A specially designed medical LLM developed by LiveIndian startup co-founded by former BharatPe chief product officer Ankur Jain topped the list Open Medical LLM Leaderboard.
Jivi MedX outperformed established LLM models such as OpenAI’s GPT-4 and Google’s Med-PaLM 2 with an average score of 91.65 across nine benchmark categories. Hosted by leading artificial intelligence platform Hugging Face, the leaderboard ranks LLMs specializing in medicine based on their performance in terms of how well they answer medical questions in exams and tests. The assessment includes medical examinations such as the US Medical Licensing Examination (USMLE), Indian Medical Entrance Examinations (AIIMS and NEET) and detailed assessments in clinical knowledge, medical genetics and occupational medicine, among others.
Founded in January 2024, Jivi is an Indian artificial intelligence startup with a team of doctors, surgeons, artificial intelligence engineers and data scientists. Jivi’s lean team consists of doctors, surgeons, AI engineers and data analysts. Developing technology that will transform the availability, affordability and quality of healthcare.
JIVI leverages its vast, proprietary medical dataset, consisting of millions of medical research articles, journals, clinical notes and other sources, to train Jivi MedX LLM. This dataset is one of the largest in the world. Jivi MedX was trained using an instruction tuning algorithm called odds ratio preference optimization (ORPO).