Project
Purpose
Our project in collaboration with the Muslim Community Center of Greater San Diego aims to grant Muslims with tools to access reliable Islamic knowledge. As AI systems are increasingly utilized and relied on for knowledge, assessing their reliability and alignment with Islamic teachings is crucial. This project looks to aid Muslims in making informed decisions by pinpointing the most trustworthy AI platforms for their Islamic questions. With access to accurate knowledge, we can mitigate the issues of misinformation and misinterpretation. Ultimately, this project strives to bridge the gap between technology and Islamic teachings, aiding Muslims in utilizing AI for their religious needs.
Methodology
To achieve this, we went through a series of steps. First, we created a list of popular AI systems such as ChatGPT, Gemini AI, Microsoft CoPilot, and Claude AI based on their accessibility and popularity amongst the public. We then developed a set of evaluation criteria that would allow us to focus on accuracy, contextual understanding, citations, and ethical sensitivity. We created 300 multiple-choice questions about Islamic teachings to cover key topics such as basic Islamic knowledge, and current issues. Each AI system was tested using the same questions, and the responses were evaluated by our team to determine their accuracy.
Anthropic Models
Google Models
OpenAI Models
- Claude 3 Opus
- Claude 3 Sonnet
- Claude 2.1
- Gemini 1.5 Pro
- Gemini 1.5 Flash
- Gemini 2.0 Flash (Beta)
- GPT-4 O1
- GPT-4 Turbo
- GPT-4