About The Job
Mercor connects elite creative and technical talent with leading AI research labs.
Headquartered in San Francisco.
Position Details
Position: AI Model Evaluation Specialist
Type: Contract
Compensation: $50–$75/hour
Commitment: 20 hours/week
Role Responsibilities
* Write realistic prompts that reflect how professionals and consumers seek domain‑specific guidance.
* Evaluate AI-generated responses for factual accuracy, regulatory or clinical correctness, and practical usefulness.
* Identify fabricated claims, incorrect references, or misleading reasoning across model outputs.
* Score and rank multiple model responses using structured rubrics across dimensions.
* Provide written justifications with specific evidence for each evaluation.
Qualifications
* Master’s degree or higher in Legal or a relevant professional field.
* Professional experience applying domain expertise in a practitioner or advisory capacity.
* Familiarity with industry‑specific standards, regulations, or clinical guidelines.
* Strong written communication and critical reasoning skills.
#J-18808-Ljbffr