AI Evaluation Engineer/Researcher

Summary

Responsibilities

• Acquire proficiency in contemporary methodologies for assessing large language models and diverse AI architectures.
  
• Plan and execute structured evaluation protocols under the guidance of senior researchers.
  
• Create and curate comprehensive evaluation datasets, test scenarios, scoring rubrics, and performance benchmarks.
  
• Evaluate model responses to determine accuracy, contextual relevance, potential bias, safety compliance, and areas of failure.
  
• Assess how AI technologies interpret specific dynamics within Nigerian, Sahelian, and broader African conflict and humanitarian landscapes.
  
• Perform comprehensive literature reviews and replicate relevant scientific experiments.
  
• Leverage Python and specialized research tools to automate assessment workflows and analyze outcome data.
  
• Document research methodologies, key findings, identified limitations, and ensure reproducibility of results.
  
• Collaborate with journalists, subject-matter experts, researchers, and engineers to integrate local insights into the evaluation framework.
  
• Formulate independent research questions and design experiments with increasing autonomy.