📢 Senior AI Engineer
🏢 Krisp
📍 Yerevan, Armenia
🔄 #FullTime
🗂️ #SecurityServices
📝 Responsibilities
- Develop and execute AI model evaluation strategies, ensuring accuracy, consistency, and fairness
- Implement automated and manual testing for LLM-based applications
- Collaborate with the AI Engineer to integrate testing into early-stage development
- Build and manage test datasets, ensuring high-quality, diverse, and balanced samples
- Develop synthetic data pipelines to enhance model evaluation
- Design and maintain hallucination, bias, and robustness detection frameworks
- Define and track AI performance metrics (e.g., factual accuracy, coherence, latency, response quality)
- Work closely with AI engineers to debug failures, identify root causes, and optimize model performance
- Provide feedback on prompt effectiveness, suggest improvements, and collaborate with the Prompt Engineer to refine prompts
- Implement continuous monitoring tools to track AI model drift, performance degradation, and unexpected failures
- Develop and maintain comprehensive test reports, summarizing findings and recommendations
✅ Requirements
- Experience with AI/ML testing frameworks and LLM evaluation methodologies
- Strong understanding of LLM behaviors, biases, failure modes, and edge cases
- Proficiency in Python and familiarity with ML testing frameworks (e.g., PyTest, Unittest, custom ML evaluation tools)
- Experience with test dataset management and annotation tools
- Familiarity with synthetic data generation and adversarial testing techniques
- Strong problem-solving and debugging skills to analyze AI failures and inconsistencies
- Strong English language proficiency with the ability to evaluate AI-generated text and improve prompts
IT Channel: @iJobAm_IT
Facebook: Page
Facebook: Group
LinkedIn: Page
100088126
#Career #Vacancy #Recruiter #JobOpening #ԳործԿա #Աշխատանք