Senior Technical Evaluator for AI-Generated Software Engineering, Data Science, and Systems Design Solutions
Remote
AI Summary
Evaluate and validate LLM-generated technical responses in software engineering, data science, and systems design. Ensure technical accuracy, code correctness, and adherence to best practices. Apply structured evaluation frameworks and provide actionable feedback to improve AI model performance.
Key Highlights
Evaluate LLM-generated technical responses
Ensure technical accuracy and code correctness
Apply structured evaluation frameworks and provide feedback
Key Responsibilities
Evaluate LLM-generated responses to software engineering, data science, and systems design questions
Validate technical accuracy by reviewing reasoning, explanations, and generated code
Execute and test code to confirm correctness and expected outputs
Identify logical flaws, inefficiencies, edge cases, and misleading explanations
Assess code quality, readability, algorithmic soundness, and system design choices
Apply structured evaluation frameworks, benchmarks, and taxonomies
Provide clear, actionable annotations to improve AI model performance
Ensure outputs align with real-world engineering best practices and conversational standards
Technical Skills Required
Benefits & Perks
Remote work
$60–$100/hour
Job Description
Position: Software Engineering, Data Science, and Systems Design Experts
Type: Hourly contract
Compensation: $60–$100/hour
Location: Remote
Commitment: 10–40 hours/week
Role Responsibilities
- Evaluate LLM-generated responses to software engineering, data science, and systems design questions.
- Validate technical accuracy by reviewing reasoning, explanations, and generated code.
- Execute and test code to confirm correctness and expected outputs.
- Identify logical flaws, inefficiencies, edge cases, and misleading explanations.
- Assess code quality, readability, algorithmic soundness, and system design choices.
- Apply structured evaluation frameworks, benchmarks, and taxonomies.
- Provide clear, actionable annotations to improve AI model performance.
- Ensure outputs align with real-world engineering best practices and conversational standards.
Interested in remote work opportunities in Development & Programming? Discover Development & Programming Remote Jobs featuring exclusive positions from top companies that offer flexible work arrangements.
- Strong academic background in computer science or a closely related discipline.
- Strong professional background in software engineering, data science, or systems design.
- Expertise in at least two programming languages (e.g., Python, Java, C++, JavaScript, Go, Rust, SQL, Bash).
- Strong problem-solving ability and comfort evaluating complex technical reasoning.
- Excellent written communication skills and attention to detail.
- Ability to work independently in a fully remote environment.
Browse our curated collection of remote jobs across all categories and industries, featuring positions from top companies worldwide.
- Upload resume
- Interview (15 min)
- Submit form
Similar Jobs
Explore other opportunities that match your interests
Visa Sponsorship
Relocation
Remote
Job Type
Full-time
Experience Level
Mid-Senior level
Opus Recruitment Solutions
United Kingdom
Visa Sponsorship
Relocation
Remote
Job Type
Full-time
Experience Level
Mid-Senior level
jump24
United Kingdom
Visa Sponsorship
Relocation
Remote
Job Type
Contract
Experience Level
Mid-Senior level
Signify Technology
United Kingdom