AI Summary
Design and execute annotation and evaluation workflows for AI/ML models. Collaborate with globally distributed teams to produce high-quality training data. Implement RLHF and prompt evaluation protocols using well-defined rubrics and calibration.
Key Highlights
Design and execute annotation and evaluation workflows
Collaborate with globally distributed teams
Implement RLHF and prompt evaluation protocols
Key Responsibilities
Design and execute annotation and evaluation workflows for NLP, computer vision, and multimodal tasks
Implement RLHF and prompt evaluation protocols using well-defined rubrics and calibration
Define and enforce annotation guidelines compliance; maintain edge-case documentation
Run QA evaluation audits, spot checks, inter-annotator agreement analysis, and error taxonomy tracking
Curate datasets for named entity recognition, classification, ranking, summarization, and safety policy adherence
Technical Skills Required
Benefits & Perks
Competitive hourly base pay: $30–$50 per hour
Nice to Have
Experience with RLHF pipelines, preference ranking, or rubric-based evaluation
Familiarity with computer vision annotation (bounding boxes, polygons, segmentation)
Experience building quality systems (sampling, audits, calibration sessions, disagreement resolution)
Exposure to content safety labeling or trust & safety workflows
Job Description
About The Role
Remote STEM Jobs in India at Rex.zone focus on applied engineering work that supports AI/ML training pipelines, including data labeling, RLHF evaluation, prompt evaluation, and QA evaluation for large language models. You will collaborate with globally distributed teams to produce high-quality training data and improve model performance across NLP, computer vision, and content safety labeling workflows.
Key Responsibilities
- Design and execute annotation and evaluation workflows for NLP, computer vision, and multimodal tasks
- Implement RLHF and prompt evaluation protocols using well-defined rubrics and calibration
- Define and enforce annotation guidelines compliance; maintain edge-case documentation
- Run QA evaluation audits, spot checks, inter-annotator agreement analysis, and error taxonomy tracking
- Curate datasets for named entity recognition, classification, ranking, summarization, and safety policy adherence
- Use Python and data tooling to automate checks, sampling plans, reporting, and quality metrics
Interested in remote work opportunities in Machine Learning & AI? Discover Machine Learning & AI Remote Jobs featuring exclusive positions from top companies that offer flexible work arrangements.
- Mid-senior STEM/engineering background (CS, EE, math, physics, statistics, or related)
- Strong Python skills; comfort with structured and unstructured data
- Practical understanding of LLM evaluation, data labeling, and QA evaluation concepts
- Experience with NLP tasks (e.g., NER, text classification, relevance ranking, summarization)
- Ability to interpret guidelines, resolve ambiguity, and communicate asynchronously in a remote setting
- Experience with RLHF pipelines, preference ranking, or rubric-based evaluation
- Familiarity with computer vision annotation (bounding boxes, polygons, segmentation)
- Experience building quality systems (sampling, audits, calibration sessions, disagreement resolution)
- Exposure to content safety labeling or trust & safety workflows
Browse our curated collection of remote jobs across all categories and industries, featuring positions from top companies worldwide.
- Competitive hourly base pay: $30–$50 per hour
Visit Rex.zone and search for “Remote STEM Jobs in India” to find matching roles and apply. Highlight STEM engineering impact, Python/data tooling, and experience with data labeling, RLHF, LLM evaluation, training data quality, and annotation guidelines compliance.
Similar Jobs
Explore other opportunities that match your interests
Visa Sponsorship
Relocation
Remote
Job Type
Full-time
Experience Level
Director
subatomic
United State
Visa Sponsorship
Relocation
Remote
Job Type
Full-time
Experience Level
Associate
sundayy
United State
Machine Learning Engineer - Platforms
••••••
••••••
••••••
Job Type
••••••
Experience Level
••••••
ut md anderson
United State