AI Summary
Develop and improve LLM training pipelines with RLHF, data labeling, and model evaluation. Implement QA evaluation checks and coordinate NLP and CV annotation tasks.
Key Highlights
Design and ship ML components for LLM training pipelines
Partner with data operations to define annotation guidelines
Run prompt evaluation and model evaluation to diagnose failure modes
Key Responsibilities
Design and ship ML components for LLM training pipelines
Partner with data operations to define annotation guidelines and labeling instructions
Build and iterate RLHF workflows (ranking, preference data, critique signals)
Run prompt evaluation and model evaluation to diagnose failure modes
Implement QA evaluation checks for annotation guidelines compliance
Coordinate NLP tasks (e.g., named entity recognition, classification) and CV annotation (bounding boxes, segmentation)
Contribute to content safety labeling policies and sampling strategies
Technical Skills Required
Benefits & Perks
$30–$50 per hour base pay range
Remote work in the US
Collaborate with distributed teams
Job Description
Overview
Remote, full-time engineering roles for mid-senior STEM graduates in the United States working on real AI/ML systems. You will help build and improve LLM training pipelines using RLHF, data labeling programs, and model evaluation/QA evaluation to drive measurable model performance improvement.
What You Will Do
- Design and ship ML components for LLM training pipelines
- Partner with data operations to define annotation guidelines and labeling instructions
- Build and iterate RLHF workflows (ranking, preference data, critique signals)
- Run prompt evaluation and model evaluation to diagnose failure modes
- Implement QA evaluation checks for annotation guidelines compliance
- Coordinate NLP tasks (e.g., named entity recognition, classification) and CV annotation (bounding boxes, segmentation)
- Contribute to content safety labeling policies and sampling strategies
- Track improvements via offline metrics, error analysis, and dataset/versioning practices
Interested in remote work opportunities in Machine Learning & AI? Discover Machine Learning & AI Remote Jobs featuring exclusive positions from top companies that offer flexible work arrangements.
- STEM degree (or equivalent experience)
- Strong Python and software engineering fundamentals
- Hands-on ML experience in NLP and/or Computer Vision
- Familiarity with RLHF, LLM evaluation, and/or prompt evaluation concepts
- Ability to write clear specs for labeling and QA evaluation processes
Browse our curated collection of remote jobs across all categories and industries, featuring positions from top companies worldwide.
Remote (US). Collaborate asynchronously with distributed teams and may interface with AI labs, startups, and annotation vendors.
Compensation
Base pay range: $30–$50 per hour.
Similar Jobs
Explore other opportunities that match your interests
Senior Machine Learning Engineer
••••••
••••••
••••••
Job Type
••••••
Experience Level
••••••
Experis
United State
Director of Machine Learning Science
••••••
••••••
••••••
Job Type
••••••
Experience Level
••••••
the knot worldwide
United State
Senior ML Inference Engineer
••••••
••••••
••••••
Job Type
••••••
Experience Level
••••••
strive - gtm talent partner
United State