We are seeking a Machine Learning Researcher focused on RL and agentic systems to help define, design, and evaluate the datasets, tasks, environments, and benchmarks used to assess advanced AI systems.

Company: Protege

Role: Machine Learning Researcher - RL and Agentic

Location: Remote

Experience:

PhD or equivalent Master’s Degree + 4+ years industry experience in machine learning, computer science, statistics, engineering, mathematics, economics, or related quantitative fields

Qualification:

PhD or equivalent Master’s Degree in machine learning, computer science, statistics, engineering, mathematics, economics, or related quantitative fields
Master’s Degree + 4+ years industry experience

Role Focus: • Design and build datasets, tasks, and environments for benchmarking agentic systems and multi-step model behavior • Translate real-world workflows into structured tasks, interaction traces, trajectories, stateful environments, and verifiable outcomes • Develop frameworks for evaluating real-world data quality • Build quality scorecards and evaluation methods • Benchmark model behavior in RL and agentic settings • Connect model failures back to concrete dataset, environment, or task-design gaps

Additional responsibilities:

Contribute to tools and systems that automate dataset validation, environment generation, rollout analysis, benchmark construction, and evaluation workflows
Improve internal infrastructure for reproducible experimentation, benchmark management, and evaluation quality
Partner across research, engineering, and product

Nice to have:

Experience developing evaluation frameworks or performance metrics for datasets, agentic systems, or training data
Experience translating real-world workflows into structured tasks or environments for model evaluation
Experience with RLHF, RLAIF, imitation learning, reward modeling, online or offline RL, or related methods
Experience with Harbor or other agent evaluation frameworks
Publications or open-source contributions in reinforcement learning, agents, evaluation, or data-centric AI
Experience collaborating cross-functionally with product, infrastructure, or partnership teams
Experience with synthetic data generation, trajectory generation, or simulation-based environments

More skills:

data quality

Other:

We act with integrity and do the right thing
We are resourceful, resilient builders who solve hard problems and push through obstacles
Velocity matters
We communicate directly and respectfully, building trust through honest feedback and genuine care for one another
We win as one team
We take pride in our work, sweat the details, and continuously raise the bar for excellence

Machine Learning Researcher - RL and Agentic

Skills Required

Description