AI Benchmark & Red-Teaming Specialist Jobs – Remote LLM Evaluation | Turing

By

AI Benchmark & Red-Teaming Specialist – Remote LLM Evaluation Job at Turing

Company: Turing

Job Title: AI Benchmark & Red-Teaming Specialist

Category: AI / Science / Research

Location: Remote

Engagement Type: Contractor

Engagement Length: Up to 4 Weeks

Weekly Commitment: Up to 40 Hours

Time Zone Requirement: 4 Hours of PST Overlap

Experience Required: At Least 1 Year in a Relevant Research or QA Area

Compensation: Not specified in the job description

About Turing

Turing is a San Francisco-based technology company that works with leading AI research labs and global enterprises to develop and improve advanced AI systems.

The company supports frontier AI research across coding, reasoning, STEM, multilinguality, multimodality, and AI agents. It also helps businesses deploy reliable AI systems through advanced training, data, and evaluation solutions.

About the Role

Turing is looking for analytical researchers and red-teaming specialists to create challenging web-research benchmark tasks that test the capabilities and limitations of advanced AI models.

In this role, you will investigate difficult-to-find facts, create evidence-based research tasks, test them against leading large language models (LLMs), and identify why AI models fail to find accurate information or follow effective research strategies.

Your work will help improve AI evaluation frameworks and test the research, search, and reasoning abilities of next-generation AI models.

Key Responsibilities

  • Research specialized topics to discover difficult-to-find factual information.
  • Verify information using primary sources and highly credible references.
  • Cross-check evidence and investigate contradictory information.
  • Document exact source URLs and create reliable answer keys.
  • Develop clear and unambiguous benchmark prompts.
  • Test research tasks against designated advanced AI models.
  • Identify weaknesses in AI search strategies and reasoning processes.
  • Analyze model search logs, research trajectories, and intermediate outputs.
  • Determine why models fail and distinguish genuine limitations from unclear prompts.
  • Apply successful benchmark patterns across different research domains.
  • Review tasks created by other trainers when required.

Required Qualifications and Skills

  • At least 1 year of experience in investigative research, fact-checking, AI benchmark task creation, red-teaming, or technical QA review.
  • Experience authoring, testing, or reviewing AI benchmark tasks.
  • Knowledge of benchmark projects such as GAIA, BrowseComp, SimpleQA, SWE-bench, Terminal-Bench, or Humanity's Last Exam (HLE).
  • Intermediate-to-advanced web research and Open Source Intelligence (OSINT) skills.
  • Ability to search academic archives, local records, and specialized databases.
  • Strong analytical skills for examining model reasoning, search logs, and source selection.
  • Ability to write precise research prompts and evidence-based answer keys.
  • Strong fact-checking and source-verification abilities.

Preferred Qualifications

  • Bachelor's degree or higher in a relevant engineering field is required as stated in the listing's preferred qualifications section.
  • Experience designing prompt-injection or reasoning-trap tasks to test advanced AI models.
  • Advanced OSINT and investigative research capabilities.
  • Published investigative or academic research demonstrating strong source verification.
  • Specialized knowledge of local government records, regional sports history, legal archives, historical patents, or similar research areas.
  • Experience as a lead reviewer, benchmark auditor, or quality owner in AI evaluation projects.

Contract and Work Details

  • Engagement Type: Contractor.
  • Project Length: Up to 4 Weeks.
  • Weekly Commitment: Up to 40 Hours.
  • Time Zone: 4 Hours of PST Overlap Required.
  • Work Location: Remote.
  • Compensation: Not mentioned in the supplied job description.

Evaluation and Selection Process

  1. Job Interest Form: Candidates must submit the required job interest form.
  2. Assessment: Applicants must complete the required assessment.
  3. Review: Assessment results will be reviewed.
  4. Further Selection: Candidates who meet the evaluation requirements may proceed to the next stage.

Who Should Apply?

This opportunity is suitable for investigative researchers, AI benchmark specialists, red-teamers, OSINT researchers, fact-checkers, and technical QA professionals with experience evaluating AI research and reasoning capabilities.

Applicants should be comfortable conducting deep web research, verifying difficult facts, writing precise benchmark tasks, and analyzing how advanced AI models search for and interpret information.

How to Apply

Visit Turing's official website to explore available opportunities and verify the application process.

Visit Turing Official Website

Note: This is Turing's general official website, not a verified direct application link for this specific role. Check the original job listing or contact the recruiter for the job interest form and assessment link.

AQZORA Jobs

Discover Global Remote Jobs & Career Opportunities

Apply Now