Job Description

TITLE: Data Quality Analyst (Technical Writer)

ANTICIPATION DURATON: 6 months

LOCATION: San Jose, CA

We are seeking a highly skilled data quality expert with a strong understanding of Large Language Models (LLMs) to join our team as a Quality Analyst for Human-created Data. In this role, you will liaise closely with the highly technical modeling teams as well as vendor leads, understand the data needs of LLM modeling teams, and ensure that the data produced by expert content creators meets the highest quality standards. You’ll need a strong ability to understand LLM use cases, have a nuanced understanding of good versus great content, and be able to use quantitative and qualitative techniques to analyze LLM datasets.

Responsibilities:

Ensure high quality during task development

• Define data requirements based on a deep understanding of the modeling team goals and study of model loss patterns
• Design detailed instructions for content creator teams, illustrating nuanced differences with clear examples
• Rapidly iterate with content-production vendors on small batches of data to arrive at desired data quality

Measure and demonstrate data quality during data collection

• Drive processes to deliver benchmark datasets for assessing data quality
• Benchmark vendor data quality against competitor models
• Use quantitative techniques to analyze vendor-produced data to assure high quality and drive rater pool optimization
• Audit datasets for quality issues and develop tools for accelerating qualitative analysis
• Use modeling and experimentation techniques to demonstrate data impact and identify blind spots

Communicate on data quality

• Send out regular reports to project stakeholders on data quality
• Track key metrics such as iteration time and vendor quality audit results
• Author best practices and guideline documents (e.g. guidelines for teams wanting to work with vendors for human data, best practices for instruction design, etc)

Mandatory:

• Bachelor's degree in English, Creative Writing, Linguistics, Science, Computer Science, or a related field.
• 5+ years of experience in Technical Writing, Quality Assurance or a related field
• Excellent writing and editing skills, with the ability to adapt to different styles and tones.
• Strong analytical and problem-solving skills.
• Ability to work independently and as part of a team.

Preferred:

• PhD degree in English, Creative Writing, Linguistics, Science, Computer Science, or a related field.
• 5+ years of experience in Technical Writing, Quality Assurance or a related field * Excellent writing and editing skills, with the ability to adapt to different styles and tones.
• Strong analytical and problem-solving skills.
• Ability to work independently and as part of a team.

The hourly pay rate range for this position is $64.28 to $87.85/hr (dependent on factors including but not limited to client requirements, experience, statutory considerations, and location). Benefits available to full-time employees: medical, dental, vision, disability, life insurance, 401k and commuter benefits.

Synergis is proud to be an Equal Opportunity Employer. We value diversity and do not discriminate on the basis of race, color, ethnicity, national origin, religion, age, gender, gender identity, political affiliation, sexual orientation, marital status, disability, military/veteran status, or any other status protected by applicable law.

Qualified applicants with arrest or conviction records will be considered for employment in accordance with the requirements of applicable state and local laws, including but not limited to, the San Francisco Fair Chance Ordinance, the City of Los Angeles’ Fair Chance Initiative for Hiring Ordinance, the Los Angeles County Fair Chance Ordinance for Employers and the California Fair Chance Act.

For immediate consideration, please forward your resume to kcorbin@synergishr.com

If you require assistance or an accommodation in the application or employment process, please contact us at kcorbin@synergishr.com.

Synergis is a workforce solutions partner serving thousands of businesses and job seekers nationwide. Our digital world has accelerated the need for businesses to build IT ecosystems that enable growth and innovation along with enhancing the Total Experience (TX). Synergis partners with our clients at the intersection of talent and transformation to scale their balanced teams of tech, digital and creative professionals. Learn more about Synergis at

Job Tags

Similar Jobs