Member of Technical Staff (Data Scientist, Evals)

Benefits

Parental leave

Not verified

Non-birth-parent leave

Not verified

Family-building benefits

• Fertility benefits: Not verified
• Adoption assistance: Not verified
• Surrogacy assistance: Not verified

Mental health support

Not verified

Relocation assistance

Not verified

Childcare support

Not verified

Learning budget

Not verified

Verification

Not verified

Salary

Not verified

Was this benefit information wrong? Tell us.

Market context

U.S. role benchmark (BLS OEWS)

$111,944 U.S. median for this role

Projected growth (BLS Employment Projections)

+13.7% - Much faster than average

Matched to SOC 15-1252 - Data and ML aggregate by role bucket.

Source: U.S. Bureau of Labor Statistics, OEWS, May 2024 and Employment Projections, 2024-2034.

Role

Role function

Data From the posting source

Seniority

Mid From the posting source

Schedule

Shift type

Not verified

Weekend work

Not verified

Application

Cover letter

Not verified

Assessment

Not verified

Deadline

Not stated

Where they hire

State eligibility is not yet verified.

About this role

Member of Technical Staff (Data Scientist, Evals) London, UK, United Kingdom, Belgrade, San Francisco, Berlin Perplexity serves tens of millions of users daily with reliable, high-quality answers grounded in an LLM-first search engine and our specialized data sources. We aim to use the latest models as they are released, but the intelligence frontier is a jagged one, and popular benchmarks do not effectively cover our use cases. In this role, you will build specialized evals to improve answer quality across Perplexity, covering search-based LLM answers and other scenarios popular with our users. Responsibilities - Architect and maintain automated evaluation pipelines to assess answer quality across Perplexity's products, ensuring high standards for accuracy and helpfulness - Design evaluation sets and methods specifically to measure the impact of tool calls (particularly web search retrieval) on the final answer's quality - Develop VLM-based solutions to programmatically evaluate how final answers render visually across different platforms and devices - Continuously review public benchmarks and academic evaluations for their applicability to the Perplexity product, adapting and incorporating them into our regular performance measurements - Operate within a small, high-impact team where your evaluation metrics directly shape product changes, collaborating closely with technical leadership to measure and improve Answer Quality Qualifications - PhD or MS in a technical field or equivalent experience - 4+ years of experience in data science or machine learning - Strong proficiency in Python and SQL (expected to write production-grade code) - Experience building within a modern cloud data stack, specifically AWS

Read the full description at jobs.ashbyhq.com. FewerJobs shows a preview and links to the original posting.