# Evaluations - Member of Technical Staff — Simile AI

- Location: San Francisco, CA
- Posted pay: $200k–$400k/yr
- Posted: Jun 30, 2026
- Status: open (checked Nov 21, 2026)
- Apply (employer's site): https://jobs.ashbyhq.com/simile/33d75074-c23b-4a1f-bfdb-129bcc5be662
- Employer record: https://visajobs.io/company/simile-ai.md

## Sponsorship history
- Tier: Employer history only
- With this exact title: 0

## Description
About the Company
Pilots don't train with real passengers. Actors don't rehearse with real audiences. Yet, the most consequential decisions in society are often pushed straight to production.
Simile is changing that. We have built the first AI simulation of society, populated by generative agents based on real humans. Our research pioneered the field of AI-based simulation, proving it is possible to model human behavior with high accuracy. Today, we are developing a Foundation Model to predict human behavior in any situation, at any scale.
We have raised $300M in total funding, with a $2B post-money valuation in our latest round. Our investors include Greenoaks, Index Ventures, Hanabi, A*, Bain Capital Ventures, Factory, CVS Health Ventures, and Definition.
About the Role
As a Member of Technical Staff, Model Evaluations at Simile, you will help define how simulations of human behavior should be tested and when their predictions are useful enough to guide real decisions. You will develop measurement methods, challenge existing assumptions, and turn evidence into priorities for modeling and product. This is an opportunity to help establish evaluation practices for an emerging field while building systems that people use.
Evaluation at Simile brings together model evals, statistics, behavioral science, research methodology, product quality, and human judgment. Because our models simulate people, populations, markets, and groups, our evals must account for distributions, noisy human ground truth, uncertainty, qualitative outputs, and customer decisions. You will work with rich human data, including surveys, long-form interviews, customer studies, qualitative research, transactions, product interactions, and other real-world behavioral traces.
Questions you could investigate include: How can we tell whether an interview elicits faithful information? When do simulated responses to interventions match observed behavior? Can we predict an individual’s future choices better than population baselines? You will turn questions like these into studies, working tools, and evidence that guides what we build.
The core question for this role is simple:
How do we know when a simulation of human behavior is good enough to trust?
Areas we are especially hiring for
We are building a team with complementary strengths. Depth in one of the four areas below can be a strong fit; we do not expect expertise in all four. We also welcome specialists and generalists in broader model and product evaluation, research engineering, and related fields. Your workstream will reflect your strengths and the team’s priorities.
Human Data and Measurement: Build reliable systems for collecting and evaluating human data. Assess whether interviews and other collection methods elicit faithful, sufficiently rich accounts of people’s experiences and behavior, and whether leading questions or unsupported detail distort the resulting data. Design rubrics, expert review, and annotation workflows; measure data quality, validity, and rater agreement; and turn these into repeatable programs that can run at scale.
Behavioral Experimentalist / Builder: Design and build creative experiments across diverse populations, settings, and behavioral tasks. Develop tools and collection workflows to run studies at scale, test whether simulations reproduce human responses to interventions, and assess how findings generalize beyond each study.
Behavioral Measurement and Causal Inference: Develop scalable strategies to learn causal relationships from large, diverse behavioral datasets. Make identification assumptions explicit and test robustness through sensitivity analyses and validation. Use this evidence to evaluate whether simulations reproduce intervention effects and support defensible counterfactual predictions across populations and settings.
Individual and Longitudinal Evaluation: Evaluate whether simulations capture stable differences between individuals and changes in their behavior over time, beyond what population averages or simple baselines predict. Build benchmarks using repeated observations, temporal holdouts, and individual trajectories. Ideally, extend this work to interactions between people, validating multi-agent behavior against observed human behavior.
What you will work on
You will own a defined evaluation problem or workstream, with scope and independence matched to your experience. Depending on the workstream, you may build on existing datasets and benchmarks or develop new studies, data resources, and evaluation infrastructure. Your work may include:
Build the measurement layer for behavioral simulation: Design evals, metrics, rubrics, datasets, dashboards, and workflows that measure whether Simile’s models are accurately predicting human behavior across customer use cases, populations, question types, and decision contexts.
Partner with modeling to improve models: Evaluate new model versions, diagnose regressions, identify priority areas for model-improvement cycles, and maintain stable eval suites that represent capabilities customers actually care about.
Contribute to product and applied evals: Build evals for qualitative responses, retrieval, survey generation, AI-generated research reports, customer-facing outputs, and other product surfaces where model quality directly shapes customer trust. Turn subjective quality concerns into concrete rubrics, labeled data, automated graders, release criteria, and model-improvement signals.
Make ground truth and uncertainty legible: Develop rigorous ways to compare simulated responses against human data, customer studies, Simile-collected ground truth, and behavioral datasets. Help the company reason about sampling error, uncertainty, calibration, margin of error, representativeness, and what “ground truth” means when human behavior is inherently noisy.
Automate evaluation workflows: Use modern agentic coding tools to rapidly build internal tools, inspect model outputs, create labeli

> History, not a promise: past H-1B filings do not mean an employer will sponsor you or a given role. Data: U.S. DOL OFLC LCA disclosure (FY2024–FY2026 for role counts, certified LCAs) and the USCIS H-1B Employer Data Hub. Not legal advice. Methodology: https://visajobs.io/methodology
