True Talent Solutions Partners
Full-Stack Engineer, Agent Evaluation & Reliability
- California
- onsite
- Full time
$120,000–$180,000 / yearPosted
Free to apply. You’ll finish on the employer’s own site.
- TypeScript
- Node.js
- Machine Learning
- PostgreSQL
About the Role
Join an AI/ML engineering team building infrastructure for agent evaluation and continuous improvement. This backend-leaning full-stack role combines foundational systems work with end-to-end product delivery, helping make agent workflows more reliable for enterprise customers.
What You'll Do
Build backend systems that power agent evaluation and continuous-improvement workflows.
Design data models for evaluation datasets, experiment runs, agent traces, knowledge versions, findings, and customer-defined quality criteria.
Build and maintain enterprise integrations, including authentication, OAuth, webhooks, incremental syncs, backfills, rate-limit handling, retries, and monitoring.
Own features from technical design and implementation through deployment, monitoring, customer feedback, and iteration.
Work with customers to understand their systems, diagnose failures, and turn specific needs into reusable product capabilities.
Take features through the full stack while contributing to architecture and product decisions.
What We're Looking For
Two to four years of experience in a full-stack or backend engineering role, building and shipping production systems.
Production experience with TypeScript and Node.js or Bun.
Hands-on machine learning model development experience, in addition to software engineering experience.
Experience at a fast-paced startup, from early stage through Series D.
Production experience designing PostgreSQL schemas and working with queues or worker systems.
A track record of owning projects end to end and building backend systems involving data modeling, integrations, observability, or distributed workflow debugging.
Understanding of how evaluating nondeterministic agent systems differs from conventional unit testing.
Experience with agent infrastructure, LLM evaluation, tracing, or agent memory systems in production or through substantial side projects is useful.
Compensation & Benefits
Base salary is $120,000 to $180,000 annually. Visa sponsorship is available, including H-1B transfers, OPT, and O-1.
Location
On-site in San Francisco, California, United States.
Not the right role?
Search jobs from 47,000+ company career pages, many not on LinkedIn or Indeed.