Staff ML Engineer - Databricks

Overview

Being good neighbors – helping people, investing in our communities, and making the world a better place – is who we are at State Farm. It is at the core of how we operate and the reason for our success. Come join a #1 team and do some good!

HYBRID: Hybrid Work Arrangement: A “Hybrid” position means selected candidates will be assigned to a facility and will spend some time working both in the office and from home. For Enterprise Technology, a hybrid work arrangement will require approximately 40% in-office time each month, beginning in late 2027.

HUB LOCATIONS: Bloomington, IL; Dunwoody, GA; Richardson, TX; or Tempe, AZ

SPONSORSHIP: Applicants for this position are required to be eligible to lawfully work in the U.S. immediately; employer will not sponsor applicants for U.S. work authorization (e.g. H-1B visa) for this opportunity

Responsibilities

We are seeking a hands-on Staff Machine Learning Engineer to architect, build, and scale enterprise-grade AI/ML systems, real-time decisioning capabilities, and agentic AI applications.

This role will help develop our next-generation Next Best Action (NBA) and recommendation platform, leveraging Databricks, AWS, and modern AI/ML technologies. The focus is on ML systems engineering, architecture, and productionization rather than developing predictive models from scratch.

• ML Architecture & Engineering: Design and build scalable, production-grade ML systems, APIs, feature pipelines, and reusable engineering components. • Decisioning & Recommendation Engines: Architect real-time or near-real-time decisioning systems integrating predictive models, business rules, eligibility, prioritization, and optimization. • Closed-Loop Learning: Build feedback mechanisms that capture customer interactions and business outcomes to continuously improve recommendations. • Databricks & AWS Engineering: Develop reliable data and ML pipelines using Python, PySpark, SQL, Databricks, and AWS. • MLOps: Implement model deployment, serving, versioning, monitoring, CI/CD, and retraining workflows. • Agentic AI: Architect and develop production-grade agentic AI applications, including LLM orchestration, tool integration, evaluation, and guardrails. • Technical Leadership: Lead architecture decisions, establish engineering standards, mentor engineers, and collaborate across technical and business teams.

Qualifications

Required

• 8+ years of experience in ML engineering, software engineering, or related technical disciplines, with demonstrated technical leadership. • Programming: Expert-level Python and strong SQL skills; experience with APIs, microservices, and production software development. • Data Engineering: Expert-level PySpark and Apache Spark; experience building reliable data and feature pipelines. • Cloud & ML Platforms: Deep hands-on Databricks expertise (Delta Lake, Unity Catalog, Workflows) and strong AWS experience (S3, IAM, compute, and application integration). • ML Engineering & MLOps: Experience supporting the full production ML lifecycle, including model integration, deployment, serving, monitoring, versioning, and retraining. Familiarity with MLflow and CI/CD. • Real-Time Decisioning: Experience building low-latency inference services, decisioning APIs, and scalable recommendation systems. • Agentic AI: Hands-on experience developing LLM-powered or agentic AI applications, including orchestration, tool calling, and enterprise integrations. • System Architecture: Proven ability to independently architect and deliver complex, reliable enterprise systems.