Join us

Democratizing Machine Learning at Netflix: Building the Model Lifecycle Graph

Netflix's Saish Sali, Nipun Kumar, and Sura Elamurugu describe the Metadata Service (MDS), a graph layer built to connect siloed ML tooling (model registry, pipeline orchestrator, experimentation platform, feature store, dataset platform, identity) across personalization, studio, payments, and ads.

The system assigns every ML asset a global AIP URI, ingests thin change events from each source over Kafka and SNS/SQS, then hydrates the full state from the source of truth so out-of-order or dropped events self-correct, with Datomic holding entities and reified edges and Elasticsearch powering search.

Background enrichment jobs walk multi-hop chains (model to pipeline run to A/B test cell to experiment) to materialize cross-system relationships, turning queries like "which experiments are running this model" or impact analysis on a feature change into a single graph traversal.


Give a Pawfive to this post!


Only registered users can post comments. Please, login or signup.

Start writing about what excites you in tech — connect with developers, grow your voice, and get rewarded.

Join other developers and claim your FAUN.dev() account now!

Avatar

Kala #GenAI

FAUN.dev()

@kala
Generative AI Weekly Newsletter, Kala. Curated GenAI news, tutorials, tools and more!
Developer Influence
7

Influence

1

Total Hits

181

Posts