Meta, formerly known as Facebook, has shared measurement techniques it developed to manage its AI models with other organisations. Operating at large-scale, Meta currently has thousands of AI models to comply with regulations across the world.
- Its Machine Learning Operations (ML-Ops) ecosystem has been created to manage these models.
- The measurement method aims to consolidate key elements of model management and offer flexible, decentralised tools to varying product teams with different needs.
- Meta has developed a machine learning maturity framework to describe model outcomes and used a hierarchical framework structure to group metrics for separate purposes.
- By measuring the time since the last training of a model, Meta was able to gain an understanding of the risks of concept drift, which means that the statistical representation that a model learns from its data changes, making the model’s predictions less valid.
- Meta consolidates some logging information into a single platform to avoid inconsistencies.
- To work with a diverse range of AI systems, it created scalable decentralised data ingestion points and a central data store.
- Finally, Meta consistently defines and labels its AI concepts to avoid ambiguity.
