Introducing FlexOlmo: a new paradigm for language model training and data collaboration

FlexOlmo empowers data owners to train models on their own turf, syncing up later to build a powerhouse shared model. Data stays secret, yet the model still crushes it, rivaling its all-data counterpart. And with differential privacy, it keeps snoops at bay, boasting a mere 0.7% data extraction rate.