Learning Collatz - The Mother of all Rabbit Holes

Researchers trained small transformer models to predict the "long Collatz step," an arithmetic rule for the infamous unsolved Collatz conjecture, achieving surprisingly high accuracy up to 99.8%. The models did not learn the universal algorithm, but instead showed quantized learning, mastering specific input classes defined by their binary structure. Error analysis revealed that mistakes were systematic and explainable by simple rules, demonstrating that transformers can learn complex arithmetic functions by focusing on special cases rather than hallucinating. This study provides a new method for AI interpretability by leveraging the known mathematical structure of the problem to analyze the model's learning process.