Building with GitHub Copilot

The Training Corpus: Billions of Lines of Code

GitHub Copilot was trained using advanced deep learning techniques, a branch of machine learning built on artificial neural networks. This is usually opposed to symbolic/rule-based AI or other AI techniques that dominated earlier AI research. These networks consist of layers of interconnected nodes that progressively transform raw input into higher-level representations, much like how the human brain processes sensory information. In Copilot's case, the model architecture is based on the transformer family, which uses mechanisms such as self-attention to capture long-range dependencies in sequences of text or code.