Building with GitHub Copilot

How Tools Work in GitHub Copilot

We have seen a list of tools (using the /list slash command) available in GitHub Copilot. create_new_workspace, edit_files, fetch_webpage, and more are some of them. Copilot uses them to augment its capabilities:

• Search and edit files in your workspace.
• Run terminal commands.
• Fetch and analyze web pages.
• Access GitHub repositories, issues, and pull requests.
• And more...

However, it's important to understand that these so-called tools are, in reality, not specific to Copilot but are a general concept in the world of LLMs. A tool is an API, function, or system the LLM can invoke when it decides it needs extra information or actions that it cannot perform on its own.

LLMs are limited by the cutoff date of their training data and their inability to perform real-time actions. Tools like search engines and web browsers help them overcome these limitations. LLMs also lack the ability to solve complex mathematical, logical, arithmetic, or programming tasks. Tools like calculators, code interpreters, and sandboxes help them with these tasks and so on.

In GitHub Copilot, a tool could be built-in features of VS Code (like the terminal or the workspace tool) or external APIs (like a web browser or the GitHub API).

Tools are transparent to the user and are invoked automatically by the LLM when needed. Here's how a simple flow looks:

1 - The user asks a question.
2 - The LLM analyzes whether it can answer directly (using its generative capabilities) or if it needs to use a tool.
3 - If a tool is needed, the LLM emits a structured call (e.g., JSON with function name + arguments).
4 - The system invokes the tool, runs it, and returns results (text, JSON, files, etc.).
5 - The LLM incorporates the result into its reply.

Here are more details about the flow of information when a tool is invoked in our context:

1 - Developer Interaction