Local AI Engineering with Ollama

Installing Ollama

Ollama ships as a single binary plus a background service. The installer handles both. The installation is straightforward and is documented on the download page.

For Linux users (our case), we will run the following command:

curl -fsSL https://ollama.com/install.sh | \
    OLLAMA_VERSION=0.30.0 sh

Note that all tools, scripts, and commands in this book were tested in an Ubuntu box.

The installer script does several things you should know about:

• Sets up sudo if not root.
• Picks an install dir from $PATH (/usr/local, /usr, or /).
• Downloads ollama-linux-$ARCH.tar.zst (falls back to .tgz), extracts the binary and lib/ollama/.
• On Jetson boards (a series of embedded computing boards from Nvidia), pulls an extra jetpack5 or jetpack6 package.
• If systemctl exists, configures it to run the service automatically.
• Detects the GPU and either installs CUDA drivers (NVIDIA's GPU compute toolkit, required for Ollama to use NVIDIA cards), pulls the ROCm package for AMD (AMD's equivalent stack, shipped as a separate download because it's large), or warns CPU-only mode if neither is available.

If you don't have a GPU, you'll see a warning like this:

WARNING: No NVIDIA/AMD GPU detected. Ollama will run in CPU-only mode.