DHIVA LABS · LEARN

Appearance

Install ​

LocalRAG is installed from its source code on GitHub. The current release is v0.1.0. There are two ways to run it:

  • Docker Compose (recommended): one command starts LocalRAG and Ollama in two containers.
  • pip: install LocalRAG as a Python package and use an Ollama you run yourself.

Both work on Linux, on x86-64 and arm64.

Option A: Docker ​

What you need ​

  • Docker with the Compose plugin (docker compose).
  • Git.
  • Disk space for the models: the answer model you pull into Ollama, and the embedding model, about 130 MB.

Steps ​

  1. Get the source code:

    bash
    git clone https://github.com/Dhiva-Labs/localrag.git
cd localrag

  2. Build and start both containers in the background:

    bash
    docker compose up --build -d

    The first build downloads Python packages and can take several minutes.

  3. Download the answer model into the Ollama container. You do this once:

    bash
    docker compose exec ollama ollama pull llama3.2

  4. Open http://localhost:8090 in your browser.

Your documents, the vector store and the downloaded embedding model live in a Docker volume called localrag-data. The Ollama models live in a volume called ollama. Both survive docker compose down, but docker compose down -v deletes them.

WARNING

The Compose file publishes port 8090 on all of the computer's network connections, and LocalRAG has no login. Anyone who can reach that port can read answers from your documents and delete them. On a shared network, block the port with a firewall, or change the ports line in docker-compose.yml to "127.0.0.1:8090:8090" so only your own computer can connect.

Option B: pip ​

What you need ​

  • Python 3.10 or newer.
  • Ollama installed and running.

Steps ​

  1. Download the answer model into Ollama, once:

    bash
    ollama pull llama3.2

  2. Get the source code:

    bash
    git clone https://github.com/Dhiva-Labs/localrag.git
cd localrag

  3. On x86-64, install the CPU-only build of PyTorch first. This avoids several gigabytes of graphics card libraries:

    bash
    pip install torch --index-url https://download.pytorch.org/whl/cpu

  4. Install LocalRAG:

    bash
    pip install .

  5. Start the server:

    bash
    localrag serve

  6. Open http://localhost:8090 in your browser.

With pip, LocalRAG keeps its data in a folder called data inside the folder where you start it. Start it from the same folder each time, or set LOCALRAG_DATA_DIR. See Configuration.

TIP

Use a Python virtual environment (python3 -m venv .venv and source .venv/bin/activate) so LocalRAG's packages do not mix with your system's.

The first document downloads a model ​

The first time you add a document, LocalRAG downloads the embedding model, about 130 MB. After that it works without a network connection.

Check that it works ​

Open http://localhost:8090/api/health. A working server returns something like:

json
{"status": "ok", "chroma": true, "ollama": true, "model": "llama3.2", "docs": 0}

If ollama is false, LocalRAG cannot reach Ollama. See Troubleshooting.

Next steps ​