LangChain BasicsLesson 2

Running Open Source Models Locally

LangChain Basics sent your question to OpenAI over the internet. This lesson points the same script shape at Ollama on your PC — same invoke call, no API bill.

What changes

OpenAI scripts use ChatOpenAI and read OPENAI_API_KEY. Local scripts use ChatOllama and talk to localhost:11434 — the URL you already saved in .env during Project Setup.

You installed Ollama and pulled llama3.2 in Ollama setup. LangChain does not replace Ollama — your Python file sends HTTP requests to the Ollama app in the background.

Before you run

Start the Ollama app (or keep it running from earlier). Confirm llama3.2 is on disk and the API responds:

PowerShell — check Ollama
PS C:\projects\langchain-course> ollama list
NAME ID SIZE MODIFIED
llama3.2:latest a80c4f2acd… 2.0 GB 2 days ago
PS C:\projects\langchain-course> curl http://localhost:11434
Ollama is running
ollama list confirms llama3.2 is on disk. The local API listens on port 11434 by default.

The script

Create hello_ollama.py in your project folder. Same HTML question as hello_langchain.py so you can compare the two outputs side by side.

◇hello_ollama.py
"""hello_ollama.py — Ollama via LangChain"""
import os
from dotenv import load_dotenv
from langchain_ollama import ChatOllama
from langchain_core.messages import HumanMessage
load_dotenv()
llm = ChatOllama(
model="llama3.2",
base_url=os.getenv("OLLAMA_BASE_URL", "http://localhost:11434"),
)
question = "What does the HTML <a> tag do?"
response = llm.invoke([HumanMessage(content=question)])
print(response.content)
Same shape as hello_langchain.py — only the client class and connection change.

Run it

Activate the venv, then:

python hello_ollama.py
PowerShell — (.venv) active
(.venv) PS C:\projects\langchain-course> python hello_ollama.py
… Ollama may need a few seconds on the first run …
The <a> tag marks a hyperlink to another page or file.
No API key needed. First run can take a few seconds while Ollama starts up. The exact sentence may differ between runs.

OpenAI vs Ollama in code

PieceOpenAI (cloud)Ollama (local)
Client classChatOpenAIChatOllama
Name in codegpt-4o-minillama3.2
AuthOPENAI_API_KEYnone (local)
Callllm.invoke([HumanMessage(...)])

Later lessons change the name in one spot. See LangChain Ollama docs for other Ollama options.

If it fails

  • Connection refused — Ollama is not running. Open the app or run ollama serve.
  • model not found — run ollama pull llama3.2 again.
  • Very slow or hangs — common on the first run while Ollama loads llama3.2. Close other heavy apps if your laptop struggles.

What's Next

Ollama is wired up. Next: tune temperature and max_tokens.