Embeddings, Vector Stores & RAGLesson 11 of 11

Building a Simple RAG Chatbot

Put the script from End-to-End RAG Pipeline behind st.chat_input. The demo loads html_notes.txt, builds Chroma once with @st.cache_resource, then calls chain.invoke on each submit.

Before you run

Complete End-to-End RAG Pipeline and Introduction to Streamlit first. Activate the venv from Project Setup, then install:

pip install streamlit langchain-text-splitters chromadb

Keep a valid OPENAI_API_KEY in .env from OpenAI Account Setup.

Same pipeline as End-to-End RAG, plus Streamlit chat:

LoaderTextLoader -> html_notes.txt

↓

Text SplitterRecursiveCharacterTextSplitter -> chunks

↓

Vector StoreOpenAIEmbeddings -> Chroma (@st.cache_resource)

↓

Chain + chatretriever -> LCEL -> st.chat_input

Steps 1-3 match End-to-End RAG Pipeline. Step 4 calls chain.invoke from st.chat_input, same pattern as Introduction to Streamlit.

Load once with cache

Streamlit reruns the script on each submit. Without a cache, TextLoader and Chroma.from_documents would run again on every message. Wrap the setup in @st.cache_resource:

@st.cache_resource
def load_rag_chain():
    docs = TextLoader(str(NOTES_PATH), encoding="utf-8").load()
    chunks = RecursiveCharacterTextSplitter(
        chunk_size=150, chunk_overlap=30
    ).split_documents(docs)

    vectorstore = Chroma.from_documents(
        documents=chunks,
        embedding=OpenAIEmbeddings(model="text-embedding-3-small"),
        persist_directory=str(DB_DIR),
    )
    retriever = vectorstore.as_retriever(search_kwargs={"k": 2})

    prompt = ChatPromptTemplate.from_template(
        "Answer in one or two short sentences using only the context below.\n\n"
        "Context:\n{context}\n\n"
        "Question: {question}"
    )
    llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)

    return (
        {"context": retriever | format_docs, "question": RunnablePassthrough()}
        | prompt
        | llm
        | StrOutputParser()
    )

Chat input

Store turns in st.session_state. On submit, show the question, call chain.invoke, print the string with st.write:

if "messages" not in st.session_state:
    st.session_state.messages = []

for msg in st.session_state.messages:
    st.chat_message(msg["role"]).write(msg["content"])

if question := st.chat_input("Ask about HTML tags..."):
    st.session_state.messages.append({"role": "human", "content": question})
    st.chat_message("human").write(question)

    chain = load_rag_chain()
    with st.chat_message("ai"):
        answer = chain.invoke(question)
        st.write(answer)

    st.session_state.messages.append({"role": "ai", "content": answer})

Run the demo

Download the ZIP so simple_rag_chatbot_samples/ sits next to the script, then run:

simple_rag_chatbot_demo.py

Includes simple_rag_chatbot_samples/html_notes.txt. Creates simple_rag_chatbot_chroma_db/ on first run.

Download ZIP .py only

Unzip so simple_rag_chatbot_samples/ sits next to the script.

◇simple_rag_chatbot_demo.py

simple_rag_chatbot_demo.py

# @st.cache_resource: loader, splitter, Chroma, chain

st.chat_input -> chain.invoke -> st.chat_message

python simple_rag_chatbot_demo.py

PowerShell - (.venv) active

(.venv) PS C:\projects\langchain-course> python simple_rag_chatbot_demo.py

Starting Streamlit server - open http://localhost:8501 in your browser.

You can now view your Streamlit app in your browser.

Local URL: http://localhost:8501

Human: How do I make a link on a page?

AI: Use the <a> tag with an href attribute to create a hyperlink.

Human: What does the <title> tag do?

AI: The <title> tag sets the page title shown in the browser tab.

First question is slow. @st.cache_resource loads and embeds html_notes.txt once. Later questions skip that step.

Ask How do I make a link on a page?. The reply should mention the <a> tag. Wording may vary slightly if you rerun the script.

If it fails

Missing file: simple_rag_chatbot_samples/html_notes.txt. Unzip simple_rag_chatbot_demo.zip so the sample folder sits beside simple_rag_chatbot_demo.py.
AuthenticationError. Check OPENAI_API_KEY and load_dotenv().
ModuleNotFoundError: streamlit. Run pip install streamlit.
Port already in use. Stop other Streamlit apps or run streamlit run simple_rag_chatbot_demo.py --server.port 8502.

Streamlit chat API.

What's Next

RAG module is done. Next: Agents — when the model picks its own steps.

← PREVIOUS

End-to-End RAG Pipeline

What are Agents?