How to Build a Simple AI Chatbot in Python with Ollama

You can build an AI chatbot in Python by sending a user’s message to a language model and displaying its response. To support a conversation, the application also sends relevant earlier messages with each request.

This tutorial builds a terminal chatbot with Python and Ollama, a tool that can run language models on your computer. You will download a small model, write a Python script and add conversation memory, reset commands and basic error handling.

The example uses a local model and does not require a hosted AI API key. It is a learning application: it does not browse the web, access company records or perform actions in other systems.

What you will build

The chatbot will accept text, generate a reply and retain a short history while the program is running. It uses Python’s standard library, so there are no third party Python packages to install for this example.

This is a generative chatbot. A rule based chatbot instead chooses from predefined responses, while a retrieval chatbot selects relevant existing material. These approaches can be combined, but a few keyword rules alone do not create a language model.

What you need before starting

Use a supported Python 3 installation, with Python 3.11 or later as the tutorial baseline, a terminal, a text editor and an Ollama installation supported by your operating system. You need internet access for the initial software and model downloads.

The example selects Gemma 3 1B from Ollama’s model library. It is a small text model suitable for illustrating the workflow, not a claim that it is the newest or most capable model. Response quality and speed depend on the model, computer and available memory. Download size is not the same as runtime memory use.

Check the model’s licence and applicable terms before redistributing it or using it in a commercial product.

Step 1: check Python and create a project folder

On Windows, open PowerShell and check Python:

py --version
mkdir python-chatbot
cd python-chatbot

On macOS or Linux:

python3 --version
mkdir python-chatbot
cd python-chatbot

If the Python command is not recognised, install Python from python.org or an appropriate supported package source for your system. Reopen the terminal after installation.

A virtual environment is optional for this particular script because it uses only the standard library. Use a dedicated environment when adding external packages later.

Step 2: install Ollama and download the model

Install Ollama using the instructions linked from its official quickstart. Open the application or ensure its local service is running.

In your terminal, run:

ollama pull gemma3:1b
ollama list

The first command downloads the model. The second should show the installed model. Download time varies with your connection.

If the service is not running, start it in another terminal:

ollama serve

Leave that terminal open. If Ollama reports that its port is already in use, an existing instance may already be running; do not start a second server unnecessarily.

This tutorial calls the local endpoint at http://127.0.0.1:11434/api/chat. It does not select a cloud model. Ollama also documents a local only configuration using OLLAMA_NO_CLOUD=1; apply it to the Ollama server process and restart that process if you require cloud features to be disabled. See the Ollama FAQ for platform specific instructions.

Step 3: create the chatbot script

Create a file named chatbot.py inside the project folder. Copy the following complete code into it:

import json
import socket
import urllib.error
import urllib.request


API_URL = "http://127.0.0.1:11434/api/chat"
MODEL = "gemma3:1b"
MAX_TURNS = 3
MAX_INPUT_CHARS = 1000
MAX_REPLY_CHARS = 1200
TIMEOUT_SECONDS = 120

SYSTEM_MESSAGE = {
    "role": "system",
    "content": (
        "You are a helpful learning assistant. Give short, clear answers. "
        "State uncertainty when appropriate. You have no web access, "
        "company database access or ability to perform external actions."
    ),
}

# Keep local requests out of environment-configured HTTP proxies.
HTTP = urllib.request.build_opener(urllib.request.ProxyHandler({}))


def get_reply(messages):
    payload = {
        "model": MODEL,
        "messages": messages,
        "stream": False,
        "options": {"num_ctx": 4096, "num_predict": 200},
    }
    request = urllib.request.Request(
        API_URL,
        data=json.dumps(payload).encode("utf-8"),
        headers={"Content-Type": "application/json"},
        method="POST",
    )
    try:
        with HTTP.open(request, timeout=TIMEOUT_SECONDS) as response:
            data = json.loads(response.read().decode("utf-8"))
    except urllib.error.HTTPError as exc:
        if exc.code == 404:
            raise RuntimeError(
                "Model or endpoint not found. Check Ollama and run: "
                f"ollama pull {MODEL}"
            ) from exc
        raise RuntimeError(
            f"Ollama returned HTTP {exc.code}. Check its server logs."
        ) from exc
    except (TimeoutError, socket.timeout) as exc:
        raise RuntimeError(
            "The request timed out. The model may still be loading or busy."
        ) from exc
    except urllib.error.URLError as exc:
        raise RuntimeError(
            "Cannot reach Ollama. Check that its local service is running."
        ) from exc
    except (ValueError, UnicodeError) as exc:
        raise RuntimeError("Ollama returned an unreadable response.") from exc

    if not isinstance(data, dict):
        raise RuntimeError("Unexpected response format from Ollama.")
    if data.get("error"):
        raise RuntimeError("Ollama reported an error. Check its server logs.")
    message = data.get("message")
    if not isinstance(message, dict):
        raise RuntimeError("Ollama did not return a chat message.")
    reply = message.get("content")
    if not isinstance(reply, str) or not reply.strip():
        raise RuntimeError("The model returned no text. Try a shorter question.")
    return reply.strip()


def main():
    history = []
    print("Python AI chatbot. Commands: /reset and /exit")
    while True:
        try:
            user_text = input("You: ").strip()
            if user_text.lower() == "/exit":
                break
            if user_text.lower() == "/reset":
                history.clear()
                print("Bot: Conversation history cleared in this program.")
                continue
            if not user_text:
                continue
            if len(user_text) > MAX_INPUT_CHARS:
                print(f"Bot: Keep your message within {MAX_INPUT_CHARS} characters.")
                continue

            user_message = {"role": "user", "content": user_text}
            messages = [SYSTEM_MESSAGE] + history + [user_message]
            reply = get_reply(messages)
            if len(reply) > MAX_REPLY_CHARS:
                reply = reply[:MAX_REPLY_CHARS] + " [shortened]"
            print(f"Bot: {reply}")

            # Save only successful exchanges, keeping complete pairs.
            history.extend([
                user_message,
                {"role": "assistant", "content": reply},
            ])
            history = history[-2 * MAX_TURNS:]
        except RuntimeError as exc:
            print(f"Bot: {exc}")
        except (KeyboardInterrupt, EOFError):
            break
    print("Goodbye.")


if __name__ == "__main__":
    main()

Save the file as plain text with a .py extension. On Windows, ensure the editor has not saved it as chatbot.py.txt.

Step 4: run your Python chatbot

From the project folder, use:

py chatbot.py

On macOS or Linux, use:

python3 chatbot.py

Wait for the prompt, then type a short question. The first response can take longer while the model loads.

Try this sequence:

You: Explain an API in two short sentences.
You: Give a simple example of that.
You: /reset
You: What is a Python list?
You: /exit

The second question tests whether the model uses the earlier exchange. Exact wording and answer quality will vary. The sample above is a set of inputs to try, not a transcript of guaranteed model outputs.

How the chatbot code works

The request and response

get_reply() sends a JSON request to Ollama’s chat API. The request identifies the model and supplies messages with roles such as user and assistant. Setting stream to False asks for a complete response rather than a stream of partial outputs.

The script reads the generated text from message.content. It handles common request failures and rejects missing or empty reply data. Python’s urllib.request documentation explains the HTTP request facilities used here.

Conversation memory

The history list retains the last three successful question and answer pairs. Those messages are included with the next request. Once the limit is exceeded, older pairs are removed.

This is context management, not model training. The program does not update model weights or permanently teach the model new facts. Closing the process discards the Python history, and /reset clears it while the program remains open. Neither action is a claim about deleting terminal scrollback or operating system records.

Input and output limits

The script limits each input to 1,000 characters and requests up to 200 generated tokens. It also shortens exceptionally long replies before displaying and storing them.

Characters and tokens are different units. A short history helps control resource use, but these character limits do not guarantee that every possible conversation fits the configured model context. Different languages and unusual text can use tokens differently. Shorten messages or reset the conversation if context becomes a problem.

An answer can stop before completing its explanation when it reaches the generation limit. Ask a more focused question or adjust num_predict after considering latency and memory requirements.

Instructions and reliability

The system message asks for concise answers and describes the bot’s limitations. Instructions can influence behaviour, but they do not guarantee truthfulness or enforce a security boundary.

A model may produce confident errors, mishandle arithmetic or claim capabilities it does not have. The application must enforce permissions and validate consequential outputs independently.

Troubleshooting common problems

Do not automatically retry indefinitely after a timeout. The local server may still be processing the original request. Inspect its state before submitting repeated work.

Can this chatbot run offline?

After downloading the software and selected local model, this example can generate responses without a hosted AI API. It sends chat requests to the local loopback address and provides no web search tools.

Keep the model local and review the surrounding machine, software and logging configuration before handling confidential information. A local deployment is not automatically a complete privacy or security solution.

Do not expose the Ollama port directly to the public internet. Ollama’s documented default binding is local; changing its network exposure requires appropriate authentication, access controls and operational safeguards.

How to add company information accurately

This tutorial does not know your current catalogue, policies, customer records or order status. A general model should not invent those details.

For company questions, a common next step is retrieval augmented generation, or RAG. The application retrieves relevant approved documents and supplies them as context for an answer. Microsoft’s RAG overview describes this approach and its access control challenges.

A business implementation should restrict retrieval to information the user is allowed to see, preserve source references and transfer unanswered or conflicting questions to an employee. Retrieval improves access to relevant evidence; it does not guarantee that every generated answer is correct.

Live information requires a separate integration. For example, order status should come from an authorised query to the order system. Changing an order needs further permission checks and approval rules. Merely adding a document containing an order number does not create that functionality.

What changes before putting a chatbot on a website?

Keep the model connection on a controlled server and place a web application layer between it and visitors. A production system needs authentication where applicable, separate session histories, request limits, monitoring and a reliable handoff route.

Treat generated text as untrusted when displaying it in a browser. Escape plain text or sanitise rendered content. Evaluate the bot using representative questions, including missing information, misleading requests and attempts to obtain another user’s data.

Define what the bot may answer and what it must escalate. For sensitive or consequential tasks, require verification and appropriate human approval. This terminal script is not a production support system.

Frequently asked questions

Do I need an API key to build this Python chatbot?

No hosted API key is required for the selected local Ollama model. Cloud services and cloud models may have different authentication, pricing and data handling arrangements.

Do I need NLTK, ChatterBot or a vector database?

Not for this example. Python sends messages directly to Ollama using its standard library. Additional NLP libraries or retrieval infrastructure should be introduced only when they solve a specific requirement.

Does chatting with the bot train the model?

No. The script passes recent conversation messages as context. It does not perform training or update model weights.

Is the chatbot free to run?

This local configuration does not make paid hosted model API calls. You still need suitable hardware, storage and electricity, and should review software and model licensing for your intended use. Deployment and maintenance also have costs.

Can I use a different model?

Yes. Select a locally runnable chat model supported by Ollama, download it and change the MODEL value to its exact tag. Check its memory requirements, licence, prompt behaviour and response quality before using it for your application.

Can the chatbot answer current questions from the internet?

Not in this tutorial. It has no search integration. It generates answers from its model and the supplied context, so current facts must be checked against suitable sources.

Build a chatbot that fits your business

A local tutorial is a useful way to understand the interaction between Python, a model and conversation history. A business chatbot adds approved knowledge, reliable system integrations, access controls and measurable quality standards.

Explore Inventh’s AI and machine learning development services for assistants and business integrations. If you need help deciding scope, data readiness or governance, see our AI consulting services.

Discuss your chatbot project with Inventh. Share who will use it, which questions it should answer and which systems it needs to access.