Iris

Why the chatbot needed its own infrastructure

Iris started because my dad had a business idea. He wanted a cybersecurity chatbot where companies could upload their internal documents, ask questions, and get answers based on their own material.

He asked if I could help code it. I said yes before I properly understood RAG, vector databases, reranking or multi-tenancy.

When I built Iris, the production RAG tooling I encountered was fragmented. The useful parts were largely open source or early stage, models hallucinated often, citations were not a standard part of the answer, and private company documents could not simply be sent to whichever hosted service was convenient.

The chatbot was the visible part. Making it useful for a company required the rest of the system: document ingestion, tenant isolation, hybrid retrieval, reranking, grounded generation, page-level citations and deployment inside the company’s Azure environment.

Iris was my attempt to make that entire path one managed system. The simple version sounded like this:

company documents
       ↓
      Iris
       ↓
ask a question
       ↓
answer with sources

Almost the entire project ended up inside that Iris box.

Finding the right information

The first search system used embeddings. Semantic search was good at finding passages with similar meaning, but cybersecurity documents contain exact identifiers such as AC-2, CIS 5.2, product names and CVEs. Similar meaning is not enough when the exact term matters.

I added BM25 through OpenSearch for keyword retrieval and kept Qdrant for dense search. Reciprocal rank fusion combines the two rankings without comparing scores that mean completely different things.

HyDE improves the dense side by generating a hypothetical answer and embedding that instead of only embedding the question. The generated text is never shown to the user or treated as fact. It is just a search query shaped more like the passage Iris needs to find.

The next problem was repetition. Five highly relevant passages can still contain one useful idea repeated five times. Maximal marginal relevance reduces near-duplicates, source limits stop one document from taking over the context, and a cross-encoder reranks the smaller candidate set against the original question.

The final retrieval path became:

question
   ↓
HyDE
   ↓
┌─────────────────────┐
│                     │
↓                     ↓
dense search       BM25 search
Qdrant             OpenSearch
│                     │
└──────────┬──────────┘
           ↓
reciprocal rank fusion
           ↓
diversity + source limits
           ↓
cross-encoder reranking
           ↓
answer + citations

This is more work than calling a language model, but retrieval decides whether the model ever sees the right information.

Answers that can be checked

A confident answer without evidence is especially bad in cybersecurity. Iris keeps the document name and page number attached to each retrieved passage, numbers the passages sent to the model, and asks for inline citations.

If Iris says that privileged accounts require phishing-resistant MFA, the user can open the cited page and check why. The model can still be wrong, but the answer is no longer detached from the evidence it received.

That changed the product from an AI that says things into something closer to search with an explanation on top.

Knowing whether a change actually helped

Every step in that retrieval path is a decision. How many candidates to fetch, how hard to push diversity, whether HyDE earns the extra generation it costs. Changing any of them changes the answer, and reading a few replies afterwards is not a way to tell whether it changed for the better.

LangChain orchestrates the pipeline, which means LangSmith can trace it without me instrumenting every step by hand. One question expands into the HyDE generation, both retrieval calls, the fusion, the reranker and the final answer, each with its own latency and token cost. Seeing the whole thing laid out made it clear where the time actually went, which was rarely where I assumed.

Evaluation mattered more than tracing. With a set of questions and the passages that should come back for them, a change to retrieval can be scored instead of eyeballed: did the right document still surface, did the answer still cite it, did it get slower. More than one change I was confident about turned out to make no measurable difference, and I would not have known otherwise.

This is also the part of the system that lines up most directly with the observability and evaluation work in AI-103. Tracing, token analytics and detecting ungrounded answers are on the exam because they are what separates a demo from something a company can rely on.

Keeping company data separate

The other hard requirement was multi-tenancy. A document uploaded by one company must never appear in another company’s answer.

Documents, conversations and database records are scoped to a tenant and user. PostgreSQL row-level security enforces that boundary at the database layer, rather than relying entirely on every query having the correct WHERE clause. Search indexes are separated, and Microsoft Entra ID carries the user and tenant identity through documents, chats and retrieval.

Iris can search two scopes at once: private company knowledge and shared cybersecurity material. Both can contribute to an answer without combining every customer’s documents into one searchable bucket.

From chatbot to managed system

By the end, the system looked roughly like this:

frontend
   ↓
FastAPI
   ↓
┌────────────┬─────────┬────────────┐
↓            ↓         ↓            ↓
PostgreSQL   Qdrant    OpenSearch   LLM
app data     vectors   BM25         answers

The backend streams responses to the frontend. FastAPI and LangChain handle the application and model orchestration, PostgreSQL stores users, documents and conversations, Qdrant handles dense retrieval, and OpenSearch handles keywords. These components are open source and can run inside infrastructure the company controls.

Iris is also why FibrumPDF exists. PDF ingestion became the bottleneck once document collections grew, so I pulled that problem apart and built a faster parser in Go and C. It preserves headings, lists, tables and page information before Iris indexes the document.

Iris took months because every simple box had failure modes behind it. The language model ended up being one of the less interesting parts. Finding the right evidence, keeping it private and showing where it came from was the actual project.