Docs / Building a RAG App

Building a RAG App with aqoon

Retrieval-Augmented Generation (RAG) is a pattern where you retrieve relevant content from a knowledge base, then pass it as context to a language model. The result is an AI answer grounded in your own documents rather than general training data.

aqoon is a natural fit for the retrieval layer: it stores, processes, and indexes your documents, and exposes a single search API that does hybrid semantic and keyword search. Your application calls the search endpoint, takes the top results, and includes them in the prompt you send to the LLM.

Want to skip the tutorial and start with a working app? Clone a RAG starter template (FastAPI or Flask) — it includes everything below in a ready-to-run project. See the SDK & Starter Templates page.

Prerequisites

  • An aqoon account with at least one collection containing documents
  • A scoped API key with access granted to the target collection — see API Keys & Rate Limits
  • Python 3.8+ (the examples below use the aqoon SDK plus one LLM package)
  • An Anthropic or OpenAI API key for the LLM step

Step 1: Install Dependencies

pip install aqoon anthropic

If you prefer OpenAI, replace anthropic with openai — the pattern is identical.

Step 2: Search aqoon for Relevant Context

Instantiate the client with your API key, then call search(). aqoon returns ranked results — each result contains the passage text (content), the document title (title), and a similarity score.

from aqoon import Aqoon

AQOON_API_KEY = "aqn_your_api_key"

def search_aqoon(query: str, limit: int = 5) -> list[dict]:
    """Search aqoon and return a list of result chunks."""
    with Aqoon(api_key=AQOON_API_KEY) as client:
        response = client.search(query, limit=limit)
    return response.get("results", [])

Each result in the list looks like this:

{
  "content": "Retrieval-Augmented Generation (RAG) combines...",
  "title": "AI Architecture Patterns.pdf",
  "document_id": "a1b2c3d4-...",
  "collection_name": "Research",
  "score": 0.91
}

Step 3: Build the Prompt with Retrieved Context

Format the search results as context and insert them into a prompt for the LLM.

def build_prompt(query: str, chunks: list[dict]) -> str:
    """Combine retrieved chunks into a prompt."""
    if not chunks:
        return (
            f"The user asked: {query}\n\n"
            "No relevant documents were found in the knowledge base. "
            "Answer based on your general knowledge and note the absence of specific context."
        )

    context_parts = []
    for i, chunk in enumerate(chunks, start=1):
        context_parts.append(
            f"[Source {i}: {chunk['title']}]\n{chunk['content']}"
        )
    context = "\n\n---\n\n".join(context_parts)

    return (
        f"Use the following excerpts from the knowledge base to answer the question. "
        f"Cite sources by number where relevant.\n\n"
        f"{context}\n\n"
        f"Question: {query}"
    )

Step 4: Send to an LLM

Pass the prompt to Claude using the Anthropic SDK:

import anthropic

ANTHROPIC_API_KEY = "sk-ant-your-key"

def ask_claude(prompt: str) -> str:
    """Send a prompt to Claude and return the response text."""
    client = anthropic.Anthropic(api_key=ANTHROPIC_API_KEY)
    message = client.messages.create(
        model="claude-opus-4-5",
        max_tokens=1024,
        messages=[{"role": "user", "content": prompt}],
    )
    return message.content[0].text

Step 5: Full Working Example

Putting it all together:

import anthropic
from aqoon import Aqoon

AQOON_API_KEY = "aqn_your_api_key"
ANTHROPIC_API_KEY = "sk-ant-your-key"


def build_prompt(query: str, chunks: list[dict]) -> str:
    if not chunks:
        return (
            f"The user asked: {query}\n\n"
            "No relevant documents were found. Answer based on general knowledge."
        )
    context_parts = [
        f"[Source {i}: {c['title']}]\n{c['content']}"
        for i, c in enumerate(chunks, start=1)
    ]
    context = "\n\n---\n\n".join(context_parts)
    return (
        f"Use the following knowledge base excerpts to answer the question. "
        f"Cite sources by number.\n\n{context}\n\nQuestion: {query}"
    )


def rag_query(query: str) -> str:
    with Aqoon(api_key=AQOON_API_KEY) as aqoon:
        response = aqoon.search(query, limit=5)
    chunks = response.get("results", [])
    prompt = build_prompt(query, chunks)
    anthropic_client = anthropic.Anthropic(api_key=ANTHROPIC_API_KEY)
    message = anthropic_client.messages.create(
        model="claude-opus-4-5",
        max_tokens=1024,
        messages=[{"role": "user", "content": prompt}],
    )
    return message.content[0].text


if __name__ == "__main__":
    answer = rag_query("What is our policy on data retention?")
    print(answer)

Tips

  • Choose the right scope. Use a scoped API key and grant only the collections relevant to your application. This prevents the LLM from receiving context from unrelated documents.
  • Tune the result limit. More results give the model more context but increase token usage and cost. Five to ten results is a good starting point; adjust based on your typical document length and query specificity.
  • Handle no results gracefully. The build_prompt function above shows one approach: tell the model that no context was found and let it respond from general knowledge or decline to answer, depending on your use case.
  • Filter by collection. If your query is always scoped to one domain, pass collection="your-collection-slug" to search() to limit results to that collection and improve relevance.
  • Cache frequent queries. aqoon search results are deterministic for a given document set. Consider caching results for common queries in Redis to reduce API calls and latency.

Bonus: JavaScript / Node.js Example

npm install @54startups/aqoon @anthropic-ai/sdk
import Anthropic from "@anthropic-ai/sdk";
import { Aqoon } from "@54startups/aqoon";

const AQOON_API_KEY = "aqn_your_api_key";
const ANTHROPIC_API_KEY = "sk-ant-your-key";

const aqoon = new Aqoon({ apiKey: AQOON_API_KEY });

function buildPrompt(query, chunks) {
  if (chunks.length === 0) {
    return `The user asked: ${query}\n\nNo relevant documents found.`;
  }
  const context = chunks
    .map((c, i) => `[Source ${i + 1}: ${c.title}]\n${c.content}`)
    .join("\n\n---\n\n");
  return (
    `Use the following knowledge base excerpts to answer the question. ` +
    `Cite sources by number.\n\n${context}\n\nQuestion: ${query}`
  );
}

async function ragQuery(query) {
  const response = await aqoon.search(query, { limit: 5 });
  const chunks = response.results ?? [];
  const prompt = buildPrompt(query, chunks);
  const anthropic = new Anthropic({ apiKey: ANTHROPIC_API_KEY });
  const message = await anthropic.messages.create({
    model: "claude-opus-4-5",
    max_tokens: 1024,
    messages: [{ role: "user", content: prompt }],
  });
  return message.content[0].text;
}

const answer = await ragQuery("What is our refund policy?");
console.log(answer);

Using Framework Integrations

If you are already using LangChain or LlamaIndex, aqoon ships first-class retriever integrations that replace the manual search and prompt-building steps above with a single object.

Install the extras you need alongside aqoon:

# LangChain
pip install 'aqoon[langchain]'

# LlamaIndex
pip install 'aqoon[llamaindex]'

LangChain

AqoonRetriever implements LangChain's BaseRetriever interface, so it slots directly into any LCEL chain:

from aqoon.integrations.langchain import AqoonRetriever
from langchain_anthropic import ChatAnthropic
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables import RunnablePassthrough
from langchain_core.output_parsers import StrOutputParser

retriever = AqoonRetriever(api_key="aqn_your_key", k=5)
llm = ChatAnthropic(model="claude-sonnet-4-20250514")

prompt = ChatPromptTemplate.from_template(
    "Answer based on this context:\n{context}\n\nQuestion: {question}"
)

chain = (
    {"context": retriever, "question": RunnablePassthrough()}
    | prompt
    | llm
    | StrOutputParser()
)

answer = chain.invoke("What is our leave policy?")

LlamaIndex

AqoonRetriever implements LlamaIndex's BaseRetriever interface and works with any QueryEngine:

from aqoon.integrations.llamaindex import AqoonRetriever
from llama_index.core.query_engine import RetrieverQueryEngine

retriever = AqoonRetriever(api_key="aqn_your_key")
query_engine = RetrieverQueryEngine.from_args(retriever)
response = query_engine.query("What is our leave policy?")

Both integrations handle authentication, result formatting, and error handling for you. Use the manual approach shown earlier in this tutorial when you need fine-grained control over the prompt or retrieval logic.