---

title: Building RAG Knowledge Bases — Production Guide description: Complete RAG knowledge base guide: Embedding model selection, document chunking strategies, vector search optimization, and operations monitoring. Original ops tutorial. lang: en tags: [ai, rag, embedding, ops] date: 2026-10-23 original: true faq: - q: "How to choose an Embedding model?" a: "For English: text-embedding-3-small (OpenAI compatible) or text-embedding-3-large. For multilingual: bge-m3. For local deployment: GPTFuzz/bge-large-zh or all-MiniLM-L6-v2." - q: "What is the best practice for document chunking?" a: "Chunk by semantic boundaries (paragraphs/sections), not fixed character counts. Use LangChain's RecursiveCharacterTextSplitter with chunk_size=512 and chunk_overlap=64. For code docs, use the Language='python' parameter to preserve structure." - q: "Vector search recall rate is too low, what to do?" a: "Check: ① Whether the Embedding model matches the language ② Whether chunk_size is reasonable ③ Whether the similarity threshold is too high (default 0.5, try 0.3) ④ Whether hybrid search (sparse + dense) is enabled." - q: "How to evaluate RAG system performance?" a: "Use the RAGAS framework: faithfulness, answer_relevancy, context_precision. Build a test set, run evaluation scripts, and perform regular regression testing." - q: "How to back up and restore a vector database?" a: "Milvus: export .vec files regularly. Chroma: back up chroma.sqlite3 + chroma-data/ directory. After restore, rebuild indexes and verify vector counts match."

---

Building RAG Knowledge Bases — Production Guide

Use Case:沉淀 enterprise documents, ops manuals, and FAQs into a searchable knowledge base, enabling precise Q&A through RAG (Retrieval-Augmented Generation).

---

1. RAG Architecture Overview

┌─────────────┐     ┌──────────────┐     ┌─────────────┐
│  Document    │ ──▶ │  Preprocess   │ ──▶ │  Embedding   │
│  Sources     │     │  (Parse/Chunk)│     │  Vectorize   │
│ (PDF/MD/DB) │     │               │     │              │
└─────────────┘     └──────────────┘     └──────┬──────┘
                                                │
                                                ▼
┌─────────────┐     ┌──────────────┐     ┌─────────────┐
│  User Query  │ ──▶ │  Retrieval    │ ──▶ │  LLM Gen     │
│             │     │  (Vector Sim.) │     │  Contextual  │
└─────────────┘     └──────────────┘     └─────────────┘

---

2. Embedding Model Selection

2.1 Mainstream Models Comparison

Model	Dimensions	Max Tokens	English Support	Deployment
text-embedding-3-small	1536/512/256	8191	✅	API / Ollama
bge-m3	1024	8192	✅✅	Local Ollama
text-embedding-3-large	3072	8191	✅	API
all-MiniLM-L6-v2	384	256	✅	Local

2.2 Ollama Deployment for Embeddings

# Download bge-m3
ollama pull bge-m3

# Test embedding generation
curl http://localhost:11434/api/embeddings -d '{
  "model": "bge-m3",
  "prompt": "Reasons why a Kubernetes Pod is stuck in Pending state"
}'

2.3 OpenAI-Compatible Interface

For LangChain / LlamaIndex integration:

from langchain.embeddings import OllamaEmbeddings

embeddings = OllamaEmbeddings(
    model="bge-m3",
    base_url="http://localhost:11434"
)

---

3. Document Preprocessing

3.1 Format Support

Format	Parser Tool	Notes
Markdown	Native markdown	Preserve heading hierarchy
PDF	pypdf / pdfplumber	Tables need special handling
Word	python-docx	Style information lost
HTML	BeautifulSoup	Strip script tags
JSON	Native	Flatten nested structures

3.2 Chunking Strategy

from langchain.text_splitter import RecursiveCharacterTextSplitter

splitter = RecursiveCharacterTextSplitter(
    chunk_size=512,           # Max tokens per chunk
    chunk_overlap=64,         # Overlap between chunks to avoid semantic breaks
    length_function=len,
    separators=["\n\n", "\n", ". ", " ", ""]
)

chunks = splitter.split_text(document_content)

Key Parameters: - chunk_size: Adjust based on the Embedding model's max tokens (bge-m3 supports 8192) - chunk_overlap: Recommended 10-15% of chunk_size - separators: Split by semantic boundaries, prioritize \n\n (paragraphs)

---

4. Vector Database Configuration

4.1 ChromaDB (Lightweight, Small Scale)

import chromadb
from chromadb.config import Settings

client = chromadb.PersistentClient(path="/data/chroma/knowledge-base")

collection = client.get_or_create_collection(
    name="ops-manuals",
    metadata={"hnsw:space": "cosine"}
)

# Batch insert
collection.add(
    documents=chunks,
    embeddings=embeddings.embed_documents(chunks),
    ids=[f"doc_{i}" for i in range(len(chunks))]
)

4.2 Milvus (Production-Grade, Large Scale)

from pymilvus import connections, Collection, utility

connections.connect("default", host="milvus", port="19530")

# Create collection if it doesn't exist
if not utility.has_collection("ops_knowledge"):
    from pymilvus import CollectionSchema, FieldSchema, DataType, Collection
    fields = [
        FieldSchema(name="id", dtype=DataType.INT64, is_primary=True),
        FieldSchema(name="embedding", dtype=DataType.FLOAT_VECTOR, dim=1024),
        FieldSchema(name="content", dtype=DataType.VARCHAR, max_length=65535),
        FieldSchema(name="source", dtype=DataType.VARCHAR, max_length=512),
    ]
    schema = CollectionSchema(fields, "Ops knowledge base")
    collection = Collection("ops_knowledge", schema)

    # Create index
    index = {"index_type": "IVF_FLAT", "metric_type": "COSINE", "params": {"nlist": 128}}
    collection.create_index("embedding", index)

collection.load()

---

5. Retrieval and Generation

5.1 Retrieval Optimization

# Similarity threshold filtering
results = collection.query(
    query_embeddings=embeddings.embed_queries([question]),
    n_results=5,
    where={"source": {"$eq": "network"}}
)

# Only return results with similarity > 0.6
filtered = [r for r, score in zip(results['documents'], results['distances']) if score < 0.4]

5.2 Prompt Template

RAG_PROMPT = """You are an ops expert assistant. Answer the following question based on the context below.

<context>
{context}
</context>

Question: {question}

Requirements:
1. Only answer based on the context provided, do not fabricate information
2. If the context is insufficient, state so explicitly
3. Provide actionable solutions
"""

5.3 Full Pipeline

from langchain.chains import RetrievalQA
from langchain.llms import Ollama
from langchain.vectorstores import Chroma

# Initialize
db = Chroma(persist_directory="/data/chroma/knowledge-base", embedding_function=embeddings)
llm = Ollama(model="qwen2.5:7b", base_url="http://localhost:11434")

# Retrieval QA
qa_chain = RetrievalQA.from_chain_type(
    llm=llm,
    retriever=db.as_retriever(
        search_kwargs={"k": 5, "score_threshold": 0.6}
    ),
    return_source_documents=True
)

response = qa_chain({"query": "How to troubleshoot Nginx 502 errors?"})
print(response["result"])

---

6. Operations Monitoring

6.1 Index Health Checks

# ChromaDB
ls -lh /data/chroma/
sqlite3 /data/chroma/chroma.sqlite3 "SELECT COUNT(*) FROM documents;"

# Milvus
milvus-cli> show collections
milvus-cli> show entities ops_knowledge

6.2 Performance Metrics

Metric	Monitoring Method	Alert Threshold
Retrieval Latency	Prometheus histogram	P99 > 500ms
Recall Rate	RAGAS faithfulness	< 0.7
Cache Hit Rate	Redis stats	< 80%
Vector DB Disk	node_disk_usage	> 85%

6.3 Regular Update Strategy

#!/bin/bash
# update_rag_index.sh
COLLECTION="ops_knowledge"
DATE=$(date +%Y%m%d)

# Incremental update
python3 scripts/incremental_ingest.py \
    --collection $COLLECTION \
    --since "$DATE-7" \
    --embed-model bge-m3

# Full rebuild (every Sunday)
if [ "$(date +%u)" -eq 7 ]; then
    python3 scripts/full_rebuild.py --collection $COLLECTION
fi

---

7. Common Troubleshooting

Issue 1: Retrieval results are irrelevant to the query

Cause: Embedding model mismatch or similarity threshold too high.

Fix:

# Lower the threshold
search_kwargs={"k": 10, "score_threshold": 0.3}

# Switch Embedding model
embeddings = OllamaEmbeddings(model="bge-m3")

Issue 2: Semantic breaks after long document chunking

Cause: chunk_size too small or separators misconfigured.

Fix:

splitter = RecursiveCharacterTextSplitter(
    chunk_size=1024,
    chunk_overlap=128,
    separators=["\n## ", "\n# ", "\n\n", "\n", " "]
)

Issue 3: Vector DB OOM

Cause: Too many vectors without optimized indexing.

Fix: - Milvus: Increase nlist parameter, adjust index type (HNSW / IVF_PQ) - Chroma: Upgrade to version 0.5+, enable persistence

Issue 4: Poor multi-language retrieval

Cause: Single Embedding model cannot cover all languages.

Fix: Use multilingual models (bge-m3, mxbai-embed-large) or build separate databases per language.

---

8. Further Reading

• LangChain Documentation
• RAGAS Evaluation Framework
• Milvus Official Documentation
• ChromaDB Documentation

---

FAQ

How to choose an Embedding model? For English: text-embedding-3-small (OpenAI compatible) or text-embedding-3-large. For multilingual: bge-m3. For local deployment: GPTFuzz/bge-large-zh or all-MiniLM-L6-v2.

What is the best practice for document chunking? Chunk by semantic boundaries (paragraphs/sections), not fixed character counts. Use LangChain's RecursiveCharacterTextSplitter with chunk_size=512 and chunk_overlap=64. For code docs, use the Language='python' parameter to preserve structure.

Vector search recall rate is too low, what to do? Check: ① Whether the Embedding model matches the language ② Whether chunk_size is reasonable ③ Whether the similarity threshold is too high (default 0.5, try 0.3) ④ Whether hybrid search (sparse + dense) is enabled.

How to evaluate RAG system performance? Use the RAGAS framework: faithfulness, answer_relevancy, context_precision. Build a test set, run evaluation scripts, and perform regular regression testing.

How to back up and restore a vector database? Milvus: export .vec files regularly. Chroma: back up chroma.sqlite3 + chroma-data/ directory. After restore, rebuild indexes and verify vector counts match.

---