Skip to content

Backend

Eagle-RAG's backend is an industry-agnostic, multi-tenant multimodal RAG data layer for Agents and LLMs. It combines FastAPI (HTTP API), Celery (async ingest), LlamaIndex (retrieval orchestration), Milvus (per-domain vector databases), PostgreSQL (metadata/audit), and an in-process plugin microkernel (eagle_rag/plugins/). Deploy-time plugin_namespace binds a Milvus Database and PostgreSQL repository filter; request-time kb_name isolates knowledge bases inside that domain. See Plugin architecture.

Application entry: eagle_rag/api/app.py. No auth middleware on REST (intranet). Schema migrations: task db:migrate.

What is a RAG backend?

Retrieval-Augmented Generation augments an LLM with external knowledge. Eagle-RAG implements four responsibilities:

Responsibility Module Key papers
Ingest — parse, chunk, embed, index eagle_rag/ingest/ + Celery Karpukhin et al. DPR (arXiv:2004.04906)
Retrieve — ANN + graph expansion + cross-modal eagle_rag/retrievers/ G-Retriever (arXiv:2402.07629), CLIP (arXiv:2103.00020)
Route — text / visual / hybrid selection eagle_rag/router/ Self-RAG (arXiv:2310.11511)
Generate — rerank + VLM + cited answer eagle_rag/generation/ Lewis et al. RAG (arXiv:2005.11401), cross-encoder rerank (arXiv:1901.04085)

Eagle-RAG extends text RAG with a dual-pipeline architecture: structured parsing via Knowhere and visual tile encoding via PixelRAG (pixelrag_render + Qwen3-VL embedding). Both write to separate Milvus collections, fused at retrieval via four anchor fields on visual tiles.

Layered architecture

flowchart TD
    CLIENT(["Client / Agent"]) --> API["API layer<br/>FastAPI + MCP /mcp"]
    API --> KERNEL["Plugin microkernel<br/>PluginManager · HookBus · orchestrators"]
    KERNEL --> SERVICES["Service layer<br/>router engine · generation · ingest runner"]
    SERVICES --> ADAPTERS["Adapters<br/>Knowhere SDK · PixelRAG library"]
    SERVICES --> RETRIEVERS["Retrievers<br/>KnowhereGraphRetriever · PixelRAGVisualRetriever"]
    KERNEL --> RETRIEVERS
    ADAPTERS --> REPOS["Repositories + Milvus<br/>eagle_text/eagle_visual · specialized collections"]
    RETRIEVERS --> REPOS
    REPOS --> INFRA["Infrastructure<br/>PostgreSQL · Milvus · Redis · MinIO"]
    API --> TASKS["Task queue<br/>3 Celery queues + dead-letter"]
    TASKS --> ADAPTERS

Two cross-cutting flows:

  1. Ingest — document in → vectors out (API → Celery → adapters → Milvus).
  2. Query — question in → cited answer out (API → route → retrieve → rerank → VLM).

See architecture/data-flow for sequence diagrams.

Dual database strategy

Driver Placeholder Used by
asyncpg (async) $1, $2 FastAPI handlers
psycopg2 (sync) %s Celery tasks, sync stores

Celery workers cannot share the asyncpg pool. Alembic normalizes DSN in alembic/env.py.

Milvus collections and domain isolation

Each deployed instance binds one Milvus Database via MilvusClientPool (eagle_rag/index/milvus_pool.py) — db_name is set at client construction, not switched per request. Base collections in every domain Database:

Collection Dim Metric Index Embed model
eagle_text 1536 COSINE HNSW (LlamaIndex) Qwen text-embedding-v4
eagle_visual 2048 IP HNSW M=16, efConstruction=256 Qwen3-VL-Embedding-2B

Domain plugins may add specialized collections (e.g. eagle_text_biomed). Core default routing never auto-queries those collections (G4); only a domain QueryRouteClassifier or scope-aware catalog union may add them.

KB isolation inside a domain: kb_name == "{tenant}" on every query. Scope union: (kb_name in [...] or document_id in [...]). Cross-domain retrieval uses multiple instances, not Core fan-out across Milvus Databases.

LlamaIndex integration map

LlamaIndex type Eagle-RAG role
TextNode Knowhere chunks + section summaries
ImageNode Visual retrieval hits
VectorStoreIndex Text ANN over eagle_text
MilvusVectorStore LlamaIndex ↔ Milvus text bridge
BaseRetriever KnowhereGraphRetriever, PixelRAGVisualRetriever
CustomQueryEngine EagleMultimodalQueryEngine
DashScopeRerank Cross-encoder text reranking
MetadataFilters kb_name / scope / facet → Milvus expr

Visual vectors bypass LlamaIndex vector store (managed by pymilvus directly).

Documentation index

Each page includes theoretical background, code walkthrough, design tensions and tuning (parameter-level trade-offs tied to code paths), Milvus schema/expr, LlamaIndex mapping, config keys, tests, and references.

Cross-cutting tensions (where to read more)

Symptom Start here
Missing chunks in answers retrieval §8, vector-stores §8 (ef, filters)
Weak or noisy citations generation §6 (top_k/top_n, visual rerank gap)
Wrong pipeline / parser ingest-pipeline §6, router-engine §7
Ingest stuck / duplicate tiles task-queue §9 (acks, DLQ)
Tenant or scope leaks retrieval §8, kb-management §8
Plugin / namespace mismatch database §3, Plugin architecture
Page Lines Scope
api-layer 200+ FastAPI app, routers, SSE, query engine singleton
ingest-pipeline 400+ runner, router, Knowhere/PixelRAG adapters, Celery tasks
retrieval 400+ KnowhereGraphRetriever, PixelRAGVisualRetriever, filters
vector-stores 400+ eagle_text + eagle_visual schema, index params, registry
router-engine 350+ route_query selectors, EagleRouterQueryEngine
generation 350+ split, rerank, prompt, VLM streaming
task-queue 300+ Celery config, with_retry, dead-letter
storage 200+ MinIO, dedup, attachments, image store
database 250+ SQLModel tables, Alembic, ER diagram
kb-management 200+ KB registry, lifecycle, stats, health
admin-module 200+ MCP log, queue metrics, config snapshot
sessions-notifications 200+ Chat sessions, scope persistence, notifications
mcp-server 250+ Core MCP tools (core_*), plugin registration, HTTP/stdio, resilience
schemas 250+ Pydantic v2 request/response contracts

Cross-cutting concerns

Documented in the architecture section:

  • Plugin architecture — microkernel, hooks, orchestrators, MCP G3 filter, RAG-only boundary.
  • Multi-tenancy — plugin_namespace (domain) + kb_name (KB), dedup PK, Milvus db_name, repository filter, MinIO prefix.
  • Reliability — lazy singletons, fail-closed Knowhere, fail-fast PixelRAG, best-effort writes.
  • Routing matrix — ingest pipeline selection by format + content form.
  • Multimodal fusion — four visual anchor fields, parent-document retrieval.

Model stack (DeepSeek + Qwen only)

Use Model Dim
Text LLM / routing DeepSeek (deepseek-v4-pro) —
VLM generation Qwen-VL (qwen3.6-flash) —
Text embedding Qwen (text-embedding-v4) 1536
Visual embedding Qwen3-VL-Embedding-2B 2048
Text rerank Qwen (qwen3-rerank) —

No OpenAI / Cohere adapters. Configuration: eagle_rag/settings.yaml.

Key test files

Area Test file
Retrievers tests/test_retrievers.py
Router + generation tests/test_router_generation.py
Ingest routing tests/test_ingest_assets.py, tests/test_ingest_smoke.py
Knowhere sections tests/test_knowhere_sections.py
Visual chunks tests/test_knowhere_visual_chunks.py
Milvus structure tests/test_milvus_structure_fetch.py
API integration tests/test_api_query_sessions_documents_tasks.py
MCP tests/test_mcp_*.py
Plugins tests/plugins/test_*.py
Attachments tests/test_attachments_parser.py

Run: uv run pytest tests/

Quick start

uv sync
task db:migrate
uv run uvicorn eagle_rag.api.app:app --host 0.0.0.0 --port 8000

# Celery workers (separate terminals)
celery -A eagle_rag.tasks.celery_app worker -Q router_queue -c 4
celery -A eagle_rag.tasks.celery_app worker -Q knowhere_queue -c 8
celery -A eagle_rag.tasks.celery_app worker -Q pixelrag_queue -c 1

References