Skip to content

Evidence Aggregation

This page systematizes the evidence aggregation concept in Eagle-RAG's semantic-layer retrieval: how chunks recalled from multiple sources are consolidated into a coherent, de-duplicated, rankable evidence set for generation. It is the downstream of Multi-vector retrieval — multi-vector handles retrieval-time multi-space fan-out, evidence aggregation handles post-retrieval consolidation — and the upstream of Citationware RAG — the aggregated evidence set is the material basis for citations.

Positioning

This page is the concept canonical. The existing primitives are implemented (RRF fusion / structural dedupe / candidate injection / merged rerank) but at the "rank-fusion + structural dedupe" level, not semantic aggregation. Semantic-layer gaps are in §5 Roadmap.

1. Definition

Evidence aggregation = consolidating chunks recalled from multiple sources (multiple collections, modalities, graph expansion) that may be duplicate or overlapping into a set that is:

  1. De-duplicated — collapsing duplicate / near-duplicate evidence;
  2. Rankable — a unified ordering by relevance;
  3. Coherent — a compact evidence set consumable by generation within a finite context window.

Eagle-RAG's evidence aggregation happens between RetrieverOrchestrator.retrieve() and eagle_rag/router/rerank_fusion.py, as the middle layer after multi-vector fan-out and before generation.

2. Existing primitives (implemented)

Primitive Function Behavior
Cross-plan RRF fusion merge_rrf (rerank_fusion.py L37) Fuses results of multiple CollectionQueryPlan by 1/(k+rank); empty result sets contribute no phantom rank (G8)
Cross-collection dedupe dedupe_cross_collection (rerank_fusion.py L65) Collapses duplicate logical chunks by source_chunk_id or (document_id, path), keeping the higher-ranked (G32), records rrf_dedupe audit
Candidate injection inject_supplement_candidates (rerank_fusion.py L96) Ensures supplement top hits enter the rerank pool
Merged rerank rerank_merged (rerank_fusion.py L143) Per RerankPolicy: domain→RERANK_MERGED hook; general→qwen3-rerank; none→truncate

Orchestration entry

RetrieverOrchestrator.retrieve() (eagle_rag/plugins/retriever_orchestrator.py L77):

per-plan ANN
  → optional per-plan RERANK (Tier-1, within space)
  → merge_rrf (L177)                 # cross-space rank fusion
  → dedupe_cross_collection (L180)   # structural dedupe
  → optional RRF_POST_MERGE hook (L190)  # domain candidate injection
  → rerank_merged (L200)             # merged rerank
  → dedupe by source_chunk_id / (document_id, path)

3. Relationship to plugin hooks

Every step of evidence aggregation is open to plugin extension (see Plugin architecture for the hook catalog):

Hook Trigger Role in evidence aggregation
RETRIEVE_SUPPLEMENT after ANN, before RRF (L166) Domain plugins supplement recall (e.g. entity-anchored exact hits), ensuring key evidence enters the fusion pool
RRF_POST_MERGE after merge_rrf, before rerank_merged (L190) Domain plugins inject extra candidates onto the RRF result (e.g. rule-boosted chunks)
RERANK_MERGED inside rerank_merged Domain plugins rerank the merged evidence with a domain cross-encoder

dedupe_cross_collection records an rrf_dedupe audit event (PluginAudit) on collapse, so dedupe intensity is traceable via GET /health/plugins recent_decisions (see Agent observability Layer 4).

4. Evidence aggregation vs pure ranking

Boundary clarification — the current mechanism is rank-fusion + structural dedupe, not semantic aggregation:

Dimension Current capability Semantic aggregation (ideal)
Dedupe basis structural id (source_chunk_id / (doc_id, path)) semantic similarity (near-duplicates collapsed even with different ids)
Evidence organization flat ranked list grouped / clustered by sub-topic
Conflict handling none (conflicting evidence listed in parallel) conflict detection and resolution / flagging
Confidence retrieval score preserved, no aggregate confidence cross-evidence aggregate answer-level confidence
Compression none long-evidence summarization

5. Roadmap (not implemented)

The following are unimplemented design gaps, not current capabilities.

  • Semantic-similarity dedupe: dedupe_cross_collection currently collapses only by structural id. A dedupe_mode="semantic" could use a lightweight embedding (or reuse the query encoder) to compute inter-chunk similarity and collapse near-duplicates above a cos threshold. Potential extension: a semantic_threshold param on dedupe_cross_collection + a new EVIDENCE_DEDUPE hook.
  • Evidence grouping / clustering: cluster top_n evidence by sub-topic so generation can organize the answer by group. Potential extension: a new EVIDENCE_GROUP hook emitting list[EvidenceGroup] for prompt construction.
  • Cross-evidence conflict resolution: detect contradictory statements about the same fact across evidence, flag or prefer one. Potential extension: a conflict-detection hook after RERANK_MERGED.
  • Aggregate confidence: fuse multi-evidence retrieval scores into an answer-level confidence (max / mean / weighted), consumed by Citationware RAG abstain decisions and Agent observability quality metrics.
  • Evidence summarization: summarize long evidence sets when over token budget instead of plain truncation.

6. Hand-off to generation / citation

The aggregated top_n evidence enters the VLM prompt via EagleMultimodalQueryEngine._prepare_generation (see Generation §3.3), and is simultaneously built into the sources list via text_sources_from_nodes / _image_source returned to the frontend. The evidence ordering directly determines the context the VLM sees and the [n] citation order of Citationware RAG.

7. References