Skip to content

Citationware RAG

This page systematizes Eagle-RAG's "citation-first RAG" concept: every generated claim must be traceable to retrieved evidence, and citations are first-class citizens rather than optional decoration. The material basis of citations comes from the evidence set consolidated by Evidence aggregation; observability and quality validation can plug into the Langfuse score integration of Agent observability.

Current maturity

Eagle-RAG has landed answer-level [n] inline citation + hovercard, which is "prompt-guided soft citation." Claim-level span grounding, faithfulness validation, etc. are in §5 Roadmap.

1. Definition

Citationware RAG (citation-first RAG) denotes a class of RAG systems where every generated statement (claim) must be backed by retrieved evidence and carry verifiable provenance — citations are part of the answer structure, not an after-thought, optional source list.

Maturity splits into two tiers:

Tier Form Eagle-RAG status
L1 answer-level The answer text uses [n] indexes to reference the sources list; the frontend hovercard displays them implemented
L2 claim-level Each claim → source span-level grounding mapping + faithfulness validation + abstain not implemented (roadmap)

2. Existing citation chain (implemented)

2.1 Backend: citation-coordinate schema

In eagle_rag/api/schemas/query.py, TextSource and ImageSource carry the full citation coordinates:

# eagle_rag/api/schemas/query.py
class TextSource(BaseModel):       # L71
    # citation coordinates: path / document_id / score / content / page_nums / keywords / source_chunk_id
    ...

class ImageSource(BaseModel):      # L98
    # four anchors: image_id / chunk_type / parent_section / content_summary / source_chunk_id
    ...

class QueryResponse(BaseModel):    # L148
    answer: str
    sources: QuerySources          # L152 = {text: list[TextSource], image: list[ImageSource]}
    route: RouteInfo
    steps: list[QueryStep]

The four anchors of ImageSource (image_id / chunk_type / parent_section / content_summary / source_chunk_id) let visual evidence be section-attributed and cross-collection-linked (see the anchor fields in Multimodal fusion).

2.2 Backend: source construction

eagle_rag/generation/multimodal_engine.py maps the reranked NodeWithScore to sources:

Function Role
text_sources_from_nodes (L857) text nodes → TextSource list, with registry-backfilled file_name
_text_source (L780) single node → source dict (path / level / score / content)
_enrich_text_sources (L821) enriches sources with the file-name registry
_image_source (L863) visual node → ImageSource (with four anchors)

2.3 Prompt: [n] index guidance

The VLM prompt requires the model to reference reference text with [n] indexes and to describe images without fabricating URLs (see Generation §1.4 grounding and citation). This is "soft citation" — prompt-guided, not structurally guaranteed.

2.4 Frontend: inline citation hovercard

frontend/components/qa/sources-utils.ts implements inline-citation alignment and display:

Function Line Role
flattenSources L11 flattens {text, image} into a 1-based indexed list, aligning with [n] in the answer
findImageSourceIndex L27 looks up the 1-based index of a visual source by image_id
sourceCitationExcerpt L122 the evidence excerpt body of the hovercard
sourceCitationTitle L135 the title line of the hovercard
resolveAnswerImageSrc L199 remaps a VLM-fabricated external image URL to the real /images/{image_id}

3. Citation pipeline

flowchart LR
    R["retrieve"] --> RR["rerank"]
    RR --> TN["top_n evidence"]
    TN --> SRC["text_sources_from_nodes / _image_source<br/>build sources list"]
    TN --> PROMPT["VLM prompt<br/>guides [n] indexes"]
    SRC --> RESP["QueryResponse.sources"]
    PROMPT --> RESP2["QueryResponse.answer (with [n])"]
    RESP & RESP2 --> FE["frontend flattenSources<br/>1-based index aligns with [n]"]
    FE --> HC["hovercard<br/>sourceCitationExcerpt/Title"]
    FE --> IMG["resolveAnswerImageSrc<br/>image URL remap"]

Key coupling: the order of the sources list must match the order of [n] in the answer — the frontend flattenSources produces 1-based indexes, [1] maps to sources.text[0]. The evidence ordering (from Evidence aggregation) directly determines the citation numbering.

4. Current maturity positioning

What is landed is L1 answer-level soft citation:

  • the sources list carries full citation coordinates (path / document_id / page_nums / source_chunk_id);
  • the VLM prompt guides [n] indexes;
  • frontend hovercard + image URL remap;
  • [n] is prompt-guided, not structurally guaranteed — the model may omit, mis-label, or fabricate indexes;
  • no claim → source span-level mapping, no post-generation validation.

5. Roadmap (not implemented)

The following are unimplemented design gaps, not current capabilities.

  • claim → source span-level grounding mapping: today [n] is an answer-level citation. The ideal is a structured citations: list[ClaimCitation], each carrying claim_text / source_index / source_span (a character range within the source). Potential extension: add a citations field to QueryResponse; have the VLM emit structured citation JSON or post-process [n] → span.
  • faithfulness / groundedness validation: after generation, validate that each claim is truly backed by a cited source. Potential extension: add a GROUNDING_CHECK hook after RERANK_MERGED or after generation; plug into Langfuse score (faithfulness / citation coverage) for evaluation.
  • abstain / uncitable detection: when evidence is insufficient, the system should explicitly abstain ("evidence insufficient") instead of forcing a generation. Potential extension: combine with the aggregate-confidence threshold of Evidence aggregation.
  • citation coverage check: count the proportion of uncited claims in the answer and expose it as a quality metric to Agent observability.
  • conflicting-evidence annotation: when multiple sources contradict on the same claim, annotate the conflict in the citation (connects to the conflict-resolution roadmap of Evidence aggregation).

6. Connection to observability

Citation quality can be plugged into Agent observability as an observation metric:

  • Langfuse score (not implemented): faithfulness / citation coverage / abstain rate, reported as scores on the generation observation;
  • structlog AI JSONL (implemented): the generate event already records prompt/completion (truncated); it can be extended to record citation_count / uncited_claim_count;
  • Prometheus (implemented): a new metric like eagle_citation_coverage could be added (roadmap).

7. References