Build a Retrieval Budget: A Practical RAG Workflow for Agent Tasks

This article shows how to turn broad RAG retrieval into a repeatable workflow: define the decision, retrieve a narrow evidence set, inspect the full source when needed, and record what actually supported the result.

1. Start with the decision, not the document corpus

The most common RAG mistake is beginning with a vague request such as “find everything relevant.” That instruction optimizes for volume, not usefulness. Before retrieval, write the decision the agent must support in one sentence:

Decide whether the proposed article angle is genuinely new, and identify two repository sources that can ground it.

This sentence gives retrieval a job. It also tells the later reviewer what “good enough” means.

Next, define the evidence types that could answer the decision. For an article-planning task, that might be:

• recent work logs showing what has shipped or been learned;
• existing drafts showing topics that are already covered;
• implementation or policy documents describing current behavior;
• research notes containing a concrete example or unresolved tradeoff.

Do not treat all matches as equally valuable. A recent shipping log may be better evidence for “what changed lately” than an older overview, while a runtime contract should outrank a casual note when the question is about enforced behavior. Retrieval should narrow the search space; authority and freshness still determine what you trust.

2. Use a two-pass retrieval budget

A practical workflow separates discovery from confirmation.

Pass one: retrieve candidate slices

Set a small initial budget—perhaps five to ten candidate excerpts—and search using the decision’s vocabulary plus the concrete subject. For a RAG tradeoff article, useful query terms might include retrieval, RAG, canon, freshness, work log, and draft.

At this stage, the goal is not to write. Make a table with four columns:

| Candidate | Why it might matter | Authority | Freshness | |---|---|---|---| | Recent work log | May contain new operational evidence | note/checkpoint | high | | Runtime contract | Defines actual behavior | policy/contract | verify date | | Existing article draft | Prevents duplicate thesis | draft | current | | General overview | Provides terminology | guide | possibly stale |

The table exposes a key RAG tradeoff: more retrieved text can increase recall while making it harder to notice which source actually answers the question. A candidate list preserves recall without allowing every match into the final context.

Pass two: confirm the winners

Choose the two or three candidates that best match the decision, then read the complete source—not only the retrieved slice. Check surrounding headings, dates, status labels, and caveats. If two same-level sources disagree and freshness is unclear, do not silently blend them. Surface the conflict and choose a conservative interpretation.

This second pass matters because top-k retrieval is a navigation aid, not a complete representation of the source. A snippet can suggest that a feature exists while the surrounding document says it is planned, disabled, or review-gated.

3. Apply authority before similarity

Similarity is useful for finding candidates, but it is a poor final ranking rule. A highly similar note should not override a current contract merely because it contains the same words.

Use a simple ranking order:

1. current runtime contract or policy;
2. current execution specification or acceptance checklist;
3. active implementation plan;
4. recent notes, checkpoints, or stories.

Then apply freshness within the same tier. Mark each selected source with a short status such as verified, current, advisory, or needs audit. This makes the reasoning visible to the next person who reviews the output.

For example, suppose a retrieved note says that an agent can publish automatically, while the current job instructions say drafts remain on disk until approval. The job instruction governs the article workflow. The note can still be useful historical context, but it should not be presented as current behavior.

This is where a retrieval budget protects quality: with only a few selected sources, the agent has enough context to compare authority deliberately instead of unconsciously following whichever passage appeared first.

4. Convert evidence into an evidence ledger

Before drafting, record how each source will be used. A lightweight ledger is enough:

```text Decision: Is this article angle new and supportable?

Source: recent work log Use: concrete example of a retrieval or verification lesson Status: current Supports: why the workflow is needed

Source: runtime or job contract Use: current rule for handling retrieved canon and drafts Status: verified Supports: what the workflow must not claim

Source: existing draft index Use: duplicate-topic check Status: current Supports: why this thesis is distinct ```

The ledger prevents a frequent failure mode: citing a source for a claim it does not actually support. It also makes revision faster. If a reviewer challenges one sentence, you can inspect the source-to-claim link instead of rereading the entire retrieval history.

Keep negative findings too. “No recent shipping evidence found” is useful information when it explains why the article uses a how-to format rather than pretending to be a case study. Absence should narrow the claim, not invite invented detail.

5. Test the draft against the retrieval boundary

Once the draft is written, run a final boundary check. For every factual or operational claim, ask:

• Is this directly supported by a selected source?
• Is the source current enough for the claim?
• Did I turn a recommendation into a statement about existing behavior?
• Did I accidentally reuse the thesis of an existing draft?
• Would a reader know what to verify before applying this workflow?

Then inspect the article’s examples. Examples should clarify the method without masquerading as production results. A table showing how to rank sources is safe; an invented customer outcome is not.

The final check should also look for retrieval-shaped prose: paragraphs that repeat several source summaries without making a decision. If a section does not help the reader choose, verify, or act, remove it. RAG is an input technique, not a reason to expose the whole search process.

Next steps

To implement this workflow, start with one recurring task. Write its decision sentence, cap the first retrieval pass, and keep a source table. Confirm only the strongest candidates in full, rank them by authority and freshness, and preserve a small evidence ledger beside the draft. After a few runs, review which sources repeatedly prove useful and refine the query vocabulary or budget.

The goal is not maximum context. It is controlled context: enough evidence to support a decision, enough provenance to review it, and a clear boundary around what the agent does not know.