NES: opt-in global-budget cascade for prompt parts#315565
Merged
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Adds an experiment-gated “global budget” cascade for NES (xtab) prompt construction so multiple prompt parts share a single token pool with surplus flowing to later parts, similar to completions-core’s cascading prompt factory.
Changes:
- Introduces
GlobalBudgetOptions/globalBudgetinPromptOptions, with defaults for order/shares/total tokens. - Implements the cascade path in
getUserPromptand refactors recent-files helpers to support independent sub-builder invocation. - Wires new experiment settings in the xtab provider and adds unit tests for cascade parity/validation.
Show a summary per file
| File | Description |
|---|---|
| extensions/copilot/src/platform/inlineEdits/common/dataTypes/xtabPromptOptions.ts | Adds global-budget types + defaults and plumbs globalBudget into PromptOptions. |
| extensions/copilot/src/platform/configuration/common/configurationService.ts | Adds experiment-based config keys for enabling global budget and setting totalTokens. |
| extensions/copilot/src/extension/xtab/common/promptCrafting.ts | Implements the global-budget cascade and switches getUserPrompt between legacy vs cascade paths. |
| extensions/copilot/src/extension/xtab/common/recentFilesForPrompt.ts | Extracts helpers (prepareRecentCodeSnippets, exports appenders) to enable cascade sub-builders. |
| extensions/copilot/src/extension/xtab/node/xtabProvider.ts | Wires the new experiment flags into constructed promptOptions.globalBudget. |
| extensions/copilot/src/extension/xtab/test/common/promptCrafting.spec.ts | Adds tests for cascade parity (large budgets) and order validation. |
Copilot's findings
- Files reviewed: 6/6 changed files
- Comments generated: 2
roblourens
approved these changes
May 10, 2026
When opts.globalBudget is set, recentlyViewedDocuments, languageContext, neighborFiles and diffHistory share a single token pool: each part is allocated totalTokens*shares[part] plus the cumulative surplus from earlier parts in the cascade order. Mirrors completions-core's CascadingPromptFactory model. currentFile and lintOptions stay outside the budget. The legacy code path is byte-identical when globalBudget is undefined.
Adds two exp-based config keys: - chat.advanced.inlineEdits.xtabProvider.globalBudget.enabled (default false) - chat.advanced.inlineEdits.xtabProvider.globalBudget.totalTokens (default 6000) When enabled, populates PromptOptions.globalBudget with the default cascade order and shares matching today's per-part maxTokens ratios.
…onfig Address PR review: - Each sub-builder now returns tokensConsumed measured with its own internal accounting (paged-clipping line cost, raw-snippet cost for appenders, diff-entry cost for history). The cascade uses that value to compute surplus so it stays consistent with how each part charges against its own budget. - Validate GlobalBudgetOptions at runtime: no duplicates in order, every part in order has a share entry, and shares sum to ~1 (epsilon 1e-3). Tests added for each failure mode.
ulugbekna
force-pushed
the
ulugbekna/nes-global-budget-cascade
branch
from
c455673 to
2933ad2
Compare
ulugbekna
added
the
~release-cherry-pick
vs-code-engineering
Bot
added
release-cherry-pick
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Behind a new exp flag, NES prompt parts (recentlyViewedDocuments, languageContext, neighborFiles, diffHistory) share a single token pool with cascading surplus, mirroring completions-core's `CascadingPromptFactory`. `currentFile` and `lintOptions` stay outside the budget.
Today
Each NES prompt part has its own absolute `maxTokens` cap (e.g. `recentlyViewedDocuments: 2000`, `diffHistory: 1000`). Unused tokens in one part are wasted — they cannot flow to another part. By contrast, completions-core's production path uses a single `promptTokenLimit` with weighted/percentage allocation; surplus from earlier components naturally feeds later ones.
Change
Add an opt-in `globalBudget` field to `PromptOptions`. When set, `getUserPrompt` runs a cascade: each part gets `surplus + totalTokens * shares[part]`, sub-builders are invoked with that override (existing logic untouched), unspent budget cascades to the next part.
Default order: `languageContext → recentlyViewedDocuments → neighborFiles → diffHistory` (neighborFiles must follow recentlyViewed because it consults `docsInPrompt`; enforced at runtime). Default shares match today's `maxTokens` ratios over a 6000-token total — volume-neutral baseline.
Exp flags
Regression safety
When `globalBudget` is undefined, `getUserPrompt` takes the legacy code path verbatim. A new parity test asserts the cascade with a large `totalTokens` produces byte-identical output to the legacy path. All 252 xtab unit tests pass.
Commits