fix(code-index): limit embedding batch size by item count to prevent provider 422 errors

[proyectoauraorg]

Related GitHub Issue

Closes: #335

Description

Root cause: Embedders batched by token count only (MAX_BATCH_TOKENS=100000). When many small code segments are sent, they all fit in one token-based batch (e.g., 60 segments × ~100 tokens = 6000 < 100000), resulting in API calls with 60+ items. Providers like qwen3-embedding via LiteLLM enforce a maximum of 32 items per call, returning HTTP 422.

Fix:

1. Added MAX_BATCH_ITEMS = 32 constant in src/services/code-index/constants/index.ts
2. Applied item count constraint alongside the existing token limit in all four embedders:
   • OpenAICompatibleEmbedder — the one reported in [BUG] Batch Size Exceeded #335
   • OpenAIEmbedder
   • OpenRouterEmbedder
   • BedrockEmbedder

The batching condition changed from:

if (currentBatchTokens + itemTokens <= MAX_BATCH_TOKENS)

to:

if (currentBatchTokens + itemTokens <= MAX_BATCH_TOKENS && currentBatch.length < MAX_BATCH_ITEMS)

The existing codeIndex.embeddingBatchSize setting (scanner-level, default 60) remains configurable. The embedder-level limit ensures no single API call exceeds 32 items regardless of scanner batch size.

Test Procedure

• Unit tests: All 53 existing tests pass (no regressions)
• Type check: Passes cleanly
• Manual test: Configure a provider with batch limit <60 (e.g., qwen3-embedding via LiteLLM), run code indexing → should no longer get HTTP 422

Pre-Submission Checklist

• Issue Linked: Closes [BUG] Batch Size Exceeded #335
• Scope: Focused — only adds item count batching constraint to embedders
• Self-Review: Thorough review performed
• Testing: All existing tests pass
• Documentation Impact: No documentation changes needed
• Contribution Guidelines: Read and agreed

Summary by CodeRabbit

• Bug Fixes
  • Enhanced embedding batch management to enforce a maximum item-count limit across all embedding services, improving request reliability and API compliance.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: b5e15439-efa8-4e63-a78a-75f7307d61fb

📥 Commits

Reviewing files that changed from the base of the PR and between 3df406e and 7e7bdea.

📒 Files selected for processing (5)

• src/services/code-index/constants/index.ts
• src/services/code-index/embedders/bedrock.ts
• src/services/code-index/embedders/openai-compatible.ts
• src/services/code-index/embedders/openai.ts
• src/services/code-index/embedders/openrouter.ts

---

📝 Walkthrough

Walkthrough

A maximum embedding batch item limit of 32 is defined as a constant and enforced across Bedrock, OpenAI, OpenAI-compatible, and OpenRouter embedders. Each embedder's batch-building logic now checks both token capacity and item-count capacity before adding texts to a batch.

Changes

Embedding batch item limits

Layer / File(s)	Summary
Add MAX_BATCH_ITEMS constant and update all embedders src/services/code-index/constants/index.ts, src/services/code-index/embedders/bedrock.ts, src/services/code-index/embedders/openai-compatible.ts, src/services/code-index/embedders/openai.ts, src/services/code-index/embedders/openrouter.ts	The constant MAX_BATCH_ITEMS = 32 is defined and imported into all four embedder implementations. Each embedder's batch-building condition is updated to enforce currentBatch.length < MAX_BATCH_ITEMS alongside the existing token limit, ensuring batches never exceed the item count allowed by embedding APIs.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Poem

🐰 A batch of thirty-two, no more to take,
Four embedders now respect the limit's stake,
From Bedrock, OpenAI, and Router's way,
Each batch behaves the API's say. ✨

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title 'fix(code-index): limit embedding batch size by item count to prevent provider 422 errors' clearly and concisely describes the main change—adding item-count batching constraints to embedding calls.
Description check	✅ Passed	The description comprehensively covers root cause, implementation details, test results, and follows the template with all major sections completed including linked issue, description, test procedure, and pre-submission checklist.
Linked Issues check	✅ Passed	The PR fully addresses issue #335 by implementing item-count batching limits (MAX_BATCH_ITEMS=32) in all four embedders to prevent 422 errors when batch sizes exceed provider limits.
Out of Scope Changes check	✅ Passed	All changes are focused on the batching constraint fix: adding MAX_BATCH_ITEMS constant and applying it to four embedders' batching conditions, with no unrelated modifications.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 ESLint

If the error stems from missing dependencies, add them to the package.json file. For unrecoverable errors (e.g., due to private dependencies), disable the tool in the CodeRabbit configuration.

src/services/code-index/constants/index.ts

ESLint skipped: missing config or dependency (missing-dependency). The ESLint configuration references a package that is not available in the sandbox.

src/services/code-index/embedders/bedrock.ts

ESLint skipped: the ESLint configuration for this file references a package that is not available in the sandbox.

src/services/code-index/embedders/openai-compatible.ts

ESLint skipped: the ESLint configuration for this file references a package that is not available in the sandbox.

• 2 others

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!