Skip to content

fix: cap poller batch size at config limit and make cleanup/requestShutdown optional#622

Merged
jumski merged 1 commit into
mainfrom
fix/slot-backpressure-review-fixes
Jun 3, 2026
Merged

fix: cap poller batch size at config limit and make cleanup/requestShutdown optional#622
jumski merged 1 commit into
mainfrom
fix/slot-backpressure-review-fixes

Conversation

@jumski
Copy link
Copy Markdown
Contributor

@jumski jumski commented May 28, 2026

The Worker class previously held a direct reference to a postgres.Sql connection and always closed it on shutdown. This coupling meant the worker unconditionally owned the SQL connection regardless of whether it was externally provided.

This change introduces a WorkerOptions interface that replaces the positional requestShutdown and sql constructor arguments. The worker now accepts an optional cleanup callback instead of a SQL instance directly, and only invokes it if one is provided. Both createFlowWorker and createQueueWorker track whether they created the SQL connection themselves (ownsSql), and only pass a cleanup callback that closes the connection when they own it. When a SQL connection is passed in via config, the worker leaves lifecycle management of that connection to the caller.

requestShutdown on PlatformAdapter is now optional, and both worker factories use optional chaining when binding it. SupabasePlatformAdapter.stopWorker wraps the worker stop call in a try/finally block so the platform-owned SQL connection is always closed even if the worker stop throws.

The batch size limit parameter in ReadWithPollPoller and StepTaskPoller is now capped at the configured batchSize using Math.min, preventing a caller from requesting more messages than the poller is configured to handle.

New tests cover the cleanup callback invocation on worker stop, the ownsSql-conditional SQL teardown behavior, the sql.end guarantee when worker stop rejects, the optional requestShutdown type contract, and the corrected batch size capping logic in both poller implementations.

@changeset-bot
Copy link
Copy Markdown

changeset-bot Bot commented May 28, 2026

⚠️ No Changeset found

Latest commit: 5509849

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Copy link
Copy Markdown
Contributor Author

jumski commented May 28, 2026

@nx-cloud
Copy link
Copy Markdown

nx-cloud Bot commented May 28, 2026

View your CI Pipeline Execution ↗ for commit 5509849

Command Status Duration Result
nx test:types:health dsl ✅ Succeeded 12s View ↗

☁️ Nx Cloud last updated this comment at 2026-06-03 16:33:44 UTC

Copy link
Copy Markdown
Contributor Author

jumski commented Jun 3, 2026

Merge activity

  • Jun 3, 4:31 PM UTC: A user started a stack merge that includes this pull request via Graphite.
  • Jun 3, 4:33 PM UTC: Graphite rebased this pull request as part of a merge.
  • Jun 3, 4:33 PM UTC: @jumski merged this pull request with Graphite.

@jumski jumski changed the base branch from feat/slot-aware-backpressure to graphite-base/622 June 3, 2026 16:31
@jumski jumski changed the base branch from graphite-base/622 to main June 3, 2026 16:32
@jumski jumski force-pushed the fix/slot-backpressure-review-fixes branch from dc209fb to 5509849 Compare June 3, 2026 16:32
@jumski jumski merged commit 79c2313 into main Jun 3, 2026
13 checks passed
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 3, 2026

🚀 Production Deployment: Website

Successfully deployed to production!

🔗 Production URL: https://pgflow.dev

📝 Details:

  • Commit: 79c23137d181abfde78fc3951bddd9ceff5b16a0
  • View Logs

Deployed at: 2026-06-03T18:33:30+02:00

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant