feat: add GitLab Runner scaler#7473
Conversation
|
Thank you for your contribution! 🙏 Please understand that we will do our best to review your PR and give you feedback as soon as possible, but please bear with us if it takes a little longer as expected. While you are waiting, make sure to:
Once the initial tests are successful, a KEDA member will ensure that the e2e tests are run. Once the e2e tests have been successfully completed, the PR may be merged at a later date. Please be patient. Learn more about our contribution guide. |
✅ Snyk checks have passed. No issues have been found so far.
💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse. |
|
Semgrep found 21
Detected directly writing or similar in 'http.ResponseWriter.write()'. This bypasses HTML escaping that prevents cross-site scripting vulnerabilities. Instead, use the 'html/template' package and render data using 'template.Execute()'. |
jmif
force-pushed
the
jmif/gitlab-runner-scaler
branch
4 times, most recently
from
72c8f42 to
2f6af07
Compare
![semgrep-code-kedacore[bot]](https://avatars.githubusercontent.com/u/49917779?s=60&v=4){kind=small}
jmif
force-pushed
the
jmif/gitlab-runner-scaler
branch
from
2f6af07 to
6d2d790
Compare
Add a new scaler that scales based on pending jobs in GitLab CI/CD pipelines. Supports project and group scopes, tag-based job filtering, and uses the x-total header optimization for efficient job counting. Signed-off-by: jmif <jmif@users.noreply.github.com> Signed-off-by: Joe Mifsud <joe@mifsud.me>
jmif
force-pushed
the
jmif/gitlab-runner-scaler
branch
from
6d2d790 to
3f86e5d
Compare
|
/run-e2e gitlab |
There was a problem hiding this comment.
Pull request overview
This PR introduces a new GitLab Runner Scaler that enables KEDA to scale workloads based on pending jobs in GitLab CI/CD pipelines. The scaler queries the GitLab Jobs API to determine queue length and supports both project-level and group-level scoping, with tag-based filtering for job matching and efficient x-total header usage when tag filtering is not required.
Changes:
- Added new GitLab Runner scaler implementation supporting project and group scopes with tag-based job filtering
- Comprehensive unit and e2e test suites with mock GitLab API server
- Schema definitions for scaler configuration parameters
Reviewed changes
Copilot reviewed 5 out of 7 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| pkg/scalers/gitlab_runner_scaler.go | Core scaler implementation with GitLab API integration, tag filtering logic, and pagination support |
| pkg/scalers/gitlab_runner_scaler_test.go | Comprehensive unit tests covering metadata parsing, API interactions, tag filtering, and error scenarios |
| tests/scalers/gitlab_runner/gitlab_runner_test.go | E2e tests with mock nginx-based GitLab API server testing scale out/in behavior |
| schema/generated/scalers-schema.yaml | YAML schema definition for the gitlab-runner scaler parameters |
| schema/generated/scalers-schema.json | JSON schema definition for the gitlab-runner scaler parameters |
| pkg/scaling/scalers_builder.go | Registration of the new scaler in the builder |
| CHANGELOG.md | Changelog entry documenting the new scaler |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
zroubalik
left a comment
zroubalik
left a comment
There was a problem hiding this comment.
Supports
ScaledObjectandScaledJob, withFromEnvparameter variants for all configuration options.
I haven't reviewed the PR yet, but TBH I am not sure we should do this, I would rather completely remove the FromEnv method from KEDA.
CC @kedacore/keda-core-maintainers
Yes, true. Let's not allow this for new scalers. |
same with me |
|
Updated - removed all resolvedEnv/FromEnv support from the scaler. |
Remove resolvedEnv/FromEnv support per maintainer guidance that new scalers should not use this pattern. Remove redundant 'optional' from fields that already have 'default' values. Refactor URL construction to use net/url for safe encoding and path escaping. Add max page limit (100) to pagination loops to prevent unbounded iteration. Remove URL and response body from error messages to avoid SSRF information leaks. Regenerate scalers schema to reflect metadata changes. Signed-off-by: Joe Mifsud <joe@mifsud.me>
jmif
force-pushed
the
jmif/gitlab-runner-scaler
branch
from
f7869a2 to
769b6ff
Compare
|
/run-e2e gitlab |
| if rem, _ := strconv.Atoi(remaining); rem == 0 { | ||
| resetTime, _ := strconv.ParseInt(r.Header.Get("RateLimit-Reset"), 10, 64) | ||
|
|
||
| return nil, r.Header, r.StatusCode, fmt.Errorf("GitLab API rate limit exceeded, resets at %s", time.Unix(resetTime, 0)) |
| func (s *gitlabRunnerScaler) getGitLabRequest(ctx context.Context, url string) ([]byte, http.Header, int, error) { | ||
| req, err := http.NewRequestWithContext(ctx, "GET", url, nil) |
| s.logger.Info("reached maximum page limit for group projects", "maxPages", gitlabMaxPages) | ||
| break |
rickbrouwer
added
the
waiting-author-response
Add a new scaler that scales based on pending jobs in GitLab CI/CD pipelines. The scaler queries the GitLab Jobs API for pending jobs, supports both project and group scopes, tag-based job filtering, and uses the x-total header for efficient job counting when tag filtering is not needed.
Supports
ScaledObjectandScaledJob, withFromEnvparameter variants for all configuration options.Checklist
Fixes #5616
Relates to kedacore/keda-docs#1705