Skip to content

Build multi-destination replicationInfo per backend#6172

Open
maeldonn wants to merge 2 commits into
development/9.4from
improvement/CLDSRV-888/crr-multi
Open

Build multi-destination replicationInfo per backend#6172
maeldonn wants to merge 2 commits into
development/9.4from
improvement/CLDSRV-888/crr-multi

Conversation

@maeldonn
Copy link
Copy Markdown
Contributor

Issue: CLDSRV-888

@bert-e
Copy link
Copy Markdown
Contributor

bert-e commented May 18, 2026

Hello maeldonn,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Available options
name description privileged authored
/after_pull_request Wait for the given pull request id to be merged before continuing with the current one.
/bypass_author_approval Bypass the pull request author's approval
/bypass_build_status Bypass the build and test status
/bypass_commit_size Bypass the check on the size of the changeset TBA
/bypass_incompatible_branch Bypass the check on the source branch prefix
/bypass_jira_check Bypass the Jira issue check
/bypass_peer_approval Bypass the pull request peers' approval
/bypass_leader_approval Bypass the pull request leaders' approval
/approve Instruct Bert-E that the author has approved the pull request. ✍️
/create_pull_requests Allow the creation of integration pull requests.
/create_integration_branches Allow the creation of integration branches.
/no_octopus Prevent Wall-E from doing any octopus merge and use multiple consecutive merge instead
/unanimity Change review acceptance criteria from one reviewer at least to all reviewers
/wait Instruct Bert-E not to run until further notice.
Available commands
name description privileged
/help Print Bert-E's manual in the pull request.
/status Print Bert-E's current status in the pull request TBA
/clear Remove all comments from Bert-E from the history TBA
/retry Re-start a fresh build TBA
/build Re-start a fresh build TBA
/force_reset Delete integration branches & pull requests, and restart merge process from the beginning.
/reset Try to remove integration branches unless there are commits on them which do not appear on the source branch.

Status report is not available.

@codecov
Copy link
Copy Markdown

codecov Bot commented May 18, 2026

❌ 47 Tests Failed:

Tests completed Failed Passed Skipped
1805 47 1758 0
View the top 3 failed test(s) by shortest run time
"after all" hook for "should replace archived current null version in version-suspended bucket"::Put object with same key as prior object With default signature "after all" hook for "should replace archived current null version in version-suspended bucket"
Stack Traces | 0.001s run time
connect ECONNREFUSED 127.0.0.1:8000
"after all" hook for "should replace archived current null version in version-suspended bucket"::Put object with same key as prior object With v4 signature "after all" hook for "should replace archived current null version in version-suspended bucket"
Stack Traces | 0.001s run time
connect ECONNREFUSED 127.0.0.1:8000
"after each" hook for "should be 'PENDING' when object key prefix applies"::Head object 'ReplicationStatus' value With default signature "after each" hook for "should be 'PENDING' when object key prefix applies"
Stack Traces | 0.346s run time
socket hang up
View the full list of 44 ❄️ flaky test(s)
"after all" hook for "should return NoSuchKey if try to put object ACLs for nonexistent object"::PUT Object ACL With default signature "after all" hook for "should return NoSuchKey if try to put object ACLs for nonexistent object"

Flake rate in main: 100.00% (Passed 0 times, Failed 7 times)

Stack Traces | 0.001s run time
connect ECONNREFUSED 127.0.0.1:8000
"after all" hook for "should return NoSuchKey if try to put object ACLs for nonexistent object"::PUT Object ACL With v4 signature "after all" hook for "should return NoSuchKey if try to put object ACLs for nonexistent object"

Flake rate in main: 100.00% (Passed 0 times, Failed 7 times)

Stack Traces | 0.001s run time
connect ECONNREFUSED 127.0.0.1:8000
"after each" hook for "should allow unauthenticated putObjectLegalHold without VersionId"::PUT object legal hold iam action and version id With default signature "after each" hook for "should allow unauthenticated putObjectLegalHold without VersionId"

Flake rate in main: 100.00% (Passed 0 times, Failed 7 times)

Stack Traces | 0.002s run time
connect ECONNREFUSED 127.0.0.1:8000
"after each" hook for "should allow unauthenticated putObjectLegalHold without VersionId"::PUT object legal hold iam action and version id With v4 signature "after each" hook for "should allow unauthenticated putObjectLegalHold without VersionId"

Flake rate in main: 100.00% (Passed 0 times, Failed 7 times)

Stack Traces | 0.001s run time
connect ECONNREFUSED 127.0.0.1:8000
"after each" hook for "should allow unauthenticated putObjectRetention without VersionId"::PUT object retention iam action and version id With default signature "after each" hook for "should allow unauthenticated putObjectRetention without VersionId"

Flake rate in main: 100.00% (Passed 0 times, Failed 7 times)

Stack Traces | 0.001s run time
connect ECONNREFUSED 127.0.0.1:8000
"after each" hook for "should allow unauthenticated putObjectRetention without VersionId"::PUT object retention iam action and version id With v4 signature "after each" hook for "should allow unauthenticated putObjectRetention without VersionId"

Flake rate in main: 100.00% (Passed 0 times, Failed 7 times)

Stack Traces | 0.001s run time
connect ECONNREFUSED 127.0.0.1:8000
"after each" hook for "should be `undefined` when there is no bucket replication config"::Head object 'ReplicationStatus' value With v4 signature "after each" hook for "should be `undefined` when there is no bucket replication config"

Flake rate in main: 100.00% (Passed 0 times, Failed 7 times)

Stack Traces | 0.128s run time
connect ECONNREFUSED 127.0.0.1:8000
"after each" hook for "should fail if version id is invalid"::PUT object with x-scal-s3-version-id header With default signature "after each" hook for "should fail if version id is invalid"

Flake rate in main: 100.00% (Passed 0 times, Failed 7 times)

Stack Traces | 0.03s run time
The specified bucket does not exist.
"after each" hook for "should put an object and set the acl via query param"::PUT object With default signature "after each" hook for "should put an object and set the acl via query param"

Flake rate in main: 100.00% (Passed 0 times, Failed 11 times)

Stack Traces | 0.001s run time
connect ECONNREFUSED 127.0.0.1:8000
"after each" hook for "should put an object and set the acl via query param"::PUT object With v4 signature "after each" hook for "should put an object and set the acl via query param"

Flake rate in main: 100.00% (Passed 0 times, Failed 51 times)

Stack Traces | 0.001s run time
connect ECONNREFUSED 127.0.0.1:8000
"after each" hook for "should put object with valid object lock retention date and mode when object lock is enabled on the bucket"::PUT object with object lock With default signature "after each" hook for "should put object with valid object lock retention date and mode when object lock is enabled on the bucket"

Flake rate in main: 100.00% (Passed 0 times, Failed 25 times)

Stack Traces | 0.001s run time
connect ECONNREFUSED 127.0.0.1:8000
"after each" hook for "should put object with valid object lock retention date and mode when object lock is enabled on the bucket"::PUT object with object lock With v4 signature "after each" hook for "should put object with valid object lock retention date and mode when object lock is enabled on the bucket"

Flake rate in main: 100.00% (Passed 0 times, Failed 7 times)

Stack Traces | 0.001s run time
connect ECONNREFUSED 127.0.0.1:8000
"after each" hook for "should return AccessDenied putting legal hold with another account"::PUT object legal hold With default signature "after each" hook for "should return AccessDenied putting legal hold with another account"

Flake rate in main: 100.00% (Passed 0 times, Failed 7 times)

Stack Traces | 0s run time
Cannot read properties of undefined (reading 'length')
"after each" hook for "should return AccessDenied putting legal hold with another account"::PUT object legal hold With v4 signature "after each" hook for "should return AccessDenied putting legal hold with another account"

Flake rate in main: 100.00% (Passed 0 times, Failed 7 times)

Stack Traces | 0s run time
Cannot read properties of undefined (reading 'length')
"after each" hook for "should return AccessDenied putting retention with another account"::PUT object retention With default signature "after each" hook for "should return AccessDenied putting retention with another account"

Flake rate in main: 100.00% (Passed 0 times, Failed 7 times)

Stack Traces | 0.001s run time
connect ECONNREFUSED 127.0.0.1:8000
"after each" hook for "should return AccessDenied putting retention with another account"::PUT object retention With v4 signature "after each" hook for "should return AccessDenied putting retention with another account"

Flake rate in main: 100.00% (Passed 0 times, Failed 7 times)

Stack Traces | 0.002s run time
connect ECONNREFUSED 127.0.0.1:8000
"after each" hook for "should return Not Implemented error for obj. encryption using customer-provided encryption keys"::PUT object With default signature "after each" hook for "should return Not Implemented error for obj. encryption using customer-provided encryption keys"

Flake rate in main: 100.00% (Passed 0 times, Failed 7 times)

Stack Traces | 0.001s run time
connect ECONNREFUSED 127.0.0.1:8000
"after each" hook for "should return Not Implemented error for obj. encryption using customer-provided encryption keys"::PUT object With v4 signature "after each" hook for "should return Not Implemented error for obj. encryption using customer-provided encryption keys"

Flake rate in main: 100.00% (Passed 0 times, Failed 7 times)

Stack Traces | 0s run time
connect ECONNREFUSED 127.0.0.1:8000
"after each" hook for "should return tags if tags are valid"::PUT object taggings With default signature "after each" hook for "should return tags if tags are valid"

Flake rate in main: 100.00% (Passed 0 times, Failed 7 times)

Stack Traces | 0.001s run time
connect ECONNREFUSED 127.0.0.1:8000
"after each" hook for "should return tags if tags are valid"::PUT object taggings With v4 signature "after each" hook for "should return tags if tags are valid"

Flake rate in main: 100.00% (Passed 0 times, Failed 7 times)

Stack Traces | 0.001s run time
connect ECONNREFUSED 127.0.0.1:8000
"before all" hook for "should list buckets concurrently"::GET Service - AWS.S3.listBuckets With v4 signature when user has credential "before all" hook for "should list buckets concurrently"

Flake rate in main: 100.00% (Passed 0 times, Failed 7 times)

Stack Traces | 0.03s run time
socket hang up
"before all" hook for "should overwrite all user metadata and data on overwrite put"::Put object with same key as prior object With default signature "before all" hook for "should overwrite all user metadata and data on overwrite put"

Flake rate in main: 100.00% (Passed 0 times, Failed 7 times)

Stack Traces | 0.005s run time
connect ECONNREFUSED 127.0.0.1:8000
"before all" hook for "should overwrite all user metadata and data on overwrite put"::Put object with same key as prior object With v4 signature "before all" hook for "should overwrite all user metadata and data on overwrite put"

Flake rate in main: 100.00% (Passed 0 times, Failed 7 times)

Stack Traces | 0.002s run time
connect ECONNREFUSED 127.0.0.1:8000
"before all" hook for "should put object ACLs"::PUT Object ACL With default signature "before all" hook for "should put object ACLs"

Flake rate in main: 100.00% (Passed 0 times, Failed 7 times)

Stack Traces | 0.002s run time
connect ECONNREFUSED 127.0.0.1:8000
"before all" hook for "should put object ACLs"::PUT Object ACL With v4 signature "before all" hook for "should put object ACLs"

Flake rate in main: 100.00% (Passed 0 times, Failed 7 times)

Stack Traces | 0.002s run time
connect ECONNREFUSED 127.0.0.1:8000
"before each" hook for "should allow unauthenticated putObjectLegalHold without VersionId"::PUT object legal hold iam action and version id With default signature "before each" hook for "should allow unauthenticated putObjectLegalHold without VersionId"

Flake rate in main: 100.00% (Passed 0 times, Failed 7 times)

Stack Traces | 0.002s run time
connect ECONNREFUSED 127.0.0.1:8000
"before each" hook for "should allow unauthenticated putObjectLegalHold without VersionId"::PUT object legal hold iam action and version id With v4 signature "before each" hook for "should allow unauthenticated putObjectLegalHold without VersionId"

Flake rate in main: 100.00% (Passed 0 times, Failed 7 times)

Stack Traces | 0.002s run time
connect ECONNREFUSED 127.0.0.1:8000
"before each" hook for "should allow unauthenticated putObjectRetention without VersionId"::PUT object retention iam action and version id With default signature "before each" hook for "should allow unauthenticated putObjectRetention without VersionId"

Flake rate in main: 100.00% (Passed 0 times, Failed 7 times)

Stack Traces | 0.002s run time
connect ECONNREFUSED 127.0.0.1:8000
"before each" hook for "should allow unauthenticated putObjectRetention without VersionId"::PUT object retention iam action and version id With v4 signature "before each" hook for "should allow unauthenticated putObjectRetention without VersionId"

Flake rate in main: 100.00% (Passed 0 times, Failed 7 times)

Stack Traces | 0.002s run time
connect ECONNREFUSED 127.0.0.1:8000
"before each" hook for "should be `undefined` when there is no bucket replication config"::Head object 'ReplicationStatus' value With v4 signature "before each" hook for "should be `undefined` when there is no bucket replication config"

Flake rate in main: 100.00% (Passed 0 times, Failed 7 times)

Stack Traces | 0.029s run time
socket hang up
"before each" hook for "should fail if version id is invalid"::PUT object with x-scal-s3-version-id header With default signature "before each" hook for "should fail if version id is invalid"

Flake rate in main: 100.00% (Passed 0 times, Failed 7 times)

Stack Traces | 40s run time
Timeout of 40000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (.../test/object/putVersion.js)
"before each" hook for "should put an object and set the acl via query param"::PUT object With default signature "before each" hook for "should put an object and set the acl via query param"

Flake rate in main: 100.00% (Passed 0 times, Failed 7 times)

Stack Traces | 0.003s run time
connect ECONNREFUSED 127.0.0.1:8000
"before each" hook for "should put an object and set the acl via query param"::PUT object With v4 signature "before each" hook for "should put an object and set the acl via query param"

Flake rate in main: 100.00% (Passed 0 times, Failed 7 times)

Stack Traces | 0.002s run time
connect ECONNREFUSED 127.0.0.1:8000
"before each" hook for "should put object with valid object lock retention date and mode when object lock is enabled on the bucket"::PUT object with object lock With default signature "before each" hook for "should put object with valid object lock retention date and mode when object lock is enabled on the bucket"

Flake rate in main: 100.00% (Passed 0 times, Failed 25 times)

Stack Traces | 0.002s run time
connect ECONNREFUSED 127.0.0.1:8000
"before each" hook for "should put object with valid object lock retention date and mode when object lock is enabled on the bucket"::PUT object with object lock With v4 signature "before each" hook for "should put object with valid object lock retention date and mode when object lock is enabled on the bucket"

Flake rate in main: 100.00% (Passed 0 times, Failed 7 times)

Stack Traces | 0.001s run time
connect ECONNREFUSED 127.0.0.1:8000
"before each" hook for "should return AccessDenied putting legal hold with another account"::PUT object legal hold With default signature "before each" hook for "should return AccessDenied putting legal hold with another account"

Flake rate in main: 100.00% (Passed 0 times, Failed 7 times)

Stack Traces | 0.003s run time
connect ECONNREFUSED 127.0.0.1:8000
"before each" hook for "should return AccessDenied putting legal hold with another account"::PUT object legal hold With v4 signature "before each" hook for "should return AccessDenied putting legal hold with another account"

Flake rate in main: 100.00% (Passed 0 times, Failed 7 times)

Stack Traces | 0.002s run time
connect ECONNREFUSED 127.0.0.1:8000
"before each" hook for "should return AccessDenied putting retention with another account"::PUT object retention With default signature "before each" hook for "should return AccessDenied putting retention with another account"

Flake rate in main: 100.00% (Passed 0 times, Failed 7 times)

Stack Traces | 0.002s run time
connect ECONNREFUSED 127.0.0.1:8000
"before each" hook for "should return AccessDenied putting retention with another account"::PUT object retention With v4 signature "before each" hook for "should return AccessDenied putting retention with another account"

Flake rate in main: 100.00% (Passed 0 times, Failed 7 times)

Stack Traces | 0.001s run time
connect ECONNREFUSED 127.0.0.1:8000
"before each" hook for "should return Not Implemented error for obj. encryption using customer-provided encryption keys"::PUT object With default signature "before each" hook for "should return Not Implemented error for obj. encryption using customer-provided encryption keys"

Flake rate in main: 100.00% (Passed 0 times, Failed 7 times)

Stack Traces | 0.003s run time
connect ECONNREFUSED 127.0.0.1:8000
"before each" hook for "should return Not Implemented error for obj. encryption using customer-provided encryption keys"::PUT object With v4 signature "before each" hook for "should return Not Implemented error for obj. encryption using customer-provided encryption keys"

Flake rate in main: 100.00% (Passed 0 times, Failed 7 times)

Stack Traces | 0.002s run time
connect ECONNREFUSED 127.0.0.1:8000
"before each" hook for "should return tags if tags are valid"::PUT object taggings With default signature "before each" hook for "should return tags if tags are valid"

Flake rate in main: 100.00% (Passed 0 times, Failed 7 times)

Stack Traces | 0.002s run time
connect ECONNREFUSED 127.0.0.1:8000
"before each" hook for "should return tags if tags are valid"::PUT object taggings With v4 signature "before each" hook for "should return tags if tags are valid"

Flake rate in main: 100.00% (Passed 0 times, Failed 7 times)

Stack Traces | 0.002s run time
connect ECONNREFUSED 127.0.0.1:8000
should be 'PENDING' when object key prefix applies::Head object 'ReplicationStatus' value With default signature With bucket replication config should be 'PENDING' when object key prefix applies

Flake rate in main: 13.81% (Passed 668 times, Failed 107 times)

Stack Traces | 0.034s run time
socket hang up

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

@scality scality deleted a comment from bert-e May 20, 2026
@bert-e
Copy link
Copy Markdown
Contributor

bert-e commented May 20, 2026

Waiting for approval

The following approvals are needed before I can proceed with the merge:

  • the author

  • 2 peers

@maeldonn maeldonn force-pushed the improvement/CLDSRV-888/crr-multi branch from 545c4b6 to 0ff3941 Compare May 27, 2026 07:47
// The role is derived from the top-level template with the
// rule's Account (V2) substituting the account segment.
const destination = rule.destination ?? config.destination;
const role = ReplicationConfiguration.resolveDestinationRole(config.role, rule.account);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ReplicationConfiguration.resolveDestinationRole does not exist in Arsenal. It is not in the pinned 8.4.2 tag, nor on any branch in the scality/Arsenal repo. This will throw a TypeError: ReplicationConfiguration.resolveDestinationRole is not a function at runtime whenever a CRR backend is encountered.

Either a companion Arsenal PR needs to land and the dependency version bumped here, or this logic needs to be implemented inline.

— Claude Code

@claude
Copy link
Copy Markdown

claude Bot commented May 27, 2026

  • ReplicationConfiguration.resolveDestinationRole (line 166 of getReplicationInfo.js) does not exist in Arsenal — not in the pinned 8.4.2 tag, not on any branch, and not in any open PR across the scality org. This will crash at runtime for CRR backends and the unit tests referencing CRR destinations will also fail.
    - Add a companion Arsenal PR with this method and bump the arsenal dependency, or implement the role-resolution logic inline.

    Review by Claude Code

Match all enabled rules with the object's prefix (highest-priority wins
on site collision) and stamp destination/role per CRR backend. Cloud
backends get a bare entry — their bucket and credentials live in the
location config. ACL replication uses per-backend `role` as the CRR
marker so cloud backends keep their status. Legacy V1 and
comma-separated `StorageClass` remain supported on read.

Issue: CLDSRV-888
@maeldonn maeldonn force-pushed the improvement/CLDSRV-888/crr-multi branch from 0ff3941 to b98f98f Compare May 28, 2026 15:17
// services, by design)
const isCloud = site => !!replicationBackends[s3config.locationConstraints[site]?.type];
const rules = _withDefaultStorageClass(config.rules || [], s3config);
const backends = ReplicationConfiguration.resolveBackends(
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ReplicationConfiguration.resolveBackends is not a function in arsenal 8.4.2. CI shows 110 test failures with TypeError: ReplicationConfiguration.resolveBackends is not a function. The arsenal dependency in package.json needs to be bumped to a version that exports resolveBackends and resolveSourceRole as static methods on ReplicationConfiguration.

— Claude Code

@claude
Copy link
Copy Markdown

claude Bot commented May 28, 2026

  • Arsenal dependency not bumped: ReplicationConfiguration.resolveBackends and resolveSourceRole do not exist in arsenal 8.4.2. All 110 unit test failures are TypeError: ReplicationConfiguration.resolveBackends is not a function. Bump the arsenal dependency in package.json to a version that exports these static methods.
    - Prettier check failing on all 4 changed files. Run prettier --write on them.

    The refactoring logic itself (service-account gate extraction, default-storageClass helper, per-backend CRR/cloud handling in ACL flow) looks correct and well-tested, pending the arsenal dependency.

    Review by Claude Code

@maeldonn maeldonn requested review from a team, benzekrimaha and delthas May 29, 2026 08:58
@bert-e
Copy link
Copy Markdown
Contributor

bert-e commented May 29, 2026

Conflict

There is a conflict between your branch improvement/CLDSRV-888/crr-multi and the
destination branch development/9.4.

Please resolve the conflict on the feature branch (improvement/CLDSRV-888/crr-multi).

git fetch && \
git checkout origin/improvement/CLDSRV-888/crr-multi && \
git merge origin/development/9.4

Resolve merge conflicts and commit

git push origin HEAD:improvement/CLDSRV-888/crr-multi

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants