[feat] S3 uploads integrity protections

[das-Abroxas]

Goal

Implement support for S3 default data integrity protections to maintain compatibility with modern S3 clients.

AWS introduced automatic client-side integrity protections for S3 uploads. The latest S3 clients now automatically calculate CRC-based checksums for each upload and send them to S3, which validates them before storing objects. This extends data integrity protection all the way to client-side applications, protecting against corruption during network transmission. Aruna currently does not support these checksums, which breaks compatibility with clients using updated AWS SDKs. This issue tracks the implementation of checksum support to restore compatibility.

Workplan

1. Update s3s dependency to 0.12.0 and fix breaking changes

   • [chore] Update s3s to 0.12.0 #226

2. Implement support for CRC32, CRC32C, and CRC64NVME checksum calculation

   • [feat] Add CRC checksum calculation support #218

3. Update data model in the Dataproxy component to support object integrity validation

   • [feat] Extend data model for object/part checksums #225

4. Integrate object integrity validation in the Dataproxy S3 interface for the relevant operations

Definition of Done

• Single part object uploads work with checksum integrity validation for all algorithms (SHA256, MD5, CRC32, CRC32C, CRC64NVME)
• Multipart uploads work with checksum integrity validation for all algorithms (SHA256, MD5, CRC32, CRC32C, CRC64NVME)
• Checksums are stored in part/object metadata
• Compatibility verified with AWS SDK clients (boto3, Java SDK v2, AWS CLI, ...)

Test Concept

• Additional Unit tests for newly integrated checksum algorithm (CRC32, CRC32C, CRC64NVME)
• Integration tests for single part object uploads with different checksums
• Integration tests for multipart uploads with trailing checksums and full-object checksums
• Manual testing with AWS SDK v2 clients

Risks / Blockers

• s3s 0.12.0 release date:
  Version 0.12.0 of the s3s crate is scheduled for release on December 21, 2025. Tasks dependent on this release cannot be finalized until this stable version of the dependency is available.