Version 0.7 rework #14
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- X25519 private key generation - X25519 private key to/from ASN.1 encoded PEM conversion
- Currently only reads directory
- Rename command Modify to Append - Add List subcommand to Read - Implement List, Info and All subcommand of Read
# Conflicts: # Cargo.lock # crates/pithos/Cargo.toml # crates/pithos/src/main.rs # crates/pithos_lib/Cargo.toml # crates/pithos_lib/src/lib.rs # crates/pithos_lib/src/transformers/mod.rs # crates/pithos_pyo3/Cargo.toml
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #14 +/- ##
==========================================
- Coverage 69.91% 68.39% -1.53%
==========================================
Files 33 27 -6
Lines 3497 4445 +948
==========================================
+ Hits 2445 3040 +595
- Misses 1052 1405 +353 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Overview
This pull request represents a substantial development effort rewriting the core functionality for the Pithos file format specification. The changes include a complete implementation of the Pithos file format with encryption, compression, and CLI tooling, alongside the addition of basic RO-Crate metadata support. This work spans from the initial specification draft through to a fully functional system with comprehensive testing and documentation.
Key Changes
• Complete Pithos file format implementation: Added PithosWriter and PithosReader with support for encrypted file storage using X25519 key exchange and ChaCha20Poly1305 encryption
• Comprehensive CLI tooling: Implemented full command-line interface supporting file creation, extraction, appending, listing, and Crypt4GH conversion operations
• RO-Crate metadata integration: Added complete RO-Crate library with support for directory and ZIP formats, including validation framework and programmatic creation via builder patterns
• Advanced compression and chunking: Integrated FastCDC content-defined chunking with configurable Zstd compression and automatic compression ratio detection
• Robust cryptographic infrastructure: Implemented X25519 key management, ChaCha20Poly1305 encryption helpers, and secure key derivation for multi-recipient encryption
• Directory structure preservation: Added support for ingesting and recreating complete directory hierarchies with proper metadata preservation
• Extensive test coverage: Added comprehensive test suites covering encryption/decryption, serialization, key management, and file operations with utility functions and dummy data
• Remote block reading capability: Implemented foundation for content-addressable storage with IndexMap-based block indexing for future distributed storage features