[WIP] #380 feat: implement dual checksum calculation #381
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…se class refactoring
- Add separate checksums for source and test inputs for enhanced build incrementality
- Implement web-path compatible cache key format: `{source_checksum}-{test_checksum}`
- Increment cache version to v1.2 (breaking change)
- Create AbstractInputAnalyzer base class to eliminate code duplication
- Refactor SrcInputAnalyzer and TestInputAnalyzer to extend base class
- Reduce code duplication by ~300 lines (60% reduction)
- Add comprehensive tests for dual checksum functionality
Enhanced build logic:
- Source-only changes: rebuild only if source checksum differs
- Test-only changes: rebuild only if test checksum differs OR source changes
- Both changed: full rebuild required
- Neither changed: use cached results
maximiln
changed the title
|
|
||
| // Add all input files to the hash | ||
| for (Path inputFile : inputFiles) { | ||
| checksum.update(inputFile.toString().getBytes()); |
There was a problem hiding this comment.
getBytes() is using the platform's default charset, for portability between win/linux it is better to set encoding.
| String sourceChecksum = combinedChecksum.substring(0, separatorIndex); | ||
| String testChecksum = combinedChecksum.substring(separatorIndex + 1); | ||
|
|
||
| return new String[] {sourceChecksum, testChecksum}; |
| if (sourceChanged && testChanged) { | ||
| return RebuildType.FULL_REBUILD; | ||
| } else if (sourceChanged) { | ||
| return RebuildType.SOURCE_REBUILD; |
There was a problem hiding this comment.
If the source changes, then the tests should be rebuilt too. For example, they might stop compiling.
| final SortedMap<String, String> pluginDependenciesChecksum = getMutablePluginDependencies(); | ||
| final String sourceChecksum = dualCalculator.calculateSourceChecksum(); | ||
| final String testChecksum = dualCalculator.calculateTestChecksum(); | ||
| final String combinedChecksum = dualCalculator.calculateDualChecksum(); |
There was a problem hiding this comment.
this will recalculate again sourceChecksum and testChecksum - could be calculated once
| final HashChecksum checksum = config.getHashFactory().createChecksum(count); | ||
| // Create digest items for the new dual checksum approach | ||
| final List<DigestItem> items = new ArrayList<>(); | ||
| final HashChecksum checksum = config.getHashFactory().createChecksum(3); |
There was a problem hiding this comment.
what is 3 in createChecksum(3)?
| DigestItem fileDigest = DigestUtils.file(checksum, baseDirPath, file); | ||
| items.add(fileDigest); | ||
| if (compareWithBaseline) { | ||
| sourcesMatched &= checkItemMatchesBaseline(baselineHolder.get(), fileDigest); |
There was a problem hiding this comment.
that was a diff report (compare 2 buildinfos from different sources to find mismatches). is it moved or deprecated?
maximiln
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Separate Source and Test Input Checksums for Enhanced Build Incrementality #380
Enhanced build logic:
Following this checklist to help us incorporate your
contribution quickly and easily:
Note that commits might be squashed by a maintainer on merge.
This may not always be possible but is a best-practice.
mvn verifyto make sure basic checks pass.A more thorough check will be performed on your pull request automatically.
mvn -Prun-its verify).If your pull request is about ~20 lines of code you don't need to sign an
Individual Contributor License Agreement if you are unsure
please ask on the developers list.
To make clear that you license your contribution under
the Apache License Version 2.0, January 2004
you have to acknowledge this by using the following check-box.