Replace xargs -0 with POSIX-compliant alternative (fixes #11485) #11489

gnodet · 2025-11-24T15:31:23Z

Description

This PR fixes #11485 by replacing the non-POSIX-compliant xargs -0 option with a pure shell solution that works on all POSIX-compliant systems including AIX, FreeBSD, and others.

Problem

The xargs -0 option was introduced in commit aeff353 to handle special characters (pipes, quotes, spaces) in .mvn/jvm.config files. However, this option is not part of the POSIX standard and is not available on AIX, FreeBSD, and other systems, causing the Maven wrapper to fail on these platforms.

Solution

The fix replaces xargs -0 with a pure shell solution that:

Uses the read builtin with -d '' option to read NUL-terminated strings
Builds the result string incrementally in a while loop
Maintains the same functionality of handling special characters correctly
Works on all POSIX-compliant systems

Changes

Modified concat_lines() function in apache-maven/src/assembly/maven/bin/mvn
Replaced the pipeline using xargs -n 1 -0 with a while loop using read -r -d ''
Preserved all existing functionality for handling special characters

Testing

All existing tests pass, including the integration test for pipes in MAVEN_OPTS (MavenITgh10937QuotedPipesInMavenOptsTest)
Manually tested the concat_lines function with special characters
Full build completed successfully with mvn clean install

Compatibility

This change maintains backward compatibility while extending support to more platforms. The behavior for handling special characters in .mvn/jvm.config remains unchanged.

Pull Request opened by Augment Code with guidance from the PR author

bmarwell · 2025-11-24T16:27:53Z

Hi @gnodet, I just checked.
Only AIX (as far as I know) is missing out the -0 parameter. I have not checked Haiko or OpenIndiana, etc. FreeBSD does have -0.

For AIX: we could work around that by requiring AIX users of Maven to use the "Open Source Toolbox". I tested that successfully.

However, the assumption is correct that -0 is not a POSIX standard: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/xargs.html

Given we can work around this OR just require it anyway, I am open to either solution.

The xargs -0 option is not POSIX-compliant and is not available on AIX, FreeBSD, and other systems. This commit replaces the use of xargs -0 in the concat_lines function with a pure shell solution that reads NUL-terminated strings using the read builtin with -d '' option. The new implementation: - Maintains the same functionality of handling special characters like pipes, quotes, and spaces correctly - Avoids the non-portable xargs -0 option - Uses a while loop with read -r -d '' to process NUL-terminated strings - Builds the result string incrementally instead of using tr to convert newlines to spaces This fix ensures that the mvn script works correctly on all POSIX-compliant systems while preserving the behavior introduced in commit aeff353 to handle quoted pipes and other special characters in .mvn/jvm.config. Fixes apache#11485

bmarwell · 2025-11-25T11:24:28Z

apache-maven/src/assembly/maven/bin/mvn

+    result=""
+    # Read the file line by line
+    while IFS= read -r line || [ -n "$line" ]; do
+      # Convert CR to LF
+      line=$(echo "$line" | tr '\r' '\n')
+      # Remove comments
+      line=$(echo "$line" | sed 's/#.*$//')
+      # Skip empty lines
+      [ -z "$(echo "$line" | tr -d ' \t')" ] && continue
+
+      # Process each argument in the line using eval to handle quotes
+      eval "set -- $line"
+      for arg in "$@"; do
+        # Replace variables
+        arg=$(echo "$arg" | sed \
+          -e "s@\${MAVEN_PROJECTBASEDIR}@$MAVEN_PROJECTBASEDIR@g" \
+          -e "s@\$MAVEN_PROJECTBASEDIR@$MAVEN_PROJECTBASEDIR@g")
+
+        # Quote the argument if it contains spaces or special shell characters
+        case "$arg" in
+          *[\ \|\&\;\<\>\(\)\$\`\\\"\'\~\*\?\[\]\#\~\=]*)
+            arg="\"$arg\""
+            ;;
+        esac
+
+        if [ -n "$result" ]; then
+          result="$result $arg"
+        else
+          result="$arg"
+        fi
+      done


imho this should be an extra function

we might want to document why we do not use xargs -0 here and that we agreed to use posix-only arguments (did we agree on that?)

Not sure what you mean by extra function. The concat_lines function is the code above (but for an if which verifies the argument is actually a file...

I just thought we might want to extract a method/function here, because we get a lot of indentation and the method gets longer and longer.

Did we agree on "we must be POSIX compliant"? If not, we could leave xargs in here and just concentrate on the sed part.

I think 30 lines is still manageable.

Yes, unless there's a strong reason, I think keeping POSIX compliant script is better. maven-wrapper has an additional job to validate posix compliance, we may want to do the same: https://github.com/apache/maven-wrapper/blob/master/.github/workflows/shellcheck-posix.yml

Great, I fully agree!

bmarwell

AIX is gloriously stubborn — ancient userland tools (no GNU xargs -0) and a nonstandard /bin/sh make portability a landmine, so avoiding non‑POSIX flags is absolutely the right goal.

bmarwell · 2025-11-27T20:20:53Z

apache-maven/src/assembly/maven/bin/mvn

+      [ -z "$(echo "$line" | tr -d ' \t')" ] && continue
+
+      # Process each argument in the line using eval to handle quotes
+      eval "set -- $line"


This allows command substitution/globbing -> unsafe and can execute backticks/$() from .mvn/jvm.config. Is this wanted? Seems dangerous, we might want to at least document this.

We could switch to a solution similar to the one I tried on windows for #11365, i.e. using java to compile the JvmConfigParser

bmarwell · 2025-11-27T20:21:54Z

apache-maven/src/assembly/maven/bin/mvn

+    # Do not use `xargs -0` as this is not POSIX-compliant
+    while IFS= read -r line || [ -n "$line" ]; do
+      # Convert CR to LF
+      line=$(echo "$line" | tr '\r' '\n')


this can produce embedded newlines that confuse splitting if \r is in the middle of a line. But I think that is problematic anyway.

You mean in case of mixed end of lines style, with some \n, some \r and some \r\n ?

bmarwell · 2025-11-27T20:23:09Z

apache-maven/src/assembly/maven/bin/mvn

+
+        # Quote the argument if it contains spaces or special shell characters
+        case "$arg" in
+          *[\ \|\&\;\<\>\(\)\$\`\\\"\'\~\*\?\[\]\#\~\=]*)


This looks fragile. Does work correctly on backticks and double quotes? If backticks or double quotes are inside, should we not use `arg="'$arg'" instead?

This commit unifies the .mvn/jvm.config parsing approach by using JvmConfigParser.java on both Unix and Windows platforms, solving multiple related issues with special character handling. Issues Fixed: - MNG-11363: Pipe symbols (|) in jvm.config cause shell parsing errors - MNG-11485: Non-POSIX xargs -0 fails on AIX, FreeBSD, and other systems - MNG-11486: sed command fails when paths contain @ symbols Changes: - Modified mvn (Unix): Replaced awk/sed pipeline with JvmConfigParser.java compilation and execution. This ensures POSIX compliance and handles all special characters (pipes, quotes, @, etc.) correctly. - Modified mvn.cmd (Windows): Removed debug output and updated comments to reflect all special characters now handled correctly. - Added integration test MavenITgh11485AtSignInJvmConfigTest to verify @ symbol handling in jvm.config files (important for Jenkins workspaces like workspace/project_PR-350@2). The Java-based parser approach: - Handles comment removal (both # at start and end-of-line) - Performs variable substitution (${MAVEN_PROJECTBASEDIR}) - Correctly parses quoted arguments with special characters - Works consistently across all platforms - Is POSIX-compliant (no xargs -0, awk, or complex sed needed) Performance Impact: Unix script is ~200ms slower (320ms → 520ms for mvn -v) due to Java compilation, but this is acceptable for the correctness, maintainability, and cross-platform consistency benefits. Related PRs: - Fixes apache#11365 (MNG-11363: pipe symbols) - Fixes apache#11489 (MNG-11485: POSIX compliance) - Fixes apache#11499 (MNG-11486: @ symbols in paths)

gnodet · 2025-12-02T12:52:12Z

This PR has been superseded by #11365, which now fixes all three related issues:

MNG-11363: Pipe symbols in jvm.config
MNG-11485: POSIX compliance (this PR's issue)
MNG-11486: @ symbols in paths

The unified solution uses JvmConfigParser.java on both Unix and Windows platforms, ensuring consistent behavior and handling all special characters correctly.

Integration tests from this PR have been added to #11365.

gnodet added bug Something isn't working backport-to-4.0.x labels Nov 24, 2025

gnodet added this to the 4.1.0 milestone Nov 24, 2025

gnodet changed the title ~~[gh-11485] Replace xargs -0 with POSIX-compliant alternative~~ Replace xargs -0 with POSIX-compliant alternative (fixes #11485) Nov 24, 2025

gnodet force-pushed the gh-11485-xargs-posix-fix branch from 18ab53a to fbdb26b Compare November 24, 2025 16:29

gnodet force-pushed the gh-11485-xargs-posix-fix branch from fbdb26b to 3f5f5d4 Compare November 25, 2025 07:22

bmarwell requested changes Nov 25, 2025

View reviewed changes

Add an IT to check @ in cmdline argument

087a742

gnodet requested a review from bmarwell November 27, 2025 15:31

bmarwell added mvn40 mvn4 labels Nov 27, 2025

bmarwell reviewed Nov 27, 2025

View reviewed changes

gnodet mentioned this pull request Dec 2, 2025

Fix special characters in .mvn/jvm.config (fix #11363, #11485 and #11486) #11365

Open

gnodet closed this Dec 9, 2025

github-actions bot removed this from the 4.1.0 milestone Dec 9, 2025

Replace xargs -0 with POSIX-compliant alternative (fixes #11485) #11489

Replace xargs -0 with POSIX-compliant alternative (fixes #11485) #11489

Conversation

gnodet commented Nov 24, 2025

Description

Problem

Solution

Changes

Testing

Compatibility

Uh oh!

bmarwell commented Nov 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bmarwell left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gnodet commented Dec 2, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

bmarwell commented Nov 24, 2025 •

edited

Loading