tests/int: use runc features in seccomp flags test

This test (initially added by commit 58ea21d and later amended in
commit 26dc55e) currently has two major deficiencies:

1. All possible flag combinations, and their respective numeric values,
   have to be explicitly listed. Currently we support 3 flags, so
   there is only 7 combinations, but adding more flags will become
   increasingly difficult.

2. It requires kernel 4.17 (for SECCOMP_FILTER_FLAG_SPEC_ALLOW), and not
   doing any tests when running on an older kernel. This, too, will make
   it more difficult to add extra flags in the future.

Both issues can be solved by using runc features which now prints all
known and supported runc flags. We still have to hardcode individual
flags to test, and their numeric values, but most of other work is coded
now. In particular:

 * The test checks that all the supported flags are being tested. In
   other words, this is a sanity check that if a new flag is added to
   runc, the test is amended appropriately.

 * Flags that are known but not supported (say because of an older
   kernel) are removed from the test. This makes it possible to run
   the test on older kernels, removing the limitation (2) above.

 * The test calculates the powerset (all possible combinations) of
   supported flags, and their numeric values. This makes it easier
   to add more flags, removing the limitation (1) above.

The downside to this is the test code is somewhat complicated.

Signed-off-by: Kir Kolyshkin <[email protected]>

Author: kolyshkin

tests/integration/seccomp.bats

@@ -67,10 +67,6 @@ function teardown() {
 }
 
 @test "runc run [seccomp] (SECCOMP_FILTER_FLAG_*)" {
-	# Linux 4.14: SECCOMP_FILTER_FLAG_LOG
-	# Linux 4.17: SECCOMP_FILTER_FLAG_SPEC_ALLOW
-	requires_kernel 4.17
-
 	update_config '   .process.args = ["/bin/sh", "-c", "mkdir /dev/shm/foo"]
 			| .process.noNewPrivileges = false
 			| .linux.seccomp = {
@@ -79,18 +75,68 @@ function teardown() {
 				"syscalls":[{"names":["mkdir"], "action":"SCMP_ACT_ERRNO"}]
 			}'
 
-	declare -A FLAGS=(
-		['REMOVE']=4 # No setting, use built-in default.
-		['EMPTY']=0  # Empty set of flags.
-		['"SECCOMP_FILTER_FLAG_LOG"']=2
-		['"SECCOMP_FILTER_FLAG_SPEC_ALLOW"']=4
-		['"SECCOMP_FILTER_FLAG_TSYNC"']=0 # tsync flag is ignored.
-		['"SECCOMP_FILTER_FLAG_LOG","SECCOMP_FILTER_FLAG_SPEC_ALLOW"']=6
-		['"SECCOMP_FILTER_FLAG_LOG","SECCOMP_FILTER_FLAG_TSYNC"']=2
-		['"SECCOMP_FILTER_FLAG_SPEC_ALLOW","SECCOMP_FILTER_FLAG_TSYNC"']=4
-		['"SECCOMP_FILTER_FLAG_LOG","SECCOMP_FILTER_FLAG_SPEC_ALLOW","SECCOMP_FILTER_FLAG_TSYNC"']=6
+	# This is a set of all possible flag combinations to test.
+	declare -A TEST_CASES=(
+		['EMPTY']=0 # Special value: empty set of flags.
+	)
+
+	# List of flags to test and their respective numeric values.
+	declare -A flags=(
+		["TSYNC"]=0 # Supported but ignored by runc, thus 0.
+		["LOG"]=2
+		["SPEC_ALLOW"]=4
+		# XXX: add new seccomp flags above this line.
 	)
-	for key in "${!FLAGS[@]}"; do
+
+	# Get the list of flags supported by runc/seccomp/kernel.
+	mapfile -t sup_flags < <(__runc features |
+		jq '.linux.seccomp.supported_flags' | grep 'SECCOMP' |
+		sed -e 's/^ *"SECCOMP_FILTER_FLAG_//' -e 's/",*$//')
+
+	# Check that any new seccomp flags added are actually tested.
+	for f in "${sup_flags[@]}"; do
+		if ! [ -v 'flags[$f]' ]; then
+			echo "ERROR: seccomp flag $f not added to the test; fix the text!" 1>&2
+			exit 1
+		fi
+	done
+
+	# Filter out flags not supported by the current seccomp/kernel,
+	# so this test can be run on older systems.
+	for f in "${!flags[@]}"; do
+		found=0
+		for v in "${sup_flags[@]}"; do
+			if [ "$f" = "$v" ]; then
+				found=1
+				break
+			fi
+		done
+		[ "$found" -eq 0 ] && unset 'flags[$f]'
+	done
+
+	# Special value: do not set any flags in spec. In this case, runc
+	# should set SPEC_ALLOW flag in this case, if it is supported,
+	# or 0 otherwise.
+	TEST_CASES['REMOVE']=${flags["SPEC_ALLOW"]-0}
+
+	# Add all possible combinations (i.e., a powerset) of seccomp flags
+	# and their expected numeric values to TEST_CASES.
+	if [ "${#flags[@]}" -gt 0 ]; then
+		for fc in $(eval echo "$(printf "{'\"SECCOMP_FILTER_FLAG_'%s'\",',}" "${!flags[@]}")"); do
+			# Calculate the numeric value of the flags set.
+			val=0
+			for k in "${!flags[@]}"; do
+				if [[ "$fc" == *"$k\","* ]]; then
+					((val += flags[$k])) || true
+				fi
+			done
+			# Remove the last comma from fc.
+			TEST_CASES[${fc/%,/}]=$val
+		done
+	fi
+
+	# Finally, run the tests.
+	for key in "${!TEST_CASES[@]}"; do
 		case "$key" in
 		'REMOVE')
 			update_config ' del(.linux.seccomp.flags)'
@@ -108,7 +154,7 @@ function teardown() {
 		[[ "$output" == *"mkdir:"*"/dev/shm/foo"*"Operation not permitted"* ]]
 
 		# Check the numeric flags value, as printed in the debug log, is as expected.
-		exp="\"seccomp filter flags: ${FLAGS[$key]}\""
+		exp="\"seccomp filter flags: ${TEST_CASES[$key]}\""
 		echo "flags $key, expecting $exp"
 		[[ "$output" == *"$exp"* ]]
 	done