Skip to content

Revamp testing. #289

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

Revamp testing. #289

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
4147f3b
Revamp testing.
neoblizz Nov 17, 2025
0a19762
Use preamble
neoblizz Nov 18, 2025
4983672
Apply Ruff auto-fixes
github-actions[bot] Nov 18, 2025
e5002f3
Merge branch 'main' into muhosama/testing-infra
neoblizz Nov 18, 2025
b49a3e8
Disable ring-reduce.
neoblizz Nov 22, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions .github/scripts/container_build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,9 +36,10 @@ if [ "$CONTAINER_RUNTIME" = "apptainer" ]; then

elif [ "$CONTAINER_RUNTIME" = "docker" ]; then
echo "[INFO] Checking Docker images..."
IMAGE_NAME="iris-dev-triton-aafec41"
# Use GitHub variable if set, otherwise default to iris-dev
IMAGE_NAME=${DOCKER_IMAGE_NAME:-"iris-dev"}

# Check if the triton image exists
# Check if the image exists
if docker image inspect "$IMAGE_NAME" &> /dev/null; then
echo "[INFO] Using existing Docker image: $IMAGE_NAME"
else
Expand Down
4 changes: 3 additions & 1 deletion .github/scripts/container_exec.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,9 @@ if [ "$CONTAINER_RUNTIME" = "apptainer" ]; then
exit $EXIT_CODE

elif [ "$CONTAINER_RUNTIME" = "docker" ]; then
IMAGE_NAME=${CUSTOM_IMAGE:-${DOCKER_IMAGE_NAME:-"iris-dev-triton-aafec41"}}
# Use custom image if provided, otherwise use GitHub variable or default
# GitHub Actions sets DOCKER_IMAGE_NAME, locally defaults to iris-dev
IMAGE_NAME=${CUSTOM_IMAGE:-${DOCKER_IMAGE_NAME:-"iris-dev"}}

if ! docker image inspect "$IMAGE_NAME" &> /dev/null; then
echo "[ERROR] Docker image $IMAGE_NAME not found" >&2
Expand Down
3 changes: 2 additions & 1 deletion .github/scripts/container_run.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,8 @@ if [ "$CONTAINER_RUNTIME" = "apptainer" ]; then
bash apptainer/run.sh "$@"
elif [ "$CONTAINER_RUNTIME" = "docker" ]; then
echo "[INFO] Running with Docker..."
IMAGE_NAME=${1:-"iris-dev-triton-aafec41"}
# Use GitHub variable if set, otherwise default to iris-dev
IMAGE_NAME=${1:-${DOCKER_IMAGE_NAME:-"iris-dev"}}
WORKSPACE_DIR=${2:-"$(pwd)"}
bash docker/run.sh "$IMAGE_NAME" "$WORKSPACE_DIR"
fi
Expand Down
47 changes: 24 additions & 23 deletions .github/scripts/run_tests.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,16 +3,28 @@
# Copyright (c) 2025 Advanced Micro Devices, Inc. All rights reserved.
#
# Run Iris tests in a container
# Usage: run_tests.sh <num_ranks> [gpu_devices]
# Usage: run_tests.sh <test_dir> <num_ranks> [gpu_devices]
# test_dir: subdirectory under tests/ (e.g., examples, unittests, ccl)
# num_ranks: number of GPU ranks (1, 2, 4, or 8)
# gpu_devices: comma-separated GPU device IDs (optional)

set -e

NUM_RANKS=$1
GPU_DEVICES=${2:-""}
TEST_DIR=$1
NUM_RANKS=$2
GPU_DEVICES=${3:-""}

if [ -z "$NUM_RANKS" ]; then
echo "[ERROR] NUM_RANKS not provided"
echo "Usage: $0 <num_ranks> [gpu_devices]"
if [ -z "$TEST_DIR" ] || [ -z "$NUM_RANKS" ]; then
echo "[ERROR] Missing required arguments"
echo "Usage: $0 <test_dir> <num_ranks> [gpu_devices]"
echo " test_dir: examples, unittests, or ccl"
echo " num_ranks: 1, 2, 4, or 8"
exit 1
fi

# Validate test directory
if [ ! -d "tests/$TEST_DIR" ]; then
echo "[ERROR] Test directory tests/$TEST_DIR does not exist"
exit 1
fi

Expand All @@ -29,23 +41,12 @@ fi
set -e
pip install -e .

# Run examples tests
for test_file in tests/examples/test_*.py; do
echo \"Testing: \$test_file with $NUM_RANKS ranks\"
python tests/run_tests_distributed.py --num_ranks $NUM_RANKS \"\$test_file\" -v --tb=short --durations=10
# Run tests in the specified directory
for test_file in tests/$TEST_DIR/test_*.py; do
if [ -f \"\$test_file\" ]; then
echo \"Testing: \$test_file with $NUM_RANKS ranks\"
python tests/run_tests_distributed.py --num_ranks $NUM_RANKS \"\$test_file\" -v --tb=short --durations=10
fi
done

# Run unit tests
for test_file in tests/unittests/test_*.py; do
echo \"Testing: \$test_file with $NUM_RANKS ranks\"
python tests/run_tests_distributed.py --num_ranks $NUM_RANKS \"\$test_file\" -v --tb=short --durations=10
done

# Run ccl tests
# DISABLED: CCL host-side APIs have issues for some data types/algorithms
# for test_file in tests/ccl/test_*.py; do
# echo \"Testing: \$test_file with $NUM_RANKS ranks\"
# python tests/run_tests_distributed.py --num_ranks $NUM_RANKS \"\$test_file\" -v --tb=short --durations=10
# done
"

3 changes: 3 additions & 0 deletions .github/workflows/iris-external-validation-test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,9 @@ concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.ref }}
cancel-in-progress: ${{ github.ref != 'refs/heads/main' }}

env:
DOCKER_IMAGE_NAME: ${{ vars.DOCKER_IMAGE_NAME || 'iris-dev-triton-aafec41' }}

jobs:
build-container-image:
runs-on: [self-hosted, mi3008x]
Expand Down
3 changes: 3 additions & 0 deletions .github/workflows/iris-performance-regression-test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,9 @@ concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.ref }}
cancel-in-progress: ${{ github.ref != 'refs/heads/main' }}

env:
DOCKER_IMAGE_NAME: ${{ vars.DOCKER_IMAGE_NAME || 'iris-dev-triton-aafec41' }}

jobs:
build-container-image:
runs-on: [self-hosted, mi3008x]
Expand Down
198 changes: 0 additions & 198 deletions .github/workflows/iris-pip-install-test.yml
View file
Open in desktop
Copy link
Collaborator

@mawad-amd mawad-amd Nov 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like us to continue testing against the three different installs:

  • pip install git+https://github.com/${{ github.repository }}.git@${{ github.sha }}
  • pip install -e .
  • pip install .

Sometimes source structure and import assumptions are only captured in one vs the other.

Copy link
Member Author

@neoblizz neoblizz Nov 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mawad-amd why don't we just make sure the installs work there?

Copy link
Collaborator

@mawad-amd mawad-amd Nov 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just pip install without running the tests you mean?

Copy link
Collaborator

@mawad-amd mawad-amd Nov 24, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are issues that won't show up unless we actually run the tests. If we really have to test against one and do some smoke tests against the other, then I would prefer we always do the full test suite against pip install git+https://github.com/${{ github.repository }}.git@${{ github.sha }}

This file was deleted.

Loading
Loading