Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
151 commits
Select commit Hold shift + click to select a range
0f18af0
initial commit with partition logic
gwct May 5, 2025
03aa209
todo list
gwct May 5, 2025
42069a6
default partition test
gwct May 5, 2025
541d143
changed name of plugin subfolder
gwct May 5, 2025
62519fe
default partition test
gwct May 5, 2025
715cef9
fixing gpu partition
gwct May 5, 2025
704f49c
refactor logic, resource error checks, printing resource table
gwct May 5, 2025
9ec5247
Intro for cannon
nsohail19 May 6, 2025
e23789f
Default resource parameters
nsohail19 May 6, 2025
7e2e4f0
Change slurm executor to cannon
nsohail19 May 6, 2025
15b3104
Update units for resources
nsohail19 May 6, 2025
e33e006
finalizing error messages and tests, added --cannon-resources option
gwct May 6, 2025
35c9ddc
Update GPU flags with slurm-extra
nsohail19 May 6, 2025
1a62ffc
Include instructions to run tests
nsohail19 May 6, 2025
1b6a9cf
Add setting up profile
nsohail19 May 6, 2025
c369ce6
added warning for threads
gwct May 6, 2025
fdab80a
Merge pull request #1 from harvardinformatics/cannon-partitions
nsohail19 May 6, 2025
f14863f
removed print statement
gwct May 6, 2025
cdc24d3
Fix flags for example
nsohail19 May 6, 2025
e2d4adc
add upstream version check workflow
gwct May 7, 2025
48bc07e
disabling previous workflows
gwct May 7, 2025
8b6b68e
Merge pull request #2 from harvardinformatics/versioning
gwct May 7, 2025
f702858
versioning actions
gwct May 7, 2025
f04b0e7
check upstream typo
gwct May 7, 2025
852e732
check upstream typo
gwct May 7, 2025
3218903
check_upstream_release.py typo
gwct May 7, 2025
1037524
another typo
gwct May 7, 2025
95e49b3
trying to get version string in auto pr
gwct May 7, 2025
9313af4
typos
gwct May 7, 2025
0d734ae
simulated test
gwct May 7, 2025
32ed6a3
removing invalid line from sync-upstream
gwct May 7, 2025
af2b389
combining check and sync upstream workflows
gwct May 7, 2025
f3193ce
removing workflow files from auto-pr generation
gwct May 7, 2025
9ca5d7d
fixing workflow to exclude workflows on upstream sync
gwct May 7, 2025
fab46b8
trying to fix auto-sync
gwct May 7, 2025
c4199ac
fix upstream in sync workflow
gwct May 7, 2025
bcd0eab
fixing sync workflow...
gwct May 7, 2025
32f6d7e
rm redundant upstream add to workflow
gwct May 7, 2025
54c7194
upstream sync workflow
gwct May 7, 2025
8f441d4
upstream sync workflow
gwct May 7, 2025
bade1d7
upstream sync workflow
gwct May 7, 2025
cefaf0a
upstream sync workflow
gwct May 7, 2025
6e71b32
upstream sync workflow
gwct May 7, 2025
23c2d15
refactor sync workflow for 3-way merge
gwct May 7, 2025
daebe2d
sync workflow again
gwct May 7, 2025
04e22ab
switching pr creation to gh
gwct May 7, 2025
939ebb4
switching pr creation to gh
gwct May 7, 2025
1929399
switching pr creation to gh
gwct May 7, 2025
39a93bd
switching pr creation to gh
gwct May 7, 2025
08a935d
trying a different token
gwct May 8, 2025
1148326
trying again
gwct May 8, 2025
aa54f59
trying sleep
gwct May 8, 2025
52117c2
trying sleep
gwct May 8, 2025
19aaa69
trying sync again
gwct May 8, 2025
d2f13b2
trying sync again
gwct May 8, 2025
acb8855
trying sync again
gwct May 8, 2025
4b8a2d0
trying sync again
gwct May 8, 2025
3e1b54c
trying sync again
gwct May 8, 2025
10df6cb
fixing sync and pr workflows
gwct May 8, 2025
87c95c9
fixing sync and pr workflows
gwct May 8, 2025
3126529
fixing sync and pr workflows
gwct May 8, 2025
3359537
fixing sync and pr workflows
gwct May 8, 2025
f72e124
fixing sync and pr workflows
gwct May 8, 2025
29dd786
adding pypi release action
gwct May 8, 2025
46b1c7e
pypi release action
gwct May 8, 2025
1f4fb50
pypi release action
gwct May 8, 2025
42b9fb8
pypi release action
gwct May 8, 2025
7f3fbe7
moved many functions to new cannon library to minimize changes with u…
gwct May 8, 2025
9b8a685
Merge pull request #9 from harvardinformatics/clean-up
gwct May 9, 2025
e8121db
testing the sync workflow to ensure it doesnt create duplicate prs
gwct May 9, 2025
bf36b07
testing the sync workflow to ensure it doesnt create duplicate prs
gwct May 9, 2025
c813d39
testing the sync workflow to ensure it doesnt create duplicate prs
gwct May 9, 2025
768415d
testing the sync workflow to ensure it doesnt create duplicate prs
gwct May 9, 2025
3455abf
update docs
gwct May 9, 2025
1dd0ca4
update docs
gwct May 9, 2025
dfe154a
Merge pull request #10 from harvardinformatics/readme-0509
gwct May 9, 2025
615991a
readme
gwct May 9, 2025
521c840
testing [bump-local]
gwct May 9, 2025
baef29f
Version 1.2.1.post2
github-actions[bot] May 9, 2025
07c25a9
testing [bump-local]
gwct May 9, 2025
a6991ae
Version 1.2.1.post3
github-actions[bot] May 9, 2025
ccfea49
testing [bump-local]
gwct May 9, 2025
15494d4
Version 1.2.1.post4
github-actions[bot] May 9, 2025
142ce2b
deleting TODO since all tasks are complete
gwct May 9, 2025
72305ff
Merge remote-tracking branch 'upstream/main'
gwct May 9, 2025
93003cd
testing issue workflow
gwct May 10, 2025
8a2dd4e
testing issue workflow
gwct May 10, 2025
de210ee
testing issue workflow
gwct May 10, 2025
50758f9
testing issue workflow
gwct May 10, 2025
c8331ce
testing issue workflow
gwct May 10, 2025
a94da3e
Merge remote-tracking branch 'upstream/main'
gwct May 10, 2025
7af6166
testing issue workflow
gwct May 10, 2025
123957a
removed workflow test
gwct May 10, 2025
512323a
testing release workflow [bump-release]
gwct May 10, 2025
f85f354
Version 1.2.2.post1
github-actions[bot] May 10, 2025
c56c52c
testing release workflow [bump-release]
gwct May 10, 2025
74855ef
Version 1.2.2.post1
github-actions[bot] May 10, 2025
fbe36c2
testing release workflow [bump-release]
gwct May 10, 2025
4874712
Version 1.2.2.post2
github-actions[bot] May 10, 2025
0407b67
testing release workflow [bump-release]
gwct May 10, 2025
cc2dfb4
Version 1.2.2.post1
github-actions[bot] May 10, 2025
e0a5dc6
testing release workflow [bump-release]
gwct May 10, 2025
e944c41
Version 1.2.2.post1
github-actions[bot] May 10, 2025
d68b77c
testing release workflow [bump-release]
gwct May 10, 2025
5e094f9
Version 1.2.2.post1
github-actions[bot] May 10, 2025
7306077
testing release workflow [bump-release]
gwct May 10, 2025
e0e77b2
Version 1.2.2.post2
github-actions[bot] May 10, 2025
f775b97
testing release workflow [bump-release]
gwct May 10, 2025
6175f0d
Version 1.2.2.post3
github-actions[bot] May 10, 2025
ed4674a
testing release workflow [bump-release]
gwct May 11, 2025
2c37c5e
Version 1.2.2.post4
github-actions[bot] May 11, 2025
4c23148
testing release workflow [bump-release]
gwct May 11, 2025
e9825a1
Version 1.2.2.post4
github-actions[bot] May 11, 2025
3b0d9af
testing release workflow [bump-release]
gwct May 11, 2025
13f4db1
Version 1.2.2.post4
github-actions[bot] May 11, 2025
636df2c
testing release workflow [bump-release]
gwct May 11, 2025
c062280
Version 1.2.2.post1
github-actions[bot] May 11, 2025
cf8cfc9
testing release workflow [bump-release]
gwct May 11, 2025
be3620d
Version 1.2.2.post1
github-actions[bot] May 11, 2025
613665f
re-adding pypi release action
gwct May 11, 2025
3983ddd
minor sync workflow edit
gwct May 11, 2025
0592b1e
Update README.md
gwct May 11, 2025
d9a263a
Merge remote-tracking branch 'upstream/main'
gwct May 19, 2025
007bb32
[bump-release]
gwct May 19, 2025
94872ab
Version 1.3.6.post1
github-actions[bot] May 19, 2025
3579ca8
paring the further docs to specific cannon info
gwct May 27, 2025
4126ae5
Merge pull request #21 from harvardinformatics/docs
gwct May 27, 2025
f13cc36
[bump-release] trigger release
gwct May 27, 2025
eca2e3f
Version 1.3.6.post2
github-actions[bot] May 27, 2025
c47a9e9
workflow release version
gwct May 27, 2025
ed3c939
docs
github-actions[bot] May 28, 2025
1ac1aa0
Merge remote-tracking branch 'upstream/main'
gwct Jun 14, 2025
7195014
v1.4.0.post0
gwct Jun 14, 2025
a551485
[bump-v1.4.0.post0]
gwct Jun 14, 2025
71ee31b
Version 1.4.0.post1
github-actions[bot] Jun 14, 2025
8d48527
[bump-release]
gwct Jun 14, 2025
62bdff0
[bump-release]
gwct Jun 14, 2025
dfc3df7
Version 1.4.0.post1
github-actions[bot] Jun 14, 2025
c6eac79
Merge remote-tracking branch 'upstream/main'
gwct Jul 7, 2025
330649b
Merge branch 'main' of github.com:harvardinformatics/snakemake-execut…
gwct Jul 7, 2025
0c12c72
[bump-release]
gwct Jul 7, 2025
64df6ae
Version 1.5.0.post1
github-actions[bot] Jul 7, 2025
0a0bf0e
adding token to upstream check workflow
gwct Jul 22, 2025
29f43c2
Merge branch 'main' of github.com:harvardinformatics/snakemake-execut…
gwct Jul 22, 2025
7d0264e
Merge remote-tracking branch 'upstream/main' [bump-release]
gwct Jul 25, 2025
5382584
Version 1.6.0.post1
github-actions[bot] Jul 25, 2025
45403fc
Merge remote-tracking branch 'upstream/main'
gwct Sep 3, 2025
cef75cf
Merge remote-tracking branch 'upstream/main'
gwct Sep 18, 2025
f7e3d21
Version 1.8.0.post1
github-actions[bot] Sep 18, 2025
b3bac3f
[bump-release]
gwct Sep 18, 2025
e480357
updating to slurm plugin 1.9.2; required changes to how gpus are spec…
gwct Oct 29, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/config.yml
Original file line number Diff line number Diff line change
@@ -1 +1 @@
blank_issues_enabled: false
blank_issues_enabled: true
14 changes: 14 additions & 0 deletions .github/pr-types.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# .github/semantic.yml
types:
- feat
- fix
- chore
- docs
- style
- refactor
- test
- perf
- build
- ci
- revert
- auto-sync
33 changes: 33 additions & 0 deletions .github/releases.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# .github/release.yml
changelog:
# PRs carrying any of these labels will be omitted entirely
exclude:
labels:
- ignore-for-release
authors:
- dependabot[bot]

# Now define one or more categories for PRs you _do_ want to include
categories:
- title: Features
labels:
- feat

- title: Bug Fixes
labels:
- fix
# within this category you could also skip certain labels/authors
exclude:
labels:
- docs

- title: Maintenance
labels:
- chore
- ci
- refactor
exclude:
- build
- test
- perf
- style
77 changes: 77 additions & 0 deletions .github/scripts/check_upstream_release.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
import requests
import toml
import sys
import os
from packaging.version import Version, InvalidVersion

UPSTREAM_REPO = "snakemake/snakemake-executor-plugin-slurm"
PYPROJECT = "pyproject.toml"

EXIT_OK = 0 # no changes
EXIT_ERROR = 1 # script failure
EXIT_NEW_RELEASE = 42 # desired trigger

def error(msg):
print(f"ERROR: {msg}", file=sys.stderr)
sys.exit(EXIT_ERROR)

def get_latest_upstream_version():
url = f"https://api.github.com/repos/{UPSTREAM_REPO}/releases/latest"
headers = {"Accept": "application/vnd.github+json"}

token = os.environ.get("GITHUB_TOKEN")
if token:
headers["Authorization"] = f"Bearer {token}"

r = requests.get(url, headers=headers)
if r.status_code != 200:
error(f"Failed to fetch upstream release info: {r.status_code}, response: {r.text}")

try:
version_str = r.json()["tag_name"].lstrip("v")
return Version(version_str)
except Exception:
error("Failed to parse upstream version from tag")

def get_local_main_version():
try:
data = toml.load(PYPROJECT)
version_str = data["tool"]["poetry"]["version"]
base = Version(version_str).base_version # base_version is a str
return Version(base) # convert base_version string back to Version object
except Exception as e:
error(f"Could not parse pyproject.toml version: {e}")

def main():
upstream = get_latest_upstream_version()
local = get_local_main_version()

print(f"Local main version = {local}")
print(f"Upstream version = {upstream}")

if upstream > local:
print("New upstream release detected!")
sys.exit(EXIT_NEW_RELEASE)
else:
print("No new release")
sys.exit(EXIT_OK)

if __name__ == "__main__":
######
## TESTS
# # Simulated version for testing
# simulated_upstream_version = "1.2.99"

# # Print output GitHub Actions will parse
# print(f"Local main version = 1.2.1")
# print(f"Upstream version = {simulated_upstream_version}")
# print("New upstream release detected!")

# # Emit output for GitHub Actions
# #print(f"upstream-version={simulated_upstream_version}")

# # Exit code triggers the sync workflow
# sys.exit(42)
######

main()
85 changes: 85 additions & 0 deletions .github/scripts/merge_test.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
#!/usr/bin/env bash

set -euo pipefail

# CONFIGURATION
UPSTREAM_REMOTE="upstream"
UPSTREAM_BRANCH="main"
FORK_BRANCH="main"
TEMP_DIR=".merge-tmp"

EXCLUDES=(
"TODO.md"
"tests/Snakefile"
"tests/tests.py"
"tests/__pycache__"
"tests/profiles"
)
# Check if the script is run from the correct directory
echo "Preparing for local 3-way sync test (upstream → cannon)..."

# Ensure remotes are up to date
git fetch $UPSTREAM_REMOTE
git fetch origin

# Find the common ancestor (merge base)
BASE_COMMIT=$(git merge-base origin/$FORK_BRANCH $UPSTREAM_REMOTE/$UPSTREAM_BRANCH)
echo "Using merge base: $BASE_COMMIT"

# Prepare temp workspace
mkdir -p $TEMP_DIR/base $TEMP_DIR/theirs $TEMP_DIR/mine $TEMP_DIR/preview

MERGED=false

# Gather tracked files from upstream
FILES=$(git ls-tree -r --name-only $UPSTREAM_REMOTE/$UPSTREAM_BRANCH)

for SRC in $FILES; do
# (skip exclusions and map paths as before)

# Map slurm ➞ cannon
if [[ "$SRC" == snakemake_executor_plugin_slurm/* ]]; then
DEST="${SRC/snakemake_executor_plugin_slurm/snakemake_executor_plugin_cannon}"
else
DEST="$SRC"
fi

[[ ! -f "$DEST" ]] && continue

echo "Merging $SRC -> $DEST"

TMP_BASE="$TEMP_DIR/base/$(basename "$DEST")"
TMP_THEIRS="$TEMP_DIR/theirs/$(basename "$DEST")"
TMP_MINE="$TEMP_DIR/mine/$(basename "$DEST")"

git show "$BASE_COMMIT:$SRC" > "$TMP_BASE" 2>/dev/null || continue
git show "$UPSTREAM_REMOTE/$UPSTREAM_BRANCH:$SRC" > "$TMP_THEIRS" || continue
cp "$DEST" "$TMP_MINE"

if git merge-file --marker-size=30 "$TMP_MINE" "$TMP_BASE" "$TMP_THEIRS"; then
echo "Clean merge completed: $DEST"
else
echo "Conflict detected: $DEST (manual resolution may be required)"
fi

echo "Diff between $DEST and merged result:"
diff -u "$DEST" "$TMP_MINE" || true
echo

read -p "Apply merged result to $DEST? [y/N]: " CONFIRM
if [[ "$CONFIRM" == "y" || "$CONFIRM" == "Y" ]]; then
cp "$TMP_MINE" "$DEST"
echo "Applied merged version to $DEST"
MERGED=true
else
echo "Skipped applying merged version to $DEST"
fi

rm -f "$TMP_BASE" "$TMP_THEIRS" "$TMP_MINE"
done

echo ""
echo "Merge simulation complete."
echo "Merged files are available under: $TEMP_DIR/preview/"
echo "No files in your working directory were modified."
echo "To apply changes, manually copy from preview/ into your repo and commit."
Original file line number Diff line number Diff line change
Expand Up @@ -18,12 +18,21 @@ jobs:
- name: Checkout repository
uses: actions/checkout@v4

- name: Verify logo file exists
run: |
ls -la ${{ github.workspace }}/pr_logo/
if [ ! -f "${{ github.workspace }}/pr_logo/long_logo_hpc.png" ]; then
echo "ERROR: Logo file not found!"
exit 1
fi
echo "Logo file found at: ${{ github.workspace }}/pr_logo/long_logo_hpc.png"

- name: Post to Mastodon
uses: snakemake/mastodon-release-post-action@main # == latest
with:
access-token: ${{ secrets.MASTODONBOT }}
pr-title: ${{ github.event.head_commit.message }}
image: "${{ github.workspace }}/pr_logo/long_logo_hpc.png"
image: "long_logo_hpc.png"
image-description: "Snakemake HPC logo for Mastodon"
message: |
Beep, Beep - I am your friendly #Snakemake release announcement bot.
Expand Down
79 changes: 79 additions & 0 deletions .github/workflows/bump-and-tag.yml.disabled
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
name: Version Bump

on:
push:
branches: [main]
workflow_dispatch:

jobs:
bump-and-tag:
name: Bump version & optionally tag
if: |
github.event_name == 'workflow_dispatch' ||
(
github.event_name == 'push' &&
startsWith(github.ref, 'refs/heads/main') &&
contains(github.event.head_commit.message, '[bump-')
)
runs-on: ubuntu-latest
permissions:
contents: write

steps:
- name: Checkout
uses: actions/checkout@v3

- name: Set up Python & Poetry
uses: actions/setup-python@v4
with:
python-version: '3.x'
- run: pip install poetry

- name: Detect bump mode
id: mode
run: |
MSG="${{ github.event.head_commit.message }}"
if [[ "$MSG" == *"[bump-release]"* ]]; then
MODE=release
else
MODE=local
fi
echo "mode=$MODE" >> "$GITHUB_OUTPUT"
echo "MODE=$MODE" >> "$GITHUB_ENV"
echo "::notice::Detected bump mode: $MODE"

- name: Compute next post-release version
id: bump
run: |
FULL=$(poetry version -s)
UPSTREAM="${FULL%%.post*}"
if [[ "$FULL" =~ \.post([0-9]+)$ ]]; then
NEXT=$((BASH_REMATCH[1] + 1))
else
NEXT=1
fi
NEW="${UPSTREAM}.post${NEXT}"
poetry version "$NEW"
echo "new_version=$NEW" >> "$GITHUB_OUTPUT"
echo "::notice::Will bump to: $NEW"

- name: Commit bumped version
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git commit -am "Version ${{ steps.bump.outputs.new_version }}"

- name: Tag release (only in release mode)
if: env.MODE == 'release'
run: |
echo "Tagging v${{ steps.bump.outputs.new_version }}"
git tag "v${{ steps.bump.outputs.new_version }}"

- name: Push commit and tag
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
git push origin HEAD
if [ "$MODE" = "release" ]; then
git push origin "v${{ steps.bump.outputs.new_version }}"
fi
Loading
Loading