Skip to content

chore: use the new CLI to run the server #171

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Oct 16, 2025
+27 −18
Merged

chore: use the new CLI to run the server #171

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 27 additions & 18 deletions controllers/resource_helper.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,30 +53,39 @@ var validConfigMapKeyRegex = regexp.MustCompile(`^[a-zA-Z0-9]([a-zA-Z0-9\-_.]*[a
var startupScript = `
set -e

if python -c "
# Determine which CLI to use based on llama-stack version
VERSION_CODE=$(python -c "
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we make the script itself a shell HERE document like in

read -r -d '' detect_version_script <<EOT
....
EOT

VERSION_CODE=$(python -c "$detect_version_script")

That makes it easier with formatting etc I guess. You could also use single quote around EOT to avoid variable expansion.

btw, I'm not super happy with having shell scripts that call python code embedded in golang binaries, including hard-coded version numbers.

Also, I wonder why isn't this part of the distribution imnage as the distribution image knows best how to start up itself ?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's ok if we skip that HERE document for now as it doesn't make this hack much beautifuler anyways :)

import sys
from importlib.metadata import version
from packaging import version as pkg_version

try:
from importlib.metadata import version
from packaging import version as pkg_version

llama_version = version('llama_stack')
print(f'Determined llama-stack version {llama_version}')
if pkg_version.parse(llama_version) < pkg_version.parse('0.2.17'):
print('llama-stack version is less than 0.2.17 usin old module path llama_stack.distribution.server.server to start the server')
sys.exit(0)
print(f'Detected llama-stack version: {llama_version}', file=sys.stderr)

v = pkg_version.parse(llama_version)

if v < pkg_version.parse('0.2.17'):
print('Using legacy module path (llama_stack.distribution.server.server)', file=sys.stderr)
print(0)
elif v < pkg_version.parse('0.3.0'):
print('Using core module path (llama_stack.core.server.server)', file=sys.stderr)
print(1)
else:
print('llama-stack version is greater than or equal to 0.2.17 using new module path llama_stack.core.server.server to start the server')
sys.exit(1)
print('Using new CLI command (llama stack run)', file=sys.stderr)
print(2)
except Exception as e:
print(f'Failed to determine version: assume newer version if we cannot determine using new module path llama_stack.core.server.server to start the server: {e}')
sys.exit(1)

"; then
python3 -m llama_stack.distribution.server.server --config /etc/llama-stack/run.yaml
else
python3 -m llama_stack.core.server.server /etc/llama-stack/run.yaml
fi`
print(f'Version detection failed, defaulting to new CLI: {e}', file=sys.stderr)
print(2)
")

# Execute the appropriate CLI based on version
case $VERSION_CODE in
0) python3 -m llama_stack.distribution.server.server --config /etc/llama-stack/run.yaml ;;
1) python3 -m llama_stack.core.server.server /etc/llama-stack/run.yaml ;;
2) llama stack run /etc/llama-stack/run.yaml ;;
*) echo "Invalid version code: $VERSION_CODE, using new CLI"; llama stack run /etc/llama-stack/run.yaml ;;
esac`

// validateConfigMapKeys validates that all ConfigMap keys contain only safe characters.
// Note: This function validates key names only. PEM content validation is performed
Expand Down