To support port 8443 instead of 8080 for chatbot and service communication over on-prem

[justjais]

Jira Issue: https://issues.redhat.com/browse/AAP-50667

Description

To support port 8443 instead of 8080 for chatbot and service communication over on-prem

Testing

Steps to test

1. Pull down the PR
2. Tested and verified over saas deoloyment
3. Tested and verified over operator and the setup details are as follows:

Cluster setup is at:

https://console-openshift-console.apps.aap-test2.w6n5.p1.openshiftapps.com/, under aap-50667 namespace, as verified the chatbot status ref:

> curl -s "https://my-aiconnect-aap-50667.apps.aap-test2.w6n5.p1.openshiftapps.com/api/v1/health/status/chatbot/
"{"chatbot-service": "ok", "streaming-chatbot-service": "ok"}

Model pipeline configuration used:

ModelPipelineChatBot:
  config:
    inference_url: 'https://my-aiconnect-chatbot-api.aap-50667.svc.cluster.local:8443'
    model_id: 'granite-3.3-8b-instruct'
    enable_health_check: 'True'
    verify_ssl: true
    ca_cert_file: '/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt'
  provider: 'http'
ModelPipelineStreamingChatBot:
  config:
    inference_url: 'https://my-aiconnect-chatbot-api.aap-50667.svc.cluster.local:8443'
    model_id: 'granite-3.3-8b-instruct'
    enable_health_check: 'True'
    verify_ssl: true
    ca_cert_file: '/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt'

Llama-stack is serving using https, as: https://0.0.0.0:8321/

Scenarios tested

NA

Production deployment

• This code change is ready for production on its own
• This PR should be released in 2.4 (cherry-pick should be created)
• This PR should be released in 2.5 (cherry-pick should be created)
• This code change requires the following considerations before going to production:

[rooftopcellist]

Quick assessment:

• This will likely only work on Openshift because that custom CA path is only mounted into container in Openshift. (/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt)
• ContainerPort in that PR should match the port the container is listening on (8321 currently as declared by the Service targetPort)
• Currently, in our other operators, we mount the custom CA cert to the path the python version in the container expects certs to be used so that they are automatically picked up. Looks like the same thing happens here in the model deployment.

In openshift, there are tricks to use annotations to automatically set up certs, but we don't have that available to us in k8s.

[justjais]

@rooftopcellist thanks for reviewing the PR, I updated and verified the suggestion made as:

This will likely only work on Openshift because that custom CA path is only mounted into container in Openshift. (/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt)
Update the config as:

{% if is_openshift %}
        ca_cert_file: '/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt'
{% else %}
        ca_cert_file: '/etc/ssl/certs/ca-certificates.crt'
{% endif %}

This would help admins to pass the custom CA, is that correct expectation?

ContainerPort in that PR should match the port the container is listening on (8321 currently as declared by the Service targetPort)
Updated the port to 8321 from 8443, here

- containerPort: 8321

And, post update the communication happens as expected w/o issues.

Around your following comment:

Currently, in our other operators, we mount the custom CA cert to the path the python version in the container expects certs to be used so that they are automatically picked up. Looks like the same thing happens here in the model deployment.

I believe you meant the following model and chat deployment:

initContainers:
  - name: configure-bundle-ca-cert
    command:
      - /bin/sh
      - -c
      - |
        mkdir -p /etc/pki/ca-trust/extracted/{java,pem,openssl,edk2}
        update-ca-trust extract --output /etc/pki/ca-trust/extracted
    volumeMounts:
      - name: "ca-trust-extracted"
        mountPath: "/etc/pki/ca-trust/extracted"
      - name: "{{ ansible_operator_meta.name }}-bundle-cacert"
        mountPath: /etc/pki/ca-trust/source/anchors/bundle-ca.crt
        subPath: bundle-ca.crt

So, based on your suggestion you mean to make use of existing bundle CA system that's already properly configured for Python containers, instead of pointing to platform-specific certificate paths like:

{% if bundle_ca_crt %}
    ca_cert_file: '/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem'
{% else %}
  # Fallback to platform-specific paths
{% endif %}

Correct?

In openshift, there are tricks to use annotations to automatically set up certs, but we don't have that available to us in k8s.

Please excuse me for my understanding as I thought operators would be used only via Openshift. And, I wanted to understand that will bundling CA could provide operator-level automation for both platforms(openshift/k8s)

[manstis]

The service-ca-operator is installed:

e.g. https://console-openshift-console.apps.stage2-west.v2dz.p1.openshiftapps.com/k8s/cluster/config.openshift.io~v1~ClusterOperator/service-ca

Seems to be installed with OCP:

https://docs.redhat.com/en/documentation/openshift_container_platform/4.11/html/operators/cluster-operators-ref

[manstis]

I think the volumeMounts may need revisiting.

WDYT?

[manstis]

From my (limited) understanding of TLS; this LGTM 👍

[manstis]

@justjais The failing molecule tests:

2025-09-03T20:27:58.9045432Z TASK [Re-emit failure] *********************************************************
2025-09-03T20:27:58.9046025Z �[1;30mtask path: /home/runner/work/ansible-ai-connect-operator/ansible-ai-connect-operator/molecule/default/verify.yml:68�[0m
2025-09-03T20:27:58.9046169Z �[31mfatal: [localhost]: FAILED! => {�[0m
2025-09-03T20:27:58.9046297Z �[31m    "changed": false,�[0m
2025-09-03T20:27:58.9046484Z �[31m    "msg": {�[0m
2025-09-03T20:27:58.9046611Z �[31m        "result": {�[0m
2025-09-03T20:27:58.9046832Z �[31m            "assertion": "service_host_response.json.status == 'ok'",�[0m
2025-09-03T20:27:58.9046959Z �[31m            "changed": false,�[0m
2025-09-03T20:27:58.9047085Z �[31m            "evaluated_to": false,�[0m
2025-09-03T20:27:58.9047294Z �[31m            "failed": true,�[0m
2025-09-03T20:27:58.9047737Z �[31m            "msg": "/check endpoint did not return expected response. Expected HTTP200, OK."�[0m
2025-09-03T20:27:58.9047831Z �[31m        }�[0m
2025-09-03T20:27:58.9048139Z �[31m    }�[0m
2025-09-03T20:27:58.9048295Z �[31m}�[0m

Are probably caused by ansible/ansible-ai-connect-service#1757

The test expects ok whereas the above PR changed it to be good.

[manstis]

@justjais There may be other tests too.. you should check.

For example https://github.com/ansible/ansible-ai-connect-operator/blob/main/molecule/default/tasks/2_0_model_config_test.yml#L32

[manstis]

LGTM 👍

Thank-you.

a