Skip to content

Conversation

dbwiddis
Copy link
Member

Description

The ML Client requires a few runtime dependencies that are not included in the shadow JAR, primarily associated with validation of ML Connector fields. The requirement to have these dependencies by downstream consumers of the client is not documented nor obvious.

In the Maven dependency ecosystem, the provided dependency scope is a clear indicator of a required runtime dependency that the project is not providing itself:

provided
This is much like compile, but indicates you expect the JDK or a container to provide the dependency at runtime. For example, when building a web application for the Java Enterprise Edition, you would set the dependency on the Servlet API and related Java EE APIs to scope provided because the web container provides those classes. A dependency with this scope is added to the classpath used for compilation and test, but not the runtime classpath. It is not transitive.

This PR appends a <dependency> section to the published opensearch-ml-client POM file on the Central Repository. The additional section added will be:

<dependencies>
  <dependency>
    <groupId>org.json</groupId>
    <artifactId>json</artifactId>
    <version>20231013</version>
    <scope>provided</scope>
  </dependency>
  <dependency>
    <groupId>com.jayway.jsonpath</groupId>
    <artifactId>json-path</artifactId>
    <version>2.9.0</version>
    <scope>provided</scope>
  </dependency>
  <dependency>
    <groupId>org.apache.commons</groupId>
    <artifactId>commons-text</artifactId>
    <version>1.10.0</version>
    <scope>provided</scope>
  </dependency>
</dependencies>

Note that there is an additional dependency in StringUtils on com.networknt:json-schema-validator:1.4.0. Currently this is not required by the ML Client, but if any future changes to the client include schema validation, that should be added to this list.

This has no impact on actual included dependencies; it's simply a documentation signal to clearly indicate to consumers of this client that they must provide these dependencies themselves.

Related Issues

Check List

  • New functionality has been documented.
  • Commits are signed per the DCO using --signoff.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@dbwiddis dbwiddis requested a deployment to ml-commons-cicd-env-require-approval August 22, 2025 22:06 — with GitHub Actions Waiting
@dbwiddis dbwiddis requested a deployment to ml-commons-cicd-env-require-approval August 22, 2025 22:06 — with GitHub Actions Waiting
@dbwiddis dbwiddis requested a deployment to ml-commons-cicd-env-require-approval August 22, 2025 22:06 — with GitHub Actions Waiting
@dbwiddis dbwiddis requested a deployment to ml-commons-cicd-env-require-approval August 22, 2025 22:06 — with GitHub Actions Waiting
@dbwiddis dbwiddis force-pushed the client-dependencies branch from 8ef79c2 to 3c8413b Compare August 22, 2025 22:10
@dbwiddis dbwiddis had a problem deploying to ml-commons-cicd-env-require-approval August 22, 2025 22:12 — with GitHub Actions Error
@dbwiddis dbwiddis had a problem deploying to ml-commons-cicd-env-require-approval August 22, 2025 22:12 — with GitHub Actions Failure
@dbwiddis dbwiddis had a problem deploying to ml-commons-cicd-env-require-approval August 22, 2025 22:12 — with GitHub Actions Error
@dbwiddis dbwiddis had a problem deploying to ml-commons-cicd-env-require-approval August 22, 2025 22:12 — with GitHub Actions Failure
withXml {
def dependencies = asNode().appendNode('dependencies')

def addProvidedDependency = { group, artifact, version ->
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fixing this issue. Have you verified this? Any impact to other plugins using ml-commons clients like neural-search?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This really only changes the POM which wouuld in theory add transitive dependencies, ecxept:

  • we don't do transitive dependencies in OpenSearch (the source of the problem!)
  • it's in scope <provided> which means it wouldn't be imported transitively if it did. This is purely a user documentation change.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did test it with flow framework by deleting those imports (and getting runtime errors) even with the new snapshot containing the new values in the pom in my maven local repository, so it confirms these imports aren't included.

@dbwiddis dbwiddis requested a deployment to ml-commons-cicd-env-require-approval August 29, 2025 01:22 — with GitHub Actions Waiting
@dbwiddis dbwiddis requested a deployment to ml-commons-cicd-env-require-approval August 29, 2025 01:22 — with GitHub Actions Waiting
@dbwiddis dbwiddis requested a deployment to ml-commons-cicd-env-require-approval August 29, 2025 01:22 — with GitHub Actions Waiting
@dbwiddis dbwiddis requested a deployment to ml-commons-cicd-env-require-approval August 29, 2025 01:22 — with GitHub Actions Waiting
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] Downstream dependencies struggle to find transitive dependencies in StringUtils
2 participants