Configurable Inference timeout during Query time

[Samiul-TheSoccerFan]

This PR focuses on introducing user configurable inference timeout settings and use that as timeout during inference calls. Currently, it is hardcoded to 10s and the goal is to make it configurable.

Setup

PUT _inference/sparse_embedding/my-elser-model
{
  "service": "elser",
  "service_settings": {
    "num_allocations": 1,
    "num_threads": 1
  },
  "task_settings": {}
}

PUT my-semantic-index-5
{
  "mappings": {
    "properties": {
      "writer": {
        "type": "semantic_text",
        "inference_id": "my-elser-model"
      },
      "reader": {
        "type": "semantic_text"
      }
    }
  }
}

PUT my-semantic-index-6
{
  "mappings": {
    "properties": {
      "writer": {
        "type": "semantic_text",
        "inference_id": "my-elser-model"
      },
      "reader": {
        "type": "semantic_text"
      }
    }
  }
}

POST my-semantic-index-5/_doc/1
{
  "writer": "Little Red Riding Hood",
  "reader": ["inference test", "another inference test"]
}

POST my-semantic-index-5/_doc/2
{
  "writer": "Another Little Red Riding Hood",
  "reader": ["inference test", "another inference test"]
}

POST my-semantic-index-5/_doc/3
{
   "writer": "Another Little Red Riding Hood",
  "reader": ["inference test", "another inference test"]
}

POST my-semantic-index-6/_doc/1
{
  "writer": "Little Red Riding Hood",
  "reader": ["inference test", "another inference test"]
}

POST my-semantic-index-6/_doc/2
{
  "writer": "Another Little Red Riding Hood",
  "reader": ["inference test", "another inference test"]
}

POST my-semantic-index-6/_doc/3
{
   "writer": "Another Little Red Riding Hood",
  "reader": ["inference test", "another inference test"]
}

GET the default settings:

GET /my-semantic-index-5/_settings

GET /my-semantic-index-5/_settings?include_defaults=true

GET /my-semantic-index-6/_settings

GET /my-semantic-index-6/_settings?include_defaults=true

Update the inference timeout value:

PUT /my-semantic-index-6/_settings
{
  "index": {
    "semantic_text": {
      "inference_timeout": "1s"
    }
  }
}

GET the updated settings:

GET /my-semantic-index-5/_settings

GET /my-semantic-index-5/_settings?include_defaults=true

GET /my-semantic-index-6/_settings

GET /my-semantic-index-6/_settings?include_defaults=true

[elasticsearchmachine]

Pinging @elastic/search-eng (Team:SearchOrg)

[elasticsearchmachine]

Pinging @elastic/search-relevance (Team:Search - Relevance)

[elasticsearchmachine]

Hi @Samiul-TheSoccerFan, I've created a changelog YAML for you.

[Samiul-TheSoccerFan]

@Mikep86 Do we need to ping ML team in the PR too?

[Mikep86]

@Samiul-TheSoccerFan Yes, we should ping the ML team since it touches code they own

[elasticsearchmachine]

Pinging @elastic/ml-core (Team:ML)

[jonathan-buttner]

I left a comment, if you could take a look that'd be great!

[kderusso]

Change looks good to me, but some more tests need to be updated

[jonathan-buttner]

I left a few suggestions.

[Mikep86]

Partial review, I think we have unhandled edge cases and potentially divergent default values to manage.

[Mikep86]

We only want to apply this timeout if the input type is SEARCH or INTERNAL_SEARCH. Which brings up another edge case: If we allow timeout to be null now, we need to set default timeouts for the other input types as well.

[Samiul-TheSoccerFan]

@elasticmachine update branch

[Mikep86]

Got to everything except SageMakerServiceTests. I will take a look at those in a follow-up review.

[Mikep86]

And a review of SageMakerServiceTests :)

[Samiul-TheSoccerFan]

@elasticmachine update branch

[davidkyle]

I have a refactor PR that will probably cause a merge conflict with this PR

In #131759 the clusterService member is removed from all the Inference Service implementations as intellij was reporting it as being unused. @Samiul-TheSoccerFan do you need the clusterService here? I'm happy to close my PR without merging otherwise if you just need it for the ElasticsearchInternalService I can work around that.

[Samiul-TheSoccerFan]

@davidkyle Yes, we do need to pass the clusterService to all services so it become available to their super classes (SageMaker, and SenderService). The unused issue hopefully will be go away once we merge this PR.

[Mikep86]

One little thing to fix up, then we're good to go 👍

[Samiul-TheSoccerFan]

@elasticmachine update branch

[Samiul-TheSoccerFan]

@elasticmachine update branch

[Mikep86]

LGTM!