Files
OpenSearch-Docs-Cn/_query-dsl/specialized/neural-sparse.md
T
242c35db59 add pretrained model description and saving disk description (#5383)
* add description and saving disk

Signed-off-by: xinyual <[email protected]>

* update

Signed-off-by: xinyual <[email protected]>

* update

Signed-off-by: xinyual <[email protected]>

* Update _ml-commons-plugin/pretrained-models.md

Co-authored-by: kolchfa-aws <[email protected]>
Signed-off-by: xinyual <[email protected]>

* Update _query-dsl/specialized/neural-sparse.md

Co-authored-by: kolchfa-aws <[email protected]>
Signed-off-by: xinyual <[email protected]>

* Update _search-plugins/neural-sparse-search.md

Co-authored-by: kolchfa-aws <[email protected]>
Signed-off-by: xinyual <[email protected]>

* Update _search-plugins/neural-sparse-search.md

Co-authored-by: kolchfa-aws <[email protected]>
Signed-off-by: xinyual <[email protected]>

* Update _search-plugins/neural-sparse-search.md

Co-authored-by: kolchfa-aws <[email protected]>
Signed-off-by: xinyual <[email protected]>

* Moved recommended model choice options outside the table

Signed-off-by: Fanit Kolchina <[email protected]>

* Add a link for more info

Signed-off-by: Fanit Kolchina <[email protected]>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: kolchfa-aws <[email protected]>

---------

Signed-off-by: xinyual <[email protected]>
Signed-off-by: xinyual <[email protected]>
Signed-off-by: Fanit Kolchina <[email protected]>
Signed-off-by: kolchfa-aws <[email protected]>
Co-authored-by: kolchfa-aws <[email protected]>
Co-authored-by: Fanit Kolchina <[email protected]>
Co-authored-by: Nathan Bower <[email protected]>
2023-10-25 11:59:49 -04:00

2.1 KiB

layout, title, parent, grand_parent, nav_order
layout title parent grand_parent nav_order
default Neural sparse Specialized queries Query DSL 55

Neural sparse query

Introduced 2.11 {: .label .label-purple }

Use the neural_sparse query for vector field search in sparse neural search.

Request fields

Include the following request fields in the neural_sparse query:

"neural_sparse": {
  "<vector_field>": {
    "query_text": "<query_text>",
    "model_id": "<model_id>",
    "max_token_score": "<max_token_score>"
  }
}

The top-level vector_field specifies the vector field against which to run a search query. The following table lists the other neural_sparse query fields.

Field | Data type | Required/Optional | Description :--- | :--- | :--- query_text | String | Required | The query text from which to generate vector embeddings. model_id | String | Required | The ID of the sparse encoding model or tokenizer model that will be used to generate vector embeddings from the query text. The model must be deployed in OpenSearch before it can be used in sparse neural search. For more information, see Using custom models within OpenSearch and Semantic search. max_token_score | Float | Optional | The theoretical upper bound of the score for all tokens in the vocabulary (required for performance optimization). For OpenSearch-provided pretrained sparse embedding models, we recommend setting max_token_score to 2 for amazon/neural-sparse/opensearch-neural-sparse-encoding-doc-v1 and to 3.5 for amazon/neural-sparse/opensearch-neural-sparse-encoding-v1.

Example request

GET my-nlp-index/_search
{
  "query": {
    "neural_sparse": {
      "passage_embedding": {
        "query_text": "Hi world",
        "model_id": "aP2Q8ooBpBj3wT4HVS8a",
        "max_token_score": 2
      }
    }
  }
}

{% include copy-curl.html %}