Compare commits

...
Author SHA1 Message Date
aetter ab00a05549 Comment out Java for now 2021-09-30 13:48:32 -07:00
aetter 96f94c649e Merge branch 'main' into python-client 2021-09-21 16:40:46 -07:00
aetter 66ca05e714 Update links 2021-09-21 13:56:25 -07:00
aetter 65f333d038 Update python.md 2021-09-21 13:40:38 -07:00
aetter 30378f0076 Sentence case access control page names 2021-09-21 12:34:59 -07:00
aetter 4ffdce6377 Minor improvements
Troubleshooting expired certificates, settings, broken link, etc.
2021-09-21 11:34:44 -07:00
aetter 21ae64aac9 Move to h2 2021-09-17 14:17:21 -07:00
aetter ba9f432622 Fix table 2021-09-16 14:38:01 -07:00
Keith Chan 35a7b186fc Fixed broken link 2021-09-14 14:03:29 -07:00
Ashwin Kumar 6a36ee4535 Merge pull request #175 from opensearch-project/helm_update
Updated helm project path
2021-09-14 12:31:58 -07:00
ashwinkumar12345 2a0cf500cc updated helm project path 2021-09-14 11:10:11 -07:00
aetter 50f3fa51d8 Update python.md 2021-09-10 15:01:43 -07:00
Ashwin Kumar 5a1a4cfdef Merge pull request #162 from opensearch-project/count
Added count REST API
2021-09-10 14:25:17 -07:00
ashwinkumar12345 b030194981 incorporated feedback 2021-09-10 14:25:00 -07:00
aetter c507de5b9f Minor wording changes for Beats OSS 2021-09-09 11:22:13 -07:00
aetter 2c713e43ad Update index.md 2021-09-08 10:08:33 -07:00
aetter 243c7315e4 Update generate-certificates.md 2021-09-08 08:34:57 -07:00
Andrew Etter 0fc6a9e057 Merge pull request #165 from opensearch-project/upgrade-tip
Add note about enabling compatibility setting during upgrade
2021-09-07 18:04:15 -07:00
aetter eaba608cfd Update some Dashboards settings paths 2021-09-07 15:28:29 -07:00
Keith Chan b1d3a5906b Language tweaks to create-index 2021-09-01 19:13:23 -07:00
aetter 34068f01bd Add note about enabling compatibility setting during upgrade 2021-09-01 16:29:19 -07:00
aetter ecea83707d Adds Lucene version variable, minor improvements to REST API and query DSL 2021-09-01 11:37:54 -07:00
Andrew Etter f29b04bdb9 Merge pull request #164 from opensearch-project/1.0.1
Version bumps for 1.0.1
2021-09-01 11:27:32 -07:00
aetter cda046f727 Version bumps for 1.0.1 2021-09-01 11:25:39 -07:00
aetter ffe3f77993 Light edits to data streams 2021-08-31 10:50:48 -07:00
aetter 60b1859f1e Remove "About data streams," "Date math support for index names," and "Simulate multi-component templates" 2021-08-31 10:43:48 -07:00
ashwinkumar12345 baa5622f9d minor changes 2021-08-31 10:29:20 -07:00
ashwinkumar12345 dc48c6ced5 added count API 2021-08-30 12:58:30 -07:00
Keith Chan 62611a7605 Merge pull request #161 from opensearch-project/document-rest-apis
Tweaked document REST APIs
2021-08-30 12:46:52 -07:00
Keith Chan 94503ea959 Language tweaks 2021-08-30 11:54:46 -07:00
keithhc2 449177c07e Fixed typos 2021-08-30 11:47:02 -07:00
keithhc2 f464d01041 Revert "Fixed typos"
This reverts commit d941cb6ad7.
2021-08-30 11:44:35 -07:00
keithhc2 36694537f6 Space? 2021-08-30 11:42:52 -07:00
keithhc2 26d3dfe351 Maybe this is the label 2021-08-30 11:42:00 -07:00
keithhc2 7faade1bdf Maybe this can resolve conflicts 2021-08-30 11:38:59 -07:00
keithhc2 5ddabb88bd Fixed heading and styling 2021-08-30 11:35:11 -07:00
keithhc2 edd9d2e313 Fixed even more typos 2021-08-30 11:34:01 -07:00
keithhc2 d941cb6ad7 Fixed typos 2021-08-30 11:32:24 -07:00
keithhc2 17f91f09a3 Language tweaks everywhere 2021-08-30 11:29:19 -07:00
aetter 36853b6484 Update external link to Dashboards developer guide 2021-08-27 12:46:55 -07:00
Andrew Etter eaeeb97630 Merge pull request #123 from closingin/feature/add-osd-dev-server-docs
feat: add a link to the developer guide
2021-08-27 12:46:07 -07:00
Andrew Etter dd2d93793a Merge pull request #156 from opensearch-project/ad_shingle_size
Changed window size to shingle size for AD
2021-08-27 12:37:37 -07:00
Andrew Etter 6c5432db1f Merge pull request #157 from opensearch-project/minimal-dashboards-redirects
Some redirects for OpenSearch Dashboards 1.0.0
2021-08-27 09:30:05 -07:00
aetter c972869893 Link to pip, because why not 2021-08-26 11:50:46 -07:00
aetter ac9acb3c62 Adds some statements around compatibility
Should (hopefully) apply to all clients.
2021-08-26 09:15:50 -07:00
ashwinkumar12345 78621f98e3 minor change 2021-08-25 16:34:46 -07:00
aetter bdffb128d8 Some redirects for OpenSearch Dashboards 1.0.0 2021-08-25 15:17:51 -07:00
aetter 0376c4b6d9 Fix link. 2021-08-25 14:55:03 -07:00
Andrew Etter ff8cd66090 Merge pull request #155 from ict-one-nl/patch-1
Use same attribute name as cluster page
2021-08-25 14:52:13 -07:00
ashwinkumar12345 df42f777f9 changed window size to shingle size 2021-08-25 11:21:56 -07:00
ict-one-nl f3a9bad35c Use same attribute name as cluster page
Simple fix, but because https://opensearch.org/docs/opensearch/cluster/ uses the node.attr.temp setting and this page uses the box_type setting a less experienced user like me easily makes a mistake (not comprehending the exact workings). Using the temp attribute here as well would have saved me a couple of hours and some grey hairs.
2021-08-25 13:54:54 +02:00
aetter 30facfe628 Typo 2021-08-19 14:38:30 -07:00
aetter 9ce5d95786 Clean up spacing 2021-08-19 13:23:14 -07:00
aetter 0bf8624824 Getting started content for Python 2021-08-19 12:56:53 -07:00
closingin 8e3451358b feat: add a link to the developer guide
Signed-off-by: closingin <[email protected]>
2021-08-19 08:47:17 +02:00
58 changed files with 543 additions and 398 deletions
+1 -1
View File
@@ -196,7 +196,7 @@ If you're making major changes to the documentation and need to see the rendered
## New releases
1. Branch.
1. Change the `opensearch_version` and `opensearch_major_version` variables in `_config.yml`.
1. Change the `opensearch_version` and `opensearch_major_minor_version` variables in `_config.yml`.
1. Start up a new cluster using the updated Docker Compose file in `docs/install/docker.md`.
1. Update the version table in `version-history.md`.
+8 -2
View File
@@ -27,12 +27,18 @@ PUT _cluster/settings
}
```
[Just like any other setting]({{site.url}}{{site.baseurl}}/opensearch/configuration/), the alternative is to add the following line to `opensearch.yml` on each node and then restart the node:
```yml
compatibility.override_main_response_version: true
```
## Downloads
You can download the OpenSearch output plugin for Logstash from [OpenSearch downloads](https://opensearch.org/downloads.html). The Logstash output plugin is compatible with OpenSearch and Elasticsearch OSS (7.10.2 or lower).
These versions of Beats offer the best compatibility with OpenSearch. For more information, see the [compatibility matrices](#compatibility-matrices).
These are the latest versions of Beats OSS with OpenSearch compatibility. For more information, see the [compatibility matrices](#compatibility-matrices).
- [Filebeat OSS 7.12.1](https://www.elastic.co/downloads/past-releases/filebeat-oss-7-12-1)
- [Metricbeat OSS 7.12.1](https://www.elastic.co/downloads/past-releases/metricbeat-oss-7-12-1)
@@ -41,7 +47,7 @@ These versions of Beats offer the best compatibility with OpenSearch. For more i
- [Winlogbeat OSS 7.12.1](https://www.elastic.co/downloads/past-releases/winlogbeat-oss-7-12-1)
- [Auditbeat OSS 7.12.1](https://elastic.co/downloads/past-releases/auditbeat-oss-7-12-1)
Some users report compatibility issues with ingest pipelines on these versions of Beats. If you use ingest pipelines with OpenSearch, consider using the 7.10.2 versions of Beats OSS instead.
Some users report compatibility issues with ingest pipelines on these versions of Beats. If you use ingest pipelines with OpenSearch, consider using the 7.10.2 versions of Beats instead.
{: .note }
+15 -1
View File
@@ -9,6 +9,20 @@ redirect_from:
# OpenSearch client compatibility
OpenSearch provides clients for several popular programming languages, with more coming. In general, clients are compatible with clusters running the same major version of OpenSearch (`major.minor.patch`).
For example, a 1.0.0 client works with an OpenSearch 1.1.0 cluster, but might not support any non-breaking API changes in OpenSearch 1.1.0. A 1.2.0 client works with the same cluster, but might allow you to pass unsupported options in certain functions. We recommend using the same version for both, but if your tests pass after a cluster upgrade, you don't necessarily need to upgrade your clients immediately.
{% comment %}
* [OpenSearch Java client]({{site.url}}{{site.baseurl}}/clients/java/)
{% endcomment %}
* [OpenSearch Python client]({{site.url}}{{site.baseurl}}/clients/python/)
* [OpenSearch JavaScript (Node.js) client]({{site.url}}{{site.baseurl}}/clients/javascript/)
* [OpenSearch Go client]({{site.url}}{{site.baseurl}}/clients/go/)
## Legacy clients
Most clients that work with Elasticsearch OSS 7.10.2 *should* work with OpenSearch, but the latest versions of those clients might include license or version checks that artificially break compatibility. This page includes recommendations around which versions of those clients to use for best compatibility with OpenSearch.
Client | Recommended version
@@ -18,7 +32,7 @@ Client | Recommended version
[Python Elasticsearch client](https://pypi.org/project/elasticsearch/7.13.4/) | 7.13.4
[Elasticsearch Node.js client](https://www.npmjs.com/package/@elastic/elasticsearch/v/7.13.0) | 7.13.0
Clients exist for a wide variety of languages, so if you test a client and verify that it works, please [submit a PR](https://github.com/opensearch-project/documentation-website/pulls) and add it to this table.
If you test a legacy client and verify that it works, please [submit a PR](https://github.com/opensearch-project/documentation-website/pulls) and add it to this table.
{% comment %}
+1 -1
View File
@@ -1,7 +1,7 @@
---
layout: default
title: Java high-level REST client
nav_order: 97
nav_order: 60
---
# Java high-level REST client
+128
View File
@@ -0,0 +1,128 @@
---
layout: default
title: Python client
nav_order: 70
---
# Python client
The OpenSearch Python client provides a more natural syntax for interacting with your cluster. Rather than sending HTTP requests to a given URL, you can create an OpenSearch client for your cluster and call the client's built-in functions.
{% comment %}
`opensearch-py` is the lower-level of the two Python clients. If you want a general client for assorted operations, it's a great choice. If you want a higher-level client strictly for indexing and search operations, consider [opensearch-dsl-py]({{site.url}}{{site.baseurl}}/clients/python-dsl/).
{% endcomment %}
## Setup
To add the client to your project, install it using [pip](https://pip.pypa.io/):
```bash
pip install opensearch-py
```
Then import it like any other module:
```python
from opensearchpy import OpenSearch
```
If you prefer to add the client manually or just want to examine the source code, see [opensearch-py on GitHub](https://github.com/opensearch-project/opensearch-py).
## Sample code
```python
from opensearchpy import OpenSearch
host = 'localhost'
port = 9200
auth = ('admin', 'admin') # For testing only. Don't store credentials in code.
ca_certs_path = '/full/path/to/root-ca.pem' # Provide a CA bundle if you use intermediate CAs with your root CA.
# Optional client certificates if you don't want to use HTTP basic authentication.
# client_cert_path = '/full/path/to/client.pem'
# client_key_path = '/full/path/to/client-key.pem'
# Create the client with SSL/TLS enabled, but hostname verification disabled.
client = OpenSearch(
hosts = [{'host': host, 'port': port}],
http_compress = True, # enables gzip compression for request bodies
http_auth = auth,
# client_cert = client_cert_path,
# client_key = client_key_path,
use_ssl = True,
verify_certs = True,
ssl_assert_hostname = False,
ssl_show_warn = False,
ca_certs = ca_certs_path
)
# Create an index with non-default settings.
index_name = 'python-test-index'
index_body = {
'settings': {
'index': {
'number_of_shards': 4
}
}
}
response = client.indices.create(index_name, body=index_body)
print('\nCreating index:')
print(response)
# Add a document to the index.
document = {
'title': 'Moneyball',
'director': 'Bennett Miller',
'year': '2011'
}
id = '1'
response = client.index(
index = index_name,
body = document,
id = id,
refresh = True
)
print('\nAdding document:')
print(response)
# Search for the document.
q = 'miller'
query = {
'size': 5,
'query': {
'multi_match': {
'query': q,
'fields': ['title^2', 'director']
}
}
}
response = client.search(
body = query,
index = index_name
)
print('\nSearch results:')
print(response)
# Delete the document.
response = client.delete(
index = index_name,
id = id
)
print('\nDeleting document:')
print(response)
# Delete the index.
response = client.indices.delete(
index = index_name
)
print('\nDeleting index:')
print(response)
```
+2 -1
View File
@@ -5,8 +5,9 @@ baseurl: "/docs" # the subpath of your site, e.g. /blog
url: "https://opensearch.org" # the base hostname & protocol for your site, e.g. http://example.com
permalink: /:path/
opensearch_version: 1.0.0
opensearch_version: 1.0.1
opensearch_major_minor_version: 1.0
lucene_version: 8_8_2
# Build settings
markdown: kramdown
+3
View File
@@ -5,9 +5,12 @@ nav_order: 1
has_children: false
has_toc: false
redirect_from:
- /docs/opensearch-dashboards/
- /dashboards/
---
{%- comment -%}The `/docs/opensearch-dashboards/` redirect is specifically to support the UI links in OpenSearch Dashboards 1.0.0.{%- endcomment -%}
# OpenSearch Dashboards
OpenSearch Dashboards is the default visualization tool for data in OpenSearch. It also serves as a user interface for many of the OpenSearch plugins, including security, alerting, Index State Management, SQL, and more.
+15 -30
View File
@@ -28,6 +28,21 @@ If you don't want to use the all-in-one installation options, you can install th
</tr>
</thead>
<tbody>
<tr>
<td>1.0.1</td>
<td>
<pre>alertingDashboards 1.0.0.0
anomalyDetectionDashboards 1.0.0.0
ganttChartDashboards 1.0.0.0
indexManagementDashboards 1.0.1.0
notebooksDashboards 1.0.0.0
queryWorkbenchDashboards 1.0.0.0
reportsDashboards 1.0.1.0
securityDashboards 1.0.1.0
traceAnalyticsDashboards 1.0.0.0
</pre>
</td>
</tr>
<tr>
<td>1.0.0</td>
<td>
@@ -40,36 +55,6 @@ queryWorkbenchDashboards 1.0.0.0
reportsDashboards 1.0.0.0
securityDashboards 1.0.0.0
traceAnalyticsDashboards 1.0.0.0
</pre>
</td>
</tr>
<tr>
<td>1.0.0-rc1</td>
<td>
<pre>alertingDashboards 1.0.0.0-rc1
anomalyDetectionDashboards 1.0.0.0-rc1
ganttChartDashboards 1.0.0.0-rc1
indexManagementDashboards 1.0.0.0-rc1
notebooksDashboards 1.0.0.0-rc1
queryWorkbenchDashboards 1.0.0.0-rc1
reportsDashboards 1.0.0.0-rc1
securityDashboards 1.0.0.0-rc1
traceAnalyticsDashboards 1.0.0.0-rc1
</pre>
</td>
</tr>
<tr>
<td>1.0.0-beta1</td>
<td>
<pre>alertingDashboards 1.0.0.0-beta1
anomalyDetectionDashboards 1.0.0.0-beta1
ganttChartDashboards 1.0.0.0-beta1
indexManagementDashboards 1.0.0.0-beta1
notebooksDashboards 1.0.0.0-beta1
queryWorkbenchDashboards 1.0.0.0-beta1
reportsDashboards 1.0.0.0-beta1
securityDashboards 1.0.0.0-beta1
traceAnalyticsDashboards 1.0.0.0-beta1
</pre>
</td>
</tr>
+4
View File
@@ -2,8 +2,12 @@
layout: default
title: WMS map server
nav_order: 5
redirect_from:
- /docs/opensearch-dashboards/maptiles/
---
{%- comment -%}The `/docs/opensearch-dashboards/maptiles/` redirect is specifically to support the UI links in OpenSearch Dashboards 1.0.0.{%- endcomment -%}
# Configure WMS map server
OpenSearch Dashboards includes default map tiles, but if you need more specialized maps, you can configure OpenSearch Dashboards to use a WMS map server:
+7
View File
@@ -0,0 +1,7 @@
---
layout: default
title: Dashboards developer guide
nav_order: 2
permalink: /dashboards-developer-guide/
redirect_to: https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/DEVELOPER_GUIDE.md
---
+2 -2
View File
@@ -347,7 +347,7 @@ Parameter | Description | Type | Required | Default
### allocation
Allocate the index to a node with a specific attribute.
Allocate the index to a node with a specific attribute set [like this]({{site.url}}{{site.baseurl}}/opensearch/cluster/#advanced-step-7-set-up-a-hot-warm-architecture).
For example, setting `require` to `warm` moves your data only to "warm" nodes.
The `allocation` operation has the following parameters:
@@ -363,7 +363,7 @@ Parameter | Description | Type | Required
"actions": [
{
"allocation": {
"require": { "box_type": "warm" }
"require": { "temp": "warm" }
}
}
]
+3 -5
View File
@@ -79,13 +79,11 @@ This formula provides a good starting point, but make sure to test with a repres
For example, for a cluster with 3 data nodes, each with 8G of JVM heap size, a maximum memory percentage of 10% (default), and the entity size of the detector as 1MB: the total number of unique entities supported is (8.096 * 10^9 * 0.1 / 1M ) * 3 = 2429.
#### Set a window size
#### Set a shingle size
Set the number of aggregation intervals from your data stream to consider in a detection window. It's best to choose this value based on your actual data to see which one leads to the best results for your use case.
Set the number of aggregation intervals from your data stream to consider in a detection window. It’s best to choose this value based on your actual data to see which one leads to the best results for your use case.
Based on experiments performed on a wide variety of one-dimensional data streams, we recommend using a window size between 1 and 16. The default window size is 8. If you set the category field for high cardinality, the default window size is 1.
If you expect missing values in your data or if you want to base the anomalies on the current interval, choose 1. If your data is continuously ingested and you want to base the anomalies on multiple intervals, choose a larger window size.
The anomaly detector expects the shingle size to be in the range of 1 and 60. The default shingle size is 8. We recommend that you don't choose 1 unless you have two or more features. Smaller values might increase [recall](https://en.wikipedia.org/wiki/Precision_and_recall) but also false positives. Larger values might be useful for ignoring noise in a signal.
#### Preview sample anomalies
+1 -1
View File
@@ -1,6 +1,6 @@
---
layout: default
title: Create Dashboards
title: Create PerfTop Dashboards
parent: Performance Analyzer
nav_order: 2
---
+2 -2
View File
@@ -48,11 +48,11 @@ Otherwise, just specify the OpenSearch endpoint:
./opensearch-perf-top-macos --dashboard dashboards/<dashboard>.json --endpoint my-cluster.my-domain.com
```
PerfTop has four pre-built dashboards in the `dashboards` directory, but you can also [create your own]({{site.url}}{{site.baseurl}}/dashboards/).
PerfTop has four pre-built dashboards in the `dashboards` directory, but you can also [create your own]({{site.url}}{{site.baseurl}}/monitoring-plugins/pa/dashboards/).
You can also load the pre-built dashboards (ClusterOverview, ClusterNetworkMemoryAnalysis, ClusterThreadAnalysis, or NodeAnalysis) without the JSON files, such as `--dashboard ClusterThreadAnalysis`.
PerfTop has no interactivity. Start the application, monitor the dashboard, and press esc, q, or Ctrl + C to quit.
PerfTop has no interactivity. Start the application, monitor the dashboard, and press Esc, Q, or Ctrl + C to quit.
{: .note }
+8
View File
@@ -65,4 +65,12 @@ PUT _cluster/settings
You can find `opensearch.yml` in `/usr/share/opensearch/config/opensearch.yml` (Docker) or `/etc/opensearch/opensearch.yml` (most Linux distributions) on each node.
You don't mark settings in `opensearch.yml` as persistent or transient, and settings use the flat form:
```yml
cluster.name: my-application
action.auto_create_index: true
compatibility.override_main_response_version: true
```
The demo configuration includes a number of settings for the security plugin that you should modify before using OpenSearch for a production workload. To learn more, see [Security]({{site.url}}{{site.baseurl}}/security-plugin/).
+3 -29
View File
@@ -8,39 +8,13 @@ nav_order: 13
If you're ingesting continuously generated time-series data such as logs, events, and metrics into OpenSearch, you're likely in a scenario where the number of documents grows rapidly and you don't need to update older documents.
A typical workflow to manage time-series data involves multiple steps such as creating a rollover index alias, defining a write index, and defining common mappings and settings for the backing indices.
A typical workflow to manage time-series data involves multiple steps, such as creating a rollover index alias, defining a write index, and defining common mappings and settings for the backing indices.
Data streams simplify this bootstrapping process and enforce a setup that best suits time-series data, such as being designed primarily for append-only data, and ensuring that each document has a timestamp field.
Data streams simplify this process and enforce a setup that best suits time-series data, such as being designed primarily for append-only data and ensuring that each document has a timestamp field.
A data stream is internally composed of multiple backing indices. Search requests are routed to all the backing indices, while indexing requests are routed to the latest write index. You can use [ISM]({{site.url}}{{site.baseurl}}/im-plugin/ism/index/) policies to automatically handle rollovers or deletion of indices in a data stream, based on your use case.
A data stream is internally composed of multiple backing indices. Search requests are routed to all the backing indices, while indexing requests are routed to the latest write index. [ISM]({{site.url}}{{site.baseurl}}/im-plugin/ism/index/) policies let you automatically handle index rollovers or deletions.
## About data streams
A data stream consists of one or more hidden auto-generated backing indices. These backing indices are named using the following convention:
```
.ds-<data-stream-name>-<generation-id>
```
For example, `.ds-logs-redis-000003`, where generation-id is a six-digit, zero-padded integer that acts as a cumulative count of the data stream’s rollovers, starting at `000001`.
The most recently created backing index is the data stream’s write index. You can’t add documents directly to any of the backing indices. You can only add them via the data stream handle:
![data stream indexing diagram]({{site.url}}{{site.baseurl}}/images/data_stream_indexing.png)
The data stream routes search requests to all of its backing indices. It uses the timestamp field to intelligently route search requests to the right set of indices and shards:
![data stream indexing diagram]({{site.url}}{{site.baseurl}}/images/data_stream_searching.png)
The following operations are not supported on the write index because they might hinder the indexing operation:
- close
- clone
- delete
- shrink
- split
## Get started with data streams
### Step 1: Create an index template
-60
View File
@@ -1,60 +0,0 @@
---
layout: default
title: Date math support for index names
nav_order: 92
---
# Date math support for index names
Date math is shorthand arithmetic for finding relative dates.
If you're indexing time-series data with the dates mapped in the index names, you can use date math in your queries to filter index names and limit the number of searched indices.
## Date math syntax
The date math syntax for an index name is as follows:
```
<static_name{date_math_expr{date_format|time_zone}}>
```
- `static_name`: The unchanged or static portion of the index name. To use the characters `{` and `}` in the static part of an index name, escape them with a backslash `\`.
- `date_math_expr`: The changing or dynamic portion of the index name that’s computed by the date math expression. For example, `now+1h` adds one hour, `now-1d` subtracts one hour, and `now/d` rounds down to the nearest day, where `now` represents the current timestamp.
- `date_format`: (Optional) Specify the format for the computed date. The default value is `YYYY.MM.dd`. Make sure that you’re using the correct small or capital letters in the date format. For example, `mm` denotes minute of hour, while `MM` denotes month of year. Similarly, `hh` denotes the hour in the `1-12` range in combination with AM/PM, while `HH` denotes the hour in the `0-23` 24-hour range.
- `time_zone`: (Optional) Specify the timezone offset. The default value is UTC. For example, the UTC time offset for PST is `-08:00`.
## Date math example
You must enclose date math index names within angle brackets.
If today is 22nd March, 2024:
- `<logstash-{now/d}>` resolves to `logstash-2024.03.22`
- `<logstash-{now/M}>` resolves to `logstash-2024.03.01`
- `<logstash-{now/M{YYYY.MM}}>` resolves to `logstash-2024.03`
- `<logstash-{now/M-1M{YYYY.MM}}>` resolves to `logstash-2024.02`
- `<logstash-{now/d{yyyy.MM.dd|+12:00}}>` resolves to `logstash-2024.03.23`
You need to encode all special characters in URI format:
Special characters | URI format
:--- | :---
`<` | %3C
`>` | %3E
`/` | %2F
`{` | %7B
`}` | %7D
`|` | %7C
`+` | %2B
`:` | %3A
`,` | %2C
`\` | %5C
If you are searching for errors in your daily logs with the default Logstash index name format `logstash-YYYY.MM.dd`, you can use date math to restrict the search to indices of the past three days:
```
# GET <logstash-{now/d-2d}>,<logstash-{now/d-1d}>,<logstash-{now/d}>/_search
GET %3Clogstash-%7Bnow%2Fd-2d%7D%3E%2C%3Clogstash-%7Bnow%2Fd-1d%7D%3E%2C%3Clogstash-%7Bnow%2Fd%7D%3E/_search
```
This date math expression is evaluated at runtime.
-65
View File
@@ -328,71 +328,6 @@ GET logs-2020-01-01
}
```
### Simulate multi-component templates
For index templates composed of multiple component templates, you can simulate applying a new template to verify whether the settings are applied as you expect.
To simulate the settings that would be applied to a specific index name:
```json
POST _index_template/_simulate_index/<index_name>
```
To simulate the settings that would be applied from an existing template:
```json
POST _index_template/_simulate/<index_template>
```
You can also specify a template definition in the simulate request:
```json
POST _index_template/_simulate
{
"index_patterns": [
"logs-2020-01-*"
],
"template": {
"settings" : {
"index.number_of_shards" : 3
}
},
"composed_of": ["component_template_1", "component_template_2"]
}
```
The `_simulate` API returns the final settings, mappings, and aliases that will be applied to indices that match the index pattern. You can also see any overlapping templates whose configuration is superseded by the simulated template body or higher priority templates:
```json
{
"template" : {
"settings" : {
"index" : {
"number_of_shards" : "3"
}
},
"mappings" : {
"properties" : {
"@timestamp" : {
"type" : "date"
},
"ip_address" : {
"type" : "ip"
}
}
},
"aliases" : { }
},
"overlapping" : [
{
"name" : "daily_logs",
"index_patterns" : [
"logs-2020-01-*"
]
}
]
}
```
## Index template options
+3
View File
@@ -5,9 +5,12 @@ nav_order: 1
has_children: false
has_toc: false
redirect_from:
- /docs/opensearch/
- /opensearch/
---
{%- comment -%}The `/docs/opensearch/` redirect is specifically to support the UI links in OpenSearch Dashboards 1.0.0.{%- endcomment -%}
# Introduction to OpenSearch
OpenSearch is a distributed search and analytics engine based on [Apache Lucene](https://lucene.apache.org/). After adding your data to OpenSearch, you can perform full-text searches on it with all of the features you might expect: search by field, search multiple indices, boost fields, rank results by score, sort results by field, and aggregate results.
+3 -3
View File
@@ -31,16 +31,16 @@ The default Helm chart deploys a three-node cluster. We recommend that you have
## Install OpenSearch using Helm
1. Clone the [opensearch-devops](https://github.com/opensearch-project/opensearch-devops/) repository:
1. Clone the [helm-charts](https://github.com/opensearch-project/helm-charts) repository:
```bash
git clone https://github.com/opensearch-project/opensearch-devops.git
git clone https://github.com/opensearch-project/helm-charts
```
1. Change to the `opensearch` directory:
```bash
cd Helm/opensearch
cd charts/opensearch
```
1. Package the Helm chart:
+17 -33
View File
@@ -29,6 +29,23 @@ If you don't want to use the all-in-one OpenSearch installation options, you can
</tr>
</thead>
<tbody>
<tr>
<td>1.0.1</td>
<td>
<pre>opensearch-alerting 1.0.0.0
opensearch-anomaly-detection 1.0.0.0
opensearch-asynchronous-search 1.0.0.0
opensearch-index-management 1.0.1.0
opensearch-job-scheduler 1.0.0.0
opensearch-knn 1.0.0.0
opensearch-notebooks 1.0.0.0
opensearch-performance-analyzer 1.0.1.0
opensearch-reports-scheduler 1.0.0.0
opensearch-security 1.0.1.0
opensearch-sql 1.0.0.0
</pre>
</td>
</tr>
<tr>
<td>1.0.0</td>
<td>
@@ -43,39 +60,6 @@ opensearch-performance-analyzer 1.0.0.0
opensearch-reports-scheduler 1.0.0.0
opensearch-security 1.0.0.0
opensearch-sql 1.0.0.0
</pre>
</td>
</tr>
<tr>
<td>1.0.0-rc1</td>
<td>
<pre>opensearch-alerting 1.0.0.0-rc1
opensearch-anomaly-detection 1.0.0.0-rc1
opensearch-asynchronous-search 1.0.0.0-rc1
opensearch-index-management 1.0.0.0-rc1
opensearch-job-scheduler 1.0.0.0-rc1
opensearch-knn 1.0.0.0-rc1
opensearch-notebooks 1.0.0.0-rc1
opensearch-performance-analyzer 1.0.0.0-rc1
opensearch-reports-scheduler 1.0.0.0-rc1
opensearch-security 1.0.0.0-rc1
opensearch-sql 1.0.0.0-rc1
</pre>
</td>
</tr>
<tr>
<td>1.0.0-beta1</td>
<td>
<pre>opensearch-alerting 1.0.0.0-beta1
opensearch-anomaly-detection 1.0.0.0-beta1
opensearch-asynchronous-search 1.0.0.0-beta1
opensearch-index-management 1.0.0.0-beta1
opensearch-job-scheduler 1.0.0.0-beta1
opensearch-knn 1.0.0.0-beta1
opensearch-performance-analyzer 1.0.0.0-beta1
opensearch-reports-scheduler 1.0.0.0-beta1
opensearch-security 1.0.0.0-beta1
opensearch-sql 1.0.0.0-beta1
</pre>
</td>
</tr>
+2 -2
View File
@@ -88,8 +88,8 @@ GET opensearch_dashboards_sample_data_ecommerce/_search
}
```
The cardinality count is approximate.
If you had tens of thousands of products in your store, an accurate cardinality calculation requires loading all the values into a hash set and returning its size. This approach doesn't scale well because it requires more memory and causes high latency.
Cardinality count is approximate.
If you have tens of thousands of products in your hypothetical store, an accurate cardinality calculation requires loading all the values into a hash set and returning its size. This approach doesn't scale well; it requires huge amounts of memory and can cause high latencies.
You can control the trade-off between memory and accuracy with the `precision_threshold` setting. This setting defines the threshold below which counts are expected to be close to accurate. Above this value, counts might become a bit less accurate. The default value of `precision_threshold` is 3,000. The maximum supported value is 40,000.
+7 -7
View File
@@ -21,7 +21,7 @@ This page lists all full-text query types and common options. Given the sheer nu
## Match
Creates a [boolean query](https://lucene.apache.org/core/8_4_0/core/org/apache/lucene/search/BooleanQuery.html) that returns results if the search term is present in the field.
Creates a [boolean query](https://lucene.apache.org/core/{{site.lucene_version}}/core/org/apache/lucene/search/BooleanQuery.html) that returns results if the search term is present in the field.
The most basic form of the query provides only a field (`title`) and a term (`wind`):
@@ -126,7 +126,7 @@ GET _search
## Match boolean prefix
Similar to [match](#match), but creates a [prefix query](https://lucene.apache.org/core/8_4_0/core/org/apache/lucene/search/PrefixQuery.html) out of the last term in the query string.
Similar to [match](#match), but creates a [prefix query](https://lucene.apache.org/core/{{site.lucene_version}}/core/org/apache/lucene/search/PrefixQuery.html) out of the last term in the query string.
```json
GET _search
@@ -164,7 +164,7 @@ GET _search
## Match phrase
Creates a [phrase query](https://lucene.apache.org/core/8_4_0/core/org/apache/lucene/search/PhraseQuery.html) that matches a sequence of terms.
Creates a [phrase query](https://lucene.apache.org/core/{{site.lucene_version}}/core/org/apache/lucene/search/PhraseQuery.html) that matches a sequence of terms.
```json
GET _search
@@ -198,7 +198,7 @@ GET _search
## Match phrase prefix
Similar to [match phrase](#match-phrase), but creates a [prefix query](https://lucene.apache.org/core/8_4_0/core/org/apache/lucene/search/PrefixQuery.html) out of the last term in the query string.
Similar to [match phrase](#match-phrase), but creates a [prefix query](https://lucene.apache.org/core/{{site.lucene_version}}/core/org/apache/lucene/search/PrefixQuery.html) out of the last term in the query string.
```json
GET _search
@@ -410,7 +410,7 @@ Option | Valid values | Description
`allow_leading_wildcard` | Boolean | Whether `*` and `?` are allowed as the first character of a search term. The default is true.
`analyze_wildcard` | Boolean | Whether OpenSearch should attempt to analyze wildcard terms. Some analyzers do a poor job at this task, so the default is false.
`analyzer` | `standard, simple, whitespace, stop, keyword, pattern, <language>, fingerprint` | The analyzer you want to use for the query. Different analyzers have different character filters, tokenizers, and token filters. The `stop` analyzer, for example, removes stop words (e.g. "an," "but," "this") from the query string.
`auto_generate_synonyms_phrase_query` | Boolean | A value of true (default) automatically generates [phrase queries](https://lucene.apache.org/core/8_4_0/core/org/apache/lucene/search/PhraseQuery.html) for multi-term synonyms. For example, if you have the synonym `"ba, batting average"` and search for "ba," OpenSearch searches for `ba OR "batting average"` (if this option is true) or `ba OR (batting AND average)` (if this option is false).
`auto_generate_synonyms_phrase_query` | Boolean | A value of true (default) automatically generates [phrase queries](https://lucene.apache.org/core/{{site.lucene_version}}/core/org/apache/lucene/search/PhraseQuery.html) for multi-term synonyms. For example, if you have the synonym `"ba, batting average"` and search for "ba," OpenSearch searches for `ba OR "batting average"` (if this option is true) or `ba OR (batting AND average)` (if this option is false).
`boost` | Floating-point | Boosts the clause by the given multiplier. Useful for weighing clauses in compound queries. The default is 1.0.
`cutoff_frequency` | Between `0.0` and `1.0` or a positive integer | This value lets you define high and low frequency terms based on number of occurrences in the index. Numbers between 0 and 1 are treated as a percentage. For example, 0.10 is 10%. This value means that if a word occurs within the search field in more than 10% of the documents on the shard, OpenSearch considers the word "high frequency" and deemphasizes it when calculating search score.<br /><br />Because this setting is *per shard*, testing its impact on search results can be challenging unless a cluster has many documents.
`enable_position_increments` | Boolean | When true, result queries are aware of position increments. This setting is useful when the removal of stop words leaves an unwanted "gap" between terms. The default is true.
@@ -420,7 +420,7 @@ Option | Valid values | Description
`fuzzy_transpositions` | Boolean | Setting `fuzzy_transpositions` to true (default) adds swaps of adjacent characters to the insert, delete, and substitute operations of the `fuzziness` option. For example, the distance between `wind` and `wnid` is 1 if `fuzzy_transpositions` is true (swap "n" and "i") and 2 if it is false (delete "n", insert "n"). <br /><br />If `fuzzy_transpositions` is false, `rewind` and `wnid` have the same distance (2) from `wind`, despite the more human-centric opinion that `wnid` is an obvious typo. The default is a good choice for most use cases.
`lenient` | Boolean | Setting `lenient` to true lets you ignore data type mismatches between the query and the document field. For example, a query string of "8.2" could match a field of type `float`. The default is false.
`low_freq_operator` | `and, or` | The operator for low-frequency terms. The default is `or`. See [Common terms](#common-terms) queries and `operator` in this table.
`max_determinized_states` | Positive integer | The maximum number of "[states](https://lucene.apache.org/core/8_4_0/core/org/apache/lucene/util/automaton/Operations.html#DEFAULT_MAX_DETERMINIZED_STATES)" (a measure of complexity) that Lucene can create for query strings that contain regular expressions (e.g. `"query": "/wind.+?/"`). Larger numbers allow for queries that use more memory. The default is 10,000.
`max_determinized_states` | Positive integer | The maximum number of "[states](https://lucene.apache.org/core/{{site.lucene_version}}/core/org/apache/lucene/util/automaton/Operations.html#DEFAULT_MAX_DETERMINIZED_STATES)" (a measure of complexity) that Lucene can create for query strings that contain regular expressions (e.g. `"query": "/wind.+?/"`). Larger numbers allow for queries that use more memory. The default is 10,000.
`max_expansions` | Positive integer | Fuzzy queries "expand to" a number of matching terms that are within the distance specified in `fuzziness`. Then OpenSearch tries to match those terms against its indices. `max_expansions` specifies the maximum number of terms that the fuzzy query expands to. The default is 50.
`minimum_should_match` | Positive or negative integer, positive or negative percentage, combination | If the query string contains multiple search terms and you used the `or` operator, the number of terms that need to match for the document to be considered a match. For example, if `minimum_should_match` is 2, "wind often rising" does not match "The Wind Rises." If `minimum_should_match` is 1, it matches. This option also has `low_freq` and `high_freq` properties for [Common terms](#common-terms) queries.
`operator` | `or, and` | If the query string contains multiple search terms, whether all terms need to match (`and`) or only one term needs to match (`or`) for a document to be considered a match.
@@ -428,7 +428,7 @@ Option | Valid values | Description
`prefix_length` | `0` (default) or a positive integer | The number of leading characters that are not considered in fuzziness.
`quote_field_suffix` | String | This option lets you search different fields depending on whether terms are wrapped in quotes. For example, if `quote_field_suffix` is `".exact"` and you search for `"lightly"` (in quotes) in the `title` field, OpenSearch searches the `title.exact` field. This second field might use a different type (e.g. `keyword` rather than `text`) or a different analyzer. The default is null.
`rewrite` | `constant_score, scoring_boolean, constant_score_boolean, top_terms_N, top_terms_boost_N, top_terms_blended_freqs_N` | Determines how OpenSearch rewrites and scores multi-term queries. The default is `constant_score`.
`slop` | `0` (default) or a positive integer | Controls the degree to which words in a query can be misordered and still be considered a match. From the [Lucene documentation](https://lucene.apache.org/core/8_4_0/core/org/apache/lucene/search/PhraseQuery.html#getSlop--): "The number of other words permitted between words in query phrase. For example, to switch the order of two words requires two moves (the first move places the words atop one another), so to permit re-orderings of phrases, the slop must be at least two. A value of zero requires an exact match."
`slop` | `0` (default) or a positive integer | Controls the degree to which words in a query can be misordered and still be considered a match. From the [Lucene documentation](https://lucene.apache.org/core/{{site.lucene_version}}/core/org/apache/lucene/search/PhraseQuery.html#getSlop--): "The number of other words permitted between words in query phrase. For example, to switch the order of two words requires two moves (the first move places the words atop one another), so to permit re-orderings of phrases, the slop must be at least two. A value of zero requires an exact match."
`tie_breaker` | `0.0` (default) to `1.0` | Changes the way OpenSearch scores searches. For example, a `type` of `best_fields` typically uses the highest score from any one field. If you specify a `tie_breaker` value between 0.0 and 1.0, the score changes to highest score + `tie_breaker` * score for all other matching fields. If you specify a value of 1.0, OpenSearch adds together the scores for all matching fields (effectively defeating the purpose of `best_fields`).
`time_zone` | UTC offset | The time zone to use (e.g. `-08:00`) if the query string contains a date range (e.g. `"query": "wind rises release_date[2012-01-01 TO 2014-01-01]"`). The default is `UTC`.
`type` | `best_fields, most_fields, cross-fields, phrase, phrase_prefix` | Determines how OpenSearch executes the query and scores the results. The default is `best_fields`.
+3
View File
@@ -5,8 +5,11 @@ nav_order: 27
has_children: true
redirect_from:
- /opensearch/query-dsl/
- /docs/opensearch/query-dsl/
---
{%- comment -%}The `/docs/opensearch/query-dsl/` redirect is specifically to support the UI links in OpenSearch Dashboards 1.0.0.{%- endcomment -%}
# Query DSL
While you can use HTTP request parameters to perform simple searches, you can also use the OpenSearch query domain-specific language (DSL), which provides a wider range of search options. The query DSL uses the HTTP request body, so you can more easily customize your queries to get the exact results that you want.
+6 -4
View File
@@ -430,7 +430,7 @@ Wildcard queries tend to be slow because they need to iterate over a lot of term
## Regex
Use the `regex` query to search for terms that match a regular expression.
Use the `regexp` query to search for terms that match a regular expression.
This regular expression matches any single uppercase or lowercase letter:
@@ -439,12 +439,14 @@ GET shakespeare/_search
{
"query": {
"regexp": {
"play_name": "H[a-zA-Z]+mlet"
"play_name": "[a-zA-Z]amlet"
}
}
}
```
Regular expressions are applied to the terms in the field and not the entire value of the field.
A few important notes:
The efficiency of your regular expression depends a lot on the patterns you write. Make sure that you write `regex` queries with either a prefix or suffix to improve performance.
- Regular expressions are applied to the terms in the field (i.e. tokens), not the entire field.
- Regular expressions use the Lucene syntax, which differs from more standardized implementations. Test thoroughly to ensure that you receive the results you expect. To learn more, see [the Lucene documentation](https://lucene.apache.org/core/{{site.lucene_version}}/core/index.html).
- `regexp` queries can be expensive operations and require the `search.allow_expensive_queries` setting to be set to `true`. Before making frequent `regexp` queries, test their impact on cluster performance and examine alternative queries for achieving similar results.
+103
View File
@@ -0,0 +1,103 @@
---
layout: default
title: Count
parent: REST API reference
nav_order: 150
---
# Count
Introduced 1.0
{: .label .label-purple }
The count API gives you quick access to the number of documents that match a query.
You can also use it to check the document count of an index, data stream, or cluster.
## Example
To see the number of documents that match a query:
```json
GET opensearch_dashboards_sample_data_logs/_count
{
"query": {
"term": {
"response": "200"
}
}
}
```
The following call to the search API produces equivalent results:
```json
GET opensearch_dashboards_sample_data_logs/_search
{
"query": {
"term": {
"response": "200"
}
},
"size": 0,
"track_total_hits": true
}
```
To see the number of documents in an index:
```json
GET opensearch_dashboards_sample_data_logs/_count
```
To check for the number of documents in a [data stream]({{site.url}}{{site.baseurl}}/opensearch/data-streams/), replace the index name with the data stream name.
To see the number of documents in your cluster:
```json
GET _count
```
Alternatively, you could use the [cat indices]({{site.url}}{{site.baseurl}}/opensearch/rest-api/cat/cat-indices/) and [cat count]({{site.url}}{{site.baseurl}}/opensearch/rest-api/cat/cat-count/) APIs to see the number of documents per index or data stream.
{: .note }
## Path and HTTP methods
```
GET <target>/_count/<id>
POST <target>/_count/<id>
```
## URL parameters
All count parameters are optional.
Parameter | Type | Description
:--- | :--- | :---
`allow_no_indices` | Boolean | If false, the request returns an error if any wildcard expression or index alias targets any closed or missing indices. Default is false.
`analyzer` | String | The analyzer to use in the query string.
`analyze_wildcard` | Boolean | Specifies whether to analyze wildcard and prefix queries. Default is false.
`default_operator` | String | Indicates whether the default operator for a string query should be AND or OR. Default is OR.
`df` | String | The default field in case a field prefix is not provided in the query string.
`expand_wildcards` | String | Specifies the type of index that wildcard expressions can match. Supports comma-separated values. Valid values are `all` (match any index), `open` (match open, non-hidden indices), `closed` (match closed, non-hidden indices), `hidden` (match hidden indices), and `none` (deny wildcard expressions). Default is `open`.
`ignore_unavailable` | Boolean | Specifies whether to include missing or closed indices in the response. Default is false.
`lenient` | Boolean | Specifies whether OpenSearch should accept requests if queries have format errors (for example, querying a text field for an integer). Default is false.
`min_score` | Float | Include only documents with a minimum `_score` value in the result.
`routing` | String | Value used to route the operation to a specific shard.
`preference` | String | Specifies which shard or node OpenSearch should perform the count operation on.
`terminate_after` | Integer | The maximum number of documents OpenSearch should process before terminating the request.
## Response
```json
{
"count" : 14074,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
}
}
```
+11 -11
View File
@@ -83,9 +83,8 @@ index.number_of_routing_shards | The number of routing shards used to split an i
index.shard.check_on_startup | Whether the index's shards should be checked for corruption. Available options are `false` (do not check for corruption), `checksum` (check for physical corruption), and `true` (check for both physical and logical corruption). Default is `false`.
index.codec | The compression type to use to compress stored data. Available values are `best_compression` and `default`.
index.routing_partition_size | The number of shards a custom routing value can go to. Routing helps an imbalanced cluster by relocating values to a subset of shards rather than just a single shard. To enable, set this value to greater than 1 but less than `index.number_of_shards`. Default is 1.
index.soft_deletes_enabled | Whether to retain a history of operations performed on the index.
index.soft_deletes.retention_lease.period | The maximum amount of time to retain a shard's history of operations. Default is `12h`.
index.load_fixed_bitset_filters_eagerly | Indicates whether cached filters should be pre-loaded for nested queries. Available options are `true` and `false`. Default is `true`.
index.load_fixed_bitset_filters_eagerly | Whether OpenSearch should pre-load cached filters. Available options are `true` and `false`. Default is `true`.
index.hidden | Whether the index should be hidden. Hidden indices are not returned as part of queries that have wildcards. Available options are `true` and `false`. Default is `false`.
#### Dynamic index Settings
@@ -93,28 +92,29 @@ index.hidden | Whether the index should be hidden. Hidden indices are not return
Setting | Description
:--- | :---
index.number_of_replicas | The number of replica shards each primary shard should have. For example, if you have 4 primary shards and set `index.number_of_replicas` to 3, the index has 12 replica shards. Default is 1.
index.auto_expand_replicas | Automatically expands the number of replicas based on the amount of data nodes in the cluster. Specify a lower bound and upper limit (for example, 0-9), or `all` for the upper limit. For example, if you have 5 data nodes and set `index.auto_expand_replicas` to 0-3, then the cluster does not autoamtically add another replica shard. However, if you set this value to `0-all` and add 2 more nodes for a total of 7, the cluster will expand to now have 6 replica shards. Default is disabled.
index.auto_expand_replicas | Whether the cluster should automatically add replica shards based on the number of data nodes. Specify a lower bound and upper limit (for example, 0-9), or `all` for the upper limit. For example, if you have 5 data nodes and set `index.auto_expand_replicas` to 0-3, then the cluster does not autoamtically add another replica shard. However, if you set this value to `0-all` and add 2 more nodes for a total of 7, the cluster will expand to now have 6 replica shards. Default is disabled.
index.search.idle.after | Amount of time a shard should wait for a search or get request until it goes idle. Default is `30s`.
index.refresh_interval | How often the index should refresh, which publishes its most recent changes and makes them available for searching. Can be set to `-1` to disable refreshing. Default is `1s`.
index.max_result_window | The maximum value of `from` + `size` for searches to the index. `from` is the starting index to search from, and `size` is the amount of results to return. Default: 10000.
index.max_inner_result_window | The maximum value of `from` + `size` for inner hits definitions and top hits aggregations to the index. `from` is the starting index to search from, and `size` is the amount of top hits to return. Default is 100.
index.max_inner_result_window | Maximum value of `from` + `size` to return nested search hits and most relevant document aggregated during the query. `from` is the starting index to search from, and `size` is the amount of top hits to return. Default is 100.
index.max_rescore_window | The maximum value of `window_size` for rescore requests to the index. Rescore requests reorder the index's documents and return a new score, which can be more precise. Default is the same as index.max_inner_result_window or 10000 by default.
index.max_docvalue_fields_search | Maximum amount of `docvalue_fields` allowed in a query. Default is 100.
index.max_script_fields | Maximum amount of `script_fields` allowed in a query. Default is 32.
index.max_ngram_diff | Maximum allowed difference between `min_gram` and `max_gram` values for `NGramTokenizer` and `NGramTokenFilter`. Default is 1.
index.max_shingle_diff | Maximum allowed difference between `max_shingle_size` and `min_shingle_size` for the `shingle` token filter. Default is 3.
index.max_refresh_listeners | Maximum amount of refresh listeners available on the index's shards.
index.max_ngram_diff | Maximum difference between `min_gram` and `max_gram` values for `NGramTokenizer` and `NGramTokenFilter` fields. Default is 1.
index.max_shingle_diff | Maximum difference between `max_shingle_size` and `min_shingle_size` to feed into the `shingle` token filter. Default is 3.
index.max_refresh_listeners | Maximum amount of refresh listeners each shard is allowed to have.
index.analyze.max_token_count | Maximum amount of tokens that can return from the `_analyze` API operation. Default is 10000.
index.highlight.max_analyzed_offset | The maximum amount of characters that will be analyzed in a highlight request. Default is 1000000.
index.max_terms_count | The maximum amount of terms that can be used in a terms query. Default is 65536.
index.highlight.max_analyzed_offset | The amount of characters a highlight request can analyze. Default is 1000000.
index.max_terms_count | The maximum amount of terms a terms query can accept. Default is 65536.
index.max_regex_length | The maximum character length of regex that can be in a regexp query. Default is 1000.
index.query.default_field | A field or list of fields that OpenSearch uses in queries in case a field isn't specified in the parameters.
index.routing.allocation.enable | Specifies options for the index's shard allocation. Available options are `all` (allow shard allocation for all shards), `primaries` (allow shard allocation only for primary shards), `new_primaries` (allow shard allocaiton only for new primary shards), and `none` (do not allow shard allocation). Default is `all`.
index.routing.rebalance.enable | Enables shard rebalancing for the index. Available options are `all` (allow shard rebalancing for all shards), `primaries`, (allow shard rebalancing only for primary shards), `replicas` (allow shard rebalancing only for replicas), and `none` (do not allow shard rebalancing). Default is `all`.
index.routing.allocation.enable | Specifies options for the index’s shard allocation. Available options are all (allow allocation for all shards), primaries (allow allocation only for primary shards), new_primaries (allow allocation only for new primary shards), and none (do not allow allocation). Default is all.
index.routing.rebalance.enable | Enables shard rebalancing for the index. Available options are `all` (allow rebalancing for all shards), `primaries` (allow rebalancing only for primary shards), `replicas` (allow rebalancing only for replicas), and `none` (do not allow rebalancing). Default is `all`.
index.gc_deletes | Amount of time to retain a deleted document's version number. Default is `60s`.
index.default_pipeline | The default ingest node pipeline for the index. If the default pipeline is set and the pipeline does not exist, then index requests fail. The pipeline name `_none` specifies that the index does not have an ingest pipeline.
index.final_pipeline | The final ingest node pipeline for the index. If the final pipeline is set and the pipeline does not exist, then index requests fail. The pipeline name `_none` specifies that the index does not have an ingest pipeline.
### Mappings
Mappings define how a documents and its fields are stored and indexed. If you're just starting to build out your cluster and data, you may not know exactly how your data should be stored. In those cases, you can use dynamic mappings, which tell OpenSearch to dynamically add data and their fields. However, if you know exactly what types your data fall under and want to enforce that standard, then you can use explicit mappings.
+1 -1
View File
@@ -32,7 +32,7 @@ POST _bulk
```
POST _bulk
POST {index}/_bulk
POST <index>/_bulk
```
Specifying the index in the path means you don't need to include it in the [request body]({{site.url}}{{site.baseurl}}/opensearch/rest-api/document-apis/bulk/#request-body).
@@ -28,7 +28,7 @@ POST sample-index1/_delete_by_query
## Path and HTTP methods
```
POST <target>/_delete_by_query
POST <index>/_delete_by_query
```
## URL parameters
@@ -37,8 +37,8 @@ All URL parameters are optional.
Parameter | Type | Description
:--- | :--- | :--- | :---
&lt;index&gt; | String | Name of the data streams, indices, or aliases to delete from. Supports wildcards. If left blank, OpenSearch searches all indices.
allow_no_indices | Boolean | False indicates to OpenSearch the request should return an error if any wildcard expression or index alias targets only missing or closed indices. Default is true.
&lt;index&gt; | String | Name or list of the data streams, indices, or aliases to delete from. Supports wildcards. If left blank, OpenSearch searches all indices.
allow_no_indices | Boolean | Whether to ignore wildcards that don’t match any indices. Default is `true`.
analyzer | String | The analyzer to use in the query string.
analyze_wildcard | Boolean | Specifies whether to analyze wildcard and prefix queries. Default is false.
conflicts | String | Indicates to OpenSearch what should happen if the delete by query operation runs into a version conflict. Valid options are `abort` and `proceed`. Default is `abort`.
@@ -47,19 +47,18 @@ df | String | The default field in case a field prefix is not provided in the qu
expand_wildcards | String | Specifies the type of index that wildcard expressions can match. Supports comma-separated values. Valid values are `all` (match any index), `open` (match open, non-hidden indices), `closed` (match closed, non-hidden indices), `hidden` (match hidden indices), and `none` (deny wildcard expressions). Default is `open`.
from | Integer | The starting index to search from. Default is 0.
ignore_unavailable | Boolean | Specifies whether to include missing or closed indices in the response. Default is false.
lenient | Boolean | Specifies whether OpenSearch should ignore format-based query failures (for example, querying a text field for an integer). Default is false.
max_docs | Integer | Maximum amount of documents the operation should process. Default is all documents.
preference | String | Specifies the shard or node OpenSearch should perform the operation on.
q | String | Query in the Lucene query string syntax.
request_cache | Boolean | Specifies whether OpenSearch should use the request cache for the request. Default is whether it's enabled in the index's settings.
refresh | Boolean | Specifies whether OpenSearch should refresh all of the shards involved in the delete request once the operation finishes. Default is false.
lenient | Boolean | Specifies whether OpenSearch should accept requests if queries have format errors (for example, querying a text field for an integer). Default is false.
max_docs | Integer | How many documents the delete by query operation should process at most. Default is all documents.
preference | String | Specifies which shard or node OpenSearch should perform the delete by query operation on.
q | String | Lucene query string's query.
request_cache | Boolean | Specifies whether OpenSearch should use the request cache. Default is whether it’s enabled in the index’s settings.
refresh | Boolean | If true, OpenSearch refreshes shards to make the delete by query operation available to search results. Valid options are `true`, `false`, and `wait_for`, which tells OpenSearch to wait for a refresh before executing the operation. Default is `false`.
requests_per_second | Integer | Specifies the request's throttling in sub-requests per second. Default is -1, which means no throttling.
routing | String | Value used to route the operation to a specific shard.
scroll | Time | Amount of time the search context should be open.
scroll_size | Integer | Size of the scroll request of the operation. Default is 1000.
scroll_size | Integer | Size of the operation's scroll requests. Default is 1000.
search_type | String | Whether OpenSearch should use global term and document frequencies calculating revelance scores. Valid choices are `query_then_fetch` and `dfs_query_then_fetch`. `query_then_fetch` scores documents using local term and document frequencies for the shard. It’s usually faster but less accurate. `dfs_query_then_fetch` scores documents using global term and document frequencies across all shards. It’s usually slower but more accurate. Default is `query_then_fetch`.
search_timeout | Time | Amount of time until timeout for the search request. Default is no timeout.
slices | Integer | Number of sub-tasks OpenSearch should divide this task into. Default is 1, which means OpenSearch should not divide this task.
search_timeout | Time | How long to wait until OpenSearch deems the request timed out. Default is no timeout.
sort | String | A comma-separated list of &lt;field&gt; : &lt;direction&gt; pairs to sort by.
_source | String | Specifies whether to include the `_source` field in the response.
_source_excludes | String | A comma-separated list of source fields to exclude from the response.
@@ -21,20 +21,20 @@ DELETE /sample-index1/_doc/1
## Path and HTTP methods
```
DELETE /<index-name>/_doc/<_id>
DELETE /<index>/_doc/<_id>
```
## URL parameters
Parameter | Type | Description | Required
:--- | :--- | :--- | :---
&lt;index-name&gt; | String | The index to delete from. | Yes
&lt;index&gt; | String | The index to delete from. | Yes
&lt;_id&gt; | String | The ID of the document to delete. | Yes
if_seq_no | Integer | Only perform the delete operation if the document's version number matches the specified number. | No
if_primary_term | Integer | Only perform the delete operation if the document has the specified primary term. | No
refresh | Enum | If true, OpenSearch refreshes shards to make the operation visible to searching. Valid options are `true`, `false`, and `wait_for`, which tells OpenSearch to wait for a refresh before executing the operation. Default is false. | No
refresh | Enum | If true, OpenSearch refreshes shards to make the delete operation available to search results. Valid options are `true`, `false`, and `wait_for`, which tells OpenSearch to wait for a refresh before executing the operation. Default is `false`. | No
routing | String | Value used to route the operation to a specific shard. | No
timeout | Time | How long to wait for a response from the cluster. | No
timeout | Time | How long to wait for a response from the cluster. Default is `1m`. | No
version | Integer | The version of the document to delete, which must match the last updated version of the document. | No
version_type | Enum | Retrieves a specifically typed document. Available options are `external` (retrieve the document if the specified version number is greater than the document's current version) and `external_gte` (retrieve the document if the specified version number is greater than or equal to the document's current version). For example, to delete version 3 of a document, use `/_doc/1?version=3&version_type=external`. | No
wait_for_active_shards | String | The number of active shards that must be available before OpenSearch processes the delete request. Default is 1 (only the primary shard). Set to `all` or a positive integer. Values greater than 1 require replicas. For example, if you specify a value of 3, the index must have two replicas distributed across two additional nodes for the operation to succeed. | No

Some files were not shown because too many files have changed in this diff Show More