minor fix

Adds an Agents and Ingestion tools section

Signed-off-by: Kyle Davis <kyledvs@amazon.com>

.gitignore

@@ -3,3 +3,4 @@ _site
 .jekyll-metadata
 .DS_Store
 Gemfile.lock
+.idea

NOTICE

@@ -1 +1 @@
-Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
+Copyright 2021 OpenSearch contributors.

README.md

@@ -273,4 +273,4 @@ This project is licensed under the Apache-2.0 License.
 
 ## Copyright
 
-Copyright Amazon.com, Inc. or its affiliates. All rights reserved.
+Copyright 2021 OpenSearch contributors.

_layouts/default.html

@@ -73,8 +73,8 @@ layout: table_wrappers
       {% endif %}
     </nav>
     <footer class="site-footer">
-      <p class="text-small text-grey-dk-100">See a problem? Submit <a href="https://github.com/opensearch-project/documentation-website/issues">issues</a> or <a href="https://github.com/opensearch-project/documentation-website/edit/main/{{ page.path }}">edit this page</a> on GitHub.</p>
-      <p class="text-small text-grey-dk-100 mb-0">© Amazon Web Services, Inc. or its affiliates. All rights reserved.</p>
+      <p class="text-small text-grey-dk-100">See a problem? Submit <a href="https://github.com/opensearch-project/documentation-website/issues">issues</a> or <a href="https://github.com/opensearch-project/documentation-website/edit/main/{{ page.path }}">edit this page</a> on <a href="https://github.com/opensearch-project/documentation-website/">GitHub</a>.</p>
+      <p class="text-small text-grey-dk-100 mb-0">© 2021 OpenSearch contributors. This documentation is under the Apache License 2.0.</p>
     </footer>
   </div>
   <div class="main" id="top">

docs/ad/api.md

@@ -28,7 +28,7 @@ This command creates a detector named `http_requests` that finds anomalies based
 #### Request
 
 ```json
-POST _opensearch/_anomaly_detection/detectors
+POST _plugins/_anomaly_detection/detectors
 {
   "name": "test-detector",
   "description": "Test detector",
@@ -143,7 +143,7 @@ To set a category field for high cardinality:
 #### Request
 
 ```json
-POST _opensearch/_anomaly_detection/detectors
+POST _plugins/_anomaly_detection/detectors
 {
   "name": "Host OK Rate Detector",
   "description": "ok rate",
@@ -243,7 +243,7 @@ To create a historical detector:
 #### Request
 
 ```json
-POST _opensearch/_anomaly_detection/detectors
+POST _plugins/_anomaly_detection/detectors
 {
   "name": "test1",
   "description": "test historical detector",
@@ -312,7 +312,7 @@ Passes a date range to the anomaly detector to return any anomalies within that
 #### Request
 
 ```json
-POST _opensearch/_anomaly_detection/detectors/<detectorId>/_preview
+POST _plugins/_anomaly_detection/detectors/<detectorId>/_preview
 {
   "period_start": 1588838250000,
   "period_end": 1589443050000
@@ -452,7 +452,7 @@ Starts a real-time or historical anomaly detector job.
 #### Request
 
 ```json
-POST _opensearch/_anomaly_detection/detectors/<detectorId>/_start
+POST _plugins/_anomaly_detection/detectors/<detectorId>/_start
 ```
 
 #### Sample response
@@ -476,7 +476,7 @@ Stops a real-time or historical anomaly detector job.
 #### Request
 
 ```json
-POST _opensearch/_anomaly_detection/detectors/<detectorId>/_stop
+POST _plugins/_anomaly_detection/detectors/<detectorId>/_stop
 ```
 
 #### Sample response
@@ -494,8 +494,8 @@ Returns all results for a search query.
 #### Request
 
 ```json
-GET _opensearch/_anomaly_detection/detectors/results/_search
-POST _opensearch/_anomaly_detection/detectors/results/_search
+GET _plugins/_anomaly_detection/detectors/results/_search
+POST _plugins/_anomaly_detection/detectors/results/_search
 
 {
   "query": {
@@ -596,7 +596,7 @@ To see an ordered set of anomaly records for an entity with an anomaly within a
 #### Request
 
 ```json
-POST _opensearch/_anomaly_detection/detectors/results/_search
+POST _plugins/_anomaly_detection/detectors/results/_search
 {
   "query": {
     "bool": {
@@ -782,7 +782,7 @@ To get the latest task:
 #### Request
 
 ```json
-GET _opensearch/_anomaly_detection/detectors/<detector_id>?task=true
+GET _plugins/_anomaly_detection/detectors/<detector_id>?task=true
 ```
 
 To query the anomaly results with `task_id`:
@@ -790,7 +790,7 @@ To query the anomaly results with `task_id`:
 #### Request
 
 ```json
-GET _opensearch/_anomaly_detection/detectors/results/_search
+GET _plugins/_anomaly_detection/detectors/results/_search
 {
   "query": {
     "term": {
@@ -940,7 +940,7 @@ To delete a detector, you need to first stop the detector.
 #### Request
 
 ```json
-DELETE _opensearch/_anomaly_detection/detectors/<detectorId>
+DELETE _plugins/_anomaly_detection/detectors/<detectorId>
 ```
 
 
@@ -975,7 +975,7 @@ To update a detector, you need to first stop the detector.
 #### Request
 
 ```json
-PUT _opensearch/_anomaly_detection/detectors/<detectorId>
+PUT _plugins/_anomaly_detection/detectors/<detectorId>
 {
   "name": "test-detector",
   "description": "Test detector",
@@ -1091,7 +1091,7 @@ To update a historical detector:
 #### Request
 
 ```json
-PUT _opensearch/_anomaly_detection/detectors/<detectorId>
+PUT _plugins/_anomaly_detection/detectors/<detectorId>
 {
   "name": "test1",
   "description": "test historical detector",
@@ -1145,7 +1145,7 @@ Returns all information about a detector based on the `detector_id`.
 #### Request
 
 ```json
-GET _opensearch/_anomaly_detection/detectors/<detectorId>
+GET _plugins/_anomaly_detection/detectors/<detectorId>
 ```
 
 #### Sample response
@@ -1215,7 +1215,7 @@ Use `job=true` to get anomaly detection job information.
 #### Request
 
 ```json
-GET _opensearch/_anomaly_detection/detectors/<detectorId>?job=true
+GET _plugins/_anomaly_detection/detectors/<detectorId>?job=true
 ```
 
 #### Sample response
@@ -1304,7 +1304,7 @@ Use `task=true` to get historical detector task information.
 #### Request
 
 ```json
-GET _opensearch/_anomaly_detection/detectors/<detectorId>?task=true
+GET _plugins/_anomaly_detection/detectors/<detectorId>?task=true
 ```
 
 #### Sample response
@@ -1491,8 +1491,8 @@ Returns all anomaly detectors for a search query.
 #### Request
 
 ```json
-GET _opensearch/_anomaly_detection/detectors/_search
-POST _opensearch/_anomaly_detection/detectors/_search
+GET _plugins/_anomaly_detection/detectors/_search
+POST _plugins/_anomaly_detection/detectors/_search
 
 Sample Input:
 {
@@ -1597,10 +1597,10 @@ Provides information about how the plugin is performing.
 #### Request
 
 ```json
-GET _opensearch/_anomaly_detection/stats
-GET _opensearch/_anomaly_detection/<nodeId>/stats
-GET _opensearch/_anomaly_detection/<nodeId>/stats/<stat>
-GET _opensearch/_anomaly_detection/stats/<stat>
+GET _plugins/_anomaly_detection/stats
+GET _plugins/_anomaly_detection/<nodeId>/stats
+GET _plugins/_anomaly_detection/<nodeId>/stats/<stat>
+GET _plugins/_anomaly_detection/stats/<stat>
 ```
 
 #### Sample response
@@ -1697,7 +1697,7 @@ Create a monitor to set up alerts for the detector.
 #### Request
 
 ```json
-POST _opensearch/_alerting/monitors
+POST _plugins/_alerting/monitors
 {
   "type": "monitor",
   "name": "test-monitor",
@@ -1919,23 +1919,23 @@ It also helps track the initialization percentage, the required shingles, and th
 #### Request
 
 ```json
-GET _opensearch/_anomaly_detection/detectors/<detectorId>/_profile/
-GET _opensearch/_anomaly_detection/detectors/<detectorId>/_profile?_all=true
-GET _opensearch/_anomaly_detection/detectors/<detectorId>/_profile/<type>
-GET /_opensearch/_anomaly_detection/detectors/<detectorId>/_profile/<type1>,<type2>
+GET _plugins/_anomaly_detection/detectors/<detectorId>/_profile/
+GET _plugins/_anomaly_detection/detectors/<detectorId>/_profile?_all=true
+GET _plugins/_anomaly_detection/detectors/<detectorId>/_profile/<type>
+GET /_plugins/_anomaly_detection/detectors/<detectorId>/_profile/<type1>,<type2>
 ```
 
 #### Sample Responses
 
 ```json
-GET _opensearch/_anomaly_detection/detectors/<detectorId>/_profile
+GET _plugins/_anomaly_detection/detectors/<detectorId>/_profile
 
 {
     "state":"DISABLED",
     "error":"Stopped detector: AD models memory usage exceeds our limit."
 }
 
-GET _opensearch/_anomaly_detection/detectors/<detectorId>/_profile?_all=true&pretty
+GET _plugins/_anomaly_detection/detectors/<detectorId>/_profile?_all=true&pretty
 
 {
   "state": "RUNNING",
@@ -1970,7 +1970,7 @@ GET _opensearch/_anomaly_detection/detectors/<detectorId>/_profile?_all=true&pre
   }
 }
 
-GET _opensearch/_anomaly_detection/detectors/<detectorId>/_profile/total_size_in_bytes
+GET _plugins/_anomaly_detection/detectors/<detectorId>/_profile/total_size_in_bytes
 
 {
   "total_size_in_bytes" : 13369344
@@ -1984,7 +1984,7 @@ You can use this data to estimate how much memory is required for anomaly detect
 #### Request
 
 ```json
-GET /_opensearch/_anomaly_detection/detectors/<detectorId>/_profile?_all=true&pretty
+GET /_plugins/_anomaly_detection/detectors/<detectorId>/_profile?_all=true&pretty
 
 {
   "state": "RUNNING",
@@ -2043,7 +2043,7 @@ If there are no anomaly results for an entity, either the entity doesn't have an
 #### Request
 
 ```json
-GET /_opensearch/_anomaly_detection/detectors/<detectorId>/_profile?_all=true&entity=i-00f28ec1eb8997686
+GET /_plugins/_anomaly_detection/detectors/<detectorId>/_profile?_all=true&entity=i-00f28ec1eb8997686
 {
   "category_field": "host",
   "value": "i-00f28ec1eb8997686",
@@ -2067,8 +2067,8 @@ For a historical detector, specify `_all` or `ad_task` to see information about
 #### Request
 
 ```json
-GET _opensearch/_anomaly_detection/detectors/<detectorId>/_profile?_all
-GET _opensearch/_anomaly_detection/detectors/<detectorId>/_profile/ad_task
+GET _plugins/_anomaly_detection/detectors/<detectorId>/_profile?_all
+GET _plugins/_anomaly_detection/detectors/<detectorId>/_profile/ad_task
 ```
 
 #### Sample Responses

docs/ad/index.md

@@ -52,7 +52,7 @@ A feature is the field in your index that you want to check for anomalies. A det
 
 For example, if you choose `min()`, the detector focuses on finding anomalies based on the minimum values of your feature. If you choose `average()`, the detector finds anomalies based on the average values of your feature.
 
-A multi-feature model correlates anomalies across all its features. The [curse of dimensionality](https://en.wikipedia.org/wiki/Curse_of_dimensionality) makes it less likely for multi-feature models to identify smaller anomalies as compared to a single-feature model. Adding more features might negatively impact the [precision and recall](https://en.wikipedia.org/wiki/Precision_and_recall) of a model. A higher proportion of noise in your data might further amplify this negative impact. Selecting the optimal feature set is usually an iterative process. We recommend experimenting with a historical detector with different feature sets and checking the precision before moving on to real-time detectors. By default, the maximum number of features for a detector is 5. You can adjust this limit with the `opendistro.anomaly_detection.max_anomaly_features` setting.
+A multi-feature model correlates anomalies across all its features. The [curse of dimensionality](https://en.wikipedia.org/wiki/Curse_of_dimensionality) makes it less likely for multi-feature models to identify smaller anomalies as compared to a single-feature model. Adding more features might negatively impact the [precision and recall](https://en.wikipedia.org/wiki/Precision_and_recall) of a model. A higher proportion of noise in your data might further amplify this negative impact. Selecting the optimal feature set is usually an iterative process. We recommend experimenting with a historical detector with different feature sets and checking the precision before moving on to real-time detectors. By default, the maximum number of features for a detector is 5. You can adjust this limit with the `plugins.anomaly_detection.max_anomaly_features` setting.
 {: .note }
 
 1. On the **Model configuration** page, enter the **Feature name**.

docs/ad/security.md

@@ -35,7 +35,7 @@ Next, enable the following setting:
 PUT _cluster/settings
 {
   "transient": {
-    "opendistro.anomaly_detection.filter_by_backend_roles": "true"
+    "plugins.anomaly_detection.filter_by_backend_roles": "true"
   }
 }
 ```
@@ -46,7 +46,7 @@ For example, consider two users: `alice` and `bob`.
 `alice` has an analyst backend role:
 
 ```json
-PUT _opensearch/_security/api/internalusers/alice
+PUT _plugins/_security/api/internalusers/alice
 {
   "password": "alice",
   "backend_roles": [
@@ -59,7 +59,7 @@ PUT _opensearch/_security/api/internalusers/alice
 `bob` has a human-resources backend role:
 
 ```json
-PUT _opensearch/_security/api/internalusers/bob
+PUT _plugins/_security/api/internalusers/bob
 {
   "password": "bob",
   "backend_roles": [
@@ -72,7 +72,7 @@ PUT _opensearch/_security/api/internalusers/bob
 Both `alice` and `bob` have full access to anomaly detection:
 
 ```json
-PUT _opensearch/_security/api/rolesmapping/anomaly_full_access
+PUT _plugins/_security/api/rolesmapping/anomaly_full_access
 {
   "backend_roles": [],
   "hosts": [],

docs/ad/settings.md

@@ -17,26 +17,26 @@ For example, to update the retention period of the result index:
 PUT _cluster/settings
 {
   "transient": {
-    "opendistro.anomaly_detection.ad_result_history_retention_period": "5m"
+    "plugins.anomaly_detection.ad_result_history_retention_period": "5m"
   }
 }
 ```
 
 Setting | Default | Description
 :--- | :--- | :---
-`opendistro.anomaly_detection.enabled` | True | Whether the anomaly detection plugin is enabled or not. If disabled, all detectors immediately stop running.
-`opendistro.anomaly_detection.max_anomaly_detectors` | 1,000 | The maximum number of non-high cardinality detectors (no category field) users can create.
-`opendistro.anomaly_detection.max_multi_entity_anomaly_detectors` | 10 | The maximum number of high cardinality detectors (with category field) in a cluster.
-`opendistro.anomaly_detection.max_anomaly_features` | 5 | The maximum number of features for a detector.
-`opendistro.anomaly_detection.ad_result_history_rollover_period` | 12h | How often the rollover condition is checked. If `true`, the plugin rolls over the result index to a new index.
-`opendistro.anomaly_detection.ad_result_history_max_docs` | 250000000 | The maximum number of documents in one result index. The plugin only counts refreshed documents in the primary shards.
-`opendistro.anomaly_detection.ad_result_history_retention_period` | 30d | The maximum age of the result index.  If its age exceeds the threshold, the plugin deletes the rolled over result index. If the cluster has only one result index, the plugin keeps the index even if it's older than its configured retention period.
-`opendistro.anomaly_detection.max_entities_per_query` | 1,000 | The maximum unique values per detection interval for high cardinality detectors. By default, if the category field has more than 1,000 unique values in a detector interval, the plugin selects the top 1,000 values and orders them by `doc_count`.
-`opendistro.anomaly_detection.max_entities_for_preview` | 30 | The maximum unique category field values displayed with the preview operation for high cardinality detectors. If the category field has more than 30 unique values, the plugin selects the top 30 values and orders them by `doc_count`.
-`opendistro.anomaly_detection.max_primary_shards` | 10 | The maximum number of primary shards an anomaly detection index can have.
-`opendistro.anomaly_detection.filter_by_backend_roles` | False | When you enable the security plugin and set this to `true`, the plugin filters results based on the user's backend role(s).
-`opendistro.anomaly_detection.max_cache_miss_handling_per_second` | 100 | High cardinality detectors use a cache to store active models. In the event of a cache miss, the cache gets the models from the model checkpoint index. Use this setting to limit the rate of fetching models. Because the thread pool for a GET operation has a queue of 1,000, we recommend setting this value below 1,000.
-`opendistro.anomaly_detection.max_batch_task_per_node` | 2 | Starting a historical detector triggers a batch task. This setting is the number of batch tasks that you can run per data node. You can tune this setting from 1 to 1000. If the data nodes can't support all batch tasks and you're not sure if the data nodes are capable of running more historical detectors, add more data nodes instead of changing this setting to a higher value.
-`opendistro.anomaly_detection.max_old_ad_task_docs_per_detector` | 10 | You can run the same historical detector many times. For each run, the anomaly detection plugin creates a new task. This setting is the number of previous tasks the plugin keeps. Set this value to at least 1 to track its last run. You can keep a maximum of 1,000 old tasks to avoid overwhelming the cluster.
-`opendistro.anomaly_detection.batch_task_piece_size` | 1000 | The date range for a historical task is split into smaller pieces and the anomaly detection plugin runs the task piece by piece. Each piece contains 1,000 detection intervals by default. For example, if detector interval is 1 minute and one piece is 1000 minutes, the feature data is queried every 1,000 minutes. You can change this setting from 1 to 10,000.
-`opendistro.anomaly_detection.batch_task_piece_interval_seconds` | 5 | Add a time interval between historical detector tasks. This interval prevents the task from consuming too much of the available resources and starving other operations like search and bulk index. You can change this setting from 1 to 600 seconds.
+`plugins.anomaly_detection.enabled` | True | Whether the anomaly detection plugin is enabled or not. If disabled, all detectors immediately stop running.
+`plugins.anomaly_detection.max_anomaly_detectors` | 1,000 | The maximum number of non-high cardinality detectors (no category field) users can create.
+`plugins.anomaly_detection.max_multi_entity_anomaly_detectors` | 10 | The maximum number of high cardinality detectors (with category field) in a cluster.
+`plugins.anomaly_detection.max_anomaly_features` | 5 | The maximum number of features for a detector.
+`plugins.anomaly_detection.ad_result_history_rollover_period` | 12h | How often the rollover condition is checked. If `true`, the plugin rolls over the result index to a new index.
+`plugins.anomaly_detection.ad_result_history_max_docs` | 250000000 | The maximum number of documents in one result index. The plugin only counts refreshed documents in the primary shards.
+`plugins.anomaly_detection.ad_result_history_retention_period` | 30d | The maximum age of the result index.  If its age exceeds the threshold, the plugin deletes the rolled over result index. If the cluster has only one result index, the plugin keeps the index even if it's older than its configured retention period.
+`plugins.anomaly_detection.max_entities_per_query` | 1,000 | The maximum unique values per detection interval for high cardinality detectors. By default, if the category field has more than 1,000 unique values in a detector interval, the plugin selects the top 1,000 values and orders them by `doc_count`.
+`plugins.anomaly_detection.max_entities_for_preview` | 30 | The maximum unique category field values displayed with the preview operation for high cardinality detectors. If the category field has more than 30 unique values, the plugin selects the top 30 values and orders them by `doc_count`.
+`plugins.anomaly_detection.max_primary_shards` | 10 | The maximum number of primary shards an anomaly detection index can have.
+`plugins.anomaly_detection.filter_by_backend_roles` | False | When you enable the security plugin and set this to `true`, the plugin filters results based on the user's backend role(s).
+`plugins.anomaly_detection.max_cache_miss_handling_per_second` | 100 | High cardinality detectors use a cache to store active models. In the event of a cache miss, the cache gets the models from the model checkpoint index. Use this setting to limit the rate of fetching models. Because the thread pool for a GET operation has a queue of 1,000, we recommend setting this value below 1,000.
+`plugins.anomaly_detection.max_batch_task_per_node` | 2 | Starting a historical detector triggers a batch task. This setting is the number of batch tasks that you can run per data node. You can tune this setting from 1 to 1000. If the data nodes can't support all batch tasks and you're not sure if the data nodes are capable of running more historical detectors, add more data nodes instead of changing this setting to a higher value.
+`plugins.anomaly_detection.max_old_ad_task_docs_per_detector` | 10 | You can run the same historical detector many times. For each run, the anomaly detection plugin creates a new task. This setting is the number of previous tasks the plugin keeps. Set this value to at least 1 to track its last run. You can keep a maximum of 1,000 old tasks to avoid overwhelming the cluster.
+`plugins.anomaly_detection.batch_task_piece_size` | 1000 | The date range for a historical task is split into smaller pieces and the anomaly detection plugin runs the task piece by piece. Each piece contains 1,000 detection intervals by default. For example, if detector interval is 1 minute and one piece is 1000 minutes, the feature data is queried every 1,000 minutes. You can change this setting from 1 to 10,000.
+`plugins.anomaly_detection.batch_task_piece_interval_seconds` | 5 | Add a time interval between historical detector tasks. This interval prevents the task from consuming too much of the available resources and starving other operations like search and bulk index. You can change this setting from 1 to 600 seconds.

docs/agents-and-ingestion-tools/index.md

@@ -0,0 +1,63 @@
+---
+layout: default
+title: Agents and Ingestion Tools
+nav_order: 100
+has_children: false
+has_toc: false
+---
+
+# Agents and Ingestion Tools
+
+Historically, there have been multiple popular agents and ingestion tools that work with Elasticsearch. Some of which are Beats, Logstash, Fluentd, FluentBit, and Open Telemetry. OpenSearch aims to continue to support a broad set of agents and ingestion tools, but not all have been tested or have explicitly added OpenSearch compatibility. To make getting started with OpenSearch easier, a [version value](https://github.com/opensearch-project/OpenSearch/issues/693) is being added to the OpenSearch YML. This will let you set OpenSearch 1.x clusters to report open source Elasticsearch 7.10.2. By reporting 7.10.2, the cluster will be able to connect with tools that do version checking. This is intended to be an intermediate solution while OpenSearch support is added to more tools.
+
+
+## Compatibility Matrices
+
+So far we have built compatibility matrices for OpenSearch, Elasticsearch OSS 7.x, Open Distro for Elasticsearch (ODFE) 1.x, Logstash OSS 7.x, and Beats OSS 7.x. **This page is a living document. If there are other versions or software you would like to add to the page please [submit a PR](https://github.com/opensearch-project/documentation-website/edit/main/docs/agents-and-ingestion-tools/index.md) and add yourself to the "Credits and Thanks you" section at the bottom of the page. We greatly appreciate the help!** 
+
+Note that if a cell in any of the matrices is *italicized* that means the value is theoretically what it should be based on other documentation or release notes, but it is still pending testing. Also note that for Logstash we have included a column for version 7.13.x with the [proposed OpenSearch output plugin](https://github.com/opensearch-project/OpenSearch/issues/820).
+
+### Compatibility Matrix for Logstash
+
+|	|Logstash OSS 7.x to 7.11.x	|Logstash OSS 7.12.x*	|Logstash 7.13.x without OpenSearch output plugin^	|Logstash 7.13.x with OpenSearch output plugin**^	|
+|---	|---	|---	|---	|---	|
+|Elasticsearch OSS v7.x to v7.9.x	|*Yes*	|*Yes*	|*No*	|*Yes*	|
+|Elasticsearch OSS v7.10.2^	|*Yes*	|*Yes*	|*No*	|*Yes*	|
+|ODFE OSS v1.x to 1.12	|*Yes*	|*Yes*	|*No*	|*Yes*	|
+|ODFE 1.13^	|*Yes*	|*Yes*	|*No*	|*Yes*	|
+|Amazon Elasticsearch Service 7.x to 7.9	|Yes	|Yes	|*No*	|*Yes*	|
+|Amazon Elasticsearch Service 7.10^	|Yes	|Yes	|*No*	|*Yes*	|
+|Amazon Elasticsearch Service 7.x to 7.9 with IAM	|[Yes with Amazon ES output plugin](https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-managedomains-logstash.html)	|[Yes with Amazon ES output plugin](https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-managedomains-logstash.html)	|No	|*Yes*	|
+|Amazon Elasticsearch Service 7.10 with IAM^	|[Yes with Amazon ES output plugin](https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-managedomains-logstash.html)	|[Yes with Amazon ES output plugin](https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-managedomains-logstash.html)	|No	|*Yes*	|
+|OpenSearch 1.0^	|[Yes via version setting](https://github.com/opensearch-project/OpenSearch/issues/693)	|[Yes via version setting](https://github.com/opensearch-project/OpenSearch/issues/693)	|*No*	|*Yes*	|
+
+\*Most current compatible version with Elasticsearch OSS\
+\*\*Planning to build\
+^ Most current version of software\
+*Italicized cells are based on documentation or release notes but are still pending testing to validate*
+
+### Compatibility Matrix for Beats
+
+|	|Beats OSS 7.x to 7.11.x*	|Beats OSS 7.12.x**	|Beats 7.13.x^	|
+|---	|---	|---	|---	|
+|Elasticsearch OSS v7.x to v7.9.x	|*Yes*	|*Yes*	|No	|
+|Elasticsearch OSS v7.10.2^	|*Yes*	|*Yes*	|No	|
+|ODFE OSS v1.x to 1.12	|*Yes*	|*Yes*	|No	|
+|ODFE 1.13^	|*Yes*	|*Yes*	|No	|
+|Amazon Elasticsearch Service 7.x to 7.9	|*Yes*	|*Yes*	|No	|
+|Amazon Elasticsearch Service 7.10^	|*Yes*	|*Yes*	|No	|
+|OpenSearch 1.0^	|[Yes via version setting](https://github.com/opensearch-project/OpenSearch/issues/693)	|[Yes via version setting](https://github.com/opensearch-project/OpenSearch/issues/693)	|No	|
+|Logstash OSS 7.x to 7.11.x*	|*Yes*	|*Yes*	|*Yes*	|
+|Logstash OSS 7.12.x**	|*Yes*	|*Yes*	|*Yes*	|
+|Logstash 7.13.x with OpenSearch output plugin	|*Yes*	|*Yes*	|*Yes*	|
+
+\*Beats OSS includes all Apache 2.0 Beats agents (Filebeat, Metricbeat, Auditbeat, Heartbeat, Winlogbeat, Packetbeat)\
+\*\*Most current compatible version with Elasticsearch OSS\
+^ Most current version of software\
+*Italicized cells are based on documentation or release notes but are still pending testing to validate*
+
+### Credits and Thank You for Contributing and Testing
+
+* [VijayanB](https://github.com/VijayanB)
+* [vamshin](https://github.com/vamshin)
+* [anirudha](https://github.com/anirudha)

docs/alerting/api.md

@@ -24,7 +24,7 @@ Use the alerting API to programmatically manage monitors and alerts.
 #### Request
 
 ```json
-POST _opensearch/_alerting/monitors
+POST _plugins/_alerting/monitors
 {
   "type": "monitor",
   "name": "test-monitor",
@@ -252,7 +252,7 @@ When you update a monitor, include the current version number as a parameter. Op
 #### Request
 
 ```json
-PUT _opensearch/_alerting/monitors/<monitor_id>
+PUT _plugins/_alerting/monitors/<monitor_id>
 {
   "type": "monitor",
   "name": "test-monitor",
@@ -372,7 +372,7 @@ PUT _opensearch/_alerting/monitors/<monitor_id>
 #### Request
 
 ```
-GET _opensearch/_alerting/monitors/<monitor_id>
+GET _plugins/_alerting/monitors/<monitor_id>
 ```
 
 #### Sample response
@@ -439,15 +439,15 @@ GET _opensearch/_alerting/monitors/<monitor_id>
 
 ## Monitor stats
 
-Returns statistics about the alerting feature. Use `_opensearch/_alerting/stats` to find node IDs and metrics. Then you can drill down using those values.
+Returns statistics about the alerting feature. Use `_plugins/_alerting/stats` to find node IDs and metrics. Then you can drill down using those values.
 
 #### Request
 
 ```json
-GET _opensearch/_alerting/stats
-GET _opensearch/_alerting/stats/<metric>
-GET _opensearch/_alerting/<node-id>/stats
-GET _opensearch/_alerting/<node-id>/stats/<metric>
+GET _plugins/_alerting/stats
+GET _plugins/_alerting/stats/<metric>
+GET _plugins/_alerting/<node-id>/stats
+GET _plugins/_alerting/<node-id>/stats/<metric>
 ```
 
 #### Sample response
@@ -631,7 +631,7 @@ GET _opensearch/_alerting/<node-id>/stats/<metric>
 #### Request
 
 ```
-DELETE _opensearch/_alerting/monitors/<monitor_id>
+DELETE _plugins/_alerting/monitors/<monitor_id>
 ```
 
 #### Sample response
@@ -662,7 +662,7 @@ DELETE _opensearch/_alerting/monitors/<monitor_id>
 #### Request
 
 ```json
-GET _opensearch/_alerting/monitors/_search
+GET _plugins/_alerting/monitors/_search
 {
   "query": {
     "match" : {
@@ -762,7 +762,7 @@ You can add the optional `?dryrun=true` parameter to the URL to show the results
 #### Request
 
 ```json
-POST _opensearch/_alerting/monitors/<monitor_id>/_execute
+POST _plugins/_alerting/monitors/<monitor_id>/_execute
 ```
 
 #### Sample response
@@ -793,7 +793,7 @@ Returns an array of all alerts.
 #### Request
 
 ```json
-GET _opensearch/_alerting/monitors/alerts
+GET _plugins/_alerting/monitors/alerts
 ```
 
 #### Response
@@ -862,7 +862,7 @@ GET _opensearch/_alerting/monitors/alerts
 #### Request
 
 ```json
-POST _opensearch/_alerting/monitors/<monitor-id>/_acknowledge/alerts
+POST _plugins/_alerting/monitors/<monitor-id>/_acknowledge/alerts
 {
   "alerts": ["eQURa3gBKo1jAh6qUo49"]
 }
@@ -886,7 +886,7 @@ POST _opensearch/_alerting/monitors/<monitor-id>/_acknowledge/alerts
 #### Requests
 
 ```json
-POST _opensearch/_alerting/destinations
+POST _plugins/_alerting/destinations
 {
   "name": "my-destination",
   "type": "slack",
@@ -895,7 +895,7 @@ POST _opensearch/_alerting/destinations
   }
 }
 
-POST _opensearch/_alerting/destinations
+POST _plugins/_alerting/destinations
 {
   "type": "custom_webhook",
   "name": "my-custom-destination",
@@ -939,7 +939,7 @@ POST _opensearch/_alerting/destinations
 #### Request
 
 ```json
-PUT _opensearch/_alerting/destinations/<destination-id>
+PUT _plugins/_alerting/destinations/<destination-id>
 {
   "name": "my-updated-destination",
   "type": "slack",
@@ -976,7 +976,7 @@ Retrieve one destination.
 #### Requests
 
 ```json
-GET _opensearch/_alerting/destinations/<destination-id>
+GET _plugins/_alerting/destinations/<destination-id>
 ```
 
 #### Sample response
@@ -1021,7 +1021,7 @@ Retrieve all destinations.
 #### Requests
 
 ```json
-GET _opensearch/_alerting/destinations
+GET _plugins/_alerting/destinations
 ```
 
 #### Sample response
@@ -1064,7 +1064,7 @@ GET _opensearch/_alerting/destinations
 #### Request
 
 ```
-DELETE _opensearch/_alerting/destinations/<destination-id>
+DELETE _plugins/_alerting/destinations/<destination-id>
 ```
 
 #### Sample response
@@ -1092,7 +1092,7 @@ DELETE _opensearch/_alerting/destinations/<destination-id>
 
 #### Request
 ```json
-POST _opensearch/_alerting/destinations/email_accounts
+POST _plugins/_alerting/destinations/email_accounts
 {
   "name": "example_account",
   "email": "example@email.com",
@@ -1124,7 +1124,7 @@ POST _opensearch/_alerting/destinations/email_accounts
 
 #### Request
 ```json
-PUT _opensearch/_alerting/destinations/email_accounts/<email_account_id>
+PUT _plugins/_alerting/destinations/email_accounts/<email_account_id>
 {
   "name": "example_account",
   "email": "example@email.com",
@@ -1155,7 +1155,7 @@ PUT _opensearch/_alerting/destinations/email_accounts/<email_account_id>
 
 #### Request
 ```json
-GET _opensearch/_alerting/destinations/email_accounts/<email_account_id>
+GET _plugins/_alerting/destinations/email_accounts/<email_account_id>
 {
   "name": "example_account",
   "email": "example@email.com",
@@ -1186,7 +1186,7 @@ GET _opensearch/_alerting/destinations/email_accounts/<email_account_id>
 
 #### Request
 ```
-DELETE _opensearch/_alerting/destinations/email_accounts/<email_account_id>
+DELETE _plugins/_alerting/destinations/email_accounts/<email_account_id>
 ```
 #### Sample response
 
@@ -1213,7 +1213,7 @@ DELETE _opensearch/_alerting/destinations/email_accounts/<email_account_id>
 #### Request
 
 ```json
-POST _opensearch/_alerting/destinations/email_accounts/_search
+POST _plugins/_alerting/destinations/email_accounts/_search
 {
   "from": 0,
   "size": 20,
@@ -1279,7 +1279,7 @@ POST _opensearch/_alerting/destinations/email_accounts/_search
 #### Request
 
 ```json
-POST _opensearch/_alerting/destinations/email_groups
+POST _plugins/_alerting/destinations/email_groups
 {
   "name": "example_email_group",
   "emails": [{
@@ -1313,7 +1313,7 @@ POST _opensearch/_alerting/destinations/email_groups
 #### Request
 
 ```json
-PUT _opensearch/_alerting/destinations/email_groups/<email_group_id>
+PUT _plugins/_alerting/destinations/email_groups/<email_group_id>
 {
   "name": "example_email_group",
   "emails": [{
@@ -1345,7 +1345,7 @@ PUT _opensearch/_alerting/destinations/email_groups/<email_group_id>
 
 #### Request
 ```json
-GET _opensearch/_alerting/destinations/email_groups/<email_group_id>
+GET _plugins/_alerting/destinations/email_groups/<email_group_id>
 {
   "name": "example_email_group",
   "emails": [{
@@ -1377,7 +1377,7 @@ GET _opensearch/_alerting/destinations/email_groups/<email_group_id>
 
 #### Request
 ```
-DELETE _opensearch/_alerting/destinations/email_groups/<email_group_id>
+DELETE _plugins/_alerting/destinations/email_groups/<email_group_id>
 ```
 #### Sample response
 
@@ -1404,7 +1404,7 @@ DELETE _opensearch/_alerting/destinations/email_groups/<email_group_id>
 #### Request
 
 ```json
-POST _opensearch/_alerting/destinations/email_groups/_search
+POST _plugins/_alerting/destinations/email_groups/_search
 {
   "from": 0,
   "size": 20,

docs/alerting/settings.md

@@ -34,26 +34,26 @@ All settings are available using the OpenSearch `_cluster/settings` API. None re
 
 Setting | Default | Description
 :--- | :--- | :---
-`opensearch.scheduled_jobs.enabled` | true | Whether the alerting plugin is enabled or not. If disabled, all monitors immediately stop running.
-`opendistro.alerting.index_timeout` | 60s | The timeout for creating monitors and destinations using the REST APIs.
-`opendistro.alerting.request_timeout` | 10s | The timeout for miscellaneous requests from the plugin.
-`opendistro.alerting.action_throttle_max_value` | 24h | The maximum amount of time you can set for action throttling. By default, this value displays as 1440 minutes in OpenSearch Dashboards.
-`opendistro.alerting.input_timeout` | 30s | How long the monitor can take to issue the search request.
-`opendistro.alerting.bulk_timeout` | 120s | How long the monitor can write alerts to the alert index.
-`opendistro.alerting.alert_backoff_count` | 3 | The number of retries for writing alerts before the operation fails.
-`opendistro.alerting.alert_backoff_millis` | 50ms | The amount of time to wait between retries---increases exponentially after each failed retry.
-`opendistro.alerting.alert_history_rollover_period` | 12h | How frequently to check whether the `.opendistro-alerting-alert-history-write` alias should roll over to a new history index and whether the Alerting plugin should delete any history indices.
-`opendistro.alerting.move_alerts_backoff_millis` | 250 | The amount of time to wait between retries---increases exponentially after each failed retry.
-`opendistro.alerting.move_alerts_backoff_count` | 3 | The number of retries for moving alerts to a deleted state after their monitor or trigger has been deleted.
-`opendistro.alerting.monitor.max_monitors` | 1000 | The maximum number of monitors users can create.
-`opendistro.alerting.alert_history_max_age` | 30d | The oldest document to store in the `.opensearch-alert-history-<date>` index before creating a new index. If the number of alerts in this time period does not exceed `alert_history_max_docs`, alerting creates one history index per period (e.g. one index every 30 days).
-`opendistro.alerting.alert_history_max_docs` | 1000 | The maximum number of alerts to store in the `.opensearch-alert-history-<date>` index before creating a new index.
-`opendistro.alerting.alert_history_enabled` | true | Whether to create `.opendistro-alerting-alert-history-<date>` indices.
-`opendistro.alerting.alert_history_retention_period` | 60d | The amount of time to keep history indices before automatically deleting them.
-`opendistro.alerting.destination.allow_list` | ["chime", "slack", "custom_webhook", "email", "test_action"] | The list of allowed destinations. If you don't want to allow users to a certain type of destination, you can remove it from this list, but we recommend leaving this setting as-is.
-`opendistro.alerting.filter_by_backend_roles` | "false" | Restricts access to monitors by backend role. See [Alerting security](../security/).
-`opensearch.scheduled_jobs.sweeper.period` | 5m | The alerting feature uses its "job sweeper" component to periodically check for new or updated jobs. This setting is the rate at which the sweeper checks to see if any jobs (monitors) have changed and need to be rescheduled.
-`opensearch.scheduled_jobs.sweeper.page_size` | 100 | The page size for the sweeper. You shouldn't need to change this value.
-`opensearch.scheduled_jobs.sweeper.backoff_millis` | 50ms | The amount of time the sweeper waits between retries---increases exponentially after each failed retry.
-`opensearch.scheduled_jobs.sweeper.retry_count` | 3 | The total number of times the sweeper should retry before throwing an error.
-`opensearch.scheduled_jobs.request_timeout` | 10s | The timeout for the request that sweeps shards for jobs.
+`plugins.scheduled_jobs.enabled` | true | Whether the alerting plugin is enabled or not. If disabled, all monitors immediately stop running.
+`plugins.alerting.index_timeout` | 60s | The timeout for creating monitors and destinations using the REST APIs.
+`plugins.alerting.request_timeout` | 10s | The timeout for miscellaneous requests from the plugin.
+`plugins.alerting.action_throttle_max_value` | 24h | The maximum amount of time you can set for action throttling. By default, this value displays as 1440 minutes in OpenSearch Dashboards.
+`plugins.alerting.input_timeout` | 30s | How long the monitor can take to issue the search request.
+`plugins.alerting.bulk_timeout` | 120s | How long the monitor can write alerts to the alert index.
+`plugins.alerting.alert_backoff_count` | 3 | The number of retries for writing alerts before the operation fails.
+`plugins.alerting.alert_backoff_millis` | 50ms | The amount of time to wait between retries---increases exponentially after each failed retry.
+`plugins.alerting.alert_history_rollover_period` | 12h | How frequently to check whether the `.opendistro-alerting-alert-history-write` alias should roll over to a new history index and whether the Alerting plugin should delete any history indices.
+`plugins.alerting.move_alerts_backoff_millis` | 250 | The amount of time to wait between retries---increases exponentially after each failed retry.
+`plugins.alerting.move_alerts_backoff_count` | 3 | The number of retries for moving alerts to a deleted state after their monitor or trigger has been deleted.
+`plugins.alerting.monitor.max_monitors` | 1000 | The maximum number of monitors users can create.
+`plugins.alerting.alert_history_max_age` | 30d | The oldest document to store in the `.opendistro-alert-history-<date>` index before creating a new index. If the number of alerts in this time period does not exceed `alert_history_max_docs`, alerting creates one history index per period (e.g. one index every 30 days).
+`plugins.alerting.alert_history_max_docs` | 1000 | The maximum number of alerts to store in the `.opendistro-alert-history-<date>` index before creating a new index.
+`plugins.alerting.alert_history_enabled` | true | Whether to create `.opendistro-alerting-alert-history-<date>` indices.
+`plugins.alerting.alert_history_retention_period` | 60d | The amount of time to keep history indices before automatically deleting them.
+`plugins.alerting.destination.allow_list` | ["chime", "slack", "custom_webhook", "email", "test_action"] | The list of allowed destinations. If you don't want to allow users to a certain type of destination, you can remove it from this list, but we recommend leaving this setting as-is.
+`plugins.alerting.filter_by_backend_roles` | "false" | Restricts access to monitors by backend role. See [Alerting security](../security/).
+`plugins.scheduled_jobs.sweeper.period` | 5m | The alerting feature uses its "job sweeper" component to periodically check for new or updated jobs. This setting is the rate at which the sweeper checks to see if any jobs (monitors) have changed and need to be rescheduled.
+`plugins.scheduled_jobs.sweeper.page_size` | 100 | The page size for the sweeper. You shouldn't need to change this value.
+`plugins.scheduled_jobs.sweeper.backoff_millis` | 50ms | The amount of time the sweeper waits between retries---increases exponentially after each failed retry.
+`plugins.scheduled_jobs.sweeper.retry_count` | 3 | The total number of times the sweeper should retry before throwing an error.
+`plugins.scheduled_jobs.request_timeout` | 10s | The timeout for the request that sweeps shards for jobs.

docs/im/index-rollups/index.md

@@ -63,7 +63,7 @@ Review your configuration and select **Create**.
 
 ### Step 5: Search the target index
 
-You can use the standard `_search` API to search the target index. Make sure that the query matches the constraints of the target index. For example, if don’t set up terms aggregations on a field, you don’t receive results for terms aggregations. If you don’t set up the maximum aggregations, you don’t receive results for maximum aggregations.
+You can use the standard `_search` API to search the target index. Make sure that the query matches the constraints of the target index. For example, if you don’t set up terms aggregations on a field, you don’t receive results for terms aggregations. If you don’t set up the maximum aggregations, you don’t receive results for maximum aggregations.
 
 You can’t access the internal structure of the data in the target index because the plugin automatically rewrites the query in the background to suit the target index. This is to make sure you can use the same query for the source and target index.
 
@@ -219,7 +219,7 @@ Create an index rollup job.
 This example picks the `order_date`, `customer_gender`, `geoip.city_name`, `geoip.region_name`, and `day_of_week` fields and rolls them into an `example_rollup` target index:
 
 ```json
-PUT _opensearch/_rollup/jobs/example
+PUT _plugins/_rollup/jobs/example
 {
   "rollup": {
     "enabled": true,

docs/im/index-rollups/rollup-api.md

@@ -28,8 +28,8 @@ You must provide the `seq_no` and `primary_term` parameters.
 #### Request
 
 ```json
-PUT _opensearch/_rollup/jobs/<rollup_id> // Create
-PUT _opensearch/_rollup/jobs/<rollup_id>?if_seq_no=1&if_primary_term=1 // Update
+PUT _plugins/_rollup/jobs/<rollup_id> // Create
+PUT _plugins/_rollup/jobs/<rollup_id>?if_seq_no=1&if_primary_term=1 // Update
 {
   "rollup": {
     "source_index": "nyc-taxi-data",
@@ -141,7 +141,7 @@ Returns all information about an index rollup job based on the `rollup_id`.
 #### Request
 
 ```json
-GET _opensearch/_rollup/jobs/<rollup_id>
+GET _plugins/_rollup/jobs/<rollup_id>
 ```
 
 
@@ -166,7 +166,7 @@ Deletes an index rollup job based on the `rollup_id`.
 #### Request
 
 ```json
-DELETE _opensearch/_rollup/jobs/<rollup_id>
+DELETE _plugins/_rollup/jobs/<rollup_id>
 ```
 
 #### Sample response
@@ -185,8 +185,8 @@ Start or stop an index rollup job.
 #### Request
 
 ```json
-POST _opensearch/_rollup/jobs/<rollup_id>/_start
-POST _opensearch/_rollup/jobs/<rollup_id>/_stop
+POST _plugins/_rollup/jobs/<rollup_id>/_start
+POST _plugins/_rollup/jobs/<rollup_id>/_stop
 ```
 
 
@@ -206,7 +206,7 @@ Returns detailed metadata information about the index rollup job and its current
 #### Request
 
 ```json
-GET _opensearch/_rollup/jobs/<rollup_id>/_explain
+GET _plugins/_rollup/jobs/<rollup_id>/_explain
 ```
 
 

docs/im/ism/api.md

@@ -28,7 +28,7 @@ Creates a policy.
 #### Request
 
 ```json
-PUT _opensearch/_ism/policies/policy_1
+PUT _plugins/_ism/policies/policy_1
 {
   "policy": {
     "description": "ingesting logs",
@@ -145,7 +145,7 @@ Adds a policy to an index. This operation does not change the policy if the inde
 #### Request
 
 ```json
-POST _opensearch/_ism/add/index_1
+POST _plugins/_ism/add/index_1
 {
   "policy_id": "policy_1"
 }
@@ -171,7 +171,7 @@ Updates a policy. Use the `seq_no` and `primary_term` parameters to update an ex
 #### Request
 
 ```json
-PUT _opensearch/_ism/policies/policy_1?if_seq_no=7&if_primary_term=1
+PUT _plugins/_ism/policies/policy_1?if_seq_no=7&if_primary_term=1
 {
   "policy": {
     "description": "ingesting logs",
@@ -288,7 +288,7 @@ Gets the policy by `policy_id`.
 #### Request
 
 ```json
-GET _opensearch/_ism/policies/policy_1
+GET _plugins/_ism/policies/policy_1
 ```
 
 
@@ -358,7 +358,7 @@ Removes any ISM policy from the index.
 #### Request
 
 ```json
-POST _opensearch/_ism/remove/index_1
+POST _plugins/_ism/remove/index_1
 ```
 
 
@@ -385,7 +385,7 @@ In this example, the policy applied on the `index_1` index is changed to `policy
 #### Request
 
 ```json
-POST _opensearch/_ism/change_policy/index_1
+POST _plugins/_ism/change_policy/index_1
 {
   "policy_id": "policy_1",
   "state": "delete",
@@ -417,7 +417,7 @@ Retries the failed action for an index. For the retry call to succeed, ISM must
 #### Request
 
 ```json
-POST _opensearch/_ism/retry/index_1
+POST _plugins/_ism/retry/index_1
 {
   "state": "delete"
 }
@@ -443,7 +443,7 @@ Gets the current state of the index. You can use index patterns to get the statu
 #### Request
 
 ```json
-GET _opensearch/_ism/explain/index_1
+GET _plugins/_ism/explain/index_1
 ```
 
 
@@ -469,7 +469,7 @@ Deletes the policy by `policy_id`.
 #### Request
 
 ```json
-DELETE _opensearch/_ism/policies/policy_1
+DELETE _plugins/_ism/policies/policy_1
 ```
 
 

docs/im/ism/index.md

@@ -41,7 +41,7 @@ After you create a policy, your next step is to attach this policy to an index o
 You can set up an `ism_template` in the policy so when you create an index that matches the ISM template pattern, the index will have this policy attached to it:
 
 ```json
-PUT _opensearch/_ism/policies/policy_id
+PUT _plugins/_ism/policies/policy_id
 {
   "policy": {
     "description": "Example policy.",

docs/im/ism/policies.md

@@ -201,7 +201,7 @@ Deletes a managed index.
 Rolls an alias over to a new index when the managed index meets one of the rollover conditions.
 
 The index format must match the pattern: `^.*-\d+$`. For example, `(logs-000001)`.
-Set `index.opendistro.index_state_management.rollover_alias` as the alias to rollover.
+Set `index.plugins.index_state_management.rollover_alias` as the alias to rollover.
 
 Parameter | Description | Type | Example | Required
 :--- | :--- |:--- |:--- |
@@ -526,7 +526,7 @@ The following sample template policy is for a rollover use case.
 1. Create a policy with an `ism_template` field:
 
    ```json
-   PUT _opensearch/_ism/policies/rollover_policy
+   PUT _plugins/_ism/policies/rollover_policy
    {
      "policy": {
        "description": "Example rollover policy.",
@@ -561,7 +561,7 @@ The following sample template policy is for a rollover use case.
    {
      "index_patterns": ["log*"],
      "settings": {
-       "opendistro.index_state_management.rollover_alias": "log"
+       "plugins.index_state_management.rollover_alias": "log"
      }
    }
    ```

docs/im/ism/settings.md

@@ -17,17 +17,17 @@ All settings are available using the OpenSearch `_cluster/settings` operation. N
 
 Setting | Default | Description
 :--- | :--- | :---
-`opendistro.index_state_management.enabled` | True | Specifies whether ISM is enabled or not.
-`opendistro.index_state_management.job_interval` | 5 minutes | The interval at which the managed index jobs are run.
-`opendistro.index_state_management.coordinator.sweep_period` | 10 minutes | How often the routine background sweep is run.
-`opendistro.index_state_management.coordinator.backoff_millis` | 50 milliseconds | The backoff time between retries for failures in the `ManagedIndexCoordinator` (such as when we update managed indices).
-`opendistro.index_state_management.coordinator.backoff_count` | 2 | The count of retries for failures in the `ManagedIndexCoordinator`.
-`opendistro.index_state_management.history.enabled` | True | Specifies whether audit history is enabled or not. The logs from ISM are automatically indexed to a logs document.
-`opendistro.index_state_management.history.max_docs` | 2,500,000 | The maximum number of documents before rolling over the audit history index.
-`opendistro.index_state_management.history.max_age` | 24 hours | The maximum age before rolling over the audit history index.
-`opendistro.index_state_management.history.rollover_check_period` | 8 hours | The time between rollover checks for the audit history index.
-`opendistro.index_state_management.history.rollover_retention_period` | 30 days | How long audit history indices are kept.
-`opendistro.index_state_management.allow_list` | All actions | List of actions that you can use.
+`plugins.index_state_management.enabled` | True | Specifies whether ISM is enabled or not.
+`plugins.index_state_management.job_interval` | 5 minutes | The interval at which the managed index jobs are run.
+`plugins.index_state_management.coordinator.sweep_period` | 10 minutes | How often the routine background sweep is run.
+`plugins.index_state_management.coordinator.backoff_millis` | 50 milliseconds | The backoff time between retries for failures in the `ManagedIndexCoordinator` (such as when we update managed indices).
+`plugins.index_state_management.coordinator.backoff_count` | 2 | The count of retries for failures in the `ManagedIndexCoordinator`.
+`plugins.index_state_management.history.enabled` | True | Specifies whether audit history is enabled or not. The logs from ISM are automatically indexed to a logs document.
+`plugins.index_state_management.history.max_docs` | 2,500,000 | The maximum number of documents before rolling over the audit history index.
+`plugins.index_state_management.history.max_age` | 24 hours | The maximum age before rolling over the audit history index.
+`plugins.index_state_management.history.rollover_check_period` | 8 hours | The time between rollover checks for the audit history index.
+`plugins.index_state_management.history.rollover_retention_period` | 30 days | How long audit history indices are kept.
+`plugins.index_state_management.allow_list` | All actions | List of actions that you can use.
 
 
 ## Audit history indices

docs/im/refresh-analyzer/index.md

@@ -13,7 +13,7 @@ has_toc: false
 With ISM installed, you can refresh search analyzers in real time with the following API:
 
 ```json
-POST /_opensearch/_refresh_search_analyzers/<index or alias or wildcard>
+POST /_plugins/_refresh_search_analyzers/<index or alias or wildcard>
 ```
 For example, if you change the synonym list in your analyzer, the change takes effect without you needing to close and reopen the index.
 

docs/knn/api.md

@@ -1,7 +1,7 @@
 ---
 layout: default
 title: API
-nav_order: 4
+nav_order: 5
 parent: k-NN
 has_children: false
 ---
@@ -15,7 +15,7 @@ The k-NN plugin adds two API operations to help you better manage the plugin's f
 
 The k-NN `stats` API provides information about the current status of the k-NN plugin. The plugin keeps track of both cluster-level and node-level statistics. Cluster-level statistics have a single value for the entire cluster. Node-level statistics have a single value for each node in the cluster. You can filter the query by `nodeId` and `statName`:
 ```
-GET /_opensearch/_knn/nodeId1,nodeId2/stats/statName1,statName2
+GET /_plugins/_knn/nodeId1,nodeId2/stats/statName1,statName2
 ```
 
 Statistic |  Description
@@ -45,7 +45,7 @@ Statistic |  Description
 ### Usage
 
 ```json
-GET /_opensearch/_knn/stats?pretty
+GET /_plugins/_knn/stats?pretty
 {
     "_nodes" : {
         "total" : 1,
@@ -87,7 +87,7 @@ GET /_opensearch/_knn/stats?pretty
 ```
 
 ```json
-GET /_opensearch/_knn/HYMrXXsBSamUkcAjhjeN0w/stats/circuit_breaker_triggered,graph_memory_usage?pretty
+GET /_plugins/_knn/HYMrXXsBSamUkcAjhjeN0w/stats/circuit_breaker_triggered,graph_memory_usage?pretty
 {
     "_nodes" : {
         "total" : 1,
@@ -121,7 +121,7 @@ After the process finishes, you can start searching against the indices with no
 This request performs a warmup on three indices:
 
 ```json
-GET /_opensearch/_knn/warmup/index1,index2,index3?pretty
+GET /_plugins/_knn/warmup/index1,index2,index3?pretty
 {
   "_shards" : {
     "total" : 6,

docs/knn/approximate-knn.md

@@ -1,7 +1,7 @@
 ---
 layout: default
 title: Approximate search
-nav_order: 1
+nav_order: 2
 parent: k-NN
 has_children: false
 has_math: true
@@ -19,7 +19,7 @@ Because the graphs are constructed during indexing, it is not possible to apply
 
 To use the k-NN plugin's approximate search functionality, you must first create a k-NN index with setting `index.knn` to `true`. This setting tells the plugin to create HNSW graphs for the index.
 
-Additionally, if you're using the approximate k-nearest neighbor method, specify `knn.space_type` to the space you're interested in. You can't change this setting after it's set. To see what spaces we support, see [spaces](#spaces). By default, `index.knn.space_type` is `l2`. For more information about index settings, such as algorithm parameters you can tweak to tune performance, see [Index settings](../settings#index-settings).
+Additionally, if you're using the approximate k-nearest neighbor method, specify `knn.space_type` to the space you're interested in. You can't change this setting after it's set. To see what spaces we support, see [spaces](#spaces). By default, `index.knn.space_type` is `l2`. For more information about index settings, such as algorithm parameters you can tweak to tune performance, see [Index settings](../knn-index#index-settings).
 
 Next, you must add one or more fields of the `knn_vector` data type. This example creates an index with two `knn_vector` fields and uses cosine similarity:
 
@@ -29,19 +29,37 @@ PUT my-knn-index-1
   "settings": {
     "index": {
       "knn": true,
-      "knn.space_type": "cosinesimil"
+      "knn.algo_param.ef_search": 100
     }
   },
   "mappings": {
     "properties": {
-      "my_vector1": {
-        "type": "knn_vector",
-        "dimension": 2
-      },
-      "my_vector2": {
-        "type": "knn_vector",
-        "dimension": 4
-      }
+        "my_vector1": {
+          "type": "knn_vector",
+          "dimension": 4,
+          "method": {
+            "name": "hnsw",
+            "space_type": "l2",
+            "engine": "nmslib",
+            "parameters": {
+              "ef_construction": 128,
+              "m": 24
+            } 
+          }
+        },
+        "my_vector2": {
+          "type": "knn_vector",
+          "dimension": 4,
+          "method": {
+            "name": "hnsw",
+            "space_type": "cosinesimil",
+            "engine": "nmslib",
+            "parameters": {
+              "ef_construction": 256,
+              "m": 48
+            } 
+          }
+        }
     }
   }
 }
@@ -144,6 +162,11 @@ A space corresponds to the function used to measure the distance between two poi
     <td>\[ Distance(X, Y) = \sum_{i=1}^n (X_i - Y_i) \]</td>
     <td>1 / (1 + Distance Function)</td>
   </tr>
+  <tr>
+    <td>linf</td>
+    <td>\[ Distance(X, Y) = Max(X_i - Y_i) \]</td>
+    <td>1 / (1 + Distance Function)</td>
+  </tr>
   <tr>
     <td>cosinesimil</td>
     <td>\[ 1 - {A &middot; B \over \|A\| &middot; \|B\|} = 1 -
@@ -152,9 +175,9 @@ A space corresponds to the function used to measure the distance between two poi
     <td>1 / (1 + Distance Function)</td>
   </tr>
   <tr>
-    <td>hammingbit</td>
-    <td style="text-align:center">Distance = countSetBits(X \(\oplus\) Y)</td>
-    <td>1 / (1 + Distance Function)</td>
+    <td>innerproduct</td>
+    <td>\[ Distance(X, Y) = - {A &middot; B} \]</td>
+    <td>if (Distance Function >= 0) 1 / (1 + Distance Function) else -Distance Function + 1</td>
   </tr>
 </table>
 

docs/knn/jni-library.md

@@ -1,7 +1,7 @@
 ---
 layout: default
 title: JNI library
-nav_order: 5
+nav_order: 6
 parent: k-NN
 has_children: false
 ---

docs/knn/knn-index.md

@@ -0,0 +1,58 @@
+---
+layout: default
+title: k-NN Index
+nav_order: 1
+parent: k-NN
+has_children: false
+---
+
+# k-NN Index
+
+## `knn_vector` datatype
+The k-NN plugin introduces a custom data type, the `knn_vector`, that allows users to ingest their k-NN vectors
+into an OpenSearch index.
+
+```json
+"my_vector": {
+  "type": "knn_vector",
+  "dimension": 4,
+  "method": {
+    "name": "hnsw",
+    "space_type": "l2",
+    "engine": "nmslib",
+    "parameters": {
+      "ef_construction": 128,
+      "m": 24
+    } 
+  }
+}
+```
+
+Mapping Pararameter | Required | Default | Updateable | Description
+:--- | :--- | :--- | :--- | :---
+`type` | true | n/a | false | The type of the field
+`dimension` | true | n/a | false | The vector dimension for the field
+`method` | false | null | false | The configuration for the Approximate nearest neighbor method
+`method.name` | true, if `method` is specified | n/a | false | The identifier for the nearest neighbor method. Currently, "hnsw" is the only valid method.
+`method.space_type` | false | "l2" | false | The vector space used to calculate the distance between vectors. Refer to [here](../approximate-knn#spaces)) to see available spaces.
+`method.engine` | false | "nmslib" | false | The approximate k-NN library to use for indexing and search. Currently, "nmslib" is the only valid engine.
+`method.parameters` | false | null | false | The parameters used for the nearest neighbor method.
+`method.parameters.ef_construction` | false | 512 | false | The size of the dynamic list used during k-NN graph creation. Higher values lead to a more accurate graph, but slower indexing speed. Only valid for "hnsw" method.
+`method.parameters.m` | false | 16 | false | The number of bidirectional links that the plugin creates for each new element. Increasing and decreasing this value can have a large impact on memory consumption. Keep this value between 2-100. Only valid for "hnsw" method
+
+## Index settings
+
+Additionally, the k-NN plugin introduces several index settings that can be used to configure the k-NN structure as well.
+
+At the moment, several parameters defined in the settings are in the deprecation process. Those parameters should be set 
+in the mapping instead of the index settings. Parameters set in the mapping will override the parameters set in the 
+index settings. Setting the parameters in the mapping allows an index to have multiple `knn_vector` fields with 
+different parameters.
+
+Setting | Default | Updateable | Description
+:--- | :--- | :--- | :---
+`index.knn` | false | false | Whether the index should build hnsw graphs for the `knn_vector` fields. If set to false, the `knn_vector` fields will be stored in doc values, but Approximate k-NN search functionality will be disabled. 
+`index.knn.algo_param.ef_search` | 512 | true | The size of the dynamic list used during k-NN searches. Higher values lead to more accurate but slower searches. 
+`index.knn.algo_param.ef_construction` | 512 | false | (Deprecated in 1.0.0. Use the mapping parameters to set this value instead.) Refer to mapping definition.
+`index.knn.algo_param.m` | 16 | false | (Deprecated in 1.0.0. Use the mapping parameters to set this value instead.) Refer to mapping definition.
+`index.knn.space_type` | "l2" | false | (Deprecated in 1.0.0. Use the mapping parameters to set this value instead.) Refer to mapping definition.

docs/knn/knn-score-script.md

@@ -1,7 +1,7 @@
 ---
 layout: default
 title: Exact k-NN with scoring script
-nav_order: 2
+nav_order: 3
 parent: k-NN
 has_children: false
 has_math: true
@@ -298,6 +298,11 @@ A space corresponds to the function used to measure the distance between two poi
     <td>\[ Distance(X, Y) = \sum_{i=1}^n (X_i - Y_i) \]</td>
     <td>1 / (1 + Distance Function)</td>
   </tr>
+  <tr>
+    <td>linf</td>
+    <td>\[ Distance(X, Y) = Max(X_i - Y_i) \]</td>
+    <td>1 / (1 + Distance Function)</td>
+  </tr>
   <tr>
     <td>cosinesimil</td>
     <td>\[ {A &middot; B \over \|A\| &middot; \|B\|} =
@@ -305,6 +310,11 @@ A space corresponds to the function used to measure the distance between two poi
     where \(\|A\|\) and \(\|B\|\) represent normalized vectors.</td>
     <td>1 + Distance Function</td>
   </tr>
+  <tr>
+    <td>innerproduct</td>
+    <td>\[ Distance(X, Y) = -{A &middot; B} \]</td>
+    <td>if (Distance Function >= 0) 1 / (1 + Distance Function) else -Distance Function + 1</td>
+  </tr>
   <tr>
     <td>hammingbit</td>
     <td style="text-align:center">Distance = countSetBits(X \(\oplus\) Y)</td>

docs/knn/painless-functions.md

@@ -1,7 +1,7 @@
 ---
 layout: default
 title: k-NN Painless extensions
-nav_order: 3
+nav_order: 4
 parent: k-NN
 has_children: false
 has_math: true

docs/knn/performance-tuning.md

@@ -2,7 +2,7 @@
 layout: default
 title: Performance tuning
 parent: k-NN
-nav_order: 7
+nav_order: 8
 ---
 
 # Performance tuning
@@ -64,7 +64,7 @@ Take the following steps to improve search performance:
    To avoid this latency penalty during your first queries, you can use the warmup API operation on the indices you want to search:
 
    ```json
-   GET /_opensearch/_knn/warmup/index1,index2,index3?pretty
+   GET /_plugins/_knn/warmup/index1,index2,index3?pretty
    {
      "_shards" : {
        "total" : 6,
@@ -88,7 +88,7 @@ Recall depends on multiple factors like number of vectors, number of dimensions,
 
 To configure recall, adjust the algorithm parameters of the HNSW algorithm exposed through index settings. Algorithm parameters that control recall are `m`, `ef_construction`, and `ef_search`. For more information about how algorithm parameters influence indexing and search recall, see [HNSW algorithm parameters](https://github.com/nmslib/hnswlib/blob/master/ALGO_PARAMS.md). Increasing these values can help recall and lead to better search results, but at the cost of higher memory utilization and increased indexing time. 
 
-The default recall values work on a broader set of use cases, but make sure to run your own experiments on your data sets and choose the appropriate values. For index-level settings, see [Index settings](../settings#index-settings).
+The default recall values work on a broader set of use cases, but make sure to run your own experiments on your data sets and choose the appropriate values. For index-level settings, see [Index settings](../knn-index#index-settings).
 
 ## Estimating memory usage
 

docs/knn/settings.md

@@ -2,25 +2,12 @@
 layout: default
 title: Settings
 parent: k-NN
-nav_order: 6
+nav_order: 7
 ---
 
 # k-NN settings
 
-The k-NN plugin adds several new index and cluster settings.
-
-
-## Index settings
-
-The default values work well for most use cases, but you can change these settings when you create the index.
-
-Setting | Default | Description
-:--- | :--- | :---
-`index.knn.algo_param.ef_search` | 512 | The size of the dynamic list used during k-NN searches. Higher values lead to more accurate but slower searches.
-`index.knn.algo_param.ef_construction` | 512 | The size of the dynamic list used during k-NN graph creation. Higher values lead to a more accurate graph, but slower indexing speed.
-`index.knn.algo_param.m` | 16 | The number of bidirectional links that the plugin creates for each new element. Increasing and decreasing this value can have a large impact on memory consumption. Keep this value between 2-100.
-`index.knn.space_type` | "l2" | The vector space used to calculate the distance between vectors. Currently, the k-NN plugin supports the `l2` space (Euclidean distance) and `cosinesimil` space (cosine similarity). For more information on these spaces, see the [nmslib documentation](https://github.com/nmslib/nmslib/blob/master/manual/spaces.md).
-
+The k-NN plugin adds several new cluster settings.
 
 ## Cluster settings
 

docs/opensearch-dashboards/browser-compatibility.md

@@ -0,0 +1,17 @@
+---
+layout: default
+title: Browser compatibility
+parent: OpenSearch Dashboards
+nav_order: 3
+---
+
+# Browser compatibility
+
+OpenSearch Dashboards supports the following web browsers:
+
+- Chrome
+- Firefox
+- Safari
+- Edge (Chromium)
+
+Other Chromium-based browsers might work, as well. Internet Explorer and Microsoft Edge Legacy are **not** supported.

docs/opensearch/aggregations.md

@@ -16,13 +16,13 @@ OpenSearch can perform aggregations on massive datasets in milliseconds. Compare
 
 ## Aggregations on text fields
 
-By default, OpenSearch doesn't support aggregations on a text field.
-Because text fields are tokenized, an aggregation on a text field has to reverse the tokenization process back to its original string and then formulate an aggregation based on that. Such an operation consumes significant memory and degrades cluster performance.
+By default, OpenSearch doesn't support aggregations on a text field. Because text fields are tokenized, an aggregation on a text field has to reverse the tokenization process back to its original string and then formulate an aggregation based on that. This kind of an operation consumes significant memory and degrades cluster performance.
 
 While you can enable aggregations on text fields by setting the `fielddata` parameter to `true` in the mapping, the aggregations are still based on the tokenized words and not on the raw text.
 
 We recommend keeping a raw version of the text field as a `keyword` field that you can aggregate on.
-In this case, you can perform aggregations on the `title.raw` field, instead of the `title` field:
+
+In this case, you can perform aggregations on the `title.raw` field, instead of on the `title` field:
 
 ```json
 PUT movies
@@ -61,15 +61,13 @@ GET _search
 
 If you’re only interested in the aggregation result and not in the results of the query, set `size` to 0.
 
-In the `aggs` property (you can use `aggregations` if you want), you can define any number of aggregations.
-Each aggregation is defined by its name and one of the types of aggregations that OpenSearch supports.
+In the `aggs` property (you can use `aggregations` if you want), you can define any number of aggregations. Each aggregation is defined by its name and one of the types of aggregations that OpenSearch supports.
 
-The name of the aggregation helps you to distinguish between different aggregations in the response.
-The `AGG_TYPE` property is where you specify the type of aggregation.
+The name of the aggregation helps you to distinguish between different aggregations in the response. The `AGG_TYPE` property is where you specify the type of aggregation.
 
 ## Sample aggregation
 
-This section uses the OpenSearch Dashboards sample e-commerce data and web log data. To add the sample data, log in to OpenSearch Dashboards, choose **Home** and **Try our sample data**. For **Sample eCommerce orders** and **Sample web logs**, choose **Add data**.
+This section uses the OpenSearch Dashboards sample ecommerce data and web log data. To add the sample data, log in to OpenSearch Dashboards, choose **Home**, and then choose **Try our sample data**. For **Sample eCommerce orders** and **Sample web logs**, choose **Add data**.
 
 ### avg
 
@@ -129,7 +127,7 @@ There are three main types of aggregations:
 
 ## Nested aggregations
 
-Aggregations within aggregations are called nested or sub aggregations.
+Aggregations within aggregations are called nested or subaggregations.
 
 Metric aggregations produce simple results and can't contain nested aggregations.
 

docs/opensearch/bucket-agg.md

@@ -11,7 +11,7 @@ has_children: false
 
 Bucket aggregations categorize sets of documents as buckets. The type of bucket aggregation determines whether a given document falls into a bucket or not.
 
-You can use bucket aggregations to implement faceted navigation (usually placed as a sidebar on a search result landing page) to help you're users narrow down the results.
+You can use bucket aggregations to implement faceted navigation (usually placed as a sidebar on a search result landing page) to help your users narrow down the results.
 
 ## terms
 

docs/opensearch/install/docker-security.md

@@ -26,7 +26,7 @@ services:
       - discovery.seed_hosts=opensearch-node1,opensearch-node2
       - cluster.initial_master_nodes=opensearch-node1,opensearch-node2
       - bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
-      - "ES_JAVA_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
+      - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
       - network.host=0.0.0.0 # required if not using the demo security configuration
     ulimits:
       memlock:
@@ -62,7 +62,7 @@ services:
       - discovery.seed_hosts=opensearch-node1,opensearch-node2
       - cluster.initial_master_nodes=opensearch-node1,opensearch-node2
       - bootstrap.memory_lock=true
-      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
+      - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m"
       - network.host=0.0.0.0
     ulimits:
       memlock:

docs/opensearch/install/docker.md

@@ -15,7 +15,7 @@ docker pull opensearchproject/opensearch:{{site.opensearch_version}}
 docker pull opensearchproject/opensearch-dashboards:{{site.opensearch_version}}
 ```
 
-To check available versions, see [Docker Hub](https://hub.docker.com/u/opensearch).
+To check available versions, see [Docker Hub](https://hub.docker.com/u/opensearchproject).
 
 OpenSearch images use `centos:7` as the base image. If you run Docker locally, we recommend allowing Docker to use at least 4 GB of RAM in **Preferences** > **Resources**.
 
@@ -95,7 +95,7 @@ services:
       - discovery.seed_hosts=opensearch-node1,opensearch-node2
       - cluster.initial_master_nodes=opensearch-node1,opensearch-node2
       - bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
-      - "ES_JAVA_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
+      - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
     ulimits:
       memlock:
         soft: -1
@@ -119,7 +119,7 @@ services:
       - discovery.seed_hosts=opensearch-node1,opensearch-node2
       - cluster.initial_master_nodes=opensearch-node1,opensearch-node2
       - bootstrap.memory_lock=true
-      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
+      - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m"
     ulimits:
       memlock:
         soft: -1
@@ -149,7 +149,6 @@ volumes:
 
 networks:
   opensearch-net:
-
 ```
 
 If you override `opensearch_dashboards.yml` settings using environment variables, as seen above, use all uppercase letters and periods in place of underscores (e.g. for `opensearch.url`, specify `OPENSEARCH_URL`).

docs/opensearch/install/important-settings.md

@@ -28,7 +28,7 @@ The [sample docker-compose.yml](../docker/#sample-docker-compose-file) file also
 
   Disbles swapping (along with `memlock`). Swapping can dramatically decrease performance and stability, so you should ensure it is disabled on production clusters.
 
-- `ES_JAVA_OPTS=-Xms512m -Xmx512m`
+- `OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m`
 
   Sets the size of the Java heap (we recommend half of system RAM).
 

docs/opensearch/query-dsl/bool.md

@@ -1,7 +1,8 @@
 ---
 layout: default
 title: Boolean queries
-parent: OpenSearch
+parent: Query DSL
+grand_parent: OpenSearch
 nav_order: 45
 ---
 

docs/opensearch/query-dsl/full-text.md

@@ -1,14 +1,13 @@
 ---
 layout: default
 title: Full-text queries
-parent: OpenSearch
+parent: Query DSL
+grand_parent: OpenSearch
 nav_order: 40
 ---
 
 # Full-text queries
 
-Although you can use HTTP request parameters to perform simple searches, the OpenSearch query domain-specific language (DSL) lets you specify the full range of search options. The query DSL uses the HTTP request body. Queries specified in this way have the added advantage of being more explicit in their intent and easier to tune over time.
-
 This page lists all full-text query types and common options. Given the sheer number of options and subtle behaviors, the best method of ensuring useful search results is to test different queries against representative indices and verify the output.
 
 

docs/opensearch/query-dsl/index.md

@@ -0,0 +1,121 @@
+---
+layout: default
+title: Query DSL
+nav_order: 27
+parent: OpenSearch
+has_children: true
+---
+
+# Query DSL
+
+While you can use HTTP request parameters to perform simple searches, you can also use the OpenSearch query domain-specific language (DSL), which provides a wider range of search options. The query DSL uses the HTTP request body, so you can more easily customize your queries to get the exact results that you want.
+
+For example, the following request performs a simple search to search for a `speaker` field that has a value of `queen`.
+
+**Sample request**
+```json
+GET _search?q=speaker:queen
+```
+
+**Sample response**
+```
+{
+  "took": 87,
+  "timed_out": false,
+  "_shards": {
+  "total": 68,
+  "successful": 68,
+  "skipped": 0,
+  "failed": 0
+  },
+  "hits": {
+  "total": {
+    "value": 4080,
+    "relation": "eq"
+  },
+  "max_score": 4.4368687,
+  "hits": [
+    {
+    "_index": "new_shakespeare",
+    "_type": "_doc",
+    "_id": "28559",
+    "_score": 4.4368687,
+    "_source": {
+      "type": "line",
+      "line_id": 28560,
+      "play_name": "Cymbeline",
+      "speech_number": 20,
+      "line_number": "1.1.81",
+      "speaker": "QUEEN",
+      "text_entry": "No, be assured you shall not find me, daughter,"
+    }
+    }
+```
+
+With query DSL, however, you can include an HTTP request body to look for results more tailored to your needs. The following example shows how to search for `speaker` and `text_entry` fields that have a value of `QUEEN`.
+
+**Sample request**
+```json
+{
+  "query": {
+  "multi_match": {
+    "query": "QUEEN",
+    "fields": ["speaker", "text_entry"]
+    }
+  }
+}
+```
+
+**Sample Response**
+```json
+{
+  "took": 39,
+  "timed_out": false,
+  "_shards": {
+    "total": 68,
+    "successful": 68,
+    "skipped": 0,
+    "failed": 0
+  },
+  "hits": {
+    "total": {
+      "value": 5837,
+      "relation": "eq"
+    },
+    "max_score": 7.8623476,
+    "hits": [
+      {
+        "_index": "new_shakespeare",
+        "_type": "_doc",
+        "_id": "100763",
+        "_score": 7.8623476,
+        "_source": {
+          "type": "line",
+          "line_id": 100764,
+          "play_name": "Troilus and Cressida",
+          "speech_number": 43,
+          "line_number": "3.1.68",
+          "speaker": "PANDARUS",
+          "text_entry": "Sweet queen, sweet queen! thats a sweet queen, i faith."
+        }
+      },
+      {
+        "_index": "shakespeare",
+        "_type": "_doc",
+        "_id": "28559",
+        "_score": 5.8923807,
+        "_source": {
+          "type": "line",
+          "line_id": 28560,
+          "play_name": "Cymbeline",
+          "speech_number": 20,
+          "line_number": "1.1.81",
+          "speaker": "QUEEN",
+          "text_entry": "No, be assured you shall not find me, daughter,"
+        }
+      }
+    ]
+  }
+}
+```
+The OpenSearch query DSL comes in three varieties: term-level queries, full-text queries, and boolean queries. You can even perform more complicated searches by using different elements from each variety to find whatever data you need.

docs/opensearch/query-dsl/term.md

@@ -1,7 +1,8 @@
 ---
 layout: default
 title: Term-level queries
-parent: OpenSearch
+parent: Query DSL
+grand_parent: OpenSearch
 nav_order: 30
 ---
 

docs/opensearch/rest-api/bulk.md

@@ -0,0 +1,170 @@
+---
+layout: default
+title: Bulk
+parent: REST API reference
+grand_parent: OpenSearch
+nav_order: 15
+---
+
+# Bulk
+
+The bulk operation lets you add, update, or delete many documents in a single request. Compared to individual OpenSearch indexing requests, the bulk operation has significant performance benefits. Whenever practical, we recommend batching indexing operations into bulk requests.
+
+
+## Example
+
+```json
+POST _bulk
+{ "delete": { "_index": "movies", "_id": "tt2229499" } }
+{ "index": { "_index": "movies", "_id": "tt1979320" } }
+{ "title": "Rush", "year": 2013 }
+{ "create": { "_index": "movies", "_id": "tt1392214" } }
+{ "title": "Prisoners", "year": 2013 }
+{ "update": { "_index": "movies", "_id": "tt0816711" } }
+{ "doc" : { "title": "World War Z" } }
+
+```
+
+
+## Path and HTTP methods
+
+```
+POST _bulk
+POST {index}/_bulk
+```
+
+Specifying the index in the path means you don't need to include it in the [request body](#request-body).
+
+OpenSearch also accepts PUT requests to the `_bulk` path, but we highly recommend using POST. The accepted usage of PUT---adding or replacing a single resource at a given path---doesn't make sense for bulk requests.
+{: .note }
+
+
+## URL parameters
+
+All bulk URL parameters are optional.
+
+Parameter | Type | Description
+:--- | :--- | :---
+pipeline | String | The pipeline ID for preprocessing documents.
+refresh | Enum | Whether to refresh the affected shards after performing the indexing operations. Default is `false`. `true` makes the changes show up in search results immediately, but hurts cluster performance. `wait_for` waits for a refresh. Requests take longer to return, but cluster performance doesn't suffer.
+require_alias | Boolean | Set to `true` to require that all actions target an index alias rather than an index. Default is `false`.
+routing | String | Routes the request to the specified shard.
+timeout | Time | How long to wait for the request to return. Default `1m`.
+type | String | (Deprecated) The default document type for documents that don't specify a type. Default is `_doc`. We highly recommend ignoring this parameter and using a type of `_doc` for all indices.
+wait_for_active_shards | String | Specifies the number of active shards that must be available before OpenSearch processes the bulk request. Default is 1 (only the primary shard). Set to `all` or a positive integer. Values greater than 1 require replicas. For example, if you specify a value of 3, the index must have two replicas distributed across two additional nodes for the request to succeed.
+{% comment %}_source | List | asdf
+_source_excludes | list | asdf
+_source_includes | list | asdf{% endcomment %}
+
+
+## Request body
+
+The bulk request body follows this pattern:
+
+```
+Action and metadata\n
+Optional document\n
+Action and metadata\n
+Optional document\n
+
+```
+
+The optional JSON document doesn't need to be minified---spaces are fine---but it does need to be on a single line. OpenSearch uses newline characters to parse bulk requests and requires that the request body end with a newline character.
+
+All actions support the same metadata: `_index`, `_id`, and `_require_alias`. If you don't provide an ID, OpenSearch generates one automatically, which can make it challenging to update the document at a later time.
+
+- Create
+
+  Creates a document if it doesn't already exist and returns an error otherwise. The next line must include a JSON document.
+
+  ```json
+  { "create": { "_index": "movies", "_id": "tt1392214" } }
+  { "title": "Prisoners", "year": 2013 }
+  ```
+
+- Delete
+
+  This action deletes a document if it exists. If the document doesn't exist, OpenSearch doesn't return an error, but instead returns `not_found` under `result`. Delete actions don't require documents on the next line.
+
+  ```json
+  { "delete": { "_index": "movies", "_id": "tt2229499" } }
+  ```
+
+- Index
+
+  Index actions create a document if it doesn't yet exist and replace the document if it already exists. The next line must include a JSON document.
+
+  ```json
+  { "index": { "_index": "movies", "_id": "tt1979320" } }
+  { "title": "Rush", "year": 2013}
+  ```
+
+- Update
+
+  This action updates existing documents and returns an error if the document doesn't exist. The next line must include a full or partial JSON document, depending on how much of the document you want to update. It can also include a script or upsert for more complex document updates.
+
+  ```json
+  { "update": { "_index": "movies", "_id": "tt0816711" } }
+  { "doc" : { "title": "World War Z" } }
+  ```
+
+
+## Response
+
+In the response, pay particular attention to the top-level `errors` boolean. If true, you can iterate over the individual actions for more detailed information.
+
+```json
+{
+  "took": 11,
+  "errors": true,
+  "items": [
+    {
+      "index": {
+        "_index": "movies",
+        "_type": "_doc",
+        "_id": "tt1979320",
+        "_version": 1,
+        "result": "created",
+        "_shards": {
+          "total": 2,
+          "successful": 1,
+          "failed": 0
+        },
+        "_seq_no": 1,
+        "_primary_term": 1,
+        "status": 201
+      }
+    },
+    {
+      "create": {
+        "_index": "movies",
+        "_type": "_doc",
+        "_id": "tt1392214",
+        "status": 409,
+        "error": {
+          "type": "version_conflict_engine_exception",
+          "reason": "[tt1392214]: version conflict, document already exists (current version [1])",
+          "index": "movies",
+          "shard": "0",
+          "index_uuid": "yhizhusbSWmP0G7OJnmcLg"
+        }
+      }
+    },
+    {
+      "update": {
+        "_index": "movies",
+        "_type": "_doc",
+        "_id": "tt0816711",
+        "status": 404,
+        "error": {
+          "type": "document_missing_exception",
+          "reason": "[_doc][tt0816711]: document missing",
+          "index": "movies",
+          "shard": "0",
+          "index_uuid": "yhizhusbSWmP0G7OJnmcLg"
+        }
+      }
+    }
+  ]
+}
+```

docs/opensearch/rest-api/cluster-allocation.md

@@ -0,0 +1,145 @@
+---
+layout: default
+title: Cluster allocation explain
+parent: REST API reference
+grand_parent: OpenSearch
+nav_order: 30
+---
+
+# Cluster allocation explain
+
+The most basic cluster allocation explain request finds an unassigned shard and explains why it can't be allocated to a node.
+
+If you add some options, you can instead get information on a specific shard, including why OpenSearch assigned it to its current node.
+
+
+## Example
+
+```json
+GET /_cluster/allocation/explain?include_yes_decisions=true
+{
+  "index": "movies",
+  "shard": 0,
+  "primary": true
+}
+```
+
+
+## Path and HTTP methods
+
+```
+GET _cluster/allocation/explain
+POST _cluster/allocation/explain
+```
+
+
+## URL parameters
+
+All cluster allocation explain parameters are optional.
+
+Parameter | Type | Description
+:--- | :--- | :---
+include_yes_decisions | Boolean | OpenSearch makes a series of yes or no decisions when trying to allocate a shard to a node. If this parameter is true, OpenSearch includes the (generally more numerous) "yes" decisions in its response. Default is false.
+include_disk_info | Boolean | Whether to include information about disk usage in the response. Default is false.
+
+
+## Request body
+
+All cluster allocation explain fields are optional.
+
+Field | Type | Description
+:--- | :--- | :---
+current_node | String | If you only want an explanation if the shard happens to be on a particular node, specify that node name here.
+index | String | The name of the shard's index.
+primary | Boolean | Whether to provide an explanation for the primary shard (true) or its first replica (false), which share the same shard ID.
+shard | Integer | The shard ID that you want an explanation for.
+
+
+## Response
+
+```json
+{
+  "index": "movies",
+  "shard": 0,
+  "primary": true,
+  "current_state": "started",
+  "current_node": {
+    "id": "d8jRZcW1QmCBeVFlgOJx5A",
+    "name": "opensearch-node1",
+    "transport_address": "172.24.0.4:9300",
+    "weight_ranking": 1
+  },
+  "can_remain_on_current_node": "yes",
+  "can_rebalance_cluster": "yes",
+  "can_rebalance_to_other_node": "no",
+  "rebalance_explanation": "cannot rebalance as no target node exists that can both allocate this shard and improve the cluster balance",
+  "node_allocation_decisions": [{
+    "node_id": "vRxi4uPcRt2BtHlFoyCyTQ",
+    "node_name": "opensearch-node2",
+    "transport_address": "172.24.0.3:9300",
+    "node_decision": "no",
+    "weight_ranking": 1,
+    "deciders": [{
+        "decider": "max_retry",
+        "decision": "YES",
+        "explanation": "shard has no previous failures"
+      },
+      {
+        "decider": "replica_after_primary_active",
+        "decision": "YES",
+        "explanation": "shard is primary and can be allocated"
+      },
+      {
+        "decider": "enable",
+        "decision": "YES",
+        "explanation": "all allocations are allowed"
+      },
+      {
+        "decider": "node_version",
+        "decision": "YES",
+        "explanation": "can relocate primary shard from a node with version [1.0.0] to a node with equal-or-newer version [1.0.0]"
+      },
+      {
+        "decider": "snapshot_in_progress",
+        "decision": "YES",
+        "explanation": "no snapshots are currently running"
+      },
+      {
+        "decider": "restore_in_progress",
+        "decision": "YES",
+        "explanation": "ignored as shard is not being recovered from a snapshot"
+      },
+      {
+        "decider": "filter",
+        "decision": "YES",
+        "explanation": "node passes include/exclude/require filters"
+      },
+      {
+        "decider": "same_shard",
+        "decision": "NO",
+        "explanation": "a copy of this shard is already allocated to this node [[movies][0], node[vRxi4uPcRt2BtHlFoyCyTQ], [R], s[STARTED], a[id=x8w7QxWdQQa188HKGn0iMQ]]"
+      },
+      {
+        "decider": "disk_threshold",
+        "decision": "YES",
+        "explanation": "enough disk for shard on node, free: [35.9gb], shard size: [15.1kb], free after allocating shard: [35.9gb]"
+      },
+      {
+        "decider": "throttling",
+        "decision": "YES",
+        "explanation": "below shard recovery limit of outgoing: [0 < 2] incoming: [0 < 2]"
+      },
+      {
+        "decider": "shards_limit",
+        "decision": "YES",
+        "explanation": "total shard limits are disabled: [index: -1, cluster: -1] <= 0"
+      },
+      {
+        "decider": "awareness",
+        "decision": "YES",
+        "explanation": "allocation awareness is not enabled, set cluster setting [cluster.routing.allocation.awareness.attributes] to enable it"
+      }
+    ]
+  }]
+}
+```

docs/opensearch/rest-api/index.md

@@ -0,0 +1,11 @@
+---
+layout: default
+title: REST API reference
+nav_order: 99
+parent: OpenSearch
+has_children: true
+---
+
+# REST API reference
+
+OpenSearch uses its REST API for most operations. This _incomplete_ section includes REST API paths, HTTP verbs, supported parameters, request body details, and sample responses.

docs/security/access-control/api.md

@@ -1030,7 +1030,7 @@ GET _opensearch/_security/api/securityconfig
 
 ### Update configuration
 
-Creates or updates the existing configuration using the REST API rather than `securityadmin.sh`. This operation can easily break your existing configuration, so we recommend using `securityadmin.sh` instead. See [Access control for the API](#access-control-for-the-api) for how to enable this operation.
+Creates or updates the existing configuration using the REST API. This operation can easily break your existing configuration, so we recommend using `securityadmin.sh` instead, which is far safer. See [Access control for the API](#access-control-for-the-api) for how to enable this operation.
 
 #### Request
 
@@ -1088,7 +1088,7 @@ PUT _opensearch/_security/api/securityconfig/config
 
 ### Patch configuration
 
-Updates the existing configuration using the REST API rather than `securityadmin.sh`. This operation can easily break your existing configuration, so we recommend using `securityadmin.sh` instead. See [Access control for the API](#access-control-for-the-api) for how to enable this operation.
+Updates the existing configuration using the REST API. This operation can easily break your existing configuration, so we recommend using `securityadmin.sh` instead, which is far safer. See [Access control for the API](#access-control-for-the-api) for how to enable this operation.
 
 #### Request
 

docs/security/access-control/cross-cluster-search.md

@@ -73,7 +73,7 @@ services:
       - cluster.name=opensearch-cluster1
       - discovery.type=single-node
       - bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
-      - "ES_JAVA_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
+      - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
     ulimits:
       memlock:
         soft: -1
@@ -93,7 +93,7 @@ services:
       - cluster.name=opensearch-cluster2
       - discovery.type=single-node
       - bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
-      - "ES_JAVA_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
+      - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
     ulimits:
       memlock:
         soft: -1

docs/security/configuration/client-auth.md

@@ -87,6 +87,7 @@ response = requests.get(url, cert = (cert_file_path, key_file_path), verify=root
 print(response.text)
 ```
 
+{% comment %}
 ## Configuring Beats
 
 You can also configure your Beats so that it uses a client certificate for authentication with OpenSearch. Afterwards, it can start sending output to OpenSearch.
@@ -105,6 +106,7 @@ output.opensearch:
   ssl.certificate: "/full/path/to/client-cert.pem"
   ssl.key: "/full/path/to/to/client-cert-key.pem"
 ```
+{% endcomment %}
 
 ## Using certificates with Docker
 

docs/security/configuration/tls.md

@@ -41,7 +41,9 @@ Name | Description
 
 ## Keystore and truststore files
 
-As an alternative to certificates and private keys in PEM format, you can instead use keystore and truststore files in JKS or PKCS12/PFX format. The following settings configure the location and password of your keystore and truststore files. If you want, you can use different keystore and truststore files for the REST and the transport layer.
+As an alternative to certificates and private keys in PEM format, you can instead use keystore and truststore files in JKS or PKCS12/PFX format. For the security plugin to operate, you need certificates and private keys.
+
+The following settings configure the location and password of your keystore and truststore files. If you want, you can use different keystore and truststore files for the REST and the transport layer.
 
 
 ### Transport layer TLS
@@ -205,4 +207,4 @@ opensearch_security.ssl.http.enabled_protocols:
 
 ## (Advanced) Disable client initiated renegotiation for Java 8
 
-Set `-Djdk.tls.rejectClientInitiatedRenegotiation=true` to disable secure client initiated renegotiation, which is enabled by default. This can be set via `ES_JAVA_OPTS` in `config/jvm.options`.
+Set `-Djdk.tls.rejectClientInitiatedRenegotiation=true` to disable secure client initiated renegotiation, which is enabled by default. This can be set via `OPENSEARCH_JAVA_OPTS` in `config/jvm.options`.

docs/troubleshoot/index.md

@@ -21,11 +21,17 @@ You might see `[ERROR][c.a.o.s.s.t.OpenSearchSecuritySSLNettyTransport] [opensea
 If you encounter the error `FATAL  Error: Request Timeout after 30000ms` during startup, try running OpenSearch Dashboards on a more powerful machine. We recommend four CPU cores and 8 GB of RAM.
 
 
-## Can't open OpenSearch Dashboards on Windows
+## Encryption at rest
 
-OpenSearch Dashboards doesn't support Microsoft Edge and many versions of Internet Explorer. We recommend Firefox or Chrome.
+The operating system for each OpenSearch node handles encryption of data at rest. To enable encryption at rest in most Linux distributions, use the `cryptsetup` command:
 
+```bash
+cryptsetup luksFormat --key-file <key> <partition>
+```
 
+For full documentation on the command, see [the Linux man page](http://man7.org/linux/man-pages/man8/cryptsetup.8.html).
+
+{% comment %}
 ## Beats
 
 If you encounter compatibility issues when attempting to connect Beats to OpenSearch, make sure you're using the Apache 2.0 distribution of Beats, not the default distribution, which uses a proprietary license.
@@ -69,7 +75,7 @@ output {
   }
 }
 ```
-
+{% endcomment %}
 
 ## Can't update by script when FLS, DLS, or field masking is active