added screenshot

Add page for migrating to Dashboards

.gitignore

@@ -4,3 +4,4 @@ _site
 .DS_Store
 Gemfile.lock
 .idea
+.jekyll-cache

_dashboards/install/tls.md

@@ -0,0 +1,41 @@
+---
+layout: default
+title: Configure TLS
+parent: Install OpenSearch Dashboards
+nav_order: 40
+---
+
+# Configure TLS for OpenSearch Dashboards
+
+By default, for ease of testing and getting started, OpenSearch Dashboards runs over HTTP. To enable TLS for HTTPS, update the following settings in `opensearch_dashboards.yml`.
+
+Setting | Description
+:--- | :---
+opensearch.ssl.verificationMode | This setting is for communications between OpenSearch and OpenSearch Dashboards. Valid values are `full`, `certificate`, or `none`. We recommend `full` if you enable TLS, which enables hostname verification. `certificate` just checks the certificate, not the hostname, and `none` performs no checks (suitable for HTTP). Default is `full`.
+opensearch.ssl.certificateAuthorities | If `opensearch.ssl.verificationMode` is `full` or `certificate`, specify the full path (e.g. `[ "/usr/share/opensearch-dashboards-1.0.0/config/root-ca.pem" ]` to the certificate authority for your OpenSearch cluster.
+server.ssl.enabled | This setting is for communications between OpenSearch Dashboards and the web browser. Set to true for HTTPS, false for HTTP.
+server.ssl.certificate | If `server.ssl.enabled` is true, specify the full path (e.g. `/usr/share/opensearch-dashboards-1.0.0/config/my-client-cert.pem` to a valid client certificate for your OpenSearch cluster. You can [generate your own]({{site.url}}{{site.baseurl}}/security-plugin/configuration/generate-certificates/) or get one from a certificate authority.
+server.ssl.key | If `server.ssl.enabled` is true, specify the full path (e.g. `/usr/share/opensearch-dashboards-1.0.0/config/my-client-cert-key.pem` to the key for your client certificate. You can [generate your own]({{site.url}}{{site.baseurl}}/security-plugin/configuration/generate-certificates/) or get one from a certificate authority.
+opensearch_security.cookie.secure | If you enable TLS for OpenSearch Dashboards, change this setting to `true`. For HTTP, set it to `false`.
+
+This `opensearch_dashboards.yml` configuration shows OpenSearch and OpenSearch Dashboards running on the same machine with the demo configuration:
+
+```yml
+opensearch.hosts: ["https://localhost:9200"]
+opensearch.ssl.verificationMode: full
+opensearch.username: "kibanaserver"
+opensearch.password: "kibanaserver"
+opensearch.requestHeadersWhitelist: [ authorization,securitytenant ]
+server.ssl.enabled: true
+server.ssl.certificate: /usr/share/opensearch-1.0.0/config/client-cert.pem
+server.ssl.key: /usr/share/opensearch-1.0.0/config/client-cert-key.pem
+opensearch.ssl.certificateAuthorities: [ "/usr/share/opensearch-1.0.0/config/root-ca.pem" ]
+opensearch_security.multitenancy.enabled: true
+opensearch_security.multitenancy.tenants.preferred: ["Private", "Global"]
+opensearch_security.readonly_mode.roles: ["kibana_read_only"]
+opensearch_security.cookie.secure: true
+```
+
+If you use the Docker install, you can pass a custom `opensearch_dashboards.yml` to the container. To learn more, see the [Docker installation page]({{site.url}}{{site.baseurl}}/opensearch/install/docker/).
+
+After enabling these settings and starting OpenSearch Dashboards, you can connect to it at `https://localhost:5601`. You might have to acknowledge a browser warning if your certificates are self-signed. To avoid this sort of warning (or outright browser incompatibility), best practice is to use certificates from trusted certificate authority.

_dashboards/notebooks.md

@@ -1,28 +1,30 @@
 ---
 layout: default
-title: Notebooks (experimental)
+title: Notebooks
 nav_order: 50
 redirect_from: /docs/notebooks/
 has_children: false
 ---
 
-# OpenSearch Dashboards notebooks (experimental)
+# Notebooks
 
-Notebooks have a known issue with [tenants]({{site.url}}{{site.baseurl}}/security-plugin/access-control/multi-tenancy/). If you open a notebook and can't see its visualizations, you might be under the wrong tenant, or you might not have access to the tenant at all.
+An OpenSearch Dashboards notebook is an interface that lets you easily combine code snippets, live visualizations, and narrative text in a single notebook interface.
-{: .warning }
-
-An OpenSearch Dashboards notebook is an interface that lets you easily combine live visualizations and narrative text in a single notebook interface.
 
 Notebooks let you interactively explore data by running different visualizations that you can share with team members to collaborate on a project.
 
-A notebook is a document composed of two elements: OpenSearch Dashboards visualizations and paragraphs (Markdown). Choose multiple timelines to compare and contrast visualizations.
+A notebook is a document composed of two elements: code blocks (Markdown/SQL/PPL) and visualizations. Choose multiple timelines to compare and contrast visualizations.
+
+You can also generate [reports]({{site.url}}{{site.baseurl}}/dashboards/reporting/) directly from your notebooks.
 
 Common use cases include creating postmortem reports, designing runbooks, building live infrastructure reports, and writing documentation.
 
+Tenants in OpenSearch Dashboards are spaces for saving notebooks and other OpenSearch Dashboards objects. For more information, see [OpenSearch Dashboards multi-tenancy]({{site.url}}{{site.baseurl}}/security-plugin/access-control/multi-tenancy/).
+{: .note }
 
-## Get Started with notebooks
 
-To get started, choose **OpenSearch Dashboards Notebooks** within OpenSearch Dashboards.
+## Get started with notebooks
+
+To get started, choose **Notebooks** within OpenSearch Dashboards.
 
 
 ### Step 1: Create a notebook
@@ -32,28 +34,58 @@ A notebook is an interface for creating reports.
 1. Choose **Create notebook** and enter a descriptive name.
 1. Choose **Create**.
 
-Choose **Notebook actions** to rename, duplicate, or delete a notebook.
+Choose **Actions** to rename, duplicate, or delete a notebook.
 
 
 ### Step 2: Add a paragraph
 
-Paragraphs combine text and visualizations for describing data.
+Paragraphs combine code blocks and visualizations for describing data.
 
+#### Add a code block
 
-#### Add a markdown paragraph
+Code blocks support markdown, SQL, and PPL languages.
 
-1. To add text, choose **Add markdown paragraph**.
+Specify the input language on the first line using `%[language type]` syntax.
-1. Add rich text with markdown syntax.
+For example, type `%md` for markdown, `%sql` for SQL, and `%ppl` for PPL.
+
+##### Sample markdown block
+
+```
+%md
+Add in text formatted in markdown.
+```
 
 ![Markdown paragraph]({{site.url}}{{site.baseurl}}/images/markdown-notebook.png)
 
+##### Sample SQL block
 
-#### Add a visualization paragraph
+```sql
+%sql
+Select * from opensearch_dashboards_sample_data_flights limit 20;
+```
 
-1. To add a visualization, choose **Add OpenSearch Dashboards visualization paragraph**.
+![SQL paragraph]({{site.url}}{{site.baseurl}}/images/sql-notebook.png)
+
+##### Sample PPL block
+
+```
+%ppl
+source=opensearch_dashboards_sample_data_logs | head 20
+```
+
+![PPL paragraph]({{site.url}}{{site.baseurl}}/images/ppl-notebook.png)
+
+
+#### Add a visualization
+
+1. To add a visualization, choose **Add paragraph** and select **Visualization**.
 1. In **Title**, select your visualization and choose a date range. You can choose multiple timelines to compare and contrast visualizations.
 1. To run and save a paragraph, choose **Run**.
 
+![Visualization paragraph]({{site.url}}{{site.baseurl}}/images/visualization-notebook.png)
+
+## Paragraph actions
+
 You can perform the following actions on paragraphs:
 
 - Add a new paragraph to the top of a report.
@@ -62,3 +94,25 @@ You can perform the following actions on paragraphs:
 - Clear the outputs of all paragraphs.
 - Delete all the paragraphs.
 - Move paragraphs up and down.
+
+## Sample notebooks
+
+We prepared the following sample notebooks that showcase a variety of use cases:
+
+- Using SQL to query the OpenSearch Dashboards sample flight data.
+- Using PPL to query the OpenSearch Dashboards sample web logs data.
+- Using PPL and visualizations to perform sample root cause event analysis on the OpenSearch Dashboards sample web logs data.
+
+To add a sample notebook, choose **Actions** and select **Add sample notebooks**.
+
+## Create a report
+
+You can use notebooks to create PNG and PDF reports:
+
+1. From the top menu bar, choose **Reporting actions**.
+1. You can choose to **Download PDF** or **Download PNG**.
+
+   Reports generate asynchronously in the background and might take a few minutes, depending on the size of the report. A notification appears when your report is ready to download.
+
+1. To create a schedule-based report, choose **Create report definition**. For steps to create a report definition, see [Create reports using a definition]({{site.url}}{{site.baseurl}}/dashboards/reporting#create-reports-using-a-definition).
+1. To see all your reports, choose **View all reports**.

_dashboards/reporting.md

@@ -9,6 +9,9 @@ nav_order: 20
 
 You can use OpenSearch Dashboards to create PNG, PDF, and CSV reports. To create reports, you must have the correct permissions. For a summary of the predefined roles and the permissions they grant, see the [security plugin]({{site.url}}{{site.baseurl}}/security-plugin/access-control/users-roles#predefined-roles).
 
+CSV reports have a non-configurable 10,000 row limit. They have no explicit size limit (e.g. in MB), but extremely large documents could cause report generation to fail with an out of memory error from the V8 JavaScript engine.
+{: .tip }
+
 
 ## Create reports from Discovery, Visualize, or Dashboard
 

_opensearch/aggregations.md

@@ -1,7 +1,7 @@
 ---
 layout: default
 title: Aggregations
-nav_order: 13
+nav_order: 14
 has_children: true
 ---
 

_opensearch/data-streams.md

@@ -0,0 +1,301 @@
+---
+layout: default
+title: Data streams
+nav_order: 13
+---
+
+# Data streams
+
+If you're ingesting continuously generated time-series data such as logs, events, and metrics into OpenSearch, you're likely in a scenario where:
+
+- You’re ingesting documents that grow rapidly.
+- You don’t need to update older documents.
+- Your searches generally target the newer documents.
+
+A typical workflow to manage time-series data is as follows:
+
+- To split your data into an index for each day, use the rollover operation.
+- To perform searches on a virtual index name that gets expanded to the underlying indices, create an [index alias]({{site.url}}{{site.baseurl}}/opensearch/index-alias/).
+- To perform a write operation on an index alias, configure the latest index as the write index.
+- To configure new indices, extract common mappings and settings into an [index template]({{site.url}}{{site.baseurl}}/opensearch/index-templates/)).
+
+Even after you perform all these operations, you’re still not enforcing the best practices when dealing with time-series data. For example, you can modify the indices directly. You’re able to ingest documents without a timestamp field, which might result in slower queries.
+
+Data streams abstract the complexity and enforce the best practices for managing time-series data.
+
+With data streams, you can store append-only time-series data across multiple indices with a single endpoint for ingesting and searching data. It replaces index aliases for time-series data.
+
+## About data streams
+
+A data stream consists of one or more hidden auto-generated backing indices. These backing indices are named using the following convention:
+
+```
+.ds-<data-stream-name>-<generation-id>
+```
+
+For example, `.ds-logs-redis-000003`, where generation-id is a six-digit, zero-padded integer that acts as a cumulative count of the data stream’s rollovers, starting at `000001`.
+
+The most recently created backing index is the data stream’s write index. You can’t add documents directly to any of the backing indices. You can only add them via the data stream handle:
+
+![data stream indexing diagram]({{site.url}}{{site.baseurl}}/images/data_stream_indexing.png)
+
+The data stream routes search requests to all of its backing indices. It uses the timestamp field to intelligently route search requests to the right set of indices and shards:
+
+![data stream indexing diagram]({{site.url}}{{site.baseurl}}/images/data_stream_searching.png)
+
+The following operations are not supported on the write index because they might hinder the indexing operation:
+
+- close
+- clone
+- delete
+- shrink
+- split
+
+## Get started with data streams
+
+### Step 1: Create an index template
+
+To create a data stream, you first need to create an index template that configures a set of indices as a data stream. The `data_stream` object indicates that it’s a data stream and not a regular index template. The index pattern matches with the name of the data stream:
+
+```json
+PUT _index_template/logs-template
+{
+  "index_patterns": [
+    "my-data-stream",
+    "logs-*"
+  ],
+  "data_stream": {},
+  "priority": 100
+}
+```
+
+In this case, each ingested document must have an `@timestamp` field.
+You also have the ability to define your own custom timestamp field as a property in the `data_stream` object. You can also add index mappings and other settings here, just as you would for a regular index template.
+
+```json
+PUT _index_template/logs-template-nginx
+{
+  "index_patterns": "logs-nginx",
+  "data_stream": {
+    "timestamp_field": {
+      "name": "request_time"
+    }
+  },
+  "priority": 200,
+  "template": {
+    "settings": {
+      "number_of_shards": 1,
+      "number_of_replicas": 0
+    }
+  }
+}
+```
+
+In this case, `logs-nginx` index matches both the `logs-template` and `logs-template-nginx` templates. When you have a tie, OpenSearch selects the matching index template with the higher priority value.
+
+### Step 2: Create a data stream
+
+After you create an index template, you can create a data stream.
+You can use the data stream API to explicitly create a data stream. The data stream API initializes the first backing index:
+
+```json
+PUT _data_stream/logs-redis
+PUT _data_stream/logs-nginx
+```
+
+You can also directly start ingesting data without creating a data stream.
+
+Because we have a matching index template with a data_stream object, OpenSearch automatically creates the data stream:
+
+```json
+POST logs-staging/_doc
+{
+  "message": "login attempt failed",
+  "@timestamp": "2013-03-01T00:00:00"
+}
+```
+
+To see information about a specific data stream:
+
+```json
+GET _data_stream/logs-nginx
+```
+
+#### Sample response
+
+```json
+{
+  "data_streams" : [
+    {
+      "name" : "logs-nginx",
+      "timestamp_field" : {
+        "name" : "request_time"
+      },
+      "indices" : [
+        {
+          "index_name" : ".ds-logs-nginx-000001",
+          "index_uuid" : "-VhmuhrQQ6ipYCmBhn6vLw"
+        }
+      ],
+      "generation" : 1,
+      "status" : "GREEN",
+      "template" : "logs-template-nginx"
+    }
+  ]
+}
+```
+
+You can see the name of the timestamp field, the list of the backing indices, and the template that's used to create the data stream. You can also see the health of the data stream, which represents the lowest status of all its backing indices.
+
+To see more insights about the data stream, use the `_stats` endpoint:
+
+```json
+GET _data_stream/logs-nginx/_stats
+```
+
+#### Sample response
+
+```json
+{
+  "_shards" : {
+    "total" : 1,
+    "successful" : 1,
+    "failed" : 0
+  },
+  "data_stream_count" : 1,
+  "backing_indices" : 1,
+  "total_store_size_bytes" : 208,
+  "data_streams" : [
+    {
+      "data_stream" : "logs-nginx",
+      "backing_indices" : 1,
+      "store_size_bytes" : 208,
+      "maximum_timestamp" : 0
+    }
+  ]
+}
+```
+
+### Step 3: Ingest data into the data stream
+
+To ingest data into a data stream, you can use the regular indexing APIs. Make sure every document that you index has a timestamp field. If you try to ingest a document that doesn't have a timestamp field, you get an error.
+
+```json
+POST logs-redis/_doc
+{
+  "message": "login attempt",
+  "@timestamp": "2013-03-01T00:00:00"
+}
+```
+
+### Step 4: Searching a data stream
+
+You can search a data stream just like you search a regular index or an index alias.
+The search operation applies to all of the backing indices (all data present in the stream).
+
+```json
+GET logs-redis/_search
+{
+  "query": {
+    "match": {
+      "message": "login"
+    }
+  }
+}
+```
+
+#### Sample response
+
+```json
+{
+  "took" : 514,
+  "timed_out" : false,
+  "_shards" : {
+    "total" : 5,
+    "successful" : 5,
+    "skipped" : 0,
+    "failed" : 0
+  },
+  "hits" : {
+    "total" : {
+      "value" : 1,
+      "relation" : "eq"
+    },
+    "max_score" : 0.2876821,
+    "hits" : [
+      {
+        "_index" : ".ds-logs-redis-000001",
+        "_type" : "_doc",
+        "_id" : "-rhVmXoBL6BAVWH3mMpC",
+        "_score" : 0.2876821,
+        "_source" : {
+          "message" : "login attempt",
+          "@timestamp" : "2013-03-01T00:00:00"
+        }
+      }
+    ]
+  }
+}
+```
+
+### Step 5: Rollover a data stream
+
+A rollover operation creates a new backing index that becomes the data stream’s new write index.
+
+To perform manual rollover operation on the data stream:
+
+```json
+POST logs-redis/_rollover
+```
+
+#### Sample response
+
+```json
+{
+  "acknowledged" : true,
+  "shards_acknowledged" : true,
+  "old_index" : ".ds-logs-redis-000001",
+  "new_index" : ".ds-logs-redis-000002",
+  "rolled_over" : true,
+  "dry_run" : false,
+  "conditions" : { }
+}
+```
+
+If you now perform a `GET` operation on the `logs-redis` data stream, you see that the generation ID is incremented from 1 to 2.
+
+You can also set up an [Index State Management (ISM) policy]({{site.url}}{{site.baseurl}}/ism/policies/) to automate the rollover process for the data stream.  
+The ISM policy is applied to the backing indices at the time of their creation. When you associate a policy to a data stream, it only affects the future backing indices of that data stream.
+
+You also don’t need to provide the `rollover_alias` setting, because the ISM policy infers this information from the backing index.
+
+### Step 6: Manage data streams in OpenSearch Dashboards
+
+To manage data streams from OpenSearch Dashboards, open **OpenSearch Dashboards**, choose **Index Management**, select **Indices** or **Policy managed indices**.
+
+You see a toggle switch for data streams that you can use to show or hide indices belonging to a data stream.
+
+When you enable this switch, you see a data stream multi-select dropdown menu that you can use for filtering data streams.
+You also see a data stream column that shows you the name of the parent data stream the index is contained in.
+
+![data stream toggle]({{site.url}}{{site.baseurl}}/images/data_streams_toggle.png)
+
+You can select one or more data streams and apply an ISM policy on them. You can also apply a policy on any individual backing index.
+
+You can performing visualizations on a data stream just like you would on a regular index or index alias.
+
+### Step 7: Delete a data stream
+
+The delete operation first deletes the backing indices of a data stream and then deletes the data stream itself.
+
+To delete a data stream and all of its hidden backing indices:
+
+```json
+DELETE _data_stream/<name_of_data_stream>
+```
+
+You can use wildcards to delete more than one data stream.
+
+We recommend deleting data from a data stream using an ISM policy.
+
+You can also use [asynchronous search]({{site.url}}{{site.baseurl}}/async/index/) and [SQL]({{site.url}}{{site.baseurl}}/sql/index/) and [PPL]({{site.url}}{{site.baseurl}}/ppl/index/) to query your data stream directly. You can also use the security plugin to define granular permissions on the data stream name.

_opensearch/index-templates.md

@@ -1,23 +1,14 @@
 ---
 layout: default
 title: Index templates
-nav_order: 14
+nav_order: 15
 ---
 
 # Index templates
 
 Index templates let you initialize new indices with predefined mappings and settings. For example, if you continuously index log data, you can define an index template so that all of these indices have the same number of shards and replicas.
 
----
+### Create a template
-
-#### Table of contents
-1. TOC
-{:toc}
-
-
----
-
-## Create a template
 
 To create an index template, use a POST request:
 
@@ -106,7 +97,7 @@ GET logs-2020-01-01
 
 Any additional indices that match this pattern---`logs-2020-01-02`, `logs-2020-01-03`, and so on---will inherit the same mappings and settings.
 
-## Retrieve a template
+### Retrieve a template
 
 To list all index templates:
 
@@ -138,7 +129,7 @@ To check if a specific template exists:
 HEAD _index_template/<name>
 ```
 
-## Configure multiple templates
+### Configure multiple templates
 
 You can create multiple index templates for your indices. If the index name matches more than one template, OpenSearch merges all mappings and settings from all matching templates and applies them to the index.
 
@@ -184,7 +175,7 @@ PUT _index_template/template-02
 
 Because `template-02` has a higher `priority` value, it takes precedence over `template-01` . The `logs-2020-01-02` index would have the `number_of_shards` value as 3.
 
-## Delete a template
+### Delete a template
 
 You can delete an index template using its name:
 
@@ -192,11 +183,225 @@ You can delete an index template using its name:
 DELETE _index_template/daily_logs
 ```
 
+## Composable index templates
+
+Managing multiple index templates has the following challenges:
+
+- If you have duplication between index templates, storing these index templates results in a bigger cluster state.
+- If you want to make a change across all your index templates, you have to manually make the change for each template.
+- If an index matches multiple templates, OpenSearch might merge the templates in an unexpected way that you discover only after an index is created.
+
+You can use composable index templates to overcome these challenges. Composable index templates let you abstract common settings, mappings, and aliases into a reusable building block called a component template.
+
+You can combine component templates to compose an index template.
+
+Settings and mappings that you specify directly in the [create index]({{site.url}}{{site.baseurl}}/opensearch/rest-api/create-index/) request override any settings or mappings specified in an index template and its component templates.
+{: .note }
+
+### Create a component template
+
+Let's define two component templates⁠---`component_template_1` and `component_template_2`:
+
+#### Component template 1
+
+```json
+PUT _component_template/component_template_1
+{
+  "template": {
+    "mappings": {
+      "properties": {
+        "@timestamp": {
+          "type": "date"
+        }
+      }
+    }
+  }
+}
+```
+
+#### Component template 2
+
+```json
+PUT _component_template/component_template_2
+{
+  "template": {
+    "mappings": {
+      "properties": {
+        "ip_address": {
+          "type": "ip"
+        }
+      }
+    }
+  }
+}
+```
+
+### Use component templates to create an index template
+
+When creating index templates, you need to include the component templates in a `composed_of` list.
+
+OpenSearch applies the component templates in the order in which you specify them within the index template. The settings, mappings, and aliases that you specify inside the index template are applied last.
+
+```json
+PUT _index_template/daily_logs
+{
+  "index_patterns": [
+    "logs-2020-01-*"
+  ],
+  "template": {
+    "aliases": {
+      "my_logs": {}
+    },
+    "settings": {
+      "number_of_shards": 2,
+      "number_of_replicas": 1
+    },
+    "mappings": {
+      "properties": {
+        "timestamp": {
+          "type": "date",
+          "format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"
+        },
+        "value": {
+          "type": "double"
+        }
+      }
+    }
+  },
+  "priority": 200,
+  "composed_of": [
+    "component_template_1",
+    "component_template_2"
+  ],
+  "version": 3,
+  "_meta": {
+    "description": "using component templates"
+  }
+}
+```
+
+If you create an index named `logs-2020-01-01`, you can see that it derives its mappings and settings from both the component templates:
+
+```json
+PUT logs-2020-01-01
+GET logs-2020-01-01
+```
+
+#### Sample response
+
+```json
+{
+  "logs-2020-01-01": {
+    "aliases": {
+      "my_logs": {}
+    },
+    "mappings": {
+      "properties": {
+        "@timestamp": {
+          "type": "date"
+        },
+        "ip_address": {
+          "type": "ip"
+        },
+        "timestamp": {
+          "type": "date",
+          "format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"
+        },
+        "value": {
+          "type": "double"
+        }
+      }
+    },
+    "settings": {
+      "index": {
+        "creation_date": "1625382479459",
+        "number_of_shards": "2",
+        "number_of_replicas": "1",
+        "uuid": "rYUlpOXDSUSuZifQLPfa5A",
+        "version": {
+          "created": "7100299"
+        },
+        "provided_name": "logs-2020-01-01"
+      }
+    }
+  }
+}
+```
+
+### Simulate multi-component templates
+
+For index templates composed of multiple component templates, you can simulate applying a new template to verify whether the settings are applied as you expect.
+
+To simulate the settings that would be applied to a specific index name:
+
+```json
+POST _index_template/_simulate_index/<index_name>
+```
+
+To simulate the settings that would be applied from an existing template:
+
+```json
+POST _index_template/_simulate/<index_template>
+```
+
+You can also specify a template definition in the simulate request:
+
+```json
+POST _index_template/_simulate
+{
+  "index_patterns": [
+    "logs-2020-01-*"
+  ],
+  "template": {
+    "settings" : {
+        "index.number_of_shards" : 3
+    }
+  },
+  "composed_of": ["component_template_1", "component_template_2"]
+}
+```
+
+The `_simulate` API returns the final settings, mappings, and aliases that will be applied to indices that match the index pattern. You can also see any overlapping templates whose configuration is superseded by the simulated template body or higher priority templates:
+
+```json
+{
+  "template" : {
+    "settings" : {
+      "index" : {
+        "number_of_shards" : "3"
+      }
+    },
+    "mappings" : {
+      "properties" : {
+        "@timestamp" : {
+          "type" : "date"
+        },
+        "ip_address" : {
+          "type" : "ip"
+        }
+      }
+    },
+    "aliases" : { }
+  },
+  "overlapping" : [
+    {
+      "name" : "daily_logs",
+      "index_patterns" : [
+        "logs-2020-01-*"
+      ]
+    }
+  ]
+}
+```
+
 ## Index template options
 
 You can specify the following template options:
 
 Option | Type | Description | Required
 :--- | :--- | :--- | :---
-`priority` | `Number` | The priority of the index template.  | No
+`template` | `Object` |  Specify index settings, mappings, and aliases. | No
-`create` | `Boolean` | Whether this index template should replace an existing one. | No
+`priority` | `Integer` | The priority of the index template.  | No
+`composed_of` | `String array` |  The names of component templates applied on a new index together with the current template.  | No
+`version` | `Integer` | Specify a version number to simplify template management. Default is `null`. | No
+`_meta ` | `Object` | Specify meta information about the template. | No

_opensearch/install/compatibility.md

@@ -0,0 +1,19 @@
+---
+layout: default
+title: Compatibility
+parent: Install OpenSearch
+nav_order: 2
+---
+
+# Operating system and JVM compatibility
+
+- We recommend installing OpenSearch on RHEL- or Debian-based Linux distributions that use [systemd](https://en.wikipedia.org/wiki/Systemd), such as CentOS, Amazon Linux 2, and Ubuntu (LTS). OpenSearch should work on many Linux distributions, but we only test a handful.
+- The OpenSearch tarball ships with a compatible version of Java in the `jdk` directory. To find its version, run `./jdk/bin/java -version`. For example, the OpenSearch 1.0.0 tarball ships with Java 15 (non-LTS).
+
+{% comment %}`./jdk/bin/java -version` doesn't work on macOS with zsh at the moment, and I have no idea why. Maybe we need a macOS artifact. Regardless, the command works on Amazon Linux 2 with bash and presumably other distros. - aetter{% endcomment %}
+
+  To use a different Java installation, set the `OPENSEARCH_JAVA_HOME` environment variable to the Java install location. We recommend Java 11 (LTS), but OpenSearch also works with Java 8.
+
+OpenSearch version | Compatible Java versions | Recommended operating systems
+:--- | :--- | :---
+1.x | 8, 11 | Red Hat Enterprise Linux 7, 8; CentOS 7, 8; Amazon Linux 2; Ubuntu 16.04, 18.04, 20.04

_opensearch/install/docker.md

@@ -2,7 +2,7 @@
 layout: default
 title: Docker
 parent: Install OpenSearch
-nav_order: 1
+nav_order: 3
 ---
 
 # Docker image
@@ -16,7 +16,7 @@ docker pull opensearchproject/opensearch-dashboards:{{site.opensearch_version}}
 
 To check available versions, see [Docker Hub](https://hub.docker.com/u/opensearchproject).
 
-OpenSearch images use `centos:7` as the base image. If you run Docker locally, we recommend allowing Docker to use at least 4 GB of RAM in **Preferences** > **Resources**.
+OpenSearch images use `amazonlinux:2` as the base image. If you run Docker locally, set Docker to use at least 4 GB of RAM in **Preferences** > **Resources**.
 
 
 ---

_opensearch/logs.md

@@ -11,7 +11,7 @@ The OpenSearch logs include valuable information for monitoring cluster operatio
 - On Docker, OpenSearch writes most logs to the console and stores the remainder in `opensearch/logs/`. The tarball installation also uses `opensearch/logs/`.
 - On most Linux installations, OpenSearch writes logs to `/var/log/opensearch/`.
 
-Logs are available as `.log` (plain text) and `.json` files.
+Logs are available as `.log` (plain text) and `.json` files. Permissions for the OpenSearch logs are `-rw-r--r--` by default, meaning that any user account on the node can read them. You can change this behavior _for each log type_ in `log4j2.properties` using the `filePermissions` option. For example, you might add `appender.rolling.filePermissions = rw-r-----` to change permissions for the JSON server log. For details, see the [Log4j 2 documentation](https://logging.apache.org/log4j/2.x/manual/appenders.html#RollingFileAppender).
 
 
 ## Application logs

_opensearch/rest-api/create-alias.md

@@ -0,0 +1,80 @@
+---
+layout: default
+title: Alias
+parent: REST API reference
+grand_parent: OpenSearch
+nav_order: 8
+---
+
+# Alias
+
+An alias is a virtual pointer that you can use to reference one or more indices. Creating and updating aliases are atomic operations, so you can reindex your data and point an alias at it without any downtime.
+
+
+## Example
+
+```json
+POST _aliases
+{
+  "actions": [
+    {
+      "add": {
+        "index": "movies",
+        "alias": "movies-alias1"
+      }
+    },
+    {
+      "remove": {
+        "index": "old-index",
+        "alias": "old-index-alias"
+      }
+    }
+
+  ]
+}
+```
+
+## Path and HTTP methods
+
+```
+POST _aliases
+```
+
+## URL parameters
+
+All alias parameters are optional.
+
+Parameter | Data Type | Description
+:--- | :--- | :---
+master_timeout | Time | The amount of time to wait for a response from the master node. Default is `30s`.
+timeout | Time | The amount of time to wait for a response from the cluster. Default is `30s`.
+
+## Request body
+
+In your request body, you need to specify what action to take, the alias name, and the index you want to associate with the alias. Other fields are optional.
+
+Field | Data Type | Description | Required
+:--- | :--- | :--- | :---
+actions | Array | Set of actions you want to perform on the index. Valid options are: `add`, `remove`, and `remove_index`. You must have at least one action in the array. | Yes
+add | N/A | Adds an alias to the specified index. | No
+remove | N/A | Removes an alias from the specified index. | No
+remove_index | N/A | Deletes an index. | No
+index | String | Name of the index you want to associate with the alias. Supports wildcard expressions. | Yes if you don't supply an `indices` field in the body.
+indices | Array | Array of index names you want to associate with the alias. | Yes if you don't supply an `index` field in the body.
+alias | String | The name of the alias. | Yes if you don't supply an `aliases` field in the body.
+aliases | Array | Array of alias names. | Yes if you don't supply an `alias` field in the body.
+filter | Object | A filter to use with the alias, so the alias points to a filtered part of the index. | No
+is_hidden | Boolean | Specifies whether the alias should be hidden from results that include wildcard expressions | No
+must_exist | Boolean | Specifies whether the alias to remove must exist. | No
+is_write_index | Boolean | Specifies whether the index should be a write index. An alias can only have one write index at a time. | No
+routing | String | Used to assign a custom value to a shard for specific operations. | No
+index_routing | String | Assigns a custom value to a shard only for index operations. | No
+search_routing | String | Assigns a custom value to a shard only for search operations. | No
+
+## Response
+
+```json
+{
+    "acknowledged": true
+}
+```

_opensearch/rest-api/create-index.md

@@ -37,6 +37,12 @@ PUT /sample-index1
 }
 ```
 
+## Path and HTTP methods
+
+```
+PUT <index-name>
+```
+
 ## Index naming restrictions
 
 OpenSearch indices have the following naming restrictions:

_opensearch/rest-api/document-apis/bulk.md

@@ -1,8 +1,9 @@
 ---
 layout: default
 title: Bulk
-parent: REST API reference
+parent: Document APIs
-nav_order: 5
+grand_parent: REST API reference
+nav_order: 20
 ---
 
 # Bulk
@@ -32,7 +33,7 @@ POST _bulk
 POST {index}/_bulk
 ```
 
-Specifying the index in the path means you don't need to include it in the [request body](#request-body).
+Specifying the index in the path means you don't need to include it in the [request body]({{site.url}}{{site.baseurl}}/opensearch/rest-api/document-apis/bulk/#request-body).
 
 OpenSearch also accepts PUT requests to the `_bulk` path, but we highly recommend using POST. The accepted usage of PUT---adding or replacing a single resource at a given path---doesn't make sense for bulk requests.
 {: .note }

_opensearch/rest-api/document-apis/get-documents.md

@@ -0,0 +1,77 @@
+---
+layout: default
+title: Get document
+parent: Document APIs
+grand_parent: REST API reference
+nav_order: 5
+---
+
+# Get document
+
+After adding a JSON document to your index, you can use the get document API operation to retrieve the document's information and data.
+
+## Example
+
+```json
+GET sample-index1/_doc/1
+```
+
+## Path and HTTP methods
+
+```
+GET <index>/_doc/<_id>
+HEAD <index>/_doc/<_id>
+```
+```
+GET <index>/_source/<_id>
+HEAD <index>/_source/<_id>
+```
+
+## URL parameters
+
+All get document URL parameters are optional.
+
+Parameter | Type | Description
+:--- | :--- | :---
+preference | string | Specifies a preference of which shard to retrieve results from. Available options are `_local`, which tells the operation to retrieve results from a locally allocated shard replica, and a custom string value assigned to a specific shard replica. By default, OpenSearch executes get document operations on random shards.
+realtime | boolean | Specifies whether the operation should run in realtime. If false, the operation waits for the index to refresh to analyze the source to retrieve data, which makes the operation near-realtime. Default is true.
+refresh | boolean | If true, OpenSearch refreshes shards to make the operation visible to searching. Default is false.
+routing | string | A value used to route the operation to a specific shard.
+stored_fields | boolean | If true, the operation retrieves document fields stored in the index rather than the document's `_source`. Default is false.
+_source | string | Whether to include the `_source` field in the response body. Default is true.
+_source_excludes | string | A comma-separated list of source fields to exclude in the query response.
+_source_includes | string | A comma-separated list of source fields to include in the query response.
+version | integer | The version of the document to return, which must match the current version of the document.
+version_type | enum | Retrieves a specifically typed document. Available options are `external` (retrieve the document if the specified version number is greater than the document's current version) and `external_gte` (retrieve the document if the specified version number is greater than or equal to the document's current verison). For example, to retrieve version 3 of a document, use `/_doc/1?version=3&version_type=external`.
+
+
+## Response
+```json
+{
+  "_index": "sample-index1",
+  "_type": "_doc",
+  "_id": "1",
+  "_version": 1,
+  "_seq_no": 0,
+  "_primary_term": 9,
+  "found": true,
+  "_source": {
+    "text": "This is just some sample text."
+  }
+}
+```
+
+## Response body fields
+
+Field | Description
+:--- | :---
+_index | The name of the index.
+_type | The document's type. OpenSearch only supports one type, which is `_doc`.
+_id | The document's id.
+_version | The document's version number. Updated whenever the document changes.
+_seq_no | The sequnce number assigned when the document is indexed.
+primary_term | The primary term assigned when the document is indexed.
+found | Whether the document exists.
+_routing | The shard that the document is routed to. If the document is not routed to a particular shard, this field is omitted.
+_source | Contains the document's data if `found` is true. If `_source` is set to false or `stored_fields` is set to true in the URL parameters, this field is omitted.
+_fields | Contains the document's data that's stored in the index. Only returned if both `stored_fields` and `found` are true.

_opensearch/rest-api/document-apis/index.md

@@ -0,0 +1,28 @@
+---
+layout: default
+title: Document APIs
+parent: REST API reference
+has_children: true
+nav_order: 7
+---
+
+# Document APIs
+
+The document APIs allow you to handle documents relative to your index, such as adding, updating, and deleting documents.
+
+Document APIs are separated into two categories: single document operations and multi-document operations. Multi-document operations offer performance advantages over submitting many individual requests, so whenever practical, we recommend that you use multi-document operations.
+
+## Single document operations
+
+- Index
+- Get
+- Delete
+- Update
+
+## Multi-document operations
+
+- Bulk
+- Multi get
+- Delete by query
+- Update by query
+- Reindex

_search-plugins/sql/settings.md

@@ -21,7 +21,7 @@ PUT _cluster/settings
 }
 ```
 
-Similarly, you can also update the settings by sending the request to the plugin setting endpoing `_plugins/_query/setting`:
+Similarly, you can also update the settings by sending the request to the plugin setting endpoint `_plugins/_query/setting`:
 ```json
 PUT _plugins/_query/settings
 {
@@ -37,4 +37,4 @@ Setting | Default | Description
 `plugins.sql.slowlog` | 2 seconds | Configure the time limit (in seconds) for slow queries. The plugin logs slow queries as `Slow query: elapsed=xxx (ms)` in `opensearch.log`.
 `plugins.sql.cursor.keep_alive` | 1 minute | This value configures how long the cursor context is kept open. Cursor contexts are resource heavy, so we recommend a low value.
 `plugins.query.memory_limit` | 85% | This setting configures the heap memory usage limit for the circuit breaker of the query engine.
-`plugins.query.size_limit` | 200 | The setting sets the default size of index that the query engine fetches from OpenSearch.
+`plugins.query.size_limit` | 200 | The setting sets the default size of index that the query engine fetches from OpenSearch.

_security-plugin/configuration/disable.md

@@ -38,7 +38,7 @@ If you disable the security plugin in `opensearch.yml` (or delete the plugin ent
 
    ```
    FROM opensearchproject/opensearch-dashboards:{{site.opensearch_version}}
-   RUN /usr/share/opensearch-dashboards/bin/opensearch-dashboards-plugin remove opensearchSecurityOpenSearch Dashboards
+   RUN /usr/share/opensearch-dashboards/bin/opensearch-dashboards-plugin remove security-dashboards
    COPY --chown=opensearch-dashboards:opensearch-dashboards opensearch_dashboards.yml /usr/share/opensearch-dashboards/config/
    ```
 

_security-plugin/configuration/index.md

@@ -13,12 +13,13 @@ redirect_from:
 
 The plugin includes demo certificates so that you can get up and running quickly, but before using OpenSearch in a production environment, you must configure it manually:
 
-1. [Replace the demo certificates]({{site.url}}{{site.baseurl}}/opensearch/install/docker-security)
+1. [Replace the demo certificates]({{site.url}}{{site.baseurl}}/opensearch/install/docker-security).
-1. [Reconfigure opensearch.yml to use your certificates]({{site.url}}{{site.baseurl}}/security-plugin/configuration/tls)
+1. [Reconfigure opensearch.yml to use your certificates]({{site.url}}{{site.baseurl}}/security-plugin/configuration/tls).
-1. [Reconfigure config.yml to use your authentication backend]({{site.url}}{{site.baseurl}}/security-plugin/configuration/configuration/) (if you don't plan to use the internal user database)
+1. [Reconfigure config.yml to use your authentication backend]({{site.url}}{{site.baseurl}}/security-plugin/configuration/configuration/) (if you don't plan to use the internal user database).
-1. [Modify the configuration YAML files]({{site.url}}{{site.baseurl}}/security-plugin/configuration/yaml)
+1. [Modify the configuration YAML files]({{site.url}}{{site.baseurl}}/security-plugin/configuration/yaml).
-1. [Apply changes using securityadmin.sh]({{site.url}}{{site.baseurl}}/security-plugin/configuration/security-admin)
+1. If you plan to use the internal user database, [set a password policy in opensearch.yml]({{site.url}}{{site.baseurl}}/security-plugin/configuration/yaml/#opensearchyml).
+1. [Apply changes using securityadmin.sh]({{site.url}}{{site.baseurl}}/security-plugin/configuration/security-admin).
 1. Start OpenSearch.
-1. [Add users, roles, role mappings, and tenants]({{site.url}}{{site.baseurl}}/security-plugin/access-control/index/)
+1. [Add users, roles, role mappings, and tenants]({{site.url}}{{site.baseurl}}/security-plugin/access-control/index/).
 
 If you don't want to use the plugin, see [Disable security]({{site.url}}{{site.baseurl}}/security-plugin/configuration/disable).

_security-plugin/configuration/yaml.md

@@ -89,6 +89,42 @@ snapshotrestore:
   description: "Demo snapshotrestore user"
 ```
 
+## opensearch.yml
+
+In addition to many OpenSearch settings, this file contains paths to TLS certificates and their attributes, such as distinguished names and trusted certificate authorities.
+
+```yml
+plugins.security.ssl.transport.pemcert_filepath: esnode.pem
+plugins.security.ssl.transport.pemkey_filepath: esnode-key.pem
+plugins.security.ssl.transport.pemtrustedcas_filepath: root-ca.pem
+plugins.security.ssl.transport.enforce_hostname_verification: false
+plugins.security.ssl.http.enabled: true
+plugins.security.ssl.http.pemcert_filepath: esnode.pem
+plugins.security.ssl.http.pemkey_filepath: esnode-key.pem
+plugins.security.ssl.http.pemtrustedcas_filepath: root-ca.pem
+plugins.security.allow_unsafe_democertificates: true
+plugins.security.allow_default_init_securityindex: true
+plugins.security.authcz.admin_dn:
+  - CN=kirk,OU=client,O=client,L=test, C=de
+
+plugins.security.audit.type: internal_opensearch
+plugins.security.enable_snapshot_restore_privilege: true
+plugins.security.check_snapshot_restore_write_privileges: true
+plugins.security.restapi.roles_enabled: ["all_access", "security_rest_api_access"]
+plugins.security.system_indices.enabled: true
+plugins.security.system_indices.indices: [".opendistro-alerting-config", ".opendistro-alerting-alert*", ".opendistro-anomaly-results*", ".opendistro-anomaly-detector*", ".opendistro-anomaly-checkpoints", ".opendistro-anomaly-detection-state", ".opendistro-reports-*", ".opendistro-notifications-*", ".opendistro-notebooks", ".opendistro-asynchronous-search-response*"]
+node.max_local_storage_nodes: 3
+```
+
+If you want to run your users' passwords against some validation, specify a regular expression (regex) in this file. You can also include an error message that loads when passwords don't pass validation. The following example demonstrates how to include a regex so OpenSearch requires new passwords to be a minimum of eight characters with at least one uppercase, one lowercase, one digit, and one special character.
+
+Note that OpenSearch validates only users and passwords created through OpenSearch Dashboards or the REST API.
+
+```yml
+plugins.restapi.password_validation_regex: '(?=.*[A-Z])(?=.*[^a-zA-Z\d])(?=.*[0-9])(?=.*[a-z]).{8,}'
+plugins.restapi.password_validation_error_message: "Password must be minimum 8 characters long and must contain at least one uppercase letter, one lowercase letter, one digit, and one special character."
+```
+
 
 ## roles.yml
 

_upgrade-to/dashboards-upgrade-to.md

@@ -0,0 +1,36 @@
+---
+layout: default
+title: Upgrade from Kibana OSS to OpenSearch Dashboards
+nav_order: 50
+redirect_from:
+  - /migrate/dashboards/
+---
+
+# Upgrade from Kibana OSS to OpenSearch Dashboards
+
+Kibana OSS stores its visualizations and dashboards in one or more indices (`.kibana*`) on the Elasticsearch OSS cluster. As such, the most important step is to leave those indices intact as you upgrade from Elasticsearch OSS to OpenSearch.
+
+Consider exporting all Kibana objects prior to starting the upgrade. In Kibana, choose **Stack Management**, **Saved Objects**, **Export objects**.
+{: .tip }
+
+1. After you upgrade your Elasticsearch OSS cluster to OpenSearch, stop Kibana.
+
+1. Extract the OpenSearch Dashboards tarball to a new directory.
+
+1. Port your settings from `<kibana-dir>/config/kibana.yml` to `<dashboards-dir>/config/opensearch_dashboards.yml`.
+
+   In general, settings with `elasticsearch` in their names map to `opensearch` (e.g. `elasticsearch.shardTimeout` and `opensearch.shardTimeout`) and settings with `kibana` in their names map to `opensearchDashboards` (e.g. `kibana.defaultAppId` and `opensearchDashboards.defaultAppId`). Most other settings use the same names.
+
+   For a full list of OpenSearch Dashboards settings, see [here](https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/config/opensearch_dashboards.yml){:target='\_blank'}.
+
+1. If your OpenSearch cluster uses the security plugin, preserve and modify the default settings in `opensearch_dashboards.yml`, particularly `opensearch.username` and `opensearch.password`.
+
+   If you disabled the security plugin on your OpenSearch cluster, remove or comment out all `opensearch_security` settings. Then run `rm -rf plugins/security-dashboards/` to remove the security plugin.
+
+1. Start OpenSearch Dashboards:
+
+   ```
+   ./bin/opensearch-dashboards
+   ```
+
+1. Log in, and verify that your saved searches, visualizations, and dashboards are present.

index.md

@@ -37,7 +37,7 @@ Component | Purpose
 [Anomaly Detection]({{site.url}}{{site.baseurl}}/monitoring-plugins/ad/) | Identify atypical data and receive automatic notifications
 [Asynchronous Search]({{site.url}}{{site.baseurl}}/search-plugins/async/) | Run search requests in the background
 
-You can install OpenSearch plugins [individually]({{site.url}}{{site.baseurl}}/opensearch/install/plugins/) or use the [all-in-one packages]({{site.url}}{{site.baseurl}}/opensearch/install/). Most of these OpenSearch plugins have corresponding OpenSearch Dashboards plugins that provide a convenient, unified user interface.
+Most of OpenSearch plugins have a corresponding OpenSearch Dashboards plugin that provide a convenient, unified user interface.
 
 For specifics around the project, see the [FAQ](https://opensearch.org/faq/).
 
@@ -65,7 +65,10 @@ Docker
 To learn more, see [Install and configure OpenSearch]({{site.url}}{{site.baseurl}}/opensearch/install/) and [Install and configure OpenSearch Dashboards]({{site.url}}{{site.baseurl}}/dashboards/install/).
 
 
----
+## The secure path forward
+
+OpenSearch includes a demo configuration so that you can get up and running quickly, but before using OpenSearch in a production environment, you must [configure the security plugin manually]({{site.url}}{{site.baseurl}}/security-plugin/configuration/index/): your own certificates, your own authentication method, your own users, and your own passwords.
+
 
 ## Looking for the Javadoc?