Compare commits

..
Author SHA1 Message Date
Miki 12c82932ca Merge pull request #93 from AMoo-Miki/redirect-to-new-home
Redirect to opensearch.org/docs for all requests: fix root
2021-07-12 15:19:53 -07:00
Miki 78f0f6c288 Redirect to opensearch.org/docs for all requests: fix root
Signed-off-by: Miki <[email protected]>
2021-07-12 15:17:00 -07:00
Miki 1a88ceb40a Merge pull request #92 from AMoo-Miki/redirect-to-new-home
Redirect to opensearch.org/docs for all requests
2021-07-12 15:10:58 -07:00
Miki 8a15e82845 Redirect to opensearch.org/docs for all requests
Signed-off-by: Miki <[email protected]>
2021-07-12 15:02:50 -07:00
Keith Chan 45c8214c2b Merge pull request #78 from opensearch-project/ga
Basic changes for 1.0.0
2021-07-12 13:49:39 -07:00
Keith Chan 4841a9985c Merge pull request #77 from opensearch-project/client-version-compat
Client version compat
2021-07-12 13:45:53 -07:00
aetter 49a765d0a5 Remove beta banner 2021-07-12 13:29:11 -07:00
Ashwin Kumar 67423ff9c2 Merge pull request #85 from opensearch-project/data_stream_typo
fixed typo
2021-07-12 10:22:39 -07:00
ashwinkumar12345 5a4d4dabba fixed typo 2021-07-12 10:15:32 -07:00
Ashwin Kumar 05d6b6cb5d Merge pull request #84 from opensearch-project/data_streams
added data streams
2021-07-12 10:12:19 -07:00
ashwinkumar12345 6e9249e58c added screenshot 2021-07-12 10:10:21 -07:00
Keith Chan 432005eaa0 Merge pull request #80 from opensearch-project/java-high-rest-client
Added how to connect Java high-level REST client to OpenSearch
2021-07-12 09:50:41 -07:00
aetter 3d3bc88422 Update dashboards-upgrade-to.md 2021-07-12 09:29:49 -07:00
ashwinkumar12345 75a835f29b incorporated feedback 2021-07-12 09:16:13 -07:00
keithhc2 d00637b05e Added a comment 2021-07-12 09:02:11 -07:00
ashwinkumar12345 73667cafd0 added data streams 2021-07-12 06:32:23 -07:00
aetter be8d244d8c Update version-history.md 2021-07-11 18:20:39 -07:00
aetter 1e031e8362 Merge branch 'main' into ga 2021-07-11 18:19:26 -07:00
Andrew Etter 59969664a5 Merge pull request #74 from opensearch-project/dashboards-migration
Add page for migrating to Dashboards
2021-07-11 18:18:46 -07:00
Andrew Etter 3c27d762ef Merge pull request #83 from opensearch-project/security-issues
Add SSL configuration for Dashboards and other known issues
2021-07-11 18:17:07 -07:00
aetter 47bd70cb2b Update logs.md 2021-07-11 18:14:16 -07:00
aetter b8bf7a2f1d Add link recommending password policy 2021-07-11 18:10:36 -07:00
aetter 6c6c9c619f Merge branch 'main' into security-issues 2021-07-11 18:02:29 -07:00
Andrew Etter a335f8741a Merge pull request #62 from opensearch-project/compat
Add Java and OS recommendations
2021-07-09 16:35:50 -07:00
aetter c0871cfe0d Update _config.yml 2021-07-09 16:30:46 -07:00
aetter 92b3b49ee2 Update tls.md 2021-07-09 15:59:58 -07:00
aetter cf769012ca Change file name, add some extra bits 2021-07-09 15:59:42 -07:00
Keith Chan 4a2e1c32db Merge pull request #82 from opensearch-project/password-validation
Password validation
2021-07-09 15:44:04 -07:00
aetter 79b647ee62 Add SSL configuration for Dashboards 2021-07-09 15:33:35 -07:00
Keith Chan e5bec0b980 Yet another tweak 2021-07-09 15:09:57 -07:00
keithhc2 d3d9774eaf Another minor tweak 2021-07-09 15:06:47 -07:00
keithhc2 e484b3e93f Minor language fix 2021-07-09 15:05:09 -07:00
keithhc2 d7ce813388 Added opensearch.yml and password regex rules 2021-07-09 15:03:41 -07:00
aetter 18e74399fa Walk back some changes since they're in another PR 2021-07-09 13:31:12 -07:00
Ashwin Kumar 81bf0b2655 Merge pull request #79 from opensearch-project/notebooks_update
Add SQL and PPL features to notebooks
2021-07-09 12:58:25 -07:00
keithhc2 21b4c3ab87 Addressed comments 2021-07-09 12:56:54 -07:00
keithhc2 24acbc3d56 Added how to connect Java high-level REST client to OpenSearch 2021-07-08 16:31:32 -07:00
Andrew Etter 62491a2a98 Merge pull request #73 from opensearch-project/document-apis
Added get document API and some reorganization
2021-07-08 12:16:41 -07:00
aetter faa347c20f Basic changes for 1.0.0 2021-07-08 11:47:37 -07:00
aetter 31ebf2634d Update index.md 2021-07-08 10:55:39 -07:00
aetter b81261034b Add example version, correct command 2021-07-08 10:47:51 -07:00
aetter 610c958b2d Add cluster setting for client version checks 2021-07-08 09:36:30 -07:00
aetter 5d5dc6c413 Add page for migrating to Dashboards 2021-07-07 16:23:47 -07:00
keithhc2 b79ee7cf34 Some reorganized and added get document API 2021-07-07 14:26:50 -07:00
aetter 29feadfb57 Update compatibility.md 2021-06-28 14:49:15 -07:00
aetter 93409f1333 Add Java and OS recommendations 2021-06-23 09:45:20 -07:00
243 changed files with 9 additions and 35316 deletions
+9
View File
@@ -0,0 +1,9 @@
<!DOCTYPE html>
<html lang="en-US">
<meta charset="utf-8">
<title>Redirecting&hellip;</title>
<script>location="https://opensearch.org/docs"+location.pathname</script>
<meta name="robots" content="noindex">
<h1>Redirecting&hellip;</h1>
<a href="https://opensearch.org/docs">Click here if you are not redirected.</a>
</html>
-3
View File
@@ -1,3 +0,0 @@
---
permalink: /404.html
---
-4
View File
@@ -1,4 +0,0 @@
## Code of Conduct
This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct).
For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact
[email protected] with any additional questions or comments.
-59
View File
@@ -1,59 +0,0 @@
# Contributing Guidelines
Thank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional
documentation, we greatly value feedback and contributions from our community.
Please read through this document before submitting any issues or pull requests to ensure we have all the necessary
information to effectively respond to your bug report or contribution.
## Reporting Bugs/Feature Requests
We welcome you to use the GitHub issue tracker to report bugs or suggest features.
When filing an issue, please check existing open, or recently closed, issues to make sure somebody else hasn't already
reported the issue. Please try to include as much information as you can. Details like these are incredibly useful:
* A reproducible test case or series of steps
* The version of our code being used
* Any modifications you've made relevant to the bug
* Anything unusual about your environment or deployment
## Contributing via Pull Requests
Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that:
1. You are working against the latest source on the *main* branch.
2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already.
3. You open an issue to discuss any significant work - we would hate for your time to be wasted.
To send us a pull request, please:
1. Fork the repository.
2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change.
3. Ensure local tests pass.
4. Commit to your fork using clear commit messages.
5. Send us a pull request, answering any default questions in the pull request interface.
6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation.
GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and
[creating a pull request](https://help.github.com/articles/creating-a-pull-request/).
## Finding contributions to work on
Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any 'help wanted' issues is a great place to start.
## Code of Conduct
This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct).
For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact
[email protected] with any additional questions or comments.
## Security issue notifications
If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue.
## Licensing
See the [LICENSE](LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution.
-32
View File
@@ -1,32 +0,0 @@
source "https://rubygems.org"
# Hello! This is where you manage which Jekyll version is used to run.
# When you want to use a different version, change it below, save the
# file and run `bundle install`. Run Jekyll with `bundle exec`, like so:
#
# bundle exec jekyll serve
#
# This will help ensure the proper Jekyll version is running.
# Happy Jekylling!
# gem "jekyll", "~> 3.9.0"
# This is the default theme for new Jekyll sites. You may change this to anything you like.
gem "just-the-docs", "~> 0.3.3"
# If you want to use GitHub Pages, remove the "gem "jekyll"" above and
# uncomment the line below. To upgrade, run `bundle update github-pages`.
gem 'github-pages', group: :jekyll_plugins
# If you have any plugins, put them here!
# group :jekyll_plugins do
# # gem "jekyll-feed", "~> 0.6"
# gem "jekyll-remote-theme"
# gem "jekyll-redirect-from"
# end
# Windows does not include zoneinfo files, so bundle the tzinfo-data gem
gem "tzinfo-data", platforms: [:mingw, :mswin, :x64_mingw, :jruby]
# Performance-booster for watching directories on Windows
gem "wdm", "~> 0.1.0" if Gem.win_platform?
-276
View File
@@ -1,276 +0,0 @@
# OpenSearch documentation
This repository contains the documentation for OpenSearch, the search, analytics, and visualization suite with advanced security, alerting, SQL support, automated index management, deep performance analysis, and more. You can find the rendered documentation at [docs-beta.opensearch.org](https://docs-beta.opensearch.org).
Community contributions remain essential in keeping this documentation comprehensive, useful, well-organized, and up-to-date.
## How you can help
- Do you work on one of the various OpenSearch plugins? Take a look at the documentation for the plugin. Is everything accurate? Will anything change in the near future?
Often, engineering teams can keep existing documentation up-to-date with minimal effort, thus freeing up the documentation team to focus on larger projects.
- Do you have expertise in a particular area of OpenSearch? Cluster sizing? The query DSL? Painless scripting? Aggregations? JVM settings? Take a look at the [current content](https://docs-beta.opensearch.org/docs/opensearch/) and see where you can add value. The [documentation team](#points-of-contact) is happy to help you polish and organize your drafts.
- Are you an OpenSearch Dashboards expert? How did you set up your visualizations? Why is a particular dashboard so valuable to your organization? We have [very little](https://docs-beta.opensearch.org/docs/opensearch-dashboards/) on how to use OpenSearch Dashboards, only how to install it.
- Are you a web developer? Do you want to add an optional dark mode to the documentation? A "copy to clipboard" button for our code samples? Other improvements to the design or usability? See [major changes](#major-changes) for information on building the website locally.
- Our [issue tracker](https://github.com/opensearch-project/documentation-website/issues) contains documentation bugs and other content gaps, some of which have colorful labels like "good first issue" and "help wanted."
## Points of contact
If you encounter problems or have questions when contributing to the documentation, these people can help:
- [aetter](https://github.com/aetter)
- [ashwinkumar12345](https://github.com/ashwinkumar12345)
- [keithhc2](https://github.com/keithhc2)
- [snyder114](https://github.com/snyder114)
## How we build the website
After each commit to this repository, GitHub Pages automatically uses [Jekyll](https://jekyllrb.com) to rebuild the [website](https://docs-beta.opensearch.org). The whole process takes around 30 seconds.
This repository contains many [Markdown](https://guides.github.com/features/mastering-markdown/) files in the `/docs` directory. Each Markdown file correlates with one page on the website.
Using plain text on GitHub has many advantages:
- Everything is free, open source, and works on every operating system. Use your favorite text editor, Ruby, Jekyll, and Git.
- Markdown is easy to learn and looks good in side-by-side diffs.
- The workflow is no different than contributing code. Make your changes, build locally to check your work, and submit a pull request. Reviewers check the PR before merging.
- Alternatives like wikis and WordPress are full web applications that require databases and ongoing maintenance. They also have inferior versioning and content review processes compared to Git. Static websites, such as the ones Jekyll produces, are faster, more secure, and more stable.
In addition to the content for a given page, each Markdown file contains some Jekyll [front matter](https://jekyllrb.com/docs/front-matter/). Front matter looks like this:
```
---
layout: default
title: Alerting security
nav_order: 10
parent: Alerting
has_children: false
---
```
If you're making [trivial changes](#trivial-changes), you don't have to worry about front matter.
If you want to reorganize content or add new pages, keep an eye on `has_children`, `parent`, and `nav_order`, which define the hierarchy and order of pages in the lefthand navigation. For more information, see the documentation for [our upstream Jekyll theme](https://pmarsceill.github.io/just-the-docs/docs/navigation-structure/).
## Contribute content
There are three ways to contribute content, depending on the magnitude of the change.
- [Trivial changes](#trivial-changes)
- [Minor changes](#minor-changes)
- [Major changes](#major-changes)
### Trivial changes
If you just need to fix a typo or add a sentence, this web-based method works well:
1. On any page in the documentation, click the **Edit this page** link in the lower-left.
1. Make your changes.
1. Choose **Create a new branch for this commit and start a pull request** and **Commit changes**.
### Minor changes
If you want to add a few paragraphs across multiple files and are comfortable with Git, try this approach:
1. Fork this repository.
1. Download [GitHub Desktop](https://desktop.github.com), install it, and clone your fork.
1. Navigate to the repository root.
1. Create a new branch.
1. Edit the Markdown files in `/docs`.
1. Commit, push your changes to your fork, and submit a pull request.
### Major changes
If you're making major changes to the documentation and need to see the rendered HTML before submitting a pull request, here's how to build locally:
1. Fork this repository.
1. Download [GitHub Desktop](https://desktop.github.com), install it, and clone your fork.
1. Navigate to the repository root.
1. Install [Ruby](https://www.ruby-lang.org/en/) if you don't already have it. We recommend [RVM](https://rvm.io/), but use whatever method you prefer:
```
curl -sSL https://get.rvm.io | bash -s stable
rvm install 2.6
ruby -v
```
1. Install [Jekyll](https://jekyllrb.com/) if you don't already have it:
```
gem install bundler jekyll
```
1. Install dependencies:
```
bundle install
```
1. Build:
```
sh build.sh
```
1. If the build script doesn't automatically open your web browser (it should), open [http://localhost:4000/](http://localhost:4000/).
1. Create a new branch.
1. Edit the Markdown files in `/docs`.
If you're a web developer, you can customize `_layouts/default.html` and `_sass/custom/custom.scss`.
1. When you save a file, marvel as Jekyll automatically rebuilds the site and refreshes your web browser. This process takes roughly 30 seconds.
1. When you're happy with how everything looks, commit, push your changes to your fork, and submit a pull request.
## Writing tips
1. Try to stay consistent with existing content and consistent within your new content. Don't call the same plugin KNN, k-nn, and k-NN in three different places.
1. Shorter paragraphs are better than longer paragraphs. Use headers, tables, lists, and images to make your content easier for readers to scan.
1. Use **bold** for user interface elements, *italics* for key terms or emphasis, and `monospace` for Bash commands, file names, REST paths, and code.
1. Markdown file names should be all lowercase, use hyphens to separate words, and end in `.md`.
1. Avoid future tense. Use present tense.
**Bad**: After you click the button, the process will start.
**Better**: After you click the button, the process starts.
1. "You" refers to the person reading the page. "We" refers to the OpenSearch contributors.
**Bad**: Now that we've finished the configuration, we have a working cluster.
**Better**: At this point, you have a working cluster, but we recommend adding dedicated master nodes.
1. Don't use "this" and "that" to refer to something without adding a noun.
**Bad**: This can cause high latencies.
**Better**: This additional loading time can cause high latencies.
1. Use active voice.
**Bad**: After the request is sent, the data is added to the index.
**Better**: After you send the request, the OpenSearch cluster indexes the data.
1. Introduce acronyms before using them.
**Bad**: Reducing customer TTV should accelerate our ROIC.
**Better**: Reducing customer time to value (TTV) should accelerate our return on invested capital (ROIC).
1. Spell out one through nine. Start using numerals at 10. If a number needs a unit (GB, pounds, millimeters, kg, celsius, etc.), use numerals, even if the number if smaller than 10.
**Bad**: 3 kids looked for thirteen files on a six GB hard drive.
**Better**: Three kids looked for 13 files on a 6 GB hard drive.
## New releases
1. Branch.
1. Change the `opensearch_version` and `opensearch_major_version` variables in `_config.yml`.
1. Start up a new cluster using the updated Docker Compose file in `docs/install/docker.md`.
1. Update the version table in `version-history.md`.
Use `curl -XGET https://localhost:9200 -u admin:admin -k` to verify the OpenSearch version.
1. Update the plugin compatibility table in `docs/install/plugin.md`.
Use `curl -XGET https://localhost:9200/_cat/plugins -u admin:admin -k` to get the correct version strings.
1. Update the plugin compatibility table in `docs/opensearch-dashboards/plugins.md`.
Use `docker ps` to find the ID for the OpenSearch Dashboards node. Then use `docker exec -it <opensearch-dashboards-node-id> /bin/bash` to get shell access. Finally, run `./bin/opensearch-dashboards-plugin list` to get the plugins and version strings.
1. Run a build (`build.sh`), and look for any warnings or errors you introduced.
1. Verify that the individual plugin download links in `docs/install/plugins.md` and `docs/opensearch-dashboards/plugins.md` work.
1. Check for any other bad links (`check-links.sh`). Expect a few false positives for the `localhost` links.
1. Submit a PR.
## Classes within Markdown
This documentation uses a modified version of the [just-the-docs](https://github.com/pmarsceill/just-the-docs) Jekyll theme, which has some useful classes for labels and buttons:
```
[Get started](#get-started){: .btn .btn-blue }
## Get started
New
{: .label .label-green :}
```
* Labels come in default (blue), green, purple, yellow, and red.
* Buttons come in default, purple, blue, green, and outline.
* Warning, tip, and note blocks are available (`{: .warning }`, etc.).
* If an image has a white background, you can use `{: .img-border }` to add a one pixel border to the image.
These classes can help with readability, but should be used *sparingly*. Each addition of a class damages the portability of the Markdown files and makes moving to a different Jekyll theme (or a different static site generator) more difficult.
Besides, standard Markdown elements suffice for most documentation.
## Math
If you want to use the sorts of pretty formulas that [MathJax](https://www.mathjax.org) allows, add `has_math: true` to the Jekyll page metadata. Then insert LaTeX math into HTML tags with the rest of your Markdown content:
```
## Math
Some Markdown paragraph. Here's a formula:
<p>
When \(a \ne 0\), there are two solutions to \(ax^2 + bx + c = 0\) and they are
\[x = {-b \pm \sqrt{b^2-4ac} \over 2a}.\]
</p>
And back to Markdown.
```
## Code of conduct
This project has adopted an [Open Source Code of Conduct](https://opensearch.org/codeofconduct.html).
## Security
See [CONTRIBUTING](CONTRIBUTING.md#security-issue-notifications) for more information.
## License
This project is licensed under the Apache-2.0 License.
## Copyright
Copyright 2021 OpenSearch contributors.
-11
View File
@@ -1,11 +0,0 @@
** (MIT License) Just the Docs 0.3.3 - https://github.com/pmarsceill/just-the-docs
Copyright (c) 2016 Patrick Marsceill
** (MIT License) Jekyll Pure Liquid Table of Contents 1.1.0 - https://github.com/allejo/jekyll-toc
Copyright (c) 2017 Vladimir Jimenez
** (MIT License) Bootstrap Icons 1.4.1 - https://github.com/twbs/icons
Copyright (c) 2019-2020 The Bootstrap Authors
@@ -1,66 +0,0 @@
---
layout: default
title: Agents and ingestion tools
nav_order: 100
has_children: false
has_toc: false
---
# Agents and ingestion tools
Historically, many multiple popular agents and ingestion tools have worked with Elasticsearch OSS, such as Beats, Logstash, Fluentd, FluentBit, and OpenTelemetry. OpenSearch aims to continue to support a broad set of agents and ingestion tools, but not all have been tested or have explicitly added OpenSearch compatibility.
As an intermediate solution, we are adding a [version value](https://github.com/opensearch-project/OpenSearch/issues/693) to `opensearch.yml`. This change will let you set OpenSearch 1.x clusters to report version 7.10.2 (or any other arbitrary value). By reporting 7.10.2, the cluster will be able to connect with tools that check for a particular version number.
For a longer term solution, we plan to create an OpenSearch output plugin for Logstash. This plugin *does not exist yet*, but we've included it in the compatibility matrices below based on its expected behavior.
## Downloads
These versions of Logstash and Beats offer the best compatibility with OpenSearch. For more information, see the [compatibility matrices](#compatibility-matrices).
- [Logstash OSS 7.12.1](https://www.elastic.co/downloads/past-releases/logstash-oss-7-12-1)
- [Filebeat OSS 7.12.1](https://www.elastic.co/downloads/past-releases/filebeat-oss-7-12-1)
- [Metricbeat OSS 7.12.1](https://www.elastic.co/downloads/past-releases/metricbeat-oss-7-12-1)
- [Packetbeat OSS 7.12.1](https://www.elastic.co/downloads/past-releases/packetbeat-oss-7-12-1)
- [Heartbeat OSS 7.12.1](https://elastic.co/downloads/past-releases/heartbeat-oss-7-12-1)
- [Winlogbeat OSS 7.12.1](https://www.elastic.co/downloads/past-releases/winlogbeat-oss-7-12-1)
- [Auditbeat OSS 7.12.1](https://elastic.co/downloads/past-releases/auditbeat-oss-7-12-1)
## Compatibility Matrices
*Italicized* cells are untested, but indicate what a value theoretically should be based on existing information.
### Compatibility Matrix for Logstash
| | Logstash OSS 7.x to 7.11.x | Logstash OSS 7.12.x\* | Logstash 7.13.x without OpenSearch output plugin | Logstash 7.13.x with OpenSearch output plugin\*\* |
| :---| :--- | :--- | :--- | :--- |
| Elasticsearch OSS v7.x to v7.9.x | *Yes* | *Yes* | *No* | *Yes* |
| Elasticsearch OSS v7.10.2 | *Yes* | *Yes* | *No* | *Yes* |
| ODFE OSS v1.x to 1.12 | *Yes* | *Yes* | *No* | *Yes* |
| ODFE 1.13 | *Yes* | *Yes* | *No* | *Yes* |
| OpenSearch 1.0 | [Yes via version setting](https://github.com/opensearch-project/OpenSearch/issues/693) | [Yes via version setting](https://github.com/opensearch-project/OpenSearch/issues/693) | *No* | *Yes* |
\* Most current compatible version with Elasticsearch OSS.
\*\* Planning to build.
### Compatibility Matrix for Beats
| | Beats OSS 7.x to 7.11.x\*\* | Beats OSS 7.12.x\* | Beats 7.13.x |
| :--- | :--- | :--- | :--- |
| Elasticsearch OSS v7.x to v7.9.x | *Yes* | *Yes* | No |
| Elasticsearch OSS v7.10.2 | *Yes* | *Yes* | No |
| ODFE OSS v1.x to 1.12 | *Yes* | *Yes* | No |
| ODFE 1.13 | *Yes* | *Yes* | No |
| OpenSearch 1.0 | [Yes via version setting](https://github.com/opensearch-project/OpenSearch/issues/693) | [Yes via version setting](https://github.com/opensearch-project/OpenSearch/issues/693) | No |
| Logstash OSS 7.x to 7.11.x | *Yes* | *Yes* | *Yes* |
| Logstash OSS 7.12.x\* | *Yes* | *Yes* | *Yes* |
| Logstash 7.13.x with OpenSearch output plugin | *Yes* | *Yes* | *Yes* |
\* Most current compatible version with Elasticsearch OSS.
\*\* Beats OSS includes all Apache 2.0 Beats agents (i.e. Filebeat, Metricbeat, Auditbeat, Heartbeat, Winlogbeat, Packetbeat).
-103
View File
@@ -1,103 +0,0 @@
---
layout: default
title: OpenSearch CLI
nav_order: 52
has_children: false
redirect_from:
- /docs/odfe-cli/
- /docs/cli/
---
# OpenSearch CLI
The OpenSearch CLI command line interface (opensearch-cli) lets you manage your OpenSearch cluster from the command line and automate tasks.
Currently, opensearch-cli supports the [Anomaly Detection]({{site.url}}{{site.baseurl}}/monitoring-plugins/ad/) and [k-NN]({{site.url}}{{site.baseurl}}/search-plugins/knn/) plugins, along with arbitrary REST API paths. Among other things, you can use opensearch-cli to create and delete detectors, start and stop them, and check k-NN statistics.
Profiles let you easily access different clusters or sign requests with different credentials. opensearch-cli supports unauthenticated requests, HTTP basic signing, and IAM signing for Amazon Web Services.
This example moves a detector (`ecommerce-count-quantity`) from a staging cluster to a production cluster:
```bash
opensearch-cli ad get ecommerce-count-quantity --profile staging > ecommerce-count-quantity.json
opensearch-cli ad create ecommerce-count-quantity.json --profile production
opensearch-cli ad start ecommerce-count-quantity.json --profile production
opensearch-cli ad stop ecommerce-count-quantity --profile staging
opensearch-cli ad delete ecommerce-count-quantity --profile staging
```
## Install
1. [Download](https://opensearch.org/downloads.html){:target='\_blank'} and extract the appropriate installation package for your computer.
1. Make the `opensearch-cli` file executable:
```bash
chmod +x ./opensearch-cli
```
1. Add the command to your path:
```bash
export PATH=$PATH:$(pwd)
```
1. Confirm the CLI is working properly:
```bash
opensearch-cli --version
```
## Profiles
Profiles let you easily switch between different clusters and user credentials. To get started, run `opensearch-cli profile create` with the `--auth-type`, `--endpoint`, and `--name` options:
```bash
opensearch-cli profile create --auth-type basic --endpoint https://localhost:9200 --name docker-local
```
Alternatively, save a configuration file to `~/.opensearch-cli/config.yaml`:
```yaml
profiles:
- name: docker-local
endpoint: https://localhost:9200
user: admin
password: foobar
- name: aws
endpoint: https://some-cluster.us-east-1.es.amazonaws.com
aws_iam:
profile: ""
service: es
```
## Usage
opensearch-cli commands use the following syntax:
```bash
opensearch-cli <command> <subcommand> <flags>
```
For example, the following command retrieves information about a detector:
```bash
opensearch-cli ad get my-detector --profile docker-local
```
For a request to the OpenSearch CAT API, try the following command:
```bash
opensearch-cli curl get --path _cat/plugins --profile aws
```
Use the `-h` or `--help` flag to see all supported commands, subcommands, or usage for a specific command:
```bash
opensearch-cli -h
opensearch-cli ad -h
opensearch-cli ad get -h
```
-153
View File
@@ -1,153 +0,0 @@
title: OpenSearch documentation
description: >- # this means to ignore newlines until "baseurl:"
Documentation for OpenSearch, the Apache 2.0 search, analytics, and visualization suite with advanced security, alerting, SQL support, automated index management, deep performance analysis, and more.
baseurl: "" # the subpath of your site, e.g. /blog
url: "https://docs-beta.opensearch.org" # the base hostname & protocol for your site, e.g. http://example.com
permalink: /:path/
opensearch_version: 1.0.0-rc1
opensearch_major_minor_version: 1.0
# Build settings
markdown: kramdown
remote_theme: pmarsceill/[email protected]
# Kramdown settings
kramdown:
toc_levels: 2..3
logo: "/assets/images/logo.svg"
# Aux links for the upper right navigation
aux_links:
"Back to OpenSearch.org":
- "https://opensearch.org"
color_scheme: opensearch
# Define Jekyll collections
collections:
# Define a collection named "tests", its documents reside in the "_tests" directory
upgrade-to:
permalink: /:collection/:path/
output: true
opensearch:
permalink: /:collection/:path/
output: true
dashboards:
permalink: /:collection/:path/
output: true
security-plugin:
permalink: /:collection/:path/
output: true
search-plugins:
permalink: /:collection/:path/
output: true
im-plugin:
permalink: /:collection/:path/
output: true
monitoring-plugins:
permalink: /:collection/:path/
output: true
clients:
permalink: /:collection/:path/
output: true
troubleshoot:
permalink: /:collection/:path/
output: true
external_links:
permalink: /:collection/:path/
output: true
just_the_docs:
# Define the collections used in the theme
collections:
upgrade-to:
name: Upgrade to OpenSearch
# nav_exclude: true
nav_fold: true
# search_exclude: true
opensearch:
name: OpenSearch
nav_fold: true
dashboards:
name: OpenSearch Dashboards
nav_fold: true
security-plugin:
name: Security plugin
nav_fold: true
search-plugins:
name: Search plugins
nav_fold: true
im-plugin:
name: Index management plugin
nav_fold: true
monitoring-plugins:
name: Monitoring plugins
nav_fold: true
clients:
name: Clients and tools
nav_fold: true
troubleshoot:
name: Troubleshooting
nav_fold: true
external_links:
name: External links
# Enable or disable the site search
# Supports true (default) or false
search_enabled: true
search:
# Split pages into sections that can be searched individually
# Supports 1 - 6, default: 2
heading_level: 2
# Maximum amount of previews per search result
# Default: 3
previews: 3
# Maximum amount of words to display before a matched word in the preview
# Default: 5
preview_words_before: 5
# Maximum amount of words to display after a matched word in the preview
# Default: 10
preview_words_after: 10
# Set the search token separator
# Default: /[\s\-/]+/
# Example: enable support for hyphenated search words
tokenizer_separator: /[\s/]+/
# Display the relative url in search results
# Supports true (default) or false
rel_url: true
# Enable or disable the search button that appears in the bottom right corner of every page
# Supports true or false (default)
button: false
# Google Analytics Tracking (optional)
# e.g, UA-1234567-89
ga_tracking: G-BQV14XK08F
# Disable the just-the-docs theme anchor links in favor of our custom ones
# See _includes/head_custom.html
heading_anchors: false
# Adds on-hover anchor links to h2-h6
anchor_links: true
footer_content:
plugins:
- jekyll-remote-theme
- jekyll-redirect-from
# Exclude from processing.
# The following items will not be processed, by default. Create a custom list
# to override the default setting.
exclude:
- Gemfile
- Gemfile.lock
- node_modules
- vendor/bundle/
- vendor/cache/
- vendor/gems/
- vendor/ruby/
- README.md
-17
View File
@@ -1,17 +0,0 @@
---
layout: default
title: Browser compatibility
parent: OpenSearch Dashboards
nav_order: 3
---
# Browser compatibility
OpenSearch Dashboards supports the following web browsers:
- Chrome
- Firefox
- Safari
- Edge (Chromium)
Other Chromium-based browsers might work, as well. Internet Explorer and Microsoft Edge Legacy are **not** supported.
-25
View File
@@ -1,25 +0,0 @@
---
layout: default
title: Gantt charts
nav_order: 10
---
# Gantt charts
OpenSearch Dashboards includes a Gantt chart visualization. Gantt charts show the start, end, and duration of unique events in a sequence. Gantt charts are useful in trace analytics, telemetry, and anomaly detection use cases, where you want to understand interactions and dependencies between various events in a schedule.
For example, consider an index of log data. The fields in a typical set of log data, especially audit logs, contain a specific operation or event with a start time and duration.
To create a Gantt chart, perform the following steps:
1. In the visualizations menu, choose **Create visualization** and **Gantt Chart**.
1. Choose a source for the chart (e.g. some log data).
1. Under **Metrics**, choose **Event**. For log data, each log is an event.
1. Select the **Start Time** and **Duration** fields from your data set. The start time is the timestamp for the begining of an event. The duration is the amount of time to add to the start time.
1. Under **Results**, choose the number of events to display on the chart. Gantt charts sequence events from earliest to latest based on start time.
1. Choose **Panel settings** to adjust axis labels, time format, and colors.
1. Choose **Update**.
![Gantt Chart]({{site.url}}{{site.baseurl}}/images/gantt-chart.png)
This Gantt chart displays the ID of each log on the y-axis. Each bar is a unique event that spans some amount of time. Hover over a bar to see the duration of that event.
-24
View File
@@ -1,24 +0,0 @@
---
layout: default
title: About Dashboards
nav_order: 1
has_children: false
has_toc: false
redirect_from:
- /docs/opensearch-dashboards/
- /opensearch-dashboards/
- /dashboards/
---
# OpenSearch Dashboards
OpenSearch Dashboards is the default visualization tool for data in OpenSearch. It also serves as a user interface for many of the OpenSearch plugins, including security, alerting, Index State Management, SQL, and more.
## Get started with OpenSearch Dashboards
1. After starting OpenSearch Dashboards, you can access it at port 5601. For example, http://localhost:5601.
1. Log in with the default username `admin` and password `admin`.
1. Choose **Try our sample data** and add the sample flight data.
1. Choose **Discover** and search for a few flights.
1. Choose **Dashboard**, **[Flights] Global Flight Dashboard**, and wait for the dashboard to load.
-23
View File
@@ -1,23 +0,0 @@
---
layout: default
title: Docker
parent: Install OpenSearch Dashboards
nav_order: 1
---
# Run OpenSearch Dashboards using Docker
You *can* start OpenSearch Dashboards using `docker run` after [creating a Docker network](https://docs.docker.com/engine/reference/commandline/network_create/) and starting OpenSearch, but the process of connecting OpenSearch Dashboards to OpenSearch is significantly easier with a Docker Compose file.
1. Run `docker pull opensearchproject/opensearch-dashboards:{{site.opensearch_version}}`.
1. Create a [`docker-compose.yml`](https://docs.docker.com/compose/compose-file/) file appropriate for your environment. A sample file that includes OpenSearch Dashboards is available on the OpenSearch [Docker installation page]({{site.url}}{{site.baseurl}}/opensearch/install/docker#sample-docker-compose-file).
Just like `opensearch.yml`, you can pass a custom `opensearch_dashboards.yml` to the container in the Docker Compose file.
{: .tip }
1. Run `docker-compose up`.
Wait for the containers to start. Then see the [OpenSearch Dashboards documentation]({{site.url}}{{site.baseurl}}/).
1. When finished, run `docker-compose down`.
-12
View File
@@ -1,12 +0,0 @@
---
layout: default
title: Install OpenSearch Dashboards
nav_order: 1
has_children: true
redirect_from:
- /dashboards/install/
---
# Install and configure OpenSearch Dashboards
OpenSearch Dashboards has two installation options at this time: Docker images and tarballs.
-223
View File
@@ -1,223 +0,0 @@
---
layout: default
title: OpenSearch Dashboards plugins
parent: Install OpenSearch Dashboards
nav_order: 50
---
# Standalone plugin install
If you don't want to use the all-in-one installation options, you can install the various plugins for OpenSearch Dashboards individually.
---
#### Table of contents
1. TOC
{:toc}
---
## Plugin compatibility
<table>
<thead style="text-align: left">
<tr>
<th>OpenSearch Dashboards version</th>
<th>Plugin versions</th>
</tr>
</thead>
<tbody>
<tr>
<td>1.0.0-rc1</td>
<td>
<pre>alertingDashboards 1.0.0.0-rc1
anomalyDetectionDashboards 1.0.0.0-rc1
ganttChartDashboards 1.0.0.0-rc1
indexManagementDashboards 1.0.0.0-rc1
notebooksDashboards 1.0.0.0-rc1
queryWorkbenchDashboards 1.0.0.0-rc1
reportsDashboards 1.0.0.0-rc1
securityDashboards 1.0.0.0-rc1
traceAnalyticsDashboards 1.0.0.0-rc1
</pre>
</td>
</tr>
<tr>
<td>1.0.0-beta1</td>
<td>
<pre>alertingDashboards 1.0.0.0-beta1
anomalyDetectionDashboards 1.0.0.0-beta1
ganttChartDashboards 1.0.0.0-beta1
indexManagementDashboards 1.0.0.0-beta1
notebooksDashboards 1.0.0.0-beta1
queryWorkbenchDashboards 1.0.0.0-beta1
reportsDashboards 1.0.0.0-beta1
securityDashboards 1.0.0.0-beta1
traceAnalyticsDashboards 1.0.0.0-beta1
</pre>
</td>
</tr>
</tbody>
</table>
## Prerequisites
- A compatible OpenSearch cluster
- The corresponding OpenSearch plugins [installed on that cluster]({{site.url}}{{site.baseurl}}/opensearch/install/plugins/)
- The corresponding version of [OpenSearch Dashboards]({{site.url}}{{site.baseurl}}/) (e.g. OpenSearch Dashboards 1.0.0 works with OpenSearch 1.0.0)
## Install
Navigate to the OpenSearch Dashboards home directory (likely `/usr/share/opensearch-dashboards`) and run the install command for each plugin.
{% comment %}
#### Security OpenSearch Dashboards
```bash
sudo bin/opensearch-dashboards-plugin install https://d3g5vo6xdbdb9a.cloudfront.net/downloads/opensearch-dashboards-plugins/opensearch-security/opensearchSecurityOpenSearch Dashboards-{{site.opensearch_major_minor_version}}.0.1.zip
```
This plugin provides a user interface for managing users, roles, mappings, action groups, and tenants.
#### Alerting OpenSearch Dashboards
```bash
sudo bin/opensearch-dashboards-plugin install https://d3g5vo6xdbdb9a.cloudfront.net/downloads/opensearch-dashboards-plugins/opensearch-alerting/opensearchAlertingOpenSearch Dashboards-{{site.opensearch_major_minor_version}}.0.0.zip
```
This plugin provides a user interface for creating monitors and managing alerts.
#### Index State Management OpenSearch Dashboards
```bash
sudo bin/opensearch-dashboards-plugin install https://d3g5vo6xdbdb9a.cloudfront.net/downloads/opensearch-dashboards-plugins/opensearch-index-management/opensearchIndexManagementOpenSearch Dashboards-{{site.opensearch_major_minor_version}}.0.1.zip
```
This plugin provides a user interface for managing policies.
#### Anomaly Detection OpenSearch Dashboards
```bash
sudo bin/opensearch-dashboards-plugin install https://d3g5vo6xdbdb9a.cloudfront.net/downloads/opensearch-dashboards-plugins/opensearch-anomaly-detection/opensearchAnomalyDetectionOpenSearch Dashboards-{{site.opensearch_major_minor_version}}.0.0.zip
```
This plugin provides a user interface for adding detectors.
#### Query Workbench OpenSearch Dashboards
```bash
sudo bin/opensearch-dashboards-plugin install https://d3g5vo6xdbdb9a.cloudfront.net/downloads/opensearch-dashboards-plugins/opensearch-query-workbench/opensearchQueryWorkbenchOpenSearch Dashboards-{{site.opensearch_major_minor_version}}.0.0.zip
```
This plugin provides a user interface for using SQL queries to explore your data.
#### Trace Analytics
```bash
sudo bin/opensearch-dashboards-plugin install https://d3g5vo6xdbdb9a.cloudfront.net/downloads/opensearch-dashboards-plugins/opensearch-trace-analytics/opensearchTraceAnalyticsOpenSearch Dashboards-{{site.opensearch_major_minor_version}}.2.0.zip
```
This plugin uses distributed trace data (indexed in OpenSearch using Data Prepper) to display latency trends, error rates, and more.
#### Notebooks OpenSearch Dashboards
```bash
sudo bin/opensearch-dashboards-plugin install https://d3g5vo6xdbdb9a.cloudfront.net/downloads/opensearch-dashboards-plugins/opensearch-notebooks/opensearchNotebooksOpenSearch Dashboards-{{site.opensearch_major_minor_version}}.2.0.zip
```
This plugin lets you combine OpenSearch Dashboards visualizations and narrative text in a single interface.
#### Reports OpenSearch Dashboards
```bash
# x86 Linux
sudo bin/opensearch-dashboards-plugin install https://d3g5vo6xdbdb9a.cloudfront.net/downloads/opensearch-dashboards-plugins/opensearch-reports/linux/x64/opensearchReportsOpenSearch Dashboards-{{site.opensearch_major_minor_version}}.2.0-linux-x64.zip
# ARM64 Linux
sudo bin/opensearch-dashboards-plugin install https://d3g5vo6xdbdb9a.cloudfront.net/downloads/opensearch-dashboards-plugins/opensearch-reports/linux/arm64/opensearchReportsOpenSearch Dashboards-{{site.opensearch_major_minor_version}}.2.0-linux-arm64.zip
# x86 Windows
sudo bin/opensearch-dashboards-plugin install https://d3g5vo6xdbdb9a.cloudfront.net/downloads/opensearch-dashboards-plugins/opensearch-reports/windows/x64/opensearchReportsOpenSearch Dashboards-{{site.opensearch_major_minor_version}}.2.0-windows-x64.zip
```
This plugin lets you export and share reports from OpenSearch Dashboards dashboards, visualizations, and saved searches.
#### Gantt Chart OpenSearch Dashboards
```bash
sudo bin/opensearch-dashboards-plugin install https://d3g5vo6xdbdb9a.cloudfront.net/downloads/opensearch-dashboards-plugins/opensearch-gantt-chart/opensearchGanttChartOpenSearch Dashboards-{{site.opensearch_major_minor_version}}.0.0.zip
```
This plugin adds a new Gantt chart visualization.
{% endcomment %}
## List installed plugins
To check your installed plugins:
```bash
sudo bin/opensearch-dashboards-plugin list
```
## Remove plugins
To remove a plugin:
```bash
sudo bin/opensearch-dashboards-plugin remove <plugin-name>
```
Then remove all associated entries from `opensearch_dashboards.yml`.
For certain plugins, you must also remove the "optimze" bundle. This is a sample command for the Anomaly Detection plugin:
```bash
sudo rm /usr/share/opensearch-dashboards/optimize/bundles/opensearch-anomaly-detection-opensearch-dashboards.*
```
Then restart OpenSearch Dashboards. After you remove any plugin, OpenSearch Dashboards performs an optimize operation the next time you start it. This operation takes several minutes even on fast machines, so be patient.
## Update plugins
OpenSearch Dashboards doesn’t update plugins. Instead, you have to remove the old version and its optimized bundle, reinstall them, and restart OpenSearch Dashboards:
1. Remove the old version:
```bash
sudo bin/opensearch-dashboards-plugin remove <plugin-name>
```
1. Remove the optimized bundle:
```bash
sudo rm /usr/share/opensearch-dashboards/optimize/bundles/<bundle-name>
```
1. Reinstall the new version:
```bash
sudo bin/opensearch-dashboards-plugin install <plugin-name>
```
1. Restart OpenSearch Dashboards.
For example, to remove and reinstall the anomaly detection plugin:
```bash
sudo bin/opensearch-plugin remove opensearch-anomaly-detection
sudo rm /usr/share/opensearch-dashboards/optimize/bundles/opensearch-anomaly-detection-opensearch-dashboards.*
sudo bin/opensearch-dashboards-plugin install <AD OpenSearch Dashboards plugin artifact URL>
```
-30
View File
@@ -1,30 +0,0 @@
---
layout: default
title: Tarball
parent: Install OpenSearch Dashboards
nav_order: 30
---
# Run OpenSearch Dashboards using the tarball
1. Download the tarball from the [OpenSearch downloads page](https://opensearch.org/downloads.html){:target='\_blank'}.
1. Extract the TAR file to a directory and change to that directory:
```bash
# x64
tar -zxf opensearch-dashboards-{{site.opensearch_version}}-linux-x64.tar.gz
cd opensearch-dashboards{% comment %}# ARM64
tar -zxf opensearch-dashboards-{{site.opensearch_version}}-linux-arm64.tar.gz
cd opensearch-dashboards{% endcomment %}
```
1. If desired, modify `config/opensearch_dashboards.yml`.
1. Run OpenSearch Dashboards:
```bash
./bin/opensearch-dashboards
```
1. See the [OpenSearch Dashboards documentation]({{site.url}}{{site.baseurl}}/opensearch-dashboards/).
-29
View File
@@ -1,29 +0,0 @@
---
layout: default
title: WMS map server
nav_order: 5
---
# Configure WMS map server
OpenSearch Dashboards includes default map tiles, but if you need more specialized maps, you can configure OpenSearch Dashboards to use a WMS map server:
1. Open OpenSearch Dashboards at `https://<host>:<port>`. For example, [https://localhost:5601](https://localhost:5601).
1. If necessary, log in.
1. Choose **Management** and **Advanced Settings**.
1. Locate `visualization:tileMap:WMSdefaults`.
1. Change `enabled` to true and add the URL of a valid WMS map server:
```json
{
"enabled": true,
"url": "<wms-map-server-url>",
"options": {
"format": "image/png",
"transparent": true
}
}
```
Map services often have licensing fees or restrictions. You're responsible for all such considerations on any map server that you specify.
{: .note }
-118
View File
@@ -1,118 +0,0 @@
---
layout: default
title: Notebooks
nav_order: 50
redirect_from: /docs/notebooks/
has_children: false
---
# Notebooks
An OpenSearch Dashboards notebook is an interface that lets you easily combine code snippets, live visualizations, and narrative text in a single notebook interface.
Notebooks let you interactively explore data by running different visualizations that you can share with team members to collaborate on a project.
A notebook is a document composed of two elements: code blocks (Markdown/SQL/PPL) and visualizations. Choose multiple timelines to compare and contrast visualizations.
You can also generate [reports]({{site.url}}{{site.baseurl}}/dashboards/reporting/) directly from your notebooks.
Common use cases include creating postmortem reports, designing runbooks, building live infrastructure reports, and writing documentation.
Tenants in OpenSearch Dashboards are spaces for saving notebooks and other OpenSearch Dashboards objects. For more information, see [OpenSearch Dashboards multi-tenancy]({{site.url}}{{site.baseurl}}/security-plugin/access-control/multi-tenancy/).
{: .note }
## Get started with notebooks
To get started, choose **Notebooks** within OpenSearch Dashboards.
### Step 1: Create a notebook
A notebook is an interface for creating reports.
1. Choose **Create notebook** and enter a descriptive name.
1. Choose **Create**.
Choose **Actions** to rename, duplicate, or delete a notebook.
### Step 2: Add a paragraph
Paragraphs combine code blocks and visualizations for describing data.
#### Add a code block
Code blocks support markdown, SQL, and PPL languages.
Specify the input language on the first line using `%[language type]` syntax.
For example, type `%md` for markdown, `%sql` for SQL, and `%ppl` for PPL.
##### Sample markdown block
```
%md
Add in text formatted in markdown.
```
![Markdown paragraph]({{site.url}}{{site.baseurl}}/images/markdown-notebook.png)
##### Sample SQL block
```sql
%sql
Select * from opensearch_dashboards_sample_data_flights limit 20;
```
![SQL paragraph]({{site.url}}{{site.baseurl}}/images/sql-notebook.png)
##### Sample PPL block
```
%ppl
source=opensearch_dashboards_sample_data_logs | head 20
```
![PPL paragraph]({{site.url}}{{site.baseurl}}/images/ppl-notebook.png)
#### Add a visualization
1. To add a visualization, choose **Add paragraph** and select **Visualization**.
1. In **Title**, select your visualization and choose a date range. You can choose multiple timelines to compare and contrast visualizations.
1. To run and save a paragraph, choose **Run**.
![Visualization paragraph]({{site.url}}{{site.baseurl}}/images/visualization-notebook.png)
## Paragraph actions
You can perform the following actions on paragraphs:
- Add a new paragraph to the top of a report.
- Add a new paragraph to the bottom of a report.
- Run all the paragraphs at the same time.
- Clear the outputs of all paragraphs.
- Delete all the paragraphs.
- Move paragraphs up and down.
## Sample notebooks
We prepared the following sample notebooks that showcase a variety of use cases:
- Using SQL to query the OpenSearch Dashboards sample flight data.
- Using PPL to query the OpenSearch Dashboards sample web logs data.
- Using PPL and visualizations to perform sample root cause event analysis on the OpenSearch Dashboards sample web logs data.
To add a sample notebook, choose **Actions** and select **Add sample notebooks**.
## Create a report
You can use notebooks to create PNG and PDF reports:
1. From the top menu bar, choose **Reporting actions**.
1. You can choose to **Download PDF** or **Download PNG**.
Reports generate asynchronously in the background and might take a few minutes, depending on the size of the report. A notification appears when your report is ready to download.
1. To create a schedule-based report, choose **Create report definition**. For steps to create a report definition, see [Create reports using a definition]({{site.url}}{{site.baseurl}}/dashboards/reporting#create-reports-using-a-definition).
1. To see all your reports, choose **View all reports**.
-57
View File
@@ -1,57 +0,0 @@
---
layout: default
title: Reporting
nav_order: 20
---
# Reporting
You can use OpenSearch Dashboards to create PNG, PDF, and CSV reports. To create reports, you must have the correct permissions. For a summary of the predefined roles and the permissions they grant, see the [security plugin]({{site.url}}{{site.baseurl}}/security-plugin/access-control/users-roles#predefined-roles).
CSV reports have a non-configurable 10,000 row limit. They have no explicit size limit (e.g. in MB), but extremely large documents could cause report generation to fail with an out of memory error from the V8 JavaScript engine.
{: .tip }
## Create reports from Discovery, Visualize, or Dashboard
Quickly generate an on-demand report from the current view.
1. From the top menu bar, choose **Reporting**.
1. For dashboards or visualizations, choose **Download PDF** or **Download PNG**. From the Discover page, choose **Download CSV**.
Reports generate asynchronously in the background and might take a few minutes, depending on the size of the report. A notification appears when your report is ready to download.
1. To create a schedule-based report, choose **Create report definition**. Then proceed to [Create reports using a definition](#create-reports-using-a-definition). This option pre-fills many of the fields for you based on the visualization, dashboard, or data you were viewing.
## Create reports using a definition
Definitions let you generate reports on a periodic schedule.
1. From the navigation panel, choose **Reporting**.
1. Choose **Create**.
1. Under **Report settings**, enter a name and optional description for your report.
1. Choose the **Report Source** (i.e. the page from which the report is generated). You can generate reports from the **Dashboard**, **Visualize**, or **Discover** pages.
1. Select your dashboard, visualization, or saved search. Then choose a time range for the report.
1. Choose an appropriate file format for the report.
1. (Optional) Add a header or footer to the report. Headers and footers are only available for dashboard or visualization reports.
1. Under **Report trigger**, choose either **On-demand** or **Schedule**.
For scheduled reports, select either **Recurring** or **Cron based**. You can receive reports daily or at some other time interval. Cron expressions give you even more flexiblity. See [Cron expression reference]({{site.url}}{{site.baseurl}}/monitoring-plugins/alerting/cron/) for more information.
1. Choose **Create**.
## Troubleshooting
### Chromium fails to launch with OpenSearch Dashboards
While creating a report for dashboards or visualizations, you might see a the following error:
![OpenSearch Dashboards reporting pop-up error message]({{site.url}}{{site.baseurl}}/images/reporting-error.png)
This problem can occur for two reasons:
- You don't have the correct version of `headless-chrome` to match the operating system on which OpenSearch Dashboards is running. Download the correct version [here](https://github.com/opensearch-project/dashboards-reports/releases/tag/chromium-1.12.0.0).
- You're missing additional dependencies. Install the required dependencies for your operating system from the [additional libraries](https://github.com/opensearch-project/dashboards-reports/blob/main/dashboards-reports/rendering-engine/headless-chrome/README.md#additional-libaries) section.
-7
View File
@@ -1,7 +0,0 @@
---
layout: default
title: Javadoc
nav_order: 1
permalink: /javadoc/
redirect_to: https://opensearch.org/docs/javadocs/
---
File diff suppressed because it is too large Load Diff
-234
View File
@@ -1,234 +0,0 @@
---
layout: default
title: Index rollups API
parent: Index rollups
redirect_from: /docs/ism/rollup-api/
nav_order: 9
---
# Index rollups API
Use the index rollup operations to programmatically work with index rollup jobs.
---
#### Table of contents
- TOC
{:toc}
---
## Create or update an index rollup job
Creates or updates an index rollup job.
You must provide the `seq_no` and `primary_term` parameters.
#### Request
```json
PUT _plugins/_rollup/jobs/<rollup_id> // Create
PUT _plugins/_rollup/jobs/<rollup_id>?if_seq_no=1&if_primary_term=1 // Update
{
"rollup": {
"source_index": "nyc-taxi-data",
"target_index": "rollup-nyc-taxi-data",
"schedule": {
"interval": {
"period": 1,
"unit": "Days"
}
},
"description": "Example rollup job",
"enabled": true,
"page_size": 200,
"delay": 0,
"roles": [
"rollup_all",
"nyc_taxi_all",
"example_rollup_index_all"
],
"continuous": false,
"dimensions": {
"date_histogram": {
"source_field": "tpep_pickup_datetime",
"fixed_interval": "1h",
"timezone": "America/Los_Angeles"
},
"terms": {
"source_field": "PULocationID"
},
"metrics": [
{
"source_field": "passenger_count",
"metrics": [
{
"avg": {}
},
{
"sum": {}
},
{
"max": {}
},
{
"min": {}
},
{
"value_count": {}
}
]
}
]
}
}
}
```
You can specify the following options.
Options | Description | Type | Required
:--- | :--- |:--- |:--- |
`source_index` | The name of the detector. | `string` | Yes
`target_index` | Specify the target index that the rolled up data is ingested into. You could either create a new target index or use an existing index. The target index cannot be a combination of raw and rolled up data. | `string` | Yes
`schedule` | Schedule of the index rollup job which can be an interval or a cron expression. | `object` | Yes
`schedule.interval` | Specify the frequency of execution of the rollup job. | `object` | No
`schedule.interval.start_time` | Start time of the interval. | `timestamp` | Yes
`schedule.interval.period` | Define the interval period. | `string` | Yes
`schedule.interval.unit` | Specify the time unit of the interval. | `string` | Yes
`schedule.interval.cron` | Optionally, specify a cron expression to define therollup frequency. | `list` | No
`schedule.interval.cron.expression` | Specify a Unix cron expression. | `string` | Yes
`schedule.interval.cron.timezone` | Specify timezones as defined by the IANA Time Zone Database. Defaults to UTC. | `string` | No
`description` | Optionally, describe the rollup job. | `string` | No
`enabled` | When true, the index rollup job is scheduled. Default is true. | `boolean` | Yes
`continuous` | Specify whether or not the index rollup job continuously rolls up data forever or just executes over the current data set once and stops. Default is false. | `boolean` | Yes
`error_notification` | Set up a Mustache message template sent for error notifications. For example, if an index rollup job fails, the system sends a message to a Slack channel. | `object` | No
`page_size` | Specify the number of buckets to paginate through at a time while rolling up. | `number` | Yes
`delay` | Specify time value to delay execution of the index rollup job. | `time_unit` | No
`dimensions` | Specify aggregations to create dimensions for the roll up time window. | `object` | Yes
`dimensions.date_histogram` | Specify either fixed_interval or calendar_interval, but not both. Either one limits what you can query in the target index. | `object` | No
`dimensions.date_histogram.fixed_interval` | Specify the fixed interval for aggregations in milliseconds, seconds, minutes, hours, or days. | `string` | No
`dimensions.date_histogram.calendar_interval` | Specify the calendar interval for aggregations in minutes, hours, days, weeks, months, quarters, or years. | `string` | No
`dimensions.date_histogram.field` | Specify the date field used in date histogram aggregation. | `string` | No
`dimensions.date_histogram.timezone` | Specify the timezones as defined by the IANA Time Zone Database. The default is UTC. | `string` | No
`dimensions.terms` | Specify the term aggregations that you want to roll up. | `object` | No
`dimensions.terms.fields` | Specify terms aggregation for compatible fields. | `object` | No
`dimensions.histogram` | Specify the histogram aggregations that you want to roll up. | `object` | No
`dimensions.histogram.field` | Add a field for histogram aggregations. | `string` | Yes
`dimensions.histogram.interval` | Specify the histogram aggregation interval for the field. | `long` | Yes
`dimensions.metrics` | Specify a list of objects that represent the fields and metrics that you want to calculate. | `nested object` | No
`dimensions.metrics.field` | Specify the field that you want to perform metric aggregations on. | `string` | No
`dimensions.metrics.field.metrics` | Specify the metric aggregations you want to calculate for the field. | `multiple strings` | No
#### Sample response
```json
{
"_id": "rollup_id",
"_seqNo": 1,
"_primaryTerm": 1,
"rollup": { ... }
}
```
## Get an index rollup job
Returns all information about an index rollup job based on the `rollup_id`.
#### Request
```json
GET _plugins/_rollup/jobs/<rollup_id>
```
#### Sample response
```json
{
"_id": "my_rollup",
"_seqNo": 1,
"_primaryTerm": 1,
"rollup": { ... }
}
```
---
## Delete an index rollup job
Deletes an index rollup job based on the `rollup_id`.
#### Request
```json
DELETE _plugins/_rollup/jobs/<rollup_id>
```
#### Sample response
```json
200 OK
```
---
## Start or stop an index rollup job
Start or stop an index rollup job.
#### Request
```json
POST _plugins/_rollup/jobs/<rollup_id>/_start
POST _plugins/_rollup/jobs/<rollup_id>/_stop
```
#### Sample response
```json
200 OK
```
---
## Explain an index rollup job
Returns detailed metadata information about the index rollup job and its current progress.
#### Request
```json
GET _plugins/_rollup/jobs/<rollup_id>/_explain
```
#### Sample response
```json
{
"example_rollup": {
"rollup_id": "example_rollup",
"last_updated_time": 1602014281,
"continuous": {
"next_window_start_time": 1602055591,
"next_window_end_time": 1602075591
},
"status": "running",
"failure_reason": null,
"stats": {
"pages_processed": 342,
"documents_processed": 489359,
"rollups_indexed": 3420,
"index_time_in_ms": 30495,
"search_time_in_ms": 584922
}
}
}
```
-156
View File
@@ -1,156 +0,0 @@
---
layout: default
title: Index transforms
nav_order: 20
has_children: true
redirect_from: /docs/im/index-transforms/
has_toc: false
---
# Index transforms
Whereas index rollup jobs let you reduce data granularity by rolling up old data into condensed indices, transform jobs let you create a different, summarized view of your data centered around certain fields, so you can visualize or analyze the data in different ways.
For example, suppose that you have airline data that’s scattered across multiple fields and categories, and you want to view a summary of the data that’s organized by airline, quarter, and then price. You can use a transform job to create a new, summarized index that’s organized by those specific categories.
You can use transform jobs in two ways:
1. Use the OpenSearch Dashboards UI to specify the index you want to transform and any optional data filters you want to use to filter the original index. Then select the fields you want to transform and the aggregations to use in the transformation. Finally, define a schedule for your job to follow.
2. Use the transforms API to specify all the details about your job: the index you want to transform, target groups you want the transformed index to have, any aggregations you want to use to group columns, and a schedule for your job to follow.
OpenSearch Dashboards provides a detailed summary of the jobs you created and their relevant information, such as associated indices and job statuses. You can review and edit your job’s details and selections before creation, and even preview a transformed index’s data as you’re choosing which fields to transform. However, you can also use the REST API to create transform jobs and preview transform job results, but you must know all of the necessary settings and parameters to submit them as part of the HTTP request body. Submitting your transform job configurations as JSON scripts offers you more portability, allowing you to share and replicate your transform jobs, which is harder to do using OpenSearch Dashboards.
Your use cases will help you decide which method to use to create transform jobs.
## Create a transform job
If you don't have any data in your cluster, you can use the sample flight data within OpenSearch Dashboards to try out transform jobs. Otherwise, after launching OpenSearch Dashboards, choose **Index Management**. Select **Transform Jobs**, and choose **Create Transform Job**.
### Step 1: Choose indices
1. In the **Job name and description** section, specify a name and an optional description for your job.
2. In the **Indices** section, select the source and target index. You can either select an existing target index or create a new one by entering a name for your new index. If you want to transform just a subset of your source index, choose **Add Data Filter**, and use the OpenSearch query DSL to specify a subset of your source index. For more information about the OpenSearch query DSL, see [query DSL]({{site.url}}{{site.baseurl}}/opensearch/query-dsl/).
3. Choose **Next**.
### Step 2: Select fields to transform
After specifying the indices, you can select the fields you want to use in your transform job, as well as whether to use groupings or aggregations.
You can use groupings to place your data into separate buckets in your transformed index. For example, if you want to group all of the airport destinations within the sample flight data, you can group the `DestAirportID` field into a target field of `DestAirportID_terms` field, and you can find the grouped airport IDs in your transformed index after the transform job finishes.
On the other hand, aggregations let you perform simple calculations. For example, you can include an aggregation in your transform job to define a new field of `sum_of_total_ticket_price` that calculates the sum of all airplane tickets, and then analyze the newly summer data within your transformed index.
1. In the data table, select the fields you want to transform and expand the drop-down menu within the column header to choose the grouping or aggregation you want to use.
Currently, transform jobs support histogram, date_histogram, and terms groupings. For more information about groupings, see [Bucket Aggregations]({{site.url}}{{site.baseurl}}/opensearch/bucket-agg/). In terms of aggregations, you can select from `sum`, `avg`, `max`, `min`, `value_count`, `percentiles`, and `scripted_metric`. For more information about aggregations, see [Metric Aggregations]({{site.url}}{{site.baseurl}}/opensearch/metric-agg/).
2. Repeat step 1 for any other fields that you want to transform.
3. After selecting the fields that you want to transform and verifying the transformation, choose **Next**.
### Step 3: Specify a schedule
You can configure transform jobs to run once or multiple times on a schedule. Transform jobs are enabled by default.
1. For **transformation execution frequency**, select **Define by fixed interval** and specify a **transform interval**.
2. Under **Advanced**, specify an optional amount for **Pages per execution**. A larger number means more data is processed in each search request, but also uses more memory and causes higher latency. Exceeding allowed memory limits can cause exceptions and errors to occur.
3. Choose **Next**.
### Step 4: Review and confirm details
After confirming your transform job’s details are correct, choose **Create Transform Job**. If you want to edit any part of the job, choose **Edit** of the section you want to change, and make the necessary changes. You can’t change aggregations or groupings after creating a job.
### Step 5: Search through the transformed index.
Once the transform job finishes, you can use the `_search` API operation to search the target index.
```json
GET <target_index>/_search
```
For example, after running a transform job that transforms the flight data based on a `DestAirportID` field, you can run the following request that returns all of the fields that have a value of `SFO`.
**Sample Request**
```json
GET finished_flight_job/_search
{
"query": {
"match": {
"DestAirportID_terms" : "SFO"
}
}
}
```
**Sample Response**
```json
{
"took" : 3,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 4,
"relation" : "eq"
},
"max_score" : 3.845883,
"hits" : [
{
"_index" : "finished_flight_job",
"_type" : "_doc",
"_id" : "dSNKGb8U3OJOmC4RqVCi1Q",
"_score" : 3.845883,
"_source" : {
"transform._id" : "sample_flight_job",
"transform._doc_count" : 14,
"Carrier_terms" : "Dashboards Airlines",
"DestAirportID_terms" : "SFO"
}
},
{
"_index" : "finished_flight_job",
"_type" : "_doc",
"_id" : "_D7oqOy7drx9E-MG96U5RA",
"_score" : 3.845883,
"_source" : {
"transform._id" : "sample_flight_job",
"transform._doc_count" : 14,
"Carrier_terms" : "Logstash Airways",
"DestAirportID_terms" : "SFO"
}
},
{
"_index" : "finished_flight_job",
"_type" : "_doc",
"_id" : "YuZ8tOt1OsBA54e84WuAEw",
"_score" : 3.6988301,
"_source" : {
"transform._id" : "sample_flight_job",
"transform._doc_count" : 11,
"Carrier_terms" : "ES-Air",
"DestAirportID_terms" : "SFO"
}
},
{
"_index" : "finished_flight_job",
"_type" : "_doc",
"_id" : "W_-e7bVmH6eu8veJeK8ZxQ",
"_score" : 3.6988301,
"_source" : {
"transform._id" : "sample_flight_job",
"transform._doc_count" : 10,
"Carrier_terms" : "JetBeats",
"DestAirportID_terms" : "SFO"
}
}
]
}
}
```
File diff suppressed because it is too large Load Diff
-14
View File
@@ -1,14 +0,0 @@
---
layout: default
title: About Index Management
nav_order: 1
has_children: false
redirect_from:
- /im-plugin/
---
# About Index Management
OpenSearch Dashboards
{: .label .label-yellow :}
The Index Management (IM) plugin lets you automate recurring index management activities and reduce storage costs.
File diff suppressed because it is too large Load Diff
-104
View File
@@ -1,104 +0,0 @@
---
layout: default
title: Index State Management
nav_order: 3
has_children: true
redirect_from:
- /docs/im/ism/
- /docs/ism/
has_toc: false
---
# Index State Management
OpenSearch Dashboards
{: .label .label-yellow :}
If you analyze time-series data, you likely prioritize new data over old data. You might periodically perform certain operations on older indices, such as reducing replica count or deleting them.
Index State Management (ISM) is a plugin that lets you automate these periodic, administrative operations by triggering them based on changes in the index age, index size, or number of documents. Using the ISM plugin, you can define *policies* that automatically handle index rollovers or deletions to fit your use case.
For example, you can define a policy that moves your index into a `read_only` state after 30 days and then deletes it after a set period of 90 days. You can also set up the policy to send you a notification message when the index is deleted.
You might want to perform an index rollover after a certain amount of time or run a `force_merge` operation on an index during off-peak hours to improve search performance during peak hours.
To use the ISM plugin, your user role needs to be mapped to the `all_access` role that gives you full access to the cluster. To learn more, see [Users and roles]({{site.url}}{{site.baseurl}}/security-plugin/access-control/users-roles/).
{: .note }
## Get started with ISM
To get started, choose **Index Management** in OpenSearch Dashboards.
### Step 1: Set up policies
A policy is a set of rules that describes how an index should be managed. For information about creating a policy, see [Policies]({{site.url}}{{site.baseurl}}/im-plugin/ism/policies/).
1. Choose the **Index Policies** tab.
2. Choose **Create policy**.
3. In the **Name policy** section, enter a policy ID.
4. In the **Define policy** section, enter your policy.
5. Choose **Create**.
After you create a policy, your next step is to attach this policy to an index or indices.
You can set up an `ism_template` in the policy so when you create an index that matches the ISM template pattern, the index will have this policy attached to it:
```json
PUT _plugins/_ism/policies/policy_id
{
"policy": {
"description": "Example policy.",
"default_state": "...",
"states": [...],
"ism_template": {
"index_patterns": ["index_name-*"],
"priority": 100
}
}
}
```
For an example ISM template policy, see [Sample policy with ISM template]({{site.url}}{{site.baseurl}}/im-plugin/ism/policies#sample-policy-with-ism-template).
Older versions of the plugin include the `policy_id` in an index template, so when an index is created that matches the index template pattern, the index will have the policy attached to it:
```json
PUT _index_template/<template_name>
{
"index_patterns": [
"index_name-*"
],
"template": {
"settings": {
"opendistro.index_state_management.policy_id": "policy_id"
}
}
}
```
The `opendistro.index_state_management.policy_id` setting is deprecated. You can continue to automatically manage newly created indices with the ISM template field.
{: .note }
### Step 2: Attach policies to indices
1. Choose **Indices**.
2. Choose the index or indices that you want to attach your policy to.
3. Choose **Apply policy**.
4. From the **Policy ID** menu, choose the policy that you created.
You can see a preview of your policy.
5. If your policy includes a rollover operation, specify a rollover alias.
Make sure that the alias that you enter already exists. For more information about the rollover operation, see [rollover]({{site.url}}{{site.baseurl}}/im-plugin/ism/policies#rollover).
6. Choose **Apply**.
After you attach a policy to an index, ISM creates a job that runs every 5 minutes by default to perform policy actions, check conditions, and transition the index into different states. To change the default time interval for this job, see [Settings]({{site.url}}{{site.baseurl}}/im-plugin/ism/settings/).
If you want to use an OpenSearch operation to create an index with a policy already attached to it, see [create index]({{site.url}}{{site.baseurl}}/im-plugin/ism/api#create-index).
### Step 3: Manage indices
1. Choose **Managed Indices**.
2. To change your policy, see [Change Policy]({{site.url}}{{site.baseurl}}/im-plugin/ism/managedindices#change-policy).
3. To attach a rollover alias to your index, select your policy and choose **Add rollover alias**.
Make sure that the alias that you enter already exists. For more information about the rollover operation, see [rollover]({{site.url}}{{site.baseurl}}/im-plugin/ism/policies#rollover).
4. To remove a policy, choose your policy, and then choose **Remove policy**.
5. To retry a policy, choose your policy, and then choose **Retry policy**.
For information about managing your policies, see [Managed Indices]({{site.url}}{{site.baseurl}}/im-plugin/ism/managedindices/).
-74
View File
@@ -1,74 +0,0 @@
---
layout: default
title: Managed Indices
nav_order: 3
parent: Index State Management
redirect_from: /docs/ism/managedindices/
has_children: false
---
# Managed indices
You can change or update a policy using the managed index operations.
This table lists the fields of managed index operations.
Parameter | Description | Type | Required | Read Only
:--- | :--- |:--- |:--- |
`name` | The name of the managed index policy. | `string` | Yes | No
`index` | The name of the managed index that this policy is managing. | `string` | Yes | No
`index_uuid` | The uuid of the index. | `string` | Yes | No
`enabled` | When `true`, the managed index is scheduled and run by the scheduler. | `boolean` | Yes | No
`enabled_time` | The time the managed index was last enabled. If the managed index process is disabled, then this is null. | `timestamp` | Yes | Yes
`last_updated_time` | The time the managed index was last updated. | `timestamp` | Yes | Yes
`schedule` | The schedule of the managed index job. | `object` | Yes | No
`policy_id` | The name of the policy used by this managed index. | `string` | Yes | No
`policy_seq_no` | The sequence number of the policy used by this managed index. | `number` | Yes | No
`policy_primary_term` | The primary term of the policy used by this managed index. | `number` | Yes | No
`policy_version` | The version of the policy used by this managed index. | `number` | Yes | Yes
`policy` | The cached JSON of the policy for the `policy_version` that's used during runs. If the policy is null, it means that this is the first execution of the job and the latest policy document is read in/saved. | `object` | No | No
`change_policy` | The information regarding what policy and state to change to. | `object` | No | No
`policy_name` | The name of the policy to update to. To update to the latest version, set this to be the same as the current `policy_name`. | `string` | No | Yes
`state` | The state of the managed index after it finishes updating. If no state is specified, it's assumed that the policy structure did not change. | `string` | No | Yes
The following example shows a managed index policy:
```json
{
"managed_index": {
"name": "my_index",
"index": "my_index",
"index_uuid": "sOKSOfkdsoSKeofjIS",
"enabled": true,
"enabled_time": 1553112384,
"last_updated_time": 1553112384,
"schedule": {
"interval": {
"period": 1,
"unit": "MINUTES",
"start_time": 1553112384
}
},
"policy_id": "log_rotation",
"policy_version": 1,
"policy": {...},
"change_policy": null
}
}
```
## Change policy
You can change any managed index policy, but ISM has a few constraints in place to make sure that policy changes don't break indices.
If an index is stuck in its current state, never proceeding, and you want to update its policy immediately, make sure that the new policy includes the same state---same name, same actions, same order---as the old policy. In this case, even if the policy is in the middle of executing an action, ISM applies the new policy.
If you update the policy without including an identical state, ISM updates the policy only after all actions in the current state finish executing. Alternately, you can choose a specific state in your old policy after which you want the new policy to take effect.
To change a policy using OpenSearch Dashboards, do the following:
- Under **Managed indices**, choose the indices that you want to attach the new policy to.
- To attach the new policy to indices in specific states, choose **Choose state filters**, and then choose those states.
- Under **Choose New Policy**, choose the new policy.
- To start the new policy for indices in the current state, choose **Keep indices in their current state after the policy takes effect**.
- To start the new policy in a specific state, choose **Start from a chosen state after changing policies**, and then choose the default start state in your new policy.
File diff suppressed because it is too large Load Diff

Some files were not shown because too many files have changed in this diff Show More