Compare commits
46
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
12c82932ca | ||
|
|
78f0f6c288 | ||
|
|
1a88ceb40a | ||
|
|
8a15e82845 | ||
|
|
45c8214c2b | ||
|
|
4841a9985c | ||
|
|
49a765d0a5 | ||
|
|
67423ff9c2 | ||
|
|
5a4d4dabba | ||
|
|
05d6b6cb5d | ||
|
|
6e9249e58c | ||
|
|
432005eaa0 | ||
|
|
3d3bc88422 | ||
|
|
75a835f29b | ||
|
|
d00637b05e | ||
|
|
73667cafd0 | ||
|
|
be8d244d8c | ||
|
|
1e031e8362 | ||
|
|
59969664a5 | ||
|
|
3c27d762ef | ||
|
|
47bd70cb2b | ||
|
|
b8bf7a2f1d | ||
|
|
6c6c9c619f | ||
|
|
a335f8741a | ||
|
|
c0871cfe0d | ||
|
|
92b3b49ee2 | ||
|
|
cf769012ca | ||
|
|
4a2e1c32db | ||
|
|
79b647ee62 | ||
|
|
e5bec0b980 | ||
|
|
d3d9774eaf | ||
|
|
e484b3e93f | ||
|
|
d7ce813388 | ||
|
|
18e74399fa | ||
|
|
81bf0b2655 | ||
|
|
21b4c3ab87 | ||
|
|
24acbc3d56 | ||
|
|
62491a2a98 | ||
|
|
faa347c20f | ||
|
|
31ebf2634d | ||
|
|
b81261034b | ||
|
|
610c958b2d | ||
|
|
5d5dc6c413 | ||
|
|
b79ee7cf34 | ||
|
|
29feadfb57 | ||
|
|
93409f1333 |
@@ -0,0 +1,9 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en-US">
|
||||
<meta charset="utf-8">
|
||||
<title>Redirecting…</title>
|
||||
<script>location="https://opensearch.org/docs"+location.pathname</script>
|
||||
<meta name="robots" content="noindex">
|
||||
<h1>Redirecting…</h1>
|
||||
<a href="https://opensearch.org/docs">Click here if you are not redirected.</a>
|
||||
</html>
|
||||
@@ -1,4 +0,0 @@
|
||||
## Code of Conduct
|
||||
This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct).
|
||||
For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact
|
||||
[email protected] with any additional questions or comments.
|
||||
@@ -1,59 +0,0 @@
|
||||
# Contributing Guidelines
|
||||
|
||||
Thank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional
|
||||
documentation, we greatly value feedback and contributions from our community.
|
||||
|
||||
Please read through this document before submitting any issues or pull requests to ensure we have all the necessary
|
||||
information to effectively respond to your bug report or contribution.
|
||||
|
||||
|
||||
## Reporting Bugs/Feature Requests
|
||||
|
||||
We welcome you to use the GitHub issue tracker to report bugs or suggest features.
|
||||
|
||||
When filing an issue, please check existing open, or recently closed, issues to make sure somebody else hasn't already
|
||||
reported the issue. Please try to include as much information as you can. Details like these are incredibly useful:
|
||||
|
||||
* A reproducible test case or series of steps
|
||||
* The version of our code being used
|
||||
* Any modifications you've made relevant to the bug
|
||||
* Anything unusual about your environment or deployment
|
||||
|
||||
|
||||
## Contributing via Pull Requests
|
||||
Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that:
|
||||
|
||||
1. You are working against the latest source on the *main* branch.
|
||||
2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already.
|
||||
3. You open an issue to discuss any significant work - we would hate for your time to be wasted.
|
||||
|
||||
To send us a pull request, please:
|
||||
|
||||
1. Fork the repository.
|
||||
2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change.
|
||||
3. Ensure local tests pass.
|
||||
4. Commit to your fork using clear commit messages.
|
||||
5. Send us a pull request, answering any default questions in the pull request interface.
|
||||
6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation.
|
||||
|
||||
GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and
|
||||
[creating a pull request](https://help.github.com/articles/creating-a-pull-request/).
|
||||
|
||||
|
||||
## Finding contributions to work on
|
||||
Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any 'help wanted' issues is a great place to start.
|
||||
|
||||
|
||||
## Code of Conduct
|
||||
This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct).
|
||||
For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact
|
||||
[email protected] with any additional questions or comments.
|
||||
|
||||
|
||||
## Security issue notifications
|
||||
If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue.
|
||||
|
||||
|
||||
## Licensing
|
||||
|
||||
See the [LICENSE](LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution.
|
||||
@@ -1,32 +0,0 @@
|
||||
source "https://rubygems.org"
|
||||
|
||||
# Hello! This is where you manage which Jekyll version is used to run.
|
||||
# When you want to use a different version, change it below, save the
|
||||
# file and run `bundle install`. Run Jekyll with `bundle exec`, like so:
|
||||
#
|
||||
# bundle exec jekyll serve
|
||||
#
|
||||
# This will help ensure the proper Jekyll version is running.
|
||||
# Happy Jekylling!
|
||||
# gem "jekyll", "~> 3.9.0"
|
||||
|
||||
# This is the default theme for new Jekyll sites. You may change this to anything you like.
|
||||
gem "just-the-docs", "~> 0.3.3"
|
||||
|
||||
# If you want to use GitHub Pages, remove the "gem "jekyll"" above and
|
||||
# uncomment the line below. To upgrade, run `bundle update github-pages`.
|
||||
|
||||
gem 'github-pages', group: :jekyll_plugins
|
||||
|
||||
# If you have any plugins, put them here!
|
||||
# group :jekyll_plugins do
|
||||
# # gem "jekyll-feed", "~> 0.6"
|
||||
# gem "jekyll-remote-theme"
|
||||
# gem "jekyll-redirect-from"
|
||||
# end
|
||||
|
||||
# Windows does not include zoneinfo files, so bundle the tzinfo-data gem
|
||||
gem "tzinfo-data", platforms: [:mingw, :mswin, :x64_mingw, :jruby]
|
||||
|
||||
# Performance-booster for watching directories on Windows
|
||||
gem "wdm", "~> 0.1.0" if Gem.win_platform?
|
||||
@@ -1,276 +0,0 @@
|
||||
# OpenSearch documentation
|
||||
|
||||
This repository contains the documentation for OpenSearch, the search, analytics, and visualization suite with advanced security, alerting, SQL support, automated index management, deep performance analysis, and more. You can find the rendered documentation at [docs-beta.opensearch.org](https://docs-beta.opensearch.org).
|
||||
|
||||
Community contributions remain essential in keeping this documentation comprehensive, useful, well-organized, and up-to-date.
|
||||
|
||||
|
||||
## How you can help
|
||||
|
||||
- Do you work on one of the various OpenSearch plugins? Take a look at the documentation for the plugin. Is everything accurate? Will anything change in the near future?
|
||||
|
||||
Often, engineering teams can keep existing documentation up-to-date with minimal effort, thus freeing up the documentation team to focus on larger projects.
|
||||
|
||||
- Do you have expertise in a particular area of OpenSearch? Cluster sizing? The query DSL? Painless scripting? Aggregations? JVM settings? Take a look at the [current content](https://docs-beta.opensearch.org/docs/opensearch/) and see where you can add value. The [documentation team](#points-of-contact) is happy to help you polish and organize your drafts.
|
||||
|
||||
- Are you an OpenSearch Dashboards expert? How did you set up your visualizations? Why is a particular dashboard so valuable to your organization? We have [very little](https://docs-beta.opensearch.org/docs/opensearch-dashboards/) on how to use OpenSearch Dashboards, only how to install it.
|
||||
|
||||
- Are you a web developer? Do you want to add an optional dark mode to the documentation? A "copy to clipboard" button for our code samples? Other improvements to the design or usability? See [major changes](#major-changes) for information on building the website locally.
|
||||
|
||||
- Our [issue tracker](https://github.com/opensearch-project/documentation-website/issues) contains documentation bugs and other content gaps, some of which have colorful labels like "good first issue" and "help wanted."
|
||||
|
||||
|
||||
## Points of contact
|
||||
|
||||
If you encounter problems or have questions when contributing to the documentation, these people can help:
|
||||
|
||||
- [aetter](https://github.com/aetter)
|
||||
- [ashwinkumar12345](https://github.com/ashwinkumar12345)
|
||||
- [keithhc2](https://github.com/keithhc2)
|
||||
- [snyder114](https://github.com/snyder114)
|
||||
|
||||
|
||||
## How we build the website
|
||||
|
||||
After each commit to this repository, GitHub Pages automatically uses [Jekyll](https://jekyllrb.com) to rebuild the [website](https://docs-beta.opensearch.org). The whole process takes around 30 seconds.
|
||||
|
||||
This repository contains many [Markdown](https://guides.github.com/features/mastering-markdown/) files in the `/docs` directory. Each Markdown file correlates with one page on the website.
|
||||
|
||||
Using plain text on GitHub has many advantages:
|
||||
|
||||
- Everything is free, open source, and works on every operating system. Use your favorite text editor, Ruby, Jekyll, and Git.
|
||||
- Markdown is easy to learn and looks good in side-by-side diffs.
|
||||
- The workflow is no different than contributing code. Make your changes, build locally to check your work, and submit a pull request. Reviewers check the PR before merging.
|
||||
- Alternatives like wikis and WordPress are full web applications that require databases and ongoing maintenance. They also have inferior versioning and content review processes compared to Git. Static websites, such as the ones Jekyll produces, are faster, more secure, and more stable.
|
||||
|
||||
In addition to the content for a given page, each Markdown file contains some Jekyll [front matter](https://jekyllrb.com/docs/front-matter/). Front matter looks like this:
|
||||
|
||||
```
|
||||
---
|
||||
layout: default
|
||||
title: Alerting security
|
||||
nav_order: 10
|
||||
parent: Alerting
|
||||
has_children: false
|
||||
---
|
||||
```
|
||||
|
||||
If you're making [trivial changes](#trivial-changes), you don't have to worry about front matter.
|
||||
|
||||
If you want to reorganize content or add new pages, keep an eye on `has_children`, `parent`, and `nav_order`, which define the hierarchy and order of pages in the lefthand navigation. For more information, see the documentation for [our upstream Jekyll theme](https://pmarsceill.github.io/just-the-docs/docs/navigation-structure/).
|
||||
|
||||
|
||||
## Contribute content
|
||||
|
||||
There are three ways to contribute content, depending on the magnitude of the change.
|
||||
|
||||
- [Trivial changes](#trivial-changes)
|
||||
- [Minor changes](#minor-changes)
|
||||
- [Major changes](#major-changes)
|
||||
|
||||
|
||||
### Trivial changes
|
||||
|
||||
If you just need to fix a typo or add a sentence, this web-based method works well:
|
||||
|
||||
1. On any page in the documentation, click the **Edit this page** link in the lower-left.
|
||||
|
||||
1. Make your changes.
|
||||
|
||||
1. Choose **Create a new branch for this commit and start a pull request** and **Commit changes**.
|
||||
|
||||
|
||||
### Minor changes
|
||||
|
||||
If you want to add a few paragraphs across multiple files and are comfortable with Git, try this approach:
|
||||
|
||||
1. Fork this repository.
|
||||
|
||||
1. Download [GitHub Desktop](https://desktop.github.com), install it, and clone your fork.
|
||||
|
||||
1. Navigate to the repository root.
|
||||
|
||||
1. Create a new branch.
|
||||
|
||||
1. Edit the Markdown files in `/docs`.
|
||||
|
||||
1. Commit, push your changes to your fork, and submit a pull request.
|
||||
|
||||
|
||||
### Major changes
|
||||
|
||||
If you're making major changes to the documentation and need to see the rendered HTML before submitting a pull request, here's how to build locally:
|
||||
|
||||
1. Fork this repository.
|
||||
|
||||
1. Download [GitHub Desktop](https://desktop.github.com), install it, and clone your fork.
|
||||
|
||||
1. Navigate to the repository root.
|
||||
|
||||
1. Install [Ruby](https://www.ruby-lang.org/en/) if you don't already have it. We recommend [RVM](https://rvm.io/), but use whatever method you prefer:
|
||||
|
||||
```
|
||||
curl -sSL https://get.rvm.io | bash -s stable
|
||||
rvm install 2.6
|
||||
ruby -v
|
||||
```
|
||||
|
||||
1. Install [Jekyll](https://jekyllrb.com/) if you don't already have it:
|
||||
|
||||
```
|
||||
gem install bundler jekyll
|
||||
```
|
||||
|
||||
1. Install dependencies:
|
||||
|
||||
```
|
||||
bundle install
|
||||
```
|
||||
|
||||
1. Build:
|
||||
|
||||
```
|
||||
sh build.sh
|
||||
```
|
||||
|
||||
1. If the build script doesn't automatically open your web browser (it should), open [http://localhost:4000/](http://localhost:4000/).
|
||||
|
||||
1. Create a new branch.
|
||||
|
||||
1. Edit the Markdown files in `/docs`.
|
||||
|
||||
If you're a web developer, you can customize `_layouts/default.html` and `_sass/custom/custom.scss`.
|
||||
|
||||
1. When you save a file, marvel as Jekyll automatically rebuilds the site and refreshes your web browser. This process takes roughly 30 seconds.
|
||||
|
||||
1. When you're happy with how everything looks, commit, push your changes to your fork, and submit a pull request.
|
||||
|
||||
|
||||
## Writing tips
|
||||
|
||||
1. Try to stay consistent with existing content and consistent within your new content. Don't call the same plugin KNN, k-nn, and k-NN in three different places.
|
||||
|
||||
1. Shorter paragraphs are better than longer paragraphs. Use headers, tables, lists, and images to make your content easier for readers to scan.
|
||||
|
||||
1. Use **bold** for user interface elements, *italics* for key terms or emphasis, and `monospace` for Bash commands, file names, REST paths, and code.
|
||||
|
||||
1. Markdown file names should be all lowercase, use hyphens to separate words, and end in `.md`.
|
||||
|
||||
1. Avoid future tense. Use present tense.
|
||||
|
||||
**Bad**: After you click the button, the process will start.
|
||||
|
||||
**Better**: After you click the button, the process starts.
|
||||
|
||||
1. "You" refers to the person reading the page. "We" refers to the OpenSearch contributors.
|
||||
|
||||
**Bad**: Now that we've finished the configuration, we have a working cluster.
|
||||
|
||||
**Better**: At this point, you have a working cluster, but we recommend adding dedicated master nodes.
|
||||
|
||||
1. Don't use "this" and "that" to refer to something without adding a noun.
|
||||
|
||||
**Bad**: This can cause high latencies.
|
||||
|
||||
**Better**: This additional loading time can cause high latencies.
|
||||
|
||||
1. Use active voice.
|
||||
|
||||
**Bad**: After the request is sent, the data is added to the index.
|
||||
|
||||
**Better**: After you send the request, the OpenSearch cluster indexes the data.
|
||||
|
||||
1. Introduce acronyms before using them.
|
||||
|
||||
**Bad**: Reducing customer TTV should accelerate our ROIC.
|
||||
|
||||
**Better**: Reducing customer time to value (TTV) should accelerate our return on invested capital (ROIC).
|
||||
|
||||
1. Spell out one through nine. Start using numerals at 10. If a number needs a unit (GB, pounds, millimeters, kg, celsius, etc.), use numerals, even if the number if smaller than 10.
|
||||
|
||||
**Bad**: 3 kids looked for thirteen files on a six GB hard drive.
|
||||
|
||||
**Better**: Three kids looked for 13 files on a 6 GB hard drive.
|
||||
|
||||
|
||||
## New releases
|
||||
|
||||
1. Branch.
|
||||
1. Change the `opensearch_version` and `opensearch_major_version` variables in `_config.yml`.
|
||||
1. Start up a new cluster using the updated Docker Compose file in `docs/install/docker.md`.
|
||||
1. Update the version table in `version-history.md`.
|
||||
|
||||
Use `curl -XGET https://localhost:9200 -u admin:admin -k` to verify the OpenSearch version.
|
||||
|
||||
1. Update the plugin compatibility table in `docs/install/plugin.md`.
|
||||
|
||||
Use `curl -XGET https://localhost:9200/_cat/plugins -u admin:admin -k` to get the correct version strings.
|
||||
|
||||
1. Update the plugin compatibility table in `docs/opensearch-dashboards/plugins.md`.
|
||||
|
||||
Use `docker ps` to find the ID for the OpenSearch Dashboards node. Then use `docker exec -it <opensearch-dashboards-node-id> /bin/bash` to get shell access. Finally, run `./bin/opensearch-dashboards-plugin list` to get the plugins and version strings.
|
||||
|
||||
1. Run a build (`build.sh`), and look for any warnings or errors you introduced.
|
||||
1. Verify that the individual plugin download links in `docs/install/plugins.md` and `docs/opensearch-dashboards/plugins.md` work.
|
||||
1. Check for any other bad links (`check-links.sh`). Expect a few false positives for the `localhost` links.
|
||||
1. Submit a PR.
|
||||
|
||||
|
||||
## Classes within Markdown
|
||||
|
||||
This documentation uses a modified version of the [just-the-docs](https://github.com/pmarsceill/just-the-docs) Jekyll theme, which has some useful classes for labels and buttons:
|
||||
|
||||
```
|
||||
[Get started](#get-started){: .btn .btn-blue }
|
||||
|
||||
## Get started
|
||||
New
|
||||
{: .label .label-green :}
|
||||
```
|
||||
|
||||
* Labels come in default (blue), green, purple, yellow, and red.
|
||||
* Buttons come in default, purple, blue, green, and outline.
|
||||
* Warning, tip, and note blocks are available (`{: .warning }`, etc.).
|
||||
* If an image has a white background, you can use `{: .img-border }` to add a one pixel border to the image.
|
||||
|
||||
These classes can help with readability, but should be used *sparingly*. Each addition of a class damages the portability of the Markdown files and makes moving to a different Jekyll theme (or a different static site generator) more difficult.
|
||||
|
||||
Besides, standard Markdown elements suffice for most documentation.
|
||||
|
||||
|
||||
## Math
|
||||
|
||||
If you want to use the sorts of pretty formulas that [MathJax](https://www.mathjax.org) allows, add `has_math: true` to the Jekyll page metadata. Then insert LaTeX math into HTML tags with the rest of your Markdown content:
|
||||
|
||||
```
|
||||
## Math
|
||||
|
||||
Some Markdown paragraph. Here's a formula:
|
||||
|
||||
<p>
|
||||
When \(a \ne 0\), there are two solutions to \(ax^2 + bx + c = 0\) and they are
|
||||
\[x = {-b \pm \sqrt{b^2-4ac} \over 2a}.\]
|
||||
</p>
|
||||
|
||||
And back to Markdown.
|
||||
```
|
||||
|
||||
|
||||
## Code of conduct
|
||||
|
||||
This project has adopted an [Open Source Code of Conduct](https://opensearch.org/codeofconduct.html).
|
||||
|
||||
|
||||
## Security
|
||||
|
||||
See [CONTRIBUTING](CONTRIBUTING.md#security-issue-notifications) for more information.
|
||||
|
||||
|
||||
## License
|
||||
|
||||
This project is licensed under the Apache-2.0 License.
|
||||
|
||||
|
||||
## Copyright
|
||||
|
||||
Copyright 2021 OpenSearch contributors.
|
||||
-11
@@ -1,11 +0,0 @@
|
||||
** (MIT License) Just the Docs 0.3.3 - https://github.com/pmarsceill/just-the-docs
|
||||
|
||||
Copyright (c) 2016 Patrick Marsceill
|
||||
|
||||
** (MIT License) Jekyll Pure Liquid Table of Contents 1.1.0 - https://github.com/allejo/jekyll-toc
|
||||
|
||||
Copyright (c) 2017 Vladimir Jimenez
|
||||
|
||||
** (MIT License) Bootstrap Icons 1.4.1 - https://github.com/twbs/icons
|
||||
|
||||
Copyright (c) 2019-2020 The Bootstrap Authors
|
||||
@@ -1,66 +0,0 @@
|
||||
---
|
||||
layout: default
|
||||
title: Agents and ingestion tools
|
||||
nav_order: 100
|
||||
has_children: false
|
||||
has_toc: false
|
||||
---
|
||||
|
||||
# Agents and ingestion tools
|
||||
|
||||
Historically, many multiple popular agents and ingestion tools have worked with Elasticsearch OSS, such as Beats, Logstash, Fluentd, FluentBit, and OpenTelemetry. OpenSearch aims to continue to support a broad set of agents and ingestion tools, but not all have been tested or have explicitly added OpenSearch compatibility.
|
||||
|
||||
As an intermediate solution, we are adding a [version value](https://github.com/opensearch-project/OpenSearch/issues/693) to `opensearch.yml`. This change will let you set OpenSearch 1.x clusters to report version 7.10.2 (or any other arbitrary value). By reporting 7.10.2, the cluster will be able to connect with tools that check for a particular version number.
|
||||
|
||||
For a longer term solution, we plan to create an OpenSearch output plugin for Logstash. This plugin *does not exist yet*, but we've included it in the compatibility matrices below based on its expected behavior.
|
||||
|
||||
|
||||
## Downloads
|
||||
|
||||
These versions of Logstash and Beats offer the best compatibility with OpenSearch. For more information, see the [compatibility matrices](#compatibility-matrices).
|
||||
|
||||
- [Logstash OSS 7.12.1](https://www.elastic.co/downloads/past-releases/logstash-oss-7-12-1)
|
||||
- [Filebeat OSS 7.12.1](https://www.elastic.co/downloads/past-releases/filebeat-oss-7-12-1)
|
||||
- [Metricbeat OSS 7.12.1](https://www.elastic.co/downloads/past-releases/metricbeat-oss-7-12-1)
|
||||
- [Packetbeat OSS 7.12.1](https://www.elastic.co/downloads/past-releases/packetbeat-oss-7-12-1)
|
||||
- [Heartbeat OSS 7.12.1](https://elastic.co/downloads/past-releases/heartbeat-oss-7-12-1)
|
||||
- [Winlogbeat OSS 7.12.1](https://www.elastic.co/downloads/past-releases/winlogbeat-oss-7-12-1)
|
||||
- [Auditbeat OSS 7.12.1](https://elastic.co/downloads/past-releases/auditbeat-oss-7-12-1)
|
||||
|
||||
|
||||
## Compatibility Matrices
|
||||
|
||||
*Italicized* cells are untested, but indicate what a value theoretically should be based on existing information.
|
||||
|
||||
|
||||
### Compatibility Matrix for Logstash
|
||||
|
||||
| | Logstash OSS 7.x to 7.11.x | Logstash OSS 7.12.x\* | Logstash 7.13.x without OpenSearch output plugin | Logstash 7.13.x with OpenSearch output plugin\*\* |
|
||||
| :---| :--- | :--- | :--- | :--- |
|
||||
| Elasticsearch OSS v7.x to v7.9.x | *Yes* | *Yes* | *No* | *Yes* |
|
||||
| Elasticsearch OSS v7.10.2 | *Yes* | *Yes* | *No* | *Yes* |
|
||||
| ODFE OSS v1.x to 1.12 | *Yes* | *Yes* | *No* | *Yes* |
|
||||
| ODFE 1.13 | *Yes* | *Yes* | *No* | *Yes* |
|
||||
| OpenSearch 1.0 | [Yes via version setting](https://github.com/opensearch-project/OpenSearch/issues/693) | [Yes via version setting](https://github.com/opensearch-project/OpenSearch/issues/693) | *No* | *Yes* |
|
||||
|
||||
\* Most current compatible version with Elasticsearch OSS.
|
||||
|
||||
\*\* Planning to build.
|
||||
|
||||
|
||||
### Compatibility Matrix for Beats
|
||||
|
||||
| | Beats OSS 7.x to 7.11.x\*\* | Beats OSS 7.12.x\* | Beats 7.13.x |
|
||||
| :--- | :--- | :--- | :--- |
|
||||
| Elasticsearch OSS v7.x to v7.9.x | *Yes* | *Yes* | No |
|
||||
| Elasticsearch OSS v7.10.2 | *Yes* | *Yes* | No |
|
||||
| ODFE OSS v1.x to 1.12 | *Yes* | *Yes* | No |
|
||||
| ODFE 1.13 | *Yes* | *Yes* | No |
|
||||
| OpenSearch 1.0 | [Yes via version setting](https://github.com/opensearch-project/OpenSearch/issues/693) | [Yes via version setting](https://github.com/opensearch-project/OpenSearch/issues/693) | No |
|
||||
| Logstash OSS 7.x to 7.11.x | *Yes* | *Yes* | *Yes* |
|
||||
| Logstash OSS 7.12.x\* | *Yes* | *Yes* | *Yes* |
|
||||
| Logstash 7.13.x with OpenSearch output plugin | *Yes* | *Yes* | *Yes* |
|
||||
|
||||
\* Most current compatible version with Elasticsearch OSS.
|
||||
|
||||
\*\* Beats OSS includes all Apache 2.0 Beats agents (i.e. Filebeat, Metricbeat, Auditbeat, Heartbeat, Winlogbeat, Packetbeat).
|
||||
-103
@@ -1,103 +0,0 @@
|
||||
---
|
||||
layout: default
|
||||
title: OpenSearch CLI
|
||||
nav_order: 52
|
||||
has_children: false
|
||||
redirect_from:
|
||||
- /docs/odfe-cli/
|
||||
- /docs/cli/
|
||||
---
|
||||
|
||||
# OpenSearch CLI
|
||||
|
||||
The OpenSearch CLI command line interface (opensearch-cli) lets you manage your OpenSearch cluster from the command line and automate tasks.
|
||||
|
||||
Currently, opensearch-cli supports the [Anomaly Detection]({{site.url}}{{site.baseurl}}/monitoring-plugins/ad/) and [k-NN]({{site.url}}{{site.baseurl}}/search-plugins/knn/) plugins, along with arbitrary REST API paths. Among other things, you can use opensearch-cli to create and delete detectors, start and stop them, and check k-NN statistics.
|
||||
|
||||
Profiles let you easily access different clusters or sign requests with different credentials. opensearch-cli supports unauthenticated requests, HTTP basic signing, and IAM signing for Amazon Web Services.
|
||||
|
||||
This example moves a detector (`ecommerce-count-quantity`) from a staging cluster to a production cluster:
|
||||
|
||||
```bash
|
||||
opensearch-cli ad get ecommerce-count-quantity --profile staging > ecommerce-count-quantity.json
|
||||
opensearch-cli ad create ecommerce-count-quantity.json --profile production
|
||||
opensearch-cli ad start ecommerce-count-quantity.json --profile production
|
||||
opensearch-cli ad stop ecommerce-count-quantity --profile staging
|
||||
opensearch-cli ad delete ecommerce-count-quantity --profile staging
|
||||
```
|
||||
|
||||
|
||||
## Install
|
||||
|
||||
1. [Download](https://opensearch.org/downloads.html){:target='\_blank'} and extract the appropriate installation package for your computer.
|
||||
|
||||
1. Make the `opensearch-cli` file executable:
|
||||
|
||||
```bash
|
||||
chmod +x ./opensearch-cli
|
||||
```
|
||||
|
||||
1. Add the command to your path:
|
||||
|
||||
```bash
|
||||
export PATH=$PATH:$(pwd)
|
||||
```
|
||||
|
||||
1. Confirm the CLI is working properly:
|
||||
|
||||
```bash
|
||||
opensearch-cli --version
|
||||
```
|
||||
|
||||
|
||||
## Profiles
|
||||
|
||||
Profiles let you easily switch between different clusters and user credentials. To get started, run `opensearch-cli profile create` with the `--auth-type`, `--endpoint`, and `--name` options:
|
||||
|
||||
```bash
|
||||
opensearch-cli profile create --auth-type basic --endpoint https://localhost:9200 --name docker-local
|
||||
```
|
||||
|
||||
Alternatively, save a configuration file to `~/.opensearch-cli/config.yaml`:
|
||||
|
||||
```yaml
|
||||
profiles:
|
||||
- name: docker-local
|
||||
endpoint: https://localhost:9200
|
||||
user: admin
|
||||
password: foobar
|
||||
- name: aws
|
||||
endpoint: https://some-cluster.us-east-1.es.amazonaws.com
|
||||
aws_iam:
|
||||
profile: ""
|
||||
service: es
|
||||
```
|
||||
|
||||
|
||||
## Usage
|
||||
|
||||
opensearch-cli commands use the following syntax:
|
||||
|
||||
```bash
|
||||
opensearch-cli <command> <subcommand> <flags>
|
||||
```
|
||||
|
||||
For example, the following command retrieves information about a detector:
|
||||
|
||||
```bash
|
||||
opensearch-cli ad get my-detector --profile docker-local
|
||||
```
|
||||
|
||||
For a request to the OpenSearch CAT API, try the following command:
|
||||
|
||||
```bash
|
||||
opensearch-cli curl get --path _cat/plugins --profile aws
|
||||
```
|
||||
|
||||
Use the `-h` or `--help` flag to see all supported commands, subcommands, or usage for a specific command:
|
||||
|
||||
```bash
|
||||
opensearch-cli -h
|
||||
opensearch-cli ad -h
|
||||
opensearch-cli ad get -h
|
||||
```
|
||||
-153
@@ -1,153 +0,0 @@
|
||||
title: OpenSearch documentation
|
||||
description: >- # this means to ignore newlines until "baseurl:"
|
||||
Documentation for OpenSearch, the Apache 2.0 search, analytics, and visualization suite with advanced security, alerting, SQL support, automated index management, deep performance analysis, and more.
|
||||
baseurl: "" # the subpath of your site, e.g. /blog
|
||||
url: "https://docs-beta.opensearch.org" # the base hostname & protocol for your site, e.g. http://example.com
|
||||
permalink: /:path/
|
||||
|
||||
opensearch_version: 1.0.0-rc1
|
||||
opensearch_major_minor_version: 1.0
|
||||
|
||||
# Build settings
|
||||
markdown: kramdown
|
||||
remote_theme: pmarsceill/[email protected]
|
||||
|
||||
# Kramdown settings
|
||||
kramdown:
|
||||
toc_levels: 2..3
|
||||
|
||||
logo: "/assets/images/logo.svg"
|
||||
|
||||
# Aux links for the upper right navigation
|
||||
aux_links:
|
||||
"Back to OpenSearch.org":
|
||||
- "https://opensearch.org"
|
||||
color_scheme: opensearch
|
||||
|
||||
# Define Jekyll collections
|
||||
collections:
|
||||
# Define a collection named "tests", its documents reside in the "_tests" directory
|
||||
upgrade-to:
|
||||
permalink: /:collection/:path/
|
||||
output: true
|
||||
opensearch:
|
||||
permalink: /:collection/:path/
|
||||
output: true
|
||||
dashboards:
|
||||
permalink: /:collection/:path/
|
||||
output: true
|
||||
security-plugin:
|
||||
permalink: /:collection/:path/
|
||||
output: true
|
||||
search-plugins:
|
||||
permalink: /:collection/:path/
|
||||
output: true
|
||||
im-plugin:
|
||||
permalink: /:collection/:path/
|
||||
output: true
|
||||
monitoring-plugins:
|
||||
permalink: /:collection/:path/
|
||||
output: true
|
||||
clients:
|
||||
permalink: /:collection/:path/
|
||||
output: true
|
||||
troubleshoot:
|
||||
permalink: /:collection/:path/
|
||||
output: true
|
||||
external_links:
|
||||
permalink: /:collection/:path/
|
||||
output: true
|
||||
|
||||
just_the_docs:
|
||||
# Define the collections used in the theme
|
||||
collections:
|
||||
upgrade-to:
|
||||
name: Upgrade to OpenSearch
|
||||
# nav_exclude: true
|
||||
nav_fold: true
|
||||
# search_exclude: true
|
||||
opensearch:
|
||||
name: OpenSearch
|
||||
nav_fold: true
|
||||
dashboards:
|
||||
name: OpenSearch Dashboards
|
||||
nav_fold: true
|
||||
security-plugin:
|
||||
name: Security plugin
|
||||
nav_fold: true
|
||||
search-plugins:
|
||||
name: Search plugins
|
||||
nav_fold: true
|
||||
im-plugin:
|
||||
name: Index management plugin
|
||||
nav_fold: true
|
||||
monitoring-plugins:
|
||||
name: Monitoring plugins
|
||||
nav_fold: true
|
||||
clients:
|
||||
name: Clients and tools
|
||||
nav_fold: true
|
||||
troubleshoot:
|
||||
name: Troubleshooting
|
||||
nav_fold: true
|
||||
external_links:
|
||||
name: External links
|
||||
|
||||
|
||||
# Enable or disable the site search
|
||||
# Supports true (default) or false
|
||||
search_enabled: true
|
||||
|
||||
search:
|
||||
# Split pages into sections that can be searched individually
|
||||
# Supports 1 - 6, default: 2
|
||||
heading_level: 2
|
||||
# Maximum amount of previews per search result
|
||||
# Default: 3
|
||||
previews: 3
|
||||
# Maximum amount of words to display before a matched word in the preview
|
||||
# Default: 5
|
||||
preview_words_before: 5
|
||||
# Maximum amount of words to display after a matched word in the preview
|
||||
# Default: 10
|
||||
preview_words_after: 10
|
||||
# Set the search token separator
|
||||
# Default: /[\s\-/]+/
|
||||
# Example: enable support for hyphenated search words
|
||||
tokenizer_separator: /[\s/]+/
|
||||
# Display the relative url in search results
|
||||
# Supports true (default) or false
|
||||
rel_url: true
|
||||
# Enable or disable the search button that appears in the bottom right corner of every page
|
||||
# Supports true or false (default)
|
||||
button: false
|
||||
|
||||
# Google Analytics Tracking (optional)
|
||||
# e.g, UA-1234567-89
|
||||
ga_tracking: G-BQV14XK08F
|
||||
|
||||
# Disable the just-the-docs theme anchor links in favor of our custom ones
|
||||
# See _includes/head_custom.html
|
||||
heading_anchors: false
|
||||
|
||||
# Adds on-hover anchor links to h2-h6
|
||||
anchor_links: true
|
||||
|
||||
footer_content:
|
||||
|
||||
plugins:
|
||||
- jekyll-remote-theme
|
||||
- jekyll-redirect-from
|
||||
|
||||
# Exclude from processing.
|
||||
# The following items will not be processed, by default. Create a custom list
|
||||
# to override the default setting.
|
||||
exclude:
|
||||
- Gemfile
|
||||
- Gemfile.lock
|
||||
- node_modules
|
||||
- vendor/bundle/
|
||||
- vendor/cache/
|
||||
- vendor/gems/
|
||||
- vendor/ruby/
|
||||
- README.md
|
||||
@@ -1,17 +0,0 @@
|
||||
---
|
||||
layout: default
|
||||
title: Browser compatibility
|
||||
parent: OpenSearch Dashboards
|
||||
nav_order: 3
|
||||
---
|
||||
|
||||
# Browser compatibility
|
||||
|
||||
OpenSearch Dashboards supports the following web browsers:
|
||||
|
||||
- Chrome
|
||||
- Firefox
|
||||
- Safari
|
||||
- Edge (Chromium)
|
||||
|
||||
Other Chromium-based browsers might work, as well. Internet Explorer and Microsoft Edge Legacy are **not** supported.
|
||||
@@ -1,25 +0,0 @@
|
||||
---
|
||||
layout: default
|
||||
title: Gantt charts
|
||||
nav_order: 10
|
||||
---
|
||||
|
||||
# Gantt charts
|
||||
|
||||
OpenSearch Dashboards includes a Gantt chart visualization. Gantt charts show the start, end, and duration of unique events in a sequence. Gantt charts are useful in trace analytics, telemetry, and anomaly detection use cases, where you want to understand interactions and dependencies between various events in a schedule.
|
||||
|
||||
For example, consider an index of log data. The fields in a typical set of log data, especially audit logs, contain a specific operation or event with a start time and duration.
|
||||
|
||||
To create a Gantt chart, perform the following steps:
|
||||
|
||||
1. In the visualizations menu, choose **Create visualization** and **Gantt Chart**.
|
||||
1. Choose a source for the chart (e.g. some log data).
|
||||
1. Under **Metrics**, choose **Event**. For log data, each log is an event.
|
||||
1. Select the **Start Time** and **Duration** fields from your data set. The start time is the timestamp for the begining of an event. The duration is the amount of time to add to the start time.
|
||||
1. Under **Results**, choose the number of events to display on the chart. Gantt charts sequence events from earliest to latest based on start time.
|
||||
1. Choose **Panel settings** to adjust axis labels, time format, and colors.
|
||||
1. Choose **Update**.
|
||||
|
||||

|
||||
|
||||
This Gantt chart displays the ID of each log on the y-axis. Each bar is a unique event that spans some amount of time. Hover over a bar to see the duration of that event.
|
||||
@@ -1,24 +0,0 @@
|
||||
---
|
||||
layout: default
|
||||
title: About Dashboards
|
||||
nav_order: 1
|
||||
has_children: false
|
||||
has_toc: false
|
||||
redirect_from:
|
||||
- /docs/opensearch-dashboards/
|
||||
- /opensearch-dashboards/
|
||||
- /dashboards/
|
||||
---
|
||||
|
||||
# OpenSearch Dashboards
|
||||
|
||||
OpenSearch Dashboards is the default visualization tool for data in OpenSearch. It also serves as a user interface for many of the OpenSearch plugins, including security, alerting, Index State Management, SQL, and more.
|
||||
|
||||
|
||||
## Get started with OpenSearch Dashboards
|
||||
|
||||
1. After starting OpenSearch Dashboards, you can access it at port 5601. For example, http://localhost:5601.
|
||||
1. Log in with the default username `admin` and password `admin`.
|
||||
1. Choose **Try our sample data** and add the sample flight data.
|
||||
1. Choose **Discover** and search for a few flights.
|
||||
1. Choose **Dashboard**, **[Flights] Global Flight Dashboard**, and wait for the dashboard to load.
|
||||
@@ -1,23 +0,0 @@
|
||||
---
|
||||
layout: default
|
||||
title: Docker
|
||||
parent: Install OpenSearch Dashboards
|
||||
nav_order: 1
|
||||
---
|
||||
|
||||
# Run OpenSearch Dashboards using Docker
|
||||
|
||||
You *can* start OpenSearch Dashboards using `docker run` after [creating a Docker network](https://docs.docker.com/engine/reference/commandline/network_create/) and starting OpenSearch, but the process of connecting OpenSearch Dashboards to OpenSearch is significantly easier with a Docker Compose file.
|
||||
|
||||
1. Run `docker pull opensearchproject/opensearch-dashboards:{{site.opensearch_version}}`.
|
||||
|
||||
1. Create a [`docker-compose.yml`](https://docs.docker.com/compose/compose-file/) file appropriate for your environment. A sample file that includes OpenSearch Dashboards is available on the OpenSearch [Docker installation page]({{site.url}}{{site.baseurl}}/opensearch/install/docker#sample-docker-compose-file).
|
||||
|
||||
Just like `opensearch.yml`, you can pass a custom `opensearch_dashboards.yml` to the container in the Docker Compose file.
|
||||
{: .tip }
|
||||
|
||||
1. Run `docker-compose up`.
|
||||
|
||||
Wait for the containers to start. Then see the [OpenSearch Dashboards documentation]({{site.url}}{{site.baseurl}}/).
|
||||
|
||||
1. When finished, run `docker-compose down`.
|
||||
@@ -1,12 +0,0 @@
|
||||
---
|
||||
layout: default
|
||||
title: Install OpenSearch Dashboards
|
||||
nav_order: 1
|
||||
has_children: true
|
||||
redirect_from:
|
||||
- /dashboards/install/
|
||||
---
|
||||
|
||||
# Install and configure OpenSearch Dashboards
|
||||
|
||||
OpenSearch Dashboards has two installation options at this time: Docker images and tarballs.
|
||||
@@ -1,223 +0,0 @@
|
||||
---
|
||||
layout: default
|
||||
title: OpenSearch Dashboards plugins
|
||||
parent: Install OpenSearch Dashboards
|
||||
nav_order: 50
|
||||
---
|
||||
|
||||
# Standalone plugin install
|
||||
|
||||
If you don't want to use the all-in-one installation options, you can install the various plugins for OpenSearch Dashboards individually.
|
||||
|
||||
---
|
||||
|
||||
#### Table of contents
|
||||
1. TOC
|
||||
{:toc}
|
||||
|
||||
|
||||
---
|
||||
|
||||
## Plugin compatibility
|
||||
|
||||
<table>
|
||||
<thead style="text-align: left">
|
||||
<tr>
|
||||
<th>OpenSearch Dashboards version</th>
|
||||
<th>Plugin versions</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>1.0.0-rc1</td>
|
||||
<td>
|
||||
<pre>alertingDashboards 1.0.0.0-rc1
|
||||
anomalyDetectionDashboards 1.0.0.0-rc1
|
||||
ganttChartDashboards 1.0.0.0-rc1
|
||||
indexManagementDashboards 1.0.0.0-rc1
|
||||
notebooksDashboards 1.0.0.0-rc1
|
||||
queryWorkbenchDashboards 1.0.0.0-rc1
|
||||
reportsDashboards 1.0.0.0-rc1
|
||||
securityDashboards 1.0.0.0-rc1
|
||||
traceAnalyticsDashboards 1.0.0.0-rc1
|
||||
</pre>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>1.0.0-beta1</td>
|
||||
<td>
|
||||
<pre>alertingDashboards 1.0.0.0-beta1
|
||||
anomalyDetectionDashboards 1.0.0.0-beta1
|
||||
ganttChartDashboards 1.0.0.0-beta1
|
||||
indexManagementDashboards 1.0.0.0-beta1
|
||||
notebooksDashboards 1.0.0.0-beta1
|
||||
queryWorkbenchDashboards 1.0.0.0-beta1
|
||||
reportsDashboards 1.0.0.0-beta1
|
||||
securityDashboards 1.0.0.0-beta1
|
||||
traceAnalyticsDashboards 1.0.0.0-beta1
|
||||
</pre>
|
||||
</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- A compatible OpenSearch cluster
|
||||
- The corresponding OpenSearch plugins [installed on that cluster]({{site.url}}{{site.baseurl}}/opensearch/install/plugins/)
|
||||
- The corresponding version of [OpenSearch Dashboards]({{site.url}}{{site.baseurl}}/) (e.g. OpenSearch Dashboards 1.0.0 works with OpenSearch 1.0.0)
|
||||
|
||||
|
||||
## Install
|
||||
|
||||
Navigate to the OpenSearch Dashboards home directory (likely `/usr/share/opensearch-dashboards`) and run the install command for each plugin.
|
||||
|
||||
{% comment %}
|
||||
|
||||
#### Security OpenSearch Dashboards
|
||||
|
||||
```bash
|
||||
sudo bin/opensearch-dashboards-plugin install https://d3g5vo6xdbdb9a.cloudfront.net/downloads/opensearch-dashboards-plugins/opensearch-security/opensearchSecurityOpenSearch Dashboards-{{site.opensearch_major_minor_version}}.0.1.zip
|
||||
```
|
||||
|
||||
This plugin provides a user interface for managing users, roles, mappings, action groups, and tenants.
|
||||
|
||||
|
||||
#### Alerting OpenSearch Dashboards
|
||||
|
||||
```bash
|
||||
sudo bin/opensearch-dashboards-plugin install https://d3g5vo6xdbdb9a.cloudfront.net/downloads/opensearch-dashboards-plugins/opensearch-alerting/opensearchAlertingOpenSearch Dashboards-{{site.opensearch_major_minor_version}}.0.0.zip
|
||||
```
|
||||
|
||||
This plugin provides a user interface for creating monitors and managing alerts.
|
||||
|
||||
|
||||
#### Index State Management OpenSearch Dashboards
|
||||
|
||||
```bash
|
||||
sudo bin/opensearch-dashboards-plugin install https://d3g5vo6xdbdb9a.cloudfront.net/downloads/opensearch-dashboards-plugins/opensearch-index-management/opensearchIndexManagementOpenSearch Dashboards-{{site.opensearch_major_minor_version}}.0.1.zip
|
||||
```
|
||||
|
||||
This plugin provides a user interface for managing policies.
|
||||
|
||||
|
||||
#### Anomaly Detection OpenSearch Dashboards
|
||||
|
||||
```bash
|
||||
sudo bin/opensearch-dashboards-plugin install https://d3g5vo6xdbdb9a.cloudfront.net/downloads/opensearch-dashboards-plugins/opensearch-anomaly-detection/opensearchAnomalyDetectionOpenSearch Dashboards-{{site.opensearch_major_minor_version}}.0.0.zip
|
||||
```
|
||||
|
||||
This plugin provides a user interface for adding detectors.
|
||||
|
||||
|
||||
#### Query Workbench OpenSearch Dashboards
|
||||
|
||||
```bash
|
||||
sudo bin/opensearch-dashboards-plugin install https://d3g5vo6xdbdb9a.cloudfront.net/downloads/opensearch-dashboards-plugins/opensearch-query-workbench/opensearchQueryWorkbenchOpenSearch Dashboards-{{site.opensearch_major_minor_version}}.0.0.zip
|
||||
```
|
||||
|
||||
This plugin provides a user interface for using SQL queries to explore your data.
|
||||
|
||||
|
||||
#### Trace Analytics
|
||||
|
||||
```bash
|
||||
sudo bin/opensearch-dashboards-plugin install https://d3g5vo6xdbdb9a.cloudfront.net/downloads/opensearch-dashboards-plugins/opensearch-trace-analytics/opensearchTraceAnalyticsOpenSearch Dashboards-{{site.opensearch_major_minor_version}}.2.0.zip
|
||||
```
|
||||
|
||||
This plugin uses distributed trace data (indexed in OpenSearch using Data Prepper) to display latency trends, error rates, and more.
|
||||
|
||||
|
||||
#### Notebooks OpenSearch Dashboards
|
||||
|
||||
```bash
|
||||
sudo bin/opensearch-dashboards-plugin install https://d3g5vo6xdbdb9a.cloudfront.net/downloads/opensearch-dashboards-plugins/opensearch-notebooks/opensearchNotebooksOpenSearch Dashboards-{{site.opensearch_major_minor_version}}.2.0.zip
|
||||
```
|
||||
|
||||
This plugin lets you combine OpenSearch Dashboards visualizations and narrative text in a single interface.
|
||||
|
||||
|
||||
#### Reports OpenSearch Dashboards
|
||||
|
||||
```bash
|
||||
# x86 Linux
|
||||
sudo bin/opensearch-dashboards-plugin install https://d3g5vo6xdbdb9a.cloudfront.net/downloads/opensearch-dashboards-plugins/opensearch-reports/linux/x64/opensearchReportsOpenSearch Dashboards-{{site.opensearch_major_minor_version}}.2.0-linux-x64.zip
|
||||
# ARM64 Linux
|
||||
sudo bin/opensearch-dashboards-plugin install https://d3g5vo6xdbdb9a.cloudfront.net/downloads/opensearch-dashboards-plugins/opensearch-reports/linux/arm64/opensearchReportsOpenSearch Dashboards-{{site.opensearch_major_minor_version}}.2.0-linux-arm64.zip
|
||||
# x86 Windows
|
||||
sudo bin/opensearch-dashboards-plugin install https://d3g5vo6xdbdb9a.cloudfront.net/downloads/opensearch-dashboards-plugins/opensearch-reports/windows/x64/opensearchReportsOpenSearch Dashboards-{{site.opensearch_major_minor_version}}.2.0-windows-x64.zip
|
||||
```
|
||||
|
||||
This plugin lets you export and share reports from OpenSearch Dashboards dashboards, visualizations, and saved searches.
|
||||
|
||||
|
||||
#### Gantt Chart OpenSearch Dashboards
|
||||
|
||||
```bash
|
||||
sudo bin/opensearch-dashboards-plugin install https://d3g5vo6xdbdb9a.cloudfront.net/downloads/opensearch-dashboards-plugins/opensearch-gantt-chart/opensearchGanttChartOpenSearch Dashboards-{{site.opensearch_major_minor_version}}.0.0.zip
|
||||
```
|
||||
|
||||
This plugin adds a new Gantt chart visualization.
|
||||
|
||||
{% endcomment %}
|
||||
|
||||
## List installed plugins
|
||||
|
||||
To check your installed plugins:
|
||||
|
||||
```bash
|
||||
sudo bin/opensearch-dashboards-plugin list
|
||||
```
|
||||
|
||||
|
||||
## Remove plugins
|
||||
|
||||
To remove a plugin:
|
||||
|
||||
```bash
|
||||
sudo bin/opensearch-dashboards-plugin remove <plugin-name>
|
||||
```
|
||||
|
||||
Then remove all associated entries from `opensearch_dashboards.yml`.
|
||||
|
||||
For certain plugins, you must also remove the "optimze" bundle. This is a sample command for the Anomaly Detection plugin:
|
||||
|
||||
```bash
|
||||
sudo rm /usr/share/opensearch-dashboards/optimize/bundles/opensearch-anomaly-detection-opensearch-dashboards.*
|
||||
```
|
||||
|
||||
Then restart OpenSearch Dashboards. After you remove any plugin, OpenSearch Dashboards performs an optimize operation the next time you start it. This operation takes several minutes even on fast machines, so be patient.
|
||||
|
||||
|
||||
## Update plugins
|
||||
|
||||
OpenSearch Dashboards doesn’t update plugins. Instead, you have to remove the old version and its optimized bundle, reinstall them, and restart OpenSearch Dashboards:
|
||||
|
||||
1. Remove the old version:
|
||||
|
||||
```bash
|
||||
sudo bin/opensearch-dashboards-plugin remove <plugin-name>
|
||||
```
|
||||
|
||||
1. Remove the optimized bundle:
|
||||
|
||||
```bash
|
||||
sudo rm /usr/share/opensearch-dashboards/optimize/bundles/<bundle-name>
|
||||
```
|
||||
|
||||
1. Reinstall the new version:
|
||||
|
||||
```bash
|
||||
sudo bin/opensearch-dashboards-plugin install <plugin-name>
|
||||
```
|
||||
|
||||
1. Restart OpenSearch Dashboards.
|
||||
|
||||
For example, to remove and reinstall the anomaly detection plugin:
|
||||
|
||||
```bash
|
||||
sudo bin/opensearch-plugin remove opensearch-anomaly-detection
|
||||
sudo rm /usr/share/opensearch-dashboards/optimize/bundles/opensearch-anomaly-detection-opensearch-dashboards.*
|
||||
sudo bin/opensearch-dashboards-plugin install <AD OpenSearch Dashboards plugin artifact URL>
|
||||
```
|
||||
@@ -1,30 +0,0 @@
|
||||
---
|
||||
layout: default
|
||||
title: Tarball
|
||||
parent: Install OpenSearch Dashboards
|
||||
nav_order: 30
|
||||
---
|
||||
|
||||
# Run OpenSearch Dashboards using the tarball
|
||||
|
||||
1. Download the tarball from the [OpenSearch downloads page](https://opensearch.org/downloads.html){:target='\_blank'}.
|
||||
|
||||
1. Extract the TAR file to a directory and change to that directory:
|
||||
|
||||
```bash
|
||||
# x64
|
||||
tar -zxf opensearch-dashboards-{{site.opensearch_version}}-linux-x64.tar.gz
|
||||
cd opensearch-dashboards{% comment %}# ARM64
|
||||
tar -zxf opensearch-dashboards-{{site.opensearch_version}}-linux-arm64.tar.gz
|
||||
cd opensearch-dashboards{% endcomment %}
|
||||
```
|
||||
|
||||
1. If desired, modify `config/opensearch_dashboards.yml`.
|
||||
|
||||
1. Run OpenSearch Dashboards:
|
||||
|
||||
```bash
|
||||
./bin/opensearch-dashboards
|
||||
```
|
||||
|
||||
1. See the [OpenSearch Dashboards documentation]({{site.url}}{{site.baseurl}}/opensearch-dashboards/).
|
||||
@@ -1,29 +0,0 @@
|
||||
---
|
||||
layout: default
|
||||
title: WMS map server
|
||||
nav_order: 5
|
||||
---
|
||||
|
||||
# Configure WMS map server
|
||||
|
||||
OpenSearch Dashboards includes default map tiles, but if you need more specialized maps, you can configure OpenSearch Dashboards to use a WMS map server:
|
||||
|
||||
1. Open OpenSearch Dashboards at `https://<host>:<port>`. For example, [https://localhost:5601](https://localhost:5601).
|
||||
1. If necessary, log in.
|
||||
1. Choose **Management** and **Advanced Settings**.
|
||||
1. Locate `visualization:tileMap:WMSdefaults`.
|
||||
1. Change `enabled` to true and add the URL of a valid WMS map server:
|
||||
|
||||
```json
|
||||
{
|
||||
"enabled": true,
|
||||
"url": "<wms-map-server-url>",
|
||||
"options": {
|
||||
"format": "image/png",
|
||||
"transparent": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Map services often have licensing fees or restrictions. You're responsible for all such considerations on any map server that you specify.
|
||||
{: .note }
|
||||
@@ -1,118 +0,0 @@
|
||||
---
|
||||
layout: default
|
||||
title: Notebooks
|
||||
nav_order: 50
|
||||
redirect_from: /docs/notebooks/
|
||||
has_children: false
|
||||
---
|
||||
|
||||
# Notebooks
|
||||
|
||||
An OpenSearch Dashboards notebook is an interface that lets you easily combine code snippets, live visualizations, and narrative text in a single notebook interface.
|
||||
|
||||
Notebooks let you interactively explore data by running different visualizations that you can share with team members to collaborate on a project.
|
||||
|
||||
A notebook is a document composed of two elements: code blocks (Markdown/SQL/PPL) and visualizations. Choose multiple timelines to compare and contrast visualizations.
|
||||
|
||||
You can also generate [reports]({{site.url}}{{site.baseurl}}/dashboards/reporting/) directly from your notebooks.
|
||||
|
||||
Common use cases include creating postmortem reports, designing runbooks, building live infrastructure reports, and writing documentation.
|
||||
|
||||
Tenants in OpenSearch Dashboards are spaces for saving notebooks and other OpenSearch Dashboards objects. For more information, see [OpenSearch Dashboards multi-tenancy]({{site.url}}{{site.baseurl}}/security-plugin/access-control/multi-tenancy/).
|
||||
{: .note }
|
||||
|
||||
|
||||
## Get started with notebooks
|
||||
|
||||
To get started, choose **Notebooks** within OpenSearch Dashboards.
|
||||
|
||||
|
||||
### Step 1: Create a notebook
|
||||
|
||||
A notebook is an interface for creating reports.
|
||||
|
||||
1. Choose **Create notebook** and enter a descriptive name.
|
||||
1. Choose **Create**.
|
||||
|
||||
Choose **Actions** to rename, duplicate, or delete a notebook.
|
||||
|
||||
|
||||
### Step 2: Add a paragraph
|
||||
|
||||
Paragraphs combine code blocks and visualizations for describing data.
|
||||
|
||||
#### Add a code block
|
||||
|
||||
Code blocks support markdown, SQL, and PPL languages.
|
||||
|
||||
Specify the input language on the first line using `%[language type]` syntax.
|
||||
For example, type `%md` for markdown, `%sql` for SQL, and `%ppl` for PPL.
|
||||
|
||||
##### Sample markdown block
|
||||
|
||||
```
|
||||
%md
|
||||
Add in text formatted in markdown.
|
||||
```
|
||||
|
||||

|
||||
|
||||
##### Sample SQL block
|
||||
|
||||
```sql
|
||||
%sql
|
||||
Select * from opensearch_dashboards_sample_data_flights limit 20;
|
||||
```
|
||||
|
||||

|
||||
|
||||
##### Sample PPL block
|
||||
|
||||
```
|
||||
%ppl
|
||||
source=opensearch_dashboards_sample_data_logs | head 20
|
||||
```
|
||||
|
||||

|
||||
|
||||
|
||||
#### Add a visualization
|
||||
|
||||
1. To add a visualization, choose **Add paragraph** and select **Visualization**.
|
||||
1. In **Title**, select your visualization and choose a date range. You can choose multiple timelines to compare and contrast visualizations.
|
||||
1. To run and save a paragraph, choose **Run**.
|
||||
|
||||

|
||||
|
||||
## Paragraph actions
|
||||
|
||||
You can perform the following actions on paragraphs:
|
||||
|
||||
- Add a new paragraph to the top of a report.
|
||||
- Add a new paragraph to the bottom of a report.
|
||||
- Run all the paragraphs at the same time.
|
||||
- Clear the outputs of all paragraphs.
|
||||
- Delete all the paragraphs.
|
||||
- Move paragraphs up and down.
|
||||
|
||||
## Sample notebooks
|
||||
|
||||
We prepared the following sample notebooks that showcase a variety of use cases:
|
||||
|
||||
- Using SQL to query the OpenSearch Dashboards sample flight data.
|
||||
- Using PPL to query the OpenSearch Dashboards sample web logs data.
|
||||
- Using PPL and visualizations to perform sample root cause event analysis on the OpenSearch Dashboards sample web logs data.
|
||||
|
||||
To add a sample notebook, choose **Actions** and select **Add sample notebooks**.
|
||||
|
||||
## Create a report
|
||||
|
||||
You can use notebooks to create PNG and PDF reports:
|
||||
|
||||
1. From the top menu bar, choose **Reporting actions**.
|
||||
1. You can choose to **Download PDF** or **Download PNG**.
|
||||
|
||||
Reports generate asynchronously in the background and might take a few minutes, depending on the size of the report. A notification appears when your report is ready to download.
|
||||
|
||||
1. To create a schedule-based report, choose **Create report definition**. For steps to create a report definition, see [Create reports using a definition]({{site.url}}{{site.baseurl}}/dashboards/reporting#create-reports-using-a-definition).
|
||||
1. To see all your reports, choose **View all reports**.
|
||||
@@ -1,57 +0,0 @@
|
||||
---
|
||||
layout: default
|
||||
title: Reporting
|
||||
nav_order: 20
|
||||
---
|
||||
|
||||
|
||||
# Reporting
|
||||
|
||||
You can use OpenSearch Dashboards to create PNG, PDF, and CSV reports. To create reports, you must have the correct permissions. For a summary of the predefined roles and the permissions they grant, see the [security plugin]({{site.url}}{{site.baseurl}}/security-plugin/access-control/users-roles#predefined-roles).
|
||||
|
||||
CSV reports have a non-configurable 10,000 row limit. They have no explicit size limit (e.g. in MB), but extremely large documents could cause report generation to fail with an out of memory error from the V8 JavaScript engine.
|
||||
{: .tip }
|
||||
|
||||
|
||||
## Create reports from Discovery, Visualize, or Dashboard
|
||||
|
||||
Quickly generate an on-demand report from the current view.
|
||||
|
||||
1. From the top menu bar, choose **Reporting**.
|
||||
1. For dashboards or visualizations, choose **Download PDF** or **Download PNG**. From the Discover page, choose **Download CSV**.
|
||||
|
||||
Reports generate asynchronously in the background and might take a few minutes, depending on the size of the report. A notification appears when your report is ready to download.
|
||||
|
||||
1. To create a schedule-based report, choose **Create report definition**. Then proceed to [Create reports using a definition](#create-reports-using-a-definition). This option pre-fills many of the fields for you based on the visualization, dashboard, or data you were viewing.
|
||||
|
||||
|
||||
## Create reports using a definition
|
||||
|
||||
Definitions let you generate reports on a periodic schedule.
|
||||
|
||||
1. From the navigation panel, choose **Reporting**.
|
||||
1. Choose **Create**.
|
||||
1. Under **Report settings**, enter a name and optional description for your report.
|
||||
1. Choose the **Report Source** (i.e. the page from which the report is generated). You can generate reports from the **Dashboard**, **Visualize**, or **Discover** pages.
|
||||
1. Select your dashboard, visualization, or saved search. Then choose a time range for the report.
|
||||
1. Choose an appropriate file format for the report.
|
||||
1. (Optional) Add a header or footer to the report. Headers and footers are only available for dashboard or visualization reports.
|
||||
1. Under **Report trigger**, choose either **On-demand** or **Schedule**.
|
||||
|
||||
For scheduled reports, select either **Recurring** or **Cron based**. You can receive reports daily or at some other time interval. Cron expressions give you even more flexiblity. See [Cron expression reference]({{site.url}}{{site.baseurl}}/monitoring-plugins/alerting/cron/) for more information.
|
||||
|
||||
1. Choose **Create**.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Chromium fails to launch with OpenSearch Dashboards
|
||||
|
||||
While creating a report for dashboards or visualizations, you might see a the following error:
|
||||
|
||||

|
||||
|
||||
This problem can occur for two reasons:
|
||||
|
||||
- You don't have the correct version of `headless-chrome` to match the operating system on which OpenSearch Dashboards is running. Download the correct version [here](https://github.com/opensearch-project/dashboards-reports/releases/tag/chromium-1.12.0.0).
|
||||
|
||||
- You're missing additional dependencies. Install the required dependencies for your operating system from the [additional libraries](https://github.com/opensearch-project/dashboards-reports/blob/main/dashboards-reports/rendering-engine/headless-chrome/README.md#additional-libaries) section.
|
||||
@@ -1,7 +0,0 @@
|
||||
---
|
||||
layout: default
|
||||
title: Javadoc
|
||||
nav_order: 1
|
||||
permalink: /javadoc/
|
||||
redirect_to: https://opensearch.org/docs/javadocs/
|
||||
---
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,234 +0,0 @@
|
||||
---
|
||||
layout: default
|
||||
title: Index rollups API
|
||||
parent: Index rollups
|
||||
redirect_from: /docs/ism/rollup-api/
|
||||
nav_order: 9
|
||||
---
|
||||
|
||||
# Index rollups API
|
||||
|
||||
Use the index rollup operations to programmatically work with index rollup jobs.
|
||||
|
||||
---
|
||||
|
||||
#### Table of contents
|
||||
- TOC
|
||||
{:toc}
|
||||
|
||||
|
||||
---
|
||||
|
||||
## Create or update an index rollup job
|
||||
|
||||
Creates or updates an index rollup job.
|
||||
You must provide the `seq_no` and `primary_term` parameters.
|
||||
|
||||
#### Request
|
||||
|
||||
```json
|
||||
PUT _plugins/_rollup/jobs/<rollup_id> // Create
|
||||
PUT _plugins/_rollup/jobs/<rollup_id>?if_seq_no=1&if_primary_term=1 // Update
|
||||
{
|
||||
"rollup": {
|
||||
"source_index": "nyc-taxi-data",
|
||||
"target_index": "rollup-nyc-taxi-data",
|
||||
"schedule": {
|
||||
"interval": {
|
||||
"period": 1,
|
||||
"unit": "Days"
|
||||
}
|
||||
},
|
||||
"description": "Example rollup job",
|
||||
"enabled": true,
|
||||
"page_size": 200,
|
||||
"delay": 0,
|
||||
"roles": [
|
||||
"rollup_all",
|
||||
"nyc_taxi_all",
|
||||
"example_rollup_index_all"
|
||||
],
|
||||
"continuous": false,
|
||||
"dimensions": {
|
||||
"date_histogram": {
|
||||
"source_field": "tpep_pickup_datetime",
|
||||
"fixed_interval": "1h",
|
||||
"timezone": "America/Los_Angeles"
|
||||
},
|
||||
"terms": {
|
||||
"source_field": "PULocationID"
|
||||
},
|
||||
"metrics": [
|
||||
{
|
||||
"source_field": "passenger_count",
|
||||
"metrics": [
|
||||
{
|
||||
"avg": {}
|
||||
},
|
||||
{
|
||||
"sum": {}
|
||||
},
|
||||
{
|
||||
"max": {}
|
||||
},
|
||||
{
|
||||
"min": {}
|
||||
},
|
||||
{
|
||||
"value_count": {}
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
You can specify the following options.
|
||||
|
||||
Options | Description | Type | Required
|
||||
:--- | :--- |:--- |:--- |
|
||||
`source_index` | The name of the detector. | `string` | Yes
|
||||
`target_index` | Specify the target index that the rolled up data is ingested into. You could either create a new target index or use an existing index. The target index cannot be a combination of raw and rolled up data. | `string` | Yes
|
||||
`schedule` | Schedule of the index rollup job which can be an interval or a cron expression. | `object` | Yes
|
||||
`schedule.interval` | Specify the frequency of execution of the rollup job. | `object` | No
|
||||
`schedule.interval.start_time` | Start time of the interval. | `timestamp` | Yes
|
||||
`schedule.interval.period` | Define the interval period. | `string` | Yes
|
||||
`schedule.interval.unit` | Specify the time unit of the interval. | `string` | Yes
|
||||
`schedule.interval.cron` | Optionally, specify a cron expression to define therollup frequency. | `list` | No
|
||||
`schedule.interval.cron.expression` | Specify a Unix cron expression. | `string` | Yes
|
||||
`schedule.interval.cron.timezone` | Specify timezones as defined by the IANA Time Zone Database. Defaults to UTC. | `string` | No
|
||||
`description` | Optionally, describe the rollup job. | `string` | No
|
||||
`enabled` | When true, the index rollup job is scheduled. Default is true. | `boolean` | Yes
|
||||
`continuous` | Specify whether or not the index rollup job continuously rolls up data forever or just executes over the current data set once and stops. Default is false. | `boolean` | Yes
|
||||
`error_notification` | Set up a Mustache message template sent for error notifications. For example, if an index rollup job fails, the system sends a message to a Slack channel. | `object` | No
|
||||
`page_size` | Specify the number of buckets to paginate through at a time while rolling up. | `number` | Yes
|
||||
`delay` | Specify time value to delay execution of the index rollup job. | `time_unit` | No
|
||||
`dimensions` | Specify aggregations to create dimensions for the roll up time window. | `object` | Yes
|
||||
`dimensions.date_histogram` | Specify either fixed_interval or calendar_interval, but not both. Either one limits what you can query in the target index. | `object` | No
|
||||
`dimensions.date_histogram.fixed_interval` | Specify the fixed interval for aggregations in milliseconds, seconds, minutes, hours, or days. | `string` | No
|
||||
`dimensions.date_histogram.calendar_interval` | Specify the calendar interval for aggregations in minutes, hours, days, weeks, months, quarters, or years. | `string` | No
|
||||
`dimensions.date_histogram.field` | Specify the date field used in date histogram aggregation. | `string` | No
|
||||
`dimensions.date_histogram.timezone` | Specify the timezones as defined by the IANA Time Zone Database. The default is UTC. | `string` | No
|
||||
`dimensions.terms` | Specify the term aggregations that you want to roll up. | `object` | No
|
||||
`dimensions.terms.fields` | Specify terms aggregation for compatible fields. | `object` | No
|
||||
`dimensions.histogram` | Specify the histogram aggregations that you want to roll up. | `object` | No
|
||||
`dimensions.histogram.field` | Add a field for histogram aggregations. | `string` | Yes
|
||||
`dimensions.histogram.interval` | Specify the histogram aggregation interval for the field. | `long` | Yes
|
||||
`dimensions.metrics` | Specify a list of objects that represent the fields and metrics that you want to calculate. | `nested object` | No
|
||||
`dimensions.metrics.field` | Specify the field that you want to perform metric aggregations on. | `string` | No
|
||||
`dimensions.metrics.field.metrics` | Specify the metric aggregations you want to calculate for the field. | `multiple strings` | No
|
||||
|
||||
|
||||
#### Sample response
|
||||
|
||||
```json
|
||||
{
|
||||
"_id": "rollup_id",
|
||||
"_seqNo": 1,
|
||||
"_primaryTerm": 1,
|
||||
"rollup": { ... }
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
## Get an index rollup job
|
||||
|
||||
Returns all information about an index rollup job based on the `rollup_id`.
|
||||
|
||||
#### Request
|
||||
|
||||
```json
|
||||
GET _plugins/_rollup/jobs/<rollup_id>
|
||||
```
|
||||
|
||||
|
||||
#### Sample response
|
||||
|
||||
```json
|
||||
{
|
||||
"_id": "my_rollup",
|
||||
"_seqNo": 1,
|
||||
"_primaryTerm": 1,
|
||||
"rollup": { ... }
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
---
|
||||
|
||||
## Delete an index rollup job
|
||||
|
||||
Deletes an index rollup job based on the `rollup_id`.
|
||||
|
||||
#### Request
|
||||
|
||||
```json
|
||||
DELETE _plugins/_rollup/jobs/<rollup_id>
|
||||
```
|
||||
|
||||
#### Sample response
|
||||
|
||||
```json
|
||||
200 OK
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
|
||||
## Start or stop an index rollup job
|
||||
|
||||
Start or stop an index rollup job.
|
||||
|
||||
#### Request
|
||||
|
||||
```json
|
||||
POST _plugins/_rollup/jobs/<rollup_id>/_start
|
||||
POST _plugins/_rollup/jobs/<rollup_id>/_stop
|
||||
```
|
||||
|
||||
|
||||
#### Sample response
|
||||
|
||||
```json
|
||||
200 OK
|
||||
```
|
||||
|
||||
|
||||
---
|
||||
|
||||
## Explain an index rollup job
|
||||
|
||||
Returns detailed metadata information about the index rollup job and its current progress.
|
||||
|
||||
#### Request
|
||||
|
||||
```json
|
||||
GET _plugins/_rollup/jobs/<rollup_id>/_explain
|
||||
```
|
||||
|
||||
|
||||
#### Sample response
|
||||
|
||||
```json
|
||||
{
|
||||
"example_rollup": {
|
||||
"rollup_id": "example_rollup",
|
||||
"last_updated_time": 1602014281,
|
||||
"continuous": {
|
||||
"next_window_start_time": 1602055591,
|
||||
"next_window_end_time": 1602075591
|
||||
},
|
||||
"status": "running",
|
||||
"failure_reason": null,
|
||||
"stats": {
|
||||
"pages_processed": 342,
|
||||
"documents_processed": 489359,
|
||||
"rollups_indexed": 3420,
|
||||
"index_time_in_ms": 30495,
|
||||
"search_time_in_ms": 584922
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
@@ -1,156 +0,0 @@
|
||||
---
|
||||
layout: default
|
||||
title: Index transforms
|
||||
nav_order: 20
|
||||
has_children: true
|
||||
redirect_from: /docs/im/index-transforms/
|
||||
has_toc: false
|
||||
---
|
||||
|
||||
# Index transforms
|
||||
|
||||
Whereas index rollup jobs let you reduce data granularity by rolling up old data into condensed indices, transform jobs let you create a different, summarized view of your data centered around certain fields, so you can visualize or analyze the data in different ways.
|
||||
|
||||
For example, suppose that you have airline data that’s scattered across multiple fields and categories, and you want to view a summary of the data that’s organized by airline, quarter, and then price. You can use a transform job to create a new, summarized index that’s organized by those specific categories.
|
||||
|
||||
You can use transform jobs in two ways:
|
||||
|
||||
1. Use the OpenSearch Dashboards UI to specify the index you want to transform and any optional data filters you want to use to filter the original index. Then select the fields you want to transform and the aggregations to use in the transformation. Finally, define a schedule for your job to follow.
|
||||
2. Use the transforms API to specify all the details about your job: the index you want to transform, target groups you want the transformed index to have, any aggregations you want to use to group columns, and a schedule for your job to follow.
|
||||
|
||||
OpenSearch Dashboards provides a detailed summary of the jobs you created and their relevant information, such as associated indices and job statuses. You can review and edit your job’s details and selections before creation, and even preview a transformed index’s data as you’re choosing which fields to transform. However, you can also use the REST API to create transform jobs and preview transform job results, but you must know all of the necessary settings and parameters to submit them as part of the HTTP request body. Submitting your transform job configurations as JSON scripts offers you more portability, allowing you to share and replicate your transform jobs, which is harder to do using OpenSearch Dashboards.
|
||||
|
||||
Your use cases will help you decide which method to use to create transform jobs.
|
||||
|
||||
## Create a transform job
|
||||
|
||||
If you don't have any data in your cluster, you can use the sample flight data within OpenSearch Dashboards to try out transform jobs. Otherwise, after launching OpenSearch Dashboards, choose **Index Management**. Select **Transform Jobs**, and choose **Create Transform Job**.
|
||||
|
||||
### Step 1: Choose indices
|
||||
|
||||
1. In the **Job name and description** section, specify a name and an optional description for your job.
|
||||
2. In the **Indices** section, select the source and target index. You can either select an existing target index or create a new one by entering a name for your new index. If you want to transform just a subset of your source index, choose **Add Data Filter**, and use the OpenSearch query DSL to specify a subset of your source index. For more information about the OpenSearch query DSL, see [query DSL]({{site.url}}{{site.baseurl}}/opensearch/query-dsl/).
|
||||
3. Choose **Next**.
|
||||
|
||||
### Step 2: Select fields to transform
|
||||
|
||||
After specifying the indices, you can select the fields you want to use in your transform job, as well as whether to use groupings or aggregations.
|
||||
|
||||
You can use groupings to place your data into separate buckets in your transformed index. For example, if you want to group all of the airport destinations within the sample flight data, you can group the `DestAirportID` field into a target field of `DestAirportID_terms` field, and you can find the grouped airport IDs in your transformed index after the transform job finishes.
|
||||
|
||||
On the other hand, aggregations let you perform simple calculations. For example, you can include an aggregation in your transform job to define a new field of `sum_of_total_ticket_price` that calculates the sum of all airplane tickets, and then analyze the newly summer data within your transformed index.
|
||||
|
||||
1. In the data table, select the fields you want to transform and expand the drop-down menu within the column header to choose the grouping or aggregation you want to use.
|
||||
|
||||
Currently, transform jobs support histogram, date_histogram, and terms groupings. For more information about groupings, see [Bucket Aggregations]({{site.url}}{{site.baseurl}}/opensearch/bucket-agg/). In terms of aggregations, you can select from `sum`, `avg`, `max`, `min`, `value_count`, `percentiles`, and `scripted_metric`. For more information about aggregations, see [Metric Aggregations]({{site.url}}{{site.baseurl}}/opensearch/metric-agg/).
|
||||
|
||||
2. Repeat step 1 for any other fields that you want to transform.
|
||||
3. After selecting the fields that you want to transform and verifying the transformation, choose **Next**.
|
||||
|
||||
### Step 3: Specify a schedule
|
||||
|
||||
You can configure transform jobs to run once or multiple times on a schedule. Transform jobs are enabled by default.
|
||||
|
||||
1. For **transformation execution frequency**, select **Define by fixed interval** and specify a **transform interval**.
|
||||
2. Under **Advanced**, specify an optional amount for **Pages per execution**. A larger number means more data is processed in each search request, but also uses more memory and causes higher latency. Exceeding allowed memory limits can cause exceptions and errors to occur.
|
||||
3. Choose **Next**.
|
||||
|
||||
### Step 4: Review and confirm details
|
||||
|
||||
After confirming your transform job’s details are correct, choose **Create Transform Job**. If you want to edit any part of the job, choose **Edit** of the section you want to change, and make the necessary changes. You can’t change aggregations or groupings after creating a job.
|
||||
|
||||
### Step 5: Search through the transformed index.
|
||||
|
||||
Once the transform job finishes, you can use the `_search` API operation to search the target index.
|
||||
|
||||
```json
|
||||
GET <target_index>/_search
|
||||
```
|
||||
|
||||
For example, after running a transform job that transforms the flight data based on a `DestAirportID` field, you can run the following request that returns all of the fields that have a value of `SFO`.
|
||||
|
||||
**Sample Request**
|
||||
|
||||
```json
|
||||
GET finished_flight_job/_search
|
||||
{
|
||||
"query": {
|
||||
"match": {
|
||||
"DestAirportID_terms" : "SFO"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Sample Response**
|
||||
|
||||
```json
|
||||
{
|
||||
"took" : 3,
|
||||
"timed_out" : false,
|
||||
"_shards" : {
|
||||
"total" : 5,
|
||||
"successful" : 5,
|
||||
"skipped" : 0,
|
||||
"failed" : 0
|
||||
},
|
||||
"hits" : {
|
||||
"total" : {
|
||||
"value" : 4,
|
||||
"relation" : "eq"
|
||||
},
|
||||
"max_score" : 3.845883,
|
||||
"hits" : [
|
||||
{
|
||||
"_index" : "finished_flight_job",
|
||||
"_type" : "_doc",
|
||||
"_id" : "dSNKGb8U3OJOmC4RqVCi1Q",
|
||||
"_score" : 3.845883,
|
||||
"_source" : {
|
||||
"transform._id" : "sample_flight_job",
|
||||
"transform._doc_count" : 14,
|
||||
"Carrier_terms" : "Dashboards Airlines",
|
||||
"DestAirportID_terms" : "SFO"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_index" : "finished_flight_job",
|
||||
"_type" : "_doc",
|
||||
"_id" : "_D7oqOy7drx9E-MG96U5RA",
|
||||
"_score" : 3.845883,
|
||||
"_source" : {
|
||||
"transform._id" : "sample_flight_job",
|
||||
"transform._doc_count" : 14,
|
||||
"Carrier_terms" : "Logstash Airways",
|
||||
"DestAirportID_terms" : "SFO"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_index" : "finished_flight_job",
|
||||
"_type" : "_doc",
|
||||
"_id" : "YuZ8tOt1OsBA54e84WuAEw",
|
||||
"_score" : 3.6988301,
|
||||
"_source" : {
|
||||
"transform._id" : "sample_flight_job",
|
||||
"transform._doc_count" : 11,
|
||||
"Carrier_terms" : "ES-Air",
|
||||
"DestAirportID_terms" : "SFO"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_index" : "finished_flight_job",
|
||||
"_type" : "_doc",
|
||||
"_id" : "W_-e7bVmH6eu8veJeK8ZxQ",
|
||||
"_score" : 3.6988301,
|
||||
"_source" : {
|
||||
"transform._id" : "sample_flight_job",
|
||||
"transform._doc_count" : 10,
|
||||
"Carrier_terms" : "JetBeats",
|
||||
"DestAirportID_terms" : "SFO"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
|
||||
```
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,14 +0,0 @@
|
||||
---
|
||||
layout: default
|
||||
title: About Index Management
|
||||
nav_order: 1
|
||||
has_children: false
|
||||
redirect_from:
|
||||
- /im-plugin/
|
||||
---
|
||||
|
||||
# About Index Management
|
||||
OpenSearch Dashboards
|
||||
{: .label .label-yellow :}
|
||||
|
||||
The Index Management (IM) plugin lets you automate recurring index management activities and reduce storage costs.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,104 +0,0 @@
|
||||
---
|
||||
layout: default
|
||||
title: Index State Management
|
||||
nav_order: 3
|
||||
has_children: true
|
||||
redirect_from:
|
||||
- /docs/im/ism/
|
||||
- /docs/ism/
|
||||
has_toc: false
|
||||
---
|
||||
|
||||
# Index State Management
|
||||
OpenSearch Dashboards
|
||||
{: .label .label-yellow :}
|
||||
|
||||
If you analyze time-series data, you likely prioritize new data over old data. You might periodically perform certain operations on older indices, such as reducing replica count or deleting them.
|
||||
|
||||
Index State Management (ISM) is a plugin that lets you automate these periodic, administrative operations by triggering them based on changes in the index age, index size, or number of documents. Using the ISM plugin, you can define *policies* that automatically handle index rollovers or deletions to fit your use case.
|
||||
|
||||
For example, you can define a policy that moves your index into a `read_only` state after 30 days and then deletes it after a set period of 90 days. You can also set up the policy to send you a notification message when the index is deleted.
|
||||
|
||||
You might want to perform an index rollover after a certain amount of time or run a `force_merge` operation on an index during off-peak hours to improve search performance during peak hours.
|
||||
|
||||
To use the ISM plugin, your user role needs to be mapped to the `all_access` role that gives you full access to the cluster. To learn more, see [Users and roles]({{site.url}}{{site.baseurl}}/security-plugin/access-control/users-roles/).
|
||||
{: .note }
|
||||
|
||||
## Get started with ISM
|
||||
|
||||
To get started, choose **Index Management** in OpenSearch Dashboards.
|
||||
|
||||
### Step 1: Set up policies
|
||||
|
||||
A policy is a set of rules that describes how an index should be managed. For information about creating a policy, see [Policies]({{site.url}}{{site.baseurl}}/im-plugin/ism/policies/).
|
||||
|
||||
1. Choose the **Index Policies** tab.
|
||||
2. Choose **Create policy**.
|
||||
3. In the **Name policy** section, enter a policy ID.
|
||||
4. In the **Define policy** section, enter your policy.
|
||||
5. Choose **Create**.
|
||||
|
||||
After you create a policy, your next step is to attach this policy to an index or indices.
|
||||
You can set up an `ism_template` in the policy so when you create an index that matches the ISM template pattern, the index will have this policy attached to it:
|
||||
|
||||
```json
|
||||
PUT _plugins/_ism/policies/policy_id
|
||||
{
|
||||
"policy": {
|
||||
"description": "Example policy.",
|
||||
"default_state": "...",
|
||||
"states": [...],
|
||||
"ism_template": {
|
||||
"index_patterns": ["index_name-*"],
|
||||
"priority": 100
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
For an example ISM template policy, see [Sample policy with ISM template]({{site.url}}{{site.baseurl}}/im-plugin/ism/policies#sample-policy-with-ism-template).
|
||||
|
||||
Older versions of the plugin include the `policy_id` in an index template, so when an index is created that matches the index template pattern, the index will have the policy attached to it:
|
||||
|
||||
```json
|
||||
PUT _index_template/<template_name>
|
||||
{
|
||||
"index_patterns": [
|
||||
"index_name-*"
|
||||
],
|
||||
"template": {
|
||||
"settings": {
|
||||
"opendistro.index_state_management.policy_id": "policy_id"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The `opendistro.index_state_management.policy_id` setting is deprecated. You can continue to automatically manage newly created indices with the ISM template field.
|
||||
{: .note }
|
||||
|
||||
### Step 2: Attach policies to indices
|
||||
|
||||
1. Choose **Indices**.
|
||||
2. Choose the index or indices that you want to attach your policy to.
|
||||
3. Choose **Apply policy**.
|
||||
4. From the **Policy ID** menu, choose the policy that you created.
|
||||
You can see a preview of your policy.
|
||||
5. If your policy includes a rollover operation, specify a rollover alias.
|
||||
Make sure that the alias that you enter already exists. For more information about the rollover operation, see [rollover]({{site.url}}{{site.baseurl}}/im-plugin/ism/policies#rollover).
|
||||
6. Choose **Apply**.
|
||||
|
||||
After you attach a policy to an index, ISM creates a job that runs every 5 minutes by default to perform policy actions, check conditions, and transition the index into different states. To change the default time interval for this job, see [Settings]({{site.url}}{{site.baseurl}}/im-plugin/ism/settings/).
|
||||
|
||||
If you want to use an OpenSearch operation to create an index with a policy already attached to it, see [create index]({{site.url}}{{site.baseurl}}/im-plugin/ism/api#create-index).
|
||||
|
||||
### Step 3: Manage indices
|
||||
|
||||
1. Choose **Managed Indices**.
|
||||
2. To change your policy, see [Change Policy]({{site.url}}{{site.baseurl}}/im-plugin/ism/managedindices#change-policy).
|
||||
3. To attach a rollover alias to your index, select your policy and choose **Add rollover alias**.
|
||||
Make sure that the alias that you enter already exists. For more information about the rollover operation, see [rollover]({{site.url}}{{site.baseurl}}/im-plugin/ism/policies#rollover).
|
||||
4. To remove a policy, choose your policy, and then choose **Remove policy**.
|
||||
5. To retry a policy, choose your policy, and then choose **Retry policy**.
|
||||
|
||||
For information about managing your policies, see [Managed Indices]({{site.url}}{{site.baseurl}}/im-plugin/ism/managedindices/).
|
||||
@@ -1,74 +0,0 @@
|
||||
---
|
||||
layout: default
|
||||
title: Managed Indices
|
||||
nav_order: 3
|
||||
parent: Index State Management
|
||||
redirect_from: /docs/ism/managedindices/
|
||||
has_children: false
|
||||
---
|
||||
|
||||
# Managed indices
|
||||
|
||||
You can change or update a policy using the managed index operations.
|
||||
|
||||
This table lists the fields of managed index operations.
|
||||
|
||||
Parameter | Description | Type | Required | Read Only
|
||||
:--- | :--- |:--- |:--- |
|
||||
`name` | The name of the managed index policy. | `string` | Yes | No
|
||||
`index` | The name of the managed index that this policy is managing. | `string` | Yes | No
|
||||
`index_uuid` | The uuid of the index. | `string` | Yes | No
|
||||
`enabled` | When `true`, the managed index is scheduled and run by the scheduler. | `boolean` | Yes | No
|
||||
`enabled_time` | The time the managed index was last enabled. If the managed index process is disabled, then this is null. | `timestamp` | Yes | Yes
|
||||
`last_updated_time` | The time the managed index was last updated. | `timestamp` | Yes | Yes
|
||||
`schedule` | The schedule of the managed index job. | `object` | Yes | No
|
||||
`policy_id` | The name of the policy used by this managed index. | `string` | Yes | No
|
||||
`policy_seq_no` | The sequence number of the policy used by this managed index. | `number` | Yes | No
|
||||
`policy_primary_term` | The primary term of the policy used by this managed index. | `number` | Yes | No
|
||||
`policy_version` | The version of the policy used by this managed index. | `number` | Yes | Yes
|
||||
`policy` | The cached JSON of the policy for the `policy_version` that's used during runs. If the policy is null, it means that this is the first execution of the job and the latest policy document is read in/saved. | `object` | No | No
|
||||
`change_policy` | The information regarding what policy and state to change to. | `object` | No | No
|
||||
`policy_name` | The name of the policy to update to. To update to the latest version, set this to be the same as the current `policy_name`. | `string` | No | Yes
|
||||
`state` | The state of the managed index after it finishes updating. If no state is specified, it's assumed that the policy structure did not change. | `string` | No | Yes
|
||||
|
||||
The following example shows a managed index policy:
|
||||
|
||||
```json
|
||||
{
|
||||
"managed_index": {
|
||||
"name": "my_index",
|
||||
"index": "my_index",
|
||||
"index_uuid": "sOKSOfkdsoSKeofjIS",
|
||||
"enabled": true,
|
||||
"enabled_time": 1553112384,
|
||||
"last_updated_time": 1553112384,
|
||||
"schedule": {
|
||||
"interval": {
|
||||
"period": 1,
|
||||
"unit": "MINUTES",
|
||||
"start_time": 1553112384
|
||||
}
|
||||
},
|
||||
"policy_id": "log_rotation",
|
||||
"policy_version": 1,
|
||||
"policy": {...},
|
||||
"change_policy": null
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Change policy
|
||||
|
||||
You can change any managed index policy, but ISM has a few constraints in place to make sure that policy changes don't break indices.
|
||||
|
||||
If an index is stuck in its current state, never proceeding, and you want to update its policy immediately, make sure that the new policy includes the same state---same name, same actions, same order---as the old policy. In this case, even if the policy is in the middle of executing an action, ISM applies the new policy.
|
||||
|
||||
If you update the policy without including an identical state, ISM updates the policy only after all actions in the current state finish executing. Alternately, you can choose a specific state in your old policy after which you want the new policy to take effect.
|
||||
|
||||
To change a policy using OpenSearch Dashboards, do the following:
|
||||
|
||||
- Under **Managed indices**, choose the indices that you want to attach the new policy to.
|
||||
- To attach the new policy to indices in specific states, choose **Choose state filters**, and then choose those states.
|
||||
- Under **Choose New Policy**, choose the new policy.
|
||||
- To start the new policy for indices in the current state, choose **Keep indices in their current state after the policy takes effect**.
|
||||
- To start the new policy in a specific state, choose **Start from a chosen state after changing policies**, and then choose the default start state in your new policy.
|
||||
File diff suppressed because it is too large
Load Diff
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user