* Make API reference top level Signed-off-by: Naarcha-AWS <[email protected]> * Fix typo on Drag and Drop page (#1633) * Fix typo on Drag and Drop page * Update _dashboards/drag-drop-wizard.md Co-authored-by: Nate Bower <[email protected]> * Update drag-drop-wizard.md Co-authored-by: Nate Bower <[email protected]> * Putting all the Docker install material on a single page (#1452) * Putting all the Docker install material on a single page Signed-off-by: JeffH-AWS <[email protected]> * Making room for revamp Signed-off-by: JeffH-AWS <[email protected]> * Intro added Signed-off-by: JeffH-AWS <[email protected]> * Continuing to flesh out the intro section and overview Signed-off-by: JeffH-AWS <[email protected]> * Overview finalized Signed-off-by: JeffH-AWS <[email protected]> * Introducing docker compose Signed-off-by: JeffH-AWS <[email protected]> * Added link to compose Signed-off-by: JeffH-AWS <[email protected]> * Continuing docker image commentary Signed-off-by: JeffH-AWS <[email protected]> * Sometimes I wonder if anyone reads these Signed-off-by: JeffH-AWS <[email protected]> * Adding notes on installing compose with pip Signed-off-by: JeffH-AWS <[email protected]> * Adding prereqs Signed-off-by: JeffH-AWS <[email protected]> * Magnets - how do they work? Signed-off-by: JeffH-AWS <[email protected]> * Almonds and peaches are part of the same plant subgenus, Amygdalus Signed-off-by: JeffH-AWS <[email protected]> * There are 293 ways to make change for a dollar Signed-off-by: JeffH-AWS <[email protected]> * A shark is the only known fish that can blink with both eyes Signed-off-by: JeffH-AWS <[email protected]> * A crocodile cannot stick its tongue out Signed-off-by: JeffH-AWS <[email protected]> * wording Signed-off-by: JeffH-AWS <[email protected]> * Reorganizing a couple paragraphs to make it flow better Signed-off-by: JeffH-AWS <[email protected]> * Forgot a word Signed-off-by: JeffH-AWS <[email protected]> * Add tip about pruning stopped containers Signed-off-by: JeffH-AWS <[email protected]> * Cleaning up Signed-off-by: JeffH-AWS <[email protected]> * Add blurb about container ls Signed-off-by: JeffH-AWS <[email protected]> * Adding the Docker Compose stuff Signed-off-by: JeffH-AWS <[email protected]> * Working on compose Signed-off-by: JeffH-AWS <[email protected]> * Continuing work on the compose section - it's a lot of info Signed-off-by: JeffH-AWS <[email protected]> * Added important settings Signed-off-by: JeffH-AWS <[email protected]> * Updates to settings that need configured Signed-off-by: JeffH-AWS <[email protected]> * Still working through compose things Signed-off-by: JeffH-AWS <[email protected]> * Fixed wording Signed-off-by: JeffH-AWS <[email protected]> * Working through compose commands and guidance Signed-off-by: JeffH-AWS <[email protected]> * Reordering/rewording Signed-off-by: JeffH-AWS <[email protected]> * More phrasing Signed-off-by: JeffH-AWS <[email protected]> * More wording in steps Signed-off-by: JeffH-AWS <[email protected]> * More wording in steps Signed-off-by: JeffH-AWS <[email protected]> * Organizing Signed-off-by: JeffH-AWS <[email protected]> * Adding stuff and things Signed-off-by: JeffH-AWS <[email protected]> * Continuing to work through the configuration steps Signed-off-by: JeffH-AWS <[email protected]> * Fixes Signed-off-by: JeffH-AWS <[email protected]> * Fixes Signed-off-by: JeffH-AWS <[email protected]> * Still working on the configuration steps Signed-off-by: JeffH-AWS <[email protected]> * Changes Signed-off-by: JeffH-AWS <[email protected]> * More work Signed-off-by: JeffH-AWS <[email protected]> * Removed perf analyzer - refer to GH issue 1555 Signed-off-by: JeffH-AWS <[email protected]> * Fixing things Signed-off-by: JeffH-AWS <[email protected]> * Adding guidance on passing settings in compose Signed-off-by: JeffH-AWS <[email protected]> * Working through dockerfile materials now Signed-off-by: JeffH-AWS <[email protected]> * wording Signed-off-by: JeffH-AWS <[email protected]> * Finalized the sample dev compose file Signed-off-by: JeffH-AWS <[email protected]> * Continuing work with configuration Signed-off-by: JeffH-AWS <[email protected]> * Finished - ready for reviews Signed-off-by: JeffH-AWS <[email protected]> * Fixed a link I forgot to change before Signed-off-by: JeffH-AWS <[email protected]> * Changes from first proofread Signed-off-by: JeffH-AWS <[email protected]> * Changed heading Signed-off-by: JeffH-AWS <[email protected]> * Addressed reviewer comments and made some changes Signed-off-by: JeffH-AWS <[email protected]> * Forgot to incorporate one change. Fixed. Signed-off-by: JeffH-AWS <[email protected]> * Final editorial changes Signed-off-by: JeffH-AWS <[email protected]> Signed-off-by: JeffH-AWS <[email protected]> * fix#1584-custom_attr_allowlist (#1636) Signed-off-by: cwillum <[email protected]> Signed-off-by: cwillum <[email protected]> * Update TERMS.md with definition for Setting (#1632) * fix#1631-Terms-setting Signed-off-by: cwillum <[email protected]> * fix#1631-Terms-setting Signed-off-by: cwillum <[email protected]> Signed-off-by: cwillum <[email protected]> * Add disclaimer about remote fs usage and an example of setting env var (#1644) * Add disclaimer about remote fs usage and an example of setting env var Signed-off-by: JeffH-AWS <[email protected]> * Enhanced wording a little bit Signed-off-by: JeffH-AWS <[email protected]> Signed-off-by: JeffH-AWS <[email protected]> * [DOC] New documentation: Self-host maps server (#1625) * Add new page self-host maps server Signed-off-by: vagimeli <[email protected]> * Added new content Signed-off-by: vagimeli <[email protected]> * Copy edit Signed-off-by: vagimeli <[email protected]> * Tech review edits Signed-off-by: vagimeli <[email protected]> * Doc review edits Signed-off-by: vagimeli <[email protected]> * Editorial review changes Signed-off-by: vagimeli <[email protected]> * Final edits Signed-off-by: vagimeli <[email protected]> Signed-off-by: vagimeli <[email protected]> * Add feedback. Signed-off-by: Naarcha-AWS <[email protected]> * Fix links Signed-off-by: Naarcha-AWS <[email protected]> Signed-off-by: Naarcha-AWS <[email protected]> Signed-off-by: JeffH-AWS <[email protected]> Signed-off-by: cwillum <[email protected]> Signed-off-by: vagimeli <[email protected]> Co-authored-by: Nate Bower <[email protected]> Co-authored-by: Jeff Huss <[email protected]> Co-authored-by: Chris Moore <[email protected]> Co-authored-by: Melissa Vagi <[email protected]>
110 lines
3.1 KiB
Markdown
110 lines
3.1 KiB
Markdown
---
|
||
layout: default
|
||
title: Scroll
|
||
nav_order: 71
|
||
---
|
||
|
||
# Scroll
|
||
Introduced 1.0
|
||
{: .label .label-purple }
|
||
|
||
You can use the `scroll` operation to retrieve a large number of results. For example, for machine learning jobs, you can request an unlimited number of results in batches.
|
||
|
||
To use the `scroll` operation, add a `scroll` parameter to the request header with a search context to tell OpenSearch how long you need to keep scrolling. This search context needs to be long enough to process a single batch of results.
|
||
|
||
Because search contexts consume a lot of memory, we suggest you don't use the `scroll` operation for frequent user queries. Instead, use the `sort` parameter with the `search_after` parameter to scroll responses for user queries.
|
||
{: .note }
|
||
|
||
## Example
|
||
|
||
To set the number of results that you want returned for each batch, use the `size` parameter:
|
||
|
||
```json
|
||
GET shakespeare/_search?scroll=10m
|
||
{
|
||
"size": 10000
|
||
}
|
||
```
|
||
|
||
OpenSearch caches the results and returns a scroll ID to access them in batches:
|
||
|
||
```json
|
||
"_scroll_id" : "DXF1ZXJ5QW5kRmV0Y2gBAAAAAAAAAAUWdmpUZDhnRFBUcWFtV21nMmFwUGJEQQ=="
|
||
```
|
||
|
||
Pass this scroll ID to the `scroll` operation to get back the next batch of results:
|
||
|
||
```json
|
||
GET _search/scroll
|
||
{
|
||
"scroll": "10m",
|
||
"scroll_id": "DXF1ZXJ5QW5kRmV0Y2gBAAAAAAAAAAUWdmpUZDhnRFBUcWFtV21nMmFwUGJEQQ=="
|
||
}
|
||
```
|
||
|
||
Using this scroll ID, you get results in batches of 10,000 as long as the search context is still open. Typically, the scroll ID does not change between requests, but it *can* change, so make sure to always use the latest scroll ID. If you don't send the next scroll request within the set search context, the `scroll` operation does not return any results.
|
||
|
||
If you expect billions of results, use a sliced scroll. Slicing allows you to perform multiple scroll operations for the same request, but in parallel.
|
||
Set the ID and the maximum number of slices for the scroll:
|
||
|
||
```json
|
||
GET shakespeare/_search?scroll=10m
|
||
{
|
||
"slice": {
|
||
"id": 0,
|
||
"max": 10
|
||
},
|
||
"query": {
|
||
"match_all": {}
|
||
}
|
||
}
|
||
```
|
||
|
||
With a single scroll ID, you get back 10 results.
|
||
You can have up to 10 IDs.
|
||
|
||
Close the search context when you’re done scrolling, because the `scroll` operation continues to consume computing resources until the timeout:
|
||
|
||
```json
|
||
DELETE _search/scroll/DXF1ZXJ5QW5kRmV0Y2gBAAAAAAAAAAcWdmpUZDhnRFBUcWFtV21nMmFwUGJEQQ==
|
||
```
|
||
|
||
To close all open scroll contexts:
|
||
|
||
```json
|
||
DELETE _search/scroll/_all
|
||
```
|
||
|
||
The `scroll` operation corresponds to a specific timestamp. It doesn't consider documents added after that timestamp as potential results.
|
||
|
||
|
||
## Path and HTTP methods
|
||
|
||
```
|
||
GET _search/scroll
|
||
POST _search/scroll
|
||
```
|
||
```
|
||
GET _search/scroll/<scroll-id>
|
||
POST _search/scroll/<scroll-id>
|
||
```
|
||
|
||
## URL parameters
|
||
|
||
All scroll parameters are optional.
|
||
|
||
Parameter | Type | Description
|
||
:--- | :--- | :---
|
||
scroll | Time | Specifies the amount of time the search context is maintained.
|
||
scroll_id | String | The scroll ID for the search.
|
||
rest_total_hits_as_int | Boolean | Whether the `hits.total` property is returned as an integer (`true`) or an object (`false`). Default is `false`.
|
||
|
||
## Response
|
||
|
||
```json
|
||
{
|
||
"succeeded": true,
|
||
"num_freed": 1
|
||
}
|
||
```
|