From 49e596c20ba3d19e6521f6520333523c1b82d594 Mon Sep 17 00:00:00 2001 From: keithhc2 Date: Wed, 5 Jan 2022 11:52:31 -0800 Subject: [PATCH 1/7] Added required parameters Signed-off-by: keithhc2 --- .../index-transforms/transforms-apis.md | 31 +++++++++++++------ 1 file changed, 21 insertions(+), 10 deletions(-) diff --git a/_im-plugin/index-transforms/transforms-apis.md b/_im-plugin/index-transforms/transforms-apis.md index 67acd298..27fdf0bb 100644 --- a/_im-plugin/index-transforms/transforms-apis.md +++ b/_im-plugin/index-transforms/transforms-apis.md @@ -128,17 +128,17 @@ You can specify the following options in the HTTP request body: Option | Data Type | Description | Required :--- | :--- | :--- | :--- enabled | Boolean | If true, the transform job is enabled at creation. | No -schedule | JSON | The schedule the transform job runs on. | Yes +schedule | Object | The schedule the transform job runs on. | Yes start_time | Integer | The Unix epoch time of the transform job's start time. | Yes description | String | Describes the transform job. | No metadata_id | String | Any metadata to be associated with the transform job. | No source_index | String | The source index whose data to transform. | Yes target_index | String | The target index the newly transformed data is added into. You can create a new index or update an existing one. | Yes -data_selection_query | JSON | The query DSL to use to filter a subset of the source index for the transform job. See [query DSL]({{site.url}}{{site.baseurl}}/opensearch/query-dsl) for more information. | Yes -page_size | Integer | The number of fields to transform at a time. Higher number means higher performance but requires more memory and can cause higher latency. (Default: 1) | Yes +data_selection_query | Object | The query DSL to use to filter a subset of the source index for the transform job. See [query DSL]({{site.url}}{{site.baseurl}}/opensearch/query-dsl) for more information. | Yes +page_size | Integer | The number of fields to transform at a time. Higher number means higher performance but requires more memory and can cause higher latency. Default is 1. | Yes groups | Array | Specifies the grouping(s) to use in the transform job. Supported groups are `terms`, `histogram`, and `date_histogram`. For more information, see [Bucket Aggregations]({{site.url}}{{site.baseurl}}/opensearch/bucket-agg). | Yes if not using aggregations source_field | String | The field(s) to transform | Yes -aggregations | JSON | The aggregations to use in the transform job. Supported aggregations are: `sum`, `max`, `min`, `value_count`, `avg`, `scripted_metric`, and `percentiles`. For more information, see [Metric Aggregations]({{site.url}}{{site.baseurl}}/opensearch/metric-agg). | Yes if not using groups +aggregations | Object | The aggregations to use in the transform job. Supported aggregations are: `sum`, `max`, `min`, `value_count`, `avg`, `scripted_metric`, and `percentiles`. For more information, see [Metric Aggregations]({{site.url}}{{site.baseurl}}/opensearch/metric-agg). | Yes if not using groups ## Update a transform job Introduced 1.0 @@ -250,12 +250,23 @@ PUT _plugins/_transform/ } ``` -The `Update` operation supports the following URL parameters: +The Update operation supports the following URL parameters: Parameter | Description | Required :---| :--- | :--- -`if_seq_no` | Only perform the transform operation if the last operation that changed the transform job has the specified sequence number. | No -`if_primary_term` | Only perform the transform operation if the last operation that changed the transform job has the specified sequence term. | No +`if_seq_no` | Only perform the transform operation if the last operation that changed the transform job has the specified sequence number. | Yes +`if_primary_term` | Only perform the transform operation if the last operation that changed the transform job has the specified sequence term. | Yes + +You can update the following fields: + +Option | Data Type | Description +:--- | :--- | :--- +schedule | Object | The schedule the transform job runs on. Contains the fields `interval.start_time`, `interval.period`, and `interval.unit`. +start_time | Integer | The Unix epoch start time of the transform job. +period | Integer | How often to execute the transform job. +unit | String | The unit of time associated with the execution period. Available options are `Minutes`, `Hours`, and `Days`. +description | Integer | Describes the transform job. +page_size | Integer | The number of fields to transform at a time. Higher number means higher performance but requires more memory and can cause higher latency. Default is 1. ## Get a transform job's details Introduced 1.0 @@ -399,11 +410,11 @@ You can specify these options as the `GET` API operation’s URL parameters to f Parameter | Description | Required :--- | :--- | :--- -from | The starting index to search from. (Default: 0) | No -size | Specifies the amount of results to return (Default: 10) | No +from | The starting index to search from. Default is 0. | No +size | Specifies the amount of results to return. Default is 1. | No search |The search term to use to filter results. | No sortField | The field to sort results with. | No -sortDirection | Specifies the direction to sort results in. Can be `ASC` or `DESC`. (Default: ASC) | No +sortDirection | Specifies the direction to sort results in. Can be `ASC` or `DESC`. Default is ASC. | No For example, this request returns two results starting from the eighth index. From 155cbaaf5ee8d3a6b81aa35b33aa5fa3962058cd Mon Sep 17 00:00:00 2001 From: keithhc2 Date: Mon, 31 Jan 2022 11:32:22 -0800 Subject: [PATCH 2/7] Fixed page size description Signed-off-by: keithhc2 --- _im-plugin/index-transforms/transforms-apis.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_im-plugin/index-transforms/transforms-apis.md b/_im-plugin/index-transforms/transforms-apis.md index 27fdf0bb..d04d6073 100644 --- a/_im-plugin/index-transforms/transforms-apis.md +++ b/_im-plugin/index-transforms/transforms-apis.md @@ -135,7 +135,7 @@ metadata_id | String | Any metadata to be associated with the transform job. | N source_index | String | The source index whose data to transform. | Yes target_index | String | The target index the newly transformed data is added into. You can create a new index or update an existing one. | Yes data_selection_query | Object | The query DSL to use to filter a subset of the source index for the transform job. See [query DSL]({{site.url}}{{site.baseurl}}/opensearch/query-dsl) for more information. | Yes -page_size | Integer | The number of fields to transform at a time. Higher number means higher performance but requires more memory and can cause higher latency. Default is 1. | Yes +page_size | Integer | The maximum number of buckets OpenSearch can use to transform data. Higher number means higher performance but requires more memory and can cause higher latency. Default is 1. | Yes groups | Array | Specifies the grouping(s) to use in the transform job. Supported groups are `terms`, `histogram`, and `date_histogram`. For more information, see [Bucket Aggregations]({{site.url}}{{site.baseurl}}/opensearch/bucket-agg). | Yes if not using aggregations source_field | String | The field(s) to transform | Yes aggregations | Object | The aggregations to use in the transform job. Supported aggregations are: `sum`, `max`, `min`, `value_count`, `avg`, `scripted_metric`, and `percentiles`. For more information, see [Metric Aggregations]({{site.url}}{{site.baseurl}}/opensearch/metric-agg). | Yes if not using groups From 97ba383e8a04b96772ba24fca3d46ab728794aec Mon Sep 17 00:00:00 2001 From: keithhc2 Date: Mon, 31 Jan 2022 11:51:17 -0800 Subject: [PATCH 3/7] Took out default page_size Signed-off-by: keithhc2 --- _im-plugin/index-transforms/transforms-apis.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_im-plugin/index-transforms/transforms-apis.md b/_im-plugin/index-transforms/transforms-apis.md index d04d6073..3927096f 100644 --- a/_im-plugin/index-transforms/transforms-apis.md +++ b/_im-plugin/index-transforms/transforms-apis.md @@ -266,7 +266,7 @@ start_time | Integer | The Unix epoch start time of the transform job. period | Integer | How often to execute the transform job. unit | String | The unit of time associated with the execution period. Available options are `Minutes`, `Hours`, and `Days`. description | Integer | Describes the transform job. -page_size | Integer | The number of fields to transform at a time. Higher number means higher performance but requires more memory and can cause higher latency. Default is 1. +page_size | Integer | The number of fields to transform at a time. Higher number means higher performance but requires more memory and can cause higher latency. ## Get a transform job's details Introduced 1.0 From 6f436a44d5c945f6d33e277d45891b52c031afb9 Mon Sep 17 00:00:00 2001 From: keithhc2 Date: Mon, 31 Jan 2022 14:47:28 -0800 Subject: [PATCH 4/7] Missed a page_size default Signed-off-by: keithhc2 --- _im-plugin/index-transforms/transforms-apis.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_im-plugin/index-transforms/transforms-apis.md b/_im-plugin/index-transforms/transforms-apis.md index 3927096f..1d08b5de 100644 --- a/_im-plugin/index-transforms/transforms-apis.md +++ b/_im-plugin/index-transforms/transforms-apis.md @@ -135,7 +135,7 @@ metadata_id | String | Any metadata to be associated with the transform job. | N source_index | String | The source index whose data to transform. | Yes target_index | String | The target index the newly transformed data is added into. You can create a new index or update an existing one. | Yes data_selection_query | Object | The query DSL to use to filter a subset of the source index for the transform job. See [query DSL]({{site.url}}{{site.baseurl}}/opensearch/query-dsl) for more information. | Yes -page_size | Integer | The maximum number of buckets OpenSearch can use to transform data. Higher number means higher performance but requires more memory and can cause higher latency. Default is 1. | Yes +page_size | Integer | The maximum number of buckets OpenSearch can use to transform data. Higher number means higher performance but requires more memory and can cause higher latency. | Yes groups | Array | Specifies the grouping(s) to use in the transform job. Supported groups are `terms`, `histogram`, and `date_histogram`. For more information, see [Bucket Aggregations]({{site.url}}{{site.baseurl}}/opensearch/bucket-agg). | Yes if not using aggregations source_field | String | The field(s) to transform | Yes aggregations | Object | The aggregations to use in the transform job. Supported aggregations are: `sum`, `max`, `min`, `value_count`, `avg`, `scripted_metric`, and `percentiles`. For more information, see [Metric Aggregations]({{site.url}}{{site.baseurl}}/opensearch/metric-agg). | Yes if not using groups From 639d0df9b36a04b60098949fb6d2f1e6078dc418 Mon Sep 17 00:00:00 2001 From: keithhc2 Date: Wed, 2 Feb 2022 12:41:36 -0800 Subject: [PATCH 5/7] Language tweaks Signed-off-by: keithhc2 --- _im-plugin/index-transforms/transforms-apis.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/_im-plugin/index-transforms/transforms-apis.md b/_im-plugin/index-transforms/transforms-apis.md index 1d08b5de..c260b2b6 100644 --- a/_im-plugin/index-transforms/transforms-apis.md +++ b/_im-plugin/index-transforms/transforms-apis.md @@ -135,7 +135,7 @@ metadata_id | String | Any metadata to be associated with the transform job. | N source_index | String | The source index whose data to transform. | Yes target_index | String | The target index the newly transformed data is added into. You can create a new index or update an existing one. | Yes data_selection_query | Object | The query DSL to use to filter a subset of the source index for the transform job. See [query DSL]({{site.url}}{{site.baseurl}}/opensearch/query-dsl) for more information. | Yes -page_size | Integer | The maximum number of buckets OpenSearch can use to transform data. Higher number means higher performance but requires more memory and can cause higher latency. | Yes +page_size | Integer | The number of buckets ISM processes and indexes concurrently. Higher number means better performance but requires more memory. If your machine runs out of memory, ISM automatically adjusts this field and retries until the operation succeeds. | Yes groups | Array | Specifies the grouping(s) to use in the transform job. Supported groups are `terms`, `histogram`, and `date_histogram`. For more information, see [Bucket Aggregations]({{site.url}}{{site.baseurl}}/opensearch/bucket-agg). | Yes if not using aggregations source_field | String | The field(s) to transform | Yes aggregations | Object | The aggregations to use in the transform job. Supported aggregations are: `sum`, `max`, `min`, `value_count`, `avg`, `scripted_metric`, and `percentiles`. For more information, see [Metric Aggregations]({{site.url}}{{site.baseurl}}/opensearch/metric-agg). | Yes if not using groups @@ -266,7 +266,7 @@ start_time | Integer | The Unix epoch start time of the transform job. period | Integer | How often to execute the transform job. unit | String | The unit of time associated with the execution period. Available options are `Minutes`, `Hours`, and `Days`. description | Integer | Describes the transform job. -page_size | Integer | The number of fields to transform at a time. Higher number means higher performance but requires more memory and can cause higher latency. +page_size | Integer | The number of buckets ISM processes and indexes concurrently. Higher number means better performance but requires more memory. If your machine runs out of memory, ISM automatically adjusts this field and retries until the operation succeeds. ## Get a transform job's details Introduced 1.0 From 02d56f73b055b38ebd44d5d7def54cd8b2c256e1 Mon Sep 17 00:00:00 2001 From: keithhc2 Date: Fri, 4 Feb 2022 15:47:57 -0800 Subject: [PATCH 6/7] Missed a zero Signed-off-by: keithhc2 --- _im-plugin/index-transforms/transforms-apis.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_im-plugin/index-transforms/transforms-apis.md b/_im-plugin/index-transforms/transforms-apis.md index c260b2b6..ea5e349e 100644 --- a/_im-plugin/index-transforms/transforms-apis.md +++ b/_im-plugin/index-transforms/transforms-apis.md @@ -411,7 +411,7 @@ You can specify these options as the `GET` API operation’s URL parameters to f Parameter | Description | Required :--- | :--- | :--- from | The starting index to search from. Default is 0. | No -size | Specifies the amount of results to return. Default is 1. | No +size | Specifies the amount of results to return. Default is 10. | No search |The search term to use to filter results. | No sortField | The field to sort results with. | No sortDirection | Specifies the direction to sort results in. Can be `ASC` or `DESC`. Default is ASC. | No From f7ae12155db34a754281d9a8e7df3f2bbe6de522 Mon Sep 17 00:00:00 2001 From: keithhc2 Date: Tue, 8 Feb 2022 10:12:09 -0800 Subject: [PATCH 7/7] ISM -> IM Signed-off-by: keithhc2 --- _im-plugin/index-transforms/transforms-apis.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/_im-plugin/index-transforms/transforms-apis.md b/_im-plugin/index-transforms/transforms-apis.md index ea5e349e..7ddf6150 100644 --- a/_im-plugin/index-transforms/transforms-apis.md +++ b/_im-plugin/index-transforms/transforms-apis.md @@ -135,7 +135,7 @@ metadata_id | String | Any metadata to be associated with the transform job. | N source_index | String | The source index whose data to transform. | Yes target_index | String | The target index the newly transformed data is added into. You can create a new index or update an existing one. | Yes data_selection_query | Object | The query DSL to use to filter a subset of the source index for the transform job. See [query DSL]({{site.url}}{{site.baseurl}}/opensearch/query-dsl) for more information. | Yes -page_size | Integer | The number of buckets ISM processes and indexes concurrently. Higher number means better performance but requires more memory. If your machine runs out of memory, ISM automatically adjusts this field and retries until the operation succeeds. | Yes +page_size | Integer | The number of buckets IM processes and indexes concurrently. Higher number means better performance but requires more memory. If your machine runs out of memory, IM automatically adjusts this field and retries until the operation succeeds. | Yes groups | Array | Specifies the grouping(s) to use in the transform job. Supported groups are `terms`, `histogram`, and `date_histogram`. For more information, see [Bucket Aggregations]({{site.url}}{{site.baseurl}}/opensearch/bucket-agg). | Yes if not using aggregations source_field | String | The field(s) to transform | Yes aggregations | Object | The aggregations to use in the transform job. Supported aggregations are: `sum`, `max`, `min`, `value_count`, `avg`, `scripted_metric`, and `percentiles`. For more information, see [Metric Aggregations]({{site.url}}{{site.baseurl}}/opensearch/metric-agg). | Yes if not using groups @@ -266,7 +266,7 @@ start_time | Integer | The Unix epoch start time of the transform job. period | Integer | How often to execute the transform job. unit | String | The unit of time associated with the execution period. Available options are `Minutes`, `Hours`, and `Days`. description | Integer | Describes the transform job. -page_size | Integer | The number of buckets ISM processes and indexes concurrently. Higher number means better performance but requires more memory. If your machine runs out of memory, ISM automatically adjusts this field and retries until the operation succeeds. +page_size | Integer | The number of buckets IM processes and indexes concurrently. Higher number means better performance but requires more memory. If your machine runs out of memory, IM automatically adjusts this field and retries until the operation succeeds. ## Get a transform job's details Introduced 1.0