Caroline
ef83f6d7d5
* Added metrics section to Aggregate processor page. Signed-off-by: carolxob <[email protected]> * Added Metrics section to individual Processors pages. Signed-off-by: carolxob <[email protected]> * Added metrics section for JSON processor. Signed-off-by: carolxob <[email protected]> * Added metrics sections. Changed Default is to Default value is. Signed-off-by: carolxob <[email protected]> * Corrected references from AWS S3 to Amazon S3. Signed-off-by: carolxob <[email protected]> * Minor updates to Metrics sections and phrasing. Signed-off-by: carolxob <[email protected]> * Updated Action link. Signed-off-by: carolxob <[email protected]> * Updates based on tech review feedback. Signed-off-by: carolxob <[email protected]> * Updates based on tech review feedback. Signed-off-by: carolxob <[email protected]> * Tech review feedback. Signed-off-by: carolxob <[email protected]> * Minor updates to buffer_size and batch_size default values. Signed-off-by: carolxob <[email protected]> * Edits to Metrics sections for each processor. Signed-off-by: carolxob <[email protected]> * Update made based ondoc review feedback. Signed-off-by: carolxob <[email protected]> * Minor updates to intro text for processor pages. Minor adjustements to other text for clarity. Signed-off-by: carolxob <[email protected]> * Minor edits. Signed-off-by: carolxob <[email protected]> * Adjustements to phrasing, fixed typos. Signed-off-by: carolxob <[email protected]> * Minor updates to word choice and corrected a typo. Signed-off-by: carolxob <[email protected]> * Minor edit. Signed-off-by: carolxob <[email protected]> * Made updates based ondoc review feedback. Signed-off-by: carolxob <[email protected]> * Updates to http-source. Signed-off-by: carolxob <[email protected]> * Added common processors table to affected docs. Signed-off-by: carolxob <[email protected]> * Minor update to one file. Signed-off-by: carolxob <[email protected]> * Minor update based on tech review feedback. Signed-off-by: carolxob <[email protected]> * Minor edits. Signed-off-by: carolxob <[email protected]> * Major editorial feedback incorporated through key-value.md. Signed-off-by: carolxob <[email protected]> * Incorporated major editorial feedback thup to service-map-stateful. Signed-off-by: carolxob <[email protected]> * Incorporated major editorial feedback for Processors section. Signed-off-by: carolxob <[email protected]> * Major editorial updates, specifically to inclusion of text introducing option configuration tables. Signed-off-by: carolxob <[email protected]> * Major editorial feedback through otel-trace.md incorporated. Signed-off-by: carolxob <[email protected]> * Major editorial edits incorporated. Signed-off-by: carolxob <[email protected]> * Technical feedback and editorial feedback incorporated. Signed-off-by: carolxob <[email protected]> * Incorporated missing editorial feedback. Signed-off-by: carolxob <[email protected]> * Minor adjustements to OpenSearch sink. Signed-off-by: carolxob <[email protected]> * Minor changes to capitalization. Signed-off-by: carolxob <[email protected]> * Minor edits. Signed-off-by: carolxob <[email protected]> * Made one instance of processor name consistent with other references. Signed-off-by: carolxob <[email protected]> * Minor update based on editorial feedback. Signed-off-by: carolxob <[email protected]> --------- Signed-off-by: carolxob <[email protected]>
3.8 KiB
3.8 KiB
layout, title, parent, grand_parent, nav_order
| layout | title | parent | grand_parent | nav_order |
|---|---|---|---|---|
| default | aggregate | Processors | Pipelines | 45 |
aggregate
Overview
The aggregate processor groups events based on the keys provided and performs an action on each group. The following table describes the options you can use to configure the aggregate processor.
| Option | Required | Type | Description |
|---|---|---|---|
| identification_keys | Yes | List | An unordered list by which to group events. Events with the same values as these keys are put into the same group. If an event does not contain one of the identification_keys, then the value of that key is considered to be equal to null. At least one identification_key is required (for example, ["sourceIp", "destinationIp", "port"]). |
| action | Yes | AggregateAction | The action to be performed for each group. One of the available aggregate actions must be provided or you can create custom aggregate actions. remove_duplicates and put_all are the available actions. For more information, see Creating New Aggregate Actions. |
| group_duration | No | String | The amount of time that a group should exist before it is concluded automatically. Supports ISO_8601 notation strings ("PT20.345S", "PT15M", etc.) as well as simple notation for seconds ("60s") and milliseconds ("1500ms"). Default value is 180s. |
Metrics
The following table describes common Abstract processor metrics.
| Metric name | Type | Description |
|---|---|---|
recordsIn |
Counter | Metric representing the ingress of records to a pipeline component. |
recordsOut |
Counter | Metric representing the egress of records from a pipeline component. |
timeElapsed |
Timer | Metric representing the time elapsed during execution of a pipeline component. |
The aggregate processor includes the following custom metrics.
Counter
actionHandleEventsOut: The number of events that have been returned from thehandleEventcall to the configured action.actionHandleEventsDropped: The number of events that have not been returned from thehandleEventcall to the configured action.actionHandleEventsProcessingErrors: The number of calls made tohandleEventfor the configured action that resulted in an error.actionConcludeGroupEventsOut: The number of events that have been returned from theconcludeGroupcall to the configured action.actionConcludeGroupEventsDropped: The number of events that have not been returned from thecondludeGroupcall to the configured action.actionConcludeGroupEventsProcessingErrors: The number of calls made toconcludeGroupfor the configured action that resulted in an error.
Gauge
currentAggregateGroups: The current number of groups. This gauge decreases when a group concludes and increases when an event initiates the creation of a new group.