CarolineandNaarcha-AWS
0249991f76
* Creating PR with first file. Signed-off-by: carolxob <[email protected]> * Adding newly created files to PR. Signed-off-by: carolxob <[email protected]> * Reorganized files and added appropriate metadata to map ToC correctly. Signed-off-by: carolxob <[email protected]> * Moved Authoring pipelines page. Signed-off-by: carolxob <[email protected]> * Minor ToC updates. Signed-off-by: carolxob <[email protected]> * Minor ToC updates to Sources section for Data Prepper. Signed-off-by: carolxob <[email protected]> * Updated Buffers section under Data Prepper. Signed-off-by: carolxob <[email protected]> * Minor update to otelmetricssource. Signed-off-by: carolxob <[email protected]> * Restructured ToC in Processors section for Data Prepper. Signed-off-by: carolxob <[email protected]> * Minor filename change. Signed-off-by: carolxob <[email protected]> * Adjustments to metadata in ToC. Signed-off-by: carolxob <[email protected]> * Minor edit. Signed-off-by: carolxob <[email protected]> * Fixed nav order in metadata. Signed-off-by: carolxob <[email protected]> * Minor edit. Signed-off-by: carolxob <[email protected]> * Minor update top metadata for ToC. Signed-off-by: carolxob <[email protected]> * Adjustmenets to Toc order. Signed-off-by: carolxob <[email protected]> * Minor adjustments to ToC metadata. Signed-off-by: carolxob <[email protected]> * Minor adjustments to Sinks section. Signed-off-by: carolxob <[email protected]> * Adjustements to high level ToC. Signed-off-by: carolxob <[email protected]> * Minor adjustement to Pipelines.md Signed-off-by: carolxob <[email protected]> * Minor update. Signed-off-by: carolxob <[email protected]> * Slight reorganization. Removed two placeholder pages for now. Signed-off-by: carolxob <[email protected]> * Removed a page and replaced with pipelines content. Signed-off-by: carolxob <[email protected]> * Minor changes/additions to content for placeholder pages. Signed-off-by: carolxob <[email protected]> * Minor update to page link. Signed-off-by: carolxob <[email protected]> * Minor adjustments to ToC metadata. Signed-off-by: carolxob <[email protected]> * Minor edits. Signed-off-by: carolxob <[email protected]> * Removed /clients from redirects to correct nav order. Signed-off-by: carolxob <[email protected]> * Minor edits. Signed-off-by: carolxob <[email protected]> * Minor adjustments to ToC metadata. Signed-off-by: carolxob <[email protected]> * Minor adjustments. Signed-off-by: carolxob <[email protected]> * Minor adjustment ot metadata. Signed-off-by: carolxob <[email protected]> * TOC link fixes Signed-off-by: Naarcha-AWS <[email protected]> * Changed page name. Signed-off-by: carolxob <[email protected]> * Corrected references to Peer Forwarder. Signed-off-by: carolxob <[email protected]> * Renamed Data Prepper folder. Signed-off-by: carolxob <[email protected]> * Minor updates to phrasing and capitalization. Signed-off-by: carolxob <[email protected]> * Minor phrasing update. Signed-off-by: carolxob <[email protected]> * Minor phrasing update. Signed-off-by: carolxob <[email protected]> * Minor change. Signed-off-by: carolxob <[email protected]> * Minor change to change S3 Source to S3Source. Signed-off-by: carolxob <[email protected]> * Updated references to peer forwarder and changed capitalization. Signed-off-by: carolxob <[email protected]> * Updated capitalization for peer forwarder. Signed-off-by: carolxob <[email protected]> * Made edits based on doc review feedback. Signed-off-by: carolxob <[email protected]> * Update to one word. Signed-off-by: carolxob <[email protected]> --------- Signed-off-by: carolxob <[email protected]> Signed-off-by: Naarcha-AWS <[email protected]> Co-authored-by: Naarcha-AWS <[email protected]>
3.1 KiB
3.1 KiB
layout, title, parent, grand_parent, nav_order
| layout | title | parent | grand_parent | nav_order |
|---|---|---|---|---|
| default | http_source | Sources | Pipelines | 5 |
http_source
This is a source plugin that supports HTTP protocol. Currently ONLY support Json UTF-8 codec for incoming request, e.g. [{"key1": "value1"}, {"key2": "value2"}].
| Option | Required | Type | Description |
|---|---|---|---|
| port | No | Integer | The port the source is running on. Default is 2021. Valid options are between 0 and 65535. |
| health_check_service | No | Boolean | Enables health check service on /health endpoint on the defined port. Default is false. |
| unauthenticated_health_check | No | Boolean | Determines whether or not authentication is required on the health check endpoint. Data Prepper ignores this option if no authentication is defined. Default is false. |
| request_timeout | No | Integer | The request timeout in millis. Default is 10_000. |
| thread_count | No | Integer | The number of threads to keep in the ScheduledThreadPool. Default is 200. |
| max_connection_count | No | Integer | The maximum allowed number of open connections. Default is 500. |
| max_pending_requests | No | Integer | The maximum number of allowed tasks in ScheduledThreadPool work queue. Default is 1024. |
| authentication | No | Object | An authentication configuration. By default, this creates an unauthenticated server for the pipeline. This uses pluggable authentication for HTTPS. To use basic authentication define the http_basic plugin with a username and password. To provide customer authentication, use or create a plugin that implements ArmeriaHttpAuthenticationProvider. |
| ssl | No | Boolean | Enables TLS/SSL. Default is false. |
| ssl_certificate_file | Conditionally | String | SSL certificate chain file path or AWS S3 path. S3 path example s3://<bucketName>/<path>. Required if ssl is true and use_acm_certificate_for_ssl is false. |
| ssl_key_file | Conditionally | String | SSL key file path or AWS S3 path. S3 path example s3://<bucketName>/<path>. Required if ssl is true and use_acm_certificate_for_ssl is false. |
| use_acm_certificate_for_ssl | No | Boolean | Enables TLS/SSL using certificate and private key from AWS Certificate Manager (ACM). Default is false. |
| acm_certificate_arn | Conditionally | String | ACM certificate ARN. The ACM certificate takes preference over S3 or a local file system certificate. Required if use_acm_certificate_for_ssl is set to true. |
| acm_private_key_password | No | String | ACM private key password that decrypts the private key. If not provided, Data Prepper generates a random password. |
| acm_certificate_timeout_millis | No | Integer | Timeout in milliseconds for ACM to get certificates. Default is 120000. |
| aws_region | Conditionally | String | AWS region to use ACM or S3. Required if use_acm_certificate_for_ssl is set to true or ssl_certificate_file and ssl_key_file is AWS S3 path. |