CarolineandNaarcha-AWS
0249991f76
* Creating PR with first file. Signed-off-by: carolxob <[email protected]> * Adding newly created files to PR. Signed-off-by: carolxob <[email protected]> * Reorganized files and added appropriate metadata to map ToC correctly. Signed-off-by: carolxob <[email protected]> * Moved Authoring pipelines page. Signed-off-by: carolxob <[email protected]> * Minor ToC updates. Signed-off-by: carolxob <[email protected]> * Minor ToC updates to Sources section for Data Prepper. Signed-off-by: carolxob <[email protected]> * Updated Buffers section under Data Prepper. Signed-off-by: carolxob <[email protected]> * Minor update to otelmetricssource. Signed-off-by: carolxob <[email protected]> * Restructured ToC in Processors section for Data Prepper. Signed-off-by: carolxob <[email protected]> * Minor filename change. Signed-off-by: carolxob <[email protected]> * Adjustments to metadata in ToC. Signed-off-by: carolxob <[email protected]> * Minor edit. Signed-off-by: carolxob <[email protected]> * Fixed nav order in metadata. Signed-off-by: carolxob <[email protected]> * Minor edit. Signed-off-by: carolxob <[email protected]> * Minor update top metadata for ToC. Signed-off-by: carolxob <[email protected]> * Adjustmenets to Toc order. Signed-off-by: carolxob <[email protected]> * Minor adjustments to ToC metadata. Signed-off-by: carolxob <[email protected]> * Minor adjustments to Sinks section. Signed-off-by: carolxob <[email protected]> * Adjustements to high level ToC. Signed-off-by: carolxob <[email protected]> * Minor adjustement to Pipelines.md Signed-off-by: carolxob <[email protected]> * Minor update. Signed-off-by: carolxob <[email protected]> * Slight reorganization. Removed two placeholder pages for now. Signed-off-by: carolxob <[email protected]> * Removed a page and replaced with pipelines content. Signed-off-by: carolxob <[email protected]> * Minor changes/additions to content for placeholder pages. Signed-off-by: carolxob <[email protected]> * Minor update to page link. Signed-off-by: carolxob <[email protected]> * Minor adjustments to ToC metadata. Signed-off-by: carolxob <[email protected]> * Minor edits. Signed-off-by: carolxob <[email protected]> * Removed /clients from redirects to correct nav order. Signed-off-by: carolxob <[email protected]> * Minor edits. Signed-off-by: carolxob <[email protected]> * Minor adjustments to ToC metadata. Signed-off-by: carolxob <[email protected]> * Minor adjustments. Signed-off-by: carolxob <[email protected]> * Minor adjustment ot metadata. Signed-off-by: carolxob <[email protected]> * TOC link fixes Signed-off-by: Naarcha-AWS <[email protected]> * Changed page name. Signed-off-by: carolxob <[email protected]> * Corrected references to Peer Forwarder. Signed-off-by: carolxob <[email protected]> * Renamed Data Prepper folder. Signed-off-by: carolxob <[email protected]> * Minor updates to phrasing and capitalization. Signed-off-by: carolxob <[email protected]> * Minor phrasing update. Signed-off-by: carolxob <[email protected]> * Minor phrasing update. Signed-off-by: carolxob <[email protected]> * Minor change. Signed-off-by: carolxob <[email protected]> * Minor change to change S3 Source to S3Source. Signed-off-by: carolxob <[email protected]> * Updated references to peer forwarder and changed capitalization. Signed-off-by: carolxob <[email protected]> * Updated capitalization for peer forwarder. Signed-off-by: carolxob <[email protected]> * Made edits based on doc review feedback. Signed-off-by: carolxob <[email protected]> * Update to one word. Signed-off-by: carolxob <[email protected]> --------- Signed-off-by: carolxob <[email protected]> Signed-off-by: Naarcha-AWS <[email protected]> Co-authored-by: Naarcha-AWS <[email protected]>
38 lines
3.1 KiB
Markdown
38 lines
3.1 KiB
Markdown
---
|
|
layout: default
|
|
title: http_source
|
|
parent: Sources
|
|
grand_parent: Pipelines
|
|
nav_order: 5
|
|
---
|
|
|
|
# http_source
|
|
|
|
This is a source plugin that supports HTTP protocol. Currently ONLY support Json UTF-8 codec for incoming request, e.g. `[{"key1": "value1"}, {"key2": "value2"}]`.
|
|
|
|
Option | Required | Type | Description
|
|
:--- | :--- | :--- | :---
|
|
port | No | Integer | The port the source is running on. Default is `2021`. Valid options are between `0` and `65535`.
|
|
health_check_service | No | Boolean | Enables health check service on `/health` endpoint on the defined port. Default is `false`.
|
|
unauthenticated_health_check | No | Boolean | Determines whether or not authentication is required on the health check endpoint. Data Prepper ignores this option if no authentication is defined. Default is `false`.
|
|
request_timeout | No | Integer | The request timeout in millis. Default is `10_000`.
|
|
thread_count | No | Integer | The number of threads to keep in the ScheduledThreadPool. Default is `200`.
|
|
max_connection_count | No | Integer | The maximum allowed number of open connections. Default is `500`.
|
|
max_pending_requests | No | Integer | The maximum number of allowed tasks in ScheduledThreadPool work queue. Default is `1024`.
|
|
authentication | No | Object | An authentication configuration. By default, this creates an unauthenticated server for the pipeline. This uses pluggable authentication for HTTPS. To use basic authentication define the `http_basic` plugin with a `username` and `password`. To provide customer authentication, use or create a plugin that implements [ArmeriaHttpAuthenticationProvider](https://github.com/opensearch-project/data-prepper/blob/main/data-prepper-plugins/armeria-common/src/main/java/com/amazon/dataprepper/armeria/authentication/ArmeriaHttpAuthenticationProvider.java).
|
|
ssl | No | Boolean | Enables TLS/SSL. Default is false.
|
|
ssl_certificate_file | Conditionally | String | SSL certificate chain file path or AWS S3 path. S3 path example `s3://<bucketName>/<path>`. Required if `ssl` is true and `use_acm_certificate_for_ssl` is false.
|
|
ssl_key_file | Conditionally | String | SSL key file path or AWS S3 path. S3 path example `s3://<bucketName>/<path>`. Required if `ssl` is true and `use_acm_certificate_for_ssl` is false.
|
|
use_acm_certificate_for_ssl | No | Boolean | Enables TLS/SSL using certificate and private key from AWS Certificate Manager (ACM). Default is false.
|
|
acm_certificate_arn | Conditionally | String | ACM certificate ARN. The ACM certificate takes preference over S3 or a local file system certificate. Required if `use_acm_certificate_for_ssl` is set to true.
|
|
acm_private_key_password | No | String | ACM private key password that decrypts the private key. If not provided, Data Prepper generates a random password.
|
|
acm_certificate_timeout_millis | No | Integer | Timeout in milliseconds for ACM to get certificates. Default is 120000.
|
|
aws_region | Conditionally | String | AWS region to use ACM or S3. Required if `use_acm_certificate_for_ssl` is set to true or `ssl_certificate_file` and `ssl_key_file` is AWS S3 path.
|
|
|
|
<!--- ## Configuration
|
|
|
|
Content will be added to this section.
|
|
|
|
## Metrics
|
|
|
|
Content will be added to this section. ---> |