### PartitionSpec JSON Schema Example

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/APIReference/API_PartitionSpec

An example of the JSON schema for `PartitionSpec`, illustrating how identity transformations are applied to source columns to produce partition data.

```JSON
"partitionSpec": { "identity": [ {"sourceName": "column1"}, {"sourceName": "column2"}, {"sourceName": "column3"} ] }
```

--------------------------------

### LOGTOJSON Configuration with Custom Fields Example

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/dev/pre-processing

A specific configuration example for the `LOGTOJSON` option, demonstrating how to use `customFieldNames` to define specific field names when converting Apache Common Log entries to JSON format.

```JSON
{
    "optionName": "LOGTOJSON",
    "logFormat": "COMMONAPACHELOG",
    "customFieldNames": ["f1", "f2", "f3", "f4", "f5", "f6", "f7"]
}
```

--------------------------------

### Example S3 Object Key with Dynamic Partitioning Evaluation

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/dev/dynamic-partitioning-s3bucketprefix

This example illustrates the final S3 object key path generated by Amazon Kinesis Data Firehose after evaluating dynamic partitioning expressions. It shows how the "Prefix" configured in the destination is resolved at runtime with actual data values, leading to a structured S3 path for the delivered record.

```Text
s3://my-logs-prod/customer_id=1234567890/device=mobile/year=2019/month=08/day=09/hour=20/my-delivery-stream-2019-08-09-23-55-09-a9fa96af-e4e4-409f-bac3-1f804714faaa
```

--------------------------------

### SINGLELINE Processing Option Configuration Example

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/dev/pre-processing

An example configuration for the `SINGLELINE` processing option, which converts multi-line records into single-line records by removing newline characters, leading spaces, and trailing spaces.

```JSON
{
    "optionName": "SINGLELINE"
}
```

--------------------------------

### ListDeliveryStreams Sample HTTP Request

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/APIReference/API_ListDeliveryStreams

An example HTTP POST request demonstrating how to call `ListDeliveryStreams` to retrieve up to three `DirectPut` Firehose streams.

```HTTP
POST / HTTP/1.1
Host: firehose.<region>.<domain>
Content-Length: <PayloadSizeBytes>
User-Agent: <UserAgentString>
Content-Type: application/x-amz-json-1.1
Authorization: <AuthParams>
Connection: Keep-Alive
X-Amz-Date: <Date>
X-Amz-Target: Firehose_20150804.ListDeliveryStreams
{
    "DeliveryStreamType": "DirectPut",
    "Limit": 3
}
```

--------------------------------

### Example Apache Common Log Entry (Input)

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/dev/pre-processing

A sample Apache Common Log entry illustrating the input format before processing by Kinesis Firehose with a `LOGTOJSON` configuration.

```Text
123.45.67.89 - - [27/Oct/2000:09:27:09 -0400] "GET /java/javaResources.html HTTP/1.0" 200
```

--------------------------------

### Sample HTTP Response for Listing Tags for a Firehose Stream

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/APIReference/API_ListTagsForDeliveryStream

An example HTTP 200 OK response showing the tags returned by the ListTagsForDeliveryStream API for a Firehose stream.

```HTTP
HTTP/1.1 200 OK
x-amzn-RequestId: <RequestId>
Content-Type: application/x-amz-json-1.1
Content-Length: <PayloadSizeBytes>
Date: <Date>
{
  "HasMoreTags": "false",
  "Tags" : [
     {
       "Key": "Project",
       "Value": "myProject"
     },
     {
       "Key": "Environment",
       "Value": "Production"
     }
   ]
}
```

--------------------------------

### Example HTTP Request to Tag a Firehose Stream

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/APIReference/API_TagDeliveryStream

Illustrates a sample HTTP POST request to add two tags ('Project' and 'Environment') to a specified Firehose delivery stream named 'exampleDeliveryStreamName'.

```HTTP
POST / HTTP/1.1
Host: firehose.<region>.<domain>
Content-Length: <PayloadSizeBytes>
User-Agent: <UserAgentString>
Content-Type: application/x-amz-json-1.1
Authorization: <AuthParams>
Connection: Keep-Alive
X-Amz-Date: <Date>
X-Amz-Target: Firehose_20150804.TagDeliveryStream
{
  "DeliveryStreamName": "exampleDeliveryStreamName",
  "Tags": [
    {
      "Key": "Project",
      "Value": "myProject"
    },
    {
      "Key": "Environment",
      "Value": "Production"
    }
  ]
}
```

--------------------------------

### Example HTTP Endpoint URL for Firehose (US Region)

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/dev/create-destination

Provides a concrete example of an HTTP endpoint URL for Amazon Data Firehose, specifically for the US region.

```plaintext
https://listener-aws-metrics-stream-us.logz.io/
```

--------------------------------

### ListDeliveryStreams Sample HTTP Response

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/APIReference/API_ListDeliveryStreams

An example HTTP 200 OK response for the `ListDeliveryStreams` operation, showing a list of delivery stream names and indicating that more streams are available.

```HTTP
HTTP/1.1 200 OK
x-amzn-RequestId: <RequestId>
Content-Type: application/x-amz-json-1.1
Content-Length: <PayloadSizeBytes>
Date: <Date>
{
    "DeliveryStreamNames": [
        "some_delivery_stream",
        "another_example_delivery_stream",
        "last_stream_in_first_listing"
    ],
    "HasMoreDeliveryStreams": true
}
```

--------------------------------

### Example X-Amz-Firehose-Common-Attributes Header Value

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/dev/httpdeliveryrequestresponse

An example JSON object illustrating the structure and content of the `X-Amz-Firehose-Common-Attributes` header, showing how common attributes like deployment context and device types can be represented.

```JSON
"commonAttributes": {
    "deployment -context": "pre-prod-gamma",
    "device-types": ""
  }
```

--------------------------------

### Sample HTTP Request to List Tags for a Firehose Stream

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/APIReference/API_ListTagsForDeliveryStream

An example HTTP POST request demonstrating how to call the ListTagsForDeliveryStream API to retrieve tags for a specified Firehose stream.

```HTTP
POST / HTTP/1.1
Host: firehose.<region>.<domain>
Content-Length: <PayloadSizeBytes>
User-Agent: <UserAgentString>
Content-Type: application/x-amz-json-1.1
Authorization: <AuthParams>
Connection: Keep-Alive
X-Amz-Date: <Date>
X-Amz-Target: Firehose_20150804.ListTagsForDeliveryStream
{
  "DeliveryStreamName": "exampleDeliveryStreamName"
}
```

--------------------------------

### Amazon Data Firehose S3 Prefix Firehose Namespace Examples

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/dev/s3-prefixes

Provides examples for both 'error-output-type' and 'random-string' values within the 'firehose' namespace, showing their evaluation in custom S3 prefixes for error handling and high throughput scenarios.

```AWS Firehose Prefix Syntax
myPrefix/result=!{firehose:error-output-type}/!{timestamp:yyyy/MM/dd}
```

```AWS Firehose Prefix Syntax
myPrefix/!{firehose:random-string}/
```

--------------------------------

### LOGTOJSON Processing Option Configuration Example

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/dev/pre-processing

An example configuration for the `LOGTOJSON` processing option, which converts records from various log formats (e.g., Apache, Syslog) to JSON format. It supports specifying the log format, overriding patterns, and custom field names.

```JSON
{
    "optionName": "LOGTOJSON",
    "logFormat": "logformat",
    "matchPattern": "yourregexpattern",
    "customFieldNames": [ "field1", "field2", … ]
}
```

--------------------------------

### Amazon Kinesis Firehose CreateDeliveryStream API Sample Request

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/APIReference/API_CreateDeliveryStream

An example HTTP POST request to create an Amazon Kinesis Firehose delivery stream named `exampleStreamName` with an Amazon S3 destination configuration.

```HTTP
POST / HTTP/1.1
Host: firehose.<region>.<domain>
Content-Length: <PayloadSizeBytes>
User-Agent: <UserAgentString>
Content-Type: application/x-amz-json-1.1
Authorization: <AuthParams>
Connection: Keep-Alive
X-Amz-Date: <Date>
X-Amz-Target: Firehose_20150804.CreateDeliveryStream
{
    "DeliveryStreamName": "exampleStreamName",
    "S3DestinationConfiguration": {
        "RoleARN": "insert-role-ARN",
        "BucketARN": "insert-bucket-ARN",
        "BufferingHints": {
            "SizeInMBs": 3,
            "IntervalInSeconds": 60
        },
        "CompressionFormat": "ZIP"
    }
}
```

--------------------------------

### LOGTOJSON Configuration for Apache Common Log

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/dev/pre-processing

A specific configuration example for the `LOGTOJSON` option, demonstrating how to convert Apache Common Log entries into JSON format using default field names.

```JSON
{
    "optionName": "LOGTOJSON",
    "logFormat": "COMMONAPACHELOG"
}
```

--------------------------------

### CSVTOJSON Processing Option Configuration Example

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/dev/pre-processing

An example configuration for the `CSVTOJSON` processing option, used to convert records from a delimiter-separated format to JSON format. It allows specifying custom field names and the delimiter character.

```JSON
{
    "optionName": "CSVTOJSON",
    "customFieldNames": [ "field1", "field2", ... ],
    "delimiter": "yourdelimiter"
}
```

--------------------------------

### Example Firehose Response Bodies for Success and Failure

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/dev/httpdeliveryrequestresponse

Presents JSON examples for both successful (HTTP 200) and failed (HTTP 4xx/5xx) responses from a custom HTTPS endpoint to Amazon Kinesis Data Firehose. The failure example includes an `errorMessage` field, while the success example only requires `requestId` and `timestamp`.

```JSON
Failure Case (HTTP Response Code 4xx or 5xx)
{
  "requestId": "ed4acda5-034f-9f42-bba1-f29aea6d7d8f",
  "timestamp": "1578090903599",
  "errorMessage": "Unable to deliver records due to unknown error."
}
Success case (HTTP Response Code 200)
{
  "requestId": "ed4acda5-034f-9f42-bba1-f29aea6d7d8f",
  "timestamp": 1578090903599
}
```

--------------------------------

### Example HTTP Response for Tagging a Firehose Stream

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/APIReference/API_TagDeliveryStream

Shows a sample successful HTTP 200 OK response after tagging a Firehose delivery stream, indicating the operation's success.

```HTTP
HTTP/1.1 200 OK
x-amzn-RequestId: <RequestId>
Content-Type: application/x-amz-json-1.1
Content-Length: <PayloadSizeBytes>
Date: <Date>
```

--------------------------------

### Amazon Data Firehose DatabaseSourceDescription Object API

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/APIReference/API_DatabaseSourceDescription

This API documentation defines the structure and properties of the DatabaseSourceDescription object, which configures how Amazon Data Firehose connects to and reads from a source database endpoint. It includes details for specifying database and column patterns, authentication, VPC setup, endpoint and port, snapshot tracking, SSL mode, and the database engine type.

```APIDOC
DatabaseSourceDescription:
  description: The top level object for database source description.
  properties:
    Columns:
      description: The list of column patterns in source database endpoint for Firehose to read from.
      type: DatabaseColumnList object
      required: No
    Databases:
      description: The list of database patterns in source database endpoint for Firehose to read from.
      type: DatabaseList object
      required: No
    DatabaseSourceAuthenticationConfiguration:
      description: The structure to configure the authentication methods for Firehose to connect to source database endpoint.
      type: DatabaseSourceAuthenticationConfiguration object
      required: No
    DatabaseSourceVPCConfiguration:
      description: The details of the VPC Endpoint Service which Firehose uses to create a PrivateLink to the database.
      type: DatabaseSourceVPCConfiguration object
      required: No
    Endpoint:
      description: The endpoint of the database server.
      type: String
      constraints:
        length: Min 1, Max 255
        pattern: ^(?!s*$).+
      required: No
    Port:
      description: The port of the database. This can be one of the following values.
        - 3306 for MySQL database type
        - 5432 for PostgreSQL database type
      type: Integer
      constraints:
        range: Min 0, Max 65535
      required: No
    SnapshotInfo:
      description: The structure that describes the snapshot information of a table in source database endpoint that Firehose reads.
      type: Array of DatabaseSnapshotInfo objects
      required: No
    SnapshotWatermarkTable:
      description: The fully qualified name of the table in source database endpoint that Firehose uses to track snapshot progress.
      type: String
      constraints:
        length: Min 1, Max 129
        pattern: [\u0001-\uFFFF]*
      required: No
    SSLMode:
      description: The mode to enable or disable SSL when Firehose connects to the database endpoint.
      type: String
      constraints:
        valid_values: Disabled | Enabled
      required: No
    SurrogateKeys:
      description: The optional list of table and column names used as unique key columns when taking snapshot if the tables don’t have primary keys configured.
      type: Array of strings
      constraints:
        length: Min 1, Max 194
        pattern: [\u0001-\uFFFF]*
      required: No
    Tables:
      description: The list of table patterns in source database endpoint for Firehose to read from.
      type: DatabaseTableList object
      required: No
    Type:
      description: The type of database engine. This can be one of the following values.
        - MySQL
        - PostgreSQL
      type: String
      constraints:
        valid_values: MySQL | PostgreSQL
      required: No
```

--------------------------------

### Example JSON Output from Kinesis Firehose Log Conversion

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/dev/pre-processing

The resulting JSON output after a sample Apache Common Log entry has been processed by Kinesis Firehose using a `LOGTOJSON` configuration with a custom match pattern, demonstrating the extracted and mapped fields.

```JSON
{"host":"123.45.67.89","ident":null,"authuser":null,"datetime":"27/Oct/2000:09:27:09 -0400","request":"GET /java/javaResources.html HTTP/1.0","response":"200"}
```

--------------------------------

### DeleteDeliveryStream Sample HTTP Request

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/APIReference/API_DeleteDeliveryStream

An example HTTP POST request to delete a Firehose stream named 'exampleStreamName', demonstrating the required headers and JSON payload.

```HTTP
POST / HTTP/1.1
Host: firehose.<region>.<domain>
Content-Length: <PayloadSizeBytes>
User-Agent: <UserAgentString>
Content-Type: application/x-amz-json-1.1
Authorization: <AuthParams>
Connection: Keep-Alive
X-Amz-Date: <Date>
X-Amz-Target: Firehose_20150804.DeleteDeliveryStream
{
    "DeliveryStreamName": "exampleStreamName"
}
```

--------------------------------

### Snowflake Data Delivery Error Record Example

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/dev/retry

An example JSON structure for a record indicating a data delivery failure to a Snowflake destination. This specific example shows an 'InvalidColumns' error, detailing the reason for the failure and other relevant metadata.

```JSON
{
    "attemptsMade": 3,
    "arrivalTimestamp": 1594265943615,
    "errorCode": "Snowflake.InvalidColumns",
    "errorMessage": "Snowpipe Streaming does not support columns of type AUTOINCREMENT, IDENTITY, GEO, or columns with a default value or collation",
    "attemptEndingTimestamp": 1712937865543,
    "rawData": "c2FtcGxlIHJhdyBkYXRh"
}
```

--------------------------------

### DescribeDeliveryStream API Request Example

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/APIReference/API_DescribeDeliveryStream

This snippet shows a sample HTTP POST request to the Kinesis Firehose DescribeDeliveryStream API. It includes the necessary HTTP headers for authentication and content type, along with a JSON payload specifying the 'DeliveryStreamName' to query.

```HTTP
POST / HTTP/1.1
Host: firehose.<region>.<domain>
Content-Length: <PayloadSizeBytes>
User-Agent: <UserAgentString>
Content-Type: application/x-amz-json-1.1
Authorization: <AuthParams>
Connection: Keep-Alive
X-Amz-Date: <Date>
X-Amz-Target: Firehose_20150804.DescribeDeliveryStream
{
    "DeliveryStreamName": "exampleStreamName"
}
```

--------------------------------

### AWS Kinesis Firehose PutRecordBatch Sample HTTP Request

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/APIReference/API_PutRecordBatch

An example HTTP request demonstrating how to put two records into a Firehose stream named 'some_delivery_stream' using the PutRecordBatch API.

```HTTP
POST / HTTP/1.1
Host: firehose.<region>.<domain>
Content-Length: <PayloadSizeBytes>
User-Agent: <UserAgentString>
Content-Type: application/x-amz-json-1.1
Authorization: <AuthParams>
Connection: Keep-Alive
X-Amz-Date: <Date>
X-Amz-Target: Firehose_20150804.PutRecordBatch
{
   "DeliveryStreamName": "some_delivery_stream",
   "Records": [
      {
         "Data": "Some data blob."
      },
      {
         "Data": "Another blob of data."
      }
   ]
}
```

--------------------------------

### Sample Lambda Output for Firehose Amazon MSK Transformation

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/dev/data-transformation-status-model

A JSON example demonstrating the expected structure of a Lambda function's output when transforming records for Amazon Data Firehose, specifically for Amazon MSK destinations. It includes the record ID, transformation result, and base64-encoded Kafka record value.

```JSON
{
   "recordId": "<recordId from the Lambda input>",
   "result": "Ok",
   "kafkaRecordValue": "<Base64 encoded Transformed data>"
}
```

--------------------------------

### AWS Firehose Prefix and ErrorOutputPrefix Evaluation Examples

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/dev/s3-prefixes

Illustrates various combinations of `Prefix` and `ErrorOutputPrefix` values and their evaluated outcomes, demonstrating how timestamp and error-output-type expressions are processed by AWS Firehose under different scenarios.

```Configuration
Input | Evaluated prefix (at 10:30 AM UTC on Aug 27, 2018)
--- | ---
`Prefix`: Unspecified  `ErrorOutputPrefix`: `myFirehoseFailures/!{firehose:error-output-type}/` | `Prefix`: `2018/08/27/10`  `ErrorOutputPrefix`: `myFirehoseFailures/processing-failed/`
`Prefix`: `!{timestamp:yyyy/MM/dd}`  `ErrorOutputPrefix`: Unspecified | Invalid input: `ErrorOutputPrefix` can't be null when Prefix contains expressions
`Prefix`: `myFirehose/DeliveredYear=!{timestamp:yyyy}/anyMonth/rand=!{firehose:random-string}`  `ErrorOutputPrefix`: `myFirehoseFailures/!{firehose:error-output-type}/!{timestamp:yyyy}/anyMonth/!{timestamp:dd}` | `Prefix`: `myFirehose/DeliveredYear=2018/anyMonth/rand=5abf82daaa5`  `ErrorOutputPrefix`: `myFirehoseFailures/processing-failed/2018/anyMonth/10`
`Prefix`: `myPrefix/year=!{timestamp:yyyy}/month=!{timestamp:MM}/day=!{timestamp:dd}/hour=!{timestamp:HH}/`  `ErrorOutputPrefix`: `myErrorPrefix/year=!{timestamp:yyyy}/month=!{timestamp:MM}/day=!{timestamp:dd}/hour=!{timestamp:HH}/!{firehose:error-output-type}` | `Prefix`: `myPrefix/year=2018/month=07/day=06/hour=23/`  `ErrorOutputPrefix`: `myErrorPrefix/year=2018/month=07/day=06/hour=23/processing-failed`
`Prefix`: `myFirehosePrefix/`  `ErrorOutputPrefix`: Unspecified | `Prefix`: `myFirehosePrefix/2018/08/27/`  `ErrorOutputPrefix`: `myFirehosePrefix/processing-failed/2018/08/27/`
```

--------------------------------

### Access Denied Exception for Firehose Tagging

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/dev/firehose-tagging-basics

This snippet shows an example of an AccessDeniedException that occurs when a user attempts to create a Firehose stream with tags but lacks the firehose:TagDeliveryStream permission in their IAM policy.

```Text
AccessDeniedException
User: arn:aws:sts::x:assumed-role/x/x is not authorized to perform: firehose:TagDeliveryStream on resource: arn:aws:firehose:us-east-1:x:deliverystream/x with an explicit deny in an identity-based policy.
```

--------------------------------

### Amazon Kinesis Firehose CreateDeliveryStream API Sample Response

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/APIReference/API_CreateDeliveryStream

An example HTTP 200 OK response received after successfully creating an Amazon Kinesis Firehose delivery stream, including the `DeliveryStreamARN`.

```HTTP
HTTP/1.1 200 OK
x-amzn-RequestId: <RequestId>
Content-Type: application/x-amz-json-1.1
Content-Length: <PayloadSizeBytes>
Date: <Date>
{
    "DeliveryStreamARN": "arn:aws:firehose:us-east-1:814985986679:deliverystream/exampleStreamName"
}
```

--------------------------------

### AWS Kinesis Firehose PutRecordBatch Sample HTTP Response

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/APIReference/API_PutRecordBatch

An example HTTP response for a successful PutRecordBatch operation, showing the returned RecordIds for the processed records.

```HTTP
HTTP/1.1 200 OK
x-amzn-RequestId: <RequestId>
Content-Type: application/x-amz-json-1.1
Content-Length: <PayloadSizeBytes>
Date: <Date>
{
    "FailedPutCount": 0,
    "RequestResponses": [
        {
            "RecordId": "AJJBALlfiFN9HyhPj6Dc+XqcRlTjqyIbr927TsEmWWpN39EK/JbRTbXDZFNCCrWIs/4YUd3gjYUs6giZn76TuI4vv0ljDOMFJvLzqLuupu93RscZRmpW1CP8DeiFzJJGvqIr1LRE/MDozYenOz+v1ZIqEjECvfMwIz6silvDaGuYtIhVzwd8yWaDGFCX4ODes6W07W9W7Q5ViMPUKSt6F0nn7OhVkJ3/"
        },
        {
            "RecordId": "goGaFS919Mmv7lYET0oMaw+UL9iFpzi1O0o+csoIc31SmvkqpwzQuT0RPqZ7QqfR1FJ+HxJciW/8paFMWPByJ6qVDhiE7TtJxARKaP4YDccvWHXhD7x6Y4bc9AHZ0uzy+BEuTTnH5Zsip1IrEPgu8lj8a7fd6sluTe/kZ54GVG+Sc105IrxIu08kPbRiUTWL+Wg0A3HzOQeePqokRKKqmoXLG0pzx/8O"
        }
    ]
}
```

--------------------------------

### Splunk Data Delivery Error Record Example

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/dev/retry

This JSON example illustrates a typical error record generated when Amazon Data Firehose fails to deliver data to Splunk. It shows specific error details such as 'Splunk.AckTimeout', timestamps, and the raw data, indicating that data is backed up to S3 upon delivery failure.

```JSON
{
  "attemptsMade": 0,
  "arrivalTimestamp": 1506035354675,
  "errorCode": "Splunk.AckTimeout",
  "errorMessage": "Did not receive an acknowledgement from HEC before the HEC acknowledgement timeout expired. Despite the acknowledgement timeout, it's possible the data was indexed successfully in Splunk. Amazon Data Firehose backs up in Amazon S3 data for which the acknowledgement timeout expired.",
  "attemptEndingTimestamp": 13626284715507,
  "rawData": "MiAyNTE2MjAyNzIyMDkgZW5pLTA1ZjMyMmQ1IDIxOC45Mi4xODguMjE0IDE3Mi4xNi4xLjE2NyAyNTIzMyAxNDMzIDYgMSA0MCAxNTA2MDM0NzM0IDE1MDYwMzQ3OTQgUkVKRUNUIE9LCg==",
  "EventId": "49577193928114147339600778471082492393164139877200035842.0"
}
```

--------------------------------

### Sample Lambda Output for Firehose Kinesis Data Streams/Direct PUT Transformation

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/dev/data-transformation-status-model

A JSON example demonstrating the expected structure of a Lambda function's output when transforming records for Amazon Data Firehose, specifically for Kinesis Data Streams or Direct PUT destinations. It includes the record ID, transformation result, and base64-encoded transformed data.

```JSON
{
   "recordId": "<recordId from the Lambda input>",
   "result": "Ok",
   "data": "<Base64 encoded Transformed data>"
}
```

--------------------------------

### Firehose Metadata Examples for Snowflake Destination

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/dev/create-destination

Examples of metadata JSON structures that Amazon Kinesis Firehose can deliver to a Snowflake table, depending on the data source type. These structures provide contextual information about the data's origin and ingestion time.

```JSON
{
"firehoseDeliveryStreamName" : "streamname",
"IngestionTime" : "timestamp"
}
```

```JSON
{
"kinesisStreamName" : "streamname",
"kinesisShardId" : "Id",
"kinesisPartitionKey" : "key",
"kinesisSequenceNumber" : "1234",
"subsequenceNumber" : "2334",
"IngestionTime" : "timestamp"
}
```

--------------------------------

### Start Server-Side Encryption for AWS Kinesis Firehose Stream - Sample Request

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/APIReference/API_StartDeliveryStreamEncryption

Demonstrates an HTTP POST request to initiate server-side encryption for a specified Amazon Kinesis Firehose delivery stream using the StartDeliveryStreamEncryption API action.

```HTTP
POST / HTTP/1.1
Host: firehose.<region>.<domain>
Content-Length: <PayloadSizeBytes>
User-Agent: <UserAgentString>
Content-Type: application/x-amz-json-1.1
Authorization: <AuthParams>
Connection: Keep-Alive
X-Amz-Date: <Date>
X-Amz-Target: Firehose_20150804.StartDeliveryStreamEncryption
{
  "DeliveryStreamName": "exampleDeliveryStreamName"
}
```

--------------------------------

### DeleteDeliveryStream Sample HTTP Response

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/APIReference/API_DeleteDeliveryStream

An example HTTP 200 OK response indicating successful deletion of a Firehose stream, showing typical response headers.

```HTTP
HTTP/1.1 200 OK
x-amzn-RequestId: <RequestId>
Content-Type: application/x-amz-json-1.1
Content-Length: <PayloadSizeBytes>
Date: <Date>
```

--------------------------------

### HTTP Endpoint Data Delivery Error Record Example

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/dev/retry

An example JSON structure representing an error record generated by Amazon Data Firehose when data delivery to an HTTP endpoint fails. It includes details such as the number of attempts, timestamps, error code, and the specific error message received from the endpoint.

```JSON
{
	"attemptsMade":5,
	"arrivalTimestamp":1594265943615,
	"errorCode":"HttpEndpoint.DestinationException",
	"errorMessage":"Received the following response from the endpoint destination. {\"requestId\": \"109777ac-8f9b-4082-8e8d-b4f12b5fc17b\", \"timestamp\": 1594266081268, \"errorMessage\": \"Unauthorized\"}",
	"attemptEndingTimestamp":1594266081318,
	"rawData":"c2FtcGxlIHJhdyBkYXRh",
	"subsequenceNumber":0,
	"dataId":"49607357361271740811418664280693044274821622880012337186.0"
}
```

--------------------------------

### Example AccessDeniedException for Firehose Tagging

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/dev/create-configure-advanced

This snippet illustrates an AccessDeniedException error message received when a user attempts to create a Firehose stream with tags but lacks the necessary 'firehose:TagDeliveryStream' permission, indicating an explicit deny in an identity-based policy.

```Text
AccessDeniedException
User: arn:aws:sts::x:assumed-role/x/x is not authorized to perform: firehose:TagDeliveryStream on resource: arn:aws:firehose:us-east-1:x:deliverystream/x with an explicit deny in an identity-based policy.
```

--------------------------------

### UntagDeliveryStream API Sample Response

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/APIReference/API_UntagDeliveryStream

An example HTTP 200 OK response indicating successful tag removal from an Amazon Data Firehose delivery stream.

```HTTP
HTTP/1.1 200 OK
x-amzn-RequestId: <RequestId>
Content-Type: application/x-amz-json-1.1
Content-Length: <PayloadSizeBytes>
Date: <Date>
```

--------------------------------

### DescribeDeliveryStream API Response Example

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/APIReference/API_DescribeDeliveryStream

This snippet illustrates a sample HTTP 200 OK response from the Kinesis Firehose DescribeDeliveryStream API. The response body contains a 'DeliveryStreamDescription' object, detailing various attributes such as stream type, status, ARN, creation timestamp, encryption configuration, and destination configurations (S3 and Extended S3).

```JSON
HTTP/1.1 200 OK
x-amzn-RequestId: <RequestId>
Content-Type: application/x-amz-json-1.1
Content-Length: <PayloadSizeBytes>
Date: <Date>
{
  "DeliveryStreamDescription": {
    "DeliveryStreamType": "DirectPut",
    "HasMoreDestinations": false,
    "VersionId": "1",
    "CreateTimestamp": 1517595920.596,
    "DeliveryStreamARN": "arn:aws:firehose:us-east-1:111222333444:deliverystream/exampleStreamName",
    "DeliveryStreamStatus": "ACTIVE",
    "DeliveryStreamName": "exampleStreamName",
    "DeliveryStreamEncryptionConfiguration": {
      "Status": "DISABLED"
    },
    "Destinations": [
      {
        "DestinationId": "destinationId-000000000001",
        "ExtendedS3DestinationDescription": {
          "RoleARN": "arn:aws:iam::111222333444:role/exampleStreamName",
          "Prefix": "",
          "BufferingHints": {
            "IntervalInSeconds": 60,
            "SizeInMBs": 1
          },
          "EncryptionConfiguration": {
            "NoEncryptionConfig": "NoEncryption"
          },
          "CompressionFormat": "UNCOMPRESSED",
          "S3BackupMode": "Disabled",
          "CloudWatchLoggingOptions": {
            "Enabled": true,
            "LogStreamName": "S3Delivery",
            "LogGroupName": "/aws/kinesisfirehose/exampleStreamName"
          },
          "BucketARN": "arn:aws:s3:::somebucket",
          "ProcessingConfiguration": {
            "Enabled": false,
            "Processors": []
          }
        },
        "S3DestinationDescription": {
          "RoleARN": "arn:aws:iam::111222333444:role/exampleStreamName",
          "Prefix": "",
          "BufferingHints": {
            "IntervalInSeconds": 60,
            "SizeInMBs": 1
          },
          "EncryptionConfiguration": {
            "NoEncryptionConfig": "NoEncryption"
          },
          "CompressionFormat": "UNCOMPRESSED",
          "CloudWatchLoggingOptions": {
            "Enabled": true,
            "LogStreamName": "S3Delivery",
            "LogGroupName": "/aws/kinesisfirehose/exampleStreamName"
          },
          "BucketARN": "arn:aws:s3:::somebucket"
        }
      }
    ]
  }
}
```

--------------------------------

### MSKSourceConfiguration API Reference

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/APIReference/API_MSKSourceConfiguration

Defines the configuration for an Amazon MSK cluster used as a source for a Firehose delivery stream, including authentication, cluster ARN, topic name, and read start timestamp.

```APIDOC
MSKSourceConfiguration:
  description: The configuration for the Amazon MSK cluster to be used as the source for a delivery stream.
  properties:
    AuthenticationConfiguration:
      type: AuthenticationConfiguration object
      required: Yes
      description: The authentication configuration of the Amazon MSK cluster.
    MSKClusterARN:
      type: String
      required: Yes
      description: The ARN of the Amazon MSK cluster.
      constraints:
        length: Min 1, Max 512
        pattern: arn:.*
    TopicName:
      type: String
      required: Yes
      description: The topic name within the Amazon MSK cluster.
      constraints:
        length: Min 1, Max 255
        pattern: '[a-zA-Z0-9\\._\\-]+'
    ReadFromTimestamp:
      type: Timestamp
      required: No
      description: The start date and time in UTC for the offset position within your MSK topic from where Firehose begins to read. By default, this is set to timestamp when Firehose becomes Active. If you want to create a Firehose stream with Earliest start position from SDK or CLI, you need to set the ReadFromTimestamp parameter to Epoch (1970-01-01T00:00:00Z).
```

--------------------------------

### Restart AWS Kinesis Agent Service

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/dev/agent-config-settings

Commands to stop and start or directly restart the AWS Kinesis Agent service after modifying its configuration file. These commands ensure that changes to `/etc/aws-kinesis/agent.json` are applied.

```bash
sudo service aws-kinesis-agent stop
sudo service aws-kinesis-agent start
```

```bash
sudo service aws-kinesis-agent restart
```

--------------------------------

### API Reference: MSKSourceDescription Object

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/APIReference/API_MSKSourceDescription

Defines the structure and properties of the MSKSourceDescription object used in Amazon Data Firehose to specify details about an Amazon MSK cluster source. It includes properties like authentication, start timestamps, ARN, and topic name.

```APIDOC
MSKSourceDescription:
  Description: Details about the Amazon MSK cluster used as the source for a Firehose stream.
  Properties:
    AuthenticationConfiguration:
      Type: AuthenticationConfiguration object
      Required: No
      Description: The authentication configuration of the Amazon MSK cluster.
    DeliveryStartTimestamp:
      Type: Timestamp
      Required: No
      Description: Firehose starts retrieving records from the topic within the Amazon MSK cluster starting with this timestamp.
    MSKClusterARN:
      Type: String
      Required: No
      Length Constraints: Minimum length of 1. Maximum length of 512.
      Pattern: arn:.*
      Description: The ARN of the Amazon MSK cluster.
    ReadFromTimestamp:
      Type: Timestamp
      Required: No
      Description: The start date and time in UTC for the offset position within your MSK topic from where Firehose begins to read. By default, this is set to timestamp when Firehose becomes Active. If you want to create a Firehose stream with Earliest start position from SDK or CLI, you need to set the `ReadFromTimestampUTC` parameter to Epoch (1970-01-01T00:00:00Z).
    TopicName:
      Type: String
      Required: No
      Length Constraints: Minimum length of 1. Maximum length of 255.
      Pattern: [a-zA-Z0-9\\._\\-]+
      Description: The topic name within the Amazon MSK cluster.
```

--------------------------------

### Start Server-Side Encryption for AWS Kinesis Firehose Stream - Sample Response

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/APIReference/API_StartDeliveryStreamEncryption

Shows a successful HTTP 200 OK response received after initiating server-side encryption for an Amazon Kinesis Firehose delivery stream.

```HTTP
HTTP/1.1 200 OK
x-amzn-RequestId: <RequestId>
Content-Type: application/x-amz-json-1.1
Content-Length: <PayloadSizeBytes>
Date: <Date>
```

--------------------------------

### HTTP Request Example: StopDeliveryStreamEncryption

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/APIReference/API_StopDeliveryStreamEncryption

An example HTTP POST request to disable server-side encryption for an Amazon Data Firehose stream.

```HTTP
POST / HTTP/1.1
Host: firehose.<region>.<domain>
Content-Length: <PayloadSizeBytes>
User-Agent: <UserAgentString>
Content-Type: application/x-amz-json-1.1
Authorization: <AuthParams>
Connection: Keep-Alive
X-Amz-Date: <Date>
X-Amz-Target: Firehose_20150804.StopDeliveryStreamEncryption
{
  "DeliveryStreamName": "exampleDeliveryStreamName"
}
```

--------------------------------

### HTTP Response Example: StopDeliveryStreamEncryption

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/APIReference/API_StopDeliveryStreamEncryption

An example HTTP 200 OK response indicating successful disabling of server-side encryption for an Amazon Data Firehose stream.

```HTTP
HTTP/1.1 200 OK
x-amzn-RequestId: <RequestId>
Content-Type: application/x-amz-json-1.1
Content-Length: <PayloadSizeBytes>
Date: <Date>
```

--------------------------------

### StartDeliveryStreamEncryption Request Parameters

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/APIReference/API_StartDeliveryStreamEncryption

Details the parameters accepted by the StartDeliveryStreamEncryption API, including their types, constraints, and whether they are required for the operation.

```APIDOC
DeliveryStreamEncryptionConfigurationInput:
  Description: Used to specify the type and Amazon Resource Name (ARN) of the KMS key needed for Server-Side Encryption (SSE).
  Type: DeliveryStreamEncryptionConfigurationInput object
  Required: No
DeliveryStreamName:
  Description: The name of the Firehose stream for which you want to enable server-side encryption (SSE).
  Type: String
  Length Constraints: Minimum length of 1. Maximum length of 64.
  Pattern: [a-zA-Z0-9_.-]+
  Required: Yes
```

--------------------------------

### API Reference for TableCreationConfiguration

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/APIReference/API_TableCreationConfiguration

Defines the structure and properties of the TableCreationConfiguration API, used to enable automatic table creation in Amazon Data Firehose. This API is part of a preview release and is subject to change.

```APIDOC
TableCreationConfiguration:
  Description: The configuration to enable automatic table creation.
  Properties:
    Enabled:
      Description: Specify whether you want to enable automatic table creation.
      Type: Boolean
      Required: Yes
```

--------------------------------

### Reference for Amazon Kinesis Data Firehose Destination Buffering Hint Ranges and Defaults

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/dev/create-configure-backup

Provides a comprehensive table detailing the allowed buffering size (in MB) and buffering interval (in seconds) ranges, along with their default values, for various Amazon Kinesis Data Firehose destinations. This information is crucial when configuring or reconfiguring buffering hints via API or console.

```APIDOC
Destination Buffering Hint Ranges and Defaults:
| Destination | Buffering size in MB (default in parenthesis) | Buffering interval in seconds (default in parenthesis) |
| --- | --- | --- |
| Amazon S3 | 1-128 (5) | 0-900 (300) |
| Apache Iceberg Tables | 1-128 (5) | 0-900 (300) |
| Amazon Redshift | 1-128 (5) | 0-900 (300) |
| OpenSearch Serverless | 1-100 (5) | 0-900 (300) |
| OpenSearch | 1-100 (5) | 0-900 (300) |
| Splunk | 1-5 (5) | 0-60 (60) |
| Datadog | 1-4 (4) | 0-900 (60) |
| Coralogix | 1-64 (6) | 0-900 (60) |
| Dynatrace | 1-64 (5) | 0-900 (60) |
| Elastic | 1 | 0-900 (60) |
| Honeycomb | 1-64 (15) | 0-900 (60) |
| HTTP endpoint | 1-64 (5) | 0-900 (60) |
| LogicMonitor | 1-64 (5) | 0-900 (60) |
| Logzio | 1-64 (5) | 0-900 (60) |
| mongoDB | 1-16 (5) | 0-900 (60) |
| newRelic | 1-64 (5) | 0-900 (60) |
| sumoLogic | 1-64 (1) | 0-900 (60) |
| Splunk Observability Cloud | 1-64 (1) | 0-900 (60) |
| Snowflake | 1 - 128 (1) | 0 - 900 (0) |
```

--------------------------------

### IcebergDestinationConfiguration API Reference

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/APIReference/API_IcebergDestinationConfiguration

Detailed API documentation for the IcebergDestinationConfiguration object, used to specify destination settings for Apache Iceberg Tables in Amazon Data Firehose.

```APIDOC
IcebergDestinationConfiguration:
  CatalogConfiguration:
    Type: CatalogConfiguration object
    Required: Yes
    Description: Configuration describing where the destination Apache Iceberg Tables are persisted.
  RoleARN:
    Type: String
    Length Constraints: Minimum length of 1. Maximum length of 512.
    Pattern: `arn:.*:iam::\d{12}:role/[a-zA-Z_0-9+=,.@\-_/]+`
    Required: Yes
    Description: The Amazon Resource Name (ARN) of the IAM role to be assumed by Firehose for calling Apache Iceberg Tables.
  S3Configuration:
    Type: S3DestinationConfiguration object
    Required: Yes
    Description: Describes the configuration of a destination in Amazon S3.
  AppendOnly:
    Type: Boolean
    Required: No
    Description: Describes whether all incoming data for this delivery stream will be append only (inserts only and not for updates and deletes) for Iceberg delivery. This feature is only applicable for Apache Iceberg Tables. The default value is false. If you set this value to true, Firehose automatically increases the throughput limit of a stream based on the throttling levels of the stream. If you set this parameter to true for a stream with updates and deletes, you will see out of order delivery.
  BufferingHints:
    Type: BufferingHints object
    Required: No
    Description: Describes hints for the buffering to perform before delivering data to the destination. These options are treated as hints, and therefore Firehose might choose to use different values when it is optimal. The `SizeInMBs` and `IntervalInSeconds` parameters are optional. However, if specify a value for one of them, you must also provide a value for the other.
  CloudWatchLoggingOptions:
    Type: CloudWatchLoggingOptions object
    Required: No
    Description: Describes the Amazon CloudWatch logging options for your Firehose stream.
  DestinationTableConfigurationList:
    Type: Array of DestinationTableConfiguration objects
    Required: No
    Description: Provides a list of `DestinationTableConfigurations` which Firehose uses to deliver data to Apache Iceberg Tables. Firehose will write data with insert if table specific configuration is not provided here.
  ProcessingConfiguration:
    Type: ProcessingConfiguration object
    Required: No
    Description: Describes a data processing configuration.
  RetryOptions:
    Type: RetryOptions object
    Required: No
    Description: The retry behavior in case Firehose is unable to deliver data to a destination.
  S3BackupMode:
    Type: String
    Valid Values: `FailedDataOnly | AllData`
    Required: No
    Description: Describes how Firehose will backup records. Currently,S3 backup only supports `FailedDataOnly`.
  SchemaEvolutionConfiguration:
    Type: SchemaEvolutionConfiguration object
    Required: No
    Description: The configuration to enable automatic schema evolution. Amazon Data Firehose is in preview release and is subject to change.
  TableCreationConfiguration:
    Type: TableCreationConfiguration object
    Required: No
    Description: The configuration to enable automatic table creation. Amazon Data Firehose is in preview release and is subject to change.
```

--------------------------------

### JSON Input Examples for Firehose Multi-Record Deaggregation

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/dev/dynamic-partitioning-multirecord-deaggergation

Examples demonstrating correct and incorrect JSON input formats for multi-record deaggregation in Amazon Data Firehose. Valid inputs include single-line concatenated JSON objects or newline-delimited JSON objects (JSONL). An array of JSON objects is explicitly stated as an invalid input for deaggregation.

```JSON
{"a":1}{"a":2}
```

```JSON
{"a":1}\n{"a":2}
```

```JSON
[{"a":1}, {"a":2}]
```

--------------------------------

### Configure Buffering Hints for New Firehose Streams using CreateDeliveryStream API

Source: https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html/dev/create-configure-backup

Explains how to set buffering size and interval when creating a new Amazon Kinesis Data Firehose delivery stream. It specifies the parameters SizeInMBs and IntervalInSeconds within the DestinationConfiguration parameter.

```APIDOC
CreateDeliveryStream API:
  Purpose: Configures buffering hints for new Amazon Kinesis Data Firehose delivery streams.
  Parameters:
    DestinationConfiguration:
      Type: Object
      Description: Contains configuration for the destination, including buffering hints.
      Properties:
        SizeInMBs:
          Type: Integer
          Description: The buffering size in MBs.
          Range: Varies by destination (e.g., 1-128 for S3, 1-5 for Splunk).
          Default: Varies by destination (e.g., 5 for S3, 1 for Snowflake).
        IntervalInSeconds:
          Type: Integer
          Description: The buffering interval in seconds.
          Range: Varies by destination (e.g., 0-900 for S3, 0-60 for Splunk).
          Default: Varies by destination (e.g., 300 for S3, 0 for Snowflake).
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.