### Terraform Usage Example

Source: https://github.com/coralogix/coralogix-aws-shipper/blob/master/examples/s3-multiple-buckets-one-lambda/terraform/README.md

Follow these steps to initialize, configure, and apply the Terraform configuration. Copy the example tfvars file and update it with your specific settings. The output `bucket_names_csv` is crucial for configuring the Coralogix shipper.

```bash
cd examples/s3-multiple-buckets-one-lambda/terraform
cp terraform.tfvars.example terraform.tfvars

terraform init
terraform apply
```

--------------------------------

### Terraform Initialization and Apply

Source: https://github.com/coralogix/coralogix-aws-shipper/blob/master/examples/firehose-metrics-private-link/terraform/README.md

Navigate to the Terraform directory, copy the example configuration, edit it with your Coralogix PrivateLink service name, and then initialize and apply the Terraform configuration.

```bash
cd examples/firehose-metrics-private-link/terraform
cp terraform.tfvars.example terraform.tfvars
# Edit terraform.tfvars — service name is required.

terraform init
terraform apply
```

--------------------------------

### Configure StarlarkScript parameter

Source: https://github.com/coralogix/coralogix-aws-shipper/blob/master/README.md

Examples of valid formats for the StarlarkScript parameter based on source type.

```text
s3://my-bucket/scripts/transform.star
```

```text
https://raw.githubusercontent.com/user/repo/transform.star
```

```text
ZGVmIHRyYW5zZm9ybShldmVudCk6CiAgICByZXR1cm4gW2V2ZW50XQ==
```

```python
def transform(event):
    return [event]
```

--------------------------------

### Terraform Initialization and Apply

Source: https://github.com/coralogix/coralogix-aws-shipper/blob/master/examples/s3-kms-encrypted-logs/terraform/README.md

Initialize Terraform and apply the configuration to provision the KMS key and S3 bucket. Use the outputs for the shipper deployment.

```bash
cd examples/s3-kms-encrypted-logs/terraform
cp terraform.tfvars.example terraform.tfvars

terraform init
terraform apply
```

--------------------------------

### Run SAM Build

Source: https://github.com/coralogix/coralogix-aws-shipper/blob/master/examples/firehose-metrics-private-link/README.md

Execute `sam build` before `sam deploy` if the packaged Lambda binary is missing or stale. This is necessary for SAM deployments.

```bash
sam build
```

--------------------------------

### Define dynamic application or subsystem name

Source: https://github.com/coralogix/coralogix-aws-shipper/blob/master/README.md

Use the double-brace syntax to reference metadata keys for dynamic naming.

```text
{{ metadata.key | r'regex' }}
```

```text
{{ s3.bucket }}
```

```text
{{ cw.log.group }}
```

--------------------------------

### Package the Custom Resource

Source: https://github.com/coralogix/coralogix-aws-shipper/blob/master/README.md

Creates a zip archive for the GovCloud custom resource containing the index.py handler.

```bash
cd custom-resource-govcloud
zip -j custom-resource.zip index.py
cd ..
```

--------------------------------

### Deploy Coralogix AWS Shipper with SAM

Source: https://github.com/coralogix/coralogix-aws-shipper/blob/master/examples/firehose-metrics-private-link/terraform/README.md

Build and deploy the Coralogix AWS Shipper using the AWS SAM CLI. Ensure you pass the same parameters as specified in parameters.example.json.

```bash
sam build --use-container
sam deploy --guided
```

--------------------------------

### Deploy the CloudFormation Stack

Source: https://github.com/coralogix/coralogix-aws-shipper/blob/master/README.md

Deploys the CloudFormation template using the uploaded S3 artifacts and required integration parameters.

```bash
aws cloudformation deploy \
  --template-file template-govcloud.yaml \
  --stack-name coralogix-shipper \
  --region "${GOVCLOUD_REGION}" \
  --capabilities CAPABILITY_IAM CAPABILITY_NAMED_IAM \
  --parameter-overrides \
    LambdaCodeBucket="${GOVCLOUD_BUCKET}" \
    LambdaCodeKey="${PREFIX}/${VERSION}/bootstrap.zip" \
    CustomResourceCodeBucket="${GOVCLOUD_BUCKET}" \
    CustomResourceCodeKey="${PREFIX}/${VERSION}/custom-resource.zip" \
    CustomDomain="YOUR_CORALOGIX_DOMAIN" \
    ApiKey="YOUR_API_KEY_OR_SECRET_ARN" \
    ApplicationName="your-app" \
    IntegrationType="S3" \
    S3BucketName="your-log-bucket"
```

--------------------------------

### Deploy Coralogix Shipper with SAM

Source: https://github.com/coralogix/coralogix-aws-shipper/blob/master/examples/firehose-metrics-private-link/README.md

Use SAM to deploy the Coralogix AWS Shipper, especially for large templates. Ensure `sam build` is run first if the Lambda binary is missing or stale.

```bash
cd /path/to/coralogix-aws-shipper

PARAM_FILE=examples/firehose-metrics-private-link/parameters.json
OVERRIDES=$(jq -r '.[] | "\(.ParameterKey)=\(.ParameterValue)"' "$PARAM_FILE" | paste -sd' ' -)

sam deploy \
  --template-file template.yaml \
  --stack-name YOUR_STACK_NAME \
  --region eu-north-1 \
  --capabilities CAPABILITY_IAM CAPABILITY_NAMED_IAM CAPABILITY_AUTO_EXPAND \
  --resolve-s3 \
  --no-confirm-changeset \
  --parameter-overrides $OVERRIDES
```

--------------------------------

### Download the Shipper Lambda Package

Source: https://github.com/coralogix/coralogix-aws-shipper/blob/master/README.md

Downloads the Coralogix AWS Shipper zip file from the specified regional S3 bucket.

```bash
curl -fLsS -o bootstrap.zip \
  'https://coralogix-serverless-repo-eu-central-1.s3.eu-central-1.amazonaws.com/coralogix-aws-shipper.zip'
```

--------------------------------

### Upload Artifacts to GovCloud S3

Source: https://github.com/coralogix/coralogix-aws-shipper/blob/master/README.md

Uploads the bootstrap and custom resource zip files to the designated GovCloud S3 bucket.

```bash
GOVCLOUD_BUCKET="your-govcloud-artifacts-bucket"
GOVCLOUD_REGION="us-gov-west-1"
VERSION="1.4.8"
PREFIX="coralogix-aws-shipper"

aws s3 cp bootstrap.zip \
  "s3://${GOVCLOUD_BUCKET}/${PREFIX}/${VERSION}/bootstrap.zip" \
  --region "${GOVCLOUD_REGION}"

aws s3 cp custom-resource-govcloud/custom-resource.zip \
  "s3://${GOVCLOUD_BUCKET}/${PREFIX}/${VERSION}/custom-resource.zip" \
  --region "${GOVCLOUD_REGION}"
```

--------------------------------

### Deploy Coralogix Shipper with CloudFormation

Source: https://github.com/coralogix/coralogix-aws-shipper/blob/master/examples/firehose-metrics-private-link/README.md

Deploy the Coralogix AWS Shipper using pure CloudFormation by uploading the template to S3 and then updating the stack. This method does not rebuild the Rust `bootstrap` binary.

```bash
aws cloudformation update-stack \
  --stack-name YOUR_STACK_NAME \
  --region eu-north-1 \
  --template-url https://YOUR_BUCKET.s3.eu-north-1.amazonaws.com/coralogix-aws-shipper-template.yaml \
  --parameters file://examples/firehose-metrics-private-link/parameters.json \
  --capabilities CAPABILITY_IAM CAPABILITY_NAMED_IAM CAPABILITY_AUTO_EXPAND
```

--------------------------------

### Configure Starlark script via Base64 encoding

Source: https://github.com/coralogix/coralogix-aws-shipper/blob/master/README.md

Encode complex multi-line scripts into Base64 format for easier parameter embedding. The system automatically detects and decodes these strings.

```bash
# Encode your script (works on macOS and Linux)
cat transform.star | base64

# The shipper handles both wrapped and unwrapped base64 output
# Use the output as the StarlarkScript parameter
StarlarkScript: ZGVmIHRyYW5zZm9ybShldmVudCk6CiAgICByZXR1cm4gW2V2ZW50XQ==
```

--------------------------------

### Test Spoke Ingestion with AWS CLI

Source: https://github.com/coralogix/coralogix-aws-shipper/blob/master/examples/sqs-multi-account-central-lambda/terraform/README.md

After the event source mapping is created, use this AWS CLI command to send a test message to the spoke SQS queue. Ensure you use the correct profile for the spoke account.

```bash
QUEUE_URL="$(terraform output -raw additional_sqs_queue_url)"
aws sqs send-message --queue-url "$QUEUE_URL" --message-body '{"message":"spoke test"}' --profile your-spoke-profile
```

--------------------------------

### Convert CloudFormation Parameters with jq

Source: https://github.com/coralogix/coralogix-aws-shipper/blob/master/examples/firehose-metrics-private-link/terraform/README.md

This command uses `jq` to convert the JSON array of CloudFormation parameters into a space-separated key-value format suitable for the `aws cloudformation deploy` command's `--parameter-overrides` argument.

```bash
jq -r '.[] | "\(.ParameterKey)=\(.ParameterValue)"' ../parameters.json | paste -sd' ' -
# Paste as: aws cloudformation deploy ... --parameter-overrides Key1=Val1 Key2=Val2 ...
```

--------------------------------

### Configure Starlark script via S3

Source: https://github.com/coralogix/coralogix-aws-shipper/blob/master/README.md

Use an S3 URI to fetch the Starlark script. The CloudFormation template automatically manages the required permissions.

```yaml
StarlarkScript: s3://my-config-bucket/starlark/transform.star
```

--------------------------------

### Extract dynamic name from JSON

Source: https://github.com/coralogix/coralogix-aws-shipper/blob/master/README.md

Reference a JSON key directly from the log event to set the application name.

```text
{{ $.eventSource }}
```

--------------------------------

### Extract dynamic name using regex

Source: https://github.com/coralogix/coralogix-aws-shipper/blob/master/README.md

Apply a regular expression to a metadata key to capture a specific portion of the value.

```text
{{ s3.object.key | r'AWSLogs\/.+\/(.*)$' }}
```

--------------------------------

### Configure AWS Spoke Provider

Source: https://github.com/coralogix/coralogix-aws-shipper/blob/master/examples/sqs-multi-account-central-lambda/terraform/README.md

Configure the AWS provider for the spoke account, specifying the region and authentication method (e.g., profile). Ensure this is set up in your `providers.tf` file.

```hcl
provider "aws" {
  alias  = "spoke"
  region = var.spoke_region != "" ? var.spoke_region : var.aws_region
  profile = "your-spoke-profile"
}
```

--------------------------------

### Configure Starlark script via HTTP/HTTPS

Source: https://github.com/coralogix/coralogix-aws-shipper/blob/master/README.md

Host the Starlark script on any accessible HTTP or HTTPS endpoint.

```yaml
StarlarkScript: https://raw.githubusercontent.com/myorg/scripts/main/transform.star
```

--------------------------------

### Define Basic Lambda Execution Role Permission

Source: https://github.com/coralogix/coralogix-aws-shipper/blob/master/README.md

Use this YAML snippet to define the trust relationship for the Lambda execution role, allowing the service to assume the role.

```yaml
        Statement:
          - Effect: "Allow"
            Principal:
              Service: "lambda.amazonaws.com"
            Action: "sts:AssumeRole"
```

--------------------------------

### Transform plain text logs

Source: https://github.com/coralogix/coralogix-aws-shipper/blob/master/README.md

Handle unstructured log strings by checking for specific content.

```python
def transform(event):
    # event is "Feb 11 12:00:00 myhost sshd[1234]: Accepted publickey"
    if "sshd" in event:
        return [{"original": event, "service": "sshd"}]
    return [event]
```

--------------------------------

### Enrich logs with metadata

Source: https://github.com/coralogix/coralogix-aws-shipper/blob/master/README.md

Add custom fields to the log event before it is sent to Coralogix.

```python
def transform(event):
    event["processed"] = True
    event["source"] = "aws-shipper"
    return [event]
```

--------------------------------

### Transform JSON logs

Source: https://github.com/coralogix/coralogix-aws-shipper/blob/master/README.md

Access fields directly when the input event is a dictionary.

```python
def transform(event):
    # event is {"level": "INFO", "msg": "hello"}
    if event["level"] == "DEBUG":
        return []
    return [event]
```

--------------------------------

### Unnest JSON array

Source: https://github.com/coralogix/coralogix-aws-shipper/blob/master/README.md

Extract individual log entries from a batched JSON structure.

```json
{"logs": [{"msg": "log1"}, {"msg": "log2"}, {"msg": "log3"}]}
```

```python
def transform(event):
    if "logs" in event and type(event["logs"]) == "list":
        return event["logs"]
    return [event]
```

--------------------------------

### Redact sensitive values with regex

Source: https://github.com/coralogix/coralogix-aws-shipper/blob/master/README.md

Use regex functions to mask sensitive information in log messages.

```python
def transform(event):
    msg = event.get("message", "")
    if re_match(r"password=\S+", msg):
        event["message"] = re_sub(r"password=\S+", "password=***", msg)
    return [event]
```

--------------------------------

### Define transform function

Source: https://github.com/coralogix/coralogix-aws-shipper/blob/master/README.md

The required function signature for all Starlark transformation scripts.

```python
def transform(event):
    return [event]
```

--------------------------------

### Filter debug logs

Source: https://github.com/coralogix/coralogix-aws-shipper/blob/master/README.md

Exclude specific logs based on field values.

```python
def transform(event):
    if event.get("level") == "DEBUG":
        return []  # Filter out debug logs
    return [event]
```

--------------------------------

### Parse embedded JSON

Source: https://github.com/coralogix/coralogix-aws-shipper/blob/master/README.md

Use the parse_json function to handle JSON strings nested within log fields.

```python
def transform(event):
    # event is {"message": "{\"user\": \"alice\", \"action\": \"login\"}"}
    inner = parse_json(event["message"])
    return [inner]
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.