### Install and Start Tinybird CLI

Source: https://www.tinybird.co/docs/forward/guides/migrate-from-clickhouse

Install the Tinybird CLI, authenticate your account, and start the local development environment.

```bash
# Install Tinybird CLI
curl https://tinybird.co | sh

# Authenticate
tb login

# Start local development environment
tb local start
```

--------------------------------

### Confluent Cloud Setup Guide Link

Source: https://www.tinybird.co/docs/forward/ingest-data/connectors/kafka/troubleshooting

Refer to the Confluent Cloud setup guide for detailed authentication instructions when using API key authentication.

```markdown
[Confluent Cloud setup guide](/forward/guides/confluent-cloud-setup) for API key authentication
```

--------------------------------

### AWS MSK Setup Guide Link

Source: https://www.tinybird.co/docs/forward/ingest-data/connectors/kafka/troubleshooting

Refer to the AWS MSK setup guide for detailed authentication instructions when using IAM authentication.

```markdown
[AWS MSK setup guide](/forward/guides/aws-msk-setup) for IAM authentication
```

--------------------------------

### Start Tinybird Locally

Source: https://www.tinybird.co/templates/vercel-log-drains

Installs and starts Tinybird locally for development. This includes logging in and starting the development server.

```bash
curl -LsSf https://tbrd.co/fwd | sh
cd tinybird
tb local start
tb login
tb dev
token ls  # copy an admin token
```

--------------------------------

### Install and Run Next.js Application

Source: https://www.tinybird.co/templates/vercel-log-drains

Installs project dependencies and starts the Next.js development server for the log analyzer dashboard.

```bash
cd dashboard/log-analyzer
npm install
npm run dev
```

--------------------------------

### Start Docker Containers

Source: https://www.tinybird.co/docs/forward/ingest-data/connectors/kafka

Starts the necessary Docker containers for Kafka. Ensure Docker is installed and running.

```bash
docker compose up
```

--------------------------------

### Install ClickHouse Go Client

Source: https://www.tinybird.co/docs/forward/guides/connect-clickhouse-go

Add the ClickHouse Go client to your project using go mod init and go get.

```bash
go mod init your-project
go get github.com/ClickHouse/clickhouse-go/v2

```

--------------------------------

### Install ClickHouse Go Client

Source: https://www.tinybird.co/docs/forward/guides/connect-clickhouse-go

Add the ClickHouse Go client to your project using go mod init and go get.

```bash
go mod init your-project
go get github.com/ClickHouse/clickhouse-go/v2
```

--------------------------------

### Test Run Output Example

Source: https://www.tinybird.co/docs/forward/dev-reference/commands/tb-test

Illustrates the output of the 'tb test run' command, showing the build process, test environment setup, and individual test case results.

```bash
Running against Tinybird Local

» Building test environment
✓ Done!

» Running tests
* get_sets_by_theme_year.yaml
✓ get_sets_by_theme_year_2000s_range passed
✓ get_sets_by_theme_year_1990s_range passed

```

--------------------------------

### Common tb CLI Examples

Source: https://www.tinybird.co/docs/llms-full.txt

Examples demonstrating how to access help, check the version, initialize a project in development mode, or deploy to the cloud.

```bash
tb --help
tb --version
tb init --type cli --dev-mode manual
tb --cloud deploy
```

--------------------------------

### Start Local Server and Build Project (Python SDK)

Source: https://www.tinybird.co/docs/forward/development-workflow/examples

Start the local development server and build the project using uv commands with the Python SDK.

```shell
tb local start --daemon
uv run tinybird build
```

--------------------------------

### File Structure Example with Daily Partitioning

Source: https://www.tinybird.co/docs/forward/copy-export-data/s3-sink

Demonstrates file structure created by setting the file template to include daily partitioning.

```text
Invoices
├── dt=2023-07-07
│ └── H23.csv
│ └── H22.csv
│ └── H21.csv
│ └── ...
├── dt=2023-07-06
│ └── H23.csv
│ └── H22.csv
```

--------------------------------

### Initialize Tinybird Project with Specific Options

Source: https://www.tinybird.co/docs/forward/dev-reference/commands/tb-init

Example demonstrating initialization with a CLI project type, manual development mode, current folder, and GitHub CI/CD templates.

```bash
tb init --type cli --dev-mode manual --folder . --cicd github
```

--------------------------------

### Using toYYYYMM for Monthly Partitions

Source: https://www.tinybird.co/docs/sql-reference/engines/mergetree

This example demonstrates using the `toYYYYMM` function to create monthly partitions, which is recommended for better query performance.

```sql
toYYYYMM(date_column)
```

--------------------------------

### Python Example: Fetching Data with Requests

Source: https://www.tinybird.co/docs/forward/guides/consume-apis-in-a-notebook

Demonstrates how to fetch data from an API using the 'requests' library in Python. Ensure you have the library installed (`pip install requests`).

```python
import requests

url = "https://api.example.com/data"
headers = {
    "Authorization": "Bearer YOUR_API_KEY"
}

response = requests.get(url, headers=headers)

if response.status_code == 200:
    data = response.json()
    print(data)
else:
    print(f"Error: {response.status_code}")
```

--------------------------------

### Truncate to Start of Month

Source: https://www.tinybird.co/docs/sql-reference/functions/date-time-functions

Shows how to get the first day of the month for a specified date or datetime.

```sql
SELECT toStartOfMonth(toDateTime('2023-04-21 10:20:30'))
```

--------------------------------

### Initialize Python Project and Add SDK

Source: https://www.tinybird.co/docs/forward/quickstarts/python-sdk

Create a new directory, initialize a Python SDK project using uv, and add the necessary Tinybird and dotenv packages.

```bash
mkdir tinybird-python-quickstart
cd tinybird-python-quickstart
git init

uv init --name tinybird-sdk-python-sample --python 3.13
uv add tinybird-sdk python-dotenv

uv run tinybird init --folder src/tinybird/
```

--------------------------------

### Build Output Example

Source: https://www.tinybird.co/docs/forward/development-workflow/local-development

Example output after a successful project build, showing created resources and build completion time.

```shell
» Building project...
✓ datasources/user_actions.datasource created
✓ endpoints/user_actions_line_chart.pipe created
✓ Build completed in 0.2s

```

--------------------------------

### Get MSK Cluster ARN

Source: https://www.tinybird.co/docs/forward/guides/aws-msk-setup

Example of an AWS MSK Cluster ARN. This is required for IAM policy configuration.

```text
arn:aws:kafka:us-east-1:123456789012:cluster/example-cluster/abc123-def456-789
```

--------------------------------

### Initialize a new Tinybird project with specific options

Source: https://www.tinybird.co/docs/forward/dev-reference/commands/tb-init

Example demonstrating how to initialize a new Tinybird project with a CLI type, manual development mode, and specifying the current folder.

```shell
tb init --type cli --dev-mode manual --folder .
```

--------------------------------

### URL parameter authenticated request

Source: https://www.tinybird.co/docs/api-reference

Example of an authenticated GET request to the Tinybird API including a query parameter.

```shell
curl -X GET \n"https://<your_host>/v0/sql?q=SELECT * FROM <pipe>&token=<token>"
```

--------------------------------

### Recommended Partitioning Functions

Source: https://www.tinybird.co/docs/sql-reference/engines/mergetree

Provides examples of recommended functions for creating monthly or yearly partitions to avoid performance degradation.

```sql
toYYYYMM(date_column)
```

```sql
toYear(date_column)
```

--------------------------------

### Get Environment Variable

Source: https://www.tinybird.co/docs/api-reference/environment-variables-api

Example of how to retrieve an environment variable using its name. Requires an admin token for authorization.

```bash
curl \"https://$TB_HOST/v0/variables/test_password\" \
-H \"Authorization: Bearer <ADMIN token>\"
```

--------------------------------

### Example of Partitioning by Year

Source: https://www.tinybird.co/docs/sql-reference/engines/mergetree

Demonstrates how to partition data by year using the toYear() function. Avoid overly granular partition keys as they can severely degrade write performance.

```sql
PARTITION BY toYear(date_column)
```

--------------------------------

### Start Docker Compose with Named AWS Profile Credentials

Source: https://www.tinybird.co/docs/forward/ingest-data/connectors/s3

Use the AWS CLI to export credentials from a named profile, then start your Tinybird Local Docker Compose setup. This method is useful for managing multiple AWS configurations.

```bash
# Export credentials from a named profile
eval "$(aws configure export-credentials --profile my-profile --format env)"

# Start Docker Compose
docker compose up -d

```

--------------------------------

### Run the Demo App

Source: https://www.tinybird.co/docs/forward/guides/consume-apis-nextjs

Execute this command in your terminal to start the Next.js development server locally.

```shell
npm run dev

```

--------------------------------

### Install Tinybird SDK in Repository

Source: https://www.tinybird.co/docs/forward/quickstarts/typescript-sdk

Add the Tinybird SDK to your project's dependencies to ensure commands function as shown in examples.

```bash
pnpm add @tinybirdco/sdk
```

--------------------------------

### MergeTree Engine Configuration Example

Source: https://www.tinybird.co/docs/sql-reference/engines/mergetree

This example demonstrates how to configure a MergeTree table with schema, sorting key, partition key, primary key, TTL, and settings.

```sql
SCHEMA >
        `event_date` Date `json:$.date`,
        `event_type` LowCardinality(String) `json:$.event_type`,
        `user_id` UInt64 `json:$.user_id`,
        `payload` JSON `json:$.payload`
    
    [ENGINE "MergeTree"]
    [ENGINE_SORTING_KEY "event_type, event_date, user_id"]
    [ENGINE_PARTITION_KEY "toYYYYMM(event_date)"]
    [ENGINE_PRIMARY_KEY "event_type, event_date"]
    [ENGINE_TTL "event_date + INTERVAL 30 DAY"]
    [ENGINE_SETTINGS index_granularity=8192]
```

--------------------------------

### Bootstrap Servers Configuration Example

Source: https://www.tinybird.co/docs/forward/ingest-data/connectors/kafka

Example of configuring multiple bootstrap servers for Kafka connection. Use comma-separated values for multiple brokers.

```plaintext
KAFKA_BOOTSTRAP_SERVERS broker1:9092,broker2:9092,broker3:9092
```

--------------------------------

### URL Parameter Authenticated Request

Source: https://www.tinybird.co/docs/api-reference

Example of an authenticated GET request to the SQL API endpoint, demonstrating the use of URL parameters.

```APIDOC
## GET /v0/sql

### Description
Executes a SQL query against Tinybird.

### Method
GET

### Endpoint
/v0/sql

### Parameters
#### Query Parameters
- **q** (string) - Required - The SQL query to execute.
- **token** (string) - Required - The authentication token.

### Request Example
```shell
curl -X GET "https://<your_host>/v0/sql?q=SELECT * FROM <pipe>&token=<token>"
```

### Response
#### Success Response (200)
(Response structure not detailed in source)

#### Response Example
(Response example not detailed in source)
```

--------------------------------

### Create a preview environment with Tinybird CLI

Source: https://www.tinybird.co/docs/forward/dev-reference/commands/python-sdk-cli

Create a preview environment and deploy project resources. Use `--dry-run` to see changes without applying them, `--check` for validation, and `--name` to specify a custom name for the preview environment.

```bash
tinybird preview
```

```bash
tinybird preview --dry-run
```

```bash
tinybird preview --check
```

```bash
tinybird preview --name pr_482
```

--------------------------------

### Text: Results of toStartOfWeek examples

Source: https://www.tinybird.co/docs/sql-reference/functions/date-time-functions

Displays the output from the `toStartOfWeek` SQL query, showing the calculated start of the week for each input date and mode.

```text
Row 1:
──────
toStartOfWeek(toDateTime('2023-04-21 10:20:30')): 2023-04-16
toStartOfWeek(toDateTime('2023-04-21 10:20:30'), 1): 2023-04-17
toStartOfWeek(toDate('2023-04-24')): 2023-04-23
toStartOfWeek(toDate('2023-04-24'), 1): 2023-04-24
```

--------------------------------

### Define an S3 Connection

Source: https://www.tinybird.co/docs/forward/dev-reference/python-sdk-resources

Use `define_s3_connection` to configure a connection to an S3 bucket. This is a partial example showing the start of the S3 connection definition.

```python
from tinybird_sdk import (
 define_dynamodb_connection,
 define_gcs_connection,
 define_kafka_connection,
 define_s3_connection,
 secret,
)

landing_s3 = define_s3_connection("landing_s3", {

```

--------------------------------

### Start Local Server and Build Project (TypeScript SDK)

Source: https://www.tinybird.co/docs/forward/development-workflow/examples

Start the local development server and build the project using npm commands with the TypeScript SDK.

```shell
tb local start --daemon
npx tinybird build --local
```

--------------------------------

### Get Current Time and Sleep Example

Source: https://www.tinybird.co/docs/sql-reference/functions/date-time-functions

Demonstrates the use of `now()`, `nowInBlock()`, and `sleep()` functions. The `SETTINGS max_block_size = 1` clause influences how `nowInBlock()` behaves.

```sql
SELECT
    now(),
    nowInBlock(),
    sleep(1)
FROM numbers(3)
SETTINGS max_block_size = 1
FORMAT PrettyCompactMonoBlock
```

--------------------------------

### Example API Endpoint with Query Parameters

Source: https://www.tinybird.co/docs/forward/query-data/query-parameters

This example shows how to call an API endpoint with query parameters like 'lim'. Replace 'your_token' with your actual API token.

```shell
curl "https://api.tinybird.co/v0/pipes/tr_pipe.json?lim=20&token=<your_token>"

```

--------------------------------

### Get Environment Variable Example

Source: https://www.tinybird.co/docs/api-reference/environment-variables-api

Use this cURL command to retrieve the value of a specific environment variable. Replace placeholders with your actual host and token.

```shell
curl \
 -X GET \"https://$TB_HOST/v0/variables/test_password\" \
 -H \"Authorization: Bearer <ADMIN token>\"
```

--------------------------------

### Example Usage of Global Options

Source: https://www.tinybird.co/docs/forward/dev-reference/commands/global-options

Demonstrates how to use global options like `--host` before a command. This sets the API endpoint for the command.

```bash
tb --host https://api.tinybird.co workspace ls
```

--------------------------------

### Example of Comment Handling by tb fmt

Source: https://www.tinybird.co/docs/llms-full.txt

Illustrates how 'tb fmt' handles comments. It removes comments starting with '#', but preserves comments within '%{% comment %}' blocks.

```sql
% comment this is a comment and fmt keeps it %%

SELECT
  % comment this is another comment and fmt keeps it %
  count() c
FROM stock_prices_1m
```

--------------------------------

### Pipe File Instructions - Basic

Source: https://www.tinybird.co/docs/forward/dev-reference/datafiles/pipe-files

Available instructions for .pipe files, including '%' for templating, 'DESCRIPTION' for metadata, and 'TAGS' for categorization.

```pipe
%
```

```pipe
DESCRIPTION <markdown_string>
```

```pipe
TAGS <tag_names>
```

--------------------------------

### Get Day of Week with toDayOfWeek

Source: https://www.tinybird.co/docs/sql-reference/functions/date-time-functions

Extracts the day of the week from a datetime value. The second argument specifies the starting day of the week (e.g., 1 for Monday).

```sql
SELECT
toDayOfWeek(toDateTime('2023-04-21')),
toDayOfWeek(toDateTime('2023-04-21'), 1)

```

--------------------------------

### Create Token Help (CLI)

Source: https://www.tinybird.co/docs/administration/auth-tokens

Access help information for creating static tokens using the CLI.

```bash
tb token create static --help
```

--------------------------------

### Local Tinybird Development Setup

Source: https://www.tinybird.co/docs/forward/development-workflow/examples

Start the Tinybird local development server and build the project locally. This is the initial step for developing and testing changes before deploying.

```shell
tb local start --daemon
npx tinybird build --local

```

--------------------------------

### ClickHouse Setup Plan

Source: https://www.tinybird.co/

This snippet shows a basic setup plan for ClickHouse, including comments for self-hosting.

```markdown
# Self-hosting ClickHouse
```

--------------------------------

### Get Positions of Maximum Interval Intersections

Source: https://www.tinybird.co/docs/sql-reference/functions/aggregate-functions

Use maxIntersectionsPosition to identify the starting points where the maximum number of intervals overlap. It also skips NULL or 0 values.

```sql
WITH intervals AS (
 SELECT arrayJoin([(1, 5), (3, 7), (6, 10), (2, 4)]) AS interval_pair
)
SELECT maxIntersectionsPosition(interval_pair.1, interval_pair.2) FROM intervals

```

--------------------------------

### Start development server with Tinybird CLI

Source: https://www.tinybird.co/docs/forward/dev-reference/commands/python-sdk-cli

Build the server-side components and watch for file changes during development. Use `--with-connections` to include connections or `--no-connections` to exclude them.

```bash
tinybird dev
```

```bash
tinybird dev --with-connections
```

```bash
tinybird dev --no-connections
```

--------------------------------

### Example Login Command

Source: https://www.tinybird.co/docs/forward/guides/add-self-managed-region-cli

An example of the `tb login` command with a specific host URL.

```bash
tb login --host https://<host>\n
```

--------------------------------

### Run a SELECT COUNT(*) Query

Source: https://www.tinybird.co/docs/llms-full.txt

Execute a simple SELECT COUNT(*) statement to get the total number of rows in a data source. This is a basic example of querying data.

```bash
tb sql "SELECT count(*) from tinybird.endpoint_errors"
```

--------------------------------

### Get MSK Bootstrap Servers

Source: https://www.tinybird.co/docs/forward/guides/aws-msk-setup

Example of a bootstrap server string for an AWS MSK cluster. Use the port that matches your authentication method (e.g., 9098 for SASL/IAM).

```text
b-1.example-cluster.abc123.c2.kafka.us-east-1.amazonaws.com:9098,b-2.example-cluster.abc123.c2.kafka.us-east-1.amazonaws.com:9098
```

--------------------------------

### Create Kafka Topic and Send Sample Messages

Source: https://www.tinybird.co/docs/forward/ingest-data/connectors/kafka/guides/local-development

Commands to create a Kafka topic and produce sample messages for unit testing.

```bash
# Create a test topic
docker exec -it broker /opt/kafka/bin/kafka-topics.sh --create \
  --topic test-events \
  --bootstrap-server localhost:9092

# Send sample messages
cat <<EOF | docker exec -i broker /opt/kafka/bin/kafka-console-producer.sh \
  --topic test-events \
  --bootstrap-server localhost:9092
{"user_id": "123", "event": "click", "timestamp": "2024-01-01T00:00:00Z"}
{"user_id": "456", "event": "view", "timestamp": "2024-01-01T00:01:00Z"}
{"user_id": "123", "event": "purchase", "timestamp": "2024-01-01T00:02:00Z"}
EOF
```

--------------------------------

### Connecting to Tinybird ClickHouse with @clickhouse/client

Source: https://www.tinybird.co/docs/forward/guides/integrate-with-open-source

Example of using the @clickhouse/client library in JavaScript to connect to Tinybird's ClickHouse HTTP interface and execute a query. Ensure you have the library installed.

```javascript
// Using @clickhouse/client (HTTP interface)
import { createClient } from '@clickhouse/client'

const client = createClient({
 host: 'https://clickhouse.<region>.tinybird.co',
 username: 'optional_workspace_name', // optional
 password: 'tb_your_read_token'
})

const resultSet = await client.query({
 query: 'SELECT * FROM user_events LIMIT 10',
 format: 'JSON'
})

const data = await resultSet.json()
console.log(data)
```

--------------------------------

### First Value Example with and without RESPECT NULLS

Source: https://www.tinybird.co/docs/sql-reference/functions/aggregate-functions

Shows how to use the first_value function to get the first non-null value and how to include NULLs using the RESPECT NULLS modifier.

```sql
WITH cte AS (SELECT arrayJoin([NULL, 'Amsterdam', 'New York', 'Tokyo', 'Valencia', NULL]) as city)
SELECT first_value(city), first_value(city) RESPECT NULLS FROM cte

```

--------------------------------

### Successful response for getting pipe information

Source: https://www.tinybird.co/docs/api-reference/pipe-api

Example of a successful JSON response when retrieving pipe information, detailing the pipe's ID, name, and its pipeline structure with nodes.

```json
{
    "id": "t_bd1c62b5e67142bd9bf9a7f113a2b6ea",
    "name": "events_pipe",
    "pipeline": {
        "nodes": [{
            "name": "events_ds_0"
            "sql": "select * from events_ds_log__raw",
            "materialized": false
        }, {
            "name": "events_ds",
            "sql": "select * from events_ds_0 where valid = 1",
            "materialized": false
        }]
    }
}
```