### Install S2 CLI from Release Binaries Source: https://s2.dev/docs/quickstart Installs the S2 CLI by downloading and executing a script from release binaries. This method is versatile and works across different architectures and operating systems. It adds the binary to `~/.s2/bin` and updates the shell profile. ```bash curl -fsSL s2.dev/install.sh | bash ``` -------------------------------- ### Install S2 CLI using Cargo Source: https://s2.dev/docs/quickstart Installs the S2 CLI using the Cargo package manager, requiring Rust and Cargo to be pre-installed. This method is suitable for any system with Rust development tools. ```bash cargo install --locked s2-cli ``` -------------------------------- ### Install S2 CLI using Homebrew Source: https://s2.dev/docs/quickstart Installs the S2 CLI on macOS and Linux systems that have Homebrew package manager installed. This is a straightforward installation method for supported operating systems. ```bash brew install s2-streamstore/s2/s2 ``` -------------------------------- ### Create and Append Data to an S2 Stream Source: https://s2.dev/docs/quickstart Creates a new stream within a specified basin and demonstrates how to append data to it. This example uses `nc` to pipe data from a telnet stream into the S2 stream, simulating real-time data ingestion. ```bash s2 create-stream s2://${BASIN}/starwars export BASIN="" # Set the basin name for new terminal nc starwars.s2.dev 23 | s2 append s2://${BASIN}/starwars ``` -------------------------------- ### Configure S2 CLI Access Token Source: https://s2.dev/docs/quickstart Persists your S2 access token for the CLI. This command is essential for authenticating your operations with the S2 service. Replace `` with your actual token. ```bash s2 config set --access-token ``` -------------------------------- ### Initialize S2 Client and Get Stream (TypeScript) Source: https://s2.dev/docs/sdk/languages Demonstrates how to initialize the S2 client using an access token and then access a specific basin and stream. This is the starting point for most stream operations. ```typescript import { S2 } from "@s2-dev/streamstore"; const client = new S2({ accessToken: process.env.S2_ACCESS_TOKEN!, }); const basin = client.basin("my-basin"); const stream = basin.stream("my-stream"); ``` -------------------------------- ### Install Bento CLI Source: https://s2.dev/docs/integrations/bento This command installs the latest version of the Bento CLI using a curl script. It downloads and executes the installation script, making the Bento CLI available on your system. Ensure your system has curl installed. ```bash curl -Lsf https://warpstreamlabs.github.io/bento/sh/install | bash ``` -------------------------------- ### Retrieve Metrics (Rust) Source: https://s2.dev/docs/sdk/account-and-basins Illustrates fetching account, basin, and stream metrics using the s2 Rust SDK. Examples cover account-level active basins over 30 days, basin-level storage hourly, and stream-level storage by minute. Requires the s2 Rust crate and proper async setup. ```rust use std::time::{Duration, SystemTime}; let now = SystemTime::now(); let thirty_days_ago = now - Duration::from_secs(30 * 24 * 3600); let six_hours_ago = now - Duration::from_secs(6 * 3600); let hour_ago = now - Duration::from_secs(3600); let account_metrics = s2 .get_account_metrics(GetAccountMetricsInput::active_basins(thirty_days_ago, now)) .await?; let basin_metrics = s2 .get_basin_metrics( GetBasinMetricsInput::storage("events".parse()?, six_hours_ago, now) .with_interval(TimeseriesInterval::Hour), ) .await?; let stream_metrics = s2 .get_stream_metrics( GetStreamMetricsInput::storage("events".parse()?, "user-actions".parse()?, hour_ago, now) .with_interval(TimeseriesInterval::Minute), ) .await?; ``` -------------------------------- ### Estimate End-to-End Latency with S2 Ping Source: https://s2.dev/docs/quickstart Creates a stream with the 'express' storage class and then runs the `s2 ping` command to estimate the end-to-end latency for data transfer to that stream. This helps in understanding performance characteristics. ```bash s2 create-stream s2://${BASIN}/test/express --storage-class express s2 ping s2://${BASIN}/test/express ``` -------------------------------- ### S2S Session Setup Source: https://s2.dev/docs/api/records/overview Details on how to establish an S2S session, including required headers and success response. ```APIDOC ## S2S Session Setup ### Description This section describes the process of initiating an S2S session, including the necessary request headers and the expected success response. ### Method POST ### Endpoint `https://{basin}.b.aws.s2.dev/v1/streams/{stream_name}/session` ### Headers - **Content-Type**: `s2s/proto` - Signals that a session is being requested. - **Accept-Encoding**: Specifies supported compression algorithms (e.g., `zstd`, `gzip`). ### Response #### Success Response (200 OK) - Establishes the session. ``` -------------------------------- ### Create and Verify an S2 Basin Source: https://s2.dev/docs/quickstart Creates a new basin, which is a fundamental organizational unit in S2. It also includes a command to list basins and verify the newly created one is active. Basin names have specific length and character requirements. ```bash export BASIN="" s2 create-basin ${BASIN} s2 list-basins ``` -------------------------------- ### Install S2 CLI using Release Binaries (curl) Source: https://s2.dev/docs/index Downloads and installs the S2 CLI using a curl script for Linux and macOS. The script places the binary in `~/.s2/bin` and updates the shell profile. ```bash curl -fsSL s2.dev/install.sh | bash ``` -------------------------------- ### Install S2 CLI using Cargo Source: https://s2.dev/docs/index Installs the S2 CLI on any system with Rust and Cargo installed. This method requires a Rust development environment to be present. ```bash cargo install --locked s2-cli ``` -------------------------------- ### Verify Bento S2 Plugin Support Source: https://s2.dev/docs/integrations/bento This command checks if the installed Bento CLI supports the S2 plugin. It lists all available Bento plugins and filters the output to show only those ending with 's2'. This ensures compatibility for S2 stream operations. ```bash bento list | grep 's2$' ``` -------------------------------- ### Install S2 CLI using Homebrew Source: https://s2.dev/docs/index Installs the S2 CLI on macOS and Linux systems with Homebrew. This is a straightforward installation method for users who have Homebrew set up. ```bash brew install s2-streamstore/s2/s2 ``` -------------------------------- ### Install @s2-dev/resumable-stream Package Source: https://s2.dev/docs/integrations/chat-sdk Installs the necessary npm package for enabling resumable streams in your AI chatbot. This is the first step to integrating S2.dev's backend capabilities. ```bash npm install @s2-dev/resumable-stream ``` -------------------------------- ### Read Data from an S2 Stream Source: https://s2.dev/docs/quickstart Initiates a read session to consume data from an S2 stream. This command is typically run in a separate terminal from the append session to observe the data in motion. ```bash s2 read s2://${BASIN}/starwars 2> /dev/null ``` -------------------------------- ### S2 Example Job Data Structure Source: https://s2.dev/docs/integrations/flink An example data structure representing per-product features derived from user actions in an e-commerce use case. This JSON output illustrates the format of computed features, specifically 'top_queries' for a given product. ```json [ 112140, { "top_queries": [ "mid-century modern", "contemporary living", "fashion", "home and living", "kitchen" ] } ] ``` -------------------------------- ### Run woof-pipeline with Bento Source: https://s2.dev/docs/integrations/bento Executes the configured woof pipeline using the 'bento' command-line tool. This command initiates the data processing defined in the 'woof-pipeline.yml' file. ```bash bento -c woof-pipeline.yml ``` -------------------------------- ### GET /basins Source: https://s2.dev/docs/api/basins/list Retrieves a list of basins within your account. You can filter basins by a prefix, paginate results using `start_after`, and control the number of results with `limit`. ```APIDOC ## GET /basins ### Description Retrieves a list of basins. Supports filtering by prefix, pagination, and limiting results. ### Method GET ### Endpoint /basins ### Parameters #### Query Parameters - **prefix** (string) - Optional - Filter to basins whose names begin with this prefix. - **start_after** (string) - Optional - Filter to basins whose names lexicographically start after this string. It must be greater than or equal to the `prefix` if specified. - **limit** (integer) - Optional - Number of results, up to a maximum of 1000. Defaults to 1000. ### Request Example ```json { "example": "" } ``` ### Response #### Success Response (200) - **basins** (array) - Matching basins. Each basin object contains `name`, `state`, and optionally `scope`. - **has_more** (boolean) - Indicates that there are more basins that match the criteria. #### Response Example ```json { "basins": [ { "name": "example-basin-name", "scope": "aws:us-east-1", "state": "active" } ], "has_more": true } ``` #### Error Response (400, 403, 408) - **code** (string) - Error code. - **message** (string) - Error message. #### Error Response Example ```json { "code": "invalid_parameter", "message": "Invalid parameter provided." } ``` ``` -------------------------------- ### Initialize S2 Client and Get Stream (Go) Source: https://s2.dev/docs/sdk/languages Shows how to create an S2 client from environment variables and then obtain a reference to a specific stream within a basin. This is a common pattern for Go applications interacting with S2. ```go package main import ( "fmt" "github.com/s2-streamstore/s2-sdk-go/s2" ) func main() { client := s2.NewFromEnvironment(nil) basin := client.Basin("my-basin") stream := basin.Stream("my-stream") fmt.Printf("Created client for stream: %+v\n", stream) } ``` -------------------------------- ### Get Stream Storage Metric at Current Moment Source: https://s2.dev/docs/api/metrics/overview Fetches the storage metric for a specific stream at the current moment. This example uses `curl` to make a GET request and pipes the output to `jq` for pretty-printing the JSON response. It requires specifying the basin, stream, metric set ('storage'), and current start and end timestamps. ```bash # Get storage at current moment for a stream. curl \ --request GET \ --url "https://aws.s2.dev/v1/metrics/${BASIN}/${STREAM}?set=storage&start=$(date +%s)&end=$(date +%s)" \ --header "Authorization: Bearer ${S2_ACCESS_TOKEN}" \ | jq ``` -------------------------------- ### View Products Table in Playground Database Source: https://s2.dev/docs/integrations/sequin Execute a bash command to connect to the running Postgres container within the Sequin Docker Compose setup. This command queries the 'sequin_playground' database to display the contents of the 'products' table, showing columns like id, name, and price. ```bash docker exec -i sequin-sequin_postgres-1 \ psql -U postgres -d sequin_playground -c \ "select id, name, price from products;" ``` -------------------------------- ### GET /metrics Source: https://s2.dev/docs/api/metrics/account Retrieves account-level metrics. You can specify the metric set, start and end timestamps, and the aggregation interval. ```APIDOC ## GET /metrics ### Description Retrieves account-level metrics. You can specify the metric set, start and end timestamps, and the aggregation interval. ### Method GET ### Endpoint /metrics ### Parameters #### Query Parameters - **set** (string) - Required - Metric set to return. Must be one of: `active-basins`, `account-ops`. - **start** (integer) - Optional - Start timestamp as Unix epoch seconds. - **end** (integer) - Optional - End timestamp as Unix epoch seconds. - **interval** (string) - Optional - Interval to aggregate over for timeseries metric sets. Must be one of: `minute`, `hour`, `day`. ### Response #### Success Response (200) - **values** (array) - Metrics comprising the set. Each item can be a `ScalarMetric`, `AccumulationMetric`, `GaugeMetric`, or `LabelMetric`. #### Error Response (400, 403, 408) - **code** (string) - Error code. - **message** (string) - Error message. ### Response Example ```json { "values": [ { "scalar": { "name": "active-basins", "unit": "count", "value": 5 } } ] } ``` ``` -------------------------------- ### Fetch Navigation and Other Pages Source: https://s2.dev/docs/api/streams/create-or-reconfigure Instructions on how to fetch navigation and other pages from the documentation. ```APIDOC ## Fetch Navigation and Other Pages ### Description To find navigation and other pages in this documentation, fetch the `llms.txt` file from the specified URL. ### Method GET ### Endpoint `https://s2.dev/docs/llms.txt` ### Parameters N/A ### Request Example ```bash curl https://s2.dev/docs/llms.txt ``` ### Response #### Success Response (200) - **Content** (string) - The content of the `llms.txt` file, which may contain navigation links and other page information. #### Response Example (Example content depends on the actual file content) ``` -------------------------------- ### Create S2 Client and Access Stream (Go) Source: https://s2.dev/docs/sdk/overview Shows how to create an S2 client from environment variables and then access a specific stream within a basin. This example uses the s2-sdk-go package and assumes the S2_ACCESS_TOKEN is set in the environment. ```go package main import ( "fmt" "github.com/s2-streamstore/s2-sdk-go/s2" ) func main() { client := s2.NewFromEnvironment(nil) basin := client.Basin("my-basin") stream := basin.Stream("my-stream") fmt.Printf("Created client for stream: %+v\n", stream) } ``` -------------------------------- ### GET /websites/s2_dev/metrics/storage Source: https://s2.dev/docs/api/metrics/basin Retrieves the gauge of cumulative stored bytes, per hour, aggregated across all streams within the basin. Requires start and end timestamps. ```APIDOC ## GET /websites/s2_dev/metrics/storage ### Description Retrieves the gauge of cumulative stored bytes, per hour, aggregated across all streams within the basin. ### Method GET ### Endpoint /websites/s2_dev/metrics/storage ### Parameters #### Query Parameters - **start** (timestamp) - Required - The start timestamp for the metric retrieval. - **end** (timestamp) - Required - The end timestamp for the metric retrieval. ### Response #### Success Response (200) - **gauge** (number) - The cumulative stored bytes. #### Response Example ```json { "gauge": 1234567890 } ``` ``` -------------------------------- ### Retrieve Metrics (Go) Source: https://s2.dev/docs/sdk/account-and-basins Shows how to fetch account, basin, and stream metrics using the s2 Go client. This includes examples for active basins over 30 days, hourly storage usage for a basin, and minute-level storage for a stream. Requires the s2 Go SDK and a configured client. ```go now := time.Now().Unix() thirtyDaysAgo := now - 30*24*3600 sixHoursAgo := now - 6*3600 hourAgo := now - 3600 accountMetrics, _ := client.Metrics.Account(ctx, &s2.AccountMetricsArgs{ Set: s2.AccountMetricSetActiveBasins, Start: &thirtyDaysAgo, End: &now, }) hourInterval := s2.TimeseriesIntervalHour basinMetrics, _ := client.Metrics.Basin(ctx, &s2.BasinMetricsArgs{ Basin: "events", Set: s2.BasinMetricSetStorage, Start: &sixHoursAgo, End: &now, Interval: &hourInterval, }) minuteInterval := s2.TimeseriesIntervalMinute streamMetrics, _ := client.Metrics.Stream(ctx, &s2.StreamMetricsArgs{ Basin: "events", Stream: "user-actions", Set: s2.StreamMetricSetStorage, Start: &hourAgo, End: &now, Interval: &minuteInterval, }) ``` -------------------------------- ### Get Account-Level Active Basins Metric Set Source: https://s2.dev/docs/api/metrics/overview Retrieves the account-level 'active-basins' metric set using a GET request. Requires specifying the metric set, start and end timestamps, and an authorization token. The time period should not exceed 30 days. ```bash curl \ --request GET \ --url "https://aws.s2.dev/v1/metrics?set=active-basins&start=${START}&end=${END}" \ --header "Authorization: Bearer ${S2_ACCESS_TOKEN}" ``` -------------------------------- ### GET /websites/s2_dev/metrics/read-throughput Source: https://s2.dev/docs/api/metrics/basin Retrieves the accumulation of read throughput in bytes, aggregated across all streams within the basin. Requires start and end timestamps and accepts an optional interval. ```APIDOC ## GET /websites/s2_dev/metrics/read-throughput ### Description Retrieves the accumulation of read throughput in bytes, aggregated across all streams within the basin. This metric set does not currently separate egress traffic by client origin. ### Method GET ### Endpoint /websites/s2_dev/metrics/read-throughput ### Parameters #### Query Parameters - **start** (timestamp) - Required - The start timestamp for the metric retrieval. - **end** (timestamp) - Required - The end timestamp for the metric retrieval. - **interval** (string) - Optional - The aggregation interval for the metrics ('minute', 'hour', 'day'). Defaults to 'hour'. ### Response #### Success Response (200) - **accumulation** (number) - The total read throughput in bytes. #### Response Example ```json { "accumulation": 5000000000 } ``` ``` -------------------------------- ### Fetch LLM Documentation Source: https://s2.dev/docs/api/basins/get-config Instructions on how to fetch the LLM documentation file. ```APIDOC ## Fetch LLM Documentation ### Description To find navigation and other pages in this documentation, fetch the `llms.txt` file from the specified URL. ### Method GET ### Endpoint https://s2.dev/docs/llms.txt ### Response #### Success Response (200) - **Content**: Text file containing LLM documentation and navigation information. #### Response Example ``` # Content of llms.txt ... ``` ``` -------------------------------- ### GET /websites/s2_dev/metrics/read-ops Source: https://s2.dev/docs/api/metrics/basin Returns accumulation metrics of read operations (unary and streaming) aggregated across all streams within the basin. Requires start and end timestamps and accepts an optional interval. ```APIDOC ## GET /websites/s2_dev/metrics/read-ops ### Description Returns two accumulation metrics of read operations, aggregated across all streams within the basin: one for unary reads, and another for streaming reads. A streaming read operation is defined as up to one minute of a streaming read response, from a single stream, using a single TCP connection. A unary read operation is a single read request that does not return a streaming body. ### Method GET ### Endpoint /websites/s2_dev/metrics/read-ops ### Parameters #### Query Parameters - **start** (timestamp) - Required - The start timestamp for the metric retrieval. - **end** (timestamp) - Required - The end timestamp for the metric retrieval. - **interval** (string) - Optional - The aggregation interval for the metrics ('minute', 'hour', 'day'). Defaults to 'hour'. ### Response #### Success Response (200) - **accumulation** (object) - An object containing read operations. - **unary** (number) - The count of unary read operations. - **streaming** (number) - The count of streaming read operations. #### Response Example ```json { "accumulation": { "unary": 2000, "streaming": 750 } } ``` ``` -------------------------------- ### Initialize S2 Client and Get Stream (Rust) Source: https://s2.dev/docs/sdk/languages Illustrates the initialization of the S2 client in Rust, requiring an access token and parsing string identifiers for basins and streams. Error handling is included via the '?' operator. ```rust use s2_sdk::{S2, types::S2Config}; let client = S2::new(S2Config::new(std::env::var("S2_ACCESS_TOKEN")?))?; let basin = client.basin("my-basin".parse()?); let stream = basin.stream("my-stream".parse()?); ``` -------------------------------- ### GET /websites/s2_dev/metrics/append-ops Source: https://s2.dev/docs/api/metrics/basin Returns the accumulation of append operations aggregated across all streams within the basin, grouped by storage class. Requires start and end timestamps and accepts an optional interval. ```APIDOC ## GET /websites/s2_dev/metrics/append-ops ### Description Returns the accumulation of append operations aggregated across all streams within the basin, grouped by storage class. An append operation is defined as one minute of appends to a single stream, using a single TCP connection. ### Method GET ### Endpoint /websites/s2_dev/metrics/append-ops ### Parameters #### Query Parameters - **start** (timestamp) - Required - The start timestamp for the metric retrieval. - **end** (timestamp) - Required - The end timestamp for the metric retrieval. - **interval** (string) - Optional - The aggregation interval for the metrics ('minute', 'hour', 'day'). Defaults to 'hour'. ### Response #### Success Response (200) - **accumulation** (object) - An object containing append operations grouped by storage class. - **storage_class_name** (number) - The count of append operations for a specific storage class. #### Response Example ```json { "accumulation": { "express": 500, "standard": 1500 } } ``` ``` -------------------------------- ### GET /websites/s2_dev/metrics/basin-ops Source: https://s2.dev/docs/api/metrics/basin Retrieves a set of accumulation metrics, one per basin operation, with non-zero values within the specified period. Requires start and end timestamps and accepts an optional interval. ```APIDOC ## GET /websites/s2_dev/metrics/basin-ops ### Description Retrieves a set of accumulation metrics, one per basin operation. Each basin-level request is a single operation, and this endpoint will return as many timeseries as there are distinct basin operations with non-zero values within the specified period. ### Method GET ### Endpoint /websites/s2_dev/metrics/basin-ops ### Parameters #### Query Parameters - **start** (timestamp) - Required - The start timestamp for the metric retrieval. - **end** (timestamp) - Required - The end timestamp for the metric retrieval. - **interval** (string) - Optional - The aggregation interval for the metrics ('minute', 'hour', 'day'). Defaults to 'hour'. ### Response #### Success Response (200) - **accumulation** (object) - An object containing metrics for each distinct basin operation. - **operation_name** (number) - The count of a specific basin operation. #### Response Example ```json { "accumulation": { "list_buckets": 100, "get_object": 5000, "put_object": 2000 } } ``` ``` -------------------------------- ### Insert New Product Record via SQL (Bash) Source: https://s2.dev/docs/integrations/sequin This command demonstrates how to insert a new product record into the 'products' table using SQL within a Dockerized PostgreSQL instance. The change will then be reflected in the S2 stream. ```bash docker exec -i sequin-sequin_postgres-1 \ psql -U postgres -d sequin_playground -c \ "insert into products (name, price) values ('Organic Honey (16 oz)', 12.99);" ``` -------------------------------- ### GET /v1/metrics Source: https://s2.dev/docs/api/metrics/overview Retrieves metric sets evaluated at the account, basin, or stream level. Common parameters include 'set', 'start', 'end', and 'interval'. ```APIDOC ## GET /v1/metrics ### Description Retrieves metric sets evaluated at the account, basin, or stream level. Common parameters include 'set', 'start', 'end', and 'interval'. ### Method GET ### Endpoint /v1/metrics ### Parameters #### Query Parameters - **set** (string) - Required - Specifies the requested metric set. - **start** (integer) - Required - Unix epoch timestamp for the start of the period (in seconds). - **end** (integer) - Required - Unix epoch timestamp for the end of the period (in seconds). - **interval** (string) - Optional - Specifies the duration for timeseries data (e.g., for 'accumulation' or 'gauge' metric types). ### Request Example ```bash curl \ --request GET \ --url "https://aws.s2.dev/v1/metrics?set=active-basins&start=${START}&end=${END}" \ --header "Authorization: Bearer ${S2_ACCESS_TOKEN}" ``` ### Response #### Success Response (200) - **values** (array) - Contains the metric data. - **gauge** (object) - Represents gauge metric data. - **name** (string) - The name of the gauge metric. - **unit** (string) - The unit of the gauge metric. - **values** (array) - Array of [epoch, value] tuples. #### Response Example ```json { "values": [ { "gauge": { "name": "metered_size", "unit": "bytes", "values": [ [ 1748473860, 589.0 ] ] } } ] } ``` ``` -------------------------------- ### GET /websites/s2_dev/metrics/append-throughput Source: https://s2.dev/docs/api/metrics/basin Returns accumulation metrics of appended throughput in bytes, aggregated across all streams within the basin, separated by storage class. Requires start and end timestamps and accepts an optional interval. ```APIDOC ## GET /websites/s2_dev/metrics/append-throughput ### Description Returns two accumulation metrics of appended throughput in bytes, aggregated across all streams within the basin: one for appends against express storage class streams, and another for appends against standard storage class streams. ### Method GET ### Endpoint /websites/s2_dev/metrics/append-throughput ### Parameters #### Query Parameters - **start** (timestamp) - Required - The start timestamp for the metric retrieval. - **end** (timestamp) - Required - The end timestamp for the metric retrieval. - **interval** (string) - Optional - The aggregation interval for the metrics ('minute', 'hour', 'day'). Defaults to 'hour'. ### Response #### Success Response (200) - **accumulation** (object) - An object containing append throughput by storage class. - **express** (number) - The appended throughput in bytes for express storage class. - **standard** (number) - The appended throughput in bytes for standard storage class. #### Response Example ```json { "accumulation": { "express": 2000000000, "standard": 3000000000 } } ``` ``` -------------------------------- ### Resource Matching and Permissions Source: https://s2.dev/docs/api/access-tokens/issue This section details how to match resources using a prefix and defines the structure for permitted operation groups, including account, basin, and stream-level permissions. ```APIDOC ## Resource Matching ### Description Match all resources that start with a specified prefix. An empty string matches all resources. ### Method N/A (Schema definition) ### Endpoint N/A (Schema definition) ### Parameters #### Request Body - **prefix** (string) - Required - Match all resources that start with this prefix. Use an empty string to match all resources. ### Response N/A (Schema definition) --- ## Permitted Operation Groups ### Description Defines the structure for permitted operation groups, specifying access levels (read/write) for account, basin, and stream resources. ### Method N/A (Schema definition) ### Endpoint N/A (Schema definition) ### Parameters #### Request Body - **account** (object | null) - Optional - Account-level access permissions. See `ReadWritePermissions`. - **basin** (object | null) - Optional - Basin-level access permissions. See `ReadWritePermissions`. - **stream** (object | null) - Optional - Stream-level access permissions. See `ReadWritePermissions`. ### Response N/A (Schema definition) --- ## ReadWritePermissions ### Description Defines read and write permissions for a specific resource level. ### Method N/A (Schema definition) ### Endpoint N/A (Schema definition) ### Parameters #### Request Body - **read** (boolean) - Optional - Read permission. Defaults to `false`. - **write** (boolean) - Optional - Write permission. Defaults to `false`. ### Response N/A (Schema definition) --- ## Operation Types ### Description Enumerates the possible operations that can be performed. ### Method N/A (Schema definition) ### Endpoint N/A (Schema definition) ### Parameters #### Request Body - **Operation** (string) - Enum - One of the following: `list-basins`, `create-basin`, `delete-basin`, `reconfigure-basin`, `get-basin-config`, `issue-access-token`, `revoke-access-token`, `list-access-tokens`, `list-streams`, `create-stream`, `delete-stream`, `get-stream-config`, `reconfigure-stream`, `check-tail`, `append`, `read`, `trim`, `fence`, `account-metrics`, `basin-metrics`, `stream-metrics`. ### Response N/A (Schema definition) --- ## Authentication ### Description Bearer authentication using an access token. ### Method N/A (Schema definition) ### Endpoint N/A (Schema definition) ### Parameters #### Header - **Authorization** (string) - Required - Bearer authentication header of the form `Bearer `, where `` is your access token. ### Response N/A (Schema definition) ``` -------------------------------- ### Deployment and Development Commands for Y-S2 Source: https://s2.dev/docs/integrations/y-s2 This section outlines the necessary npm commands for installing dependencies, deploying the Y-S2 worker to Cloudflare, and running it locally for development purposes. ```bash # Install dependencies npm install # Deploy to Cloudflare Workers npm run deploy # Or run locally for development npm run dev ``` -------------------------------- ### Project Configuration Source: https://s2.dev/docs/api/basins/get-config This section details the properties available for configuring a project, including timestamping modes and retention settings. ```APIDOC ## Project Configuration Details ### Description Provides details on the configurable properties for a project, such as timestamping modes and uncapped timestamp behavior. ### Method GET ### Endpoint /websites/s2_dev ### Parameters #### Query Parameters - **mode** (TimestampingMode | null) - Optional - Timestamping mode for appends that influences how timestamps are handled. - **uncapped** (boolean | null) - Optional - Allow client-specified timestamps to exceed the arrival time. If this is `false` or not set, client timestamps will be capped at the arrival time. ### Response #### Success Response (200) - **mode** (TimestampingMode | null) - The timestamping mode for appends. - **uncapped** (boolean | null) - Indicates if client-specified timestamps can exceed the arrival time. #### Response Example ```json { "mode": "client-prefer", "uncapped": true } ``` ### Schemas #### TimestampingMode - **type**: string - **enum**: ["client-prefer", "client-require", "arrival"] #### InfiniteRetention - **type**: object ``` -------------------------------- ### Create S2 Client and Access Stream (Rust) Source: https://s2.dev/docs/sdk/overview Illustrates initializing an S2 client with an access token from environment variables and then accessing a stream within a basin. This requires the s2_sdk crate and the S2_ACCESS_TOKEN environment variable. ```rust use s2_sdk::{S2, types::S2Config}; let client = S2::new(S2Config::new(std::env::var("S2_ACCESS_TOKEN")?))?; let basin = client.basin("my-basin".parse()?); let stream = basin.stream("my-stream".parse()?); ``` -------------------------------- ### Create S2 Basin Source: https://s2.dev/docs/integrations/bento This command creates a new S2 basin, which acts as a container for streams. Basin names must be globally unique, between 8 and 48 characters, and follow specific naming conventions (lowercase letters, numbers, hyphens, not starting or ending with a hyphen). ```bash export MY_BASIN_NAME="" s2 create-basin ${MY_BASIN_NAME} ``` -------------------------------- ### Delete Product via SQL (Bash) Source: https://s2.dev/docs/integrations/sequin This bash command illustrates how to delete a product ('Blueberries (6 oz)') from the 'products' table using SQL. The deletion event will be sent to the S2 stream. ```bash docker exec -i sequin-sequin_postgres-1 \ psql -U postgres -d sequin_playground -c \ "delete from products where name = 'Blueberries (6 oz)';" ``` -------------------------------- ### GET /v1/streams/{stream_name}/records Source: https://s2.dev/docs/api/records/read Retrieves a batch of records from a stream or continuously streams records. You can specify the starting point using `seq_num`, `timestamp`, or `tail_offset`. The read can be bounded by `count`, `bytes`, or `until` a specific timestamp. Supports long polling via the `wait` parameter and Server-Sent Events (SSE) by setting the `Accept: text/event-stream` header. ```APIDOC ## GET /v1/streams/{stream_name}/records ### Description Retrieves a batch of records from a stream or continuously streams records. You can specify the starting point using `seq_num`, `timestamp`, or `tail_offset`. The read can be bounded by `count`, `bytes`, or `until` a specific timestamp. Supports long polling via the `wait` parameter and Server-Sent Events (SSE) by setting the `Accept: text/event-stream` header. ### Method GET ### Endpoint `/v1/streams/{stream_name}/records` ### Parameters #### Query Parameters - **seq_num** (integer) - Optional - The sequence number to start reading from. - **timestamp** (integer) - Optional - The timestamp (in milliseconds) to start reading from. - **tail_offset** (integer) - Optional - The number of records before the tail to start reading from. - **count** (integer) - Optional - The maximum number of records to return. - **bytes** (integer) - Optional - The maximum size in bytes of the records to return. - **until** (integer) - Optional - The timestamp (in milliseconds) until which records should be read. - **wait** (integer) - Optional - The number of seconds to wait for new records if the stream is empty or the end is reached. A non-zero value enables long polling. - **clamp** (boolean) - Optional - If true, the read will clamp to the available records if the starting point is beyond the tail. Defaults to false. #### Request Headers - **Accept** (string) - Optional - Set to `text/event-stream` to receive Server-Sent Events. - **Authorization** (string) - Required - Bearer token for authentication. ### Response #### Success Response (200) - **records** (array) - An array of record objects. Each record object contains `seq_num`, `timestamp`, and `body`. - **tail** (object) - Information about the current tail of the stream, including `seq_num` and `timestamp`. #### Error Response (416 Range Not Satisfiable) Returned when the requested starting point is out of range and `clamp` is false. ### Request Example (SSE) ```bash curl --request GET \ --url 'https://hello-world.b.aws.s2.dev/v1/streams/test/records?seq_num=0' \ --header "Authorization: Bearer ${S2_ACCESS_TOKEN}" \ --header 'Accept: text/event-stream' ``` ### Response Example (SSE - Batch Event) ```json { "event": "batch", "id": "4,5,74", "data": "{\"records\":[{\"seq_num\":4,\"timestamp\":1750101582144,\"body\":\"hello\"}],\"tail\":{\"seq_num\":5,\"timestamp\":1750101582144}}" } ``` ### Response Example (SSE - Ping Event) ```json { "event": "ping", "data": "{\"timestamp\":1750101582144}" } ``` ### Response Example (SSE - Error Event) ```json { "event": "error", "data": "Something went wrong" } ``` ### Response Example (SSE - Done Event) ```json { "event": "done", "data": "[DONE]" } ``` ``` -------------------------------- ### Initialize S2 Client and Access Stream (Rust) Source: https://s2.dev/docs/sdk/typescript Initializes the S2 client using an access token from environment variables and then accesses a specific basin and stream. This example shows the fundamental structure for using the Rust SDK. ```rust use s2_sdk::{S2, types::S2Config}; let client = S2::new(S2Config::new(std::env::var("S2_ACCESS_TOKEN")?))?; let basin = client.basin("my-basin".parse()?); let stream = basin.stream("my-stream".parse()?); ``` -------------------------------- ### GET /streams/{stream} Source: https://s2.dev/docs/api/streams/get-config Retrieves the configuration for a specific stream. This endpoint allows you to get details about retention policies, storage class, and timestamping behavior. ```APIDOC ## GET /streams/{stream} ### Description Retrieves the configuration for a specific stream. ### Method GET ### Endpoint /streams/{stream} ### Parameters #### Path Parameters - **stream** (string) - Required - Stream name. ### Request Example ```json { "example": "No request body for GET request" } ``` ### Response #### Success Response (200) - **delete_on_empty** (object | null) - Delete-on-empty configuration. - **retention_policy** (object | null) - Retention policy for the stream. - **storage_class** (string | null) - Storage class for recent writes. - **timestamping** (object | null) - Timestamping behavior. #### Response Example ```json { "example": { "delete_on_empty": { "min_age_secs": 0 }, "retention_policy": { "age": 604800 }, "storage_class": "standard", "timestamping": { "mode": "append" } } } ``` ``` -------------------------------- ### Structured Errors - Starting Point Out of Range (416) Source: https://s2.dev/docs/api/error-codes Indicates that the requested starting point for a read operation is outside the valid range of the stream. ```APIDOC ## Starting Point Out of Range (416) `416` without a `Range` header? Unconventional, but the shoe fits. See [the precise conditions](records/read#starting-point-out-of-range) under which this is returned in response to a `read` request. ```json { "tail": { "seq_num": 42, "timestamp": 1767996832000 } } ``` The `tail` field indicates the current end position of the stream. ``` -------------------------------- ### Update Product Price via SQL (Bash) Source: https://s2.dev/docs/integrations/sequin This bash command demonstrates updating the price of an existing product ('Avocados (3 pack)') in the 'products' table using SQL. This update will be reflected in the S2 stream. ```bash docker exec -i sequin-sequin_postgres-1 \ psql -U postgres -d sequin_playground -c \ "update products set price = 7.99 where name = 'Avocados (3 pack)';" ``` -------------------------------- ### Create and Populate S2 Source Streams Source: https://s2.dev/docs/integrations/bento This script creates multiple S2 streams with the prefix 'pup/' and appends 10,000 random numbers (woofs) to each. It defines an array of pup names, iterates through them to create streams, and then generates and appends random data using echo and s2 append. ```bash PUP_NAMES=("buddy" "yoyo" "scooby") for PUP in "${PUP_NAMES[@]}"; do s2 create-stream "s2://${MY_BASIN_NAME}/pup/${PUP}" # Generate and append 10000 random woofs. for _ in {1..10000}; do echo "${RANDOM}" done | s2 append "s2://${MY_BASIN_NAME}/pup/${PUP}" done ```