### Manual Setup: Start Iceberg REST Catalog Source: https://github.com/supabase/iceberg-js/blob/main/test/integration/TESTING-DOCKER.md Manually starts the Docker Compose services for the Iceberg REST Catalog and MinIO in detached mode. ```bash docker compose up -d ``` -------------------------------- ### Manual Setup: Run Test Script Source: https://github.com/supabase/iceberg-js/blob/main/test/integration/TESTING-DOCKER.md Executes the local catalog integration test script using tsx. ```bash npx tsx test/integration/test-local-catalog.ts ``` -------------------------------- ### Run Individual Compatibility Test (ESM Project) Source: https://github.com/supabase/iceberg-js/blob/main/test/compatibility/README.md Navigate to a specific compatibility test project (e.g., esm-project), install its dependencies, and run its tests. ```bash cd test/compatibility/esm-project npm install npm test ``` -------------------------------- ### Manual Setup: Verify Catalog Running Source: https://github.com/supabase/iceberg-js/blob/main/test/integration/TESTING-DOCKER.md Checks if the Iceberg REST Catalog is running by sending a request to its configuration endpoint. ```bash curl http://localhost:8181/v1/config ``` -------------------------------- ### Clone Repository and Install Dependencies Source: https://github.com/supabase/iceberg-js/blob/main/CONTRIBUTING.md Clone the iceberg-js repository and install project dependencies using pnpm. ```bash git clone https://github.com/YOUR_USERNAME/iceberg-js.git cd iceberg-js pnpm install ``` -------------------------------- ### TSDoc Example for createTable Function Source: https://github.com/supabase/iceberg-js/blob/main/CONTRIBUTING.md This example demonstrates the required TSDoc format for public APIs, including parameter, return, and throws tags, along with a runnable code example. ```typescript /** * Creates a new table in the catalog. * * @param namespace - Namespace to create the table in * @param request - Table creation request including name, schema, partition spec * @returns Table metadata for the created table * @throws {IcebergError} If the table already exists or namespace doesn't exist * * @example * ```typescript * const metadata = await catalog.createTable( * { namespace: ['analytics'] }, * { name: 'events', schema: { ... } } * ); * ``` */ async createTable(namespace: NamespaceIdentifier, request: CreateTableRequest): Promise ``` -------------------------------- ### Manual Setup: Stop Containers Source: https://github.com/supabase/iceberg-js/blob/main/test/integration/TESTING-DOCKER.md Stops and removes all containers, networks, and volumes defined in the Docker Compose file. Use this to clean up resources. ```bash docker compose down -v ``` -------------------------------- ### Run Integration Tests (with cleanup) Source: https://github.com/supabase/iceberg-js/blob/main/test/integration/TESTING-DOCKER.md Executes integration tests, starting Docker services, running tests, and cleaning up containers afterward. Use this for CI environments. ```bash pnpm test:integration:ci ``` -------------------------------- ### Running Integration Tests with Docker Source: https://github.com/supabase/iceberg-js/blob/main/README.md Commands to start Docker services, run integration tests against a local Iceberg REST Catalog, and clean up Docker resources. Useful for local development and CI. ```bash # Start Docker services and run integration tests pnpm test:integration # Or manually docker compose up -d npx tsx test/integration/test-local-catalog.ts docker compose down -v ``` -------------------------------- ### Conventional Commits Examples Source: https://github.com/supabase/iceberg-js/blob/main/CONTRIBUTING.md Illustrative examples of commit messages following the Conventional Commits specification, including breaking changes. ```bash feat: add support for view operations fix: handle empty namespace list correctly feat(auth): add OAuth2 authentication support docs: update README with new examples test: add integration tests for table updates feat!: change auth config structure BREAKING CHANGE: auth configuration now uses a discriminated union ``` -------------------------------- ### Run Integration Tests (keep containers) Source: https://github.com/supabase/iceberg-js/blob/main/test/integration/TESTING-DOCKER.md Starts Docker services, waits for them to be ready, and runs integration tests. Containers remain running after the test for debugging purposes. ```bash pnpm test:integration ``` -------------------------------- ### Initialize Iceberg REST Catalog with Access Delegation Source: https://github.com/supabase/iceberg-js/blob/main/README.md Example of initializing the `IcebergRestCatalog` client with bearer token authentication and requesting vended credentials for data access. Use `Result` variants to access credentials. ```typescript import { IcebergRestCatalog } from 'iceberg-js' const catalog = new IcebergRestCatalog({ baseUrl: 'https://catalog.example.com/iceberg/v1', auth: { type: 'bearer', token: 'your-token' }, // Request vended credentials for data access accessDelegation: ['vended-credentials'], }) // To access vended credentials (storage-credentials, server config), use the // *Result variants — `loadTable`/`createTable`/`registerTable` return only // the bare `TableMetadata` and would discard credentials. const result = await catalog.loadTableResult({ namespace: ['analytics'], name: 'events' }) // result['storage-credentials'], result.config, result.etag, result.metadata ``` -------------------------------- ### Install iceberg-js Source: https://github.com/supabase/iceberg-js/blob/main/README.md Install the iceberg-js package using npm. This command is used to add the library to your project's dependencies. ```bash npm install iceberg-js ``` -------------------------------- ### Initialize IcebergRestCatalog Source: https://github.com/supabase/iceberg-js/blob/main/README.md Instantiate the IcebergRestCatalog client with catalog configuration, including base URL, warehouse, and authentication details. The warehouse is optional and can be resolved via a GET /v1/config request. ```typescript import { IcebergRestCatalog } from 'iceberg-js' const catalog = new IcebergRestCatalog({ baseUrl: 'https://my-catalog.example.com', warehouse: 'my-warehouse', // optional; resolved via GET /v1/config auth: { type: 'bearer', token: process.env.ICEBERG_TOKEN, }, }) ``` -------------------------------- ### Development Commands Source: https://github.com/supabase/iceberg-js/blob/main/README.md Common commands for developing the iceberg-js library, including dependency installation, building, testing (unit, integration, compatibility), and code formatting/linting. ```bash # Install dependencies pnpm install # Build the library pnpm run build # Run unit tests pnpm test # Run integration tests (requires Docker) pnpm test:integration # Run integration tests with cleanup (for CI) pnpm test:integration:ci # Run compatibility tests (all module systems) pnpm test:compatibility # Format code pnpm run format # Lint and test pnpm run check ``` -------------------------------- ### Create Namespace with Iceberg REST Catalog Source: https://github.com/supabase/iceberg-js/blob/main/README.md Example of creating a new namespace with optional properties using the `createNamespace` method. The namespace identifier and metadata, including properties, are provided as arguments. ```typescript await catalog.createNamespace({ namespace: ['analytics'] }, { properties: { owner: 'data-team' } }) ``` -------------------------------- ### Troubleshooting: Port Conflict Configuration Source: https://github.com/supabase/iceberg-js/blob/main/test/integration/TESTING-DOCKER.md Example of how to reconfigure Docker Compose ports if the default ports (8181, 9000) are already in use. Remember to update the test script accordingly. ```yaml ports: - '8182:8181' # Use 8182 instead of 8181 ``` -------------------------------- ### Update Table Request Body - Remove Properties (1.0) Source: https://github.com/supabase/iceberg-js/blob/main/MIGRATION.md This example demonstrates removing table properties using the `remove-properties` action in the `updateTable` request body. ```typescript await catalog.updateTable( { namespace: ['analytics'], name: 'events' }, { requirements: [], updates: [ { action: 'remove-properties', removals: ['stale-property'], }, ], } ) ``` -------------------------------- ### Conditional Load Table Source: https://github.com/supabase/iceberg-js/blob/main/MIGRATION.md Perform a conditional GET request for table metadata using the `ifNoneMatch` option. Returns `null` if the table has not been modified (304 Not Modified). ```typescript loadTable(id, { ifNoneMatch }) ``` -------------------------------- ### Update Table Request Body - Set Properties (1.0) Source: https://github.com/supabase/iceberg-js/blob/main/MIGRATION.md In 1.0, `updateTable` uses a spec-shaped body with `requirements` and `updates`. This example shows setting table properties. ```typescript await catalog.updateTable( { namespace: ['analytics'], name: 'events' }, { requirements: [], updates: [ { action: 'set-properties', updates: { 'read.split.target-size': '134217728' }, }, ], } ) ``` -------------------------------- ### Update Namespace Properties with Iceberg REST Catalog Source: https://github.com/supabase/iceberg-js/blob/main/README.md Example of updating namespace properties, including setting new properties and removing existing ones, using the `updateNamespaceProperties` method. Requires namespace identifier and an update/removal object. ```typescript await catalog.updateNamespaceProperties( { namespace: ['analytics'] }, { updates: { owner: 'data-team' }, removals: ['stale_property'] } ) ``` -------------------------------- ### Build the Project Source: https://github.com/supabase/iceberg-js/blob/main/CONTRIBUTING.md Build the library for distribution or use watch mode for development. ```bash pnpm run build ``` ```bash pnpm run dev ``` -------------------------------- ### Build and Run All Compatibility Tests Source: https://github.com/supabase/iceberg-js/blob/main/test/compatibility/README.md Build the iceberg-js package and then execute all compatibility tests using the provided shell script. ```bash pnpm build bash test/compatibility/run-all.sh ``` -------------------------------- ### Initialize Catalog with Access Delegation Source: https://github.com/supabase/iceberg-js/blob/main/MIGRATION.md Configure the IcebergRestCatalog with access delegation enabled, specifying the token for authentication. ```typescript const catalog = new IcebergRestCatalog({ baseUrl: 'https://catalog.example.com', warehouse: 'my-warehouse', auth: { type: 'bearer', token: process.env.ICEBERG_TOKEN }, accessDelegation: ['vended-credentials'], }) ``` -------------------------------- ### Create a Table Source: https://github.com/supabase/iceberg-js/blob/main/README.md Use the `createTable` method to define and create a new table. This requires the namespace, table name, schema definition, partition specification, write order, and table properties. ```typescript // Create a table await catalog.createTable( { namespace: ['analytics'] }, { name: 'events', schema: { type: 'struct', fields: [ { id: 1, name: 'id', type: 'long', required: true }, { id: 2, name: 'timestamp', type: 'timestamp', required: true }, { id: 3, name: 'user_id', type: 'string', required: false }, ], 'schema-id': 0, 'identifier-field-ids': [1], }, 'partition-spec': { 'spec-id': 0, fields: [], }, 'write-order': { 'order-id': 0, fields: [], }, properties: { 'write.format.default': 'parquet', }, } ) ``` -------------------------------- ### Linting and Full Project Check Source: https://github.com/supabase/iceberg-js/blob/main/CONTRIBUTING.md Run ESLint for code issues and a full project check including lint, type-check, test, and build. ```bash pnpm run lint ``` ```bash pnpm run check ``` -------------------------------- ### Run Unit Tests Source: https://github.com/supabase/iceberg-js/blob/main/CONTRIBUTING.md Execute unit tests using Vitest. Use watch mode for test-driven development. ```bash pnpm test ``` ```bash pnpm test:watch ``` -------------------------------- ### Run Integration Tests Source: https://github.com/supabase/iceberg-js/blob/main/CONTRIBUTING.md Run integration tests against a real Iceberg REST Catalog in Docker. Use the 'ci' variant for automatic cleanup. ```bash pnpm test:integration ``` ```bash pnpm test:integration:ci ``` -------------------------------- ### Catalog Configuration (0.x) Source: https://github.com/supabase/iceberg-js/blob/main/MIGRATION.md In 0.x, the catalog name was passed directly to the `IcebergRestCatalog` constructor and manually appended to the URL path. ```typescript const catalog = new IcebergRestCatalog({ baseUrl: 'https://catalog.example.com', catalogName: 'my-warehouse', auth: { type: 'bearer', token: process.env.ICEBERG_TOKEN }, }) ``` -------------------------------- ### IcebergRestCatalog Constructor Source: https://github.com/supabase/iceberg-js/blob/main/README.md Initializes a new instance of the IcebergRestCatalog client. It requires the base URL of the REST catalog and accepts optional configurations for warehouse, catalog name, authentication, custom fetch, and access delegation. ```APIDOC ## `new IcebergRestCatalog(options)` Creates a new catalog client instance. **Options:** - `baseUrl` (string, required): Base URL of the REST catalog - `warehouse` (string, optional): Warehouse identifier. On first use, the client calls `GET /v1/config?warehouse=…` and uses the server-returned `overrides.prefix` for all subsequent requests. - `catalogName` (string, optional): Permanent alias for `warehouse`, kept for backward compatibility. If both are provided, `warehouse` wins. - `auth` (AuthConfig, optional): Authentication configuration - `fetch` (typeof fetch, optional): Custom fetch implementation - `accessDelegation` (AccessDelegation[], optional): Access delegation mechanisms to request from the server **Authentication types:** ```typescript // No authentication { type: 'none' } // Bearer token { type: 'bearer', token: 'your-token' } // Custom header { type: 'header', name: 'X-Custom-Auth', value: 'secret' } // Custom function { type: 'custom', getHeaders: async () => ({ 'Authorization': 'Bearer ...' }) } ``` **Access Delegation:** Supported delegation mechanisms: - `vended-credentials`: Server provides temporary credentials (e.g., AWS STS tokens) for accessing table data - `remote-signing`: Server signs data access requests on behalf of the client ``` -------------------------------- ### Catalog Configuration (1.0) Source: https://github.com/supabase/iceberg-js/blob/main/MIGRATION.md In 1.0, the `warehouse` parameter is used, which the client uses to fetch overrides from `/v1/config`. ```typescript const catalog = new IcebergRestCatalog({ baseUrl: 'https://catalog.example.com', warehouse: 'my-warehouse', auth: { type: 'bearer', token: process.env.ICEBERG_TOKEN }, }) ``` -------------------------------- ### Troubleshooting: Check Catalog Logs Source: https://github.com/supabase/iceberg-js/blob/main/test/integration/TESTING-DOCKER.md Follows the logs of the Iceberg REST catalog service to diagnose issues. Useful for verifying startup progress and identifying errors. ```bash docker compose logs -f iceberg-rest ``` -------------------------------- ### Type Checking and Formatting Source: https://github.com/supabase/iceberg-js/blob/main/CONTRIBUTING.md Run TypeScript type checking and format code using Prettier. ```bash pnpm run type-check ``` ```bash pnpm run format ``` -------------------------------- ### createTable Source: https://github.com/supabase/iceberg-js/blob/main/README.md Creates a new table in the catalog with the specified schema and partition specification. ```APIDOC ## `createTable` ### Description Creates a new table in the catalog. ### Method Signature `createTable(namespace: NamespaceIdentifier, request: CreateTableRequest): Promise` ### Parameters #### Path Parameters - **namespace** (NamespaceIdentifier) - Required - The namespace for the new table. - **request** (CreateTableRequest) - Required - The details of the table to create, including name, schema, and partition specification. ### Request Example ```json { "namespace": ["analytics"], "name": "events", "schema": { "type": "struct", "fields": [ { "id": 1, "name": "id", "type": "long", "required": true }, { "id": 2, "name": "timestamp", "type": "timestamp", "required": true } ], "schema-id": 0 }, "partition-spec": { "spec-id": 0, "fields": [ { "source-id": 2, "field-id": 1000, "name": "ts_day", "transform": "day" } ] } } ``` ### Response #### Success Response (200) - **TableMetadata** - Metadata of the newly created table. ``` -------------------------------- ### createNamespace Source: https://github.com/supabase/iceberg-js/blob/main/README.md Creates a new namespace with optional metadata properties. ```APIDOC ## `createNamespace(id: NamespaceIdentifier, metadata?: NamespaceMetadata)` Create a new namespace with optional properties. **Parameters:** - `id` (NamespaceIdentifier, required): An object specifying the namespace to create. Example: `{ namespace: ['analytics'] }` - `metadata` (NamespaceMetadata, optional): An object containing properties for the new namespace. Example: `{ properties: { owner: 'data-team' } }` **Returns:** - `Promise` ### Request Example ```typescript await catalog.createNamespace({ namespace: ['analytics'] }, { properties: { owner: 'data-team' } }) ``` ``` -------------------------------- ### Browser Usage with IcebergRestCatalog Source: https://github.com/supabase/iceberg-js/blob/main/README.md Instantiate IcebergRestCatalog for use in modern browsers supporting native fetch. Requires importing the class and configuring the base URL and authentication. ```typescript import { IcebergRestCatalog } from 'iceberg-js' const catalog = new IcebergRestCatalog({ baseUrl: 'https://public-catalog.example.com/iceberg/v1', auth: { type: 'none' }, }) const namespaces = await catalog.listNamespaces() ``` -------------------------------- ### Create a New Table Source: https://github.com/supabase/iceberg-js/blob/main/README.md Use `createTable` to define and add a new table to the catalog. Requires namespace and table schema details. ```typescript const metadata = await catalog.createTable( { namespace: ['analytics'] }, { name: 'events', schema: { type: 'struct', fields: [ { id: 1, name: 'id', type: 'long', required: true }, { id: 2, name: 'timestamp', type: 'timestamp', required: true }, ], 'schema-id': 0, }, 'partition-spec': { 'spec-id': 0, fields: [ { 'source-id': 2, 'field-id': 1000, name: 'ts_day', transform: 'day', }, ], }, } ) ``` -------------------------------- ### Integration Test Script: Run with Cleanup Source: https://github.com/supabase/iceberg-js/blob/main/test/integration/TESTING-DOCKER.md Executes the integration test script and ensures that Docker containers are cleaned up automatically after the test completes. ```bash bash scripts/test-integration.sh --cleanup ``` -------------------------------- ### Run Local Checks Command Source: https://github.com/supabase/iceberg-js/blob/main/CONTRIBUTING.md Commands to run local checks, including linting and integration tests, before committing changes. ```bash pnpm run check pnpm test:integration ``` -------------------------------- ### Iterating All Namespaces with Pagination (1.0) Source: https://github.com/supabase/iceberg-js/blob/main/MIGRATION.md This snippet shows how to iterate through all namespaces across multiple pages using `nextPageToken`. ```typescript let pageToken: string | undefined do { const page = await catalog.listNamespaces({ pageSize: 100, pageToken }) for (const ns of page.namespaces) { // ... } pageToken = page.nextPageToken } while (pageToken) ``` -------------------------------- ### Custom Fetch Implementation for IcebergRestCatalog Source: https://github.com/supabase/iceberg-js/blob/main/README.md Configure IcebergRestCatalog with a custom fetch implementation for advanced use cases like proxying or instrumentation. This allows injecting a specific HTTP client or logic. ```typescript import { IcebergRestCatalog } from 'iceberg-js' const catalog = new IcebergRestCatalog({ baseUrl: 'https://catalog.example.com/iceberg/v1', auth: { type: 'bearer', token: 'token' }, fetch: myCustomFetch, }) ``` -------------------------------- ### listNamespaces Source: https://github.com/supabase/iceberg-js/blob/main/README.md Lists namespaces, optionally under a parent namespace, with support for cursor-based pagination. ```APIDOC ## `listNamespaces(options?)` List namespaces, optionally under a parent namespace, with cursor-based pagination. **Parameters:** - `options` (object, optional) - `parent` (object): Specifies the parent namespace to list children from. Example: `{ namespace: ['analytics'] }` - `pageSize` (number, optional): The maximum number of namespaces to return per page. - `pageToken` (string, optional): A token to retrieve the next page of results. **Returns:** - `Promise<{ namespaces: Array<{ namespace: string[] }>, nextPageToken?: string }>`: An object containing a list of namespaces and an optional token for the next page. ### Request Example ```typescript const { namespaces } = await catalog.listNamespaces() // namespaces: [{ namespace: ['default'] }, { namespace: ['analytics'] }] const { namespaces: children } = await catalog.listNamespaces({ parent: { namespace: ['analytics'] }, }) // Pagination const page1 = await catalog.listNamespaces({ pageSize: 100 }) const page2 = await catalog.listNamespaces({ pageSize: 100, pageToken: page1.nextPageToken, }) ``` ``` -------------------------------- ### Integration Test Script: Custom Wait Timeout Source: https://github.com/supabase/iceberg-js/blob/main/test/integration/TESTING-DOCKER.md Runs the integration test script with a custom wait timeout, overriding the default of 60 seconds. Useful for slower environments. ```bash bash scripts/test-integration.sh --wait-timeout 120 ``` -------------------------------- ### List Namespaces with Iceberg REST Catalog Source: https://github.com/supabase/iceberg-js/blob/main/README.md Demonstrates listing namespaces, optionally under a parent namespace, and with cursor-based pagination. The `listNamespaces` method returns an array of namespace objects and a potential `nextPageToken`. ```typescript const { namespaces } = await catalog.listNamespaces() // namespaces: [{ namespace: ['default'] }, { namespace: ['analytics'] }] const { namespaces: children } = await catalog.listNamespaces({ parent: { namespace: ['analytics'] }, }) // Pagination const page1 = await catalog.listNamespaces({pageSize: 100}) const page2 = await catalog.listNamespaces({ pageSize: 100, pageToken: page1.nextPageToken, }) ``` -------------------------------- ### List Tables with Iceberg REST Catalog Source: https://github.com/supabase/iceberg-js/blob/main/README.md Demonstrates listing tables within a namespace, supporting cursor-based pagination. The `listTables` method requires a namespace identifier and optionally accepts pagination options. ```typescript const { identifiers } = await catalog.listTables({ namespace: ['analytics'] }) // identifiers: [{ namespace: ['analytics'], name: 'events' }] const page1 = await catalog.listTables({ namespace: ['analytics'] }, { pageSize: 100 }) const page2 = await catalog.listTables( { namespace: ['analytics'] }, { pageSize: 100, pageToken: page1.nextPageToken } ) ``` -------------------------------- ### List Namespaces Parent Argument (1.0) Source: https://github.com/supabase/iceberg-js/blob/main/MIGRATION.md In 1.0, the `parent` option is used to specify the parent namespace, making room for pagination parameters. ```typescript const { namespaces: children } = await catalog.listNamespaces({ parent: { namespace: ['analytics'] }, }) ``` -------------------------------- ### List Namespaces with Pagination Source: https://github.com/supabase/iceberg-js/blob/main/README.md Retrieve a list of namespaces using `listNamespaces`. This method supports pagination, returning a `nextPageToken` if more results are available. Specify `pageSize` to control the number of results per page. ```typescript // List namespaces (paginated) const { namespaces, nextPageToken } = await catalog.listNamespaces({ pageSize: 50 }) ``` -------------------------------- ### List Namespaces Return Shape (1.0) Source: https://github.com/supabase/iceberg-js/blob/main/MIGRATION.md After migration, `listNamespaces` returns a paginated response including `namespaces` and `nextPageToken`. ```typescript const { namespaces, nextPageToken } = await catalog.listNamespaces() for (const ns of namespaces) { console.log(ns.namespace) } ``` -------------------------------- ### Create a Namespace Source: https://github.com/supabase/iceberg-js/blob/main/README.md Use the `createNamespace` method to create a new namespace within the catalog. This operation requires the namespace identifier. ```typescript // Create a namespace await catalog.createNamespace({ namespace: ['analytics'] }) ``` -------------------------------- ### listTables Source: https://github.com/supabase/iceberg-js/blob/main/README.md Lists tables within a specified namespace, supporting cursor-based pagination. ```APIDOC ## `listTables(namespace, options?)` List tables in a namespace, with cursor-based pagination. **Parameters:** - `namespace` (object, required): An object specifying the namespace to list tables from. Example: `{ namespace: ['analytics'] }` - `options` (object, optional) - `pageSize` (number, optional): The maximum number of tables to return per page. - `pageToken` (string, optional): A token to retrieve the next page of results. **Returns:** - `Promise<{ identifiers: Array<{ namespace: string[], name: string }>, nextPageToken?: string }>`: An object containing a list of table identifiers and an optional token for the next page. ### Request Example ```typescript const { identifiers } = await catalog.listTables({ namespace: ['analytics'] }) // identifiers: [{ namespace: ['analytics'], name: 'events' }] const page1 = await catalog.listTables({ namespace: ['analytics'] }, { pageSize: 100 }) const page2 = await catalog.listTables( { namespace: ['analytics'] }, { pageSize: 100, pageToken: page1.nextPageToken } ) ``` ``` -------------------------------- ### List Namespaces Return Shape (0.x) Source: https://github.com/supabase/iceberg-js/blob/main/MIGRATION.md Before migration, `listNamespaces` returned a flat array of namespaces. ```typescript const namespaces = await catalog.listNamespaces() for (const ns of namespaces) { console.log(ns.namespace) } ``` -------------------------------- ### List Namespaces Parent Argument (0.x) Source: https://github.com/supabase/iceberg-js/blob/main/MIGRATION.md In 0.x, the `namespace` parameter was used to list children under a specific parent namespace. ```typescript const children = await catalog.listNamespaces({ namespace: ['analytics'] }) ``` -------------------------------- ### Conventional Commits Types Source: https://github.com/supabase/iceberg-js/blob/main/CONTRIBUTING.md Defines the types of commits and their impact on versioning. Use '!' for breaking changes. ```text Types: - feat: New feature (triggers minor version bump) - fix: Bug fix (triggers patch version bump) - docs: Documentation changes only - test: Adding or updating tests - chore: Maintenance tasks, dependency updates - refactor: Code changes that neither fix bugs nor add features - perf: Performance improvements - ci: CI/CD configuration changes ``` -------------------------------- ### Load Table Result in iceberg-js v1.0 Source: https://github.com/supabase/iceberg-js/blob/main/MIGRATION.md In v1.0, use `loadTableResult` to obtain storage credentials, configuration, and the ETag along with the table metadata. ```typescript // 1.0 — use loadTableResult to receive storage-credentials, config, and the ETag const result = await catalog.loadTableResult({ namespace: ['analytics'], name: 'events', }) // result.metadata, result.config, result['storage-credentials'], result.etag ``` -------------------------------- ### Load Namespace Metadata with Iceberg REST Catalog Source: https://github.com/supabase/iceberg-js/blob/main/README.md Shows how to load metadata and properties of a specific namespace using the `loadNamespaceMetadata` method. The namespace identifier is required. ```typescript const metadata = await catalog.loadNamespaceMetadata({ namespace: ['analytics'] }) // { properties: { owner: 'data-team', ... } } ``` -------------------------------- ### Update Table with Requirements and Updates Source: https://github.com/supabase/iceberg-js/blob/main/MIGRATION.md In v1.0, updating a table involves specifying requirements and a list of update actions, such as setting properties. ```typescript updateTable({ requirements, updates: [{ action: 'set-properties', updates }] }) ``` -------------------------------- ### loadTable Source: https://github.com/supabase/iceberg-js/blob/main/README.md Loads table metadata from the catalog. Supports conditional loading using an ETag. ```APIDOC ## `loadTable` ### Description Loads table metadata from the catalog. Can be used for conditional GET requests. ### Method Signature `loadTable(id: TableIdentifier, options?: LoadTableOptions): Promise` ### Parameters #### Path Parameters - **id** (TableIdentifier) - Required - The identifier of the table to load. #### Query Parameters - **ifNoneMatch** (string) - Optional - An ETag value for conditional loading. If the table has not changed since this ETag, `null` is returned. ### Request Example ```json { "namespace": ["analytics"], "name": "events" } ``` ### Response #### Success Response (200) - **TableMetadata** - Metadata of the table. #### Not Modified Response (304) - **null** - Returned if `ifNoneMatch` was provided and the table has not changed. ``` -------------------------------- ### Snapshot Selection for Load Table Source: https://github.com/supabase/iceberg-js/blob/main/MIGRATION.md Control the number of snapshots included in the server response by specifying the `snapshots` option. ```typescript loadTable(id, { snapshots: 'all' | 'refs' }) ``` -------------------------------- ### Push Changes Command Source: https://github.com/supabase/iceberg-js/blob/main/CONTRIBUTING.md Command to push local commits to the remote origin of your fork. ```bash git push origin feat/my-feature ``` -------------------------------- ### Create Git Branch Command Source: https://github.com/supabase/iceberg-js/blob/main/CONTRIBUTING.md Command to create a new feature branch from the main branch. ```bash git checkout -b feat/my-feature ``` -------------------------------- ### loadNamespaceMetadata Source: https://github.com/supabase/iceberg-js/blob/main/README.md Loads the metadata and properties associated with a specific namespace. ```APIDOC ## `loadNamespaceMetadata(id: NamespaceIdentifier)` Load namespace metadata and properties. **Parameters:** - `id` (NamespaceIdentifier, required): An object specifying the namespace to load metadata for. Example: `{ namespace: ['analytics'] }` **Returns:** - `Promise`: An object containing the namespace's properties. ### Response Example ```typescript const metadata = await catalog.loadNamespaceMetadata({ namespace: ['analytics'] }) // { properties: { owner: 'data-team', ... } } ``` ``` -------------------------------- ### Drop Namespace with Iceberg REST Catalog Source: https://github.com/supabase/iceberg-js/blob/main/README.md Demonstrates dropping an empty namespace using the `dropNamespace` method. The namespace identifier is passed as an argument. ```typescript await catalog.dropNamespace({ namespace: ['analytics'] }) ``` -------------------------------- ### Authentication Types for Iceberg REST Catalog Source: https://github.com/supabase/iceberg-js/blob/main/README.md Demonstrates various authentication methods supported by the `IcebergRestCatalog` client, including none, bearer token, custom header, and custom function. ```typescript // No authentication { type: 'none' } // Bearer token { type: 'bearer', token: 'your-token' } // Custom header { type: 'header', name: 'X-Custom-Auth', value: 'secret' } // Custom function { type: 'custom', getHeaders: async () => ({ 'Authorization': 'Bearer ...' }) } ``` -------------------------------- ### List Tables Return Shape (1.0) Source: https://github.com/supabase/iceberg-js/blob/main/MIGRATION.md After migration, `listTables` returns a paginated response with `identifiers` and `nextPageToken`. ```typescript const { identifiers: tables, nextPageToken } = await catalog.listTables({ namespace: ['analytics'], }) for (const t of tables) { console.log(t.name) } ``` -------------------------------- ### Commit Changes Command Source: https://github.com/supabase/iceberg-js/blob/main/CONTRIBUTING.md Command to commit staged changes using the Conventional Commits format. ```bash git commit -m "feat: add support for XYZ" ``` -------------------------------- ### Load Table in iceberg-js v0.x Source: https://github.com/supabase/iceberg-js/blob/main/MIGRATION.md In v0.x, `loadTable` returned metadata including inline credentials. This method is now deprecated for retrieving credentials. ```typescript // 0.x — credentials were folded into the metadata response const metadata = await catalog.loadTable({ namespace: ['analytics'], name: 'events' }) // metadata.config had the credentials ``` -------------------------------- ### Conventional Commits Message Format Source: https://github.com/supabase/iceberg-js/blob/main/CONTRIBUTING.md This is the standard format for commit messages, used for automated releases. It includes type, scope, description, and optional body/footer. ```text (): [optional body] [optional footer] ``` -------------------------------- ### loadTableResult Source: https://github.com/supabase/iceberg-js/blob/main/README.md Retrieves the result of loading a table, including metadata, configuration, and credentials, along with the ETag for future conditional requests. ```APIDOC ## `loadTableResult` ### Description Provides a spec-aligned `LoadTableResult` wrapper, exposing metadata, configuration, storage credentials, and the ETag. ### Method Signature `loadTableResult(id: TableIdentifier, options?: LoadTableOptions): Promise` ### Parameters #### Path Parameters - **id** (TableIdentifier) - Required - The identifier of the table to load. #### Query Parameters - **ifNoneMatch** (string) - Optional - An ETag value for conditional loading. ### Response #### Success Response (200) - **LoadTableResult & { etag }** - An object containing table metadata, configuration, storage credentials, and the ETag. ``` -------------------------------- ### Load Table Metadata Source: https://github.com/supabase/iceberg-js/blob/main/README.md Use `loadTable` to retrieve table metadata. Supports conditional loading using `ifNoneMatch` for ETags to avoid unnecessary fetches. ```typescript const metadata = await catalog.loadTable({ namespace: ['analytics'], name: 'events', }) ``` ```typescript // Conditional load const updated = await catalog.loadTable( { namespace: ['analytics'], name: 'events' }, { ifNoneMatch: lastSeenEtag } ) if (updated === null) { // table is unchanged since lastSeenEtag } ``` -------------------------------- ### Import TypeScript Types Source: https://github.com/supabase/iceberg-js/blob/main/README.md The library exports all necessary types for working with Iceberg objects in TypeScript. Import them as needed. ```typescript import type { // Identifiers NamespaceIdentifier, TableIdentifier, // Schema / type system TableSchema, TableField, IcebergType, PartitionSpec, SortOrder, // Requests / responses CreateTableRequest, CommitTableRequest, CommitTableResponse, LoadTableResult, LoadTableResultWithEtag, TableMetadata, UpdateNamespacePropertiesRequest, UpdateNamespacePropertiesResponse, // Method options ListNamespacesOptions, ListNamespacesResult, ListTablesOptions, ListTablesResult, LoadTableOptions, // Catalog config CatalogConfig, StorageCredential, // Table update / requirement unions (full spec coverage) TableUpdate, TableRequirement, // Auth / delegation AuthConfig, AccessDelegation, } from 'iceberg-js' ``` -------------------------------- ### Drop a Table Source: https://github.com/supabase/iceberg-js/blob/main/README.md Use `dropTable` to remove a table from the catalog. Requires the table's namespace and name. ```typescript await catalog.dropTable({ namespace: ['analytics'], name: 'events' }) ``` -------------------------------- ### List Tables Return Shape (0.x) Source: https://github.com/supabase/iceberg-js/blob/main/MIGRATION.md Before migration, `listTables` returned a flat array of table identifiers. ```typescript const tables = await catalog.listTables({ namespace: ['analytics'] }) for (const t of tables) { console.log(t.name) } ``` -------------------------------- ### Update Table Request Body (0.x) Source: https://github.com/supabase/iceberg-js/blob/main/MIGRATION.md In 0.x, `updateTable` accepted a simpler object for table updates, such as setting properties. ```typescript await catalog.updateTable( { namespace: ['analytics'], name: 'events' }, { properties: { 'read.split.target-size': '134217728' } } ) ``` -------------------------------- ### dropTable Source: https://github.com/supabase/iceberg-js/blob/main/README.md Removes a table from the catalog. ```APIDOC ## `dropTable` ### Description Drops a table from the catalog. ### Method Signature `dropTable(id: TableIdentifier): Promise` ### Parameters #### Path Parameters - **id** (TableIdentifier) - Required - The identifier of the table to drop. ### Request Example ```json { "namespace": ["analytics"], "name": "events" } ``` ### Response #### Success Response (200) - **void** - Indicates the table was successfully dropped. ``` -------------------------------- ### Stop Catalog (Keep Data) Source: https://github.com/supabase/iceberg-js/blob/main/test/integration/TESTING-DOCKER.md Stops the running Docker containers without removing their associated data volumes. ```bash docker compose stop ``` -------------------------------- ### dropNamespace Source: https://github.com/supabase/iceberg-js/blob/main/README.md Drops a specified namespace. The namespace must be empty before it can be dropped. ```APIDOC ## `dropNamespace(id: NamespaceIdentifier)` Drop a namespace. The namespace must be empty. **Parameters:** - `id` (NamespaceIdentifier, required): An object specifying the namespace to drop. Example: `{ namespace: ['analytics'] }` **Returns:** - `Promise` ### Request Example ```typescript await catalog.dropNamespace({ namespace: ['analytics'] }) ``` ``` -------------------------------- ### Update Table Properties Source: https://github.com/supabase/iceberg-js/blob/main/README.md Use `updateTable` to commit changes to a table's properties. Requires specifying requirements and the updates to apply. ```typescript const updated = await catalog.updateTable( { namespace: ['analytics'], name: 'events' }, { requirements: [{ type: 'assert-current-schema-id', 'current-schema-id': 0 }], updates: [{ action: 'set-properties', updates: { 'read.split.target-size': '134217728' } }], } ) ``` -------------------------------- ### updateNamespaceProperties Source: https://github.com/supabase/iceberg-js/blob/main/README.md Updates properties of a namespace by setting new values or removing existing ones. ```APIDOC ## `updateNamespaceProperties(id, request)` Set or remove namespace properties. **Parameters:** - `id` (NamespaceIdentifier, required): An object specifying the namespace to update. Example: `{ namespace: ['analytics'] }` - `request` (object, required): An object containing updates and removals for namespace properties. - `updates` (object): Key-value pairs of properties to set or update. - `removals` (string[]): An array of property keys to remove. **Returns:** - `Promise` ### Request Example ```typescript await catalog.updateNamespaceProperties( { namespace: ['analytics'] }, { updates: { owner: 'data-team' }, removals: ['stale_property'] } ) ``` ``` -------------------------------- ### Conventional Commits Breaking Changes Source: https://github.com/supabase/iceberg-js/blob/main/CONTRIBUTING.md Specifies how to indicate breaking changes in commit messages, either via '!' or the BREAKING CHANGE footer. ```text Breaking changes: - Use feat!: or fix!: for breaking changes (triggers major version bump) - Or include BREAKING CHANGE: in the commit footer ``` -------------------------------- ### updateTable / commitTable Source: https://github.com/supabase/iceberg-js/blob/main/README.md Commits updates to a table's properties or schema. ```APIDOC ## `updateTable` / `commitTable` ### Description Commits updates to a table using the spec-aligned `{ requirements?, updates }` shape. ### Method Signature `updateTable(id: TableIdentifier, request: CommitTableRequest): Promise` `commitTable(id: TableIdentifier, request: CommitTableRequest): Promise` ### Parameters #### Path Parameters - **id** (TableIdentifier) - Required - The identifier of the table to update. - **request** (CommitTableRequest) - Required - An object containing requirements and updates for the table. - **requirements** (Array) - Optional - Assertions about the current state of the table. - **updates** (Array) - Required - The actions to perform on the table. ### Request Example ```json { "namespace": ["analytics"], "name": "events", "requirements": [ { "type": "assert-current-schema-id", "current-schema-id": 0 } ], "updates": [ { "action": "set-properties", "updates": { "read.split.target-size": "134217728" } } ] } ``` ### Response #### Success Response (200) - **CommitTableResponse** - Response indicating the success of the commit operation. ``` -------------------------------- ### Handle Iceberg Errors Source: https://github.com/supabase/iceberg-js/blob/main/README.md All API errors throw an `IcebergError`. Catch this error to inspect details like status code, error type, and message. ```typescript import { IcebergError } from 'iceberg-js' try { await catalog.loadTable({ namespace: ['test'], name: 'missing' }) } catch (error) { if (error instanceof IcebergError) { console.log(error.status) // 404 console.log(error.icebergType) // 'NoSuchTableException' console.log(error.message) // 'Table does not exist' } } ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.