Depot

https://stage-tech.github.io/stage-depot
Admin
Depot is a data infrastructure platform that enables teams to build, deploy, and operate data...
Tokens:226,891
Snippets:1,825
Trust Score:8.3
Update:7 months ago
Show doc for...
Tokens:
Raw
### Install Linux CLI Executable (Bash)

Source: https://stage-tech.github.io/stage-depot/docs/development/components/cli/installation

Installs the downloaded Linux executable to '/usr/local/bin/depot', makes it executable, and then runs the 'depot' command to verify the installation. Requires sudo privileges for moving the file.

```bash
#! /bin/bash
DEPOT_TARGET="/usr/local/bin/depot"
sudo mv ~/Downloads/depot-linux $DEPOT_TARGET
chmod +x $DEPOT_TARGET
depot
```

--------------------------------

### GraphQL Setup and Schema Download

Source: https://stage-tech.github.io/stage-depot/docs/development/api/gql

Instructions on how to install and use the `graphql-cli` tool to initialize your GraphQL configuration and download the schema from the server.

```APIDOC
## GraphQL CLI Setup

### Description
This section provides instructions for installing the `graphql-cli` and initializing your `.graphqlconfig` file to manage GraphQL endpoints and schema paths. It also details how to download the schema from the server.

### Installation

**Using npm:**
```bash
npm install -g graphql-cli
```

**Using Yarn:**
```bash
yarn global add graphql-cli
```

**Using pnpm:**
```bash
pnpm add -g graphql-cli
```

### Initialization
Initialize your `.graphqlconfig` file to configure endpoints and schema paths:
```bash
graphql init
```

### Schema Download
Download the schema from the server:
```bash
graphql get-schema
```
```

--------------------------------

### Collection Processing Examples

Source: https://stage-tech.github.io/stage-depot/docs/development/schema/field-types/schema-expressions-new

Demonstrates checking for specific elements within collections and defining complex conditions for collection elements. These examples are conceptual and language-agnostic.

```pseudocode
// Check for specific elements
value in allowedValues

// Complex collection conditions
p.x > 10 && p.y < 100 for any p of points
```

--------------------------------

### Install macOS CLI Executable (Bash)

Source: https://stage-tech.github.io/stage-depot/docs/development/components/cli/installation

Installs the downloaded macOS executable to '/usr/local/bin/depot', makes it executable, and adds it to the system's security policy. Finally, it runs the 'depot' command to verify the installation.

```bash
#! /bin/bash
DEPOT_TARGET="/usr/local/bin/depot"
mv ~/Downloads/depot-macos $DEPOT_TARGET
chmod +x $DEPOT_TARGET
spctl --add --label "stage-depot" $DEPOT_TARGET
spctl --enable --label "stage-depot"
depot
```

--------------------------------

### Data Aggregation Examples

Source: https://stage-tech.github.io/stage-depot/docs/development/schema/field-types/schema-expressions-new

Illustrates common data aggregation functions such as calculating averages and counting items that meet specific criteria. These examples are conceptual and language-agnostic.

```pseudocode
// Calculate averages
avg(amount)

// Count matching items
count(status == 'active')
```

--------------------------------

### Install Depot CLI on Linux

Source: https://stage-tech.github.io/stage-depot/docs/deployments-cdk/tutorials/tutorial-snowflake

Installs the Depot CLI by moving the downloaded binary to a system path, making it executable, and verifying the installation. Requires sudo privileges for moving the file to /usr/local/bin.

```bash
#! /bin/bash  
DEPOT_TARGET="/usr/local/bin/depot"  
sudo mv ~/Downloads/depot-linux $DEPOT_TARGET  
chmod +x $DEPOT_TARGET  
depot --version  

```

--------------------------------

### Basic Filtering Examples

Source: https://stage-tech.github.io/stage-depot/docs/development/schema/field-types/schema-expressions-new

Provides examples of basic filtering conditions in a query language. It demonstrates filtering by a single country, multiple conditions including country and state, and pattern matching using ILIKE.

```javascript
// Basic filtering
country == 'US'

// Multiple conditions
country == 'US' && state == 'California'

// Pattern matching
city ILIKE 'san%'
```

--------------------------------

### Get Operation with Expansion Request Example (JavaScript)

Source: https://stage-tech.github.io/stage-depot/docs/development/api/lambda

An example of a 'get' request that includes the 'expand' field to retrieve related data. This allows fetching fields beyond the main schema, such as reference or linked fields.

```javascript
const getReq = {
    schema: "MySchema",
    operation: "get",
    id: id,
    dataset: { id: "my-dataset-id" },
}
```

--------------------------------

### Basic Expression Syntax: Comparison Example

Source: https://stage-tech.github.io/stage-depot/docs/development/schema/field-types/schema-expressions-new

Provides examples of basic comparison operations in both JavaScript and Stage Depot schema formats. The Stage Depot example shows how to define a boolean field 'isUSAddress' based on the 'country' field.

```javascript
country == 'US'
```

```yaml
example.object.Address:
  type: object
  properties:
    country:
      type: string
    isUSAddress:
      type: boolean
      expression: "this.country == 'US'"
```

--------------------------------

### Get Operation Request Example (JavaScript)

Source: https://stage-tech.github.io/stage-depot/docs/development/api/lambda

An example of a request payload for the 'get' operation. This requires the schema, a unique ID for the instance, and the dataset ID. The response contains the retrieved instance or a 404 status.

```javascript
const getReq = {
    schema: "MySchema",
    id: "my-instance-id",
    operation: "get",
    dataset: { id: "my-dataset-id" },
}
```

--------------------------------

### Start and Stop Depot-Local Instance

Source: https://stage-tech.github.io/stage-depot/docs/development/testing/depot-local

Demonstrates how to start and stop the Depot-Local instance using asynchronous methods. These are typically used within testing frameworks like Jest.

```typescript
beforeAll(async () => {
    await depot.start();
});

afterAll(async () => {
    await depot.stop();
});

```

--------------------------------

### Install Node.js Dependencies

Source: https://stage-tech.github.io/stage-depot/docs/deployments-cdk/tutorials/tutorial-snowflake

Installs the necessary Node.js dependencies for the project using npm. It first ensures the correct Node.js version is selected using nvm, then runs `npm install`.

```bash
nvm use  
npm install  

```

--------------------------------

### Start Backend for Debugging

Source: https://stage-tech.github.io/stage-depot/docs/development/testing/snowflake/advanced-java-debugger

This snippet shows how to start the backend process with a version lock file parameter for debugging purposes. It allows for setting breakpoints and running under a profiler.

```bash
test-runner-server --version-lock-file=/tmp/vlf
```

--------------------------------

### Install wscat using Yarn

Source: https://stage-tech.github.io/stage-depot/docs/development/event/subscriptions

Installs the wscat command-line tool globally using Yarn. This is an alternative to npm for installing the WebSocket client.

```bash
yarn global add wscat

```

--------------------------------

### Install wscat using npm

Source: https://stage-tech.github.io/stage-depot/docs/development/event/subscriptions

Installs the wscat command-line tool globally using npm. This tool is used for connecting to WebSocket endpoints and interacting with GraphQL subscriptions.

```bash
npm install -g wscat

```

--------------------------------

### GraphQL GET Query Example

Source: https://stage-tech.github.io/stage-depot/docs/development/api/gql

Demonstrates a basic GraphQL GET query to retrieve an individual entity by its ID and optionally by version. It includes Depot's reserved fields and entity-specific fields.

```graphql
query{
  getPet(id: "123", version:1) {
    # Depot reserved fields...
    id
    created
    updated
    schema
    version
    hash
    # Entity schema specific fields...
    name
    species
    etc
  }
}

```

--------------------------------

### Install wscat using pnpm

Source: https://stage-tech.github.io/stage-depot/docs/development/event/subscriptions

Installs the wscat command-line tool globally using pnpm. This provides another package management option for installing the WebSocket client.

```bash
pnpm add -g wscat

```

--------------------------------

### Basic GraphQL Query Examples

Source: https://stage-tech.github.io/stage-depot/docs/development/api/gql

Provides examples of common GraphQL queries and mutations, including listing items with specific fields, retrieving an item by ID, and creating a new item.

```APIDOC
## Basic Query Examples

### Description
This section provides practical examples of basic GraphQL queries and mutations for interacting with Depot data.

### List Query
Retrieve a list of contacts, limiting to 5 items and selecting specific fields:
```graphql
query example {
  listContacts(filter:"true", limit: 5) {
    items{
      contactId,
      contactFirstName
    }
  }
}
```

### Get Query by ID
Retrieve a specific sample file by its ID:
```graphql
query getFileById {
  getSampleFiles(id:"e8dc6015-ecfb-4890-b5d4-674d5995003a") {
    id,
    fileName,
    version
  }
}
```

### Create Mutation
Create a new action with specified data and return its ID:
```graphql
mutation a{
  createAction(data:{
    s3Uri:"s3://foo-bar/change/data",
    operation:INSERT
  }) {
    id
  }
}
```
```

--------------------------------

### Define Product Object with Expressions

Source: https://stage-tech.github.io/stage-depot/docs/development/schema/field-types/schema-expressions-new

Example of a basic product object schema definition using Stage Depot expressions. It demonstrates computed fields like 'discountedPrice' and 'isOnSale' based on other properties.

```yaml
example.object.Product:
  type: object
  id:
    expression: "'product:' + sys.uuidv4()"
  properties:
    price:
      type: number
      description: "Product price in euros"
    discountedPrice:
      type: number
      description: "Calculated discounted price"
      expression: "this.price * (1 - this.discountPercentage)"
    isOnSale:
      type: boolean
      description: "Whether the product is currently on sale"
      expression: "this.discountPercentage > 0"
    discountPercentage:
      type: number
      description: "Current discount percentage (0-1)"

```

--------------------------------

### Multiple Sequences Data Output Example

Source: https://stage-tech.github.io/stage-depot/docs/development/schema/types/Sequences

Illustrates a possible output structure after processing the input data with the 'Multiple Sequences Usage Example' schema, showing how different sequences are assigned.

```json
{
  "name": "hearts",
  "id": "S1:1",
  "altId": "S2:1",
  "spareId": "S1:2"
}
{
  "name": "diamonds",
  "id": "S1:3",
  "altId": "S2:4",
  "spareId": "S1:4"
}
{
  "name": "clubs",
  "id": "S1:5",
  "altId": "S2:3",
  "spareId": "S1:7"
}
{
  "name": "spades",
  "id": "S1:8",
  "altId": "S2:2",
  "spareId": "S1:6"
}
```

--------------------------------

### Sort Order Examples (Command Line)

Source: https://stage-tech.github.io/stage-depot/docs/deployments-cdk/depot-cdk/gateways

These examples illustrate different ways to specify sort order for paginated results, including ascending, descending, and handling of null values. The syntax allows for specifying direction and null ordering.

```bash
--sort "myField:asc"

```

```bash
--sort "myField:desc:nulls-first"

```

```bash
--sort "myField:asc:last"

```

--------------------------------

### Initialize Local Project Dependencies

Source: https://stage-tech.github.io/stage-depot/docs/deployments-cdk/tutorials/tutorial-snowflake

Commands to set the Node.js version using NVM and install project dependencies using npm, typically run after cloning a package repository.

```bash
nvm use
npm install
```

--------------------------------

### GraphQL GET Query with Complex Nested Primary Key Example

Source: https://stage-tech.github.io/stage-depot/docs/development/api/gql

Shows an example of a GraphQL GET query utilizing a complex, nested primary key. The schema will generate inputs corresponding to the fields that constitute the primary key.

```graphql
query {
  getPetstorePetStoreNameCount(
    name: "pet1"
    store: { id: "00ec4277-efc4-41af-86fb-f2c6d69f269c" }
    location: { region: { name: "testRegion" } }
  ) {
    name
    count
    store { id, name }
    location { label, region { name, capital } }
  }
}

```

--------------------------------

### Step-by-Step Depot Test Setup

Source: https://stage-tech.github.io/stage-depot/docs/development/testing/snowflake

This snippet illustrates setting up a `DepotTest` scenario in distinct 'given', 'when', and 'then' steps. The `run()` method can be called after each step or accumulated for later execution, allowing for flexible test structuring and deferred execution.

```typescript
import * as path from "node:path";

import { MergedSchemas } from "@stage-tech/depot-schema";


const dt = DepotTest
    .in(namespace)
    .options({ sessionTimeoutMilliseconds: 90_000 });

function givenXXX() {
  dt.setData(path.join(__dirname, 'data'));
}

function whenXXX() {
  dt.transaction({
      source: "my.Source",
      target: "my.TransformTarget"
  })
}

async function thenXXX() {
  dt.check({
      expectedUri: path.join(__dirname, 'expected'),
      actual: ['my.Result']
  });
    
  await dt.run();
}

async function thenYYY() {
    dt.check({
        expectedUri: path.join(__dirname, 'expected'),
        actual: ['my.OtherResult']
    });
  
    await dt.run();
}

```

--------------------------------

### Run Tests Against Instrumented Backend

Source: https://stage-tech.github.io/stage-depot/docs/development/testing/snowflake/advanced-java-debugger

These examples demonstrate how to execute the test suite with the environment variable set, directing all tests to the instrumented backend. This allows for comprehensive debugging.

```bash
DEPOT_TEST_BACKEND_LOCKFILE=/tmp/vlf yarn run test
```

```bash
DEPOT_TEST_BACKEND_LOCKFILE=/tmp/vlf yarn test
```

--------------------------------

### Computed Fields Examples

Source: https://stage-tech.github.io/stage-depot/docs/development/schema/field-types/schema-expressions-new

Shows examples of computed fields, including conditional computations using ternary operators and providing default values using the nullish coalescing operator.

```javascript
// Conditional computation
isUSAddress ? 'Domestic' : 'International'

// Default values
name ?? 'Unknown'
```

--------------------------------

### GraphQL GET Query by Primary Key Example

Source: https://stage-tech.github.io/stage-depot/docs/development/api/gql

Illustrates how to perform a GET query on a view using its primary key. This is generated when a view schema specifies a 'primaryKey'.

```graphql
query {
  getPetNameCount(name: "pet1") {
    name
    count
  }
}

```

--------------------------------

### SQL: Text Pattern Matching with LIKE and ILIKE

Source: https://stage-tech.github.io/stage-depot/docs/development/schema/field-types/schema-expressions-new

Demonstrates the basic syntax for case-sensitive (LIKE) and case-insensitive (ILIKE) pattern matching in SQL. It explains the use of '%' and '_' wildcards and escaping special characters.

```SQL
-- Case-sensitive matching  
field LIKE pattern  
  
-- Case-insensitive matching  
field ILIKE pattern  

Pattern syntax:
  * `%` matches any sequence of characters
  * `_` matches any single character
  * Use `\` to escape special characters
```

--------------------------------

### JavaScript Test Setup for DepotLambdaClient with Mocking

Source: https://stage-tech.github.io/stage-depot/docs/development/components/depot-lambda-client

This snippet demonstrates setting up a test environment using Jest, mocking AWS Lambda client interactions with aws-sdk-client-mock, and utilizing DepotLocal for simulating depot API responses. It covers initializing DepotLocal, defining test contexts, and mocking AWS SDK calls to simulate various scenarios, including successful operations and error conditions.

```javascript
import { InvokeCommand, LambdaClient } from "@aws-sdk/client-lambda";
import {
    DepotApi,
    DepotLocal,
    EnvironmentBuilder,
    loadIntoDepotViaBatch,
    LocationBuilder,
    proxyToDepotApiMap
} from "@stage-tech/depot-local";
import { MergedSchemas } from "@stage-tech/depot-schema";
import {LoggerFactory } from "@stage-tech/stage-monitor-logging-client";
import { mockClient } from "aws-sdk-client-mock";
import {
    DepotFields,
    DepotLambdaClient,
    DepotLambdaListOnlyClient,
    PartOfRecordRequiringCodecs,
    RecordCodec,
    StandardRecordCodecs
} from "@stage-tech/depot-lambda-client";

jest.setTimeout(360_000);

const DepotLambdaMock = mockClient(LambdaClient);

describe("sample test using DepotLambdaClient + Mock + DepotLocal", () => {

    it("should do simple creation when everything's good", async () =>
        await withContext(async (context) => {
            const myPet = await context.myPetDepotClient.create({
                /* TypeScript will require all mandatory properties, reject unknown properties if you put a constant here */
            });
            expect(myPet.version).toBeGreaterThanOrEqual(1);
        }));


    it("should retrieve objects even with simulated disruption", async () =>
        await withContext(async (context) => {
            await context.myPetDepotClient.create({
                id: "pet007",
            });

            DepotLambdaMock.reset()
                .on(InvokeCommand)
                .resolvesOnce(dynamoDbErrorResponse)
                .resolvesOnce(dynamoDbErrorResponse)
                .resolvesOnce(dynamoDbErrorResponse)
                .callsFake(proxyToDepotApiMap({[DATASET_ALIAS]: myPetDepotApi}));

            // this will show that the Depot Lambda Client will hit three failures before
            // finally resuming successfully

            const myPet = await context.myPetDepotClient.get("pet007");
            expect(myPet.version).toBeGreaterThanOrEqual(1);
        }));
});

const depotLocal = new DepotLocal({
    depot: {},
    dynamodb: {}
});

beforeAll(async () => {
    await depotLocal.start();
})

afterAll(async () => {
    await depotLocal.stop();
})

const MyTestNamespace: MergedSchemas = {
    // build it the way you want, reading a Depot Package or manually like this:
    [MyPetSchemaName]: MyPetSchema
};

function withContext<T>(action: (context: TestContext) => Promise<T>): Promise<T> {
    const myPetDepotApi = depotLocal
        .deploy(new EnvironmentBuilder(MyTestNamespace).location(LocationBuilder.dynamoDbWithTestScan()).build())
        .api(0);

    DepotLambdaMock.reset()
        .on(InvokeCommand)
        .callsFake(proxyToDepotApiMap({[DATASET_ALIAS]: myPetDepotApi}));

    const myPetDepotClient = new DepotLambdaClient<MyPet>(new LambdaClient({}), LoggerFactory.getConsoleLogger(), {
        depotLambdaArn: "unit-test",
        datasetId: myPetDepotApi.datasetId,
        schema: MyPetSchemaName,
        retryOptions: {
            maxRetries: 5,
            initialDelayInSeconds: 0.25, // shorter than in deployed code
            backoffFactor: 1.01 // really small backoff factor
        },
        recordCodec: StandardRecordCodecs.DepotFields
    });

    return action({myPetDepotApi, myPetDepotClient});
}

const dynamoDbErrorResponse = {
    StatusCode: 200,
    Payload: textEncoder.encode(
        `{
      "status": {
        "code": 500,
        "message": "Internal Server Error"
      },
      "data": {
        "code": "UnknownError",
        "message": "Internal server error (Service: DynamoDb, Status Code: 500, Request ID: HHB5VAOJ87RCUBAMGCMFFL2LJNVV4KQNSO5AEMVJF66Q9ASUAAJG)"
      }
    }`
    )
};

```

--------------------------------

### Stage Depot Schema: Text Pattern Matching Examples

Source: https://stage-tech.github.io/stage-depot/docs/development/schema/field-types/schema-expressions-new

Shows how text pattern matching expressions are defined within the Stage Depot schema for filtering data. It includes examples for checking string prefixes, suffixes, and complex patterns.

```YAML
example.object.SearchFilter:
  type: object
  properties:
    city:
      type: string
    email:
      type: string
    domain:
      type: string
    startsWithL:
      type: boolean
      description: "Check if city starts with 'L' (case insensitive)"
      expression: "this.city ILIKE 'l%'"
    isGmailAddress:
      type: boolean
      description: "Check if email is from Gmail"
      expression: "this.email ILIKE '%@gmail.com'"
    matchesDomain:
      type: boolean
      description: "Check if domain matches pattern"
      expression: "this.domain LIKE '%.com' || this.domain LIKE '%.org'"
    isValidUsername:
      type: boolean
      description: "Check if username follows pattern (letters followed by numbers)"
      expression: "this.username LIKE '[A-Za-z]%[0-9]'"
```

--------------------------------

### Stamp Example: Using Caller for User Name

Source: https://stage-tech.github.io/stage-depot/docs/development/schema/field-types/Stamps

This example demonstrates how to use the 'caller.name' property within a stamp to set the 'createdByName' field based on the current user's name during object creation. It requires a User schema definition and Cognito location setup.

```yaml
stamps.Example:
  type: object
  properties:
    createdByName:
      type: string
      stamps:
        - on: create
          then: caller.name
```

--------------------------------

### Stage Depot Schema: User Profile Reference Example

Source: https://stage-tech.github.io/stage-depot/docs/development/schema/field-types/schema-expressions-new

An example of a UserProfile object in the Stage Depot schema, showcasing the use of 'ref' types to link to User and Organization objects. It includes expressions to derive 'isActiveEmployee', 'departmentName', and 'hasManagerAccess' based on referenced data.

```Stage Depot Schema
example.object.UserProfile:
  type: object
  properties:
    user:
      type: ref
      ref: User
    organization:
      type: ref
      ref: Organization
    isActiveEmployee:
      type: boolean
      description: "Check if user is active in organization"
      expression: "this.user.status == 'ACTIVE' && this.organization.isOperational"
    departmentName:
      type: string
      description: "Get department name through reference"
      expression: "this.user.department.name ?? 'Unassigned'"
    hasManagerAccess:
      type: boolean
      description: "Check manager status through references"
      expression: "this.user.role.level >= this.organization.managerLevel"
```

--------------------------------

### Stage Depot Schema: Content Display Coalescing and Conditional Example

Source: https://stage-tech.github.io/stage-depot/docs/development/schema/field-types/schema-expressions-new

An example of a ContentDisplay object in Stage Depot schema, demonstrating the use of null-coalescing for 'displayTitle' and ternary operators for 'displayCategory' and 'priority'. It handles cases where primary data might be missing or requires conditional formatting.

```Stage Depot Schema
example.object.ContentDisplay:
  type: object
  properties:
    title:
      type: string
    fallbackTitle:
      type: string
    category:
      type: string
    displayTitle:
      type: string
      description: "Use fallback title if main title is null"
      expression: "this.title ?? this.fallbackTitle ?? 'Untitled'"
    displayCategory:
      type: string
      description: "Format category based on type"
      expression: "this.category == 'BLOG' ? 'Blog Post' : 'Article'"
    priority:
      type: string
      description: "Complex conditional logic"
      expression: "this.isUrgent ? 'High' : (this.hasDeadline ? 'Medium' : 'Low')"
```

--------------------------------

### Deploy Environment to Depot-Local

Source: https://stage-tech.github.io/stage-depot/docs/development/testing/depot-local

Shows how to deploy a new 'Environment' to a running Depot-Local instance. It involves building an environment using EnvironmentBuilder and LocationBuilder.

```typescript
const namespace = Schemas.fromYamlFiles("test/schema/pet.yml");// Usually this is done in depot package and imported from there
const environment = new EnvironmentBuilder(namespace)
    .location(LocationBuilder.postgresDb(depotLocal))
    .build();
const deployedEnv = depot.deploy(environment);

```

--------------------------------

### Create a Simple Dataset (TypeScript)

Source: https://stage-tech.github.io/stage-depot/docs/deployments-cdk/depot-cdk/documentation/classes/Dataset

Demonstrates how to create a new Dataset instance using the Dataset constructor. It requires a scope, an ID, and properties including environment, package, name, and location. Dependencies like '@some-org/package-source' might be needed for package imports.

```typescript
const sourcePackageImport = require('@some-org/package-source');
const sourcePackage = new Package(this, "ExamplePackage", {
  name: "source",
  version: "v1",
  schemas: sourcePackageImport.schemas,
  imports: sourcePackageImport.imports
});

const sourceDataset = new Dataset(this, "Dataset1", {
  environment: environment,
  package: sourcePackage,
  name: "source",
  location: exampleLocation
});
```

--------------------------------

### Decompress macOS/Linux Executable (Bash)

Source: https://stage-tech.github.io/stage-depot/docs/development/components/cli/installation

Decompresses the downloaded executable on macOS and Linux systems. This is useful if you want to avoid the performance hit of dynamic decompression during execution.

```bash
gzexe -d depot
```

--------------------------------

### Example Test Case using TestStep (TypeScript/Jest)

Source: https://stage-tech.github.io/stage-depot/docs/development/testing/snowflake/legacy-test-step

Demonstrates how to use the `TestStep` interface within a Jest test file. It shows the import of necessary modules, definition of a test step with operation, data URIs, and assertions, and how to execute the test using the `toPassSchemaTest` matcher. This example serves as a starting point for creating schema tests.

```typescript
// import the Jest bindings
import "@stage-tech/depot-test/jest";
import { TestStep } from "@stage-tech/depot-test";

import * as pkg from "..";

//collect all schemas used in package
const namespace = JSON.stringify(pkg.mergedSchemas, null, 2);

describe("some clever test name", () => {
    it("some clever scenario name", () => {
        const step: TestStep = {
            name: "contact identities view is populated",
            operation: Operation.UPSERT,
            dataUri: "test/clever/name/data",
            expectedUri: "test/clever/name/expected",
            assertions: ["some.qualified.schema.name"]
        };
        const steps: Array<TestStep> = [step];
        expect(steps).toPassSchemaTest(namespace);
    });
});

```

--------------------------------

### Snowflake: Complex JSON Object Iteration and Filtering

Source: https://stage-tech.github.io/stage-depot/docs/development/schema/field-types/schema-expressions-new

Provides an example of iterating through a JSON array of objects in Snowflake and applying multiple conditions to filter elements. It uses FLATTEN and a WHERE clause with AND operators.

```SQL
-- Expression: p.x > 10 && p.y < 100 for any p of points  
-- SQL:  
EXISTS (  
  SELECT 1   
  FROM TABLE(FLATTEN(input => points)) AS p  
  WHERE p.value:x::number > 10   
  AND p.value:y::number < 100  
)
```

--------------------------------

### Check and Deploy CDK Changes

Source: https://stage-tech.github.io/stage-depot/docs/deployments-cdk/tutorials/cdk-quickstart

Uses the AWS CDK CLI to first preview changes with `cdk diff` and then deploy them with `cdk deploy`. These commands are essential for verifying the infrastructure modifications before and after applying them to your AWS account.

```bash
cdk diff
```

```bash
cd deploy
```

--------------------------------

### Get Item by ID Query (GraphQL)

Source: https://stage-tech.github.io/stage-depot/docs/development/api/gql

A GraphQL query example to retrieve a specific item, in this case, a sample file, by its `id`. It returns the `id`, `fileName`, and `version` of the file.

```graphql
query getFileById {
  getSampleFiles(id:"e8dc6015-ecfb-4890-b5d4-674d5995003a") {
    id,
    fileName,
    version
  }
}
```

--------------------------------

### Permission Checking Expressions

Source: https://stage-tech.github.io/stage-depot/docs/development/schema/field-types/schema-expressions-new

Provides examples of expressions used for checking user permissions. It shows how to determine if a specific permission exists within a user's roles and how to check for administrative access.

```Expression Language
permission in user.roles
role.name == 'admin' for any role of user.roles
```

--------------------------------

### Performance Monitoring for Complex Calculations

Source: https://stage-tech.github.io/stage-depot/docs/development/schema/field-types/schema-expressions-new

Describes performance monitoring strategies, focusing on tracking query execution plans and performance metrics in production. It includes an example of calculating a complex metric based on filtered data.

```yaml
example.object.MonitoredQueries:
  type: object
  properties:
    data:
      type: array
      items:
        type: object
    complexMetric:
      type: number
      description: "Complex calculation to monitor"
      expression: "avg(item.value * 1.5 for item of this.data where item.category == 'premium')"
```

--------------------------------

### Create a New Depot Environment

Source: https://stage-tech.github.io/stage-depot/docs/deployments-cdk/depot-cdk/environment

Demonstrates the creation of a new Depot environment with specified account and name. This is the foundational step for setting up a Depot environment.

```typescript
const depotEnvironment = new Environment(this, `MyEnvironment`, {
    account: { name: "example", id: Aws.ACCOUNT_ID },
    name: "test"
});
```

--------------------------------

### Snowflake Array Operations Optimization

Source: https://stage-tech.github.io/stage-depot/docs/development/schema/field-types/schema-expressions-new

Details optimized array operations in Snowflake, mentioning the efficiency of FLATTEN for large datasets and the use of specialized array functions. It provides an example SQL for checking array containment.

```yaml
example.object.SnowflakeArray:
  type: object
  properties:
    permissions:
      type: array
      items:
        type: string
    canEdit:
      type: boolean
      description: "Optimized Snowflake array check"
      expression: "'EDIT' in this.permissions"
```

--------------------------------

### Process Collections with Order Expressions

Source: https://stage-tech.github.io/stage-depot/docs/development/schema/field-types/schema-expressions-new

Examples of using Stage Depot expressions for collection processing within an order schema. It includes calculating total amount, checking for large items, and verifying stock availability.

```yaml
example.object.Order:
  type: object
  properties:
    items:
      type: array
      items:
        type: object
        properties:
          productId:
            type: string
          quantity:
            type: integer
          price:
            type: number
    totalAmount:
      type: number
      description: "Total order amount"
      expression: "sum(item.price * item.quantity for item of this.items)"
    hasLargeItems:
      type: boolean
      description: "Check if any items have large quantities"
      expression: "item.quantity > 10 for any item of this.items"
    allItemsInStock:
      type: boolean
      description: "Verify all items are in stock"
      expression: "item.quantity <= item.product.stockCount for all item of this.items"

```

--------------------------------

### Sample Text Output for Schema Compatibility Differences

Source: https://stage-tech.github.io/stage-depot/docs/development/components/cli/tools/get-schema-compatibility

This sample demonstrates the text output format of the 'get-schema-compatibility' command when a new field ('foo') is detected in a schema package not yet deployed in the target environment. It highlights missing fields and provides both manual and DDL resolution steps.

```text
[ERROR]: dataset '29011122d795' with location '76448cf51ae9' has MISSING_FIELD for foo  
[manual resolution]: create field "FOO" in table "EXAMPLE_SOURCE"."MY_CONTACTS"."PERSON"  
[ddl resolution]: alter table "EXAMPLE_SOURCE"."MY_CONTACTS"."PERSON" add "FOO" varchar not null  
  
[ERROR]: dataset '29011122d795' with location '76448cf51ae9' has MISSING_FIELD for foo  
[manual resolution]: create field "FOO" in table "EXAMPLE_SOURCE"."MY_CONTACTS_HISTORY"."PERSON"  
[ddl resolution]: alter table "EXAMPLE_SOURCE"."MY_CONTACTS_HISTORY"."PERSON" add "FOO" varchar not null  

```

--------------------------------

### Start and Stop Local Depot Server (TypeScript)

Source: https://stage-tech.github.io/stage-depot/docs/development/components/depot-local

This snippet demonstrates how to programmatically start and stop a local Depot server instance using the depot-local npm package in a TypeScript project. It requires Docker to be installed and running. The code initializes DepotLocal with specified ports for Depot and DynamoDB, then uses beforeAll and afterAll hooks to manage the server lifecycle.

```typescript
const depot = new DepotLocal({
  depot: {
    port: 8089
  },
  dynamodb: {
    port: 8000
  }
});

beforeAll(async () => {
  await depot.start();
});

afterAll(async () => {
  await depot.stop();
});
```

--------------------------------

### Initialize Depot-Local Connector

Source: https://stage-tech.github.io/stage-depot/docs/development/testing/depot-local

Initializes the Depot-Local connector with various configuration options for different database types. Requires '@stage-tech/depot-local' to be installed.

```typescript
import {DepotLocal} from "@stage-tech/depot-local";

const depot = new DepotLocal({
    depot: {}, //required and is used to pass additional configuration to depot
    dynamodb: {}, //required and is used to pass additional configuration to dynamodb
    postgres: {}, //Optional and is used to pass additional configuration to postgres
    snowflake: {} //Optional and is used to pass additional configuration to snowflake
});

```

--------------------------------

### Add Custom Tools Directory to PATH (PowerShell)

Source: https://stage-tech.github.io/stage-depot/docs/development/components/cli/installation

Appends a custom directory (e.g., 'C:\Tools') to the system's PATH environment variable in PowerShell. This allows executables in the specified directory to be run from any location.

```powershell
$env:Path += ";C:\Tools"
```

--------------------------------

### Initialize GraphQL Configuration (graphql-cli)

Source: https://stage-tech.github.io/stage-depot/docs/development/api/gql

Command to initialize the `.graphqlconfig` file using the `graphql-cli`. This file is essential for configuring GraphQL endpoints and schema paths for your project.

```bash
graphql init
```

--------------------------------

### Query Optimization with Multiple Conditions

Source: https://stage-tech.github.io/stage-depot/docs/development/schema/field-types/schema-expressions-new

Illustrates query optimization techniques, including the use of appropriate indexes and combining multiple conditions within single iterations to avoid nested loops. It shows an example of counting active, high-priority items.

```yaml
example.object.QueryOptimized:
  type: object
  properties:
    items:
      type: array
      items:
        type: object
        properties:
          status:
            type: string
          priority:
            type: integer
    highPriorityCount:
      type: integer
      description: "Optimized query with multiple conditions"
      expression: "count(item.status == 'active' && item.priority > 5 for item of this.items)"
```

--------------------------------

### Initialize and Use Depot Lambda Client in TypeScript

Source: https://stage-tech.github.io/stage-depot/docs/development/components/depot-lambda-client

Demonstrates how to initialize the DepotLambdaClient with AWS Lambda client, logger, and configuration including ARN, dataset ID, schema, retry options, and record codec. It also shows examples of creating and patching records.

```typescript
import { LambdaClient } from "@aws-sdk/client-lambda";
import { DepotLambdaClient } from '@stage-tech/depot-lambda-client'
import { IStageLogger } from "@stage-tech/stage-monitor-logging-client";
import { ConsoleLogger } from "typedoc/dist/lib/utils";

interface MyPet {
    /* ... fields from Depot schema definition file via GraphQL or manually ... */
}

const lambdaClient = new LambdaClient({});
const logger = LoggerFactory.getConsoleLogger();

const petDepotLambdaClient = new DepotLambdaClient<MyPet>(
    lambdaClient,
    logger,
    {
        depotLambdaArn: 'arn:aws:.....:depot-XXXXXXXXX-depot-lambda-gateway', // get it from CDK, likely via an env var
        datasetId: 'my-dataset-alias',
        schema: "my.Pet", // should match <MyPet>
        retryOptions: { // retryOptions is opt-in, if absent no retries in case of transient outages
            maxRetries: 5,
            initialDelayInSeconds: 1.2,
            backoffFactor: 1.5
        },
        recordCodec: StandardRecordCodecs.DepotFields
    });

async function example() {
    const myPet = await petDepotLambdaClient.create({
        /* TypeScript will require all mandatory properties, reject unknown properties if you put a constant here */
    });
    // myPet is of type MyPet & DepotFields
    // so id, version, created, updated & hash are defined

    await petDepotLambdaClient.patch(myPet.id, { /* someField: someNewValue */});
}

```

--------------------------------

### Interact with GQL API (Apollo Client Mutation)

Source: https://stage-tech.github.io/stage-depot/docs/development/testing/depot-local

Shows how to perform a mutation request using the GQL API and Apollo Client. Retrieves the GQL API using `deployedEnv.gqlApi(0)`.

```typescript
const gqlApi = deployedEnv.gqlApi(0)
const create = await gqlApi.mutate<PetStore, PetStore>({
    mutation: gql`
        mutation createPet($name: String!, $age: Int!) {
            createPetPet(id: "0987", data: { name: $name, age: $age }, expressions: {}) {
                id
                created
                updated
                schema
                version
                hash
                name
            }
        }
    `,
    variables: {
        name: "ren",
        age: 2
    },
    context: {
        headers: {Authorization: "Bearer token1"}
    }
});

```

--------------------------------

### Snowflake JSON Operations Optimization

Source: https://stage-tech.github.io/stage-depot/docs/development/schema/field-types/schema-expressions-new

Covers optimized JSON processing in Snowflake, highlighting the performance benefits of colon notation for JSON paths and the potential use of lateral joins for complex iterations. It includes an example SQL for counting enabled settings.

```yaml
example.object.SnowflakeJSON:
  type: object
  properties:
    config:
      type: object
      properties:
        settings:
          type: array
          items:
            type: object
    enabledCount:
      type: integer
      description: "Optimized JSON processing"
      expression: "count(setting.enabled for setting of this.config.settings)"
```
Tokens:
Raw