Zapi SDK

https://zpi.web.id/docs
Admin
The Zapi SDK is the official TypeScript SDK for the Zapi scraper catalog, providing typed end-to-end...
Tokens:7,082
Snippets:59
Trust Score:2.3
Update:3 months ago
Show doc for...
Tokens:
Raw
### Install zpi-sdk with npm

Source: https://zpi.web.id/docs

Install the zpi-sdk package using npm.

```bash
npm i zpi-sdk
```

--------------------------------

### Install zpi-sdk with bun

Source: https://zpi.web.id/docs

Install the zpi-sdk package using bun.

```bash
bun add zpi-sdk
```

--------------------------------

### Install zpi-sdk with pnpm

Source: https://zpi.web.id/docs

Install the zpi-sdk package using pnpm.

```bash
pnpm add zpi-sdk
```

--------------------------------

### Install zpi-sdk with yarn

Source: https://zpi.web.id/docs

Install the zpi-sdk package using yarn.

```bash
yarn add zpi-sdk
```

--------------------------------

### Quickstart Zapi Client Usage

Source: https://zpi.web.id/docs

Create a ZpiClient instance with an API key and run a scraper job. The SDK automatically detects the correct HTTP verb for the given endpoint.

```typescript
import { ZpiClient } from "zpi-sdk";

const client = new ZpiClient({ apiKey: "zpi_..." });

// projectKey is "category:scraper" — no HTTP method needed,
// the SDK detects and remembers the right verb per endpoint
const profile = await client.run("social:instagram", "profile", {
  username: "instagram",
});
console.log(profile);
```

--------------------------------

### List, Get, and Schema Catalog Items

Source: https://zpi.web.id/docs

Demonstrates fetching a list of catalog items with filtering, retrieving details for a specific scraper, and getting the schema for an endpoint. No authentication is required for these operations.

```typescript
const { items } = await client.catalog.list({ cat: "social", limit: 20 });
const detail = await client.catalog.get("social:instagram");
const schema = await client.catalog.schema("social:instagram", "profile");
```

--------------------------------

### run() Method

Source: https://zpi.web.id/docs

Issues a single scraper call to the ZPI API. The SDK auto-detects the HTTP method for the endpoint, which is then memoized. GET parameters are sent as the query string, and POST parameters as the JSON body. Path parameters are included as fields in the `params` object.

```APIDOC
## run<T = unknown>(projectKey, endpoint, params?, opts?): Promise<T>

### Description
Issues a single scraper call to the ZPI API. The request URL is `{baseURL}/v1/{projectKey}/{endpoint}[/{pathRest}]`. The SDK auto-detects the endpoint's HTTP verb, which is memoized per endpoint. GET parameters become the query string, POST parameters the JSON body. Path parameters like `:slug` are plain fields in `params`.

### Method
Auto-detected (and memoized) per endpoint, or explicitly set via `opts.method`.

### Endpoint
`{baseURL}/v1/{projectKey}/{endpoint}[/{pathRest}]`

### Parameters
#### Path Parameters
- **projectKey** (string) - Required - The key of the project to run.
- **endpoint** (string) - Required - The specific endpoint within the project.
- **pathRest** (string) - Optional - Appended to the URL after the endpoint segment.

#### Query Parameters
(Derived from `params` for GET requests)

#### Request Body
(Derived from `params` for POST requests)

#### `params` Object
- **(any field)** (any) - Used for path parameters (e.g., `slug`) or request body fields.

#### `opts` Object
- **method** ("GET" | "POST") - Optional - Override for the auto-detected HTTP method.
- **signal** (AbortSignal) - Optional - External abort signal.
- **timeoutMs** (number) - Optional - Per-call timeout override.
- **idempotencyKey** (string) - Optional - Required for POST requests to be retry-eligible.
- **headers** (Record<string, string>) - Optional - Per-request header overrides.

### Request Example
```typescript
// Auto-detected method
const spot = await client.run("finance:goldprice", "spot", {});

// Path parameters
await client.run("finance:goldprice", "article", { slug: "gold-hits-ath" });

// Explicit method
const data = await client.run(
  "social:instagram",
  "profile",
  { username: "instagram" },
  { method: "GET" }
);
```

### Response
#### Success Response (200)
- **T** (any) - The deserialized response data, typed as `T` (defaults to `unknown`).

#### Response Example
(Varies based on the endpoint)
```

--------------------------------

### Get Scraper Details

Source: https://zpi.web.id/docs

Retrieves the full details for a specific scraper using its slug.

```APIDOC
## GET /api/scrapers/{slug}

### Description
Retrieves the full details for a specific scraper using its slug.

### Method
GET

### Endpoint
/api/scrapers/{slug}

### Parameters
#### Path Parameters
- **slug** (string) - Required - The slug of the scraper (e.g., "social:instagram" or "instagram").

### Response
#### Success Response (200)
- Returns the full scraper detail object.

### Request Example
```json
{
  "example": "client.catalog.get(\"social:instagram\")"
}
```

### Response Example
```json
{
  "example": "{ ...scraper details... }"
}

```

--------------------------------

### Get Bulk Job Status

Source: https://zpi.web.id/docs

Retrieves the status and results of a previously submitted bulk job.

```APIDOC
## GET /v1/bulk/{jobId}

### Description
Retrieves the status and results of a bulk scraping job using its unique job ID.

### Method
GET

### Endpoint
/v1/bulk/{jobId}

### Parameters
#### Path Parameters
- **jobId** (string) - Required - The unique identifier for the bulk job.

### Response
(Response details for bulk job status not explicitly defined in source.)
```

--------------------------------

### Get Endpoint Schema

Source: https://zpi.web.id/docs

Retrieves the field schema for a specific endpoint of a scraper.

```APIDOC
## GET /api/public/scrapers/{slug}/endpoints/{endpoint}/schema

### Description
Retrieves the field schema for a specific endpoint of a scraper.

### Method
GET

### Endpoint
/api/public/scrapers/{slug}/endpoints/{endpoint}/schema

### Parameters
#### Path Parameters
- **slug** (string) - Required - The slug of the scraper (e.g., "social:instagram" or "instagram").
- **endpoint** (string) - Required - The name of the endpoint (e.g., "profile").

### Response
#### Success Response (200)
- **fields** (object) - An object describing the fields for the endpoint.

### Request Example
```json
{
  "example": "client.catalog.schema(\"social:instagram\", \"profile\")"
}
```

### Response Example
```json
{
  "example": "{ fields: { username: { type: \"string\" } } }"
}

```

--------------------------------

### Get Scraper Statistics

Source: https://zpi.web.id/docs

Retrieves statistics for a specific scraper, including request counts and success rate.

```APIDOC
## GET /api/public/scrapers/{slug}/stats

### Description
Retrieves statistics for a specific scraper, including request counts and success rate.

### Method
GET

### Endpoint
/api/public/scrapers/{slug}/stats

### Parameters
#### Path Parameters
- **slug** (string) - Required - The slug of the scraper (e.g., "social:instagram" or "instagram").

### Response
#### Success Response (200)
- **requests** (number) - The total number of requests made to the scraper.
- **successRate** (number) - The success rate of the scraper's requests.

### Request Example
```json
{
  "example": "client.catalog.stats(\"social:instagram\")"
}
```

### Response Example
```json
{
  "example": "{ requests: 1000, successRate: 0.95 }"
}

```

--------------------------------

### Initialize ZpiClient with API Key

Source: https://zpi.web.id/docs

Instantiate the ZpiClient with your API key. The API key is required and sent as the 'x-api-key' header. The key is stored in a private field and is not exposed through serialization.

```typescript
import { ZpiClient } from "zpi-sdk";

const client = new ZpiClient({ apiKey: process.env.ZPI_API_KEY });

// The key lives in a private field and never leaks through serialization:
JSON.stringify(client); // "{}"
```

--------------------------------

### Import ZpiClient in Deno

Source: https://zpi.web.id/docs

Import the ZpiClient from the npm registry for use in Deno.

```typescript
import { ZpiClient } from "npm:zpi-sdk";
```

--------------------------------

### Create and Use MCP Client

Source: https://zpi.web.id/docs

Instantiate the MCP client with your API key and base URL to interact with the Zapi MCP server. This client supports listing available tools and calling them with specified parameters.

```typescript
import { createMcpClient } from "zpi-sdk/mcp";

const mcp = createMcpClient({
  apiKey: "zpi_...",
  baseURL: "https://api.zpi.web.id",
});
const tools = await mcp.listTools();
const result = await mcp.callTool("run_scraper", { /* ... */ });
```

--------------------------------

### client.bulk.submit(projectKey, endpoint, items)

Source: https://zpi.web.id/docs

Submits a bulk job by POSTing to the /v1/{projectKey}/{endpoint}/bulk endpoint. Automatically attaches an Idempotency-Key for safe retries. Returns a BulkJob handle.

```APIDOC
## client.bulk.submit(projectKey, endpoint, items)

### Description
Submits a bulk job by POSTing to the `/v1/{projectKey}/{endpoint}/bulk` endpoint. An `Idempotency-Key` is automatically attached to prevent duplicate submissions during retries. Returns a `BulkJob` handle.

### Parameters
- **projectKey** (string) - Required - The key of the project.
- **endpoint** (string) - Required - The API endpoint for the bulk operation.
- **items** (array) - Required - An array of items to process in the bulk job.

### Returns
- **BulkJob** - A handle to the submitted bulk job.
```

--------------------------------

### Handle ZPI SDK Errors

Source: https://zpi.web.id/docs

Illustrates how to catch and differentiate between various ZPI SDK errors, such as plan gate errors, rate limit errors, and general ZPI errors, using instanceof checks. This is crucial for robust API interaction.

```typescript
import {
  ZpiError,
  ZpiPlanGateError,
  ZpiRateLimitError,
} from "zpi-sdk";

try {
  const data = await client.run("social:instagram", "profile", {
    username: "instagram",
  });
} catch (e) {
  if (e instanceof ZpiPlanGateError) {
    console.log("upgrade:", e.requiredPlan, e.upgradeUrl);
  } else if (e instanceof ZpiRateLimitError) {
    console.log("retry after", e.retryAfterSec, "s");
  } else if (e instanceof ZpiError) {
    console.log(e.status, e.code, e.raw);
  }
}
```

--------------------------------

### Generate ZPI SDK Types CLI

Source: https://zpi.web.id/docs

Use the codegen CLI to crawl the ZPI catalog and emit a types-only .d.ts file. This enhances type safety for your ZPI integrations.

```bash
npx zpi codegen --base https://api.zpi.web.id --out ./zpi-sdk.gen.d.ts

# narrow to one category, authenticate for gated schemas
npx zpi codegen --filter social --key zpi_xxx
```

--------------------------------

### createMcpClient

Source: https://zpi.web.id/docs

Creates a JSON-RPC client for the remote Zapi MCP server. This client allows for making calls to server-side tools and retrieving information.

```APIDOC
## createMcpClient

### Description
Creates a JSON-RPC client for the remote Zapi MCP server. This client allows for making calls to server-side tools and retrieving information.

### Method
Not applicable (SDK function)

### Endpoint
Not applicable (SDK function)

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None (SDK function operates on configuration object)

### Request Example
```typescript
import { createMcpClient } from "zpi-sdk/mcp";

const mcp = createMcpClient({
  apiKey: "zpi_...",
  baseURL: "https://api.zpi.web.id",
});
const tools = await mcp.listTools();
const result = await mcp.callTool("run_scraper", { /* ... */ });
```

### Response
#### Success Response
- **listTools**: Returns a list of available tools.
- **callTool**: Returns the result of the executed tool.

#### Response Example
```json
// Example for listTools:
["tool1", "tool2"]

// Example for callTool (depends on the tool called):
{
  "output": "..."
}
```

### Error Handling
RPC failures throw `ZpiMcpError`.
```

--------------------------------

### Execute a Single Scraper Call with run()

Source: https://zpi.web.id/docs

Use the `run()` method to issue a single scraper call. The SDK auto-detects the endpoint's HTTP method, which is then memoized. Path parameters are handled by including them as fields in the `params` object.

```typescript
// No method to pick — auto-detected (and memoized) per endpoint.
const spot = await client.run("finance:goldprice", "spot", {});

// Endpoints with path params (e.g. article/:slug)? Path params are
// just regular fields in params — all of these are equivalent:
await client.run("finance:goldprice", "article", { slug: "gold-hits-ath" });
await client.run("finance:goldprice", "article/:slug", { slug: "gold-hits-ath" });

// Explicit method still works and skips auto-detection:
const data = await client.run(
  "social:instagram",
  "profile",
  { username: "instagram" },
  { method: "GET" }
);
```

--------------------------------

### Bulk Scrape Data

Source: https://zpi.web.id/docs

Initiates bulk scraping jobs for multiple requests.

```APIDOC
## POST /v1/{category:scraper}/{endpoint}/bulk

### Description
This endpoint allows for initiating multiple scraping jobs in a single request.

### Method
POST

### Endpoint
/v1/{category:scraper}/{endpoint}/bulk

### Parameters
#### Path Parameters
- **category** (string) - Required - The category of the scraper (e.g., 'social').
- **endpoint** (string) - Required - The specific endpoint for the scraping job (e.g., 'profile').

### Request Example
(Request body for bulk jobs not explicitly defined in source, but would typically be an array of objects similar to the single request.)

### Response
(Response details for bulk jobs not explicitly defined in source.)
```

--------------------------------

### stream(projectKey, endpoint, params?, opts?)

Source: https://zpi.web.id/docs

Returns an AsyncIterable for SSE and chunked endpoints. SSE responses yield SseEvent objects, while other bodies yield raw text chunks. Streams are never retried and will throw an error if broken.

```APIDOC
## stream(projectKey, endpoint, params?, opts?)

### Description
Returns an AsyncIterable for Server-Sent Events (SSE) and chunked endpoints. SSE responses yield `SseEvent` objects (`{ event?, data, id? }`); any other body yields raw text chunks. Streams are never retried and will throw an error if broken.

### Parameters
- **projectKey** (string) - Required - The key of the project.
- **endpoint** (string) - Required - The API endpoint to stream from.
- **params** (object) - Optional - Parameters to send with the request.
- **opts** (object) - Optional - Options for the stream, excluding `idempotencyKey`.

### Returns
- **AsyncIterable<StreamEvent>** - An asynchronous iterable yielding stream events.
```

--------------------------------

### Submit and Monitor Bulk Job

Source: https://zpi.web.id/docs

Submit a bulk job for processing and wait for its completion. The SDK automatically adds an idempotency key for safe retries. The `wait` method polls for job status with configurable options.

```typescript
const job = await client.bulk.submit("social:instagram", "profile", [
  { url: "https://instagram.com/a" },
  { url: "https://instagram.com/b" },
]);

const result = await job.wait({
  onProgress: (j) => console.log(j.succeeded, "/", j.total),
});
for (const item of result.items ?? []) {
  console.log(item.status, item.data ?? item.error);
}
```

--------------------------------

### List Catalog Categories

Source: https://zpi.web.id/docs

Retrieves all available categories in the public scraper catalog.

```APIDOC
## GET /api/public/categories

### Description
Retrieves all available categories in the public scraper catalog.

### Method
GET

### Endpoint
/api/public/categories

### Response
#### Success Response (200)
- Returns an array of category objects.

### Request Example
```json
{
  "example": "client.catalog.categories()"
}
```

### Response Example
```json
{
  "example": "[ \"social\", \"finance\" ]"
}

```

--------------------------------

### job.wait({ signal?, timeoutMs?, pollIntervalMs?, onProgress? })

Source: https://zpi.web.id/docs

Polls a bulk job with backoff until it reaches a terminal state. Allows for progress monitoring and cancellation.

```APIDOC
## job.wait({ signal?, timeoutMs?, pollIntervalMs?, onProgress? })

### Description
Polls a bulk job with exponential backoff (base 500 ms, capped at 5 s) until the job is terminal (`QUEUED | RUNNING | COMPLETED | FAILED | CANCELLED`). Allows for progress monitoring and cancellation via `AbortSignal`.

### Parameters
- **signal** (AbortSignal) - Optional - Signal to abort the polling.
- **timeoutMs** (number) - Optional - Maximum time in milliseconds to wait for the job.
- **pollIntervalMs** (number) - Optional - The base interval in milliseconds for polling.
- **onProgress** (function) - Optional - A callback function to receive job progress updates.

### Returns
- **BulkJobResult** - The result of the bulk job once it has reached a terminal state.
```

--------------------------------

### Parse and Verify ZPI Webhook

Source: https://zpi.web.id/docs

Use this function to parse and verify incoming webhook deliveries. Ensure you pass the raw request body and the correct signature and secret. It handles signature verification using Web Crypto and provides a typed event object.

```typescript
import { parseWebhook } from "zpi-sdk/webhooks";

// In your HTTP handler — pass the RAW body string, not parsed JSON:
const event = await parseWebhook(rawBody, {
  signature: req.headers["x-zpi-signature"],
  secret: process.env.ZPI_WEBHOOK_SECRET,
});
// → { id, event: "bulk.completed" | ..., data, deliveredAt }
// Throws ZpiWebhookVerifyError on a bad signature or malformed payload.

switch (event.event) {
  case "bulk.completed":
    console.log("job done:", event.data);
    break;
  case "quota.warning":
    console.log("almost out of quota");
    break;
}
```

--------------------------------

### Make a POST Request to the ZPI Web ID API

Source: https://zpi.web.id/docs

Use this cURL command to send a POST request to the ZPI Web ID API for scraping social media profiles. Ensure you replace 'zpi_xxx' with your actual API key. The request body should be a JSON object containing the username.

```curl
curl -X POST "https://api.zpi.web.id/v1/social:instagram/profile" \
  -H "x-api-key: zpi_xxx" \
  -H "content-type: application/json" \
  -d '{"username":"instagram"}'
```

--------------------------------

### parseWebhook

Source: https://zpi.web.id/docs

Verifies and parses incoming webhook deliveries from ZPI. It ensures the signature is valid using the provided secret and the raw request body.

```APIDOC
## parseWebhook

### Description
Verifies and parses incoming webhook deliveries from ZPI. It ensures the signature is valid using the provided secret and the raw request body.

### Method
Not applicable (SDK function)

### Endpoint
Not applicable (SDK function)

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None (SDK function operates on raw body and headers)

### Request Example
```typescript
import { parseWebhook } from "zpi-sdk/webhooks";

// In your HTTP handler — pass the RAW body string, not parsed JSON:
const event = await parseWebhook(rawBody, {
  signature: req.headers["x-zpi-signature"],
  secret: process.env.ZPI_WEBHOOK_SECRET,
});
// → { id, event: "bulk.completed" | ..., data, deliveredAt }
// Throws ZpiWebhookVerifyError on a bad signature or malformed payload.
```

### Response
#### Success Response
- **event** (object) - Contains event details, including event type and data.

#### Response Example
```json
{
  "id": "string",
  "event": "bulk.completed" | "bulk.failed" | "bulk.item" | "quota.warning" | "quota.exceeded" | "request.error" | "key.created" | "key.disabled" | "key.deleted" | "webhook.test",
  "data": {},
  "deliveredAt": "string"
}
```

### Error Handling
Throws `ZpiWebhookVerifyError` on a bad signature or malformed payload.
```

--------------------------------

### Scrape Profile Data

Source: https://zpi.web.id/docs

Retrieves profile data for a specified username from a given category (e.g., social:instagram).

```APIDOC
## POST /v1/{category:scraper}/{endpoint}

### Description
This endpoint initiates a scraping job for a specific category and endpoint, typically used to retrieve profile information.

### Method
POST

### Endpoint
/v1/{category:scraper}/{endpoint}

### Parameters
#### Path Parameters
- **category** (string) - Required - The category of the scraper (e.g., 'social').
- **endpoint** (string) - Required - The specific endpoint for the scraping job (e.g., 'profile').

#### Request Body
- **username** (string) - Required - The username for which to scrape data.

### Request Example
```json
{
  "username":"instagram"
}
```

### Response
#### Success Response (200)
- **project** (string) - The project identifier.
- **data** (object) - The scraped data.
- **timestamp** (string) - The timestamp of the response.

#### Response Example
```json
{
  "project": "social:instagram",
  "data": { ... },
  "timestamp": "..."
}
```
```

--------------------------------

### Stream SSE or Chunked Responses

Source: https://zpi.web.id/docs

Iterate over SSE events or raw text chunks from an endpoint. SSE responses yield SseEvent objects, while others yield raw strings. Streams are never retried and throw on failure.

```typescript
for await (const event of client.stream("ai:chat", "completions", {
  prompt: "hi",
})) {
  // SSE responses yield { event?, data, id? }; other bodies yield raw text
  console.log(typeof event === "string" ? event : event.data);
}
```

--------------------------------

### Augmenting ZPI SDK Types

Source: https://zpi.web.id/docs

Augment the ScraperMap interface to provide type safety for specific scrapers. This is typically done via codegen but can be manually defined.

```typescript
// The export {} makes this file a module AUGMENTATION —
// without it the block would shadow the real package types.
export {};

declare module "zpi-sdk" {
  interface ScraperMap {
    "social:instagram": {
      profile: {
        params: { username: string };
        result: { id: string; name: string };
      };
    };
  }
}
```

--------------------------------

### List Scrapers

Source: https://zpi.web.id/docs

Retrieves a cursor-paginated list of public scrapers. Supports filtering by category and limiting the number of results.

```APIDOC
## GET /api/scrapers/list

### Description
Retrieves a cursor-paginated list of public scrapers. Supports filtering by category and limiting the number of results.

### Method
GET

### Endpoint
/api/scrapers/list

### Parameters
#### Query Parameters
- **q** (string) - Optional - Query string for searching scrapers.
- **cat** (string) - Optional - Filter by category.
- **cursor** (string) - Optional - Cursor for pagination.
- **limit** (number) - Optional - Maximum number of items to return.

### Response
#### Success Response (200)
- **items** (array) - List of scraper objects.
- **nextCursor** (string) - Cursor for the next page of results.
- **total** (number) - Total number of scrapers available.

### Request Example
```json
{
  "example": "client.catalog.list({ cat: \"social\", limit: 20 })"
}
```

### Response Example
```json
{
  "example": "{ items: [...], nextCursor: \"...\", total: 100 }"
}

```

--------------------------------

### client.bulk.status(jobId)

Source: https://zpi.web.id/docs

Polls for the status of a specific bulk job using its jobId.

```APIDOC
## client.bulk.status(jobId)

### Description
Polls for the status of a specific bulk job using its `jobId`.

### Parameters
- **jobId** (string) - Required - The ID of the bulk job to poll.

### Returns
- **BulkJob** - The current status of the bulk job.
```
Tokens:
Raw