### Install dpkit with npm

Source: https://github.com/datisthq/dpkit/blob/main/docs/content/docs/index.md

Installs the dpkit framework with its command-line interface using npm. Adjust commands for other package managers.

```bash
npm install dpkit
```

--------------------------------

### Install dpkit CLI via Node.js (npm)

Source: https://github.com/datisthq/dpkit/blob/main/site/content/docs/overview/getting-started.md

Installs the dpkit command-line tool globally using the Node Package Manager (npm). This requires Node.js and npm to be installed on the system. After global installation, the `dp` command is available system-wide.

```bash
npm install -g dpkit
```

--------------------------------

### Install dpkit CLI via Binary (Shell)

Source: https://github.com/datisthq/dpkit/blob/main/site/content/docs/overview/getting-started.md

Installs the dpkit command-line tool by downloading and executing an installation script from a URL. This method is suitable for POSIX-compatible shells. It requires `curl` to fetch the script and `sh` to execute it. After installation, the binary can be run directly.

```sh
curl -fsSL https://dpkit.dev/install.sh | sh
```

--------------------------------

### Verify dpkit CLI Installation (Node.js)

Source: https://github.com/datisthq/dpkit/blob/main/site/content/docs/overview/getting-started.md

Verifies that the dpkit command-line tool has been successfully installed globally via npm by checking its version. This command can be run from any directory after the global installation.

```bash
dp --version
```

--------------------------------

### Install specific dpkit packages

Source: https://github.com/datisthq/dpkit/blob/main/docs/content/docs/index.md

Installs individual dpkit packages, such as core functionalities and integrations like Zenodo. Commands should be modified for different package managers.

```bash
npm install @dpkit/core @dpkit/zenodo
```

--------------------------------

### Verify dpkit Binary Installation (Shell)

Source: https://github.com/datisthq/dpkit/blob/main/site/content/docs/overview/getting-started.md

Verifies that the dpkit command-line tool has been successfully installed by checking its version. This command is executed from the directory where the binary was placed. It's recommended to add the binary's location to the system's PATH environment variable for easier access.

```sh
./dp --version
```

--------------------------------

### Install dpkit library only

Source: https://github.com/datisthq/dpkit/blob/main/docs/content/docs/index.md

Installs the core dpkit library without the CLI, useful for projects that do not require command-line utilities. Adapt commands for alternative package managers.

```bash
npm install @dpkit/lib
```

--------------------------------

### Load and log a Data Package from Zenodo

Source: https://github.com/datisthq/dpkit/blob/main/docs/content/docs/index.md

Loads a Data Package from a Zenodo record URL and logs the package descriptor to the console. Requires the 'dpkit' package to be installed.

```typescript
import { loadPackage } from "dpkit"

const { dataPackage } = await loadPackage("https://zenodo.org/records/10053903")

console.log(dataPackage)

```

--------------------------------

### Publishing Workflow Example

Source: https://github.com/datisthq/dpkit/blob/main/site/content/docs/guides/package.md

Demonstrates a typical publishing workflow, starting with validation of the local data package descriptor before publishing to CKAN.

```bash
# Validate before publishing
dp package validate datapackage.json

# Publish to CKAN
dp package publish ckan datapackage.json \
  --to-ckan-url https://your-ckan-instance.org \
  --to-ckan-api-key $CKAN_API_KEY
```

--------------------------------

### Import dpkit core package in browser

Source: https://github.com/datisthq/dpkit/blob/main/docs/content/docs/index.md

Imports the core dpkit package in a browser environment using NPM CDNs, enabling direct use of its functionalities in client-side applications.

```javascript
import { loadPackageDescriptor } from "https://esm.sh/@dpkit/core"

```

--------------------------------

### Install Dependencies with PNPM

Source: https://github.com/datisthq/dpkit/blob/main/CONTRIBUTING.md

Installs all project dependencies using PNPM, the recommended package manager for this project. This command should be run after cloning the repository.

```bash
pnpm install
```

--------------------------------

### DPKit Integration Examples

Source: https://github.com/datisthq/dpkit/blob/main/site/content/docs/guides/table.md

These examples show how DPKit commands can be integrated with each other. They cover using package commands, schema commands, and dialect commands in conjunction for comprehensive data management workflows.

```bash
# Create and validate package
dp package infer *.csv --json > datapackage.json
dp table validate --from-package datapackage.json --from-resource "main"

# Infer schema and validate table
dp schema infer data.csv --json > schema.json
dp table validate data.csv --schema schema.json

# Infer dialect and use for table operations
dp dialect infer data.csv --json > dialect.json
dp table explore data.csv --dialect dialect.json

```

--------------------------------

### Run CLI in Dev Mode with Bun

Source: https://github.com/datisthq/dpkit/blob/main/docs/content/docs/overview/contributing.md

Example of how to run the dpkit CLI in development mode using Bun. This is recommended for CLI compilation and development.

```bash
bun cli/main.ts
```

--------------------------------

### Install @dpkit/arrow Package

Source: https://github.com/datisthq/dpkit/blob/main/docs/content/docs/guides/arrow.md

This command installs the @dpkit/arrow package using npm. Ensure you have Node.js and npm installed on your system.

```bash
npm install @dpkit/arrow
```

--------------------------------

### Connect to Different Databases - TypeScript

Source: https://github.com/datisthq/dpkit/blob/main/docs/content/docs/guides/database.md

Demonstrates establishing connections to various database types (SQLite, PostgreSQL, MySQL) using their respective connection string formats. Each example specifies the database path and the target table.

```typescript
// SQLite
const sqliteTable = await loadDatabaseTable({
  path: "sqlite://data.db",
  dialect: { table: "products" }
})

// PostgreSQL
const pgTable = await loadDatabaseTable({
  path: "postgresql://user:password@localhost:5432/mydb",
  dialect: { table: "orders" }
})

// MySQL
const mysqlTable = await loadDatabaseTable({
  path: "mysql://user:password@localhost:3306/mydb",
  dialect: { table: "customers" }
})
```

--------------------------------

### File Not Found Error Example - dp CLI

Source: https://github.com/datisthq/dpkit/blob/main/site/content/docs/guides/file.md

Illustrates the error message and solution when the `dp file describe` command is used on a non-existent file.

```bash
dp file describe missing_file.csv
# Error: File not found
# Solution: Check file path and permissions
```

--------------------------------

### Install Deno Runtime

Source: https://github.com/datisthq/dpkit/blob/main/docs/content/docs/guides/jupyter.md

Downloads and installs the Deno runtime, which is necessary for executing TypeScript code within Jupyter notebooks. Refer to Deno's official documentation for more details.

```bash
curl -fsSL https://deno.land/install.sh | sh
```

--------------------------------

### Install @dpkit/parquet Package

Source: https://github.com/datisthq/dpkit/blob/main/docs/content/docs/guides/parquet.md

Installs the @dpkit/parquet package using npm. This package is essential for utilizing Parquet file functionalities within your project.

```bash
npm install @dpkit/parquet
```

--------------------------------

### Load a CSV table with default settings

Source: https://github.com/datisthq/dpkit/blob/main/docs/content/docs/index.md

Loads a CSV file named 'data.csv' into a table format using default parsing settings. Requires the 'dpkit' package.

```typescript
import { loadTable } from "dpkit"

const table = await loadTable({ path: "data.csv" })

```

--------------------------------

### Load, save, and load Data Package from zip

Source: https://github.com/datisthq/dpkit/blob/main/docs/content/docs/index.md

Loads a Data Package descriptor from a remote URL, saves it to a local zip archive, and then loads it back as a local data package. Utilizes functions from the 'dpkit' package.

```typescript
import {
  loadPackageDescriptor,
  loadPackageFromZip,
  savePackageToZip,
  getTempFilePath,
} from "dpkit"

const archivePath = getTempFilePath()
const sourcePath = await loadPackageDescriptor(
  "https://raw.githubusercontent.com/roll/currency-codes/refs/heads/master/datapackage.json",
)

await savePackageToZip(sourcePackage, { archivePath })
const targetPackage = await loadPackageFromZip(archivePath)
console.log(targetPackage)

```

--------------------------------

### Install dpkit/json package

Source: https://github.com/datisthq/dpkit/blob/main/docs/content/docs/guides/json.md

Install the @dpkit/json package using npm. This package is part of dpkit's modular architecture and provides functionality for handling JSON data.

```bash
npm install @dpkit/json
```

--------------------------------

### Install Deno Jupyter Kernel

Source: https://github.com/datisthq/dpkit/blob/main/docs/content/docs/guides/jupyter.md

Sets up the Deno kernel for Jupyter, which enables TypeScript support within Jupyter notebooks. This step is crucial for running TypeScript code.

```bash
deno jupyter --install
```

--------------------------------

### Work with Package Dialects Workflow - Bash

Source: https://github.com/datisthq/dpkit/blob/main/site/content/docs/guides/dialect.md

Examples of working with dialects within datapackages: validating all dialects interactively, inferring an improved dialect for a specific resource, and using scripting to compare dialects.

```bash
# Validate all dialects in a package interactively
dp dialect validate --from-package datapackage.json

# Infer improved dialect for specific resource
dp dialect infer --from-package datapackage.json --from-resource "transactions"

# Compare dialects using scripting
dp dialect script --from-package datapackage.json --from-resource "users"
```

--------------------------------

### Copy Data Package and Load in VisiData (Bash)

Source: https://github.com/datisthq/dpkit/blob/main/docs/content/docs/guides/visidata.md

This snippet demonstrates how to use the dpkit CLI to copy a remote Data Package to a local folder and subsequently load the CSV files into VisiData for exploration. It requires both dpkit and VisiData to be installed.

```bash
dpkit copy https://zenodo.org/records/7559361 --toFolder dataset --withRemote
vd dataset/*.csv
```

--------------------------------

### Load a CSV table with custom dialect

Source: https://github.com/datisthq/dpkit/blob/main/docs/content/docs/index.md

Loads a CSV file specifying a custom dialect, including delimiter, header presence, and handling of initial spaces. Requires the 'dpkit' package.

```typescript
// Load with custom dialect
const table = await loadTable({
  path: "data.csv",
  dialect: {
    delimiter: ";",
    header: true,
    skipInitialSpace: true
  }
})

```

--------------------------------

### Network Issues Error Example - dp CLI

Source: https://github.com/datisthq/dpkit/blob/main/site/content/docs/guides/file.md

Shows an example of a network error that can occur when using `dp file copy` with an unreachable remote source, along with a suggested solution.

```bash
dp file copy https://unreachable.com/data.csv local.csv
# Error: Network timeout
# Solution: Check URL and network connectivity
```

--------------------------------

### Install Jupyter Notebook

Source: https://github.com/datisthq/dpkit/blob/main/docs/content/docs/guides/jupyter.md

Installs Jupyter Notebook, a web-based interactive computing environment for data science and data engineering. Supports alternative UIs like Jupyter CLI or Jupyter Desktop.

```bash
pip install jupyterlab
```

--------------------------------

### Format Recognition Limitation Example - dp CLI

Source: https://github.com/datisthq/dpkit/blob/main/site/content/docs/guides/file.md

Highlights a potential limitation where `dp file describe` might show limited information for unknown file formats, suggesting the use of `--debug` for more details.

```bash
dp file describe unknown_format.dat
# May show limited information for unknown formats
# Solution: Use --debug for more details
```

--------------------------------

### Select ODS Sheets by Number or Name

Source: https://github.com/datisthq/dpkit/blob/main/docs/content/docs/guides/ods.md

Demonstrates advanced sheet selection when loading ODS files. It provides examples for choosing a sheet by its 1-indexed number or by its specific name.

```typescript
import { loadOdsTable } from "@dpkit/ods"

// Select by sheet number (1-indexed)
const table = await loadOdsTable({
  path: "workbook.ods",
  dialect: {
    sheetNumber: 2  // Load second sheet
  }
})

// Select by sheet name
const table = await loadOdsTable({
  path: "workbook.ods",
  dialect: {
    sheetName: "Sales Data"
  }
})
```

--------------------------------

### Custom Parsing with DPKit Dialect

Source: https://github.com/datisthq/dpkit/blob/main/site/content/docs/guides/table.md

This demonstrates using custom dialect definitions for parsing complex or non-standard data formats. You can infer a dialect or provide a custom JSON file to guide the parsing process.

```bash
# Use custom dialect for parsing
dp table explore data.csv --dialect custom_dialect.json

# Convert with parsing options
dp table convert complex_data.csv output.xlsx --dialect dialect.json

```

--------------------------------

### Save ODS Table with Default and Custom Options

Source: https://github.com/datisthq/dpkit/blob/main/docs/content/docs/guides/ods.md

Shows how to save a DataFrame to an ODS file using `saveOdsTable`. It includes examples for saving with default settings and customizing the output by specifying a sheet name.

```typescript
import { saveOdsTable } from "@dpkit/ods"

// Save with default options
await saveOdsTable(table, { path: "output.ods" })

// Save with custom sheet name
await saveOdsTable(table, {
  path: "output.ods",
  dialect: {
    sheetName: "Data"
  }
})
```

--------------------------------

### Monitor File Changes Using dp file describe and cmp

Source: https://github.com/datisthq/dpkit/blob/main/site/content/docs/guides/file.md

A bash script that continuously monitors a file for changes. It uses 'dp file describe --json' to get the current state, 'cmp' to compare it with the previous state, and logs changes along with a timestamp. Includes a sleep interval to control monitoring frequency.

```bash
# Monitor file changes
while true; do
  dp file describe changing_file.csv --json > current_state.json
  if ! cmp -s current_state.json previous_state.json; then
    echo "File changed at $(date)"
    cp current_state.json previous_state.json
  fi
  sleep 60
done
```

--------------------------------

### Log File Validations with JSON Output

Source: https://github.com/datisthq/dpkit/blob/main/site/content/docs/guides/file.md

This example demonstrates how to capture validation results in JSON format using 'dp file validate --json' and then append specific details including the filename, validity status, and timestamp to a log file using 'jq'.

```bash
# Create validation log
dp file validate data.csv --json | jq '{file: "data.csv", valid: .valid, timestamp: now}' >> validation.log
```

--------------------------------

### Validate an in-memory package descriptor

Source: https://github.com/datisthq/dpkit/blob/main/docs/content/docs/index.md

Validates a package descriptor object in memory using dpkit's validation function. Outputs whether the descriptor is valid and lists any errors found.

```typescript
import { validatePackageDescriptor } from "dpkit"

const { valid, errors } = await validatePackageDescriptor({ name: "package" })

console.log(valid)
// false
console.log(errors)

```

--------------------------------

### File Backup Workflow - dp CLI

Source: https://github.com/datisthq/dpkit/blob/main/site/content/docs/guides/file.md

Demonstrates a common workflow for backing up files using copy, validate, and describe commands. Includes timestamping for backup files.

```bash
# Create backup copy
dp file copy important_data.csv backup/important_data_$(date +%Y%m%d).csv

# Validate backup integrity
dp file validate backup/important_data_20240101.csv

# Describe backup properties
dp file describe backup/important_data_20240101.csv
```

--------------------------------

### Resource Selection within Packages

Source: https://github.com/datisthq/dpkit/blob/main/site/content/docs/guides/resource.md

Demonstrates methods for selecting specific resources when working with data packages. This includes explicit specification using `--from-resource` and interactive selection prompted by the command.

```bash
# Explicitly specify resource:
dp resource explore --from-package datapackage.json --from-resource "users"

# Interactive selection (will prompt):
dp resource validate --from-package datapackage.json
```

--------------------------------

### Integration with Package Creation

Source: https://github.com/datisthq/dpkit/blob/main/site/content/docs/guides/resource.md

Shows how resource commands integrate with package commands. This workflow involves inferring and creating a datapackage.json from data files, and then using resource commands to validate and explore individual resources within that package.

```bash
# Create package, then work with individual resources
dp package infer *.csv --json > datapackage.json
dp resource validate --from-package datapackage.json --from-resource "data"
dp resource explore --from-package datapackage.json --from-resource "users"
```

--------------------------------

### Open Interactive Scripting Session with Data Resource

Source: https://github.com/datisthq/dpkit/blob/main/site/content/docs/guides/resource.md

Launches an interactive REPL environment to programmatically control a loaded data resource. Available variables include `dpkit` and the loaded `resource` object. Supports options for specifying resource location and output format.

```bash
dp resource script <descriptor-path>

# Examples:
dp resource script resource.json
dp resource script --from-package datapackage.json --from-resource "users"

# In the REPL session:
dp> resource.schema.fields.length
dp> resource.schema.fields[0].type
dp> resource.path
```

--------------------------------

### Explore Table with dp CLI

Source: https://github.com/datisthq/dpkit/blob/main/site/content/docs/guides/commands.md

Demonstrates how to explore a table using the `dp table explore` command. It shows variations for specifying the table directly, using a datapackage JSON file interactively, and specifying both the datapackage and resource non-interactively.

```bash
dp table explore table.csv
```

```bash
dp table explore -p datapackage.json
# it will ask you to select a resource
```

```bash
dp table explore -p datapackage.json -r table
```

--------------------------------

### Publish Data Package to Zenodo

Source: https://github.com/datisthq/dpkit/blob/main/site/content/docs/guides/package.md

Publishes a data package to Zenodo, facilitating academic archiving. Requires the path to the data package descriptor.

```bash
dp package publish zenodo <descriptor-path>
```

--------------------------------

### Clone dpkit Repository

Source: https://github.com/datisthq/dpkit/blob/main/CONTRIBUTING.md

Clones the dpkit monorepo from GitHub to your local machine. This is the first step to setting up the development environment.

```bash
git clone https://github.com/yourusername/dpkit.git dpkit
cd dpkit
```

--------------------------------

### Basic Data Exploration with DPKit

Source: https://github.com/datisthq/dpkit/blob/main/site/content/docs/guides/table.md

These commands perform initial data exploration and statistical analysis on CSV files. 'explore' provides a quick overview, 'describe' generates a statistical summary, and 'script' enables interactive analysis.

```bash
dp table explore data.csv

```

```bash
dp table describe data.csv

```

```bash
dp table script data.csv

```

--------------------------------

### Import DPKit Validation Function (Python)

Source: https://github.com/datisthq/dpkit/blob/main/docs/notebooks/metadata.ipynb

Imports the `validatePackageDescriptor` function from the dpkit library in Python. This is a preparatory step before using the validation functionality. It assumes the dpkit library is installed in the Python environment.

```python
from dpkit import validatePackageDescriptor
```

--------------------------------

### Remote File Handling Workflow - dp CLI

Source: https://github.com/datisthq/dpkit/blob/main/site/content/docs/guides/file.md

Illustrates downloading, validating, and describing remote files. Shows how to handle remote data efficiently.

```bash
# Download and validate remote file
dp file copy https://example.com/dataset.csv local_dataset.csv
dp file validate local_dataset.csv

# Describe remote file without downloading
dp file describe https://example.com/dataset.csv
```

--------------------------------

### Handle Comment Rows in XLSX

Source: https://github.com/datisthq/dpkit/blob/main/docs/content/docs/guides/xlsx.md

Details how to manage comment rows when loading XLSX files. `commentRows` allows specifying rows to be skipped, and `commentChar` enables skipping rows that start with a particular character.

```typescript
import { loadXlsxTable } from "@dpkit/xlsx"

// XLSX with comment rows
const table = await loadXlsxTable({
  path: "with-comments.xlsx",
  dialect: {
    commentRows: [1, 2],  // Skip first two rows
    header: true
  }
})

// Skip rows with comment character
const table = await loadXlsxTable({
  path: "data.xlsx",
  dialect: {
    commentChar: "#"  // Skip rows starting with #
  }
})
```

--------------------------------

### Handle Permission Issues with dp file copy

Source: https://github.com/datisthq/dpkit/blob/main/site/content/docs/guides/file.md

Demonstrates how to use the 'dp file copy' command and common permission-denied errors. It advises checking file permissions as a solution.

```bash
dp file copy protected_file.csv backup.csv
# Error: Permission denied
# Solution: Check file permissions
```

--------------------------------

### Explore Data Resource Structure

Source: https://github.com/datisthq/dpkit/blob/main/site/content/docs/guides/resource.md

Explores a data resource from a local or remote path, displaying its structure and metadata interactively. Can also output the structure as JSON. Supports specifying a resource within a package.

```bash
dp resource explore <descriptor-path>

# Explore resource descriptor
dp resource explore resource.json

# Explore remote resource
dp resource explore https://example.com/resource.json

# Explore resource from package
dp resource explore --from-package datapackage.json --from-resource "users"

# Export structure as JSON
dp resource explore resource.json --json
```

--------------------------------

### DPKit Validation and Debugging

Source: https://github.com/datisthq/dpkit/blob/main/site/content/docs/guides/table.md

These commands assist in identifying and resolving data validation and parsing issues. You can get detailed validation reports, explore errors interactively, and debug parsing problems with verbose output.

```bash
# Get detailed validation report
dp table validate data.csv --schema schema.json --json

# Interactive error exploration (don't quit on errors)
dp table validate data.csv --schema schema.json

# Debug parsing issues
dp table explore problematic.csv --debug

# Infer and test dialect
dp dialect infer problematic.csv --json > dialect.json
dp table explore problematic.csv --dialect dialect.json

```

--------------------------------

### Statistical Analysis and Exploration with DPKit

Source: https://github.com/datisthq/dpkit/blob/main/site/content/docs/guides/table.md

These commands focus on generating detailed statistical information about datasets. 'describe' provides comprehensive statistics, while 'script' allows for interactive statistical exploration in a REPL environment.

```bash
# Generate comprehensive statistics
dp table describe large_dataset.csv --json > stats.json

# Interactive statistical exploration
dp table script data.csv
# In REPL: analyze column distributions, correlations, etc.

```

--------------------------------

### Handle Comment Rows in ODS Files

Source: https://github.com/datisthq/dpkit/blob/main/docs/content/docs/guides/ods.md

Details how to manage comment rows within ODS files during loading. Examples include skipping a specified number of initial rows or skipping rows that begin with a specific comment character.

```typescript
import { loadOdsTable } from "@dpkit/ods"

// ODS with comment rows
const table = await loadOdsTable({
  path: "with-comments.ods",
  dialect: {
    commentRows: [1, 2],  // Skip first two rows
    header: true
  }
})

// Skip rows with comment character
const table = await loadOdsTable({
  path: "data.ods",
  dialect: {
    commentChar: "#"  // Skip rows starting with #
  }
})
```

--------------------------------

### Interactive Scripting with Data Package using dp package script

Source: https://github.com/datisthq/dpkit/blob/main/site/content/docs/guides/package.md

Opens an interactive scripting session (REPL) with a loaded Data Package, providing access to the `dpkit` library and the `dataPackage` object for programmatic interaction.

```bash
dp package script <descriptor-path>

# Examples:
# Start scripting session
dp package script datapackage.json

# In the REPL session:
dp> dataPackage.resources.length
dp> dataPackage.resources[0].schema.fields
```

--------------------------------

### Publish Package to Zenodo (TypeScript)

Source: https://context7.com/datisthq/dpkit/llms.txt

Publishes a data package to Zenodo. This operation creates a new deposition, uploads all resource files, and includes the datapackage.json descriptor. Requires a Zenodo API key and configuration for sandbox or production environments.

```typescript
import { savePackageToZenodo } from '@dpkit/zenodo'

const dataPackage = {
  title: 'Research Output 2024',
  description: 'Experimental data from study',
  version: '1.0.0',
  licenses: [{ name: 'CC-BY-4.0' }],
  contributors: [
    { title: 'Jane Researcher', role: 'author' }
  ],
  resources: [
    {
      name: 'measurements',
      path: './data/measurements.csv',
      format: 'csv'
    }
  ]
}

const result = await savePackageToZenodo(dataPackage, {
  apiKey: 'your-zenodo-api-key',
  sandbox: false
})

console.log(`Deposit URL: ${result.datasetUrl}`)
console.log(`Descriptor: ${result.path}`)
```

--------------------------------

### Handle Validation Errors - TypeScript

Source: https://github.com/datisthq/dpkit/blob/main/docs/content/docs/guides/table.md

Provides an example of how to iterate through validation errors returned by `validateTable` and handle them based on their type. It demonstrates specific error cases like missing required fields, duplicate values, and pattern mismatches.

```typescript
const result = await validateTable(table, { schema })

result.errors.forEach(error => {
  switch (error.type) {
    case "cell/required":
      console.log(`Required field missing in row ${error.rowNumber}: '${error.fieldName}'`)
      break
    case "cell/unique":
      console.log(`Duplicate value in row ${error.rowNumber}: '${error.cell}'`)
      break
    case "cell/pattern":
      console.log(`Pattern mismatch: '${error.cell}' doesn't match ${error.constraint}`)
      break
  }
})
```

--------------------------------

### CLI: Publish Package to CKAN

Source: https://context7.com/datisthq/dpkit/llms.txt

Publishes a data package to a CKAN portal via the command-line interface. This command automates the process of uploading resources and creating a new dataset on the specified CKAN instance.

```bash
# Example command structure (actual command not provided in source)
dp package publish ckan --source ./path/to/datapackage.json --ckan-url https://data.gov.example --api-key YOUR_API_KEY --organization YOUR_ORG
```

--------------------------------

### Validate Package Descriptor with DPKit (JavaScript)

Source: https://github.com/datisthq/dpkit/blob/main/docs/notebooks/metadata.ipynb

Validates a package descriptor object using the `validatePackageDescriptor` function from the dpkit library. This function takes a descriptor as input and returns an object containing a boolean indicating validity and an array of errors if validation fails. It relies on the dpkit library being installed and imported.

```javascript
const { validatePackageDescriptor } = require("dpkit");

async function validateDescriptor() {
  const { valid, errors } = await validatePackageDescriptor({
    descriptor: { name: "package" },
  });
  console.log(valid);
  console.log(errors);
}

validateDescriptor();
```

--------------------------------

### Integrate dp Commands for Validation and Exploration

Source: https://github.com/datisthq/dpkit/blob/main/site/content/docs/guides/file.md

Shows how to chain DPKit commands, such as validating a file with 'dp file validate' before exploring it with 'dp table explore'. Also demonstrates describing a file and inferring its schema.

```bash
# Validate file before processing with table commands
dp file validate data.csv && dp table explore data.csv

# Describe file and then infer schema
dp file describe data.csv
dp schema infer data.csv --json > schema.json

# Copy and then create package
dp file copy remote_data.csv local_data.csv
dp package infer local_data.csv --json > datapackage.json
```

--------------------------------

### Publish Local Package to CKAN

Source: https://context7.com/datisthq/dpkit/llms.txt

Publishes a local datapackage.json to a CKAN instance. Requires CKAN URL, API key, owner organization, and dataset name. Supports remote resource fetching with the --with-remote flag.

```bash
dp package publish ckan ./datapackage.json \
  --to-ckan-url https://data.gov.example \
  --to-ckan-api-key your-api-key \
  --to-ckan-owner-org your-org \
  --to-ckan-dataset-name my-new-dataset

dp package publish ckan https://example.com/datapackage.json \
  --with-remote \
  --to-ckan-url https://data.gov.example \
  --to-ckan-api-key your-api-key \
  --to-ckan-owner-org your-org \
  --to-ckan-dataset-name imported-dataset
```

--------------------------------

### Launch Jupyter Notebook Server

Source: https://github.com/datisthq/dpkit/blob/main/docs/content/docs/guides/jupyter.md

Launches the Jupyter Notebook server in the current working directory. This command allows you to create, manage, and run Jupyter notebooks interactively.

```bash
jupyter-lab
```

--------------------------------

### Explore Resource Structure Interactively

Source: https://github.com/datisthq/dpkit/blob/main/site/content/docs/guides/resource.md

Provides an interactive interface to explore the structure and details of a data resource. Useful for understanding the schema, fields, and other metadata. Can be used with local files, resources within packages, or remote resources.

```bash
dp resource explore resource.json

# Explore resource from a package:
dp resource explore --from-package datapackage.json --from-resource "users"

# Explore remote resource:
dp resource explore https://example.com/resource.json
```

--------------------------------

### Explore Table Dialect Configuration - Bash

Source: https://github.com/datisthq/dpkit/blob/main/site/content/docs/guides/dialect.md

Explores and displays the parsing configuration of a table dialect from a local or remote descriptor path. This command provides an interactive view of the dialect's properties. It supports loading dialects from files, URLs, and datapackage resources, with an option to output the configuration as JSON.

```bash
dp dialect explore <descriptor-path>
dp dialect explore dialect.json
dp dialect explore https://example.com/dialect.json
dp dialect explore --from-package datapackage.json --from-resource "users"
dp dialect explore dialect.json --json
```

--------------------------------

### Describe File - dp CLI

Source: https://github.com/datisthq/dpkit/blob/main/site/content/docs/guides/file.md

Describes a file's properties, including size, format, encoding, and metadata. Can output results in JSON format. Useful for file diagnostics and understanding content.

```bash
dp file describe <file-path>

# Example: Describe local file
dp file describe data.csv

# Example: Describe remote file
dp file describe https://example.com/data.csv

# Example: Get description as JSON
dp file describe data.csv --json

# Example: Describe various file types
dp file describe document.pdf
dp file describe image.png
dp file describe archive.zip
```

--------------------------------

### Explore Data Package Structure using dp package explore

Source: https://github.com/datisthq/dpkit/blob/main/site/content/docs/guides/package.md

Explores a Data Package descriptor to view its structure and metadata in an interactive format or as JSON output. Can be used for both local and remote descriptor paths.

```bash
dp package explore <descriptor-path>

# Examples:
# Explore local package
dp package explore datapackage.json

# Explore remote package
dp package explore https://example.com/datapackage.json

# Export structure as JSON
dp package explore datapackage.json --json
```

--------------------------------

### Publish Data Package to CKAN

Source: https://github.com/datisthq/dpkit/blob/main/site/content/docs/guides/package.md

Publishes a data package to a CKAN instance. Requires the descriptor path and optionally accepts CKAN-specific options for authentication, URL, owner organization, and dataset name.

```bash
dp package publish ckan <descriptor-path> \
  --to-ckan-api-key <ckan-api-key> \
  --to-ckan-url <ckan-url> \
  --to-ckan-owner-org <owner-org> \
  --to-ckan-dataset-name <dataset-name>
```

```bash
# Publish to CKAN
dp package publish ckan datapackage.json \
  --to-ckan-url https://demo.ckan.org \
  --to-ckan-api-key your-api-key \
  --to-ckan-owner-org your-org
```

--------------------------------

### Validate and Describe Resources using CLI

Source: https://context7.com/datisthq/dpkit/llms.txt

Validates resource descriptors and provides detailed information about files and packages. Commands include resource validation, metadata inference from files, package validation, and file description.

```bash
# Validate a resource descriptor
dp resource validate ./resource.json

# Infer resource metadata from file
dp resource infer ./data.csv

# Validate a complete package
dp package validate ./datapackage.json

# Describe file properties
dp file describe ./data.csv
```

--------------------------------

### Work with Resources within a Package

Source: https://github.com/datisthq/dpkit/blob/main/site/content/docs/guides/resource.md

Commands to manage and interact with individual resources contained within a data package. Allows for inferring metadata for specific resources, validating the entire package, or scripting individual resources.

```bash
# Explore all resources in a package interactively
dp resource validate --from-package datapackage.json

# Infer metadata for specific resource
dp resource infer --from-package datapackage.json --from-resource "users"

# Script specific resource from package
dp resource script --from-package datapackage.json --from-resource "transactions"
```

--------------------------------

### Package Integration Workflow with DPKit

Source: https://github.com/datisthq/dpkit/blob/main/site/content/docs/guides/table.md

This workflow shows how to create a data package, validate individual tables within it, and describe tables for documentation purposes. It's useful for managing collections of related data files.

```bash
# Create package with tables
dp package infer *.csv --json > datapackage.json

# Validate individual tables
dp table validate --from-package datapackage.json --from-resource "users"

# Describe tables for documentation
dp table describe --from-package datapackage.json --from-resource "sales" --json

```

--------------------------------

### File Diagnostics Workflow - dp CLI

Source: https://github.com/datisthq/dpkit/blob/main/site/content/docs/guides/file.md

Shows how to use describe and validate commands for file diagnostics, including obtaining detailed JSON output.

```bash
# Check file properties
dp file describe suspicious_file.csv

# Validate file integrity
dp file validate suspicious_file.csv

# Get detailed diagnostics as JSON
dp file describe problematic_file.csv --json
dp file validate problematic_file.csv --json
```

--------------------------------

### Run All Tests (including checks)

Source: https://github.com/datisthq/dpkit/blob/main/CONTRIBUTING.md

Executes all tests in the project using PNPM, which includes linting and type checking. This is the primary command for verifying the project's health.

```bash
pnpm test
```

--------------------------------

### Load Package from Zenodo (TypeScript)

Source: https://context7.com/datisthq/dpkit/llms.txt

Loads a data package from a Zenodo record. This function fetches deposit metadata and file listings, merging them with any included datapackage.json descriptor. Supports public and private records using an API key.

```typescript
import { loadPackageFromZenodo } from '@dpkit/zenodo'

// Load from production Zenodo
const dataPackage = await loadPackageFromZenodo(
  'https://zenodo.org/records/1234567'
)

console.log(dataPackage.title) // 'Research Dataset 2024'
console.log(dataPackage.version) // '1.0.0'
console.log(dataPackage.licenses) // [{ name: 'CC-BY-4.0', ... }]

// Load from sandbox with API key for private deposits
const privatePackage = await loadPackageFromZenodo(
  'https://sandbox.zenodo.org/records/7654321',
  { apiKey: 'your-zenodo-api-key' }
)

console.log(`Resources: ${privatePackage.resources.length}`)
```

--------------------------------

### Explore Data Package

Source: https://github.com/datisthq/dpkit/blob/main/site/content/docs/guides/package.md

Explores the structure and contents of a data package, either from a local descriptor file or a remote URL.

```bash
dp package explore datapackage.json
```

```bash
# Explore remote package
dp package explore https://example.com/datapackage.json
```

--------------------------------

### Activate Deno Jupyter Kernel

Source: https://github.com/datisthq/dpkit/blob/main/docs/content/docs/guides/jupyter.md

Enables the Deno kernel for use in Jupyter notebooks. This activation step ensures that your notebooks can utilize the Deno environment.

```bash
deno jupyter --unstable
```

--------------------------------

### Load Data Package from Database - TypeScript

Source: https://github.com/datisthq/dpkit/blob/main/docs/content/docs/guides/database.md

Loads an entire Data Package from a database. All tables within the specified database will be loaded and represented as resources within the package.

```typescript
import { loadPackageFromDatabase } from "@dpkit/database"

const package = await loadPackageFromDatabase("sqlite://catalog.db")
// Loads all tables as package resources
```

--------------------------------

### Load Data Package from CKAN with DPKit

Source: https://context7.com/datisthq/dpkit/llms.txt

Fetches data package information from a CKAN portal, including metadata, resources, and datastore schemas. It can merge this information with a user-provided 'datapackage.json' descriptor. Requires the '@dpkit/ckan' package.

```typescript
import { loadPackageFromCkan } from '@dpkit/ckan'

// Load from CKAN instance
const dataPackage = await loadPackageFromCkan(
  'https://data.gov.example/dataset/population-2024'
)

console.log(dataPackage.name) // 'population-2024'
console.log(dataPackage.title) // 'Population Statistics 2024'
console.log(dataPackage.resources.length) // 3

// Access resources
dataPackage.resources.forEach(resource => {
  console.log(`Resource: ${resource.name}`)
  console.log(`  Format: ${resource.format}`)
  console.log(`  URL: ${resource.path}`)
  if (resource.schema) {
    console.log(`  Fields: ${resource.schema.fields.map(f => f.name).join(', ')}`)
  }
})
```

--------------------------------

### Run a Specific Vitest Test File

Source: https://github.com/datisthq/dpkit/blob/main/CONTRIBUTING.md

Executes a single test file using PNPM and Vitest. This allows for focused testing during development.

```bash
pnpm exec vitest run core/actions/__spec__/findTask.ts
```

--------------------------------

### Explore Table Schema - dp schema explore

Source: https://github.com/datisthq/dpkit/blob/main/site/content/docs/guides/schema.md

Explores a table schema from a local or remote descriptor path, displaying its field definitions and constraints interactively or as JSON. This command is useful for understanding the structure and properties of an existing schema without needing to infer it from data.

```bash
dp schema explore <descriptor-path>

# Explore schema descriptor
dp schema explore schema.json

# Explore remote schema
dp schema explore https://example.com/schema.json

# Explore schema from package resource
dp schema explore --from-package datapackage.json --from-resource "users"

# Export schema structure as JSON
dp schema explore schema.json --json
```

--------------------------------

### Explore Schema Structure

Source: https://github.com/datisthq/dpkit/blob/main/site/content/docs/guides/schema.md

Provides a way to explore the structure of a schema descriptor. This command is useful for understanding the details of a schema, such as its fields, types, and constraints. It can operate on local files or remote URLs.

```bash
dp schema explore <descriptor-path>
dp schema explore schema.json
dp schema explore https://example.com/schema.json
```

--------------------------------

### Load Package from GitHub (TypeScript)

Source: https://context7.com/datisthq/dpkit/llms.txt

Loads a data package from a GitHub repository. This function fetches repository metadata and file tree, merging it with any datapackage.json found. Supports public repositories and private ones using an API key.

```typescript
import { loadPackageFromGithub } from '@dpkit/github'

// Load from public repository
const dataPackage = await loadPackageFromGithub(
  'https://github.com/example-org/example-data'
)

console.log(dataPackage.name) // 'example-data'
console.log(dataPackage.homepage) // 'https://github.com/example-org/example-data'

// List resources
dataPackage.resources.forEach(resource => {
  console.log(`${resource.name}: ${resource.path}`)
})

// Load from private repository with token
const privatePackage = await loadPackageFromGithub(
  'https://github.com/example-org/private-data',
  { apiKey: 'ghp_your_github_token' }
)
```

--------------------------------

### Publish Package to GitHub (TypeScript)

Source: https://context7.com/datisthq/dpkit/llms.txt

Publishes a data package to a GitHub repository. It can create a new repository if needed, upload all resource files, and commit the datapackage.json descriptor. Requires a GitHub API token and repository details.

```typescript
import { savePackageToGithub } from '@dpkit/github'

const dataPackage = {
  name: 'dataset-2024',
  title: 'Dataset 2024',
  resources: [
    {
      name: 'data',
      path: './data/results.csv',
      format: 'csv'
    },
    {
      name: 'metadata',
      path: './data/metadata.json',
      format: 'json'
    }
  ]
}

const result = await savePackageToGithub(dataPackage, {
  apiKey: 'ghp_your_github_token',
  repo: 'dataset-2024',
  org: 'your-organization'  // optional, omit for user repos
})

console.log(`Repository: ${result.repoUrl}`)
console.log(`Descriptor: ${result.path}`)
```

--------------------------------

### Infer Data Resource Metadata

Source: https://github.com/datisthq/dpkit/blob/main/site/content/docs/guides/resource.md

Infers metadata, including schema, from a data file or table. Supports various table dialect and schema options for customization. Can handle local files, remote URLs, and resources within a package. Outputs results as JSON when specified.

```bash
dp resource infer <table-path>

# Infer resource from CSV file
dp resource infer data.csv

# Infer with custom delimiter
dp resource infer data.csv --delimiter ";"

# Infer from remote file
dp resource infer https://example.com/data.csv

# Infer from resource in package
dp resource infer --from-package datapackage.json --from-resource "users"

# Export as JSON
dp resource infer data.csv --json
```

--------------------------------

### Publish Package to CKAN (TypeScript)

Source: https://context7.com/datisthq/dpkit/llms.txt

Publishes a data package to a CKAN instance. This function creates a new dataset, uploads all associated resource files, and attaches the datapackage.json descriptor as a resource. Requires CKAN URL, API key, and organization details.

```typescript
import { savePackageToCkan } from '@dpkit/ckan'

const dataPackage = {
  name: 'new-dataset',
  title: 'New Dataset',
  description: 'A new dataset published to CKAN',
  resources: [
    {
      name: 'data',
      path: './data/output.csv',
      format: 'csv',
      schema: {
        fields: [
          { name: 'id', type: 'integer' },
          { name: 'value', type: 'number' }
        ]
      }
    }
  ]
}

const result = await savePackageToCkan(dataPackage, {
  ckanUrl: 'https://data.gov.example',
  apiKey: 'your-api-key-here',
  ownerOrg: 'your-organization',
  datasetName: 'new-dataset'
})

console.log(`Published to: ${result.datasetUrl}`)
console.log(`Descriptor at: ${result.path}`)
```

--------------------------------

### Load Parquet Data with @dpkit/parquet

Source: https://github.com/datisthq/dpkit/blob/main/docs/content/docs/guides/parquet.md

Demonstrates how to load data from Parquet files using the loadParquetTable function. Supports loading from local files, remote URLs, and multiple files which will be concatenated.

```typescript
import { loadParquetTable } from "@dpkit/parquet"

// Load from local file
const table = await loadParquetTable({ path: "data.parquet" })

// Load from remote URL
const table = await loadParquetTable({
  path: "https://example.com/data.parquet"
})

// Load multiple files (concatenated)
const table = await loadParquetTable({
  path: ["file1.parquet", "file2.parquet"]
})
```

--------------------------------

### Format Code with Biome

Source: https://github.com/datisthq/dpkit/blob/main/CONTRIBUTING.md

Automatically formats the codebase using Biome via PNPM. This command helps maintain consistent code style across the project.

```bash
pnpm run format
```

--------------------------------

### Script Table Schema

Source: https://github.com/datisthq/dpkit/blob/main/site/content/docs/guides/schema.md

Opens an interactive REPL session to programmatically interact with a loaded table schema. This allows for dynamic exploration and manipulation of schema definitions using available variables like 'dpkit' and 'schema'. Schemas can be loaded from local files or package resources.

```bash
dp schema script <descriptor-path>
dp schema script schema.json
dp schema script --from-package datapackage.json --from-resource "users"
# Inside the REPL:
schema.fields.length
schema.fields[0].name
schema.fields.filter(f => f.type === 'integer')
schema.primaryKey
```

--------------------------------

### Explore Table Interactively

Source: https://github.com/datisthq/dpkit/blob/main/site/content/docs/guides/table.md

Provides an interactive terminal interface to explore table data, view samples, column information, and basic statistics. Supports local files, package resources, and remote URLs, with options for schema and dialect validation.

```bash
dp table explore <table-path>

# Examples:
# Explore CSV file
dp table explore data.csv

# Explore with schema validation
dp table explore data.csv --schema schema.json

# Explore with custom dialect
dp table explore data.csv --dialect dialect.json

# Explore resource from package
dp table explore --from-package datapackage.json --from-resource "users"

# Explore remote table
dp table explore https://example.com/data.csv
```

--------------------------------

### CLI: Copy Data Package

Source: https://context7.com/datisthq/dpkit/llms.txt

Copies a data package from one location to another using the command-line interface. This command supports copying from local files, remote URLs, and includes options for downloading remote resources and converting formats.

```bash
# Copy local package to new location
dp package copy ./input/datapackage.json --to ./output/datapackage.json

# Copy and download remote resources
dp package copy https://example.com/datapackage.json --to ./local/datapackage.json --with-remote

# Copy from CKAN to local
dp package copy https://data.gov.example/dataset/my-data --to ./data/datapackage.json --with-remote
```

--------------------------------

### Performance Optimization with DPKit

Source: https://github.com/datisthq/dpkit/blob/main/site/content/docs/guides/table.md

This section covers optimizing performance for large datasets. It suggests using sampling for statistical operations and converting data to efficient formats like Parquet for repeated analysis.

```bash
# For large files, use sampling
dp table describe huge_file.csv --sample-rows 10000

# Convert to efficient formats for repeated analysis
dp table convert large_data.csv data.parquet

```

--------------------------------

### Batch File Operations - dp CLI

Source: https://github.com/datisthq/dpkit/blob/main/site/content/docs/guides/file.md

Demonstrates batch processing of files using shell loops with dp file describe and dp file validate commands. Useful for managing multiple files.

```bash
# Describe multiple files
for file in *.csv; do
  echo "Describing $file:"
  dp file describe "$file"
  echo "---"
done

# Validate all files in directory
for file in data/*.json; do
  dp file validate "$file" --json >> validation_report.json
done
```

--------------------------------

### Format Conversion Pipeline with DPKit

Source: https://github.com/datisthq/dpkit/blob/main/site/content/docs/guides/table.md

This pipeline illustrates how to convert data between different formats using DPKit. It shows converting an Excel file to CSV, validating the intermediate CSV, and finally converting it to a JSON output.

```bash
# Convert Excel to CSV for processing
dp table convert input.xlsx temp.csv

# Process and validate
dp table validate temp.csv --schema schema.json

# Convert to final format
dp table convert temp.csv output.json

```

--------------------------------

### Copy File - dp CLI

Source: https://github.com/datisthq/dpkit/blob/main/site/content/docs/guides/file.md

Copies a file from a source path to a target path. Supports local and remote sources and destinations. Useful for backups and data migration.

```bash
dp file copy <source-path> <target-path>

# Example: Copy local file
dp file copy data.csv backup.csv

# Example: Copy remote file to local
dp file copy https://example.com/data.csv local_data.csv

# Example: Copy to different directory
dp file copy data.csv ./backup/data_backup.csv
```

--------------------------------

### Publish Data Package to GitHub

Source: https://github.com/datisthq/dpkit/blob/main/site/content/docs/guides/package.md

Publishes a data package to GitHub. This command can be used to manage releases or repository files for a data package.

```bash
dp package publish github <descriptor-path>
```

--------------------------------

### Copy Data Package using dp package copy

Source: https://github.com/datisthq/dpkit/blob/main/site/content/docs/guides/package.md

Copies a local or remote Data Package to a specified destination, which can be a local folder, a ZIP archive, or a database. Supports including remote resources.

```bash
dp package copy <descriptor-path> --to-path <target>

# Examples:
# Copy package to local directory
dp package copy datapackage.json --to-path ./output

# Copy package to ZIP archive
dp package copy datapackage.json --to-path package.zip

# Copy remote package including remote resources
dp package copy https://example.com/datapackage.json --to-path ./local --with-remote
```

--------------------------------

### Run Comprehensive Code Check

Source: https://github.com/datisthq/dpkit/blob/main/CONTRIBUTING.md

Executes both linting and TypeScript type checking using PNPM. This command provides a thorough check of code quality and correctness.

```bash
pnpm run check
```