### Install Dependencies Source: https://dlthub.com/docs/examples/qdrant_zendesk/index Ensure you have the necessary qdrant-client and fastembed libraries installed for the example. ```bash pip install qdrant-client>=1.6.9 pip install fastembed>=0.1.1 ``` -------------------------------- ### Quickstart dltHub Installation with pipx Source: https://dlthub.com/docs/hub/getting-started/installation.md If you don't have uv installed, you can use pipx to run dlthub-start. The CLI will offer to install uv for you. ```sh pipx run dlthub-start ``` -------------------------------- ### Install MySQL Client Source: https://dlthub.com/docs/dlt-ecosystem/destinations/sqlalchemy.md Example of installing a specific database driver, such as mysqlclient for MySQL databases. ```sh pip install mysqlclient ``` -------------------------------- ### Start dltHub Project Source: https://dlthub.com/docs/plus/intro Scaffolds a local workspace with dltHub AI Workbench, example pipelines, and dlt[hub] installed. Ensure prerequisites are met before running. ```bash uvx dlthub-start@latest ``` -------------------------------- ### Get Workflow Usages (Python) Source: https://dlthub.com/docs/pipelines/zuora/load-data-with-python-from-zuora-to-yugabyte Gets workflow task usage sorted by day within a specified time frame. Requires start date, end date, and metrics as query parameters. ```python { "name": "get_workflows_usages", "table_name": "metric", "endpoint": { "path": "/workflows/metrics.json", "params": { "startDate": "FILL_ME_IN" # TODO: fill in required query parameter "endDate": "FILL_ME_IN" # TODO: fill in required query parameter "metrics": "FILL_ME_IN" # TODO: fill in required query parameter } } }, ``` -------------------------------- ### Get Data Query Jobs List (Python) Source: https://dlthub.com/docs/pipelines/zuora/load-data-with-python-from-zuora-to-yugabyte Returns a list of data query jobs created in the Zuora tenant. Can be filtered by status. Administrators can retrieve all jobs; others can only retrieve their own. ```python { "name": "get_data_query_jobs", "table_name": "job", "endpoint": { "path": "/query/jobs", "params": { # the parameters below can optionally be configured # "queryStatus": "OPTIONAL_CONFIG", # "pageSize": "20", } } }, ``` -------------------------------- ### Get All Orders Source: https://dlthub.com/docs/pipelines/zuora/load-data-with-python-from-zuora-to-yugabyte Retrieves information about all orders in your tenant. By default, it returns the first page of the orders. This feature requires the Orders Harmonization setting to be enabled. ```APIDOC ## GET /v1/orders ### Description Retrieves information about all orders in your tenant. By default, it returns the first page of the orders. This feature requires the Orders Harmonization setting to be enabled. ### Method GET ### Endpoint /v1/orders ### Parameters #### Query Parameters - **pageSize** (integer) - Optional - The number of orders to return per page. Defaults to 20. - **dateFilterOption** (string) - Optional - Option for filtering orders by date. - **startDate** (string) - Optional - The start date for filtering orders. - **endDate** (string) - Optional - The end date for filtering orders. ``` -------------------------------- ### developer_tutorial Source: https://dlthub.com/docs/pipelines/vimeo/load-data-with-python-from-vimeo-to-athena This method, in conjunction with our Getting Started guide, can help you learn how to use the Vimeo API. ```APIDOC ## GET /tutorial ### Description Provides a link to the Vimeo API Getting Started guide for learning how to use the API. ### Method GET ### Endpoint /tutorial ``` -------------------------------- ### Get Journal Run Information (Python) Source: https://dlthub.com/docs/pipelines/zuora/load-data-with-python-from-zuora-to-yugabyte Retrieves information about a specific journal run, including sample code for request and response. ```python { "name": "get_journal_run", "table_name": "journal_run", "endpoint": { "path": "/v1/journal-runs/{jr-number}", "params": { "jr-number": "FILL_ME_IN" # TODO: fill in required path parameter } } }, ``` -------------------------------- ### Get Subscription Product Feature by ID Source: https://dlthub.com/docs/pipelines/zuora/load-data-with-python-from-zuora-to-yugabyte Retrieves detailed information about a specific subscription product feature using its ID. Optional 'fields' parameter can be configured. ```python { "name": "object_get_subscription_product_feature", "table_name": "subscription_product_feature", "endpoint": { "path": "/v1/object/subscription-product-feature/{id}", "params": { "id": "FILL_ME_IN", "fields": "OPTIONAL_CONFIG" } } } ``` -------------------------------- ### get_upload_attempt Source: https://dlthub.com/docs/pipelines/vimeo/load-data-with-python-from-vimeo-to-neondb Retrieves information about a specific upload attempt for a user. This method, in conjunction with the Getting Started guide, can help you learn how to use the Vimeo API. ```APIDOC ## GET /users/{user_id}/uploads/{upload_id} ### Description Retrieves information about a specific upload attempt for a user. This method, in conjunction with the Getting Started guide, can help you learn how to use the Vimeo API. ### Method GET ### Endpoint /users/{user_id}/uploads/{upload_id} ### Parameters #### Path Parameters - **user_id** (string) - Required - The ID of the user. - **upload_id** (string) - Required - The ID of the upload attempt. ``` -------------------------------- ### Get Order Source: https://dlthub.com/docs/pipelines/zuora/load-data-with-python-from-zuora-to-yugabyte Retrieves the detailed information about a specified order. This feature requires the Orders Harmonization setting to be enabled. ```APIDOC ## GET /v1/orders/{orderNumber} ### Description Retrieves the detailed information about a specified order. This feature requires the Orders Harmonization setting to be enabled. ### Method GET ### Endpoint /v1/orders/{orderNumber} ### Parameters #### Path Parameters - **orderNumber** (string) - Required - The number of the order to retrieve. ``` -------------------------------- ### Get Specific Workflow Task Source: https://dlthub.com/docs/pipelines/zuora/load-data-with-python-from-zuora-to-yugabyte Retrieves a specific workflow task by its unique identifier. ```APIDOC ## GET /workflows/tasks/{task_id} ### Description Retrieves a specific workflow task by its unique identifier. ### Method GET ### Endpoint /workflows/tasks/{task_id} ### Parameters #### Path Parameters - **task_id** (string) - Required - The unique identifier of the workflow task. ### Response #### Success Response (200) - **task** (object) - The workflow task object. #### Response Example { "example": "Workflow task object" } ``` -------------------------------- ### load_setuptools_entrypoints Source: https://dlthub.com/docs/api_reference/dlt/common/configuration/plugins Loads plugins from setuptools entry points in the 'dlt' group. It instantiates these plugins and returns a list of their names. ```APIDOC ## load_setuptools_entrypoints ```python def load_setuptools_entrypoints(m: pluggy.PluginManager) -> List[str] ``` Loads entry points in group `dlt` and instantiates them to initialize plugins. Returns a list of names of top level modules/packages from detected entry points. ``` -------------------------------- ### Get Revenue Items by Charge Revenue Event Number from Zuora Source: https://dlthub.com/docs/pipelines/zuora/load-data-with-python-from-zuora-to-timescale This snippet fetches revenue items associated with a specific revenue event using the event number. It allows optional configuration of 'pageSize'. ```python { "name": "get_revenue_items_by_charge_revenue_event_number", "table_name": "revenue_event", "endpoint": { "path": "/v1/revenue-items/revenue-events/{event-number}", "params": { "event-number": "FILL_ME_IN" # TODO: fill in required path parameter # the parameters below can optionally be configured # "pageSize": "300", } } } ``` -------------------------------- ### Get Taxation Items of Invoice Item Source: https://dlthub.com/docs/pipelines/zuora/load-data-with-python-from-zuora-to-timescale Retrieves taxation items for a specific invoice item. Requires the invoice ID and item ID to be provided. ```python { "name": "get_taxation_items_of_invoice_item", "table_name": "taxation_item", "endpoint": { "path": "/v1/invoices/{invoiceId}/items/{itemId}/taxation-items", "params": { "invoiceId": "FILL_ME_IN", # TODO: fill in required path parameter "itemId": "FILL_ME_IN", # TODO: fill in required path parameter # the parameters below can optionally be configured # "pageSize": "20", } } } ``` -------------------------------- ### Get Specific Credit Memo Item Source: https://dlthub.com/docs/pipelines/zuora/load-data-with-python-from-zuora-to-yugabyte Retrieves the details of a single, specific item within a credit memo using its ID. ```APIDOC ## GET /v1/creditmemos/{creditMemoId}/items/{cmitemid} ### Description Retrieves the information about a specific item of a credit memo. A credit memo item is a single line item in a credit memo. ### Method GET ### Endpoint /v1/creditmemos/{creditMemoId}/items/{cmitemid} ### Parameters #### Path Parameters - **creditMemoId** (string) - Required - The ID of the credit memo. - **cmitemid** (string) - Required - The ID of the credit memo item. ### Response #### Success Response (200) - **item** (object) - The credit memo item object. #### Response Example { "amount": 100.00, "appliedAmount": 50.00, "createdById": "2c9380845c7ca371015c7cb40ac30015", "createdDate": "2023-01-01T10:00:00.000Z", "id": "402890245c7ca371015c7cb40ac30015", "refundAmount": 0.00, "serviceEndDate": "2023-12-31", "serviceStartDate": "2023-01-01", "sku": "PROD-001", "skuName": "Product One", "sourceItemId": "", "subscriptionId": "402890245c7ca371015c7cb40ac30015", "updatedById": "2c9380845c7ca371015c7cb40ac30015", "updatedDate": "2023-01-01T10:00:00.000Z" } ``` -------------------------------- ### Initialize dlt Project with Zendesk Source Source: https://dlthub.com/docs/walkthroughs/zendesk-weaviate.md Initializes a new dlt project by downloading and installing the verified Zendesk source. ```sh dlt init zendesk weaviate ``` -------------------------------- ### Get Workspace Install Specs Source: https://dlthub.com/docs/api_reference/dlt/_workspace/deployment/requirements Retrieves install specifications for core dlt hub packages, skipping any that are not currently installed. ```python def get_workspace_install_specs() -> List[TInstallSpec] ``` ``` -------------------------------- ### Get Credit Memo Parts Source: https://dlthub.com/docs/pipelines/zuora/load-data-with-python-from-zuora-to-yugabyte Retrieves the information about all parts of a credit memo. This operation is available if Invoice Settlement is enabled in Zuora and can show applied and unapplied portions of a credit memo. ```APIDOC ## GET /v1/creditmemos/{creditMemoId}/parts ### Description Retrieves the information about all parts of a credit memo. A credit memo can consist of an unapplied part, and several parts applied to invoices and debit memos. You can use this operation to get all the applied and unapplied portions of a credit memo. ### Method GET ### Endpoint /v1/creditmemos/{creditMemoId}/parts ### Parameters #### Path Parameters - **creditMemoId** (string) - Required - The ID of the credit memo. #### Query Parameters - **pageSize** (string) - Optional - The number of results to return per page. Defaults to 20. ``` -------------------------------- ### Bitbucket API GET Request Example Source: https://dlthub.com/docs/pipelines/bitbucket/load-data-with-python-from-bitbucket-to-cosmosdb Example of using a curl command to get the status of an import task from the Bitbucket API. This endpoint reports the status of the current import task. ```bash $ curl -u -X GET https://api.bitbucket.org/2.0/repositories///issues/import ``` -------------------------------- ### Initialize a new project with dlt Source: https://dlthub.com/docs/walkthroughs/create-new-destination.md Use this command to initialize a new dlt project with a verified source and credentials. ```sh dlt init github postgres ``` -------------------------------- ### Get Notification History Deletion Task Source: https://dlthub.com/docs/pipelines/zuora/load-data-with-python-from-zuora-to-yugabyte Retrieves the details of a notification history deletion task by its ID. This operation is available if Notification and Configurable Event features are enabled. ```APIDOC ## GET /notifications/history/tasks/{id} ### Description Get the notification history deletion task by ID. Note: This operation is only available if you have the Notification and the Configurable Event features enabled. ### Method GET ### Endpoint /notifications/history/tasks/{id} ### Parameters #### Path Parameters - **id** (string) - Required - The ID of the notification history deletion task. ``` -------------------------------- ### Get Specific Data Query Job (Python) Source: https://dlthub.com/docs/pipelines/zuora/load-data-with-python-from-zuora-to-yugabyte Retrieves a specific data query job, allowing tracking of its status and obtaining query results URL. Administrators can access all jobs; non-admins can only access their own. ```python { "name": "get_data_query_job", "table_name": "job", "endpoint": { "path": "/query/jobs/{job-id}", "params": { "job-id": "FILL_ME_IN" # TODO: fill in required path parameter } } }, ``` -------------------------------- ### Install Transformations Toolkit Source: https://dlthub.com/docs/hub/transformations/explore-and-transform.md Installs the `transformations` toolkit using `uv run`. Ensure `dlthub ai init` has been run previously. ```sh uv run dlthub ai toolkit install transformations ``` -------------------------------- ### Initialize dlt Project for Chess to Filesystem Source: https://dlthub.com/docs/pipelines/chess/load-data-with-python-from-chess-to-filesystem-gcs Initialize a new dlt project with Chess.com as the data source and a filesystem (like Google Cloud Storage) as the destination. This command sets up the basic project structure and configuration files. ```bash # create a new directory mkdir chess_pipeline cd chess_pipeline # initialize a new pipeline with your source and destination dlt init chess filesystem # install the required dependencies pip install -r requirements.txt ``` -------------------------------- ### Example requirements.txt Source: https://dlthub.com/docs/pipelines/sql_database_postgres/load-data-with-python-from-sql_database_postgres-to-snowflake Lists the required Python dependencies for the pipeline, ensuring compatibility and functionality. ```text sqlalchemy>=1.4 dlt[snowflake]>=0.4.7 ``` -------------------------------- ### Get Revenue Automation Start Date Source: https://dlthub.com/docs/pipelines/zuora/load-data-with-python-from-zuora-to-clickhouse Retrieves the configured start date for revenue automation. ```APIDOC ## GET /v1/settings/finance/revenue-automation-start-date ### Description Retrieves the revenue automation start date. ### Method GET ### Endpoint /v1/settings/finance/revenue-automation-start-date ### Response #### Success Response (200) - **name** (string) - The name of the setting. - **table_name** (string) - The table name for the setting. - **endpoint** (object) - Details about the API endpoint. - **path** (string) - The API endpoint path. ``` -------------------------------- ### Bitbucket API: Get Issue Import Status Example Source: https://dlthub.com/docs/pipelines/bitbucket/load-data-with-python-from-bitbucket-to-biganimal Example cURL command to get the status of an issue import task in a Bitbucket repository. This endpoint reports the status of the current import task. ```bash $ curl -u -X GET https://api.bitbucket.org/2.0/repositories///issues/import ``` -------------------------------- ### Load Setuptools Entry Points Source: https://dlthub.com/docs/api_reference/dlt/common/configuration/plugins Loads plugins from setuptools entry points in the 'dlt' group. It instantiates them and returns a list of loaded module names. ```python def load_setuptools_entrypoints(m: pluggy.PluginManager) -> List[str] ``` ``` -------------------------------- ### Configure Zuora Get Revenue Automation Start Date Endpoint Source: https://dlthub.com/docs/pipelines/zuora/load-data-with-python-from-zuora-to-cockroachdb Configure the API endpoint to retrieve the revenue automation start date setting. This is a simple GET request to a specific configuration path. ```python { "name": "get_revenue_automation_start_date", "table_name": "revenue_automation_start_date", "endpoint": { "path": "/v1/settings/finance/revenue-automation-start-date", } }, ``` -------------------------------- ### Initialize a Custom Pipeline (Example) Source: https://dlthub.com/docs/hub/ingestion/init.md An example of initializing a custom dlt pipeline named 'my_github_pipeline' with 'duckdb' as the destination. ```sh dlthub pipeline init my_github_pipeline duckdb ``` -------------------------------- ### Pipeline Configuration Example Source: https://dlthub.com/docs/dlt-ecosystem/verified-sources/stripe.md Demonstrates how to configure a dlt pipeline with a specific name, destination, and dataset. ```python pipeline = dlt.pipeline( pipeline_name="stripe_pipeline", # Use a custom name if desired destination="duckdb", # Choose the appropriate destination (e.g., duckdb, redshift, post) dataset_name="stripe_dataset" # Use a custom name if desired ) ``` -------------------------------- ### Get Revenue Automation Start Date Source: https://dlthub.com/docs/pipelines/zuora/load-data-with-python-from-zuora-to-athena Retrieves the revenue automation start date setting from Zuora. ```python { "name": "get_revenue_automation_start_date", "table_name": "revenue_automation_start_date", "endpoint": { "path": "/v1/settings/finance/revenue-automation-start-date", } } ``` -------------------------------- ### Get Installed Requirement String Source: https://dlthub.com/docs/api_reference/dlt/version Retrieves the requirement string for the currently installed dlt package. Useful for verifying the installed version or for use in dependency management. ```python def get_installed_requirement_string(package: str = DLT_PKG_NAME, allow_earlier: bool = False) -> str ``` -------------------------------- ### Install Python and uv on Windows Source: https://dlthub.com/docs/reference/installation.md Install pip and the uv package manager on Windows. ```bat C:\> pip3 install -U pip C:\> pip3 install uv ``` -------------------------------- ### Quick start: Load data into Lance Source: https://dlthub.com/docs/dlt-ecosystem/destinations/lance.md A basic example demonstrating how to initialize a dlt pipeline with the Lance destination and load a list of dictionaries into a Lance table. ```python import dlt movies = [ {"id": 1, "title": "Blade Runner", "year": 1982}, {"id": 2, "title": "Ghost in the Shell", "year": 1995}, {"id": 3, "title": "The Matrix", "year": 1999}, ] pipeline = dlt.pipeline( pipeline_name="movies", destination="lance", dataset_name="movies_db", ) info = pipeline.run(movies, table_name="movies") ``` -------------------------------- ### Quick Start: Retrieve Data into Pandas DataFrame or PyArrow Table Source: https://dlthub.com/docs/general-usage/dataset-access/dataset.md This example demonstrates how to get a readable dataset from a pipeline, access a specific table as a relation, and then fetch the entire table's content as either a Pandas DataFrame or a PyArrow Table. Assumes a 'pipeline' object and loaded data. ```python # Assuming you have a Pipeline object named 'pipeline'. You can create one with the dlt cli: dlt init fruitshop duckdb # and you have loaded the data of the fruitshop example source into the destination # the tables available in the destination are: # - customers # - inventory # - purchases # Step 1: Get the readable dataset from the pipeline dataset = pipeline.dataset() # Step 2: Access a table as a ReadableRelation customers_relation = dataset.table("customers") # Step 3: Fetch the entire table as a Pandas DataFrame df = customers_relation.df() # or customers_relation.df(chunk_size=50) # Alternatively, fetch as a PyArrow Table arrow_table = customers_relation.arrow() ``` -------------------------------- ### Initialize Filesystem to BigQuery pipeline Source: https://dlthub.com/docs/pipelines/filesystem/load-data-with-python-from-filesystem-to-bigquery Use the `dlt init` command to create a starter project for loading data from a filesystem source (like S3) to BigQuery. This sets up the basic directory structure and configuration files. ```bash # create a new directory mkdir my_filesystem_pipeline cd my_filesystem_pipeline # initialize a new pipeline with your source and destination dlt init filesystem bigquery # install the required dependencies pip install -r requirements.txt ``` -------------------------------- ### Install dlt Workspace Source: https://dlthub.com/docs/hub/ingestion/rest-api-source.md Installs the dlt workspace with the 'hub' extra. Use '--upgrade' to get the latest version. ```sh pip install "dlt[hub]" ``` ```sh pip install --upgrade "dlt[hub]" ``` -------------------------------- ### Apply Hints for Merging Data in Filesystem Pipeline Source: https://dlthub.com/docs/dlt-ecosystem/verified-sources/filesystem.md This example shows how to apply hints to a filesystem pipeline, specifically setting the write disposition to 'merge' and specifying 'date' as the merge key. It then loads the data into a table named 'table_name'. ```python import dlt from dlt.sources.filesystem import filesystem, read_csv ilesystem_pipe = filesystem(bucket_url="file://Users/admin/Documents/csv_files", file_glob="*.csv") | read_csv() # Tell dlt to merge on date ilesystem_pipe.apply_hints(write_disposition="merge", merge_key="date") # We load the data into the table_name table pipeline = dlt.pipeline(pipeline_name="my_pipeline", destination="duckdb") load_info = pipeline.run(filesystem_pipe.with_name("table_name")) print(load_info) ``` -------------------------------- ### Get Revenue Automation Start Date Configuration Source: https://dlthub.com/docs/pipelines/zuora/load-data-with-python-from-zuora-to-clickhouse This configuration retrieves the revenue automation start date from Zuora settings. ```python { "name": "get_revenue_automation_start_date", "table_name": "revenue_automation_start_date", "endpoint": { "path": "/v1/settings/finance/revenue-automation-start-date", } }, ``` -------------------------------- ### Get Package Install Spec Source: https://dlthub.com/docs/api_reference/dlt/_workspace/deployment/requirements Classifies the installation type of a given package name (pypi, path, editable, git). ```python def get_pkg_install_spec(pkg_name: str, *, extras: Optional[List[str]] = None) -> TInstallSpec ``` ``` -------------------------------- ### Configure SQL Server source with ConnectorX backend Source: https://dlthub.com/docs/pipelines/sql_database_mssql/load-data-with-python-from-sql_database_mssql-to-filesystem-local This example configures the SQL Server source using the 'connectorx' backend. It includes connection string details, chunk size, and options to preserve source data types and set up a separate connection string for ConnectorX. ```python sql_database( "postgresql://loader:loader@localhost:5432/dlt_data", "unsw_flow_7", "speed_test", # this is ignored by connectorx chunk_size=100000, backend="connectorx", # keep source data types detect_precision_hints=True, # just to demonstrate how to setup a separate connection string for connectorx backend_kwargs={\"conn\": \"postgresql://loader:loader@localhost:5432/dlt_data\"}, ) ``` -------------------------------- ### Configure Qdrant with local file path Source: https://dlthub.com/docs/dlt-ecosystem/destinations/qdrant.md Configure Qdrant to store the database in a local file instead of a remote instance. ```toml [destination.qdrant] qd_path = "db.qdrant" ``` -------------------------------- ### Configure Get Revenue Automation Start Date Endpoint Source: https://dlthub.com/docs/pipelines/zuora/load-data-with-python-from-zuora-to-neondb Defines the configuration for retrieving the revenue automation start date setting from Zuora. ```python { "name": "get_revenue_automation_start_date", "table_name": "revenue_automation_start_date", "endpoint": { "path": "/v1/settings/finance/revenue-automation-start-date" } } ``` -------------------------------- ### Get Organization SAML IdP Source: https://dlthub.com/docs/pipelines/cisco_meraki/load-data-with-python-from-cisco_meraki-to-athena Retrieves a specific SAML Identity Provider (IdP) configuration for the organization. Use this to get details about a particular SSO setup. ```APIDOC ## GET /organizations/{organizationId}/saml/idps/{idpId} ### Description Retrieves a specific SAML Identity Provider (IdP) configuration for the organization. ### Method GET ### Endpoint /organizations/{organizationId}/saml/idps/{idpId} ### Parameters #### Path Parameters - **organizationId** (string) - Required - The organization ID. - **idpId** (string) - Required - The ID of the SAML IdP to retrieve. ``` -------------------------------- ### MongoDB Connection URL Example Source: https://dlthub.com/docs/dlt-ecosystem/verified-sources/mongodb.md This is an example format for a MongoDB connection URL. Adjust the user, password, and host details according to your specific MongoDB setup. ```text connection_url = "mongodb://dbuser:passwd@host.or.ip:27017" ``` -------------------------------- ### Initialize dlt project for Chess.com to AWS S3 Source: https://dlthub.com/docs/pipelines/chess/load-data-with-python-from-chess-to-filesystem Create a new dlt project directory, navigate into it, and initialize a pipeline for the 'chess' source and 'filesystem' destination. This sets up the basic project structure and configuration files. ```bash # create a new directory mkdir my-chess-pipeline cd my-chess-pipeline # initialize a new pipeline with your source and destination dlt init chess filesystem # install the required dependencies pip install -r requirements.txt ``` -------------------------------- ### Example requirements.txt Source: https://dlthub.com/docs/pipelines/aladtec/load-data-with-python-from-aladtec-to-clickhouse The `requirements.txt` file lists the necessary dependencies for the generated dlt pipeline, including the dlt library itself. ```text dlt>=0.4.12 ``` -------------------------------- ### Get Zip Download Status Source: https://dlthub.com/docs/pipelines/box/load-data-with-python-from-box-to-alloydb Returns the download status of a zip archive. This endpoint can only be accessed once the download has started and is valid for 12 hours from the start of the download. ```APIDOC ## GET /zip_downloads/{zip_download_id}/status ### Description Returns the download status of a `zip` archive, allowing an application to inspect the progress of the download as well as the number of items that might have been skipped. This endpoint can only be accessed once the download has started. Subsequently this endpoint is valid for 12 hours from the start of the download. ### Method GET ### Endpoint /zip_downloads/{zip_download_id}/status ### Parameters #### Path Parameters - **zip_download_id** (string) - Required - The ID of the zip download to check the status for. ### Response #### Success Response (200) - **status** (string) - The status of the zip download (e.g., 'in_progress', 'succeeded', 'failed'). - **progress_percent** (number) - The percentage of the download that has been completed. - **downloaded_count** (integer) - The number of items that have been successfully downloaded. - **skipped_count** (integer) - The number of items that were skipped during the download. - **total_count** (integer) - The total number of items in the zip archive. - **message** (string) - A message providing more details about the status, if applicable. - **download_url** (string) - The URL to download the zip archive if the status is 'succeeded'. - **expires_at** (string) - The timestamp when the download status and URL will expire. ``` -------------------------------- ### Initialize GitHub to Filesystem Pipeline Source: https://dlthub.com/docs/pipelines/github/load-data-with-python-from-github-to-filesystem-aws Use the dlt CLI to create a starting point for a pipeline that loads data from GitHub to a filesystem destination. This command sets up the basic project structure and configuration files. ```bash # create a new directory mkdir github_pipeline cd github_pipeline # initialize a new pipeline with your source and destination dlt init github filesystem # install the required dependencies pip install -r requirements.txt ``` -------------------------------- ### Bitbucket API - Get Deployments Example Source: https://dlthub.com/docs/pipelines/bitbucket/load-data-with-python-from-bitbucket-to-filesystem-aws Example of how to retrieve a list of deployments for a specific repository using the Bitbucket API. Requires 'workspace' and 'repo_slug' parameters. ```bash $ curl -H "Authorization " \ https://api.bitbucket.org/2.0/repositories/mleu/test/deploy-key/1234 ``` -------------------------------- ### Get Workspace Webhooks Source: https://dlthub.com/docs/pipelines/bitbucket/load-data-with-python-from-bitbucket-to-athena Retrieves a paginated list of webhooks installed on a workspace. ```APIDOC ## GET /workspaces/{workspace}/hooks ### Description Returns a paginated list of webhooks installed on this workspace. ### Method GET ### Endpoint /workspaces/{workspace}/hooks ### Parameters #### Path Parameters - **workspace** (string) - Required - The workspace ID. #### Query Parameters None explicitly documented. ### Request Example None provided. ### Response #### Success Response (200) - **values** (array) - A list of webhook subscriptions. ```