### Full Configuration Example Source: https://github.com/kaggle/kagglehub/blob/main/_autodocs/configuration.md Demonstrates setting a custom cache directory, enabling verbose logging, and programmatically setting API token credentials. Includes verification and download examples. ```python import os import kagglehub from kagglehub.config import get_cache_folder, set_kaggle_api_token # Set custom cache directory os.environ['KAGGLEHUB_CACHE'] = '/data/kagglehub-cache' # Enable verbose logging os.environ['KAGGLEHUB_VERBOSITY'] = 'debug' # Set credentials programmatically set_kaggle_api_token('your-api-token-here') # Verify configuration print(f"Cache: {get_cache_folder()}") # Download with custom cache location path = kagglehub.dataset_download('owner/dataset') # Download to custom output directory (bypasses cache) path = kagglehub.dataset_download('owner/dataset', output_dir='./my-datasets') ``` -------------------------------- ### Install kagglehub Package Source: https://github.com/kaggle/kagglehub/blob/main/README.md Install the kagglehub package using pip. This is the primary method for setting up the library. ```bash pip install kagglehub ``` -------------------------------- ### Utility Script Install Source: https://github.com/kaggle/kagglehub/blob/main/_autodocs/OVERVIEW.md Function for installing utility scripts from Kaggle. ```APIDOC ## Utility Script Install ### Description Installs utility scripts made available through Kaggle. ### Method `kagglehub.utility_script_install(handle, force_download=False)` ### Parameters - **handle** (string) - Required - The identifier for the utility script. - **force_download** (boolean) - Optional - If True, forces a download even if the script is already installed. Defaults to False. ### Returns None. ``` -------------------------------- ### Package Structure Example Source: https://github.com/kaggle/kagglehub/blob/main/_autodocs/api-reference/packages.md Illustrates the standard directory layout for a Kaggle Package export, including initialization files, modules, and an assets directory. ```bash output/ └── package/ ├── __init__.py # Required: Package initialization ├── module1.py # Package modules ├── module2.py └── assets/ # Optional: Package assets ├── model.pkl └── data/ └── config.json ``` -------------------------------- ### Install Utility Script Source: https://github.com/kaggle/kagglehub/blob/main/README.md Install a Kaggle utility script to make its code available in your Python environment. Installs the latest version by default. ```python import kagglehub # Install the latest version. kagglehub.utility_script_install('bjoernjostein/physionet-challenge-utility-script') ``` -------------------------------- ### Download Kaggle Notebook Output and Import Packages Source: https://github.com/kaggle/kagglehub/blob/main/_autodocs/OVERVIEW.md Download the output of a Kaggle notebook or import a notebook as a package. Utility scripts can also be installed. ```python # Download notebook output path = kagglehub.notebook_output_download('alexisbcook/titanic-tutorial') # Import package my_package = kagglehub.package_import('username/my-package') # Install utility script script_path = kagglehub.utility_script_install('author/utility-script') ``` -------------------------------- ### Download and Load Dataset in Scripts Source: https://github.com/kaggle/kagglehub/blob/main/_autodocs/OVERVIEW.md Demonstrates how to set Kaggle API credentials via environment variables, download a dataset, load it into a Pandas DataFrame, and upload results. Ensure you have the 'kagglehub' and 'pandas' libraries installed. ```python import os import kagglehub # Set credentials via environment os.environ['KAGGLE_API_TOKEN'] = 'your-token' # Download resources dataset_path = kagglehub.dataset_download('owner/dataset') # Process data df = kagglehub.dataset_load( kagglehub.KaggleDatasetAdapter.PANDAS, 'owner/dataset', 'data.csv' ) # ... process ... # Upload results kagglehub.dataset_upload('myuser/results-dataset', './results') ``` -------------------------------- ### Package Import Source: https://github.com/kaggle/kagglehub/blob/main/_autodocs/OVERVIEW.md Functions for importing Kaggle packages and getting asset paths. ```APIDOC ## Package Import ### Description Provides functionality to import Kaggle packages and retrieve paths to their assets. ### Methods - `kagglehub.package_import(handle, force_download=False, bypass_confirmation=False)`: Imports a Kaggle package. - `kagglehub.get_package_asset_path(path)`: Retrieves the local file path for a package asset. ### Parameters for `package_import` - **handle** (string) - Required - The identifier for the package to import. - **force_download** (boolean) - Optional - If True, forces a download even if the package is already installed. Defaults to False. - **bypass_confirmation** (boolean) - Optional - If True, bypasses user confirmation prompts during import. Defaults to False. ### Parameters for `get_package_asset_path` - **path** (string) - Required - The path to the asset within the package. ### Returns - `package_import`: None. - `get_package_asset_path`: String path to the package asset. ``` -------------------------------- ### Install and Import Kaggle Utility Script Source: https://github.com/kaggle/kagglehub/blob/main/_autodocs/api-reference/utility_scripts.md Downloads a Kaggle Utility Script and adds its directory to Python's system path for direct import. Use this for reusable standalone scripts. The `force_download` parameter can be set to `True` to re-download an already cached script. ```python import kagglehub # Download and install a utility script script_path = kagglehub.utility_script_install( 'bjoernjostein/physionet-challenge-utility-script' ) # Now you can import from the utility script from utility_module import helper_function result = helper_function(data) # Force re-download script_path = kagglehub.utility_script_install( 'bjoernjostein/physionet-challenge-utility-script', force_download=True ) ``` -------------------------------- ### Utility Script Function Source: https://github.com/kaggle/kagglehub/blob/main/_autodocs/COMPLETION_SUMMARY.txt Provides functions for installing utility scripts. ```APIDOC ## utility_script_install() ### Description Installs a specified utility script. ### Method utility_script_install ### Parameters (Signature and parameter details would be here if available in source) ### Response (Response details would be here if available in source) ``` -------------------------------- ### Utility Script Operations Source: https://github.com/kaggle/kagglehub/blob/main/_autodocs/README.md Function for installing utility scripts. ```APIDOC ## Utility Scripts ### `utility_script_install(handle, force_download=False)` #### Description Installs a utility script from Kaggle. #### Parameters - **handle** (string) - Required - The unique identifier for the utility script. - **force_download** (boolean) - Optional - If true, forces a download even if the script is already installed. ``` -------------------------------- ### Package Operations Source: https://github.com/kaggle/kagglehub/blob/main/_autodocs/README.md Functions for importing Kaggle Packages and getting asset paths. ```APIDOC ## Packages ### `package_import(handle, force_download=False, bypass_confirmation=False)` #### Description Imports a Kaggle Package. #### Parameters - **handle** (string) - Required - The unique identifier for the package. - **force_download** (boolean) - Optional - If true, forces a download even if the package is already installed. - **bypass_confirmation** (boolean) - Optional - If true, bypasses the confirmation prompt before installation. ### `get_package_asset_path(path)` #### Description Returns the local file path to an asset within an installed Kaggle Package. #### Parameters - **path** (string) - Required - The path to the asset within the package. ``` -------------------------------- ### Interactive KaggleHub Login Source: https://github.com/kaggle/kagglehub/blob/main/_autodocs/OVERVIEW.md Provides examples for logging into KaggleHub, either interactively via a notebook widget or terminal prompt, or programmatically by setting the API token. ```python import kagglehub # Interactive login (notebook widget or terminal prompt) kagglehub.login() # Check who's logged in user_info = kagglehub.whoami() print(f"Logged in as: {user_info['username']}") # Programmatic credential setting from kagglehub.config import set_kaggle_api_token set_kaggle_api_token('your-api-token-here') ``` -------------------------------- ### package_import() Source: https://github.com/kaggle/kagglehub/blob/main/_autodocs/api-reference/packages.md Downloads a Kaggle Package and imports it as a Python module. It handles the entire import process, including downloading, dependency installation, and module registration. ```APIDOC ## package_import() ### Description Download a Kaggle Package and import it as a Python module. A Kaggle Package is a published Kaggle Notebook that has been exported to a Python package format (containing a `package/` directory with `__init__.py`). This function handles the complete import process: downloading from Kaggle, confirming code execution (if needed), installing dependencies, and registering the module in `sys.modules`. ### Method `package_import(handle: str, *, force_download: bool | None = False, bypass_confirmation: bool = False) -> ModuleType` ### Parameters #### Path Parameters - **handle** (str) - Required - Package handle in format `{owner}/{notebook}` or `{owner}/{notebook}/versions/{version}` #### Query Parameters - **force_download** (bool) - Optional - Force re-download and re-import even if already cached/imported. Clears existing module from `sys.modules` before re-importing. - **bypass_confirmation** (bool) - Optional - Skip the user confirmation prompt before executing untrusted code. Confirmation is always skipped in Kaggle notebooks, Colab, and for packages owned by the logged-in user. ### Returns Imported Python module (ModuleType) with the package's public API. ### Raises - `ValueError`: If the notebook is not a valid Package (missing `package/__init__.py`). - `ImportError`: If the module specification cannot be created from the downloaded files. - `UserCancelledError`: If user declines the code execution confirmation prompt. - `KaggleApiHTTPError`: If the notebook is not found or access is denied. ### Example ```python import kagglehub # Import a package (may prompt for confirmation) my_package = kagglehub.package_import('username/my-package') # Import specific version my_package = kagglehub.package_import('username/my-package/versions/2') # Force re-download and re-import my_package = kagglehub.package_import('username/my-package', force_download=True) # Skip confirmation (e.g., in automated scripts) my_package = kagglehub.package_import('username/my-package', bypass_confirmation=True) # Use imported package result = my_package.some_function() ``` ``` -------------------------------- ### Load Hugging Face Dataset from SQLite via SQL Query Source: https://github.com/kaggle/kagglehub/blob/main/README.md Loads a Hugging Face Dataset by executing a SQL query against a SQLite database file. This allows for selective loading of data directly from a database. The example also shows how to rename a column after loading. ```python import kagglehub from kagglehub import KaggleDatasetAdapter # Load a Dataset by executing a SQL query against a SQLite DB, then rename a column dataset = kagglehub.dataset_load( KaggleDatasetAdapter.HUGGING_FACE, "wyattowalsh/basketball", "nba.sqlite", sql_query="SELECT person_id, player_name FROM draft_history", ) dataset = dataset.rename_column('season', 'year') ``` -------------------------------- ### Package Functions Source: https://github.com/kaggle/kagglehub/blob/main/_autodocs/COMPLETION_SUMMARY.txt Provides functions for importing packages and getting asset paths. ```APIDOC ## package_import() ### Description Imports a specified package. ### Method package_import ### Parameters (Signature and parameter details would be here if available in source) ### Response (Response details would be here if available in source) ## get_package_asset_path() ### Description Retrieves the file path for an asset within a package. ### Method get_package_asset_path ### Parameters (Signature and parameter details would be here if available in source) ### Response (Response details would be here if available in source) ``` -------------------------------- ### Utility Script Function Source: https://github.com/kaggle/kagglehub/blob/main/_autodocs/INDEX.md Function for downloading and installing utility scripts from Kaggle. This allows for easy integration of helpful scripts into the development environment. ```APIDOC ## `utility_script_install(handle, force_download=False)` ### Description Download and install a utility script from Kaggle. ### Parameters - **`handle`** (ResourceHandle) - Required - The handle identifying the utility script to install. - **`force_download`** (bool) - Optional - Force download even if the script is already installed. ### Module kagglehub.utility_scripts ``` -------------------------------- ### Handle KaggleEnvironmentError Source: https://github.com/kaggle/kagglehub/blob/main/_autodocs/errors.md Use this snippet to handle `KaggleEnvironmentError` which indicates issues with the Kaggle environment setup. It prints an informative message when the error occurs. ```python try: path = kagglehub.model_download('owner/model/framework/variation') except KaggleEnvironmentError as e: print(f"Kaggle environment issue: {e}") ``` -------------------------------- ### Force Download Kaggle Model Source: https://github.com/kaggle/kagglehub/blob/main/README.md Download a Kaggle model, even if it has been previously downloaded to the cache. Use `force_download=True` to ensure you get the latest files or to refresh the cache. ```python # Download a model or file, even if previously downloaded to cache. kagglehub.model_download('google/bert/tensorFlow2/answer-equivalence-bem', force_download=True) ``` -------------------------------- ### Import a Kaggle Package Source: https://github.com/kaggle/kagglehub/blob/main/_autodocs/api-reference/packages.md Use `package_import` to download and import a Kaggle Package as a Python module. You can specify a handle, force re-download, or bypass confirmation prompts. This function handles the complete import process, including dependency installation. ```python import kagglehub # Import a package (may prompt for confirmation) my_package = kagglehub.package_import('username/my-package') # Import specific version my_package = kagglehub.package_import('username/my-package/versions/2') # Force re-download and re-import my_package = kagglehub.package_import('username/my-package', force_download=True) # Skip confirmation (e.g., in automated scripts) my_package = kagglehub.package_import('username/my-package', bypass_confirmation=True) # Use imported package result = my_package.some_function() ``` -------------------------------- ### Build Project Source: https://github.com/kaggle/kagglehub/blob/main/README.md Build the project using hatch. ```sh hatch build ``` -------------------------------- ### utility_script_install() Source: https://github.com/kaggle/kagglehub/blob/main/_autodocs/api-reference/utility_scripts.md Downloads a Kaggle Utility Script and adds its directory to Python's system path, enabling direct imports. ```APIDOC ## utility_script_install() ### Description Download a Kaggle Utility Script and add its directory to Python's system path. This allows code from the utility script to be imported and used directly in your Python environment. A Utility Script is a special type of Kaggle Notebook that is tagged as a "utility script" and is intended to be reused. After downloading, this function adds the script directory to `sys.path` so you can import functions and classes from it. ### Parameters #### Path Parameters - **handle** (str) - Required - Notebook handle in format `{owner}/{notebook}` or `{owner}/{notebook}/versions/{version}`. The notebook must be tagged as a utility script. - **force_download** (bool) - Optional - Force re-download even if already cached. Defaults to `False`. ### Returns String path to the downloaded utility script directory (added to `sys.path`). ### Raises - `KaggleApiHTTPError`: If the notebook is not found (404) or access is denied (403/401). - `DataCorruptionError`: If downloaded file integrity check fails. ### Example ```python import kagglehub # Download and install a utility script script_path = kagglehub.utility_script_install( 'bjoernjostein/physionet-challenge-utility-script' ) # Now you can import from the utility script from utility_module import helper_function result = helper_function(data) # Force re-download script_path = kagglehub.utility_script_install( 'bjoernjostein/physionet-challenge-utility-script', force_download=True ) ``` ``` -------------------------------- ### Load Dataset with Adapters Source: https://github.com/kaggle/kagglehub/blob/main/_autodocs/OVERVIEW.md Demonstrates how to load dataset files into various Python objects using different adapters. Specify `sql_query` for SQL files or `hf_kwargs` for Hugging Face datasets. ```python data = kagglehub.dataset_load( adapter, # KaggleDatasetAdapter.PANDAS, HUGGING_FACE, or POLARS handle, # Dataset identifier path, # File path within dataset pandas_kwargs=None, # For pandas adapter sql_query=None, # For SQL files hf_kwargs=None, # For Hugging Face adapter polars_frame_type=None, # For Polars adapter polars_kwargs=None # For Polars adapter ) ``` -------------------------------- ### Load CSV as Pandas DataFrame Source: https://github.com/kaggle/kagglehub/blob/main/_autodocs/api-reference/datasets.md Loads a CSV file into a Pandas DataFrame. Ensure the 'pandas-datasets' extra is installed. ```python import kagglehub from kagglehub import KaggleDatasetAdapter, PolarsFrameType # Load CSV as pandas DataFrame df = kagglehub.dataset_load( KaggleDatasetAdapter.PANDAS, "unsdsn/world-happiness/versions/1", "2016.csv" ) ``` -------------------------------- ### Import and Use KaggleHub Library Source: https://github.com/kaggle/kagglehub/blob/main/_autodocs/OVERVIEW.md Demonstrates basic imports and usage of core kagglehub functions for authentication, model and dataset operations, and package management. ```python import kagglehub # Authentication kagglehub.login() kagglehub.whoami() # Models kagglehub.model_download(handle, path=None, force_download=False, output_dir=None) kagglehub.model_upload(handle, local_model_dir, license_name=None, version_notes="", ...) # Datasets kagglehub.dataset_download(handle, path=None, force_download=False, output_dir=None) kagglehub.dataset_upload(handle, local_dataset_dir, version_notes="", ...) kagglehub.dataset_load(adapter, handle, path, pandas_kwargs=None, hf_kwargs=None, ...) kagglehub.load_dataset(...) # Deprecated, use dataset_load() # Competitions kagglehub.competition_download(handle, path=None, force_download=False, output_dir=None) # Notebooks kagglehub.notebook_output_download(handle, path=None, force_download=False, output_dir=None) # Packages kagglehub.package_import(handle, force_download=False, bypass_confirmation=False) kagglehub.get_package_asset_path(path) # Utility Scripts kagglehub.utility_script_install(handle, force_download=False) # Types from kagglehub import KaggleDatasetAdapter, PolarsFrameType ``` -------------------------------- ### Get Kaggle Credentials Source: https://github.com/kaggle/kagglehub/blob/main/_autodocs/api-reference/authentication.md Retrieve the currently configured Kaggle credentials. Returns `None` if no credentials are set or available. ```python def get_kaggle_credentials() -> KaggleApiCredentials | None: pass ``` -------------------------------- ### Handle UnauthenticatedError Source: https://github.com/kaggle/kagglehub/blob/main/_autodocs/errors.md Catch `UnauthenticatedError` when calling functions like `whoami()` that require authentication. This example prompts the user to log in if they are not authenticated. ```python try: user = kagglehub.whoami() except UnauthenticatedError: print("Please login first") kagglehub.login() ``` -------------------------------- ### Handle Non-existent Resource (404) Source: https://github.com/kaggle/kagglehub/blob/main/_autodocs/errors.md Demonstrates catching a not found error (HTTP 404) for a non-existent resource using KaggleApiHTTPError and suggests listing available resources. ```python # Scenario 2: Non-existent resource try: path = kagglehub.competition_download('nonexistent-competition') except KaggleApiHTTPError as e: if e.response and e.response.status_code == 404: print("Competition not found") # List available competitions ``` -------------------------------- ### Get Cache Folder Source: https://github.com/kaggle/kagglehub/blob/main/_autodocs/configuration.md Retrieves the configured cache directory. It prioritizes the KAGGLEHUB_CACHE environment variable and falls back to a default location. ```python from kagglehub.config import get_cache_folder cache_dir = get_cache_folder() ``` -------------------------------- ### Download Model and Print Path Source: https://github.com/kaggle/kagglehub/blob/main/README.md Download a model from the command line using hatch and print its path. ```sh hatch run python -c "import kagglehub; print('path: ', kagglehub.model_download('google/bert/tensorFlow2/answer-equivalence-bem'))" ``` -------------------------------- ### Load CSV Dataset into Pandas DataFrame Source: https://github.com/kaggle/kagglehub/blob/main/README.md Loads a CSV file from a Kaggle dataset into a pandas DataFrame. Ensure the 'pandas-datasets' extra is installed. ```python import kagglehub from kagglehub import KaggleDatasetAdapter # Load a DataFrame with a specific version of a CSV df = kagglehub.dataset_load( KaggleDatasetAdapter.PANDAS, "unsdsn/world-happiness/versions/1", "2016.csv", ) ``` -------------------------------- ### Upload Resource Pattern Source: https://github.com/kaggle/kagglehub/blob/main/_autodocs/OVERVIEW.md Shows the general pattern for uploading local directories as resources. The `handle` should not include a version. ```python kagglehub.{resource}_upload( handle, # Resource identifier (unversioned) local_dir, # Local directory to upload version_notes="", # Optional: version description ignore_patterns=None, # Optional: files to exclude # ... other resource-specific options ) ``` -------------------------------- ### Upload and Sign Model with Sigstore Source: https://github.com/kaggle/kagglehub/blob/main/_autodocs/api-reference/models.md Uploads a model and creates a transparent ledger entry on sigstore. Requires `kagglehub[signing]` extras. ```python # Upload and sign with sigstore kagglehub.model_upload(handle, './local/model/path', sigstore=True) ``` -------------------------------- ### Configure Hatch Environment Directory Source: https://github.com/kaggle/kagglehub/blob/main/README.md Set the hatch configuration to create virtual environments in the project folder. ```bash hatch config set dirs.env.virtual .env ``` -------------------------------- ### Download Latest Kaggle Model Source: https://github.com/kaggle/kagglehub/blob/main/README.md Download the latest version of a Kaggle model using its handle. This is the simplest way to get the most recent model. ```python import kagglehub # Download the latest version. kagglehub.model_download('google/bert/tensorFlow2/answer-equivalence-bem') ``` -------------------------------- ### Run All Tests Source: https://github.com/kaggle/kagglehub/blob/main/README.md Execute all tests for the current Python version using hatch. ```sh hatch test ``` -------------------------------- ### Run Integration Tests Source: https://github.com/kaggle/kagglehub/blob/main/README.md Execute all integration tests after setting up Kaggle API credentials. ```sh hatch test integration_tests ``` -------------------------------- ### Get Log Verbosity Source: https://github.com/kaggle/kagglehub/blob/main/_autodocs/configuration.md Retrieves the current logging verbosity level, returning it as a standard Python logging level constant (e.g., logging.INFO). ```python from kagglehub.config import get_log_verbosity log_level = get_log_verbosity() ``` -------------------------------- ### Run Python Command in Docker with Optional Dependencies Source: https://github.com/kaggle/kagglehub/blob/main/README.md Execute a Python command within a Docker container using an environment with optional dependencies installed. ```sh ./docker-hatch run extra-deps-env:python -c "print('hello world')" ``` -------------------------------- ### Format Code Source: https://github.com/kaggle/kagglehub/blob/main/README.md Format the project's code using hatch. ```sh hatch run lint:fmt ``` -------------------------------- ### Download Kaggle Model to Custom Directory Source: https://github.com/kaggle/kagglehub/blob/main/README.md Download a Kaggle model to a custom local directory using the `output_dir` argument. This allows for better organization of downloaded resources. ```python # Download to a custom local directory. kagglehub.model_download('google/bert/tensorFlow2/answer-equivalence-bem', output_dir='./models') ``` -------------------------------- ### Kagglehub Authentication and Resource Operations Source: https://github.com/kaggle/kagglehub/blob/main/_autodocs/README.md Demonstrates the import of the kagglehub library and showcases the signatures for authentication, model, dataset, competition, notebook, package, and utility script operations. This snippet provides an overview of the library's capabilities. ```python import kagglehub from kagglehub import KaggleDatasetAdapter, PolarsFrameType # Authentication kagglehub.login() kagglehub.whoami() # Models kagglehub.model_download(handle, path=None, force_download=False, output_dir=None) kagglehub.model_upload(handle, local_model_dir, license_name=None, version_notes="", ...) # Datasets kagglehub.dataset_download(handle, path=None, force_download=False, output_dir=None) kagglehub.dataset_upload(handle, local_dataset_dir, version_notes="", ...) kagglehub.dataset_load(adapter, handle, path, pandas_kwargs=None, ...) # Competitions kagglehub.competition_download(handle, path=None, force_download=False, output_dir=None) # Notebooks kagglehub.notebook_output_download(handle, path=None, force_download=False, output_dir=None) # Packages kagglehub.package_import(handle, force_download=False, bypass_confirmation=False) kagglehub.get_package_asset_path(path) # Utility Scripts kagglehub.utility_script_install(handle, force_download=False) ``` -------------------------------- ### Configure Kaggle API Credentials Source: https://github.com/kaggle/kagglehub/blob/main/integration_tests/README.md Set the Kaggle username and API key as environment variables before running integration tests. Obtain the KAGGLE_KEY from the provided internal link. ```bash export KAGGLE_USERNAME=integrationtester export KAGGLE_KEY=... ``` -------------------------------- ### Load XML Dataset into Pandas DataFrame Source: https://github.com/kaggle/kagglehub/blob/main/README.md Loads an XML file from a Kaggle dataset into a pandas DataFrame using the 'etree' parser. Ensure 'pandas-datasets' is installed. ```python import kagglehub from kagglehub import KaggleDatasetAdapter # Load a DataFrame using an XML file (with the natively available etree parser) df = dataset_load( KaggleDatasetAdapter.PANDAS, "parulpandey/covid19-clinical-trials-dataset", "COVID-19 CLinical trials studies/COVID-19 CLinical trials studies/NCT00571389.xml", pandas_kwargs={"parser": "etree"}, ) ``` -------------------------------- ### Run Docker Hatch Command Source: https://github.com/kaggle/kagglehub/blob/main/README.md Execute a docker-hatch command with specific version and pass arguments to hatch. ```bash ./docker-hatch -v 3.10 -- -v env create debug-env-with-verbose-logging ``` -------------------------------- ### Handle ColabEnvironmentError Source: https://github.com/kaggle/kagglehub/blob/main/_autodocs/errors.md Catch `ColabEnvironmentError` when performing operations that rely on the Colab environment. This example shows how to print an error message and potentially fall back to a standard cache. ```python try: path = kagglehub.dataset_download('owner/dataset') except ColabEnvironmentError as e: print(f"Colab issue: {e}") # Fall back to standard cache ``` -------------------------------- ### Run Saved Script from /tools/scripts Source: https://github.com/kaggle/kagglehub/blob/main/README.md Execute a pre-defined script located in the /tools/scripts directory using hatch. ```sh hatch run python tools/scripts/download_model.py ``` -------------------------------- ### Download Specific Version of Kaggle Model Source: https://github.com/kaggle/kagglehub/blob/main/README.md Download a specific version of a Kaggle model by appending the version number to the handle. This ensures you get a reproducible model version. ```python # Download a specific version. kagglehub.model_download('google/bert/tensorFlow2/answer-equivalence-bem/1') ``` -------------------------------- ### Cache Management Class Source: https://github.com/kaggle/kagglehub/blob/main/_autodocs/types.md Helper class for managing cache operations, including getting paths, loading, and marking completion status. Supports an optional override directory. ```python class Cache: def __init__(self, override_dir: str | None = None) -> None ... def get_path(self, handle: ResourceHandle, path: str | None = None) -> str ... def load_from_cache(self, handle: ResourceHandle, path: str | None = None) -> str | None ... def mark_as_complete(self, handle: ResourceHandle, path: str | None = None) -> None ... def mark_as_incomplete(self, handle: ResourceHandle, path: str | None = None) -> None ... def delete_from_cache(self, handle: ResourceHandle, path: str | None = None) -> str | None ... ``` -------------------------------- ### Download Competition Data with Kagglehub Source: https://github.com/kaggle/kagglehub/blob/main/_autodocs/api-reference/competitions.md Use `competition_download` to fetch entire competition datasets or specific files. You can specify a custom output directory and force re-downloads. ```python import kagglehub # Download entire competition dataset path = kagglehub.competition_download('digit-recognizer') # Download a single file path = kagglehub.competition_download('digit-recognizer', path='train.csv') # Force re-download even if cached path = kagglehub.competition_download('digit-recognizer', force_download=True) # Download to custom directory path = kagglehub.competition_download('digit-recognizer', output_dir='./competition') # Overwrite existing directory path = kagglehub.competition_download('digit-recognizer', output_dir='./competition', force_download=True) ``` -------------------------------- ### Get Authenticated User's Username Source: https://github.com/kaggle/kagglehub/blob/main/_autodocs/api-reference/authentication.md Retrieve the username of the currently authenticated Kaggle user. This function can optionally log messages during credential validation. It raises an error if no valid credentials are found. ```python import kagglehub # Get the authenticated user's username result = kagglehub.whoami() print(result["username"]) # Get username without log messages result = kagglehub.whoami(verbose=False) ``` -------------------------------- ### Download Resource Pattern Source: https://github.com/kaggle/kagglehub/blob/main/_autodocs/OVERVIEW.md Illustrates the common function signature for downloading various resources like models, datasets, or competition data. Use `force_download=True` to re-download cached files. ```python path = kagglehub.{resource}_download( handle, # Resource identifier path=None, # Optional: specific file within resource force_download=False, # Optional: re-download even if cached output_dir=None # Optional: bypass cache, download to directory ) ``` -------------------------------- ### login() Source: https://github.com/kaggle/kagglehub/blob/main/_autodocs/api-reference/authentication.md Prompts the user for their Kaggle API token and saves it globally. It can display an interactive widget in notebook environments or use `getpass` in standard interpreters. ```APIDOC ## login() ### Description Prompts the user for their Kaggle API token and saves it globally. In a notebook environment (Jupyter, Kaggle Notebook, Google Colab), this displays an interactive widget. In a standard Python interpreter, it prompts for input via `getpass`. ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Method ```python def login(validate_credentials: bool = True) -> None ``` ### Parameters - **validate_credentials** (bool) - Optional - Default: True - Whether to validate credentials after saving them. If True, makes a request to verify the token is valid. ### Returns None ### Raises - `ImportError`: If `ipywidgets` is not installed when calling from a notebook environment and `validate_credentials=True`. - `UnauthenticatedError`: If credential validation fails. ### Example ```python import kagglehub # Prompt for credentials in notebook or terminal kagglehub.login() # Prompt without validating credentials kagglehub.login(validate_credentials=False) ``` ``` -------------------------------- ### Load SQLite Database Table into Pandas DataFrame via SQL Query Source: https://github.com/kaggle/kagglehub/blob/main/README.md Loads data from a SQLite database file on Kaggle into a pandas DataFrame by executing a specified SQL query. Requires 'pandas-datasets' installation. ```python import kagglehub from kagglehub import KaggleDatasetAdapter # Load a DataFrame by executing a SQL query against a SQLite DB df = kagglehub.dataset_load( KaggleDatasetAdapter.PANDAS, "wyattowalsh/basketball", "nba.sqlite", sql_query="SELECT person_id, player_name FROM draft_history", ) ``` -------------------------------- ### Upload Model with Version Notes Source: https://github.com/kaggle/kagglehub/blob/main/_autodocs/api-reference/models.md Uploads a model and includes descriptive notes for the new version. ```python # Upload with version notes kagglehub.model_upload(handle, './local/model/path', version_notes='improved accuracy') ``` -------------------------------- ### Set Kaggle API Token Environment Variable Source: https://github.com/kaggle/kagglehub/blob/main/_autodocs/OVERVIEW.md Instructions for setting the Kaggle API token as an environment variable for programmatic authentication. ```bash # Example for Linux/macOS: # export KAGGLE_API_TOKEN="your-api-token-here" # Example for Windows (Command Prompt): # set KAGGLE_API_TOKEN=your-api-token-here # Example for Windows (PowerShell): # $env:KAGGLE_API_TOKEN="your-api-token-here" ``` -------------------------------- ### Cache Source: https://github.com/kaggle/kagglehub/blob/main/_autodocs/types.md Helper class for managing cache for Kaggle resources, with an option to override the default cache directory. It provides methods to get cache paths, load resources from cache, and manage cache completion status. ```APIDOC ## Cache ### Description Helper class for cache management with optional override directory. ### Methods - `__init__(override_dir: str | None = None)`: Initializes the Cache with an optional override directory. - `get_path(handle: ResourceHandle, path: str | None = None) -> str`: Get the cache path for a resource. - `load_from_cache(handle: ResourceHandle, path: str | None = None) -> str | None`: Load cached resource if exists and complete. - `mark_as_complete(handle: ResourceHandle, path: str | None = None) -> None`: Mark a resource download as complete. - `mark_as_incomplete(handle: ResourceHandle, path: str | None = None) -> None`: Mark a resource download as incomplete. - `delete_from_cache(handle: ResourceHandle, path: str | None = None) -> str | None`: Delete a resource from cache. ``` -------------------------------- ### Run Temporary Script from Repository Root Source: https://github.com/kaggle/kagglehub/blob/main/README.md Execute a temporary Python script placed at the root of the repository using hatch. ```sh hatch run python test_new_feature.py ``` -------------------------------- ### Get Path to Package Asset Source: https://github.com/kaggle/kagglehub/blob/main/_autodocs/api-reference/packages.md Use `get_package_asset_path` to retrieve the path to an asset file within a Kaggle Package. This works both when code is executed inside an imported package and when creating a new package in a Kaggle Notebook. ```python import kagglehub from pathlib import Path # When inside an imported package: my_package = kagglehub.package_import('username/my-package') # Get path to an asset model_path = kagglehub.get_package_asset_path('models/classifier.pkl') model = load_model(model_path) # Get path and use with pathlib config_file = kagglehub.get_package_asset_path('config.json') config_data = json.load(config_file.open()) # When creating a package in a Kaggle Notebook: # This writes to /kaggle/package_assets during development asset_path = kagglehub.get_package_asset_path('data/processed.csv') data.to_csv(asset_path) ``` -------------------------------- ### Load Excel as Dictionary of LazyFrames Source: https://github.com/kaggle/kagglehub/blob/main/README.md Loads an Excel file, returning a dictionary where keys are sheet names and values are LazyFrames for each sheet's data. Requires the 'fastexcel' engine to be installed. Use `sheet_id=0` to load all sheets. ```python import kagglehub from kagglehub import KaggleDatasetAdapter, PolarsFrameType # Load a dictionary of LazyFrames from an Excel file where the keys are sheet names # and the values are LazyFrames for each sheet's data. NOTE: As written, this requires # installing the default fastexcel engine. lf_dict = kagglehub.dataset_load( KaggleDatasetAdapter.POLARS, "theworldbank/education-statistics", "edstats-excel-zip-72-mb-/EdStatsEXCEL.xlsx", # sheet_id of 0 returns all sheets polars_kwargs={"sheet_id": 0}, ) ``` -------------------------------- ### Run All Tests for All Python Versions Source: https://github.com/kaggle/kagglehub/blob/main/README.md Execute all tests across all supported Python versions using hatch. ```sh hatch test --all ``` -------------------------------- ### NotebookHandle Source: https://github.com/kaggle/kagglehub/blob/main/_autodocs/types.md Represents a Kaggle notebook or notebook output, including its owner, slug, and an optional version number. It provides methods to check versioning, create new handles with specific versions, get string representations, and generate Kaggle URLs. ```APIDOC ## NotebookHandle ### Description Represents a Kaggle notebook or notebook output. ### Fields - **owner** (str) - Kaggle username of notebook creator - **notebook** (str) - Notebook slug/name - **version** (int or None) - Version number (None if unversioned, default=None) ### Methods - `is_versioned() -> bool`: Returns True if version is set - `with_version(version: int) -> NotebookHandle`: Returns a new handle with specified version - `__str__() -> str`: Returns the string representation (handle format) - `to_url() -> str`: Returns the Kaggle URL to the notebook ``` -------------------------------- ### Upload Kaggle Model with License Source: https://github.com/kaggle/kagglehub/blob/main/README.md Upload a local model directory to Kaggle and specify a license. This is important for defining the usage rights of your model. ```python # You can also specify a license (optional) kagglehub.model_upload(handle, local_model_dir, license_name='Apache 2.0') ``` -------------------------------- ### Enable File Logging Source: https://github.com/kaggle/kagglehub/blob/main/_autodocs/configuration.md Enable file-based logging for kagglehub by setting the KAGGLE_LOGGING_ENABLED environment variable to 1. ```bash export KAGGLE_LOGGING_ENABLED=1 ``` -------------------------------- ### Run Single Test File Source: https://github.com/kaggle/kagglehub/blob/main/README.md Execute tests for a single specified file using hatch. ```sh hatch test tests/test_.py ``` -------------------------------- ### kagglehub.dataset_load Source: https://github.com/kaggle/kagglehub/blob/main/_autodocs/api-reference/datasets.md Loads a dataset from Kaggle using a specified adapter. Supports various file types and provides options for customizing the loading process with adapter-specific keyword arguments. ```APIDOC ## kagglehub.dataset_load ### Description Loads a dataset from Kaggle using a specified adapter. Supports various file types and provides options for customizing the loading process with adapter-specific keyword arguments. ### Parameters #### Path Parameters - **adapter** (KaggleDatasetAdapter) - Yes - The adapter to use for loading: `PANDAS`, `HUGGING_FACE`, or `POLARS` - **handle** (str) - Yes - Dataset handle in format `{owner}/{dataset}` or `{owner}/{dataset}/versions/{version}` - **path** (str) - Yes - Path to a specific file within the dataset (e.g., "data.csv") #### Query Parameters - **pandas_kwargs** (dict) - No - Keyword arguments passed to the pandas `read_*` method. See pandas documentation for details. - **sql_query** (str) - No - SQL query for SQLite files. Required when reading a `.sqlite`/`.db` file. - **hf_kwargs** (dict) - No - Keyword arguments passed to `datasets.Dataset.from_pandas()`. Only for `HUGGING_FACE` adapter. - **polars_frame_type** (PolarsFrameType) - No - Controls return type: `LAZY_FRAME` (default) or `DATA_FRAME`. Only for `POLARS` adapter. - **polars_kwargs** (dict) - No - Keyword arguments passed to polars `scan_*` or `read_*` methods. Only for `POLARS` adapter. ### Returns Depends on adapter: - `PANDAS`: `pandas.DataFrame` or `dict[int|str, pandas.DataFrame]` (for multi-sheet files like Excel) - `HUGGING_FACE`: `datasets.Dataset` (via `Dataset.from_pandas()`) - `POLARS`: `polars.LazyFrame` or `polars.DataFrame`, or dict of these for multi-sheet files ### Raises - `ImportError`: If required optional dependencies are not installed. - `ValueError`: If invalid kwargs are provided for the adapter. - `KaggleApiHTTPError`: If the dataset or file is not found or access is denied. ### Supported File Types #### PANDAS Adapter | Extension | Method | |---|---| | `.csv`, `.tsv` | `pandas.read_csv()` | | `.json`, `.jsonl` | `pandas.read_json()` | | `.xml` | `pandas.read_xml()` | | `.parquet` | `pandas.read_parquet()` | | `.feather` | `pandas.read_feather()` | | `.sqlite`, `.db`, etc. | `pandas.read_sql_query()` | | `.xls`, `.xlsx`, `.odf`, etc. | `pandas.read_excel()` | #### HUGGING_FACE Adapter Same file types as PANDAS adapter. Uses `datasets.Dataset.from_pandas()` internally. #### POLARS Adapter | Extension | Methods | |---|---| | `.csv`, `.tsv` | `polars.scan_csv()` or `polars.read_csv()` | | `.json` | `polars.read_json()` | | `.jsonl` | `polars.scan_ndjson()` or `polars.read_ndjson()` | | `.parquet` | `polars.scan_parquet()` or `polars.read_parquet()` | | `.feather` | `polars.scan_ipc()` or `polars.read_ipc()` | | `.sqlite`, `.db`, etc. | `polars.read_database()` | | `.xlsx`, `.odf`, etc. | `polars.read_excel()` | ### Request Example ```python import kagglehub from kagglehub import KaggleDatasetAdapter, PolarsFrameType # Load CSV as pandas DataFrame df = kagglehub.dataset_load( KaggleDatasetAdapter.PANDAS, "unsdsn/world-happiness/versions/1", "2016.csv" ) # Load with specific columns df = kagglehub.dataset_load( KaggleDatasetAdapter.PANDAS, "robikscube/textocr-text-extraction-from-images-dataset", "annot.parquet", pandas_kwargs={"columns": ["image_id", "bbox"]} ) # Load as Hugging Face Dataset dataset = kagglehub.dataset_load( KaggleDatasetAdapter.HUGGING_FACE, "unsdsn/world-happiness/versions/1", "2016.csv" ) # Load from SQLite with query df = kagglehub.dataset_load( KaggleDatasetAdapter.PANDAS, "wyattowalsh/basketball", "nba.sqlite", sql_query="SELECT person_id, player_name FROM draft_history" ) # Load as Polars DataFrame (not LazyFrame) df = kagglehub.dataset_load( KaggleDatasetAdapter.POLARS, "robikscube/textocr-text-extraction-from-images-dataset", "annot.parquet", polars_frame_type=PolarsFrameType.DATA_FRAME, polars_kwargs={"columns": ["image_id", "bbox"]} ) # Load Excel with multiple sheets as Polars LazyFrames lf_dict = kagglehub.dataset_load( KaggleDatasetAdapter.POLARS, "theworldbank/education-statistics", "edstats-excel-zip-72-mb-/EdStatsEXCEL.xlsx", polars_kwargs={"sheet_id": 0} # 0 = all sheets ) ``` ``` -------------------------------- ### dataset_load() Source: https://github.com/kaggle/kagglehub/blob/main/_autodocs/api-reference/datasets.md Loads a Kaggle Dataset file into a Python object using a specified adapter. Downloads the file if not cached and loads it into the appropriate format (pandas DataFrame, Hugging Face Dataset, or Polars LazyFrame/DataFrame). ```APIDOC ## dataset_load() ### Description Load a Kaggle Dataset file into a Python object using the specified adapter. Downloads the file if not cached, then loads it into the appropriate format (pandas DataFrame, Hugging Face Dataset, or Polars LazyFrame/DataFrame). ### Parameters #### Path Parameters - **adapter** (KaggleDatasetAdapter) - Required - The adapter to use for loading the dataset. - **handle** (str) - Required - Dataset handle in format `{owner}/{dataset}` or `{owner}/{dataset}/versions/{version}`. - **path** (str) - Required - Path to the specific file within the dataset to load. #### Query Parameters - **pandas_kwargs** (Any) - Optional - Keyword arguments to pass to pandas when loading. - **sql_query** (str) - Optional - SQL query to execute if the dataset is in a database format. - **hf_kwargs** (Any) - Optional - Keyword arguments to pass to Hugging Face datasets when loading. - **polars_frame_type** (PolarsFrameType) - Optional - Specifies whether to load as a LazyFrame or DataFrame. - **polars_kwargs** (Any) - Optional - Keyword arguments to pass to Polars when loading. ### Returns Any - The loaded dataset object (e.g., pandas DataFrame, Hugging Face Dataset, Polars DataFrame/LazyFrame). ### Raises (Specific exceptions are not detailed in the source for this function, but would typically include errors related to file access, download failures, or adapter-specific loading issues.) ### Example ```python # Example usage would depend on the specific adapter and desired output format. # import kagglehub # from kagglehub import KaggleDatasetAdapter # import pandas as pd # # Example loading into a pandas DataFrame # adapter = KaggleDatasetAdapter() # df = kagglehub.dataset_load(adapter, 'user/dataset', 'data.csv', pandas_kwargs={'sep': ';'}) # # Example loading into a Hugging Face Dataset # from datasets import Dataset # hf_dataset = kagglehub.dataset_load(adapter, 'user/dataset', 'data.json', hf_kwargs={}) # # Example loading into a Polars LazyFrame # import polars as pl # lazy_df = kagglehub.dataset_load(adapter, 'user/dataset', 'data.parquet', polars_frame_type=pl.LazyFrame) ``` ``` -------------------------------- ### Handle Private Resource Access Denied (403) Source: https://github.com/kaggle/kagglehub/blob/main/_autodocs/errors.md Illustrates how to catch a forbidden access error (HTTP 403) for private resources using KaggleApiHTTPError and inform the user about lacking permissions. ```python # Scenario 1: Private resource without permission try: path = kagglehub.model_download('owner/private-model') except KaggleApiHTTPError as e: if e.response and e.response.status_code == 403: print("You don't have permission to access this resource") # Consider asking the owner for access ``` -------------------------------- ### Upload Kaggle Model with Version Notes Source: https://github.com/kaggle/kagglehub/blob/main/README.md Upload a local model directory to Kaggle and include version notes. This helps document changes or improvements in the new model version. ```python # You can also specify some version notes (optional) kagglehub.model_upload(handle, local_model_dir, version_notes='improved accuracy') ``` -------------------------------- ### Bypass Cache with output_dir Parameter Source: https://github.com/kaggle/kagglehub/blob/main/_autodocs/configuration.md Use the output_dir parameter in download functions to save resources directly to a specified local directory, bypassing the cache entirely. ```python path = kagglehub.dataset_download('owner/dataset', output_dir='./local/dir') ```