Kaggle

https://www.kaggle.com/docs
Admin
Kaggle is a platform for data science competitions, datasets, and machine learning projects where...
Tokens:75,286
Snippets:455
Trust Score:9.8
Update:3 weeks ago
Show doc for...
Tokens:
Raw
### Install Torch-XLA for TPU training

Source: https://www.kaggle.com/docs/tpu

Install Torch-XLA and set up the environment for TPU training in PyTorch. The commands use the nightly version and install required apt packages. After installation, you can run PyTorch code on TPU v3-8, which has 8 cores; to fully utilize the TPU, run code on all 8 cores simultaneously.

```bash
    # Step 1: Install Torch-XLA (PyTorch with Accelerated Linear Algebra (XLA) support)
    !curl https://raw.githubusercontent.com/pytorch/xla/master/contrib/scripts/env-setup.py -o pytorch-xla-env-setup.py
    !python pytorch-xla-env-setup.py --version nightly --apt-packages libomp5 libopenblas-dev

    # Step 2: Run your PyTorch code
    TPUs (TPU v3-8) have 8 cores, and each core is itself an XLA device.  
    You can run code on a single XLA device, but to take full advantage of 
    the TPU you will want to run your code on all 8 cores simultaneously. 
    For examples that demonstrate how to do this, you can refer to 
    The Ultimate PyTorch TPU Tutorial,
    I Like Clean TPU Training Kernels and I Can Not Lie,
    Super Duper Fast PyTorch TPU Kernel,
    and  XLM Roberta Large Pytorch TPU 

```

--------------------------------

### solution.csv example format

Source: https://www.kaggle.com/docs/competitions-setup

Example of the solution file format. It includes an 'id' column, a 'target_feature' column, and a 'Usage' column with values Public, Private, or Ignored. This file is hidden from participants and used for scoring.

```csv
id,target_feature,Usage
0,1,Public
1,0,Private
2,1,Ignored

```

--------------------------------

### Start Authorization Flow - GET Request

Source: https://www.kaggle.com/docs/api

Redirect the user to the Kaggle authorization endpoint. Required query parameters include client_id, redirect_uri, scope, state, response_type, response_mode, and for public clients code_challenge and code_challenge_method.

```http
GET https://www.kaggle.com/api/v1/oauth2/authorize
```

--------------------------------

### train.csv example format

Source: https://www.kaggle.com/docs/competitions-setup

Example of the training data file format. The first row is a header with feature and target column names, followed by data rows. This file is provided to participants for training their models.

```csv
input_feature1,input_feature2,target_feature
100,52.12,1
192,203.2,1

```

--------------------------------

### Docker image metadata in package/__init__.py

Source: https://www.kaggle.com/docs/packages

Example of the __docker_image__ metadata found in the Package's package/__init__.py file. Use this value to pull the correct Docker image for running the Package.

```python
__docker_image__ = 'gcr.io/kaggle-images/python@sha256:abcxyz...'
```

--------------------------------

### Authorize Kaggle MCP in Gemini CLI

Source: https://www.kaggle.com/docs/mcp

Starts OAuth 2.0 authorization for the Kaggle MCP server by running the command `/mcp auth kaggle` in Gemini CLI.

```bash
/mcp auth kaggle

```

--------------------------------

### Download Benchmark Leaderboard via cURL

Source: https://www.kaggle.com/docs/benchmarks

Use these cURL commands to retrieve leaderboard data as a JSON object. The authenticated example requires setting your KAGGLE_USERNAME and KAGGLE_KEY environment variables.

```bash
# Unauthenticated example
curl -L -o ~/Downloads/open-benchmarks_scicode_leaderboard.json \
  https://www.kaggle.com/api/v1/benchmarks/open-benchmarks/scicode/leaderboard

# Authenticated example
# Export your Kaggle username and API key
# export KAGGLE_USERNAME=
# export KAGGLE_KEY=

curl -L -u $KAGGLE_USERNAME:$KAGGLE_KEY \
  -o ~/Downloads/myusername_my-benchmark_leaderboard.json \
  https://www.kaggle.com/api/v1/benchmarks/myusername/my-benchmark/leaderboard
```

--------------------------------

### Run Docker container for a Kaggle Package

Source: https://www.kaggle.com/docs/packages

Run a Docker container from the Package's image and enter a shell. Replace gcr.io/... with the actual image tag. The image is over 20 GB. Optional arguments include --gpus all, -v for mounting a host directory, and --name for a custom container name.

```bash
docker run -it --rm \
  gcr.io/... \
  /bin/bash
```

--------------------------------

### Connect to Google Cloud Storage in Kaggle Notebook

Source: https://www.kaggle.com/docs/notebooks

Initializes a Google Cloud Storage client with your project ID. Requires a billing-enabled GCP account and may incur charges.

```python
# Set your own project id here
PROJECT_ID = 'your-google-cloud-project'
from google.cloud import storage
storage_client = storage.Client(project=PROJECT_ID)
```

--------------------------------

### Connect to BigQuery in Kaggle Notebook

Source: https://www.kaggle.com/docs/notebooks

Initializes a BigQuery client with your Google Cloud project ID. Requires a billing-enabled GCP account for non-public data and may incur charges.

```python
# Set your own project id here
PROJECT_ID = 'your-google-cloud-project'
from google.cloud import bigquery
bigquery_client = bigquery.Client(project=PROJECT_ID)
```

--------------------------------

### Download Gemma model via API with curl and wget

Source: https://www.kaggle.com/docs/models

Authenticate with environment variables and download the 2b PyTorch variation of google/gemma using curl or wget. Requires Kaggle credentials for restricted models.

```bash
# Authenticate with credentials
export KAGGLE_USERNAME=xyz
export KAGGLE_KEY=xyz

# With Curl
curl -L -o ~/Downloads/model.tar.gz  https://www.kaggle.com/api/v1/models/google/gemma/pyTorch/2b/1/download -u $KAGGLE_USERNAME:$KAGGLE_KEY

# Download specific version (here version 1)
wget https://www.kaggle.com/api/v1/models/google/gemma/pyTorch/2b/1/download --user=$KAGGLE_USERNAME --password=$KAGGLE_KEY --auth-no-challenge
```

--------------------------------

### get_competition_data_files_summary

Source: https://www.kaggle.com/docs/mcp

Gets a summary of the data files for a competition.

```APIDOC
## get_competition_data_files_summary

### Description
Gets a summary of the data files for a competition.

### Method
POST

### Endpoint
/api/competitions/data-files/summary

### Parameters
#### Path Parameters
- **competition_name** (string) - Required - The name of the competition.

### Request Example
{
  "competition_name": "string"
}

### Response
#### Success Response (200)
- **file_count** (int32) - Number of files.
- **total_size** (int64) - Total size in bytes.

#### Response Example
{
  "file_count": 0,
  "total_size": 0
}
```

--------------------------------

### get_model

Source: https://www.kaggle.com/docs/mcp

Gets metadata about a model. Requires owner_slug and model_slug.

```APIDOC
## get_model

### Description
Get a metadata about a model.

### Method
GET

### Endpoint
/models/{owner_slug}/{model_slug}

### Parameters
#### Path Parameters
- **owner_slug** (string) - Required - The slug of the owner of the model.
- **model_slug** (string) - Required - The slug of the model.

### Response
#### Success Response (200)
- **id** (int32) - The unique identifier of the model.
- **ref** (string) - The reference of the model.
- **title** (string) - The title of the model.
- **subtitle** (string) - The subtitle of the model.
- **author** (string) - The author of the model.
- **slug** (string) - The slug of the model.

#### Response Example
{
  "id": 1,
  "ref": "owner/model",
  "title": "Model Title",
  "subtitle": "Model Subtitle",
  "author": "Author",
  "slug": "model-slug"
}
```

--------------------------------

### GET https://www.kaggle.com/.well-known/oauth-protected-resource

Source: https://www.kaggle.com/docs/api

Retrieves metadata about the protected resource (Kaggle API).

```APIDOC
## GET https://www.kaggle.com/.well-known/oauth-protected-resource

### Description
Retrieves metadata about the protected resource (Kaggle API).

### Method
GET

### Endpoint
https://www.kaggle.com/.well-known/oauth-protected-resource

### Response
#### Success Response (200)
- Protected resource metadata (JSON) - Contains metadata about the Kaggle API.
```

--------------------------------

### Initialize Google AutoML client and project location

Source: https://www.kaggle.com/docs/notebooks

Initializes the AutoML client for the beta version (automl_v1beta1) and constructs the project location path. Requires a billing-enabled GCP account and the compute region must be 'us-central1' to use AutoML.

```python
# Set your own project id and compute region here
PROJECT_ID = 'your-google-cloud-project'
COMPUTE_REGION = 'us-central1'  # must be `us-central1` to use AutoML (see docs)
from google.cloud import automl_v1beta1 as automl
automl_client = automl.AutoMlClient()
project_location = automl_client.location_path(PROJECT_ID, COMPUTE_REGION)
```

--------------------------------

### Token Introspection - Invalid Token Response

Source: https://www.kaggle.com/docs/api

Example JSON response for an expired, revoked, or otherwise invalid token during introspection.

```json
{
  "active": false
}
```

--------------------------------

### download_model_variation_version

Source: https://www.kaggle.com/docs/mcp

Gets a URL where the files for a model variation version can be downloaded. Requires owner_slug, model_slug, framework, instance_slug, and version_number.

```APIDOC
## download_model_variation_version

### Description
Get a url where the files for a model variation version can be downloaded.

### Method
GET

### Endpoint
/models/{owner_slug}/{model_slug}/{framework}/{instance_slug}/versions/{version_number}

### Parameters
#### Path Parameters
- **owner_slug** (string) - Required - The slug of the owner of the model.
- **model_slug** (string) - Required - The slug of the model.
- **framework** (ModelFramework) - Required - Possible values: MODEL_FRAMEWORK_UNSPECIFIED, MODEL_FRAMEWORK_TENSOR_FLOW_1, MODEL_FRAMEWORK_TENSOR_FLOW_2, MODEL_FRAMEWORK_TF_LITE, MODEL_FRAMEWORK_TF_JS, MODEL_FRAMEWORK_PY_TORCH, MODEL_FRAMEWORK_JAX, MODEL_FRAMEWORK_FLAX, MODEL_FRAMEWORK_PAX, MODEL_FRAMEWORK_MAX_TEXT, MODEL_FRAMEWORK_GEMMA_CPP, MODEL_FRAMEWORK_GGML, MODEL_FRAMEWORK_GGUF, MODEL_FRAMEWORK_CORAL, MODEL_FRAMEWORK_SCIKIT_LEARN, MODEL_FRAMEWORK_MXNET, MODEL_FRAMEWORK_ONNX, MODEL_FRAMEWORK_KERAS, MODEL_FRAMEWORK_TRANSFORMERS, MODEL_FRAMEWORK_API, MODEL_FRAMEWORK_OTHER, MODEL_FRAMEWORK_TENSOR_RT_LLM, MODEL_FRAMEWORK_TRITON.
- **instance_slug** (string) - Required - The slug of the model instance.
- **version_number** (int32) - Required - The version number of the model instance.

#### Query Parameters
- **path** (string) - Optional - Relative path to a specific file inside the databundle.

### Response
#### Success Response (302)
- **HttpRedirect** - Returns a URL to download the model instance version.

#### Response Example
Location: https://download.example.com/model.zip
```

--------------------------------

### Set up Google Cloud credentials for private Kaggle Datasets

Source: https://www.kaggle.com/docs/tpu

Initializes Google Cloud credentials for using private Kaggle Datasets with TPUs. Requires enabling 'Google Cloud SDK' in the notebook's Add-ons menu. For public datasets, only Step 3 is needed.

```python
    # Step 1: Get the credential from the Cloud SDK
    from kaggle_secrets import UserSecretsClient
    user_secrets = UserSecretsClient()
    user_credential = user_secrets.get_gcloud_credential()
    
    # Step 2: Set the credentials
    user_secrets.set_tensorflow_credential(user_credential)

    # Step 3: Use a familiar call to get the GCS path of the dataset
    from kaggle_datasets import KaggleDatasets
    GCS_DS_PATH = KaggleDatasets().get_gcs_path()	
    
```

--------------------------------

### get_dataset_files_summary

Source: https://www.kaggle.com/docs/mcp

Get a summary of the data files in a dataset. This operation retrieves information about the files and columns in a specific dataset version.

```APIDOC
## get_dataset_files_summary

### Description
Get a summary of the data files in a dataset.

### Method
POST

### Endpoint
/api/datasets/files/summary

### Parameters
#### Path Parameters
- **owner_slug** (string) - Required - The slug of the owner of the dataset.
- **dataset_slug** (string) - Required - The slug of the dataset.
- **dataset_version_number** (int32) - Optional - The version number of the dataset.

### Request Example
{
  "owner_slug": "string",
  "dataset_slug": "string",
  "dataset_version_number": 0
}

### Response
#### Success Response (200)
- **file_summary_info** (ApiFileSummaryInfo) - Summary of files in the dataset.
  - **total_file_count** (int64) - Total number of files.
  - **file_types** (array of ApiFileExtensionSummaryInfo) - Summary of file types.
    - **extension** (string) - File extension.
    - **file_count** (int64) - Number of files with this extension.
    - **total_size** (int64) - Total size of files with this extension.
- **column_summary_info** (ApiColumnSummaryInfo) - Summary of columns in the dataset.
  - **total_column_count** (int64) - Total number of columns.
  - **column_types** (array of ApiColumnTypeSummaryInfo) - Summary of column types.
    - **column_type** (string) - Type of column.
    - **column_count** (int64) - Number of columns of this type.

#### Response Example
{
  "file_summary_info": {
    "total_file_count": 0,
    "file_types": [
      {
        "extension": "string",
        "file_count": 0,
        "total_size": 0
      }
    ]
  },
  "column_summary_info": {
    "total_column_count": 0,
    "column_types": [
      {
        "column_type": "string",
        "column_count": 0
      }
    ]
  }
}
```

--------------------------------

### PyTorch TPU training checklist

Source: https://www.kaggle.com/docs/tpu

Key steps for using TPUs with PyTorch, covering distributed training with xmp.spawn, model wrapping with MpModelWrapper, sending data to the XLA device, and using ParallelLoader for training. Also includes memory-optimized and standard model save/restore methods, and notes on loading out-of-memory datasets from localhost or GCS in TFRecord format.

```python
    #1: Startup Script 
    https://raw.githubusercontent.com/pytorch/xla/master/contrib/scripts/env-setup.py

    #2: Distributed training function mp_fn
    xmp.spawn(_mp_fn, nprocs=8, start_method='fork')

    #3: Instantiate model outside of mp_fn and use MpModelWrapper
    MX = JigsawModel()   =>    MX = xmp.MpModelWrapper(JigsawModel())

    #4: Send model to TPU device
    device = xm.xla_device()
    model = MX.to(device)

    #5: Changes to training loop: send data to device
    ids = ids.to(device, dtype=torch.long)
    token_type_ids = token_type_ids.to(device, dtype=torch.long)
    mask = mask.to(device, dtype=torch.long)
    targets = targets.to(device, dtype=torch.float)


    #6: Printing messages
    xm.master_print


    #7: Loading data
    train_dataset = … # user-defined, can be outside of mp_fn
    # in mp_fn:
    train_sampler = torch.utils.data.distributed.DistributedSampler(train_dataset,
                num_replicas=xm.xrt_world_size(),rank=xm.get_ordinal(), …)
    train_data_loader = torch.utils.data.DataLoader(train_dataset,
                                sampler=train_sampler, …)


    #8: Training on data
    for epoch in range(EPOCHS):
        para_loader = pl.ParallelLoader(train_data_loader, [device])
        train_fn(para_loader.per_device_loader(device), …)


    #9: Results from TPU
    xm.mesh_reduce


    #10: Model save / restore (memory-optimized)
    import torch_xla.utils.serialization as xser
    xser.save(model.state_dict(), f"model.bin", master_only=True)
    model.load_state_dict(xser.load(f"model.bin"))

    #11: Model save / restore (PyTorch standard)
    torch_xla.core.xla_model.save
    torch.load(...)

    #12: Out of memory datasets:
    Can be loaded from localhost
    Of loaded from GCS in TFRecord format, a TFRecords PyTorch loader exists



```

--------------------------------

### GET https://www.kaggle.com/.well-known/oauth-authorization-server

Source: https://www.kaggle.com/docs/api

Retrieves OAuth 2.0 server metadata including supported endpoints, grant types, and scopes.

```APIDOC
## GET https://www.kaggle.com/.well-known/oauth-authorization-server

### Description
Retrieves OAuth 2.0 server metadata including supported endpoints, grant types, and scopes.

### Method
GET

### Endpoint
https://www.kaggle.com/.well-known/oauth-authorization-server

### Response
#### Success Response (200)
- OAuth 2.0 server metadata (JSON) - Contains supported endpoints, grant types, and scopes.
```

--------------------------------

### Token Introspection - Active Token Response

Source: https://www.kaggle.com/docs/api

Example JSON response for an active token during introspection. Includes user details, scope, and expiration timestamp.

```json
{
  "active": true,
  "username": "johndoe",
  "user_id": 12345,
  "scope": "datasets.viewer:*",
  "exp": 1705719600
}
```

--------------------------------

### Claude Desktop configuration for Kaggle MCP

Source: https://www.kaggle.com/docs/mcp

Adds the Kaggle MCP server to Claude Desktop's configuration using the `mcpServers` key with the `mcp-remote` command and the server URL.

```json
{
  "mcpServers": {
     "kaggle": {
      "command": "npx",
      "args": [
          "mcp-remote",
          "https://www.kaggle.com/mcp"
      ]
    }
  }
}

```

--------------------------------

### get_model_variation

Source: https://www.kaggle.com/docs/mcp

Get metadata about a model variation. This operation retrieves detailed information about a specific model instance, including its framework, download URL, and training data.

```APIDOC
## get_model_variation

### Description
Get metadata about a model variation.

### Method
GET

### Endpoint
/models/{owner_slug}/{model_slug}/{framework}/{instance_slug}

### Parameters
#### Path Parameters
- **owner_slug** (string) - Required - The slug of the owner of the model.
- **model_slug** (string) - Required - The slug of the model.
- **framework** (ModelFramework) - Required - Possible values: MODEL_FRAMEWORK_UNSPECIFIED, MODEL_FRAMEWORK_TENSOR_FLOW_1, MODEL_FRAMEWORK_TENSOR_FLOW_2, MODEL_FRAMEWORK_TF_LITE, MODEL_FRAMEWORK_TF_JS, MODEL_FRAMEWORK_PY_TORCH, MODEL_FRAMEWORK_JAX, MODEL_FRAMEWORK_FLAX, MODEL_FRAMEWORK_PAX, MODEL_FRAMEWORK_MAX_TEXT, MODEL_FRAMEWORK_GEMMA_CPP, MODEL_FRAMEWORK_GGML, MODEL_FRAMEWORK_GGUF, MODEL_FRAMEWORK_CORAL, MODEL_FRAMEWORK_SCIKIT_LEARN, MODEL_FRAMEWORK_MXNET, MODEL_FRAMEWORK_ONNX, MODEL_FRAMEWORK_KERAS, MODEL_FRAMEWORK_TRANSFORMERS, MODEL_FRAMEWORK_API, MODEL_FRAMEWORK_OTHER, MODEL_FRAMEWORK_TENSOR_RT_LLM, MODEL_FRAMEWORK_TRITON.
- **instance_slug** (string) - Required - The slug of the model instance.

### Response
#### Success Response (200)
- **id** (int32) - 
- **slug** (string) - 
- **framework** (ModelFramework) - 
- **fine_tunable** (bool) - 
- **overview** (string) - 
- **usage** (string) - 
- **download_url** (string) - 
- **version_id** (int32) - 
- **version_number** (int32) - 
- **training_data** (repeated string) - 
- **url** (string) - 
- **license_name** (string) - 
- **model_instance_type** (ModelInstanceType) - 
- **base_model_instance_information** (BaseModelInstanceInformation) - 
- **external_base_model_url** (string) - 
- **total_uncompressed_bytes** (int64) - 

#### Response Example
{
  "id": 0,
  "slug": "string",
  "framework": "MODEL_FRAMEWORK_UNSPECIFIED",
  "fine_tunable": true,
  "overview": "string",
  "usage": "string",
  "download_url": "string",
  "version_id": 0,
  "version_number": 0,
  "training_data": ["string"],
  "url": "string",
  "license_name": "string",
  "model_instance_type": "MODEL_INSTANCE_TYPE_UNSPECIFIED",
  "base_model_instance_information": {
    "id": 0,
    "owner": {
      "id": 0,
      "image_url": "string",
      "is_organization": true,
      "name": "string",
      "profile_url": "string",
      "slug": "string",
      "user_tier": "USER_ACHIEVEMENT_TIER_UNSPECIFIED",
      "user_progression_opt_out": true,
      "allow_model_gating": true
    },
    "model_slug": "string",
    "instance_slug": "string",
    "framework": "MODEL_FRAMEWORK_UNSPECIFIED"
  },
  "external_base_model_url": "string",
  "total_uncompressed_bytes": "0"
}
```

--------------------------------

### start_competition_submission_upload

Source: https://www.kaggle.com/docs/mcp

Get a link to upload a competition submission file. The request includes competition name, file length, last modified time, and file name.

```APIDOC
## start_competition_submission_upload

### Description
Get a link to upload a competition submission file.

### Request: ApiStartSubmissionUploadRequest

#### Parameters
- **competition_name** (string) - Competition name.
- **content_length** (int64) - The length of the file in bytes.
- **last_modified_epoch_seconds** (int64) - The last modified time of the file in epoch seconds.
- **file_name** (string) - Comes from form upload.

### Response: ApiStartSubmissionUploadResponse
```
Tokens:
Raw