### Full Example Code for Quickstart

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/quickstart.md

This is the complete Python script demonstrating various lakeFS-spec file system operations, including setup, upload, read, list, info, and delete.

```python
import lakefs
import os

# Prepare a local file for upload
with open("demo.txt", "w") as f:
    f.write("Hello lakeFS!")

fs = lakefs.mounts.get_default_fs()
fs.put("demo.txt", "demo.txt")

with fs.open("demo.txt", "r") as f:
    print(f.read())

# List files in the repository
print("Listing files:")
for item in fs.ls("."):
    print(item)

# Get object metadata
print("\nObject info:")
print(fs.info("demo.txt"))

# Delete the file and commit
fs.rm("demo.txt")
print("\nDeleted demo.txt")

```

--------------------------------

### Bootstrap Local lakeFS Quickstart Instance

Source: https://github.com/aai-institute/lakefs-spec/blob/main/hack/README.md

Use this command to start a local lakeFS quickstart instance using Docker Compose. Ensure you have Docker and docker compose installed.

```shell
docker compose -f hack/compose.yml up
```

--------------------------------

### Install Development Dependencies

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/CONTRIBUTING.md

Navigate to the repository directory and install all development dependencies using uv.

```shell
cd lakefs-spec
uv sync --all-groups
```

--------------------------------

### Serve Documentation Locally

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/CONTRIBUTING.md

Start a local preview server for the project documentation using MkDocs.

```shell
uv run mkdocs serve
```

--------------------------------

### Install lakeFS-spec

Source: https://github.com/aai-institute/lakefs-spec/blob/main/README.md

Install the lakeFS-spec library using pip or uv.

```shell
$
pip install lakefs-spec
  # or, for example with uv:
$ uv add lakefs-spec
```

--------------------------------

### Launch local lakeFS instance with Docker Compose

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/quickstart.md

Quickly start a local lakeFS instance using Docker Compose by piping the configuration file to docker-compose. Ensure Docker is installed and running.

```shell
$ curl https://raw.githubusercontent.com/aai-institute/lakefs-spec/main/hack/compose.yml | docker-compose -f - up
```

--------------------------------

### Install lakeFS-spec using uv

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/quickstart.md

Install the lakeFS-spec package from PyPI using uv. uv can automatically create virtual environments.

```shell
uv add lakefs-spec
```

--------------------------------

### Local Instance .lakectl.yaml Configuration

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/guides/configuration.md

Example configuration for a local lakeFS instance using `~/.lakectl.yaml`.

```yaml
credentials:
  access_key_id: AKIAIOSFOLQUICKSTART
  secret_access_key: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
server:
  endpoint_url: http://127.0.0.1:8000
```

--------------------------------

### Install latest lakeFS-spec from GitHub using uv

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/quickstart.md

Install the latest pre-release version of lakeFS-spec directly from its GitHub repository using uv.

```shell
uv add git+https://github.com/aai-institute/lakefs-spec.git
```

--------------------------------

### Configure lakeFS-spec credentials

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/quickstart.md

Create a `~/.lakectl.yaml` file with credentials for the local lakeFS quickstart environment. These credentials allow lakeFS-spec to discover and access the lakeFS instance.

```yaml
credentials: # (1)!
  access_key_id: AKIAIOSFOLQUICKSTART
  secret_access_key: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
server:
  endpoint_url: http://127.0.0.1:8000
```

--------------------------------

### Install latest lakeFS-spec from GitHub using pip

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/quickstart.md

Install the latest pre-release version of lakeFS-spec directly from its GitHub repository using pip.

```shell
pip install git+https://github.com/aai-institute/lakefs-spec.git
```

--------------------------------

### Install Git Hooks with Pre-commit

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/CONTRIBUTING.md

Set up Git hooks to automatically perform code style checks before commits.

```shell
uv run pre-commit install
```

--------------------------------

### Clone the Repository

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/CONTRIBUTING.md

Clone the lakeFS-spec repository to start development.

```shell
git clone https://github.com/aai-institute/lakefs-spec.git
```

--------------------------------

### Install lakeFS-spec using pip

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/quickstart.md

Install the lakeFS-spec package from PyPI using pip. It is recommended to do this within a virtual environment.

```shell
pip install lakefs-spec
```

--------------------------------

### List Files and Get Object Metadata

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/quickstart.md

Demonstrates listing files in the repository root and retrieving metadata for the 'demo.txt' object using `ls` and `info` operations.

```python
# List files in the repository
print("Listing files:")
for item in fs.ls("."):
    print(item)

# Get object metadata
print("\nObject info:")
print(fs.info("demo.txt"))

```

--------------------------------

### Bootstrap lakeFS with Local SeaweedFS S3 Blockstore

Source: https://github.com/aai-institute/lakefs-spec/blob/main/hack/README.md

Start a lakeFS deployment with a local S3 blockstore using SeaweedFS. This requires Docker and docker compose.

```shell
docker compose -f hack/lakefs-s3-local.yml up
```

--------------------------------

### Configure Transaction Merge Options

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/guides/transactions.md

Example of configuring a transaction to use squash merge with a custom commit message.

```python
from lakefs_spec import LakeFSFileSystem
from lakefs_spec.types import MergeKwargs


merge_kwargs: MergeKwargs = {"squash_merge": True, "message": "My merge commit message"}
fs = LakeFSFileSystem()

with fs.transaction("my-repo", "main", merge_kwargs=merge_kwargs) as tx:
    ...
```

--------------------------------

### DuckDB integration with lakeFS

Source: https://github.com/aai-institute/lakefs-spec/blob/main/README.md

Register the lakeFS filesystem with DuckDB and read Parquet data. Assumes the 'quickstart' repository with sample data is available.

```python
# DuckDB -- see https://duckdb.org/docs/guides/python/filesystems.html
import duckdb
import fsspec

duckdb.register_filesystem(fsspec.filesystem("lakefs"))
res = duckdb.read_parquet("lakefs://quickstart/main/lakes.parquet")
res.show()
```

--------------------------------

### Read and Write Pandas DataFrames with lakeFS

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/guides/integrations.md

Illustrates reading and writing Pandas DataFrames in various formats from/to a lakeFS repository within a transaction. Assumes an existing lakeFS server with a 'quickstart' repository.

```python
--8<-- "docs/_code/pandas_example.py"
```

--------------------------------

### Local lakeFS Docker Compose configuration

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/quickstart.md

A local `compose.yml` file for launching a lakeFS instance. This setup is not recommended for production as it does not store data persistently.

```yaml
--8<-- "https://raw.githubusercontent.com/aai-institute/lakefs-spec/main/hack/compose.yml:3:"
```

--------------------------------

### Get Directory Info

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/guides/filesystem-usage.md

Retrieve metadata for a directory in a lakeFS repository. The returned object contains the directory name and the cumulative size of its contents.

```python
dir_info = fs.info("my-repo/my-ref/dir/")
```

--------------------------------

### Directly upload a file to lakeFS blockstore

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/guides/filesystem-usage.md

For performance with large files, use `fs.put_file()` with `use_blockstore=True` to write directly to the underlying object storage. Ensure the appropriate fsspec package (e.g., `s3fs`, `gcsfs`, `adlfs`) is installed.

```python
fs = LakeFSFileSystem()

fs.put_file("my-repo/my-ref/file.txt", "file.txt", use_blockstore=True)
```

--------------------------------

### Get Object Info

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/guides/filesystem-usage.md

Retrieve metadata for a specific file in a lakeFS repository. The returned dictionary includes details like storage location, creation timestamp, and size.

```python
from lakefs_spec import LakeFSFileSystem

fs = LakeFSFileSystem()

my_file_info = fs.info("my-repo/my-ref/my-file.txt")
```

--------------------------------

### Pandas integration with lakeFS

Source: https://github.com/aai-institute/lakefs-spec/blob/main/README.md

Read Parquet data directly from a lakeFS repository using Pandas. Assumes the 'quickstart' repository with sample data is available.

```python
# Pandas -- see https://pandas.pydata.org/docs/user_guide/io.html#reading-writing-remote-files
import pandas as pd

data = pd.read_parquet("lakefs://quickstart/main/lakes.parquet")
print(data.head())
```

--------------------------------

### Configure API Request Timeout and Content Type

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/guides/filesystem-usage.md

Customize lakeFS API requests by passing a `RequestConfig` object during `LakeFSFileSystem` initialization. This example sets a request timeout and specifies the content type.

```python
from lakefs_spec import LakeFSFileSystem
from lakefs_spec.types import RequestConfig


request_config: RequestConfig = {"request_timeout": 2, "content_type": "application/json"}
fs = LakeFSFileSystem(request_config=request_config)
```

--------------------------------

### Polars integration with lakeFS

Source: https://github.com/aai-institute/lakefs-spec/blob/main/README.md

Read Parquet data directly from a lakeFS repository using Polars. Assumes the 'quickstart' repository with sample data is available.

```python
# Polars -- see https://pola-rs.github.io/polars/user-guide/io/cloud-storage/
import polars as pl

data = pl.read_parquet("lakefs://quickstart/main/lakes.parquet", use_pyarrow=True)
print(data.head())
```

--------------------------------

### Pin GitHub Action Commit SHA

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/CONTRIBUTING.md

Use pinned commit SHAs for GitHub Actions references for security and stability. This example shows the correct format.

```yaml
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
```

--------------------------------

### Clean lakeFS SeaweedFS Blockstore Volume

Source: https://github.com/aai-institute/lakefs-spec/blob/main/hack/README.md

Remove the container and attached volume for the lakeFS SeaweedFS setup. This is useful for cleaning up storage namespaces after repository deletions.

```shell
docker compose -f hack/lakefs-s3-local.yml rm -v
```

--------------------------------

### Build Documentation Locally

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/CONTRIBUTING.md

Build the project documentation into the output folder using MkDocs.

```shell
uv run mkdocs build
```

--------------------------------

### Instantiate LakeFSFileSystem with Zero Configuration

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/guides/configuration.md

Instantiate the LakeFSFileSystem without arguments when the `.lakectl.yaml` is correctly configured in the home directory.

```python
from lakefs_spec import LakeFSFileSystem

# zero config necessary.
fs = LakeFSFileSystem()
```

--------------------------------

### Mixing Config File and Environment Variable Initialization

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/guides/configuration.md

Demonstrates how mixing config file and environment variable initializations can lead to unexpected results due to instance caching.

```python
import os
from lakefs_spec import LakeFSFileSystem

# first file system, initialized from the config file
config_fs = LakeFSFileSystem()

os.environ["LAKECTL_SERVER_ENDPOINT_URL"] = "http://my-other-lakefs.host"
os.environ["LAKECTL_CREDENTIALS_ACCESS_KEY_ID"] = "my-access-key-id"
os.environ["LAKECTL_CREDENTIALS_SECRET_ACCESS_KEY"] = "my-secret-access-key"

envvar_fs = LakeFSFileSystem()

print(config_fs is envvar_fs) # <- prints True!
```

--------------------------------

### Prepare Local File for Upload

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/quickstart.md

This snippet prepares a local file named 'demo.txt' which will be uploaded to the lakeFS repository.

```python
import lakefs
import os

# Prepare a local file for upload
with open("demo.txt", "w") as f:
    f.write("Hello lakeFS!")

```

--------------------------------

### Configure AWS Credentials File

Source: https://github.com/aai-institute/lakefs-spec/blob/main/hack/README.md

Set up your AWS credentials file to point to the local S3 blockstore. This method overwrites the existing credentials file, so back it up if necessary.

```shell
cat > $HOME/.aws/credentials <<EOL
[default]
endpoint_url = http://localhost:9001
aws_access_key_id = sandbox
aws_secret_access_key = sandbox
EOL
```

--------------------------------

### Palette and Alternate Theme Handling

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/_theme_overrides/partials/header.html

Includes palette and alternate theme partials if configured.

```html
{% if config.theme.palette %} {% if not config.theme.palette is mapping %} {% include "partials/palette.html" %} {% endif %} {% endif %} {% if not config.theme.palette is mapping %} {% include "partials/javascripts/palette.html" %} {% endif %} {% if config.extra.alternate %} {% include "partials/alternate.html" %} {% endif %}
```

--------------------------------

### Low-level fsspec filesystem usage

Source: https://github.com/aai-institute/lakefs-spec/blob/main/README.md

Upload a file, create a commit, and read back committed data using the bare lakeFS filesystem implementation. Assumes a repository named 'repo' and lakectl credentials are set up.

```python
from pathlib import Path

from lakefs_spec import LakeFSFileSystem

REPO, BRANCH = "repo", "main"

# Prepare example local data
local_path = Path("demo.txt")
local_path.write_text("Hello, lakeFS!")

# Upload to lakeFS and create a commit
fs = LakeFSFileSystem()  # will auto-discover config from ~/.lakectl.yaml

# Upload a file on a temporary transaction branch
with fs.transaction(repository=REPO, base_branch=BRANCH) as tx:
    fs.put(local_path, f"{REPO}/{tx.branch.id}/{local_path.name}")
    tx.commit(message="Add demo data")

# Read back committed file
f = fs.open(f"{REPO}/{BRANCH}/demo.txt", "rt")
print(f.readline())  # "Hello, lakeFS!"
```

--------------------------------

### Download an entire directory from lakeFS

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/guides/filesystem-usage.md

Use `fs.get()` with `recursive=True` to download a directory and its contents from lakeFS to a local directory.

```python
# structure:
#   dir/
#   ├── a.txt
#   ├── b.yaml
#   ├── c.csv
#   └── ...

# downloads the entire `dir` directory (and subdirectories) into the current directory.
fs.get("my-repo/my-ref/dir", "dir", recursive=True)
```

--------------------------------

### Run Pre-commit Checks

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/CONTRIBUTING.md

Verify code style adherence by running pre-commit checks on all files.

```shell
uv run pre-commit run --all-files
```

--------------------------------

### Configure LakeFS via Environment Variables

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/guides/configuration.md

Configure authentication with the lakeFS server using environment variables, which override values from a `.lakectl.yaml` file.

```python
import os
from lakefs_spec import LakeFSFileSystem

os.environ["LAKECTL_SERVER_ENDPOINT_URL"] = "http://my-lakefs.host"
os.environ["LAKECTL_CREDENTIALS_ACCESS_KEY_ID"] = "my-access-key-id"
os.environ["LAKECTL_CREDENTIALS_SECRET_ACCESS_KEY"] = "my-secret-access-key"

# also zero-config.
fs = LakeFSFileSystem()
```

--------------------------------

### Upload Local File to lakeFS Repository

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/quickstart.md

Uploads the locally created 'demo.txt' file to the 'main' branch of the 'repo' lakeFS repository.

```python
fs = lakefs.mounts.get_default_fs()
fs.put("demo.txt", "demo.txt")

```

--------------------------------

### Minimum .lakectl.yaml Configuration

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/guides/configuration.md

This is the minimum configuration required in `~/.lakectl.yaml` to address a lakeFS server.

```yaml
credentials:
  access_key_id: <ID>
  secret_access_key: <KEY>
server:
  endpoint_url: <LAKEFS-HOST>
```

--------------------------------

### Read Parquet and Write Partitioned CSV with PyArrow

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/guides/integrations.md

Illustrates reading a Parquet file and writing it back to a lakeFS repository as a partitioned CSV dataset using PyArrow. The filesystem parameter accepts any fsspec file system, such as LakeFSFileSystem.

```python
import pyarrow.parquet as pq
import pyarrow.csv as csv
from lakefs_spec import LakeFSFileSystem

fs = LakeFSFileSystem()

# Read Parquet from lakeFS
parquet_table = pq.read_table("lakefs://my-repo/my-bucket/data.parquet", filesystem=fs)

# Write partitioned CSV to lakeFS
# PyArrow's write_table supports writing to a filesystem object
csv.write_table(parquet_table, "lakefs://my-repo/my-bucket/partitioned_output/", filesystem=fs)

```

--------------------------------

### Configure AWS Endpoint and Credentials (Python)

Source: https://github.com/aai-institute/lakefs-spec/blob/main/hack/README.md

Set environment variables within a Python script to configure the connection to the local S3 blockstore. This allows programmatic interaction with the SeaweedFS endpoint.

```python
import os

os.environ["AWS_ENDPOINT_URL"] = "http://localhost:9001"
os.environ["AWS_ACCESS_KEY_ID"] = "sandbox"
os.environ["AWS_SECRET_ACCESS_KEY"] = "sandbox"
```

--------------------------------

### Read Data from lakeFS Repository

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/quickstart.md

Reads the content of the 'demo.txt' file from the 'main' branch of the 'repo' lakeFS repository and prints it.

```python
with fs.open("demo.txt", "r") as f:
    print(f.read())

```

--------------------------------

### Add Documentation Dependency

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/CONTRIBUTING.md

Add a dependency required for documentation generation to the 'docs' group in pyproject.toml using uv.

```shell
uv add --group docs <dep>
```

--------------------------------

### Configure AWS Endpoint and Credentials (Shell)

Source: https://github.com/aai-institute/lakefs-spec/blob/main/hack/README.md

Set environment variables to configure your shell to interact with the local S3 blockstore. These are used by lakeFS to connect to the SeaweedFS endpoint.

```shell
export AWS_ENDPOINT_URL="http://localhost:9001"
export AWS_ACCESS_KEY_ID=sandbox
export AWS_SECRET_ACCESS_KEY=sandbox
```

--------------------------------

### Search Icon and Integration

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/_theme_overrides/partials/header.html

Includes the search icon and search partial if the search plugin is configured.

```html
{% if "material/search" in config.plugins %} {% set icon = config.theme.icon.search or "material/magnify" %} {% include ".icons/" ~ icon ~ ".svg" %} {% include "partials/search.html" %} {% endif %}
```

--------------------------------

### Load and Save Datasets with Hugging Face Datasets and lakeFS

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/guides/integrations.md

Shows how to load datasets from a lakeFS repository using the `load_dataset` function and save them using `save_to_disk()`. This integration relies on fsspec for remote file access.

```python
--8<-- "docs/_code/hf_datasets_example.py"
```

--------------------------------

### Upload an entire directory to lakeFS

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/guides/filesystem-usage.md

Use `fs.put()` with `recursive=True` to upload a local directory and its contents to a specified remote path in lakeFS.

```python
# structure:
#   dir/
#   ├── a.txt
#   ├── b.yaml
#   ├── c.csv
#   └── ...

fs.put("dir", "my-repo/my-ref/dir", recursive=True)
```

--------------------------------

### Update GitHub Action Version

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/CONTRIBUTING.md

Use the 'pinact run -u' command to update GitHub Actions to newer versions.

```shell
pinact run -u
```

--------------------------------

### File Uploads with Commit and Tag Creation

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/guides/transactions.md

Upload files, create commits with messages, and apply a tag to a specific commit within a lakeFS transaction. Ensure you have initialized LakeFSFileSystem and are within a transaction context.

```python
from lakefs_spec import LakeFSFileSystem

fs = LakeFSFileSystem()

with fs.transaction("repo", "main") as tx:
    fs.put_file("train-data.txt", f"repo/{tx.branch.id}/train-data.txt")
    tx.commit(message="Add training data")
    fs.put_file("test-data.txt", f"repo/{tx.branch.id}/test-data.txt")
    sha = tx.commit(message="Add test data")
    tx.tag(sha, name="My train-test split")
```

--------------------------------

### Add Core Dependency

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/CONTRIBUTING.md

Add a core dependency needed for the package to pyproject.toml using uv.

```shell
uv add <dep>
```

--------------------------------

### Copy File and Directory

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/guides/filesystem-usage.md

Copy files or directories within the same branch, or between different branches in the same repository. Use `cp_file()` for single files and `copy()` for directories with `recursive=True`.

```python
from lakefs_spec import LakeFSFileSystem

fs = LakeFSFileSystem()

# copies a single file on the same branch to a new location.
fs.cp_file("my-repo/branch-a/file.txt", "my-repo/branch-a/file.txt.bak")

# copies a single file from branch A to branch B.
fs.cp_file("my-repo/branch-a/file.txt", "my-repo/branch-b/file.txt")

# copies the entire `my-dir` directory from branch A to branch B (which must exist).
fs.copy("my-repo/branch-a/my-dir/", "my-repo/branch-b/my-dir/", recursive=True)
```

--------------------------------

### Query and Store Data with DuckDB and lakeFS

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/guides/integrations.md

Demonstrates reading and writing data from/to a lakeFS repository using the DuckDB Python API within a transaction. This leverages fsspec for transparent data access.

```python
--8<-- "docs/_code/duckdb_example.py"
```

--------------------------------

### Run Tests

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/CONTRIBUTING.md

Execute the project tests against an ephemeral lakeFS instance using pytest.

```shell
uv run pytest
```

--------------------------------

### Download a single file from lakeFS

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/guides/filesystem-usage.md

Use `fs.get_file()` to download a single file from a specified remote lakeFS path to a local target path.

```python
from lakefs_spec import LakeFSFileSystem

fs = LakeFSFileSystem()

# remote path, then local target path.
fs.get_file("my-repo/my-ref/file.txt", "file.txt")
```

--------------------------------

### List Directory Contents

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/guides/filesystem-usage.md

List all objects within a specified directory in a lakeFS repository. Returns a list of dictionaries, each containing object metadata similar to fs.info().

```python
from lakefs_spec import LakeFSFileSystem

fs = LakeFSFileSystem()

my_dir_listing = fs.ls("my-repo/my-ref/my-dir/")
```

--------------------------------

### Add Development Dependency

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/CONTRIBUTING.md

Add a development dependency to the 'dev' group in pyproject.toml using uv.

```shell
uv add --group dev <dep>
```

--------------------------------

### Source Code Link Inclusion

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/_theme_overrides/partials/header.html

Includes the source code partial if a repository URL is configured.

```html
{% if config.repo_url %}

{% include "partials/source.html" %}

{% endif %}
```

--------------------------------

### Set LAKECTL_CONFIG_FILE Environment Variable

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/guides/configuration.md

Set the `LAKECTL_CONFIG_FILE` environment variable to specify a custom path for the configuration file before instantiating the file system.

```python
import os
from lakefs_spec import LakeFSFileSystem

os.environ["LAKECTL_CONFIG_FILE"] = "/path/to/my/configfile.yaml"

fs = LakeFSFileSystem()
```

--------------------------------

### Define MergeKwargs for Transaction Merges

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/guides/transactions.md

Defines the structure for merge options, including message, metadata, strategy, force, allow_empty, and squash_merge.

```python
class MergeKwargs(TypedDict, total=False):
    """Options to control the merge of a transaction branch into the base branch.

    This is essentially the `lakefs_sdk.Merge` model, without the optionals.
    """

    message: str
    metadata: dict[str, str]
    strategy: Literal["dest-wins", "source-wins"]
    force: bool
    allow_empty: bool
    squash_merge: bool
```

--------------------------------

### Read Parquet and Write CSV with Polars

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/guides/integrations.md

Demonstrates reading a Parquet file and saving a modified version back in CSV format to a lakeFS repository using Polars. Note that Polars does not directly support writing to remote storage via its DataFrame.write_* API.

```python
import polars as pl
from lakefs_spec import LakeFSFileSystem

fs = LakeFSFileSystem()

# Read Parquet from lakeFS
df = pl.read_parquet("lakefs://my-repo/my-bucket/data.parquet")

# Perform some transformations
df_transformed = df.with_columns(
    (pl.col("col1") * 2).alias("col1_doubled")
)

# Write transformed DataFrame to CSV in lakeFS
# Note: Polars does not directly support writing to remote storage via DataFrame.write_* API.
# This example assumes a mechanism to write to a file-like object that fsspec can handle.
df_transformed.write_csv(fs.open("lakefs://my-repo/my-bucket/output.csv", "wb"))

```

--------------------------------

### Run Zizmor for GitHub Actions Security Audit

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/CONTRIBUTING.md

Locally check GitHub Actions workflows for security issues using zizmor. This is also part of the pre-commit hooks.

```shell
uv run zizmor .github
```

--------------------------------

### Clear LakeFS FileSystem Instance Cache

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/guides/configuration.md

Clear the file system instance cache to ensure new configurations are picked up correctly when mixing initialization methods.

```python
from lakefs_spec import LakeFSFileSystem

LakeFSFileSystem.clear_instance_cache()
```

--------------------------------

### Upload a single file to lakeFS

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/guides/filesystem-usage.md

Use `fs.put_file()` to upload a single file. Specify the remote lakeFS path and the local source file path.

```python
from lakefs_spec import LakeFSFileSystem

fs = LakeFSFileSystem()

# remote path, then local target path.
fs.put_file("file.txt", "my-repo/my-ref/file.txt")
```

--------------------------------

### Lock Dependencies

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/CONTRIBUTING.md

After adding or modifying dependencies, lock all dependencies to ensure reproducible builds.

```shell
uv lock
```

--------------------------------

### Header Logo and Title Inclusion

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/_theme_overrides/partials/header.html

Includes the site logo and displays the site name, with conditional title display based on page meta.

```html
[{% include "partials/logo.html" %}]({{ config.extra.homepage | d(nav.homepage.url, true) | url }} "{{ config.site_name | e }}") {% set icon = config.theme.icon.menu or "material/menu" %} {% include ".icons/" ~ icon ~ ".svg" %}

{{ config.site_name }}

{% if page.meta and page.meta.title %} {{ config.site_name }} — **{{ page.meta.title }}** {% else %} {{ config.site_name }} — **{{ page.title }}** {% endif %}
```

--------------------------------

### Delete File and Directory

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/guides/filesystem-usage.md

Remove a single file using `rm_file()` or an entire directory recursively using `rm()` with `recursive=True` from a lakeFS branch.

```python
from lakefs_spec import LakeFSFileSystem

fs = LakeFSFileSystem()

fs.rm_file("my-repo/my-branch/my-file.txt")

# removes the entire `my-dir` directory.
fs.rm("my-repo/my-branch/my-dir/", recursive=True)
```

--------------------------------

### Check for lakeFS object existence

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/guides/filesystem-usage.md

Use `fs.exists()` to check if a file exists at a given path within a specific repository revision. Returns `False` for directories and raises errors for permission issues.

```python
from lakefs_spec import LakeFSFileSystem

fs = LakeFSFileSystem()

my_file_exists = fs.exists("my-repo/my-ref/my-file.txt")
```

--------------------------------

### Delete File from lakeFS Repository

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/quickstart.md

Removes the 'demo.txt' file from the 'main' branch of the 'repo' lakeFS repository and creates a new commit.

```python
# Delete the file and commit
fs.rm("demo.txt")
print("\nDeleted demo.txt")

```

--------------------------------

### Navigation Tabs Inclusion

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/_theme_overrides/partials/header.html

Includes the navigation tabs partial if sticky tabs and navigation features are enabled.

```html
{% if "navigation.tabs.sticky" in features %} {% if "navigation.tabs" in features %} {% include "partials/tabs.html" %} {% endif %} {% endif %}
```

--------------------------------

### Error Handling in lakeFS Transactions

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/guides/transactions.md

Demonstrates how exceptions within a transaction block prevent modifications to the target branch. The transaction branch is retained on failure for debugging when delete='onsuccess' (the default).

```python
from lakefs_spec import LakeFSFileSystem

fs = LakeFSFileSystem()

with fs.transaction("repo", "main", delete="onsuccess") as tx:
    fs.put_file("my-file.txt", f"repo/{tx.branch.id}/my-file.txt")
    tx.commit(message="Add my-file.txt")
    raise ValueError("oops!")
```

--------------------------------

### Conditional Header Class Assignment

Source: https://github.com/aai-institute/lakefs-spec/blob/main/docs/_theme_overrides/partials/header.html

Assigns CSS classes to the header based on enabled features like sticky tabs or navigation.

```html
{% set class = "md-header" %} {% if "navigation.tabs.sticky" in features %} {% set class = class ~ " md-header--shadow md-header--lifted" %} {% elif "navigation.tabs" not in features %} {% set class = class ~ " md-header--shadow" %} {% endif %}
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.