### Setup development environment with uv

Source: https://github.com/datalad/datalad/blob/maint/CONTRIBUTING.md

Commands to install the uv package manager, create a virtual environment, and install DataLad in editable mode with development dependencies.

```sh
# Install uv (fast Python package manager) if you don't have it
pip install uv

# Create a virtualenv and install for development
uv venv
source .venv/bin/activate
uv pip install -e '.[devel]'
```

--------------------------------

### Install dataset with custom destination and get data

Source: https://github.com/datalad/datalad/blob/maint/docs/source/generated/man/datalad-install.md

Installs a dataset into a custom directory and retrieves all content immediately using --get-data.

```default
% datalad install --get-data \
  -s https://github.com/datalad-datasets/longnow-podcasts.git podcasts
```

--------------------------------

### Text: Example job queue timing output

Source: https://github.com/datalad/datalad/blob/maint/docs/testing-speedup-proposal.md

Sample output from a CI run showing job start times, durations, and queueing. Demonstrates that 20 jobs start immediately while others wait, illustrating the concurrency cap effect.

```text
 started   done  queued  job
     0.1   43.6     0.0  test (3.10)
     0.1   48.5     0.0  test (3.10, true, not)
     ...   (18 more at t+0.1)
    30.9   84.8    30.9  test (3.10, not, ru_RU.UTF-8)
    31.0   71.3    30.9  test (3.10, /var/tmp/sym link, -, INFO, ...)
```

--------------------------------

### datalad install synopsis

Source: https://github.com/datalad/datalad/blob/maint/docs/source/generated/man/datalad-install.md

Shows the full command-line syntax for datalad install, including all options and arguments.

```default
datalad install [-h] [-s URL-OR-PATH] [-d DATASET] [-g] [-D DESCRIPTION] [-r] [-R
    LEVELS] [--reckless [auto|ephemeral|shared-...]] [-J NJOBS] [-c
    PROC] [--branch BRANCH] [--version] [URL-OR-PATH ...]
```

--------------------------------

### Install dataset from GitHub

Source: https://github.com/datalad/datalad/blob/maint/docs/source/generated/man/datalad-install.md

Installs a dataset from a remote URL into the current directory.

```default
% datalad install https://github.com/datalad-datasets/longnow-podcasts.git
```

--------------------------------

### Install default superdataset with shortcut

Source: https://github.com/datalad/datalad/blob/maint/docs/source/basics.rst

Installs the default superdataset from https://datasets.datalad.org/ into a datasets.datalad.org/ subdirectory. The /// shortcut points to this superdataset.

```bash
datalad install ///
```

--------------------------------

### Install dataset recursively

Source: https://github.com/datalad/datalad/blob/maint/docs/source/generated/man/datalad-install.md

Installs a dataset along with all its subdatasets using the -r option.

```default
% datalad install -r \
  https://github.com/datalad-datasets/longnow-podcasts.git
```

--------------------------------

### Install DataLad on Debian-based systems with apt-get

Source: https://github.com/datalad/datalad/blob/maint/README.md

Install DataLad on Debian-based systems after enabling NeuroDebian. Run this command as root or with sudo.

```shell
apt-get install datalad
```

--------------------------------

### Install DataLad on Gentoo-based systems with emerge

Source: https://github.com/datalad/datalad/blob/maint/README.md

Install DataLad on Gentoo-based systems after enabling the ::science overlay. Run this command as root or with sudo.

```shell
emerge datalad
```

--------------------------------

### Install dataset via SSH login style resource identifier

Source: https://github.com/datalad/datalad/blob/maint/docs/source/basics.rst

Installs a dataset using an SSH login style resource identifier, such as me@localhost:/path.

```bash
datalad install me@localhost:/path
```

--------------------------------

### Install dataset via HTTP URL

Source: https://github.com/datalad/datalad/blob/maint/docs/source/basics.rst

Installs a dataset from a regular HTTP or HTTPS URL. DataLad accepts these URLs as arguments in most places.

```bash
datalad install https://github.com/psychoinformatics-de/studyforrest-data-phase2.git
```

--------------------------------

### Set up asv benchmarking environment

Source: https://github.com/datalad/datalad/blob/maint/CONTRIBUTING.md

Initialize the .asv submodule, install asv (or the devel extras), and configure the machine for benchmarking. This is required before running benchmarks.

```sh
git submodule update --init .asv
pip install asv        # or pip install .[devel]
asv machine            # configure for your host
```

--------------------------------

### Global Configuration Example

Source: https://github.com/datalad/datalad/blob/maint/docs/design.rst

Shows the global configuration structure with dataset path, description, and exec fields.

```ini
[dataset]
path =
description =
exec =
```

--------------------------------

### Example: create GIN and GitHub siblings and configure common datasrc

Source: https://github.com/datalad/datalad/blob/maint/docs/source/generated/man/datalad-create-sibling-github.md

Example workflow: create a GIN sibling as a common data source, create a GitHub sibling for collaboration, configure the GIN storage as a common datasrc, and push to GitHub.

```default
% datalad create-sibling-gin myrepo -s gin

# the sibling on GitHub will be used for collaborative work
% datalad create-sibling-github myrepo -s github

# register the storage of the public GIN repo as a data source
% datalad siblings configure -s gin --as-common-datasrc gin-storage

# announce its availability on github
% datalad push --to github
```

--------------------------------

### Install git-annex step with pypi scenario

Source: https://github.com/datalad/datalad/blob/maint/docs/testing-speedup-proposal.md

The 'Install git-annex' step that installs git-annex via pip when the scenario starts with 'pypi', otherwise falls back to datalad-installer. It creates symlinks in /usr/local/bin to make git-annex available to non-interactive SSH sessions and sudo.

```yaml
- name: Install git-annex
  run: |
    if [[ "$_DL_ANNEX_INSTALL_SCENARIO" == pypi* ]]; then
      # "pypi", or "pypi==10.20260901.post1" to pin
      pip install "git-annex${_DL_ANNEX_INSTALL_SCENARIO#pypi}"
      # console scripts must be findable by *non-interactive ssh* sessions
      # (DATALAD_TESTS_SSH=1) and by `sudo -E`, not just by this job's PATH
      scriptdir=$(python -c 'import sysconfig; print(sysconfig.get_path("scripts"))')
      sudo ln -sf "$scriptdir"/git-annex "$scriptdir"/git-annex-shell \
                  "$scriptdir"/git-remote-annex "$scriptdir"/git-remote-tor-annex \
                  /usr/local/bin/
    else
      pip install datalad-installer
      eval datalad-installer --sudo ok -E new.env ${_DL_ANNEX_INSTALL_SCENARIO}
      cat new.env >> ~/.bashrc
    fi
```

--------------------------------

### REFLOW macro example

Source: https://github.com/datalad/datalad/blob/maint/docs/source/design/docstrings.rst

Use the REFLOW macro to enclose segments that need reformatting as the final processing step; the start macro must appear on a dedicated line.

```default
|| REFLOW >>
The API has been aligned with the some
``create_sibling_...||create-sibling-...`` commands of other GitHub-like
services, such as GOGS, GIN, GitTea.<< REFLOW ||
```

--------------------------------

### Setup files and directories with @with_tempfile and @with_tree

Source: https://github.com/datalad/datalad/blob/maint/CONTRIBUTING.md

Use these decorators to manage temporary files and directory structures for test isolation.

```python
from datalad.tests.utils_pytest import with_tempfile, with_tree

@with_tempfile(mkdir=True)
def test_something(path=None):        # path is a fresh temp dir, auto-cleaned
    ds = Dataset(path).create()

@with_tree(tree={'input.txt': 'data', 'subdir': {'nested.csv': 'a,b'}})
def test_with_content(path=None):     # tree materialised under path
    ds = Dataset(path).create(force=True)
    ds.save()
```

--------------------------------

### Basic DataLad GitHub Action usage

Source: https://github.com/datalad/datalad/blob/maint/docs/source/design/github_actions.rst

Basic usage of the DataLad GitHub Action. Installs the dataset from the given source URL and gets all its data by setting install_get_data to true.

```yaml
- uses: datalad/datalad-action@master
  with:
    datasets:
      - source: https://github.com/psychoinformatics-de/studyforrest-data-phase2
      - install_get_data: true
```

--------------------------------

### avatars.csv example content

Source: https://github.com/datalad/datalad/blob/maint/docs/source/generated/man/datalad-addurls.md

Example CSV file 'avatars.csv' used in the examples. Contains columns 'who', 'ext', and 'link'.

```default
who,ext,link
neurodebian,png,https://avatars3.githubusercontent.com/u/260793
datalad,png,https://avatars1.githubusercontent.com/u/8927200
```

--------------------------------

### Install DataLad via pip in a virtual environment

Source: https://github.com/datalad/datalad/blob/maint/README.md

Install DataLad from PyPI using a dedicated virtualenv. First install a recent version of git-annex. The commands create and enter a new virtual environment, then install DataLad.

```shell
# Create and enter a new virtual environment (optional)
virtualenv --python=python3 ~/env/datalad
. ~/env/datalad/bin/activate

# Install from PyPI
pip install datalad
```

--------------------------------

### Show help for the create subcommand

Source: https://github.com/datalad/datalad/blob/maint/docs/source/basics.rst

Displays the first 20 lines of 'datalad create --help', which explains how to create a new dataset and lists its options. The output includes a description of the command's behavior and the default identifier generation.

```ansi-color
~ % datalad create --help       | head -n20
Usage: datalad create [-h] [-f] [-D DESCRIPTION] [-d PATH] [--no-annex]
                      [--nosave] [--annex-version ANNEX_VERSION]
                      [--annex-backend ANNEX_BACKEND]
                      [--native-metadata-type LABEL] [--shared-access MODE]
                      [--git-opts STRING] [--annex-opts STRING]
                      [--annex-init-opts STRING] [--text-no-annex]
                      [PATH]

Create a new dataset from scratch.

This command initializes a new dataset at a given location, or the
current directory. The new dataset can optionally be registered in an
existing superdataset (the new dataset's path needs to be located
within the superdataset for that, and the superdataset needs to be given
explicitly). It is recommended to provide a brief description to label
the dataset's nature *and* location, e.g. "Michael's music on black
laptop". This helps humans to identify data locations in distributed
scenarios.  By default an identifier comprised of user and machine name,
plus path will be generated.
```

--------------------------------

### Web Provider Configuration Example

Source: https://github.com/datalad/datalad/blob/maint/docs/design.rst

Configuration for a web provider, including crawling options, download modes, and recursion rules.

```ini
[provider:incoming_http]
type = web
mode = (download|fast|relaxed)            # fast/relaxed/download
filename = (url|request)                  # of cause also could be _e'valuated given the bs4 link get_video_filename(link, filename)
recurse_(a|href) =                        # regexes to recurse
# mimicking scrapy
start_urls = http://...                   #
certificates =                            # if there are https -- we need to allow specifying those
allowed_domains = example.com/buga/duga   # to limit recursion
                  sample.com
excluded_hrefs =                          # do not even search for "download" URLs on given pages.  Should also allow to be a function/callback to decide based on request?
include_(a|href) =                        # what to download
exclude_(a|href) =                        # and not (even if matches)
???generators = generate_readme              # Define some additional actions to be performed....
```