### Define Basic Setup and Teardown Tasks

Source: https://airflow.apache.org/docs/apache-airflow/stable/howto/setup-and-teardown.html

Mark tasks as setup and teardown using `as_setup` and `as_teardown` methods to manage dependencies. The teardown task can be linked to the setup task directly or via the `setups` argument.

```python
create_cluster.as_setup() >> run_query >> delete_cluster.as_teardown()
create_cluster >> delete_cluster
```

```python
create_cluster >> run_query >> delete_cluster.as_teardown(setups=create_cluster)
```

--------------------------------

### Install OpenLineage Extra

Source: https://airflow.apache.org/docs/apache-airflow/stable/extra-packages-ref.html

Install the OpenLineage extra for sending OpenLineage events.

```bash
pip install 'apache-airflow[openlineage]'
```

--------------------------------

### Install OpenFaaS Extra

Source: https://airflow.apache.org/docs/apache-airflow/stable/extra-packages-ref.html

Install the OpenFaaS extra for OpenFaaS hooks.

```bash
pip install 'apache-airflow[openfaas]'
```

--------------------------------

### Basic Setup and Teardown Task

Source: https://airflow.apache.org/docs/apache-airflow/stable/_modules/airflow/example_dags/example_setup_teardown.html

Defines a root setup task and a root teardown task that depend on each other through a normal task. The `as_setup()` method marks a task as a setup task, and `as_teardown(setups=...)` marks a task as a teardown task, linking it to its corresponding setup task(s).

```python
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#   http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied.  See the License for the
# specific language governing permissions and limitations
# under the License.
"""Example DAG demonstrating the usage of setup and teardown tasks."""

from __future__ import annotations

import pendulum

from airflow.providers.standard.operators.bash import BashOperator
from airflow.sdk import DAG, TaskGroup

with DAG(
    dag_id="example_setup_teardown",
    schedule=None,
    start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
    catchup=False,
    tags=["example"],
) as dag:


    root_setup = BashOperator(task_id="root_setup", bash_command="echo 'Hello from root_setup'").as_setup()


    root_normal = BashOperator(task_id="normal", bash_command="echo 'I am just a normal task'")
    root_teardown = BashOperator(
        task_id="root_teardown", bash_command="echo 'Goodbye from root_teardown'"
    ).as_teardown(setups=root_setup)
    root_setup >> root_normal >> root_teardown

```

--------------------------------

### Install Opensearch Extra

Source: https://airflow.apache.org/docs/apache-airflow/stable/extra-packages-ref.html

Install the Opensearch extra for Opensearch hooks and operators.

```bash
pip install 'apache-airflow[opensearch]'
```

--------------------------------

### Install FAB Extra

Source: https://airflow.apache.org/docs/apache-airflow/stable/extra-packages-ref.html

Install the FAB extra for FAB auth manager.

```bash
pip install 'apache-airflow[fab]'
```

--------------------------------

### Install Informatica Extra

Source: https://airflow.apache.org/docs/apache-airflow/stable/extra-packages-ref.html

Install the Informatica extra for Informatica hooks and operators.

```bash
pip install 'apache-airflow[informatica]'
```

--------------------------------

### Install Exasol Extra

Source: https://airflow.apache.org/docs/apache-airflow/stable/extra-packages-ref.html

Install the Exasol extra for Exasol hooks and operators.

```bash
pip install 'apache-airflow[exasol]'
```

--------------------------------

### Install Git Extra

Source: https://airflow.apache.org/docs/apache-airflow/stable/extra-packages-ref.html

Install the Git extra for Git bundle and hook.

```bash
pip install 'apache-airflow[git]'
```

--------------------------------

### Install gRPC Extra

Source: https://airflow.apache.org/docs/apache-airflow/stable/extra-packages-ref.html

Install the gRPC extra for gRPC hooks and operators.

```bash
pip install 'apache-airflow[grpc]'
```

--------------------------------

### Install Samba Extra

Source: https://airflow.apache.org/docs/apache-airflow/stable/extra-packages-ref.html

Install the Samba extra for Samba hooks and operators.

```bash
pip install 'apache-airflow[samba]'
```

--------------------------------

### Install Teradata Extra

Source: https://airflow.apache.org/docs/apache-airflow/stable/extra-packages-ref.html

Install the Teradata extra for Teradata hooks and operators.

```bash
pip install 'apache-airflow[teradata]'
```

--------------------------------

### Start Airflow DAG Processor Independently

Source: https://airflow.apache.org/docs/apache-airflow/stable/installation/upgrading_to_airflow3.html

The DAG processor must now be started separately, even for local development setups.

```bash
airflow dag-processor

```

--------------------------------

### Configure Setuptools Entrypoint for Policies

Source: https://airflow.apache.org/docs/apache-airflow/stable/administration-and-deployment/cluster-policies.html

Example of configuring a setuptools entrypoint in pyproject.toml to register custom Airflow policy functions.

```toml
[build-system]
requires = ["setuptools", "wheel"]
build-backend = "setuptools.build_meta"

[project]
name = "my-airflow-plugin"
version = "0.0.1"
# ...

dependencies = ["apache-airflow>=2.6"]
[project.entry-points.'airflow.policy']
_ = 'my_airflow_plugin.policies'

```

--------------------------------

### Example SQLite Database URI

Source: https://airflow.apache.org/docs/apache-airflow/stable/howto/set-up-database.html

This is an example of the connection string format for a SQLite database file. Ensure the path is correct for your Airflow setup.

```sql
sqlite:////home/airflow/airflow.db

```

--------------------------------

### Install All Optional Dependencies

Source: https://airflow.apache.org/docs/apache-airflow/stable/extra-packages-ref.html

Installs all optional dependencies, including all providers. Not recommended for production.

```bash
pip install apache-airflow[all]
```

--------------------------------

### Install Airflow Standalone with pipx

Source: https://airflow.apache.org/docs/apache-airflow/stable/installation/index.html

Use this command to install Airflow directly from PyPI for local development and testing when you have pipx installed. It starts a minimal system with an auto-generated admin password and SQLite database.

```bash
pipx run apache-airflow standalone

```

--------------------------------

### Parallel Setup and Teardown Tasks

Source: https://airflow.apache.org/docs/apache-airflow/stable/howto/setup-and-teardown.html

Demonstrates how to define multiple setup tasks and their corresponding teardown tasks to run in parallel. This is useful for managing resources that can be provisioned or cleaned up concurrently.

```python
(
    [create_cluster, create_bucket]
    >> run_query
    >> [delete_cluster.as_teardown(setups=create_cluster), delete_bucket.as_teardown(setups=create_bucket)]
)

```

--------------------------------

### Example of Constructing Constraint File URL

Source: https://airflow.apache.org/docs/apache-airflow/stable/installation/installing-from-pypi.html

An example demonstrating how to form the URL for a specific Airflow and Python version constraint file.

```bash
https://raw.githubusercontent.com/apache/airflow/constraints-3.2.2/constraints-3.10.txt
```

--------------------------------

### Valid Templated Field Setup in Parent Constructor

Source: https://airflow.apache.org/docs/apache-airflow/stable/howto/custom-operator.html

This example demonstrates a valid setup where a subclass does not define its own constructor, and templated fields are correctly set in the parent operator.

```python
class HelloOperator(BaseOperator):
    template_fields = "foo"

    def __init__(self, foo) -> None:
        self.foo = foo


class MyHelloOperator(HelloOperator):
    template_fields = "foo"

```

--------------------------------

### Install Airflow Standalone with uvx

Source: https://airflow.apache.org/docs/apache-airflow/stable/installation/index.html

An alternative command to install Airflow for local development and testing using Astral uvx. This also starts a minimal system with an auto-generated admin password and SQLite database.

```bash
uvx apache-airflow standalone

```

--------------------------------

### Install All Core Optional Dependencies

Source: https://airflow.apache.org/docs/apache-airflow/stable/extra-packages-ref.html

Installs all optional core dependencies. Suitable for development and testing.

```bash
pip install apache-airflow[all-core]
```

--------------------------------

### airflow info

Source: https://airflow.apache.org/docs/apache-airflow/stable/cli-and-env-variables-ref.html

Displays information about the current Airflow installation and environment. Useful for debugging and understanding the Airflow setup.

```APIDOC
## airflow info

### Description
Shows information about current Airflow and environment.

### Method
CLI Command

### Endpoint
N/A

### Parameters
#### Positional Arguments
* `import_path` (string) - The import path of the database manager to use.

#### Named Arguments
* `--skip-init` (boolean) - Only remove tables; do not perform db init. Default: `False`
* `--use-migration-files` (boolean) - Use migration files to create the database instead of the ORM. This is useful for verifying that the migration files produce the same schema as the ORM models. Default: `False`
* `--verbose` (boolean) - Make logging output more verbose. Default: `False`
* `--yes` (boolean) - Do not prompt to confirm. Use with care! Default: `False`
* `--anonymize` (boolean) - Minimize any personal identifiable information. Use it when sharing output with others. Default: `False`
* `--file-io` (boolean) - Send output to file.io service and returns link. Default: `False`
* `--output` (string) - Possible choices: table, json, yaml, plain. Output format. Allowed values: json, yaml, plain, table (default: table). Default: `'table'`
* `--verbose` (boolean) - Make logging output more verbose. Default: `False`

### Request Example
```bash
airflow info --anonymize --output json
```

### Response
#### Success Response (200)
Output will be in the format specified by the `--output` argument (e.g., table, json, yaml, plain).

#### Response Example
```json
{
  "airflow_version": "2.7.1",
  "python_version": "3.9.16"
}
```
```

--------------------------------

### Example of Metrics allow_list Configuration

Source: https://airflow.apache.org/docs/apache-airflow/stable/release_notes.html

This snippet shows a complete example for configuring the metrics allow_list. Ensure this configuration is applied correctly to manage metrics access.

```yaml
metrics:
  allow_list:
    - "airflow_dag_run.running"
    - "airflow_dag_run.failed"
```

--------------------------------

### Get Current Auth Manager

Source: https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/auth-manager/index.html

Use this command to check which auth manager is currently configured in your Airflow installation.

```bash
$ airflow config get-value core auth_manager
airflow.providers.fab.auth_manager.fab_auth_manager.FabAuthManager

```

--------------------------------

### Install All Task SDK Optional Dependencies

Source: https://airflow.apache.org/docs/apache-airflow/stable/extra-packages-ref.html

Installs all optional task SDK dependencies. Useful for CI and testing.

```bash
pip install apache-airflow[all-task-sdk]
```

--------------------------------

### Nginx Proxy Configuration Example

Source: https://airflow.apache.org/docs/apache-airflow/stable/howto/run-behind-proxy.html

This is a basic Nginx configuration snippet for proxying requests. Ensure it matches your specific proxy setup.

```nginx
proxy_redirect off;
proxy_http_version 1.1;

```

--------------------------------

### Download Docker Compose and Initialize Airflow

Source: https://airflow.apache.org/docs/apache-airflow/stable/tutorial/pipeline.html

This snippet downloads the necessary Docker Compose file, sets up directories and environment variables, and initializes the Airflow database. It is used to start a local Airflow environment for the tutorial.

```bash
# Download the docker-compose.yaml file
curl -LfO 'https://airflow.apache.org/docs/apache-airflow/stable/docker-compose.yaml'

# Make expected directories and set an expected environment variable
mkdir -p ./dags ./logs ./plugins
echo -e "AIRFLOW_UID=$(id -u)" > .env

# Initialize the database
docker compose up airflow-init

# Start up all services
docker compose up

```

--------------------------------

### Install Airflow with Standard Extras

Source: https://airflow.apache.org/docs/apache-airflow/stable/extra-packages-ref.html

Install Airflow with standard hooks and operators included.

```bash
pip install apache-airflow[standard]'
```

--------------------------------

### Get Information on Airflow Queues

Source: https://airflow.apache.org/docs/apache-airflow/stable/cli-and-env-variables-ref.html

Retrieves information about queues provided by installed providers. Supports different output formats and verbose logging.

```bash
airflow providers queues [-h] [-o (table, json, yaml, plain)] [-v]
```

--------------------------------

### Example DAG Definition

Source: https://airflow.apache.org/docs/apache-airflow/stable/_modules/airflow/example_dags/example_skip_dag.html

Defines the 'example_skip_dag' with a daily schedule, start date, and tags. It instantiates the test pipeline twice with different trigger rules.

```python

with DAG(
    dag_id="example_skip_dag",
    schedule=datetime.timedelta(days=1),
    start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
    catchup=False,
    tags=["example"],
) as dag:
    create_test_pipeline("1", TriggerRule.ALL_SUCCESS)
    create_test_pipeline("2", TriggerRule.ONE_SUCCESS)

```

--------------------------------

### Configure Setuptools Entrypoint in pyproject.toml

Source: https://airflow.apache.org/docs/apache-airflow/stable/administration-and-deployment/plugins.html

Configuration in `pyproject.toml` to register an Airflow plugin via setuptools entrypoints, enabling Airflow to discover and load it.

```toml
[project.entry-points."airflow.plugins"]
my_plugin = "my_package.my_plugin:MyAirflowPlugin"
```

--------------------------------

### Get Airflow Database Connection URI

Source: https://airflow.apache.org/docs/apache-airflow/stable/howto/set-up-database.html

Use this command to retrieve the currently configured database connection URI for Airflow. This is useful for verifying your setup.

```bash
$ airflow config get-value database sql_alchemy_conn
sqlite:////tmp/airflow/airflow.db

```

--------------------------------

### Get Information on Airflow Secrets Backends

Source: https://airflow.apache.org/docs/apache-airflow/stable/cli-and-env-variables-ref.html

Retrieves information about secrets backends provided by installed providers. Options include output format and verbosity.

```bash
airflow providers secrets [-h] [-o (table, json, yaml, plain)] [-v]
```

--------------------------------

### Install Airflow with Extras and Providers

Source: https://airflow.apache.org/docs/apache-airflow/stable/installation/installing-from-pypi.html

Use this command to install a specific version of Apache Airflow along with desired extras (e.g., async) and providers (e.g., postgres, google). The constraint URL ensures compatibility with the Airflow version.

```bash
AIRFLOW_VERSION=3.2.2
PYTHON_VERSION="$(python -c 'import sys; print(f"{sys.version_info.major}.{sys.version_info.minor}")')"
CONSTRAINT_URL="https://raw.githubusercontent.com/apache/airflow/constraints-${AIRFLOW_VERSION}/constraints-${PYTHON_VERSION}.txt"
pip install "apache-airflow[async,postgres,google]==${AIRFLOW_VERSION}" --constraint "${CONSTRAINT_URL}"
```

--------------------------------

### Example .env File Format

Source: https://airflow.apache.org/docs/apache-airflow/stable/_modules/airflow/secrets/local_filesystem.html

Demonstrates the expected format for a connection string in a .env file. Supports URL-encoded characters for special symbols in passwords.

```text
MY_CONN_ID=my-conn-type://my-login:my-pa%2Fssword@my-host:5432/my-schema?param1=val1&param2=val2
```

--------------------------------

### Use BashOperator with Jinja Templating

Source: https://airflow.apache.org/docs/apache-airflow/stable/concepts/operators.html

This example shows how to use the BashOperator to pass the start of a data interval as an environment variable to a bash script using Jinja templating.

```python
bash_task = BashOperator(
    task_id="run_bash_script",
    bash_command="echo {{ ds }} {{ macros.ds_add(ds, -1) }}",
    env={
        "START_DATE": "{{ ds }}",
        "PREVIOUS_DATE": "{{ macros.ds_add(ds, -1) }}",
    },
)
```

--------------------------------

### Get Information on Airflow Connection Form Widgets

Source: https://airflow.apache.org/docs/apache-airflow/stable/cli-and-env-variables-ref.html

Retrieves information about connection form widgets provided by installed providers. Options for output format and verbosity are available.

```bash
airflow providers widgets [-h] [-o (table, json, yaml, plain)] [-v]
```

--------------------------------

### Basic Setup/Teardown with as_teardown

Source: https://airflow.apache.org/docs/apache-airflow/stable/_modules/airflow/example_dags/example_setup_teardown_taskflow.html

This snippet shows how to mark a task as a teardown task and link it to a setup task using the `as_teardown` method. Clearing the intermediate task will also clear its associated setup and teardown tasks.

```python
# [START example_setup_teardown_taskflow]
import pendulum

from airflow.sdk import DAG, setup, task, task_group, teardown

with DAG(
    dag_id="example_setup_teardown_taskflow",
    schedule=None,
    start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
    catchup=False,
    tags=["example"],
) as dag:

    @task
    def my_first_task():
        print("Hello 1")



    @task
    def my_second_task():
        print("Hello 2")

    @task
    def my_third_task():
        print("Hello 3")

    # you can set setup / teardown relationships with the `as_teardown` method.
    task_1 = my_first_task()
    task_2 = my_second_task()
    task_3 = my_third_task()
    task_1 >> task_2 >> task_3.as_teardown(setups=task_1)

    # The method `as_teardown` will mark task_3 as teardown, task_1 as setup, and
    # arrow task_1 >> task_3.
    # Now if you clear task_2, then its setup task, task_1, will be cleared in
    # addition to its teardown task, task_3

    # it's also possible to use a decorator to mark a task as setup or
    # teardown when you define it. see below.

    @setup
    def outer_setup():
        print("I am outer_setup")
        return "some cluster id"

    @teardown
    def outer_teardown(cluster_id):
        print("I am outer_teardown")
        print(f"Tearing down cluster: {cluster_id}")

    @task
    def outer_work():
        print("I am just a normal task")

    @task_group
    def section_1():
        @setup
        def inner_setup():
            print("I set up")
            return "some_cluster_id"

        @task
        def inner_work(cluster_id):
            print(f"doing some work with {cluster_id=}")

        @teardown
        def inner_teardown(cluster_id):
            print(f"tearing down {cluster_id=}")

        # this passes the return value of `inner_setup` to both `inner_work` and `inner_teardown`
        inner_setup_task = inner_setup()
        inner_work(inner_setup_task) >> inner_teardown(inner_setup_task)

    # by using the decorators, outer_setup and outer_teardown are already marked as setup / teardown
    # now we just need to make sure they are linked directly.  At a low level, what we need
    # to do so is the following::
    #     s = outer_setup()
    #     t = outer_teardown()
    #     s >> t
    #     s >> outer_work() >> t
    # Thus, s and t are linked directly, and outer_work runs in between.  We can take advantage of
    # the fact that we are in taskflow, along with the context manager on teardowns, as follows:
    with outer_teardown(outer_setup()):
        outer_work()

        # and let's put section 1 inside the outer setup and teardown tasks
        section_1()
# [END example_setup_teardown_taskflow]
```

--------------------------------

### Get Information on Airflow Logging Handlers

Source: https://airflow.apache.org/docs/apache-airflow/stable/cli-and-env-variables-ref.html

Retrieves information about task logging handlers provided by installed providers. Supports verbose output and various output formats.

```bash
airflow providers logging [-h] [-o (table, json, yaml, plain)] [-v]
```

--------------------------------

### LocalFilesystemBackend Initialization

Source: https://airflow.apache.org/docs/apache-airflow/stable/_modules/airflow/secrets/local_filesystem.html

Initializes the LocalFilesystemBackend with optional file paths for variables, connections, and configurations.

```APIDOC
## LocalFilesystemBackend Constructor

### Description
Initializes the LocalFilesystemBackend, allowing you to specify the file paths for storing Airflow variables, connections, and configurations.

### Parameters
- **variables_file_path** (str | None) - Optional. The file path to the data containing Airflow variables.
- **connections_file_path** (str | None) - Optional. The file path to the data containing Airflow connections.
- **configs_file_path** (str | None) - Optional. The file path to the data containing Airflow configurations.
```

--------------------------------

### Get External Log URL Response

Source: https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html

Example JSON response when requesting the external URL for task instance logs. This is useful for accessing logs stored in external systems.

```json
{
  "url": "string"
}
```

--------------------------------

### Install SFTP Extra

Source: https://airflow.apache.org/docs/apache-airflow/stable/extra-packages-ref.html

Install the SFTP extra for SFTP hooks, operators, and sensors.

```bash
pip install 'apache-airflow[sftp]'
```

--------------------------------

### Get Task Instance Logs Response

Source: https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html

Example JSON response for retrieving logs of a specific task instance. Includes log content and a continuation token for paginated results.

```json
{
  "content": [
    {
      "timestamp": "2019-08-24T14:15:22Z",
      "event": "string"
    }
  ],
  "continuation_token": "string"
}
```

--------------------------------

### Airflow DAG and Task Definition with BashOperator

Source: https://airflow.apache.org/docs/apache-airflow/stable/tutorial/fundamentals.html

A complete example of an Airflow DAG using BashOperator for tasks, including default arguments, schedule, start date, and task dependencies. It also shows how to document tasks and the DAG itself.

```python
import textwrap
from datetime import datetime, timedelta

# Operators; we need this to operate!
from airflow.providers.standard.operators.bash import BashOperator

# The DAG object; we'll need this to instantiate a DAG
from airflow.sdk import DAG
with DAG(
    "tutorial",
    # These args will get passed on to each operator
    # You can override them on a per-task basis during operator initialization
    default_args={
        "depends_on_past": False,
        "retries": 1,
        "retry_delay": timedelta(minutes=5),
        # 'queue': 'bash_queue',
        # 'pool': 'backfill',
        # 'priority_weight': 10,
        # 'end_date': datetime(2016, 1, 1),
        # 'wait_for_downstream': False,
        # 'execution_timeout': timedelta(seconds=300),
        # 'on_failure_callback': some_function, # or list of functions
        # 'on_success_callback': some_other_function, # or list of functions
        # 'on_retry_callback': another_function, # or list of functions
        # 'sla_miss_callback': yet_another_function, # or list of functions
        # 'on_skipped_callback': another_function, #or list of functions
        # 'trigger_rule': 'all_success'
    },
    description="A simple tutorial DAG",
    schedule=timedelta(days=1),
    start_date=datetime(2021, 1, 1),
    catchup=False,
    tags=["example"],
) as dag:

    # t1, t2 and t3 are examples of tasks created by instantiating operators
    t1 = BashOperator(
        task_id="print_date",
        bash_command="date",
    )

    t2 = BashOperator(
        task_id="sleep",
        depends_on_past=False,
        bash_command="sleep 5",
        retries=3,
    )
    t1.doc_md = textwrap.dedent(
        """
    #### Task Documentation
    You can document your task using the attributes `doc_md` (markdown),
    `doc` (plain text), `doc_rst`, `doc_json`, `doc_yaml` which gets
    rendered in the UI's Task Instance Details page.
    ![img](https://imgs.xkcd.com/comics/fixing_problems.png)
    **Image Credit:** Randall Munroe, [XKCD](https://xkcd.com/license.html)
    """
    )

    dag.doc_md = __doc__  # providing that you have a docstring at the beginning of the DAG; OR
    dag.doc_md = """
    This is a documentation placed anywhere
    """  # otherwise, type it like this
    templated_command = textwrap.dedent(
        """
    {% for i in range(5) %}
        echo "{{ ds }}"
        echo "{{ macros.ds_add(ds, 7)}}"
    {% endfor %}
    """
    )

    t3 = BashOperator(
        task_id="templated",
        depends_on_past=False,
        bash_command=templated_command,
    )

    t1 >> [t2, t3]

```

--------------------------------

### Activating Virtual Environment in systemd ExecStart

Source: https://airflow.apache.org/docs/apache-airflow/stable/howto/run-with-systemd.html

When Airflow is installed in a virtual environment, the ExecStart command in systemd unit files must be modified to activate the environment first. Replace the example path with your actual virtual environment path.

```bash
ExecStart=/bin/bash -c 'source /home/airflow/airflow_venv/bin/activate && airflow scheduler'
```

--------------------------------

### Install Airflow Meta-Package with Google Auth Extras

Source: https://airflow.apache.org/docs/apache-airflow/stable/extra-packages-ref.html

Install the Airflow meta-package with a Google authentication backend.

```bash
pip install 'apache-airflow[google-auth]'
```

--------------------------------

### Verifying URI Parsing

Source: https://airflow.apache.org/docs/apache-airflow/stable/howto/connection.html

This example demonstrates how to create a Connection object from a URI string and verify that the login and password fields are parsed correctly.

```python
>>> from airflow.sdk import Connection

>>> c = Connection(uri="my-conn-type://my-login:my-password@my-host:5432/my-schema?param1=val1&param2=val2")
>>> print(c.login)
my-login
>>> print(c.password)
my-password

```

--------------------------------

### 200 OK Response for Get Task Instances

Source: https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html

Example of a successful response when retrieving a list of task instances. This JSON structure outlines the fields available for each task instance, including its state, dates, and associated DAG and run information.

```json
{
  "task_instances": [
    {
      "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
      "task_id": "string",
      "dag_id": "string",
      "dag_run_id": "string",
      "map_index": 0,
      "logical_date": "2019-08-24T14:15:22Z",
      "run_after": "2019-08-24T14:15:22Z",
      "start_date": "2019-08-24T14:15:22Z",
      "end_date": "2019-08-24T14:15:22Z",
      "duration": 0,
      "state": "removed",
      "try_number": 0,
      "max_tries": 0,
      "task_display_name": "string",
      "dag_display_name": "string",
      "hostname": "string",
      "unixname": "string",
      "pool": "string",
      "pool_slots": 0,
      "queue": "string",
      "priority_weight": 0,
      "operator": "string",
      "operator_name": "string",
      "queued_when": "2019-08-24T14:15:22Z",
      "scheduled_when": "2019-08-24T14:15:22Z",
      "pid": 0,
      "executor": "string",
      "executor_config": "string",
      "note": "string",
      "rendered_map_index": "string",
      "rendered_fields": { },
      "trigger": {
        "id": 0,
        "classpath": "string",
        "kwargs": "string",
        "created_date": "2019-08-24T14:15:22Z",
        "queue": "string",
        "triggerer_id": 0
      },
      "triggerer_job": {
        "id": 0,
        "dag_id": "string",
        "state": "string",
        "job_type": "string",
        "start_date": "2019-08-24T14:15:22Z",
        "end_date": "2019-08-24T14:15:22Z",
        "latest_heartbeat": "2019-08-24T14:15:22Z",
        "executor_class": "string",
        "hostname": "string",
        "unixname": "string",
        "dag_display_name": "string"
      },
      "dag_version": {
        "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
        "version_number": 0,
        "dag_id": "string",
        "bundle_name": "string",
        "bundle_version": "string",
        "created_at": "2019-08-24T14:15:22Z",
        "dag_display_name": "string",
        "bundle_url": "string"
      }
    }
  ],
  "total_entries": 0,
  "next_cursor": "string",
  "previous_cursor": "string"
}
```

--------------------------------

### Nested Branching DAG Example

Source: https://airflow.apache.org/docs/apache-airflow/stable/_modules/airflow/example_dags/example_nested_branch_dag.html

This DAG demonstrates a workflow with nested branching. The join tasks use the 'none_failed_min_one_success' trigger rule to ensure they are skipped if their preceding branching tasks are skipped. This setup allows for complex conditional execution paths within a DAG.

```python
#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#   http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied.  See the License for the
# specific language governing permissions and limitations
# under the License.
"""
Example DAG demonstrating a workflow with nested branching. The join tasks are created with
``none_failed_min_one_success`` trigger rule such that they are skipped whenever their corresponding
branching tasks are skipped.
"""

from __future__ import annotations

import pendulum

from airflow.providers.standard.operators.empty import EmptyOperator
from airflow.sdk import DAG, TriggerRule, task

with DAG(
    dag_id="example_nested_branch_dag",
    start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
    catchup=False,
    schedule="@daily",
    tags=["example"],
) as dag:

    @task.branch()


[docs]
    def branch(task_id_to_return: str) -> str:
        return task_id_to_return



    branch_1 = branch.override(task_id="branch_1")(task_id_to_return="true_1")
    join_1 = EmptyOperator(task_id="join_1", trigger_rule=TriggerRule.NONE_FAILED_MIN_ONE_SUCCESS)
    true_1 = EmptyOperator(task_id="true_1")
    false_1 = EmptyOperator(task_id="false_1")

    branch_2 = branch.override(task_id="branch_2")(task_id_to_return="true_2")
    join_2 = EmptyOperator(task_id="join_2", trigger_rule=TriggerRule.NONE_FAILED_MIN_ONE_SUCCESS)
    true_2 = EmptyOperator(task_id="true_2")
    false_2 = EmptyOperator(task_id="false_2")
    false_3 = EmptyOperator(task_id="false_3")

    branch_1 >> true_1 >> join_1
    branch_1 >> false_1 >> branch_2 >> [true_2, false_2] >> join_2 >> false_3 >> join_1

```

--------------------------------

### Add Example for Defaults in conn.extras

Source: https://airflow.apache.org/docs/apache-airflow/stable/release_notes.html

This snippet provides an example for accessing default values within 'conn.extras'.

```python
from airflow.models.connection import Connection

conn = Connection(conn_type="http", host="http://localhost:8000")
conn.extra_dejson = {"retry_args": "{ \"attempts\": 3, \"delay\": 5 }"}
print(conn.extra_dejson.get("retry_args", "default_value"))
# Output: { "attempts": 3, "delay": 5 }
```

--------------------------------

### Upgrade Pip and Install Airflow

Source: https://airflow.apache.org/docs/apache-airflow/stable/start.html

Upgrades the pip package installer and then installs Apache Airflow with the 'celery' extra and specific version constraints. This ensures a reproducible installation.

```bash
pip install --upgrade pip
pip install apache-airflow[celery]==3.1.0 --constraint https://raw.githubusercontent.com/apache/airflow/constraints-3.1.0/constraints-3.12.txt

```

--------------------------------

### Verify Airflow Installation

Source: https://airflow.apache.org/docs/apache-airflow/stable/start.html

This command checks the installed Airflow version to confirm that the installation was successful.

```bash
airflow version

```

--------------------------------

### Run All-in-One Airflow Instance

Source: https://airflow.apache.org/docs/apache-airflow/stable/cli-and-env-variables-ref.html

Starts a self-contained, all-in-one copy of Airflow for development or testing purposes.

```bash
airflow standalone [-h]
```

--------------------------------

### Install Memray Separately

Source: https://airflow.apache.org/docs/apache-airflow/stable/howto/memory-profiling.html

Alternatively, install Memray with a specific version if you have an existing Airflow installation.

```bash
pip install 'memray>=1.19.0'
```