### BaseExecutor Start Method

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/_modules/airflow/executors/base_executor.html

Abstract method to be implemented by subclasses. Executors may need to perform setup actions when they are started.

```python
def start(self):  # pragma: no cover

    """Executors may need to get things started."""

```

--------------------------------

### Check Multiple Scheduler Jobs Example

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/cli-ref.html

Example for checking multiple scheduler jobs in a high availability setup. Allows specifying a limit for recent jobs.

```bash
airflow jobs check –job-type SchedulerJob –allow-multiple –limit 100
```

--------------------------------

### Example: Create User with Admin Role

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/cli-ref.html

Example of creating a user with the 'Admin' role and a specified username.

```bash
$ airflow users create 
    
> –username admin –firstname FIRST_NAME –lastname LAST_NAME –role Admin –email admin@example.org
```

--------------------------------

### Start Celery Flower UI

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/executor/celery.html

Launches the Celery Flower web UI, which provides monitoring capabilities for your Celery workers. The 'flower' Python library must be installed.

```bash
airflow celery flower

```

--------------------------------

### Start Airflow Webserver

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/start/local.html

Starts the Airflow webserver on the specified port. This is necessary to access the Airflow UI.

```bash
airflow webserver --port 8080
```

--------------------------------

### Start Airflow Webserver

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/cli-and-env-variables-ref.html

Start an Airflow webserver instance. Supports various configurations for logging, port, and worker class.

```bash
airflow webserver [-h] [-A ACCESS_LOGFILE] [-L ACCESS_LOGFORMAT] [-D] [-d]
                  [-E ERROR_LOGFILE] [-H HOSTNAME] [-l LOG_FILE] [--pid [PID]]
                  [-p PORT] [--ssl-cert SSL_CERT] [--ssl-key SSL_KEY]
                  [--stderr STDERR] [--stdout STDOUT] [-t WORKER_TIMEOUT]
                  [-k {sync,eventlet,gevent,tornado}] [-w WORKERS]
```

--------------------------------

### Install Airflow with Google Auth Extra

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/extra-packages-ref.html

Installs Airflow with the 'google_auth' extra to enable the Google authentication backend.

```bash
pip install 'apache-airflow[google_auth]'
```

--------------------------------

### Install Airflow Documentation Extras

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/extra-packages-ref.html

Use this command to install the packages required for building Airflow documentation. This is typically needed during development.

```bash
pip install 'apache-airflow[doc]'
```

--------------------------------

### Install Airflow with Celery Extra

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/extra-packages-ref.html

Installs Airflow with the 'celery' extra, enabling the CeleryExecutor and also installing the celery provider package.

```bash
pip install 'apache-airflow[celery]'
```

--------------------------------

### Install Airflow with Dask Extra

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/extra-packages-ref.html

Installs Airflow with the 'dask' extra to enable the DaskExecutor.

```bash
pip install 'apache-airflow[dask]'
```

--------------------------------

### Install Airflow with Password Extra

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/extra-packages-ref.html

Installs Airflow with the 'password' extra for password authentication of users.

```bash
pip install 'apache-airflow[password]'
```

--------------------------------

### Install Airflow with Async Extra

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/extra-packages-ref.html

Installs Airflow with the 'async' extra to enable Async worker classes for Gunicorn.

```bash
pip install 'apache-airflow[async]'
```

--------------------------------

### Start a triggerer instance

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/cli-and-env-variables-ref.html

Start a triggerer instance to manage deferred tasks. Options include daemonization, log file location, and capacity.

```bash
airflow triggerer [-h] [--capacity CAPACITY] [-D] [-l LOG_FILE] [--pid [PID]] [--stderr STDERR] [--stdout STDOUT]
```

--------------------------------

### Install Airflow with LevelDB Extra

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/extra-packages-ref.html

Installs Airflow with the 'leveldb' extra, which is required for using the leveldb extra in the Google provider.

```bash
pip install 'apache-airflow[leveldb]'
```

--------------------------------

### Check Scheduler Job Status Example

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/cli-ref.html

Example of checking if the local scheduler is working. Requires specifying the job type and hostname.

```bash
airflow jobs check –job-type SchedulerJob –hostname “$(hostname)”
```

--------------------------------

### Create and Start Kubernetes Job Watcher

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/_modules/airflow/executors/kubernetes_executor.html

Creates and starts a KubernetesJobWatcher to monitor pod statuses, retrieving the initial resource version.

```python
def _make_kube_watcher(self) -> KubernetesJobWatcher:
        resource_version = ResourceVersion().resource_version
        watcher = KubernetesJobWatcher(
            watcher_queue=self.watcher_queue,
            namespace=self.kube_config.kube_namespace,
            multi_namespace_mode=self.kube_config.multi_namespace_mode,
            resource_version=resource_version,
            scheduler_job_id=self.scheduler_job_id,
            kube_config=self.kube_config,
        )
        watcher.start()
        return watcher
```

--------------------------------

### Install Airflow with RabbitMQ Extra

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/extra-packages-ref.html

Installs Airflow with the 'rabbitmq' extra to enable RabbitMQ support as a Celery backend.

```bash
pip install 'apache-airflow[rabbitmq]'
```

--------------------------------

### Install Airflow with Kerberos Extra

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/extra-packages-ref.html

Installs Airflow with the 'kerberos' extra for Kerberos integration with services like Hadoop and Presto/Trino.

```bash
pip install 'apache-airflow[kerberos]'
```

--------------------------------

### Install Apache Airflow with Apache Pinot Extra

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/extra-packages-ref.html

Installs the apache-airflow package with the 'apache.pinot' extra, providing Apache Pinot hooks.

```bash
pip install 'apache-airflow[apache.pinot]'
```

--------------------------------

### Install Apache Airflow with Apache Cassandra Extra

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/extra-packages-ref.html

Installs the apache-airflow package with the 'apache.cassandra' extra, adding dependencies for Cassandra-related operators and hooks.

```bash
pip install 'apache-airflow[apache.cassandra]'
```

--------------------------------

### Install setuptools, wheel

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/modules_management.html

Before creating a Python package, ensure you have the necessary packaging tools installed. Upgrade pip, setuptools, and wheel using pip.

```bash
pip install --upgrade pip setuptools wheel

```

--------------------------------

### Get Previous Task Instance Start Date

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/_modules/airflow/models/taskinstance.html

Retrieves the start date of the previous task instance, optionally filtering by state. Handles cases where the previous instance or its start date might not exist.

```python
def get_previous_start_date(self, state=None, session=None):
        """
        Get previous task instance start date.

        :param state: If passed, it only take into account instances of a specific state.
        :param session: SQLAlchemy ORM Session
        """
        self.log.debug("previous_start_date was called")
        prev_ti = self.get_previous_ti(state=state, session=session)
        # prev_ti may not exist and prev_ti.start_date may be None.
        return prev_ti and prev_ti.start_date and pendulum.instance(prev_ti.start_date)
```

--------------------------------

### Get Previous Start Date

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/_modules/airflow/models/taskinstance.html

Retrieves the start date of the previous task instance, optionally filtered by state. This method is intended to fetch the start date, similar to how `get_previous_execution_date` fetches the execution date.

```python
    @provide_session

    def get_previous_start_date(
        self, state: Optional[str] = None, session: Session = None
    ) -> Optional[pendulum.DateTime]:
        """
        The start date from property previous_ti_success.

        
```

--------------------------------

### Airflow Webserver

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/cli-ref.html

Start an Airflow webserver instance.

```APIDOC
## airflow webserver

### Description
Start a webserver instance for Apache Airflow.

### Method
`airflow webserver`

### Parameters
#### Named Arguments
- **-A, --access-logfile** (string) - Path to the access log file. Use '-' to print to stderr. Defaults to '-'.
- **-L, --access-logformat** (string) - The access log format for gunicorn logs.
- **-D, --daemon** (boolean) - Run the webserver in daemon mode (background). Defaults to False.
- **-d, --debug** (boolean) - Use the Flask development server in debug mode. Defaults to False.
- **-E, --error-logfile** (string) - Path to the error log file. Use '-' to print to stderr. Defaults to '-'.
- **-H, --hostname** (string) - The hostname to run the webserver on. Defaults to '0.0.0.0'.
- **-l, --log-file** (string) - Location of the general log file.
- **--pid** (string) - Path to the PID file.
- **-p, --port** (integer) - The port to run the webserver on. Defaults to 8080.
- **--ssl-cert** (string) - Path to the SSL certificate file.
- **--ssl-key** (string) - Path to the SSL key file.
- **--stderr** (string) - Redirect stderr to this file.
- **--stdout** (string) - Redirect stdout to this file.
- **-t, --worker-timeout** (integer) - Timeout for waiting on webserver workers. Defaults to 120.
- **-k, --workerclass** (string) - The worker class to use for Gunicorn. Possible choices: sync, eventlet, gevent, tornado. Defaults to 'sync'.
- **-w, --workers** (integer) - Number of workers to run the webserver on. Defaults to 4.

### Example
```bash
airflow webserver --port 8080
airflow webserver --daemon --workers 4
```
```

--------------------------------

### Get Previous Start Date (Success)

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/_modules/airflow/models/taskinstance.html

Retrieves the start date of the previously successful DAG run. Returns None if no previous successful DAG run is found.

```python
def get_prev_start_date_success() -> Optional[pendulum.DateTime]:
    dagrun = _get_previous_dagrun_success()
    if dagrun is None:
        return None
    return timezone.coerce_datetime(dagrun.start_date)
```

--------------------------------

### run

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/_api/airflow/models/index.html

Executes the TaskInstance, handling dependencies and state changes.

```APIDOC
## run

### Description
Run TaskInstance.

### Parameters
* **verbose** (bool) - whether to turn on more verbose logging
* **ignore_all_deps** (bool) - Ignore all of the non-critical dependencies, just runs
* **ignore_depends_on_past** (bool) - Ignore depends_on_past DAG attribute
* **ignore_task_deps** (bool) - Don’t check the dependencies of this TaskInstance’s task
* **ignore_ti_state** (bool) - Disregards previous task instance state
* **mark_success** (bool) - Don’t run the task, mark its state as success
* **test_mode** (bool) - Doesn’t record success or failure in the DB
* **job_id** (Optional[str]) - Job (BackfillJob / LocalTaskJob / SchedulerJob) ID
* **pool** (str) - specifies the pool to use to run the task instance
* **session** (Session) - SQLAlchemy ORM Session.
```

--------------------------------

### Deprecated Previous Start Date Success

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/_modules/airflow/models/taskinstance.html

Provides a deprecated attribute to get the start date of the previous successful task instance. It issues a warning and directs users to use the `get_previous_start_date` method instead.

```python
@property
def previous_start_date_success(self) -> Optional[pendulum.DateTime]:
        """
        This attribute is deprecated.
        Please use `airflow.models.taskinstance.TaskInstance.get_previous_start_date` method.
        """
        warnings.warn(
            """
            This attribute is deprecated.
            Please use `airflow.models.taskinstance.TaskInstance.get_previous_start_date` method.
            "",
            DeprecationWarning,
            stacklevel=2,
        )
        return self.get_previous_start_date(state=State.SUCCESS)
```

--------------------------------

### Create setup.py for Package Metadata

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/modules_management.html

The setup.py file contains metadata for your Python package, such as its name. This is essential for building and distributing the package.

```python
import setuptools

setuptools.setup(
    name="airflow_operators",
)

```

--------------------------------

### Check Dependencies and Set Task Instance State to Running

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/_modules/airflow/models/taskinstance.html

Prepares a task instance for execution by checking its dependencies and state. If dependencies are met and the task is eligible, its state is set to RUNNING. Various flags allow for ignoring specific dependency checks or marking the task as successful without execution.

```python
    @provide_session
    def check_and_change_state_before_execution(
        self,
        verbose: bool = True,
        ignore_all_deps: bool = False,
        ignore_depends_on_past: bool = False,
        ignore_task_deps: bool = False,
        ignore_ti_state: bool = False,
        mark_success: bool = False,
        test_mode: bool = False,
        job_id: Optional[str] = None,
        pool: Optional[str] = None,
        external_executor_id: Optional[str] = None,
        session=None,
    ) -> bool:
        """
        Checks dependencies and then sets state to RUNNING if they are met. Returns
        True if and only if state is set to RUNNING, which implies that task should be
        executed, in preparation for _run_raw_task

        :param verbose: whether to turn on more verbose logging
        :type verbose: bool
        :param ignore_all_deps: Ignore all of the non-critical dependencies, just runs
        :type ignore_all_deps: bool
        :param ignore_depends_on_past: Ignore depends_on_past DAG attribute
        :type ignore_depends_on_past: bool
        :param ignore_task_deps: Don't check the dependencies of this TaskInstance's task
        :type ignore_task_deps: bool
        :param ignore_ti_state: Disregards previous task instance state
        :type ignore_ti_state: bool
        :param mark_success: Don't run the task, mark its state as success
        :type mark_success: bool
        :param test_mode: Doesn't record success or failure in the DB
        :type test_mode: bool
        :param job_id: Job (BackfillJob / LocalTaskJob / SchedulerJob) ID
        :type job_id: str
        :param pool: specifies the pool to use to run the task instance
        :type pool: str
        :param external_executor_id: The identifier of the celery executor
        :type external_executor_id: str
        :param session: SQLAlchemy ORM Session
        :type session: Session
        :return: whether the state was changed to running or not
        :rtype: bool
        """
        task = self.task
        self.refresh_from_task(task, pool_override=pool)
        self.test_mode = test_mode
        self.refresh_from_db(session=session, lock_for_update=True)
        self.job_id = job_id
        self.hostname = get_hostname()
        self.pid = None

        if not ignore_all_deps and not ignore_ti_state and self.state == State.SUCCESS:
            Stats.incr('previously_succeeded', 1, 1)

        # TODO: Logging needs cleanup, not clear what is being printed
        hr_line_break = "\n" + ("-" * 80)  # Line break
```

--------------------------------

### Install Airflow Providers Separately

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/installation/installing-from-pypi.html

Manually install a specific Airflow provider package. It's recommended to use the 'main' version of constraint files to get the latest provider versions, which are generally compatible with most Airflow versions.

```bash
PYTHON_VERSION="$(python --version | cut -d " " -f 2 | cut -d "." -f 1-2)"
# For example: 3.6
CONSTRAINT_URL="https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-${PYTHON_VERSION}.txt"
pip install "apache-airflow-providers-google" --constraint "${CONSTRAINT_URL}"
```

--------------------------------

### DummyOperator

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/_api/airflow/operators/dummy/index.html

The DummyOperator is a no-operation task. It can be used to create a task that does nothing, for example, to represent the start or end of a DAG, or as a placeholder for tasks that will be implemented later.

```APIDOC
## DummyOperator

### Description
A no-operation task. Useful for creating a task that does nothing, for example, to represent the start or end of a DAG, or as a placeholder for tasks that will be implemented later.

### Parameters
This operator does not have any specific parameters beyond the standard BaseOperator parameters.

### Example
```python
from airflow.operators.dummy import DummyOperator

start = DummyOperator(task_id='start')
end = DummyOperator(task_id='end')

start >> end
```
```

--------------------------------

### Install Airflow with Extras and Providers

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/installation/installing-from-pypi.html

Use this command to install a specific version of Airflow along with extras like 'async', 'postgres', and 'google' providers. It ensures compatibility by using a constraint file.

```bash
AIRFLOW_VERSION=2.2.2
PYTHON_VERSION="$(python --version | cut -d " " -f 2 | cut -d "." -f 1-2)"
CONSTRAINT_URL="https://raw.githubusercontent.com/apache/airflow/constraints-${AIRFLOW_VERSION}/constraints-${PYTHON_VERSION}.txt"
pip install "apache-airflow[async,postgres,google]==${AIRFLOW_VERSION}" --constraint "${CONSTRAINT_URL}"
```

--------------------------------

### Get Root Tasks of a DAG

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/_modules/airflow/models/dag.html

Retrieves all tasks within a DAG that have no upstream dependencies. These tasks are the starting points of the DAG's execution flow.

```python
from airflow.models.dag import DAG

# Assuming 'dag' is an instance of DAG
root_tasks = dag.roots

```

--------------------------------

### LocalFilesystemBackend Initialization

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/_modules/airflow/secrets/local_filesystem.html

Initializes the LocalFilesystemBackend with optional file paths for variables and connections. If paths are not provided, the respective loading will be skipped.

```python
def __init__(
        self,
        variables_file_path: Optional[str] = None,
        connections_file_path: Optional[str] = None
    ):
        super().__init__()
        self.variables_file = variables_file_path
        self.connections_file = connections_file_path
```

--------------------------------

### DAG with Naive Start Date

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/timezone.html

Example of defining a DAG with a naive datetime object for the start_date. Airflow will interpret this naive date using the default timezone.

```python
from airflow.models import DAG
from airflow.operators.dummy_operator import DummyOperator
from datetime import datetime

default_args = dict(start_date=datetime(2016, 1, 1), owner="airflow")

dag = DAG("my_dag", default_args=default_args)
op = DummyOperator(task_id="dummy", dag=dag)
print(op.owner)  # Airflow
```

--------------------------------

### run

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/_api/airflow/models/taskinstance/index.html

Executes the TaskInstance.

```APIDOC
## run

### Description
Run TaskInstance.

### Method
Not Applicable (Python method)

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Parameters
* **verbose** (bool) - whether to turn on more verbose logging
* **ignore_all_deps** (bool) - Ignore all of the non-critical dependencies, just runs
* **ignore_depends_on_past** (bool) - Ignore depends_on_past DAG attribute
* **ignore_task_deps** (bool) - Don’t check the dependencies of this TaskInstance’s task
* **ignore_ti_state** (bool) - Disregards previous task instance state
* **mark_success** (bool) - Don’t run the task, mark its state as success
* **test_mode** (bool) - Doesn’t record success or failure in the DB
* **job_id** (Optional[str]) - Job (BackfillJob / LocalTaskJob / SchedulerJob) ID
* **pool** (Optional[str]) - specifies the pool to use to run the task instance
* **session** (Session) - SQLAlchemy ORM Session
```

--------------------------------

### Get Airflow API authentication backend

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/stable-rest-api-ref.html

This command retrieves the currently configured authentication backend for the Airflow API. Ensure you have the Airflow CLI installed and configured.

```bash
$ airflow config get-value api auth_backend
airflow.api.auth.backend.basic_auth
```

--------------------------------

### Create User Response Sample (200 OK)

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/stable-rest-api-ref.html

Example of a successful response when creating a user. Includes user details and metadata.

```json
{
  "first_name": "string",
  "last_name": "string",
  "username": "string",
  "email": "string",
  "active": true,
  "last_login": "string",
  "login_count": 0,
  "failed_login_count": 0,
  "roles": [
    {
      "name": "string"
}
],
  "created_on": "string",
  "changed_on": "string"

}
```

--------------------------------

### Build Wheel Package

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/modules_management.html

Use the setup.py script to build a wheel (.whl) file for your package. This file can then be installed using pip.

```bash
python setup.py bdist_wheel

```

--------------------------------

### Get Single Event Log Entry Response Sample

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/stable-rest-api-ref.html

Example response for retrieving a single event log entry by its ID. It provides detailed information about a specific event.

```json
{
  "event_log_id": 0,
  "when": "2019-08-24T14:15:22Z",
  "dag_id": "string",
  "task_id": "string",
  "event": "string",
  "execution_date": "2019-08-24T14:15:22Z",
  "owner": "string",
  "extra": "string"
}
```

--------------------------------

### Executor Configuration Example

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/_modules/airflow/models/baseoperator.html

Demonstrates how to specify executor-specific configuration, such as running a task in a custom Docker image with the KubernetesExecutor.

```python
MyOperator(
    ...,
    executor_config={
        "KubernetesExecutor":
            {"image": "myCustomDockerImage"}
    }
)
```

--------------------------------

### Get Sensor Logger Instance

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/_modules/airflow/sensors/smart_sensor.html

Retrieves or creates a logger for a specific sensor instance, ensuring it has the correct handlers and context set. It logs the start of processing for the sensor task.

```python
import logging
from airflow.utils.log.logging_mixin import set_context
from airflow.utils.network import get_hostname


def _get_sensor_logger(self, si):
    """Return logger for a sensor instance object."""
    # The created log_id is used inside of smart sensor as the key to fetch
    # the corresponding in memory log handler.
    si.raw = False  # Otherwise set_context will fail
    log_id = "-".join(
        [si.dag_id, si.task_id, si.execution_date.strftime("%Y_%m_%dT%H_%M_%S_%f"), str(si.try_number)]
    )
    logger = logging.getLogger('airflow.task' + '.' + log_id)

    if len(logger.handlers) == 0:
        handler = self.create_new_task_handler()
        logger.addHandler(handler)
        set_context(logger, si)

        line_break = "-" * 120
        logger.info(line_break)
        logger.info(
            "Processing sensor task %s in smart sensor service on host: %s", self.ti_key, get_hostname()
        )
        logger.info(line_break)
    return logger
```

--------------------------------

### Get Airflow Variables in Python

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/concepts/variables.html

Demonstrates how to retrieve variable values using the Variable model in Python. Includes examples for normal retrieval, auto-deserializing JSON, and providing default values.

```python
from airflow.models import Variable

# Normal call style
foo = Variable.get("foo")

# Auto-deserializes a JSON value
bar = Variable.get("bar", deserialize_json=True)

# Returns the value of default_var (None) if the variable is not set
baz = Variable.get("baz", default_var=None)

```

--------------------------------

### Example DAG Package Structure

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/concepts/dags.html

Illustrates the file structure for a zip-packaged DAG, including multiple DAG files and a custom Python package.

```text
my_dag1.py
my_dag2.py
package1/__init__.py
package1/functions.py
```

--------------------------------

### Get DAG Runs Between Dates

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/_modules/airflow/models/dag.html

Retrieves a list of all DagRuns that fall within a specified start and end date (inclusive). It filters DagRuns by DAG ID and the execution date range using SQLAlchemy.

```python
@provide_session
def get_dagruns_between(self, start_date, end_date, session=None):
    """
    Returns the list of dag runs between start_date (inclusive) and end_date (inclusive).

    :param start_date: The starting execution date of the DagRun to find.
    :param end_date: The ending execution date of the DagRun to find.
    :param session:
    :return: The list of DagRuns found.
    """
    dagruns = (
        session.query(DagRun)
        .filter(
            DagRun.dag_id == self.dag_id,
            DagRun.execution_date >= start_date,
            DagRun.execution_date <= end_date,
        )
        .all()
    )

    return dagruns
```

--------------------------------

### Sample .env Export Format

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/howto/connection.html

An example of connections exported in .env format. Each line defines a connection using a URI string.

```env
airflow_db=mysql://root:plainpassword@mysql/airflow
druid_broker_default=druid://druid-broker:8082?endpoint=druid%2Fv2%2Fsql

```

--------------------------------

### Get DAG Run Dates

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/_modules/airflow/models/dag.html

Retrieves a list of dates for DAG runs within a specified start and end date interval, using the DAG's schedule interval. These dates can be used for execution.

```python
    def get_run_dates(self, start_date, end_date=None):
        """
        Returns a list of dates between the interval received as parameter using this
        dag's schedule interval. Returned dates can be used for execution dates.
        
```

--------------------------------

### Instantiate Custom Timetable

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/howto/timetable.html

Example of how to instantiate a custom timetable with parameters for a DAG.

```python
with DAG(
    timetable=SometimeAfterWorkdayTimetable(Time(8)),  # 8am.
    ...,
) as dag:
    ...

```

--------------------------------

### Get Task Instances with Filters

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/_modules/airflow/models/dag.html

Retrieves task instances based on specified date ranges, states, and session. Defaults to the last 30 days if no start date is provided. Excludes subDAGs and parent DAGs by default.

```python
    def get_task_instances(
        self, start_date=None, end_date=None, state=None, session=None
    ) -> List[TaskInstance]:
        if not start_date:
            start_date = (timezone.utcnow() - timedelta(30)).date()
            start_date = timezone.make_aware(datetime.combine(start_date, datetime.min.time()))

        return (
            self._get_task_instances(
                task_ids=None,
                start_date=start_date,
                end_date=end_date,
                run_id=None,
                state=state,
                include_subdags=False,
                include_parentdag=False,
                include_dependent_dags=False,
                exclude_task_ids=[],
                as_pk_tuple=False,
                session=session,
            )
            .join(TaskInstance.dag_run)
            .order_by(DagRun.execution_date)

            .all()
        )
```

--------------------------------

### Get Next DagRun Information

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/_modules/airflow/models/dag.html

Calculates information for the next DagRun based on the DAG's timetable, start date, and end date. It does not enforce limits like max active runs. The `restricted` parameter controls whether DAG-level start/end dates and catchup settings are considered.

```python
def next_dagrun_info(
        self,
        last_automated_dagrun: Union[None, datetime, DataInterval],
        *, 
        restricted: bool = True,
    ) -> Optional[DagRunInfo]:
        """Get information about the next DagRun of this dag after ``date_last_automated_dagrun``.

        This calculates what time interval the next DagRun should operate on
        (its execution date), and when it can be scheduled, , according to the
        dag's timetable, start_date, end_date, etc. This doesn't check max
        active run or any other "max_active_tasks" type limits, but only
        performs calculations based on the various date and interval fields of
        this dag and its tasks.

        :param date_last_automated_dagrun: The ``max(execution_date)`` of
            existing "automated" DagRuns for this dag (scheduled or backfill, 
            but not manual).
        :param restricted: If set to *False* (default is *True*), ignore
            ``start_date``, ``end_date``, and ``catchup`` specified on the DAG
            or tasks.
        :return: DagRunInfo of the next dagrun, or None if a dagrun is not
            going to be scheduled.
        """
        # Never schedule a subdag. It will be scheduled by its parent dag.
        if self.is_subdag:
            return None
        if isinstance(last_automated_dagrun, datetime):
            warnings.warn(
                "Passing a datetime to DAG.next_dagrun_info is deprecated. Use a DataInterval instead.",
                DeprecationWarning,
                stacklevel=2,
            )
            data_interval = self.infer_automated_data_interval(
                timezone.coerce_datetime(last_automated_dagrun)
            )
        else:
            data_interval = last_automated_dagrun
        if restricted:
            restriction = self._time_restriction
        else:
            restriction = TimeRestriction(earliest=None, latest=None, catchup=True)
        try:
            info = self.timetable.next_dagrun_info(
                last_automated_data_interval=data_interval,
                restriction=restriction,
            )
        except Exception:
            self.log.exception(
                "Failed to fetch run info after data interval %s for DAG %r",
                data_interval,
                self.dag_id,
            )
            info = None
        return info
```

--------------------------------

### Download, Compile, and Install SQLite from Source

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/howto/set-up-database.html

Steps to download the SQLite source code, configure build options with specific flags, compile, and install it locally. This process is used to upgrade SQLite to a version compatible with Airflow requirements.

```bash
wget https://www.sqlite.org/src/tarball/sqlite.tar.gz
tar xzf sqlite.tar.gz
cd sqlite/
export CFLAGS="-DSQLITE_ENABLE_FTS3 \
    -DSQLITE_ENABLE_FTS3_PARENTHESIS \
    -DSQLITE_ENABLE_FTS4 \
    -DSQLITE_ENABLE_FTS5 \
    -DSQLITE_ENABLE_JSON1 \
    -DSQLITE_ENABLE_LOAD_EXTENSION \
    -DSQLITE_ENABLE_RTREE \
    -DSQLITE_ENABLE_STAT4 \
    -DSQLITE_ENABLE_UPDATE_DELETE_LIMIT \
    -DSQLITE_SOUNDEX \
    -DSQLITE_TEMP_STORE=3 \
    -DSQLITE_USE_URI \
    -O2 \
    -fPIC"
export PREFIX="/usr/local"
LIBS="-lm" ./configure --disable-tcl --enable-shared --enable-tempstore=always --prefix="$PREFIX"
make
make install

```

--------------------------------

### Start Celery Executor

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/_modules/airflow/executors/celery_executor.html

Logs the initialization of the Celery Executor, indicating the number of processes used for syncing.

```python
    def start(self) -> None:
        self.log.debug('Starting Celery Executor using %s processes for syncing', self._sync_parallelism)
```

--------------------------------

### Install Airflow with Docker Extra

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/upgrading-from-1-10/index.html

Install Airflow with the 'docker' extra. This command automatically installs the 'apache-airflow-providers-docker' package along with Airflow core.

```bash
pip install 'apache-airflow[docker]'

```

--------------------------------

### Register Plugin with Setuptools Entry Point

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/plugins.html

Register your plugin using a setuptools entry point in your package's setup.py. This allows Airflow to automatically discover and load the plugin when the package is installed.

```python
from setuptools import setup

setup(
    name="my-package",
    # ...
    entry_points={
        "airflow.plugins": ["my_plugin = my_package.my_plugin:MyAirflowPlugin"]
    },
)


```

--------------------------------

### Install Airflow with CNCF Kubernetes Extra

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/extra-packages-ref.html

Installs Airflow with the 'cncf.kubernetes' extra, enabling the Kubernetes Executor and also installing the kubernetes provider package.

```bash
pip install 'apache-airflow[cncf.kubernetes]'
```

--------------------------------

### Install Core Airflow Without Providers

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/installation/installing-from-pypi.html

Install the core Apache Airflow package without any additional providers. This command uses a specific constraint file designed for installations without providers.

```bash
AIRFLOW_VERSION=2.2.2
PYTHON_VERSION="$(python --version | cut -d " " -f 2 | cut -d "." -f 1-2)"
# For example: 3.6
CONSTRAINT_URL="https://raw.githubusercontent.com/apache/airflow/constraints-${AIRFLOW_VERSION}/constraints-no-providers-${PYTHON_VERSION}.txt"
# For example: https://raw.githubusercontent.com/apache/airflow/constraints-2.2.2/constraints-no-providers-3.6.txt
pip install "apache-airflow==${AIRFLOW_VERSION}" --constraint "${CONSTRAINT_URL}"
```

--------------------------------

### Install Airflow Locally

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/start/local.html

Installs Airflow using the specified version and Python version's constraint file. Ensure AIRFLOW_HOME is set if you prefer a custom directory.

```bash
# Airflow needs a home. `~/airflow` is the default, but you can put it
# somewhere else if you prefer (optional)
export AIRFLOW_HOME=~/airflow

# Install Airflow using the constraints file
AIRFLOW_VERSION=2.2.2
PYTHON_VERSION=$(python --version | cut -d " " -f 2 | cut -d "." -f 1-2)
# For example: 3.6
CONSTRAINT_URL="https://raw.githubusercontent.com/apache/airflow/constraints-${AIRFLOW_VERSION}/constraints-${PYTHON_VERSION}.txt"
# For example: https://raw.githubusercontent.com/apache/airflow/constraints-2.2.2/constraints-3.6.txt
pip install "apache-airflow==${AIRFLOW_VERSION}" --constraint "${CONSTRAINT_URL}"

# The Standalone command will initialise the database, make a user,
# and start all components for you.
airflow standalone

# Visit localhost:8080 in the browser and use the admin account details
# shown on the terminal to login.
# Enable the example_bash_operator dag in the home page

```

--------------------------------

### Initialize Package with __init__.py

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/modules_management.html

Create an __init__.py file within your package directory to mark it as a Python package. This file can contain initialization code.

```python
print("Hello from airflow_operators")

```

--------------------------------

### Update Library Path After SQLite Installation

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/howto/set-up-database.html

After installing SQLite from source to a custom location like `/usr/local/lib`, update the `LD_LIBRARY_PATH` environment variable. This ensures that the system can find the newly installed SQLite libraries.

```bash
export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH

```

--------------------------------

### Install Prerequisites for Upgrading SQLite

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/howto/set-up-database.html

Install necessary packages like wget, tar, gzip, gcc, make, and expect on AmazonLinux or similar environments. These are required for compiling and installing a newer version of SQLite from source.

```bash
yum -y install wget tar gzip gcc make expect

```

--------------------------------

### SqlSensor Initialization

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/_modules/airflow/sensors/sql.html

Initializes the SqlSensor with connection details, SQL query, and optional success/failure criteria. Use this to set up a sensor that checks a database for specific conditions.

```python
def __init__(
    self, *, conn_id, sql, parameters=None, success=None, failure=None, fail_on_empty=False, **kwargs
):
    self.conn_id = conn_id
    self.sql = sql
    self.parameters = parameters
    self.success = success
    self.failure = failure
    self.fail_on_empty = fail_on_empty
    super().__init__(**kwargs)
```

--------------------------------

### Install Airflow with StatsD Extra

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/extra-packages-ref.html

Installs Airflow with the 'statsd' extra, which is needed by StatsD metrics.

```bash
pip install 'apache-airflow[statsd]'
```

--------------------------------

### CeleryExecutor.start

Source: https://airflow.apache.org/docs/apache-airflow/2.2.2/_api/airflow/executors/celery_executor/index.html

Initializes and starts the Celery Executor, preparing it to accept and manage tasks.

```APIDOC
## CeleryExecutor.start

### Description
Starts the Celery Executor.

### Method
Not specified (likely internal)

### Parameters
* **_self_**: The instance of the CeleryExecutor.
```