### Install Feast SDK and CLI Bash Source: https://github.com/discord/feast/blob/discord/docs/getting-started/quickstart.md Installs the core Feast Python SDK and command-line interface using pip. This is the first step to setting up Feast locally or in any Python environment. Requires pip to be installed. ```Bash pip install feast ``` -------------------------------- ### Starting Feast Web UI Bash Source: https://github.com/discord/feast/blob/discord/docs/getting-started/quickstart.md Launches the experimental Feast Web UI, providing a visual interface to browse registered features, entities, data sources, and feature services within the feature store. ```bash feast ui ``` -------------------------------- ### Initialize Feast Repository Bash Source: https://github.com/discord/feast/blob/discord/docs/getting-started/quickstart.md Initializes a new Feast feature repository named 'my_project' in the current directory using the Feast CLI. It then changes the current directory to the newly created 'feature_repo' within the project. Requires the Feast CLI to be installed. ```Bash feast init my_project cd my_project/feature_repo ``` -------------------------------- ### Install Feast Python Library Source: https://github.com/discord/feast/blob/discord/README.md Installs the core Feast Python package using pip. This is the fundamental step to get started with Feast. It requires a Python environment with pip installed. ```commandline pip install feast ``` -------------------------------- ### Running Feast Quickstart Workflow (Shell) Source: https://github.com/discord/feast/blob/discord/sdk/python/feast/templates/local/README.md Executes the `test_workflow.py` script located in the `feature_repo/` directory, which demonstrates key Feast commands like defining, retrieving, and pushing features. This command runs the full quickstart example. ```shell python test_workflow.py ``` -------------------------------- ### Launch Feast Web UI Source: https://github.com/discord/feast/blob/discord/README.md Starts the experimental Feast web user interface. This UI provides a visual way to explore the registered features and entities within your Feast feature store. ```commandline feast ui ``` -------------------------------- ### Inspect Raw Parquet Data Python Source: https://github.com/discord/feast/blob/discord/docs/getting-started/quickstart.md Uses the pandas library to read and display the contents of the sample 'driver_stats.parquet' file. This snippet helps users visualize the raw data structure and values before they are used to define Feast features. Requires the pandas library to be installed. ```Python import pandas as pd pd.read_parquet("data/driver_stats.parquet") ``` -------------------------------- ### Installing and Renaming protoc-gen-doc Go Tool Source: https://github.com/discord/feast/blob/discord/DISCORD_README.md Installs the `protoc-gen-doc` documentation generator tool using `go get`, adds the Go binary directory to the system's PATH for accessibility, and renames the executable to `protoc-gen-docs` to match a specific requirement in the project's makefile. ```sh go get -u github.com/pseudomuto/protoc-gen-doc/cmd/protoc-gen-doc export PATH=$PATH:$HOME/go/bin mv $HOME/go/bin/protoc-gen-doc $HOME/go/bin/protoc-gen-docs ``` -------------------------------- ### Executing Feast Workflow via Python Script - Shell Source: https://github.com/discord/feast/blob/discord/sdk/python/feast/templates/snowflake/README.md This command executes the `test_workflow.py` script located in the repository root. This script is designed to demonstrate the core Feast commands and workflows showcased in the quickstart, including defining, retrieving, and potentially pushing features. Running this command provides an end-to-end execution of the example setup. Requires Python to be installed and accessible. ```Shell python test_workflow.py ``` -------------------------------- ### Install Feast with Snowflake Source: https://github.com/discord/feast/blob/discord/docs/tutorials/tutorials-overview/driver-stats-on-snowflake.md Installs the 'feast' library along with the necessary dependencies required to use Snowflake as a data store. This is a prerequisite step before initializing a Feast project that leverages Snowflake. ```shell pip install 'feast[snowflake]' ``` -------------------------------- ### Installing Dependencies with Yarn Source: https://github.com/discord/feast/blob/discord/ui/CONTRIBUTING.md Installs all necessary project dependencies, including both runtime and development packages, for the Feast Web UI. This command should be run initially after cloning the repository to set up the development environment. ```shell yarn install ``` -------------------------------- ### Applying Feast Definitions Bash Source: https://github.com/discord/feast/blob/discord/docs/getting-started/quickstart.md Registers feature view and entity definitions found in the repository's Python files and deploys necessary infrastructure, such as creating online store tables. This command reads configuration from `feature_store.yaml`. ```bash feast apply ``` -------------------------------- ### Installing helm-docs Tool - Brew/Shell Source: https://github.com/discord/feast/blob/discord/docs/project/development-guide.md This command installs the `helm-docs` utility using the Homebrew package manager. This tool is used to automatically generate README files for Helm charts. ```Shell brew install norwoodj/tap/helm-docs ``` -------------------------------- ### Run Feast Test Script Source: https://github.com/discord/feast/blob/discord/docs/tutorials/tutorials-overview/driver-stats-on-snowflake.md Executes the 'test.py' Python script that is automatically generated with the Snowflake template. This script contains example code demonstrating the key Feast operations (applying features, getting historical data, materializing, and getting online features) using the configured Snowflake stores. Running this script verifies the basic setup. ```shell python test.py ``` -------------------------------- ### Initializing Feast Repository with Template (Text) Source: https://github.com/discord/feast/blob/discord/docs/reference/feast-cli-commands.md Creates a new Feast feature repository using a specific template (like `gcp`), which pre-configures settings and examples relevant to that cloud provider setup. ```text feast init -t gcp my_feature_repo ``` -------------------------------- ### Install Feast Java Modules Bash Source: https://github.com/discord/feast/blob/discord/java/serving/README.md Installs the necessary Feast Java modules from the root directory using Maven, skipping tests to speed up the build process. ```bash mvn -f java/pom.xml install -Dmaven.test.skip=true ``` -------------------------------- ### Starting Local Trino Container (Shell) Source: https://github.com/discord/feast/blob/discord/sdk/python/feast/infra/offline_stores/contrib/trino_offline_store/README.md Uses the `make` command to initiate and start a Docker container running the Trino database for local testing of the Feast Trino offline store. This command simplifies the setup process by executing predefined Docker commands. Requires a Makefile with the `start-trino-locally` target. ```sh make start-trino-locally ``` -------------------------------- ### Initializing Feast with GCP Template (Shell) Source: https://github.com/discord/feast/blob/discord/sdk/python/feast/templates/local/README.md Initializes a new Feast feature repository using the Google Cloud Platform (GCP) template. This template configures Feast to use GCP services like BigQuery for the offline store, suitable for a more scalable production setup compared to the quickstart's local file store. ```shell feast init -t gcp ``` -------------------------------- ### Ingesting Batch Features Feast Bash Source: https://github.com/discord/feast/blob/discord/docs/getting-started/quickstart.md Serializes the latest feature values up to a specified time (or incrementally since the last materialize call) from batch sources into the online feature store. This prepares the data for low-latency online retrieval. ```bash CURRENT_TIME=$(date -u +"%Y-%m-%dT%H:%M:%S") feast materialize-incremental $CURRENT_TIME ``` -------------------------------- ### Installing Pre-commit Hooks (Shell) Source: https://github.com/discord/feast/blob/discord/docs/project/development-guide.md Installs the 'pre-commit' tool using pip and then configures it to run during 'pre-commit' and 'pre-push' git hooks. This ensures linting and formatting checks run automatically before commits and pushes. Requires Python and pip. ```sh pip install pre-commit pre-commit install --hook-type pre-commit --hook-type pre-push ``` -------------------------------- ### Running Feast Demo Workflow (Shell) Source: https://github.com/discord/feast/blob/discord/sdk/python/feast/templates/rockset/README.md Executes the `test_workflow.py` script using the Python interpreter. This script runs all key Feast commands demonstrated in the quickstart, including defining, retrieving, and pushing features. Prerequisites include having Feast installed and a Python environment configured. ```Shell python test_workflow.py ``` -------------------------------- ### Configure Feast Feature Store YAML Source: https://github.com/discord/feast/blob/discord/docs/getting-started/quickstart.md Defines the core configuration for the Feast feature store using YAML. It sets the project name, specifies file paths for the registry and online store (using SQLite), and sets the provider to 'local'. This file dictates where Feast stores metadata and online feature values. ```YAML project: my_project # By default, the registry is a file (but can be turned into a more scalable SQL-backed registry) registry: data/registry.db # The provider primarily specifies default offline / online stores & storing the registry in a given cloud provider: local online_store: type: sqlite path: data/online_store.db entity_key_serialization_version: 2 ``` -------------------------------- ### Installing Pip-tools (Shell) Source: https://github.com/discord/feast/blob/discord/docs/project/development-guide.md Installs the 'pip-tools' package, which is used for managing Python dependencies by compiling 'requirements.in' files into 'requirements.txt'. Requires pip installed. ```sh pip install pip-tools ``` -------------------------------- ### Install Feast Base Package using Pip - Shell Source: https://github.com/discord/feast/blob/discord/docs/how-to-guides/feast-snowflake-gcp-aws/install-feast.md Installs the core Feast feature store library without any database or cloud-specific dependencies. This is the basic installation required to get started with Feast. ```shell pip install feast ``` -------------------------------- ### Example Feast Feature Store Configuration YAML Source: https://github.com/discord/feast/blob/discord/docs/reference/feature-repository/feature-store-yaml.md This snippet provides a basic example of a `feature_store.yaml` file. It configures a Feast feature store with a specific project name, sets up a local registry using SQLite, specifies the 'local' provider, and defines a local SQLite database for the online store. This configuration is typical for a local development setup. ```yaml project: loyal_spider registry: data/registry.db provider: local online_store: type: sqlite path: data/online_store.db ``` -------------------------------- ### Initializing Feast Python Feature Server (Bash) Source: https://github.com/discord/feast/blob/discord/docs/reference/feature-servers/python-feature-server.md This sequence of Bash commands demonstrates how to set up a new Feast repository locally, apply feature definitions from the repo, materialize sample data into the online store, and finally start the Python feature server using the Feast CLI. Prerequisites include having the Feast CLI installed. ```bash $ feast init feature_repo Creating a new Feast repository in /home/tsotne/feast/feature_repo. $ cd feature_repo $ feast apply Created entity driver Created feature view driver_hourly_stats Created feature service driver_activity Created sqlite table feature_repo_driver_hourly_stats $ feast materialize-incremental $(date +%Y-%m-%d) Materializing 1 feature views to 2021-09-09 17:00:00-07:00 into the sqlite online store. driver_hourly_stats from 2021-09-09 16:51:08-07:00 to 2021-09-09 17:00:00-07:00: 100%|████████████████████████████████████████████████████████████████| 5/5 [00:00<00:00, 295.24it/s] $ feast serve 09/10/2021 10:42:11 AM INFO:Started server process [8889] INFO: Waiting for application startup. 09/10/2021 10:42:11 AM INFO:Waiting for application startup. INFO: Application startup complete. 09/10/2021 10:42:11 AM INFO:Application startup complete. INFO: Uvicorn running on http://127.0.0.1:6566 (Press CTRL+C to quit) 09/10/2021 10:42:11 AM INFO:Uvicorn running on http://127.0.0.1:6566 (Press CTRL+C to quit) ``` -------------------------------- ### Initialize Feast Project Locally - Shell Source: https://github.com/discord/feast/blob/discord/docs/project/release-process.md Initializes a new, empty Feast project directory with basic configuration files and example feature definitions. This command is used as a post-release sanity check to quickly set up a test environment using the newly released Feast SDK. ```Shell feast init ``` -------------------------------- ### Define Feast Features and Sources Python Source: https://github.com/discord/feast/blob/discord/docs/getting-started/quickstart.md Python code defining Feast entities, data sources (FileSource, PushSource, RequestSource), feature views (FeatureView, OnDemandFeatureView), and feature services. This script structures how raw data is transformed into features and grouped for online/offline retrieval, demonstrating key Feast concepts like point-in-time correctness and on-demand transformations. Requires the Feast SDK and Pandas. ```Python # This is an example feature definition file from datetime import timedelta import pandas as pd from feast import ( Entity, FeatureService, FeatureView, Field, FileSource, PushSource, RequestSource, ) from feast.on_demand_feature_view import on_demand_feature_view from feast.types import Float32, Float64, Int64 # Define an entity for the driver. You can think of entity as a primary key used to # fetch features. driver = Entity(name="driver", join_keys=["driver_id"]) # Read data from parquet files. Parquet is convenient for local development mode. For # production, you can use your favorite DWH, such as BigQuery. See Feast documentation # for more info. driver_stats_source = FileSource( name="driver_hourly_stats_source", path="%PARQUET_PATH%", timestamp_field="event_timestamp", created_timestamp_column="created", ) # Our parquet files contain sample data that includes a driver_id column, timestamps and # three feature column. Here we define a Feature View that will allow us to serve this # data to our model online. driver_stats_fv = FeatureView( # The unique name of this feature view. Two feature views in a single # project cannot have the same name name="driver_hourly_stats", entities=[driver], ttl=timedelta(days=1), # The list of features defined below act as a schema to both define features # for both materialization of features into a store, and are used as references # during retrieval for building a training dataset or serving features schema=[ Field(name="conv_rate", dtype=Float32), Field(name="acc_rate", dtype=Float32), Field(name="avg_daily_trips", dtype=Int64), ], online=True, source=driver_stats_source, # Tags are user defined key/value pairs that are attached to each # feature view tags={"team": "driver_performance"}, ) # Defines a way to push data (to be available offline, online or both) into Feast. driver_stats_push_source = PushSource( name="driver_stats_push_source", batch_source=driver_stats_source, ) # Define a request data source which encodes features / information only # available at request time (e.g. part of the user initiated HTTP request) input_request = RequestSource( name="vals_to_add", schema=[ Field(name="val_to_add", dtype=Int64), Field(name="val_to_add_2", dtype=Int64), ], ) # Define an on demand feature view which can generate new features based on # existing feature views and RequestSource features @on_demand_feature_view( sources=[driver_stats_fv, input_request], schema=[ Field(name="conv_rate_plus_val1", dtype=Float64), Field(name="conv_rate_plus_val2", dtype=Float64), ], ) def transformed_conv_rate(inputs: pd.DataFrame) -> pd.DataFrame: df = pd.DataFrame() df["conv_rate_plus_val1"] = inputs["conv_rate"] + inputs["val_to_add"] df["conv_rate_plus_val2"] = inputs["conv_rate"] + inputs["val_to_add_2"] return df # This groups features into a model version driver_activity_v1 = FeatureService( name="driver_activity_v1", features=[ driver_stats_fv[["conv_rate"]], # Sub-selects a feature from a feature view transformed_conv_rate, # Selects all features from the feature view ], ) driver_activity_v2 = FeatureService( name="driver_activity_v2", features=[driver_stats_fv, transformed_conv_rate] ) ``` -------------------------------- ### Installing MySQL via Homebrew (Shell) Source: https://github.com/discord/feast/blob/discord/docs/project/development-guide.md Installs the MySQL database server using the Homebrew package manager. This is needed for CI dependencies, suggesting it might be required for certain integration tests. Requires Homebrew installed (macOS). ```sh brew install mysql ``` -------------------------------- ### Starting React App with Feast UI - Bash Source: https://github.com/discord/feast/blob/discord/docs/reference/alpha-web-ui.md Executes the `start` script defined in the `package.json` of the React application using Yarn. This typically launches a local development server that serves the React app containing the embedded Feast UI component. Requires Yarn and the React project setup. ```Bash yarn start ``` -------------------------------- ### Fetching Online Features via Service Feast Python Source: https://github.com/discord/feast/blob/discord/docs/getting-started/quickstart.md Retrieves the latest feature values from the online store for specific entities using a predefined `FeatureService` object. This simplifies requesting multiple related features. ```python from pprint import pprint from feast import FeatureStore feature_store = FeatureStore('.') # Initialize the feature store feature_service = feature_store.get_feature_service("driver_activity_v1") feature_vector = feature_store.get_online_features( features=feature_service, entity_rows=[ # {join_key: entity_value} {"driver_id": 1004}, {"driver_id": 1005}, ], ).to_dict() pprint(feature_vector) ``` -------------------------------- ### Full Bytewax Engine Configuration Example - YAML Source: https://github.com/discord/feast/blob/discord/docs/reference/batch-materialization/bytewax.md This comprehensive YAML example illustrates various configuration options for the Bytewax batch engine in 'feature_store.yaml'. It includes specifying the Docker image, Kubernetes namespace, image pull secrets, service account, security context capabilities, annotations (e.g., for IAM roles), and resource requests/limits for the job container. ```yaml batch_engine: type: bytewax namespace: bytewax image: bytewax/bytewax-feast:latest image_pull_secrets: - my_container_secret service_account_name: my-k8s-service-account include_security_context_capabilities: false annotations: # example annotation you might include if running on AWS EKS iam.amazonaws.com/role: arn:aws:iam:::role/MyBytewaxPlatformRole resources: limits: cpu: 1000m memory: 2048Mi requests: cpu: 500m memory: 1024Mi ``` -------------------------------- ### Installing Feast Python SDK (Editable Dev) (Shell) Source: https://github.com/discord/feast/blob/discord/docs/project/development-guide.md Installs the Feast Python SDK from the local directory in editable mode ('-e'), including all dependencies specified in the 'dev' extra. Editable mode links the installed package to the source code, allowing changes without reinstallation. Requires pip and the Feast source code. ```sh pip install -e ".[dev]" ``` -------------------------------- ### Initialize Feast Repository Source: https://github.com/discord/feast/blob/discord/README.md Initializes a new Feast feature repository with a specified name and changes the current directory into the main repository folder. This sets up the necessary file structure for defining features. ```commandline feast init my_feature_repo cd my_feature_repo/feature_repo ``` -------------------------------- ### Deploying Feast Feature Server - Shell Source: https://github.com/discord/feast/blob/discord/sdk/python/feast/templates/snowflake/README.md This command is used to deploy one or more instances of a Feast feature server. Feature servers provide low-latency endpoints for retrieving online features required by applications (e.g., for model inference). This is presented as an optional step when moving towards a production setup for serving features online. Requires the Feast CLI to be installed. ```Shell feast serve ``` -------------------------------- ### Defining a Feast Feature Service Python Source: https://github.com/discord/feast/blob/discord/docs/getting-started/quickstart.md Defines a `FeatureService` object, which acts as a logical grouping of features from one or more Feature Views. This object can be referenced when fetching features for easier management and decoupling. ```python from feast import FeatureService driver_stats_fs = FeatureService( name="driver_activity_v1", features=[driver_hourly_stats_view] ) ``` -------------------------------- ### Example project-list.json Configuration - JSON Source: https://github.com/discord/feast/blob/discord/docs/reference/alpha-web-ui.md Provides an example JSON structure for the `project-list.json` file expected by the Feast UI component when used as a module. This file defines the list of Feast projects, their descriptions, unique IDs, and the path to their respective registry files, allowing the UI to browse different projects. ```JSON { "projects": [ { "name": "Credit Score Project", "description": "Project for credit scoring team and associated models.", "id": "credit_score_project", "registryPath": "/registry.json" } ] } ``` -------------------------------- ### Building Feast Python Feature Server Binary - Make/Shell Source: https://github.com/discord/feast/blob/discord/docs/project/development-guide.md This command builds a local development binary for the Feast Python feature server, used before installing the Helm chart with local overrides. ```Shell make build-feature-server-dev ``` -------------------------------- ### Initializing Feast Repository (Text) Source: https://github.com/discord/feast/blob/discord/docs/reference/feast-cli-commands.md Creates a new Feast feature repository directory with a default structure and example files (like `feature_store.yaml`, `example.py`, `data/driver_stats.parquet`). ```text feast init my_repo_name ``` -------------------------------- ### Installing Specific gRPC and Protobuf Versions Source: https://github.com/discord/feast/blob/discord/DISCORD_README.md Installs precise versions of `grpcio`, `grpcio-tools`, `mypy-protobuf`, and `protobuf` using pip to resolve potential version conflicts or backend mismatch issues often encountered when working with generated protocol buffer code. ```sh pip install 'grpcio==1.48.1' 'grpcio-tools==1.48.1' 'mypy-protobuf==1.24' 'protobuf<3.20' ``` -------------------------------- ### Example feature_store.yaml Configuration Source: https://github.com/discord/feast/blob/discord/docs/reference/feature-store-yaml.md This snippet provides an example of a `feature_store.yaml` file for configuring a Feast feature store. It specifies the project name, the location of the registry, the provider (local in this case), and the online store details (type and path for SQLite). ```yaml project: loyal_spider registry: data/registry.db provider: local online_store: type: sqlite path: data/online_store.db ``` -------------------------------- ### Installing Feast Java Helm Chart with Overrides - Helm/Shell Source: https://github.com/discord/feast/blob/discord/docs/project/development-guide.md This command installs the Feast Java feature server Helm chart locally, overriding default values with those specified in `application-override.yaml`, typically to use a locally built Docker image. ```Shell helm install feast-release ../../../infra/charts/feast --values application-override.yaml ``` -------------------------------- ### Displaying gcloud Configuration (Shell) Source: https://github.com/discord/feast/blob/discord/docs/project/development-guide.md Shows the current configuration settings for the gcloud command-line tool, including the active account and project. This is useful for verifying that authentication and project setup were successful before running GCP integration tests. ```Shell $ gcloud config list [core] account = [your email] disable_usage_reporting = True project = [your project id] Your active configuration is: [default] ``` -------------------------------- ### Initialize Feast Project with Snowflake Template Source: https://github.com/discord/feast/blob/discord/docs/tutorials/tutorials-overview/driver-stats-on-snowflake.md Creates a new Feast feature repository using the Snowflake template. This command prompts the user for necessary Snowflake connection details (account, user, password, role, warehouse, database) and optionally uploads example data. It sets up the basic project structure and the 'feature_store.yaml' configuration file. ```shell feast init -t snowflake {feature_repo_name} Snowflake Deployment URL (exclude .snowflakecomputing.com): Snowflake User Name:: Snowflake Password:: Snowflake Role Name (Case Sensitive):: Snowflake Warehouse Name (Case Sensitive):: Snowflake Database Name (Case Sensitive):: Should I upload example data to Snowflake (overwrite table)? [Y/n]: Y cd {feature_repo_name} ``` -------------------------------- ### Fetching Online Features Feast Python Source: https://github.com/discord/feast/blob/discord/docs/getting-started/quickstart.md Retrieves the latest feature values from the online feature store for a list of entities specified by their join keys. This method is used for low-latency inference. ```python from pprint import pprint from feast import FeatureStore store = FeatureStore(repo_path=".") feature_vector = store.get_online_features( features=[ "driver_hourly_stats:conv_rate", "driver_hourly_stats:acc_rate", "driver_hourly_stats:avg_daily_trips", ], entity_rows=[ # {join_key: entity_value} {"driver_id": 1004}, {"driver_id": 1005}, ], ).to_dict() pprint(feature_vector) ``` -------------------------------- ### Apply Feast Feature Definitions Source: https://github.com/discord/feast/blob/discord/README.md Applies the feature definitions found within the current feature repository directory. This registers feature views, entities, and data sources with the Feast feature store, preparing it for data ingestion and retrieval. ```commandline feast apply ``` -------------------------------- ### Installing Feast 0.10+ via pip Source: https://github.com/discord/feast/blob/discord/docs/project/feast-0.9-vs-feast-0.10+.md Describes the simplified process of installing Feast 0.10 and later versions using the standard Python package installer, `pip`. This contrasts with Feast 0.9's complex installation involving heavy components and configurations. ```Shell pip install ``` -------------------------------- ### Building Feast UI Artifacts (Shell) Source: https://github.com/discord/feast/blob/discord/docs/project/development-guide.md Runs the 'make build-ui' command to build the necessary artifacts for the Feast UI. This is an optional step required if you plan to use the 'feast ui' command. Requires Node, Yarn, and Make installed. ```sh make build-ui ``` -------------------------------- ### Installing Project Dependencies with pip Source: https://github.com/discord/feast/blob/discord/DISCORD_README.md Upgrades the pip package manager and installs the current project in editable mode within the active Python virtual environment, allowing for local code modifications without reinstallation. ```sh pip install --upgrade pip pip install -e . ``` -------------------------------- ### Performing Batch Inference with Feast Python Source: https://github.com/discord/feast/blob/discord/docs/getting-started/quickstart.md Generates features for batch scoring models by using the `get_historical_features` call with the current timestamp for each entity. This retrieves the latest available feature values from the offline store. ```python entity_df["event_timestamp"] = pd.to_datetime("now", utc=True) training_df = store.get_historical_features( entity_df=entity_df, features=[ "driver_hourly_stats:conv_rate", "driver_hourly_stats:acc_rate", "driver_hourly_stats:avg_daily_trips", "transformed_conv_rate:conv_rate_plus_val1", "transformed_conv_rate:conv_rate_plus_val2", ], ).to_df() print("\n----- Example features -----\n") print(training_df.head()) ``` -------------------------------- ### Generating Training Data with Feast Python Source: https://github.com/discord/feast/blob/discord/docs/getting-started/quickstart.md Retrieves historical feature values for a given set of entities and timestamps (an entity DataFrame) to generate a training dataset. This uses the `get_historical_features` method, joining relevant offline sources. ```python from datetime import datetime import pandas as pd from feast import FeatureStore # Note: see https://docs.feast.dev/getting-started/concepts/feature-retrieval for # more details on how to retrieve for all entities in the offline store instead entity_df = pd.DataFrame.from_dict( { # entity's join key -> entity values "driver_id": [1001, 1002, 1003], # "event_timestamp" (reserved key) -> timestamps "event_timestamp": [ datetime(2021, 4, 12, 10, 59, 42), datetime(2021, 4, 12, 8, 12, 10), datetime(2021, 4, 12, 16, 40, 26), ], # (optional) label name -> label values. Feast does not process these "label_driver_reported_satisfaction": [1, 5, 3], # values we're using for an on-demand transformation "val_to_add": [1, 2, 3], "val_to_add_2": [10, 20, 30], } ) store = FeatureStore(repo_path=".") training_df = store.get_historical_features( entity_df=entity_df, features=[ "driver_hourly_stats:conv_rate", "driver_hourly_stats:acc_rate", "driver_hourly_stats:avg_daily_trips", "transformed_conv_rate:conv_rate_plus_val1", "transformed_conv_rate:conv_rate_plus_val2", ], ).to_df() print("----- Feature schema -----\n") print(training_df.info()) print() print("----- Example features -----\n") print(training_df.head()) ``` -------------------------------- ### Running Postgres Online Store Integration Tests (Shell) Source: https://github.com/discord/feast/blob/discord/docs/project/development-guide.md Executes integration tests specifically configured for the Postgres online store. This command is part of the contrib test suite and requires the development dependencies to be installed (`pip install -e ".[dev]"`). Note: The command in the source text appears to have a potential typo, missing 'make'. ```Shell test-python-universal-postgres-online ``` -------------------------------- ### Starting React Development Server with Yarn (Bash) Source: https://github.com/discord/feast/blob/discord/ui/README.md This command initiates the development server for the React application using the `start` script typically configured by Create React App. It compiles the application and serves it locally, often with hot-reloading enabled. ```bash yarn start ``` -------------------------------- ### Formatting Python Code (Shell) Source: https://github.com/discord/feast/blob/discord/docs/project/development-guide.md Runs the 'make format-python' command to automatically format the Python codebase according to the project's defined code style (e.g., Black). Requires Make installed and the project's formatting tools configured. ```sh make format-python ``` -------------------------------- ### Displaying Feast Init Help (Shell) Source: https://github.com/discord/feast/blob/discord/sdk/python/feast/templates/local/README.md Shows the available options and templates for the `feast init` command. Use this to see the full list of pre-configured templates for initializing a new Feast feature repository. ```shell feast init --help ``` -------------------------------- ### Configure Feast Serving Local Registry YAML Source: https://github.com/discord/feast/blob/discord/java/serving/README.md Example YAML configuration for `application-override.yaml` specifying a Feast project and a local file path for the feature registry. This file is used to override default application settings. ```yaml feast: project: feast_demo registry: /Users/[your username]/GitHub/feast-demo/feature_repo/data/registry.db entityKeySerializationVersion: 2 ``` -------------------------------- ### Configure Feast Serving Remote Registry Redis YAML Source: https://github.com/discord/feast/blob/discord/java/serving/README.md Example YAML configuration for `application-override.yaml` using a remote registry path (like GCS) and configuring a Redis store for online feature serving. Requires host, port, and optional password for Redis. ```yaml feast: project: feast_java_demo registry: gs://[YOUR BUCKET]/demo-repo/registry.db entityKeySerializationVersion: 2 activeStore: online stores: - name: online type: REDIS config: host: localhost port: 6379 password: [YOUR PASSWORD] ``` -------------------------------- ### Running Development Server with Yarn Source: https://github.com/discord/feast/blob/discord/ui/CONTRIBUTING.md Launches the Feast Web UI application in development mode. It starts a local server, typically accessible at http://localhost:3000, and automatically reloads the page when changes are made to the source code. ```shell yarn start ``` -------------------------------- ### Installing Feast SDK/CLI (Bash) Source: https://github.com/discord/feast/blob/discord/docs/reference/online-stores/dragonfly.md Install the core Feast library using pip. This command installs the necessary CLI and SDK components to interact with Feast projects. ```bash pip install feast ``` -------------------------------- ### Installing Feast Dev Build (Bash) Source: https://github.com/discord/feast/blob/discord/ui/feature_repo/README.md Navigates into the cloned Feast repository directory and installs the Feast package in editable (development) mode along with development dependencies. This command prepares the local environment for running and modifying Feast code for the demo. ```bash cd feast pip install -e ".[dev]" ``` -------------------------------- ### Running Local Feast Integration Tests (Shell) Source: https://github.com/discord/feast/blob/discord/docs/project/development-guide.md Executes the local integration test suite for Feast. This command requires Docker setup locally as it uses ephemeral containers to emulate various online stores like Datastore, DynamoDB, and Redis, testing against a file-based offline store. ```Shell make test-python-integration-local ``` -------------------------------- ### Installing Redis Cluster with Helm (Bash) Source: https://github.com/discord/feast/blob/discord/examples/java-demo/README.md Adds the Bitnami Helm repository, updates it, and installs a Redis cluster named 'my-redis' using the bitnami/redis chart. Requires Helm and a running Kubernetes cluster (like minikube). ```bash helm repo add bitnami https://charts.bitnami.com/bitnami helm repo update helm install my-redis bitnami/redis ``` -------------------------------- ### Installing Redis Cluster with Helm on Kubernetes Source: https://github.com/discord/feast/blob/discord/examples/python-helm-demo/README.md This snippet uses Helm to add the Bitnami repository and install a default Redis cluster into the Kubernetes cluster. This sets up the necessary online store for Feast. ```Bash helm repo add bitnami https://charts.bitnami.com/bitnami helm repo update helm install my-redis bitnami/redis ``` -------------------------------- ### Installing Apache Arrow C++ Libraries on Linux (Shell) Source: https://github.com/discord/feast/blob/discord/docs/reference/feature-servers/go-feature-server.md These shell commands provide the steps necessary to install the required Apache Arrow C++ libraries (`libarrow-dev`) on Debian-based Linux distributions. These libraries are needed for interoperability between Go and Python via cgo memory allocation. ```shell sudo apt update sudo apt install -y -V ca-certificates lsb-release wget wget https://apache.jfrog.io/artifactory/arrow/$(lsb_release --id --short | tr 'A-Z' 'a-z')/apache-arrow-apt-source-latest-$(lsb_release --codename --short).deb sudo apt install -y -V ./apache-arrow-apt-source-latest-$(lsb_release --codename --short).deb sudo apt update sudo apt install -y -V libarrow-dev # For C++ ``` -------------------------------- ### Installing Feast Server with Helm (Bash) Source: https://github.com/discord/feast/blob/discord/examples/java-demo/README.md Installs the Feast Helm chart from the 'feast-charts' repository, naming the release 'feast-release', and applying configuration overrides specified in the `application-override.yaml` file. Deploys the Feast Java feature server to Kubernetes. Requires Helm and the `application-override.yaml` file. ```bash helm install feast-release feast-charts/feast --values application-override.yaml ``` -------------------------------- ### Displaying Feast Version (Text) Source: https://github.com/discord/feast/blob/discord/docs/reference/feast-cli-commands.md Prints the version of the Feast SDK and CLI currently installed in the environment. ```text feast version ``` -------------------------------- ### Upgrading Pip Package Manager (Shell) Source: https://github.com/discord/feast/blob/discord/docs/project/development-guide.md Upgrades the pip package installer to the latest version. This is a recommended step to ensure compatibility and access to the newest pip features. Requires pip installed. ```sh pip install --upgrade pip ``` -------------------------------- ### Example Feast .feastignore Configuration (Text) Source: https://github.com/discord/feast/blob/discord/docs/reference/feast-ignore.md This snippet provides an example of the `.feastignore` file content, demonstrating various patterns for ignoring paths within a Feast feature repository. It shows how to ignore a directory, a specific file, files matching a pattern in a specific directory, and files matching a pattern recursively within subdirectories. ```text # Ignore virtual environment venv # Ignore a specific Python file scripts/foo.py # Ignore all Python files directly under scripts directory scripts/*.py # Ignore all "foo.py" anywhere under scripts directory scripts/**/foo.py ``` -------------------------------- ### Installing Feast Helm Chart (Bash) Source: https://github.com/discord/feast/blob/discord/infra/charts/feast/README.md Installs the Feast Helm chart from the added repository with a release name 'feast-release'. This command deploys the default Feast components onto your Kubernetes cluster. Requires the Helm CLI and a configured Kubernetes cluster. ```bash helm install feast-release feast-charts/feast ``` -------------------------------- ### Creating/Activating Python Virtual Environment (Shell) Source: https://github.com/discord/feast/blob/discord/docs/project/development-guide.md Creates a new Python virtual environment named 'venv' in the current directory and then activates it. This isolates project dependencies from the system Python installation. Requires Python 3.7+ installed. ```sh python -m venv venv/ source venv/bin/activate ``` -------------------------------- ### Installing Dev Feast Feature Server Helm Chart (Dev) Source: https://github.com/discord/feast/blob/discord/examples/python-helm-demo/README.md This command (for developers) installs the Feast feature server chart from a local path, specifically setting the image tag to 'dev'. This deploys the custom-built development image instead of a standard release image. ```Bash helm install feast-release ../../../infra/charts/feast-feature-server --set image.tag=dev --set feature_store_yaml_base64=$(base64 feature_store.yaml) ``` -------------------------------- ### Installing Feast Python Helm Chart with Overrides - Helm/Shell Source: https://github.com/discord/feast/blob/discord/docs/project/development-guide.md This command installs the Feast Python feature server Helm chart locally. It sets the image tag to 'dev' and base64 encodes the `feature_store.yaml` content to be passed as a Helm value. ```Shell helm install feast-release ../../../infra/charts/feast-feature-server --set image.tag=dev --set feature_store_yaml_base64=$(base64 feature_store.yaml) ``` -------------------------------- ### Installing Feast UI Peer Dependencies - Bash Source: https://github.com/discord/feast/blob/discord/docs/reference/alpha-web-ui.md Installs necessary peer dependencies required by the `@feast-dev/feast-ui` package using Yarn. These include various libraries like Elastic UI components, routing, data fetching, and typing libraries essential for the Feast UI component to function correctly within the React app. Requires Yarn and the project created in the previous step. ```Bash yarn add @elastic/eui @elastic/datemath @emotion/react moment prop-types inter-ui react-query react-router-dom use-query-params zod typescript query-string d3 @types/d3 ``` -------------------------------- ### Get Online Features for Inference - Python Source: https://github.com/discord/feast/blob/discord/README.md Shows how to retrieve the latest feature values for specific entities from the online store using the Feast Python SDK. This allows for low-latency access to features required for real-time model inference. ```python from pprint import pprint from feast import FeatureStore store = FeatureStore(repo_path=".") feature_vector = store.get_online_features( features=[ 'driver_hourly_stats:conv_rate', 'driver_hourly_stats:acc_rate', 'driver_hourly_stats:avg_daily_trips' ], entity_rows=[{"driver_id": 1001}] ).to_dict() pprint(feature_vector) # Make prediction # model.predict(feature_vector) ``` -------------------------------- ### Package Feast Serving JAR Bash Source: https://github.com/discord/feast/blob/discord/java/serving/README.md Packages an executable JAR file for the Feast Serving component using Maven, skipping tests. ```bash mvn -f java/serving/pom.xml package -Dmaven.test.skip=true ``` -------------------------------- ### Running Trino Offline Store Integration Tests (Shell) Source: https://github.com/discord/feast/blob/discord/docs/project/development-guide.md Executes integration tests specifically configured for the Trino offline store. This command is part of the contrib test suite and requires the development dependencies to be installed (`pip install -e ".[dev]"`). ```Shell make test-python-universal-trino ``` -------------------------------- ### Running Spark Offline Store Integration Tests (Shell) Source: https://github.com/discord/feast/blob/discord/docs/project/development-guide.md Executes integration tests specifically configured for the Spark offline store. This command is part of the contrib test suite and requires the development dependencies to be installed (`pip install -e ".[dev]"`). ```Shell make test-python-universal-spark ``` -------------------------------- ### Displaying Feast UI CLI Help - Bash Source: https://github.com/discord/feast/blob/discord/docs/reference/alpha-web-ui.md Shows the command-line options for the `feast ui` command. It lists parameters like host, port, and registry TTL, explaining how to configure the local UI server. Requires Feast CLI to be installed and run within a feature repository. ```Bash Usage: feast ui [OPTIONS] Shows the Feast UI over the current directory Options: -h, --host TEXT Specify a host for the server [default: 0.0.0.0] -p, --port INTEGER Specify a port for the server [default: 8888] -r, --registry_ttl_sec INTEGER Number of seconds after which the registry is refreshed. Default is 5 seconds. --help Show this message and exit. ``` -------------------------------- ### Bootstrapping React App with Create React App (Bash) Source: https://github.com/discord/feast/blob/discord/ui/README.md This command initializes a new React project directory using the Create React App tool, setting up the basic structure and dependencies required for a React application. ```bash npx create-react-app your-feast-ui ``` -------------------------------- ### Retrieve Azure AD Principal ID using Azure CLI - Bash Source: https://github.com/discord/feast/blob/discord/docs/tutorials/azure/README.md This snippet shows how to get the Azure Active Directory Principal ID (user object ID) of the signed-in user. This ID is required to set storage permissions for the Feast registry store during the Azure infrastructure deployment via the ARM template. ```bash # If you are using Azure portal CLI or Azure CLI 2.37.0 or above az ad signed-in-user show --query id -o tsv # If you are using Azure CLI below 2.37.0 az ad signed-in-user show --query objectId -o tsv ``` -------------------------------- ### Generating Protocol Buffer Files with Make Source: https://github.com/discord/feast/blob/discord/DISCORD_README.md Executes the `make protos` command, which triggers the project's protocol buffer compilation and generation process using the installed dependencies and tools. The `-i` flag instructs make to continue even if some commands fail. ```sh make protos -i ``` -------------------------------- ### Get Historical Features for Training - Python Source: https://github.com/discord/feast/blob/discord/README.md Demonstrates how to retrieve historical feature values using the Feast Python SDK for a given set of entities and event timestamps. It loads entity data into a pandas DataFrame and uses `get_historical_features` to join it with past feature values from the offline store, suitable for training machine learning models. ```python from feast import FeatureStore import pandas as pd from datetime import datetime entity_df = pd.DataFrame.from_dict({ "driver_id": [1001, 1002, 1003, 1004], "event_timestamp": [ datetime(2021, 4, 12, 10, 59, 42), datetime(2021, 4, 12, 8, 12, 10), datetime(2021, 4, 12, 16, 40, 26), datetime(2021, 4, 12, 15, 1 , 12) ] }) store = FeatureStore(repo_path=".") training_df = store.get_historical_features( entity_df=entity_df, features = [ 'driver_hourly_stats:conv_rate', 'driver_hourly_stats:acc_rate', 'driver_hourly_stats:avg_daily_trips' ], ).to_df() print(training_df.head()) # Train model # model = ml.fit(training_df) ``` -------------------------------- ### Running Postgres Offline Store Integration Tests (Shell) Source: https://github.com/discord/feast/blob/discord/docs/project/development-guide.md Executes integration tests specifically configured for the Postgres offline store. This command is part of the contrib test suite and requires the development dependencies to be installed (`pip install -e ".[dev]"`). Note: The command in the source text appears to have a potential typo, missing 'make'. ```Shell test-python-universal-postgres-offline ``` -------------------------------- ### Initialize Feast Store and Apply Features (Python) Source: https://github.com/discord/feast/blob/discord/docs/tutorials/tutorials-overview/driver-stats-on-snowflake.md Initializes the Feast FeatureStore object pointing to the current repository directory. It then applies the feature definitions (e.g., 'driver' entity and 'driver_stats_fv' FeatureView) defined in 'driver_repo.py' to the feature store, registering them and creating necessary infrastructure like the online store table if it doesn't exist. ```python from datetime import datetime, timedelta import pandas as pd from driver_repo import driver, driver_stats_fv from feast import FeatureStore fs = FeatureStore(repo_path=".") fs.apply([driver, driver_stats_fv]) ``` -------------------------------- ### Initializing Feast Project for Hazelcast (Bash) Source: https://github.com/discord/feast/blob/discord/sdk/python/feast/infra/online_stores/contrib/hazelcast_online_store/README.md Use the Feast CLI init command with the `-t hazelcast` flag to quickly create a new feature store project directory pre-configured to use Hazelcast as the online store in its `feature_store.yaml` file. ```bash feast init FEATURE_STORE_NAME -t hazelcast ``` -------------------------------- ### Initializing Feast Repository Locally - Bash Source: https://github.com/discord/feast/blob/discord/docs/how-to-guides/feast-snowflake-gcp-aws/create-a-feature-repository.md This command initializes a new Feast feature repository in the current directory using the default local template. It creates a basic project structure with example feature definitions and configuration. ```bash feast init ``` -------------------------------- ### Granting Roles in Snowflake (SQL) Source: https://github.com/discord/feast/blob/discord/docs/project/development-guide.md Grants the ACCOUNTADMIN and SYSADMIN roles to a specified user in Snowflake. This is a necessary setup step to ensure the user running tests has sufficient privileges to create warehouses, databases, schemas, and other objects. ```SQL grant role accountadmin, sysadmin to user user2; ``` -------------------------------- ### Initializing Feast Feature Store with Cassandra Template - Shell Source: https://github.com/discord/feast/blob/discord/sdk/python/feast/infra/online_stores/contrib/cassandra_online_store/README.md This command initializes a new Feast feature store directory and template configuration files, specifically setting up the basic structure and populating the `feature_store.yaml` with settings relevant for a Cassandra or Astra DB online store type. It prompts the user for necessary connection details. ```Shell feast init FEATURE_STORE_NAME -t cassandra ``` -------------------------------- ### Bootstrapping React App for Feast UI - Bash Source: https://github.com/discord/feast/blob/discord/docs/reference/alpha-web-ui.md Uses `create-react-app` via `npx` to quickly set up a new React application directory named `your-feast-ui`. This is the initial step to integrate the Feast UI component into a custom React project. Requires Node.js and npm/npx. ```Bash npx create-react-app your-feast-ui ``` -------------------------------- ### Example Python Test Configuration for Custom Offline Store Source: https://github.com/discord/feast/blob/discord/docs/how-to-guides/customizing-feast/adding-a-new-offline-store.md Provides a template Python code snippet demonstrating how to define a custom `FULL_REPO_CONFIGS_MODULE` for testing a custom offline store against Feast's integration test suite. It shows how to register a custom `DataSourceCreator` with a corresponding offline store type for testing purposes. This example uses a PostgreSQL `DataSourceCreator` but serves as a structural guide for implementing one for a custom store. ```python # Should go in sdk/python/feast/infra/offline_stores/contrib/postgres_repo_configuration.py from feast.infra.offline_stores.contrib.postgres_offline_store.tests.data_source import ( PostgreSQLDataSourceCreator, ) AVAILABLE_OFFLINE_STORES = [("local", PostgreSQLDataSourceCreator)] ``` -------------------------------- ### Running Tests with Yarn Source: https://github.com/discord/feast/blob/discord/ui/CONTRIBUTING.md Executes the project's test suite using the configured test runner. This command typically starts in interactive watch mode, allowing developers to quickly re-run tests as code is modified. ```shell yarn test ``` -------------------------------- ### Enabling Go Feature Server in Feast Configuration (YAML) Source: https://github.com/discord/feast/blob/discord/docs/reference/feature-servers/go-feature-server.md This YAML snippet shows the basic configuration required in `feature_store.yaml` to enable the Go feature server. It includes standard Feast configuration like project, registry, provider, and online store details, along with the crucial `go_feature_serving: True` flag. ```yaml project: my_feature_repo registry: data/registry.db provider: local online_store: type: redis connection_string: "localhost:6379" go_feature_serving: True ``` -------------------------------- ### Initializing Feast Repository Shell Source: https://github.com/discord/feast/blob/discord/sdk/python/feast/infra/online_stores/contrib/hbase_online_store/README.md This snippet initializes a new Feast feature store repository named `feature_repo` in the current directory using the `feast init` command. It then navigates into the newly created repository directory using the `cd` command, preparing the environment for further configuration and definition. ```shell feast init feature_repo cd feature_repo ``` -------------------------------- ### Initializing Feast Repository - Shell Source: https://github.com/discord/feast/blob/discord/sdk/python/feast/infra/online_stores/contrib/mysql_online_store/README.md This command initializes a new Feast feature repository directory structure. It's the first step to creating a feature store project. The `cd` command then moves into the newly created directory. ```shell feast init feature_repo cd feature_repo ``` -------------------------------- ### Materialize Features to Online Store Incrementally Source: https://github.com/discord/feast/blob/discord/README.md Calculates the current time and uses it as the end time to trigger the incremental materialization of feature values into the online store. This command makes the latest features available for low-latency online lookups. ```commandline CURRENT_TIME=$(date -u +"%Y-%m-%dT%H:%M:%S") feast materialize-incremental $CURRENT_TIME ``` -------------------------------- ### Deploying Feast Feature Server (Shell) Source: https://github.com/discord/feast/blob/discord/sdk/python/feast/templates/local/README.md Deploys instances of the Feast feature server. These servers expose endpoints for retrieving online features, enabling low-latency feature lookup for model serving or real-time applications. ```shell feast serve ``` -------------------------------- ### Viewing Feast Project Directory Structure - Bash Source: https://github.com/discord/feast/blob/discord/docs/how-to-guides/feast-snowflake-gcp-aws/create-a-feature-repository.md This command shows the typical directory structure created by the `feast init` command. It includes a data directory, an example Python file for feature definitions, and the main Feast configuration file. ```bash $ tree . └── tiny_pika ├── data │ └── driver_stats.parquet ├── example.py └── feature_store.yaml 1 directory, 3 files ```