### Install OpenAPI Generator Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/functions-python/operations_api/README.md Installs the OpenAPI Generator tool. This is a one-time setup step. ```bash ./scripts/setup-openapi-generator.sh ``` -------------------------------- ### Start Pub/Sub and Datastore Emulators Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/functions-python/batch_datasets/README.md Commands to start the local emulators for Google Cloud Pub/Sub and Datastore. Ensure these are running before starting the function locally. ```bash gcloud beta emulators pubsub start --project=project-id --host-port='localhost:8043' gcloud beta emulators datastore start --project=project-id --host-port='localhost:8044' ``` -------------------------------- ### Install API Dependencies Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/README.md Install project dependencies for the API. Ensure you are in the 'api' directory. ```bash cd api pip3 install -r requirements.txt pip3 install -r requirements_dev.txt ``` -------------------------------- ### Install and Run Pre-commit Hooks Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/README.md Installs and runs pre-commit hooks for code styling and linting. Ensure pre-commit is installed via requirements_dev.txt. ```bash pre-commit install pre-commit run --all-files ``` -------------------------------- ### Install Pub/Sub Emulator Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/functions-python/batch_datasets/README.md Command to install the Google Cloud Pub/Sub emulator. This is required for local testing of Pub/Sub interactions. ```bash gcloud components install pubsub-emulator ``` -------------------------------- ### Install Python Dependencies Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/integration-tests/README.md Install the required Python libraries for the integration tests using pip. Ensure you have Python 3.x and pip installed. ```bash pip install -r requirements.txt ``` -------------------------------- ### Example Integration Test Execution Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/integration-tests/README.md An example command to run integration tests against a local API URL and a specified data file, including tests from specific classes. Ensure the REFRESH_TOKEN is exported. ```bash ./scripts/integration-tests.sh -u "http://0.0.0.0:8080" -f "/path/to/your/data_file.csv" -c "ClassName1,ClassName2" ``` -------------------------------- ### Setup All Function Environments Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/functions-python/README.md Sets up the development environment for all functions in the project simultaneously. This is useful for comprehensive local testing. ```bash ./scripts/function-python-setup.sh --all ``` -------------------------------- ### Start Local Test Database Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/functions-python/operations_api/README.md Starts the local test database using Docker Compose. Run this command from the repository root. ```bash docker compose --env-file ./config/.env.local up -d liquibase-test ``` -------------------------------- ### Install Locust Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/load-test/README.md Install the Locust library using pip. This is a prerequisite for running the load tests. ```bash pip install locust ``` -------------------------------- ### Start Batch Datasets Function Locally Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/functions-python/batch_datasets/README.md Command to run the batch_datasets function locally using the provided script. This is part of the local development setup. ```bash ./scripts/function-python-run.sh --function_name batch_datasets ``` -------------------------------- ### Basic Logging Example Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/docs/Logging.md Demonstrates the basic usage of the logging library to log an informational message. Assumes the logging context is already initialized. ```python import logging logging.info("This message is awesome") ``` -------------------------------- ### Start Load Test Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/load-test/README.md Run a load test against the QA environment. Adjust the number of users (-u) and hatch rate (-r) as needed. ```bash locust -f gtfs_user_test.py --host=https://api-qa.mobilitydatabase.org -u 100 -r 10 ``` -------------------------------- ### Setup Function Environment Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/functions-python/README.md Sets up a self-contained environment for a specific function by creating shared folders with symbolic links to necessary packages. Use this to run functions locally. ```bash ./scripts/function-python-setup.sh --function_name batch_datasets ``` -------------------------------- ### Example Pub/Sub Message for Batch Processing Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/functions-python/batch_process_dataset/README.md An illustrative example of a Pub/Sub message payload for the batch-process-dataset function. This shows sample values for each field. ```json { "message": { "data": { "execution_id": "JLU_20250721A", "producer_url": "http://api.511.org/transit/datafeeds?operator_id=CE", "feed_stable_id": "mdb-2684", "feed_id": "2f5d7b4e-bb9b-49ae-a011-b61d7d9b53ff", "dataset_id": null, "dataset_hash": null, "authentication_type": "1", "authentication_info_url": "https://511.org/open-data/token", "api_key_parameter_name": "api_key" } } } ``` -------------------------------- ### Start Local Database with Docker Compose Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/README.md Generates an instance of the database locally using docker-compose. Ensure the .env.local file is present. ```bash docker-compose --env-file ./config/.env.local up -d --force-recreate ``` -------------------------------- ### Run Local API Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/README.md Starts the local API server. This command is also used to access the API's swagger documentation. ```bash scripts/api-start.sh ``` -------------------------------- ### Download OpenAPI Generator Script Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/README.md Download the openapi-generator-cli.sh script for one-time setup. ```bash scripts/setup-openapi-generator.sh ``` -------------------------------- ### Install yq on macOS Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/functions-python/operations_api/README.md Installs yq version 4+ on macOS using Homebrew. This is a prerequisite for schema synchronization scripts. ```bash brew install yq ``` -------------------------------- ### Deploy All Functions to Firebase Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/functions/README.md Use this command to deploy all functions to Firebase. Ensure you have Firebase CLI installed and configured. ```shell firebase deploy --only functions ``` -------------------------------- ### Response Example for Check GTFS Feed Availability Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/functions-python/tasks_executor/README.md This is an example JSON response from the 'check_gtfs_feed_availability' task. It includes summary statistics, elapsed time, and a detailed 'failures' list when verbose mode is enabled. ```json { "message": "Checked 3 feed(s): 2 succeeded, 1 failed.", "total_feeds": 3, "succeeded": 2, "failed": 1, "skip_db_update": false, "elapsed_seconds": 4.21, "failures": [ { "stable_id": "mdb-123", "error_type": "ConnectionError", "reason": "Max retries exceeded", "content_type": null, "is_zip": null } ] } ``` -------------------------------- ### Example Payload for Populate Licenses Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/functions-python/tasks_executor/README.md This JSON payload is used to populate license information. It includes a dry run option. ```json { "task": "populate_licenses", "payload": { "dry_run": true } } ``` -------------------------------- ### Example Payload for Update GeoJSON Files Task Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/functions-python/tasks_executor/src/tasks/geojson/README.md An example JSON payload demonstrating how to invoke the update_geojson_files task with specific parameters. This example sets dry_run to true, specifies 'gtfs' data type, and limits processing to 10 feeds. ```json { "dry_run": true, "data_type": "gtfs", "limit": 10 } ``` -------------------------------- ### Example GTFS Dataset PMTiles Builder Request Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/functions-python/pmtiles_builder/README.md An example of a valid request payload for the PMTiles builder function, demonstrating the format for feed and dataset stable IDs. ```json { "feed_stable_id": "mdb-1004", "dataset_stable_id": "mdb-1004-202507081807" } ``` -------------------------------- ### Example Payload for Get Validation Run Status Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/functions-python/tasks_executor/README.md This JSON payload is used to retrieve the status of a specific validation run. It requires the task name and the run ID. ```json { "task": "get_validation_run_status", "payload": { "task_name": "gtfs_validation", "run_id": "7.1.1-SNAPSHOT" } } ``` -------------------------------- ### Example Payload with Date Filter Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/functions-python/tasks_executor/src/tasks/dataset_files/README.md An example of the task payload specifying 'dry_run' as false and a date filter to process datasets downloaded after July 1, 2025. 'latest_only' is also set to true. ```json { "dry_run": false, "after_date": "2025-07-01", "latest_only": true } ``` -------------------------------- ### Example Payload with Specific Dataset ID Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/functions-python/tasks_executor/src/tasks/dataset_files/README.md An example of the task payload specifying 'dry_run' as false and a specific 'dataset_id'. This will process only the dataset with the ID 'mdb-1147-202407031702'. ```json { "dry_run": false, "dataset_id": "mdb-1147-202407031702" } ``` -------------------------------- ### Apply Terraform for Artifact Registry Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/docs/GCP.md Apply Terraform configuration specifically for the artifact registry module. This is a one-time setup step before deploying services. ```shell terraform apply -var-file=vars-.tfvars -target=module.artifact-registry ``` -------------------------------- ### Clean Function Environment Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/functions-python/README.md Removes the 'shared' and 'test_shared' folders created during the setup process for a specific function. Use this to revert changes or clean up. ```bash ./scripts/function-python-setup.sh --function_name batch_datasets --clean ``` -------------------------------- ### Run Unit Tests for a Specific Function Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/functions-python/README.md Executes unit tests for a specified function folder using the api-tests.sh script. This script also sets up the function environment and installs dependencies. ```bash scripts/api-tests.sh --folder functions-python/batch_datasets ``` -------------------------------- ### Execute Function Locally Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/functions-python/README.md Runs a specific Python function locally using the function-framework. This command creates a virtual environment with function-specific requirements installed. ```bash ./scripts/function-python-run.sh --function_name batch_datasets ``` -------------------------------- ### Example Payload for Refresh Materialized View Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/functions-python/tasks_executor/README.md This JSON payload is used to trigger the 'refresh_materialized_view' task. It includes a dry run option. ```json { "task": "refresh_materialized_view", "payload": { "dry_run": true } } ``` -------------------------------- ### Example Payload for Rebuild Missing Bounding Boxes Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/functions-python/tasks_executor/README.md This JSON payload is used to trigger the 'rebuild_missing_bounding_boxes' task. It supports a dry run option and filtering by date. ```json { "task": "rebuild_missing_bounding_boxes", "payload": { "dry_run": true, "after_date": "2025-06-01" } } ``` -------------------------------- ### Example Input Payload for Reverse Geolocation Populate Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/functions-python/reverse_geolocation_populate/README.md This JSON payload demonstrates the expected input for the Reverse Geolocation Populate Cloud Function. It includes the required 'country_code' and an optional 'admin_levels' parameter. ```json { "country_code": "FR", "admin_levels": "2,4,6" } ``` -------------------------------- ### Log Messages Without Stable ID Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/docs/Logging.md Use the standard logging library to log messages. This example shows logging without associating a specific stable ID. ```python import logging logging.info("Total feeds: %s", total_feeds) ``` -------------------------------- ### Example Payload for Backfill Dataset Hash MD5 Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/functions-python/tasks_executor/README.md This JSON payload is used to backfill MD5 hashes for existing GTFS datasets. It supports dry run, processing only the latest datasets, only missing hashes, and limiting the number of datasets processed. ```json { "task": "backfill_dataset_hash_md5", "payload": { "dry_run": true, "only_latest": true, "only_missing_hashes": true, "limit": 10 } } ``` -------------------------------- ### Example Payload for Rebuild Missing Validation Reports Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/functions-python/tasks_executor/README.md This JSON payload is used to trigger the 'rebuild_missing_validation_reports' task. It includes options for dry run, bypassing database updates, filtering by days, forcing updates, specifying a validator endpoint, limiting results, and filtering by status. ```json { "task": "rebuild_missing_validation_reports", "payload": { "dry_run": true, "bypass_db_update": true, "filter_after_in_days": null, "force_update": false, "validator_endpoint": "https://stg-gtfs-validator-web-mbzoxaljzq-ue.a.run.app", "limit": 1, "filter_statuses": ["active", "inactive", "future"] } } ``` -------------------------------- ### Example Payload for Check GTFS Feed Availability Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/functions-python/tasks_executor/README.md This JSON payload is used to check the availability of non-deprecated published GTFS feeds via HTTP HEAD requests. It includes options for dry run, skipping database updates, limiting feeds, setting concurrency, timeout, batch size, stable feed IDs, verbosity, and fallback to GET requests. ```json { "task": "check_gtfs_feed_availability", "payload": { "dry_run": true, "skip_db_update": false, "limit": null, "concurrency": 15, "timeout_seconds": 10, "batch_size": 50, "stable_feed_ids": null, "verbose": false, "fallback_to_get": true } } ``` -------------------------------- ### Create GCP Project Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/docs/GCP.md Use this command to create a new Google Cloud Platform project with a specified name. ```shell gcloud projects create PROJECT_ID --name="Mobility Feeds API" ``` -------------------------------- ### Reset Local Database and Populate with Catalog Content Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/README.md Resets the local database and populates it with catalog content. Requires the wget command to be available. ```bash ./scripts/docker-localdb-rebuild-data.sh --populate-db ``` -------------------------------- ### Initialize Terraform Backend Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/docs/GCP.md Initialize the Terraform backend configuration for a specific environment. Ensure you are in the correct directory and have a backend configuration file. ```shell terraform init -backend-config=backend-.conf ``` -------------------------------- ### Execute Python Script to Create Feed List Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/scripts/validator-acceptance-feed-list/README.md Run the Python script to generate the acceptance test feed list. Ensure you have configured the .env file with database credentials and potentially set up a database tunnel. ```bash cd scripts/validator-acceptance-feed-list cp ../../config/.env.local .env # Edit .env to point to the DB you want to query python create_acceptance_list.py --env-file .env ``` -------------------------------- ### Generate API and Database Stubs Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/README.md Generates the API and database stubs. Run this on the first run and whenever the schema changes. ```bash scripts/api-gen.sh ``` ```bash scripts/db-gen.sh ``` -------------------------------- ### Run Integration Tests with Options Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/integration-tests/README.md Execute the integration tests using the provided bash script. Specify the API URL, data file path, and optionally filter by test class names. The REFRESH_TOKEN environment variable must be set. ```bash export REFRESH_TOKEN="your_refresh_token" ./scripts/integration-tests.sh -u -f [-c ] ``` -------------------------------- ### Create Distribution Zip for All Functions Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/functions-python/README.md Builds distribution zip files for all functions in the project. This is useful for preparing multiple functions for deployment. ```bash ./scripts/function-python-build.sh --all ``` -------------------------------- ### Run API Server with Docker Compose Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/README.md Builds and runs the API server on a Docker container. Ensure OpenAPI generated files are present by running api-gen.sh first. ```bash (cd api && docker-compose up --build) ``` -------------------------------- ### Run All Tests Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/README.md Executes all tests in the project. Ensure a local test DB instance is available and empty. ```bash scripts/api-tests.sh ``` -------------------------------- ### Set GCP Project Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/docs/GCP.md Configure the local gcloud CLI to point to the newly created GCP project. ```shell gcloud config set project PROJECT_ID ``` -------------------------------- ### Initialize Logging Client Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/docs/Logging.md Call the `init_logger()` function in your main file to set the appropriate logging level and initialize the GCP cloud client, especially when not running in a local environment. ```python init_logger() ``` -------------------------------- ### API Error Response Structure Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/README.md Example of a standard API HTTP error response following the FastAPI structure. Use helper functions for standardization. ```json { "details": "The error message" } ``` -------------------------------- ### Get Validation Run Status Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/functions-python/tasks_executor/src/tasks/validation_reports/README.md This JSON payload is used to retrieve the status of a specific GTFS validation run, with an option to synchronize the workflow status. ```json { "task": "get_validation_run_status", "payload": { "validator_version": "7.0.0", "sync_workflow_status": true } } ``` -------------------------------- ### Create Distribution Zip Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/functions-python/README.md Locally creates a distribution zip file for a specific function. This zip is used for deployment and contains all necessary files. ```bash ./scripts/function-python-build.sh --function_name batch_datasets ``` -------------------------------- ### Import from Test Shared Modules Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/functions-python/README.md Shows how to import modules from the 'test_shared' directory for testing purposes. This requires the test environment to be set up. ```python from test_shared.test_utils.database_utils import get_testing_session, default_db_url ``` -------------------------------- ### Run Single Test File Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/README.md Executes a single test file. Replace .py with the actual test file name. ```bash scripts/api-tests.sh .py ``` -------------------------------- ### Run Unit Tests with HTML Coverage Report Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/functions-python/README.md Executes unit tests for all functions and generates an HTML coverage report. The reports are saved in a dedicated folder per function. ```bash scripts/api-tests.sh --folder functions-python --html_report ``` -------------------------------- ### Move index.html to root Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/docs/SwaggerUI/README.md Move the index.html file from the dist directory to the root of your repository. ```bash mv dist/index.html . ``` -------------------------------- ### Test Access Token with Curl Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/README.md Verify your access token by making a request to the metadata endpoint. Replace `[Your Access Token]` with your generated access token. ```bash curl --location 'https://api.mobilitydatabase.org/v1/metadata' \ --header 'Accept: application/json' \ --header 'Authorization: Bearer [Your Access Token]' ``` -------------------------------- ### Example Payload for Update GeoJSON Files Precision Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/functions-python/tasks_executor/README.md This JSON payload is used to update the precision of GeoJSON files. It includes options for dry run, data type, precision level, and a limit on the number of files to process. ```json { "task": "update_geojson_files_precision", "payload": { "dry_run": true, "data_type": "gtfs", "precision": 5, "limit": 10 } } ``` -------------------------------- ### Generate Access Token using Curl Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/README.md Use this command to generate an access token by providing your refresh token. Replace `[Your Refresh Token]` with the actual token obtained after registration. ```bash curl --location 'https://api.mobilitydatabase.org/v1/tokens' \ --header 'Content-Type: application/json' \ --data '{ "refresh_token": "[Your Refresh Token]" }' ``` -------------------------------- ### Reset Local Database and Populate with Test Data Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/README.md Resets the local database and populates it with test data. Use this to test or debug features requiring dataset entities. ```bash ./scripts/docker-localdb-rebuild-data.sh --populate-db --populate-test-data ``` -------------------------------- ### Import from Shared Modules Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/functions-python/README.md Demonstrates how to import modules from the 'shared' directory within your Python function code. Ensure the environment is set up using function-python-setup.sh. ```python from shared.database_gen.sqlacodegen_models import Gtfsfeed, Gtfsdataset ``` -------------------------------- ### Run API Tests Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/functions-python/operations_api/README.md Executes API tests for the operations_api folder using the repository's test runner. This is the recommended way to run tests. ```bash ./scripts/api-tests.sh --folder functions-python/operations_api ``` -------------------------------- ### Run Operations API Locally Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/functions-python/operations_api/README.md Use this command to locally execute the operations_api function. Ensure you are in the repository root. ```bash ./scripts/function-python-run.sh --function_name operations_api ``` -------------------------------- ### Run API Tests for Batch Datasets Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/functions-python/batch_datasets/README.md Command to execute the automated API tests for the batch_datasets function. This is used to verify the function's integration and correctness. ```bash ./scripts/api-tests.sh --folder functions-python/batch_datasets ``` -------------------------------- ### Login to gcloud CLI Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/docs/GCP.md Authenticate your local environment to Google Cloud Platform using application default credentials. ```shell gcloud auth application-default login ``` -------------------------------- ### Build and Push Docker Image Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/docs/GCP.md Build and push a Docker image for the feed-api service to the GCP artifact registry. Ensure the version number is set in the tfvars file. ```shell /scripts/docker-build-push.sh -project_id mobility-feeds- -service feed-api -repo_name feeds- -region northamerica-northeast1 -version ``` -------------------------------- ### Plan Terraform Changes Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/docs/GCP.md Generate and review a Terraform execution plan for a given environment. This step allows you to see the proposed changes before applying them. ```shell terraform plan -var-file=vars-.tfvars ``` -------------------------------- ### Build Operations API Distribution Zip Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/functions-python/operations_api/README.md Use this command to locally create a distribution zip artifact for the operations_api function. Ensure you are in the repository root. ```bash ./scripts/function-python-build.sh --function_name operations_api ``` -------------------------------- ### Deploy Specific Function Codebase to Firebase Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/functions/README.md Use this command to deploy a specific function identified by its codebase. Replace with the actual codebase name. ```shell firebase deploy --only functions: ``` -------------------------------- ### Update Local Database Schema and Documentation Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/README.md Updates the local database instance and related schema documentation when the database schema is modified. This command is used in conjunction with schema changes. ```bash docker-compose --env-file ./config/.env.local up schemaspy -d --force-recreate ``` -------------------------------- ### Configure Swagger UI URL Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/docs/SwaggerUI/README.md Edit the swagger-initializer.js file to set the 'url' property to your local YAML API specification file. ```javascript window.ui = SwaggerUIBundle({ url: "swagger.yaml", ... ``` -------------------------------- ### Reset Local Database and Apply Liquibase Changes Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/README.md Resets the local database and applies Liquibase changes. This is useful when switching branches with different DB schema definitions. Warning: All data within the database will be lost. ```bash ./scripts/populate-db.sh ``` -------------------------------- ### Update Swagger UI file references Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/docs/SwaggerUI/README.md Adjust references in your HTML to point to the correct paths for Swagger UI CSS and JavaScript files within the 'dist' directory. ```html ... ... ... ``` -------------------------------- ### Run Unit Tests for All Functions Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/functions-python/README.md Executes unit tests for all functions within the functions-python directory. This provides a comprehensive test run for the entire module. ```bash scripts/api-tests.sh --folder functions-python ``` -------------------------------- ### Set Local Environment Variables Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/functions-python/README.md Defines local environment variables by creating or modifying a .env.local file within a function's directory. This allows for custom configurations. ```bash export MY_AWESOME_KEY=MY_AWESOME_VALUE ``` -------------------------------- ### Reset Local Test Database Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/README.md Resets the local test database to an empty state. Run this if you have existing data in your local test DB before running tests. ```bash ./scripts/docker-localdb-rebuild-data.sh --use-test-db ``` -------------------------------- ### Execute Function Locally Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/functions-python/batch_datasets/README.md Command to send a request to the locally running batch_datasets function. This is used to test the function's behavior. ```bash curl http://localhost:8080 ``` -------------------------------- ### Payload to List Supported Tasks Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/functions-python/tasks_executor/README.md This JSON payload is used to request a list of all supported tasks from the Tasks Executor. It requires a 'list_tasks' task name with an empty payload. ```json { "name": "list_tasks", "payload": {} } ``` -------------------------------- ### Update OperationsAPI Schema Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/functions-python/operations_api/README.md Updates OperationsAPI OpenAPI components with Mobility Database Catalog API components. Run this command from the repository root. ```bash ./scripts/api-operations-update-schema.sh ``` -------------------------------- ### End-to-End Test with Limited Batch Size Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/functions-python/tasks_executor/src/tasks/validation_reports/README.md Execute a test run with a limited number of datasets using 'rebuild_missing_validation_reports'. This is useful for end-to-end testing before a full deployment. ```json { "task": "rebuild_missing_validation_reports", "payload": { "dry_run": false, "validator_endpoint": "https://stg-gtfs-validator-web-mbzoxaljzq-ue.a.run.app", "bypass_db_update": true, "limit": 10 } } ``` -------------------------------- ### Pub/Sub Message Format for Batch Processing Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/functions-python/batch_process_dataset/README.md Defines the expected structure of a Pub/Sub message for triggering the batch-process-dataset function. Ensure your messages adhere to this format. ```json { "message": { "data": { "execution_id": "execution_id", "producer_url": "producer_url", "feed_stable_id": "feed_stable_id", "feed_id": "feed_id", "dataset_id": "dataset_id", "dataset_hash": "dataset_hash", "authentication_type": "authentication_type", "authentication_info_url": "authentication_info_url", "api_key_parameter_name": "api_key_parameter_name" } } } ``` -------------------------------- ### Full GTFS Validation Run Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/functions-python/tasks_executor/src/tasks/validation_reports/README.md Initiate a full GTFS validation run by calling 'rebuild_missing_validation_reports' without a limit. The system automatically skips already-triggered datasets. ```json { "task": "rebuild_missing_validation_reports", "payload": { "dry_run": false, "validator_endpoint": "https://stg-gtfs-validator-web-mbzoxaljzq-ue.a.run.app", "bypass_db_update": true } } ``` -------------------------------- ### Run Linter Checks Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/README.md Executes linter checks using Flake8 and Black for code styling. This script ensures code quality and consistency. ```bash scripts/lint-tests.sh ``` -------------------------------- ### Clean All Function Environments Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/functions-python/README.md Removes the 'shared' and 'test_shared' folders for all functions in the project. This is a comprehensive cleanup command. ```bash ./scripts/function-python-setup.sh --all --clean ``` -------------------------------- ### Apply Terraform Changes Source: https://github.com/mobilitydata/mobility-feed-api/blob/main/docs/GCP.md Apply the planned Terraform changes to the GCP environment. This command requires Project IAM Admin role for the terraform-init project. ```shell terraform apply -var-file=vars-.tfvars ```