### Local Project Installation with setup.py develop Source: https://faust-streaming.github.io/faust/_sources/userguide/application Demonstrates the command-line usage of 'python setup.py develop' to install a project locally in development mode, allowing for immediate use of installed scripts and easy source code modifications. ```console $ python setup.py develop ``` -------------------------------- ### Start Faust Worker for Agent Example Source: https://faust-streaming.github.io/faust/userguide/agents Command to start a Faust worker specifically for the agent example defined in `examples.agent`. It uses the `-A` flag to specify the Python module containing the Faust application and `worker -l info` to launch the worker with informational logging. This assumes the `examples/agent.py` file is part of an installable package named `examples`. ```bash $ faust -A examples.agent worker -l info ``` -------------------------------- ### Get Faust Worker Help (Console) Source: https://faust-streaming.github.io/faust/_sources/playbooks/quickstart This command displays all available command-line options for the Faust worker, useful for understanding configuration and runtime parameters. ```console faust worker --help ``` -------------------------------- ### Install Faust using pip Source: https://faust-streaming.github.io/faust/userguide/installation This command installs the latest version of the Faust library using pip. It's the most straightforward method for getting started with Faust. ```bash $ pip install -U faust-streaming ``` -------------------------------- ### Run Faust Worker (Console) Source: https://faust-streaming.github.io/faust/_sources/playbooks/quickstart This command starts a Faust worker for the 'hello_world' application with informational logging enabled. Multiple instances can be run to distribute processing. ```console faust -A hello_world worker -l info ``` -------------------------------- ### Faust Worker Help Command Example Source: https://faust-streaming.github.io/faust/_sources/userguide/cli This example shows how to display the help message for the Faust worker command, which lists all available options for starting and configuring a worker instance. It's useful for understanding the full range of configurable parameters. ```console $ python examples/word_count.py worker --help Usage: word_count.py worker [OPTIONS] Start ƒaust worker instance. Options: -f, --logfile PATH Path to logfile (default is ). -l, --loglevel [crit|error|warn|info|debug] Logging level to use. --blocking-timeout FLOAT Blocking detector timeout (requires --debug). -p, --web-port RANGE[1-65535] Port to run web server on. -b, --web-bind TEXT -h, --web-host TEXT Canonical host name for the web server. --console-port RANGE[1-65535] (when --debug:) Port to run debugger console on. --help Show this message and exit. ``` -------------------------------- ### Faust Worker Startup Example (Info Logging) Source: https://faust-streaming.github.io/faust/_sources/userguide/cli This example shows how to start a Faust worker with increased logging verbosity to 'info' level. The output includes more detailed information about the worker's startup process, including web server and monitor initialization. ```console $ python examples/word_count.py worker -l info ┌ƒaµS† v1.0.0──────────────────────────────────────────┐ │ id │ word-counts │ │ transport │ kafka://localhost:9092 │ │ store │ rocksdb: │ │ web │ http://localhost:6066/ │ │ log │ -stderr- (info) │ │ pid │ 46034 │ │ hostname │ grainstate.local │ │ platform │ CPython 3.6.4 (Darwin x86_64) │ │ drivers │ aiokafka=0.4.0 aiohttp=3.0.8 │ │ datadir │ /opt/devel/faust/word-counts-data │ │ appdir │ /opt/devel/faust/word-counts-data/v1 │ └───────────┴──────────────────────────────────────────┘ starting^[2018-03-13 13:41:39,269: INFO]: [^Worker]: Starting... [2018-03-13 13:41:39,275: INFO]: [^-App]: Starting... [2018-03-13 13:41:39,271: INFO]: [^--Web]: Starting... [2018-03-13 13:41:39,272: INFO]: [^---ServerThread]: Starting... [2018-03-13 13:41:39,273: INFO]: [^--Web]: Serving on http://localhost:6066/ [2018-03-13 13:41:39,275: INFO]: [^--Monitor]: Starting... [2018-03-13 13:41:39,275: INFO]: [^--Producer]: Starting... [2018-03-13 13:41:39,317: INFO]: [^--Consumer]: Starting... [2018-03-13 13:41:39,325: INFO]: [^--LeaderAssignor]: Starting... [2018-03-13 13:41:39,325: INFO]: [^--Producer]: Creating topic word-counts-__assignor-__leader [2018-03-13 13:41:39,325: INFO]: [^--Producer]: Nodes: [0] [2018-03-13 13:41:39,668: INFO]: [^--Producer]: Topic word-counts-__assignor-__leader created. [2018-03-13 13:41:39,669: INFO]: [^--ReplyConsumer]: Starting... [2018-03-13 13:41:39,669: INFO]: [^--Agent]: Starting... [2018-03-13 13:41:39,673: INFO]: [^---OneForOneSupervisor]: Starting... [2018-03-13 13:41:39,673: INFO]: [^---Agent*: examples.word_co[.]shuffle_words]: Starting... [2018-03-13 13:41:39,673: INFO]: [^--Agent]: Starting... [2018-03-13 13:41:39,674: INFO]: [^---OneForOneSupervisor]: Starting... [2018-03-13 13:41:39,674: INFO]: [^---Agent*: examples.word_count.count_words]: Starting... [2018-03-13 13:41:39,674: INFO]: [^--Conductor]: Starting... [2018-03-13 13:41:39,674: INFO]: [^--TableManager]: Starting... [2018-03-13 13:41:39,675: INFO]: [^--Stream: <(*)Topic: posts@0x10497e5f8>]: Starting... [2018-03-13 13:41:39,675: INFO]: [^--Stream: <(*)Topic: wo...s@0x105f73b38>]: Starting... [...] ``` -------------------------------- ### Start Zookeeper Server Source: https://faust-streaming.github.io/faust/_sources/playbooks/leaderelection This console command starts the Zookeeper server, a prerequisite for running Kafka. It requires Kafka to be installed and the KAFKA_HOME environment variable to be set. ```console $KAFKA_HOME/bin/zookeeper-server-start $KAFKA_HOME/etc/kafka/zookeeper.properties ``` -------------------------------- ### Start Faust Application with app.main() Source: https://faust-streaming.github.io/faust/_sources/userguide/application This Python snippet shows how to initialize and start a Faust application using `app.main()`. This method integrates the script with the Faust command-line interface, allowing it to accept standard Faust arguments. ```python # examples/command.py import faust app = faust.App('umbrella-command-example') if __name__ == '__main__': app.main() ``` -------------------------------- ### Faust Development Environment Setup Source: https://faust-streaming.github.io/faust/_sources/contributing Commands to set up the Faust development environment. This includes installing dependencies and configuring Python to use the Faust development directory, ensuring proper project setup for coding and testing. ```console $ make develop $ make hooks ``` -------------------------------- ### Start Kafka Server Source: https://faust-streaming.github.io/faust/_sources/playbooks/leaderelection This console command starts the Kafka server, which is used as a message broker by Faust. It requires Kafka to be installed and the KAFKA_HOME environment variable to be set. ```console $KAFKA_HOME/bin/kafka-server-start $KAFKA_HOME/etc/kafka/server.properties ``` -------------------------------- ### Start Zookeeper Server Source: https://faust-streaming.github.io/faust/playbooks/leaderelection Command to start the Zookeeper server, a prerequisite for running Kafka. This command assumes the KAFKA_HOME environment variable is set to the Kafka installation directory. ```bash $KAFKA_HOME/bin/zookeeper-server-start $KAFKA_HOME/etc/kafka/zookeeper.properties ``` -------------------------------- ### Console command to run a custom Faust command Source: https://faust-streaming.github.io/faust/_sources/userguide/application This console command demonstrates how to execute a custom command defined within a Faust application. The example shows running the 'example' subcommand defined using the `@app.command()` decorator. ```console $ python examples/command.py example RUNNING EXAMPLE COMMAND ``` -------------------------------- ### Get Faust Worker Help Source: https://faust-streaming.github.io/faust/playbooks/quickstart Displays the help message for the Faust worker command, listing all available command-line options and configurations. ```bash faust worker --help ``` -------------------------------- ### Start Multiple Faust Workers Source: https://faust-streaming.github.io/faust/_sources/userguide/workers Demonstrates starting multiple Faust worker instances on the same machine, ensuring each has a unique data directory and web server port to avoid conflicts. ```console $ faust --datadir=/var/faust/worker1 -A proj -l info worker --web-port=6066 $ faust --datadir=/var/faust/worker2 -A proj -l info worker --web-port=6067 ``` -------------------------------- ### Install Faust from source code Source: https://faust-streaming.github.io/faust/userguide/installation This sequence of commands outlines the process of downloading, extracting, building, and installing Faust from its source tarball. This method is useful for development or when a specific version not available on PyPI is needed. Note that 'python setup.py install' may require elevated privileges. ```bash $ tar xvfz faust-0.0.0.tar.gz $ cd faust-0.0.0 $ python setup.py build # python setup.py install ``` -------------------------------- ### Define Faust Application (Python) Source: https://faust-streaming.github.io/faust/_sources/playbooks/quickstart This snippet defines a basic Faust application named 'hello-world' that connects to a local Kafka broker. It configures the value serializer to 'raw' and sets up a Kafka topic named 'greetings' with an agent to process and print incoming messages. ```python import faust app = faust.App( 'hello-world', broker='kafka://localhost:9092', value_serializer='raw', ) greetings_topic = app.topic('greetings') @app.agent(greetings_topic) async def greet(greetings): async for greeting in greetings: print(greeting) ``` -------------------------------- ### Start Faust Worker Instance (Python) Source: https://faust-streaming.github.io/faust/userguide/cli This example demonstrates starting a Faust worker instance for the `word_count.py` application. It shows the default output when no specific options are provided, including application details and startup messages. ```shell python examples/word_count.py worker ``` -------------------------------- ### Starting Faust App from Command Line Source: https://faust-streaming.github.io/faust/_sources/userguide/application This console command shows how to start a Faust worker instance from the command line. It uses the `faust` executable with the `-A` option to specify the application module and the `worker` subcommand. The `-l info` flag sets the logging level. ```console $ faust -A myproj worker -l info ``` -------------------------------- ### Start Faust Worker and Interact with Topics/Agents Source: https://context7.com/context7/faust-streaming_github_io_faust/llms.txt Provides various command-line examples for managing and interacting with Faust applications. This includes starting workers, sending messages to topics and agents, listing agents and tables, and resetting application state. ```bash # Start worker with app module faust -A myapp worker -l info # Start with multiple worker processes faust -A myapp worker -l info --web-port=6066 # Send messages to topic faust -A myapp send mytopic '{"key": "value"}' # Send to agent using @ prefix faust -A myapp send @myagent '{"data": "value"}' # List agents faust -A myapp agents # List tables faust -A myapp tables # Reset application state (delete tables) faust -A myapp reset # Clean old versions faust -A myapp clean-versions ``` -------------------------------- ### Faust Codec Extension: Setup for PyPI Package Source: https://faust-streaming.github.io/faust/userguide/models Provides the `setup.py` content for creating an installable Faust codec extension package. It includes package metadata, dependencies, and importantly, the `entry_points` configuration to register the custom codec with Faust under the 'faust.codecs' group, making it discoverable by Faust applications. ```python import setuptools from setuptools import find_packages setuptools.setup( name='faust-msgpack', version='1.0.0', description='Faust msgpack serialization support', author='Ola A. Normann', author_email='ola@normann.no', url='http://github.com/example/faust-msgpack', platforms=['any'], license='BSD', packages=find_packages(exclude=['ez_setup', 'tests', 'tests.*']), zip_safe=False, install_requires=['msgpack-python'], tests_require=[], entry_points={ 'faust.codecs': [ 'msgpack = faust_msgpack:msgpack', ], }, ) ``` -------------------------------- ### Start Faust Worker with Info Logging (Python) Source: https://faust-streaming.github.io/faust/userguide/cli This example shows how to start a Faust worker with an increased logging level to 'info'. This provides more detailed output during the worker's startup process, including messages from various components like the web server, monitor, producer, and consumer. ```shell python examples/word_count.py worker -l info ``` -------------------------------- ### Send Message to Faust Agent (Console) Source: https://faust-streaming.github.io/faust/_sources/playbooks/quickstart This command uses the 'faust send' utility to send a message 'Hello Faust' to the 'greet' agent within the 'hello_world' application. The '@' prefix targets an agent. ```console faust -A hello_world send @greet "Hello Faust" ``` -------------------------------- ### Faust Worker Help Command (Python) Source: https://faust-streaming.github.io/faust/userguide/cli This example displays the help message for the `faust worker` command. It lists all available options with their short and long forms, along with brief descriptions and default values, providing a comprehensive guide to worker configuration. ```shell python examples/word_count.py worker --help ``` -------------------------------- ### Define a Web View (Python) Source: https://faust-streaming.github.io/faust/_sources/userguide/application Provides an example of defining a web view using the `@app.page()` decorator in Faust. This creates a web endpoint that can handle HTTP requests. ```python # examples/view.py import faust app = faust.App('view-example') @app.page('/path/to/view/') async def myview(web, request): ``` -------------------------------- ### Python setup.py for Faust Project Source: https://faust-streaming.github.io/faust/_sources/userguide/application A minimal Python setup.py file using setuptools to define a Faust project. It specifies package details, dependencies (including faust), and Python version requirements. ```python #!/usr/bin/env python from setuptools import find_packages, setup setup( name='proj', version='1.0.0', description='Use Faust to blah blah blah', author='Ola Normann', author_email='ola.normann@example.com', url='http://proj.example.com', platforms=['any'], license='Proprietary', packages=find_packages(exclude=['tests', 'tests.*']), include_package_data=True, zip_safe=False, install_requires=['faust'], python_requires='~=3.8', ) ``` -------------------------------- ### Faust Channel Example: Putting and Getting Messages Source: https://faust-streaming.github.io/faust/_sources/userguide/channels Demonstrates how to manually create a Faust channel, send a message to it using 'put', and then asynchronously iterate over the channel to receive the message. This showcases basic channel interaction for in-memory message buffering. ```python async def main(): channel = app.channel() await channel.put(1) async for event in channel: print(event.value) # the channel is infinite so we break after first event break ``` -------------------------------- ### Setuptools Entry Point for Console Script Source: https://faust-streaming.github.io/faust/_sources/userguide/application Adds a setuptools entry point to setup.py to define a console script for the Faust application. This allows the application to be executed as a command-line tool. ```python setup( ..., entry_points={ 'console_scripts': [ 'proj = proj.app:main', ], }, ) ``` -------------------------------- ### Install aiomonitor for Faust Debugging Source: https://faust-streaming.github.io/faust/_sources/userguide/debugging Installs the aiomonitor library, which is required for enabling the debugging console in Faust. It can be installed standalone or as part of the Faust debug bundle. ```console pip install aiomonitor ``` ```console pip install -U faust-streaming[debug] ``` -------------------------------- ### Creating a Faust Codec Extension via Setuptools Source: https://faust-streaming.github.io/faust/_modules/faust/serializers/codecs Details the process of creating an installable Faust codec extension using setuptools. This involves defining a `setup.py` file with `entry_points` to register the codec and a corresponding Python module containing the codec implementation. This allows Faust to discover and load the extension automatically. ```python from setuptools import setup, find_packages setup( name='faust-msgpack', version='1.0.0', description='Faust msgpack serialization support', author='Ola A. Normann', author_email='ola@normann.no', url='http://github.com/example/faust-msgpack', platforms=['any'], license='BSD', packages=find_packages(exclude=['ez_setup', 'tests', 'tests.*']), zip_safe=False, install_requires=['msgpack-python'], tests_require=[], entry_points={ 'faust.codecs': [ 'msgpack = faust_msgpack:msgpack', ], }, ) ``` ```python from faust.serializers import codecs class raw_msgpack(codecs.Codec): def _dumps(self, obj: Any) -> bytes: return msgpack.dumps(s) def msgpack() -> codecs.Codec: return raw_msgpack() | codecs.binary() ``` -------------------------------- ### Embed Faust App as a Service Dependency Source: https://faust-streaming.github.io/faust/_sources/userguide/application This Python example shows how to integrate a Faust application into a larger service structure, likely using a framework like 'mode'. The Faust app (`faust_app`) is added as a dependency to a custom service (`MyService`) within the `on_init_dependencies` method, ensuring it's initialized correctly when the service starts. ```python import mode # Assuming faust_app is a pre-defined Faust application instance class MyService(mode.Service): def on_init_dependencies(self): return [faust_app] ``` -------------------------------- ### Faust Worker Startup Example (Default) Source: https://faust-streaming.github.io/faust/_sources/userguide/cli This example demonstrates a basic Faust worker startup using the default configuration. It shows the initial console output, including the Faust version, transport, store, web server details, and logging status. ```console $ python examples/word_count.py worker ┌ƒaµS† v1.0.0──────────────────────────────────────────┐ │ id │ word-counts │ │ transport │ kafka://localhost:9092 │ │ store │ rocksdb: │ │ web │ http://localhost:6066/ │ │ log │ -stderr- (warn) │ │ pid │ 46052 │ │ hostname │ grainstate.local │ │ platform │ CPython 3.6.4 (Darwin x86_64) │ │ drivers │ aiokafka=0.4.0 aiohttp=3.0.8 │ │ datadir │ /opt/devel/faust/word-counts-data │ │ appdir │ /opt/devel/faust/word-counts-data/v1 │ └───────────┴──────────────────────────────────────────┘ starting➢ 😊 ``` -------------------------------- ### Start Web Server in Faust Source: https://faust-streaming.github.io/faust/_modules/faust/web/drivers/aiohttp This code snippet demonstrates how to start a web server within a Faust application. It involves creating a site instance and then awaiting its start. ```python site = self._create_site() await site.start() ``` -------------------------------- ### Start a Faust Worker in Foreground Source: https://faust-streaming.github.io/faust/_sources/userguide/workers Starts a Faust worker process in the foreground for development or debugging. It specifies the application module and sets logging level to 'info'. ```console $ faust -A proj worker -l info ``` -------------------------------- ### Start Faust Worker in Foreground Source: https://faust-streaming.github.io/faust/userguide/workers Starts a Faust worker in the foreground for development or debugging purposes. It uses the 'info' log level and specifies the application module. ```bash $ faust -A proj worker -l info ``` -------------------------------- ### Python Setup.py EXTENSIONS Configuration Source: https://faust-streaming.github.io/faust/contributing Illustrates how to configure optional extensions in a Python `setup.py` file. This example shows a set of extension names that can be included for building the package. ```python EXTENSIONS = { 'debug', 'fast', 'rocksdb', 'uvloop', } ``` -------------------------------- ### Install Faust Streaming with Multiple Extensions Source: https://faust-streaming.github.io/faust/_sources/introduction Demonstrates how to install Faust Streaming with multiple dependency bundles simultaneously by separating the bundle names with commas within the square brackets. ```console pip install faust-streaming[uvloop,fast,rocksdb,datadog,redis] ``` -------------------------------- ### Console command to get help for a Faust subcommand Source: https://faust-streaming.github.io/faust/_sources/userguide/application This console command shows how to get detailed help information for a specific Faust subcommand, such as 'worker'. It displays available options and arguments for that subcommand. ```console $ python examples/command.py worker --help ``` -------------------------------- ### Start Second Faust Worker with Unique Configuration Source: https://faust-streaming.github.io/faust/userguide/workers Starts a second Faust worker, ensuring it uses a different data directory and web server port than the first worker to avoid conflicts. ```bash $ faust --datadir=/var/faust/worker2 -A proj -l info worker --web-port=6067 ``` -------------------------------- ### Create and Configure Faust Application Source: https://context7.com/context7/faust-streaming_github_io_faust/llms.txt Demonstrates how to initialize a Faust application with basic broker and storage configurations. It shows alternative methods for setting configuration parameters like broker, store, and processing guarantee. ```python import faust # Basic app with broker and storage configuration app = faust.App( 'myapp', broker='kafka://localhost:9092', store='rocksdb://', topic_partitions=8, topic_replication_factor=3 ) # Alternative: configure after instantiation app = faust.App('myapp') app.conf.broker = 'kafka://kafka.example.com:9092' app.conf.store = 'rocksdb://' app.conf.processing_guarantee = 'exactly_once' # Start the application if __name__ == '__main__': app.main() ``` -------------------------------- ### Start Multiple Faust Apps with Worker Source: https://faust-streaming.github.io/faust/userguide/application Provides examples of how to start multiple Faust applications concurrently using a single `Worker` instance. It addresses configuring individual web ports when running multiple apps. ```python worker = Worker(app1, app2, app3, app4) ``` ```python apps = [app1, app2, app3, app4] for i, app in enumerate(apps): app.conf.web_port = 6066 + i worker = Worker(*apps) ``` -------------------------------- ### Set Up Faust Development Environment Source: https://faust-streaming.github.io/faust/contributing This snippet details the commands to set up the development environment for Faust. 'make develop' installs dependencies and configures the environment for Python development, while 'make hooks' installs pre-commit hooks. 'make cdevelop' is used for installing C extensions, including RocksDB bindings. ```shell $ make develop $ make hooks $ make cdevelop ``` -------------------------------- ### Install Faust with specific feature bundles using pip Source: https://faust-streaming.github.io/faust/userguide/installation These commands demonstrate how to install Faust along with specific feature bundles using pip. Bundles like 'rocksdb', 'uvloop', 'fast', and 'redis' enable additional functionalities such as enhanced storage, faster event loops, performance optimizations, and caching. ```bash $ pip install "faust-streaming[rocksdb]" ``` ```bash $ pip install "faust-streaming[rocksdb,uvloop,fast,redis]" ``` -------------------------------- ### Bash: Display Faust Help Information Source: https://faust-streaming.github.io/faust/userguide/cli Provides an example of how to display the help message for the Faust CLI using the `--help` flag. This output lists available options, commands, and their descriptions. ```bash $ python examples/word_count.py --help ``` -------------------------------- ### Start Faust Worker with Debugging Enabled Source: https://faust-streaming.github.io/faust/_sources/userguide/debugging Starts a Faust worker with the debug option enabled. This will also initiate the aiomonitor console, typically on a local port. The output shows the worker's configuration and the aiomonitor console connection details. ```console $ faust -A myapp --debug worker -l info ┌ƒaµS† v0.9.20─────────────────────────────────────────┐ │ id │ word-counts │ │ transport │ kafka://localhost:9092 │ │ store │ rocksdb: │ │ web │ http://localhost:6066/ │ │ log │ -stderr- (info) │ │ pid │ 55522 │ │ hostname │ grainstate.local │ │ platform │ CPython 3.6.3 (Darwin x86_64) │ │ drivers │ aiokafka=0.3.2 aiohttp=2.3.7 │ │ datadir │ /opt/devel/faust/word-counts-data │ └───────────┴──────────────────────────────────────────┘ [2018-01-04 12:41:07,635: INFO]: Starting aiomonitor at 127.0.0.1:50101 [2018-01-04 12:41:07,637: INFO]: Starting console at 127.0.0.1:50101 [2018-01-04 12:41:07,638: INFO]: [^Worker]: Starting... [2018-03-13 13:41:39,275: INFO]: [^-App]: Starting... [2018-01-04 12:41:07,638: INFO]: [^--Web]: Starting... [...] ``` -------------------------------- ### Starting Multiple Faust Apps with Worker Source: https://faust-streaming.github.io/faust/_sources/userguide/application This Python code snippet illustrates how to configure a Faust `Worker` to start multiple applications. The additional apps are passed as starargs to the `Worker` constructor. The first app provided becomes the main app of the worker. ```python worker = Worker(app1, app2, app3, app4) ``` -------------------------------- ### Run Faust Main CLI Command Source: https://faust-streaming.github.io/faust/_modules/faust/app/base The `main` method serves as the entry point for the Faust command-line interface. When invoked, it first finalizes the application's configuration using `self.finalize()`, then initializes the worker environment with `self.worker_init()`, and finally triggers automatic discovery if configured via `self.conf.autodiscover` by calling `self.discover()`. This orchestrates the setup required before the main application logic begins execution. ```python def main(self) -> NoReturn: """Execute the :program:`faust` umbrella command using this app.""" from faust.cli.faust import cli self.finalize() self.worker_init() if self.conf.autodiscover: self.discover() ``` -------------------------------- ### Starting Faust Worker from Async Function Source: https://faust-streaming.github.io/faust/_sources/userguide/application This Python code demonstrates how to start a Faust worker from within an `async def` function using `await worker.start()`. It highlights the responsibility of the user to manage the event loop and gracefully shut it down using `worker.stop_and_shutdown_loop()`. ```python async def start_worker(worker: Worker) -> None: await worker.start() def manage_loop(): loop = asyncio.get_event_loop() worker = Worker(app, loop=loop) try: loop.run_until_complete(start_worker(worker)) finally: worker.stop_and_shutdown_loop() ``` -------------------------------- ### Start Faust Worker with Custom Data Directory and Web Port Source: https://faust-streaming.github.io/faust/userguide/workers Starts a Faust worker with a specified data directory and web server port. This is crucial for running multiple workers on the same machine, ensuring unique data storage and inter-worker communication. ```bash $ faust --datadir=/var/faust/worker1 -A proj -l info worker --web-port=6066 ``` -------------------------------- ### Start Kafka Server Source: https://faust-streaming.github.io/faust/playbooks/leaderelection Command to start the Kafka server. This command requires Zookeeper to be running and assumes the KAFKA_HOME environment variable is set. ```bash $KAFKA_HOME/bin/kafka-server-start $KAFKA_HOME/etc/kafka/server.properties ``` -------------------------------- ### Faust CLI: Display Help Information Source: https://faust-streaming.github.io/faust/_sources/userguide/cli Illustrates how to get help information for the Faust CLI, including available options and commands, using the `--help` flag. This output details general options and specific commands like 'agents', 'model', 'reset', etc. ```console $ python examples/word_count.py --help Usage: word_count.py [OPTIONS] COMMAND [ARGS]... Faust command-line interface. Options: -L, --loop [aio|eventlet|uvloop] Event loop implementation to use. --json / --no-json Prefer data to be emitted in json format. -D, --datadir DIRECTORY Directory to keep application state. -W, --workdir DIRECTORY Working directory to change to after start. --debug / --no-debug Enable debugging output, and the blocking detector. -q, --quiet / --no-quiet Silence output to /. -A, --app TEXT Path of Faust application to use, or the name of a module. --version Show the version and exit. --help Show this message and exit. Commands: agents List agents. model Show model detail. models List all available models as tabulated list. reset Delete local table state. send Send message to agent/topic. tables List available tables. worker Start ƒaust worker instance. ``` -------------------------------- ### Start Faust App Directly Source: https://faust-streaming.github.io/faust/_sources/userguide/application This Python code shows how to start a Faust application directly using an async function. The `app.start()` method will block until the worker shuts down. This approach is suitable for embedding Faust within other applications or environments where manual control over startup and shutdown is required. ```python async def start_app(app): await app.start() ``` -------------------------------- ### Create Faust App Instance Source: https://faust-streaming.github.io/faust/_sources/userguide/application Instantiates a Faust application with a given name, message broker, and optional storage driver. The broker is essential for communication, and the store is used for table storage. ```python >>> import faust >>> app = faust.App('example', broker='kafka://', store='rocksdb://') ``` ```python >>> app1 = faust.App('demo1') >>> app2 = faust.App('demo2') ``` ```python >>> app = faust.App( ... 'myid', ... broker='kafka://kafka.example.com', ... store='rocksdb://', ... ) ``` ```python >>> app = faust.App('myApp') >>> app.conf.broker='kafka://kafka.example.com' >>> app.conf.store='rocksdb://' >>> app.main() ``` -------------------------------- ### Start Faust App in Background Loop Source: https://faust-streaming.github.io/faust/_sources/userguide/application This Python snippet illustrates how to start a Faust application's event loop in the background using asyncio. `loop.ensure_future(app.start())` schedules the `app.start()` coroutine to run concurrently, allowing other parts of the program to execute while the Faust worker operates independently. ```python import asyncio def start_in_loop(app): loop = asyncio.get_event_loop() loop.ensure_future(app.start()) ``` -------------------------------- ### Install Faust Streaming with Bundled Dependencies Source: https://faust-streaming.github.io/faust/_sources/introduction Installs Faust Streaming with specific sets of dependencies for different functionalities. The `[bundle_name]` syntax allows for easy installation of related packages, such as optimizations or debugging tools. ```console pip install faust-streaming[rocksdb] pip install faust-streaming[redis] pip install faust-streaming[datadog] pip install faust-streaming[statsd] pip install faust-streaming[uvloop] pip install faust-streaming[eventlet] pip install faust-streaming[yaml] pip install faust-streaming[fast] pip install faust-streaming[aiodns] pip install faust-streaming[cchardet] pip install faust-streaming[ciso8601] pip install faust-straming[cython] pip install faust-streaming[orjson] pip install faust-streaming[setproctitle] pip install faust-streaming[debug] pip install faust-streaming[aiomonitor] ``` -------------------------------- ### Install Faust with Multiple Pip Extensions Source: https://faust-streaming.github.io/faust/introduction Installs Faust with a comma-separated list of extensions. This is useful for enabling multiple functionalities like uvloop, fast, rocksdb, datadog, and redis simultaneously. ```bash $ pip install faust-streaming[uvloop,fast,rocksdb,datadog,redis] ``` -------------------------------- ### Boot Strategy (`faust.app.base.BootStrategy`) Source: https://faust-streaming.github.io/faust/reference/faust.app Defines the startup strategy for a Faust application, determining the graph of services to initiate when a Faust worker starts. ```APIDOC ## `faust.app.base.BootStrategy` ### Description App startup strategy. The startup strategy defines the graph of services to start when the Faust worker for an app starts. ### Class Definition ```python class BootStrategy(_app : AppT_, _*_ , _enable_web : Optional[bool] = None_, _enable_kafka : Optional[bool] = None_, _enable_kafka_producer : Optional[bool] = None_, _enable_kafka_consumer : Optional[bool] = None_, _enable_sensors : Optional[bool] = None_) ``` ### Parameters #### Keyword Arguments - **_app** (AppT_) - The Faust application instance. - **enable_web** (Optional[bool]) - Whether to enable web services. - **enable_kafka** (Optional[bool]) - Whether to enable Kafka services (defaults to True). - **enable_kafka_producer** (Optional[bool]) - Whether to enable Kafka producer services. - **enable_kafka_consumer** (Optional[bool]) - Whether to enable Kafka consumer services. - **enable_sensors** (Optional[bool]) - Whether to enable sensor services (defaults to True). ### Methods - **agents()**: Returns services required to start agents. - **client_only()**: Returns services for client-only mode. - **kafka_client_consumer()**: Returns services for Kafka client consumer. - **kafka_conductor()**: Returns services for Kafka conductor. - **kafka_consumer()**: Returns services for Kafka consumer. - **kafka_producer()**: Returns services for Kafka producer. - **producer_only()**: Returns services for producer-only mode. - **sensors()**: Returns services for starting sensors. - **server()**: Returns services for default server mode. - **tables()**: Returns table-related services. - **web_components()**: Returns web-related services (excluding the server). - **web_server()**: Returns web server services. ### Properties - **enable_kafka** (bool) - **enable_kafka_consumer** (Optional[bool]) - **enable_kafka_producer** (Optional[bool]) - **enable_sensors** (bool) - **enable_web** (Optional[bool]) - **app** (AppT_) ``` -------------------------------- ### Start Faust App in Client-Only Mode Source: https://faust-streaming.github.io/faust/_sources/userguide/application This asynchronous Python function demonstrates how to start a Faust application in 'client-only' mode. `app.start_client()` enables the application to send RPC requests and receive replies without launching a full Faust worker, making it suitable for scenarios where only client functionalities are needed. ```python async def start_client_mode(app): await app.start_client() ``` -------------------------------- ### Get Help for Faust Worker Command Source: https://faust-streaming.github.io/faust/userguide/workers Displays the help message for the Faust worker command, listing all available command-line options and configurations. ```bash $ faust worker --help ``` -------------------------------- ### Faust Worker Startup Commands Source: https://faust-streaming.github.io/faust/_modules/faust/stores/rocksdb These commands demonstrate how to start Faust workers with specified data directories and web ports. Each worker is configured to run with informational logging. ```bash $ myproj --datadir=/var/faust/w1 worker -l info --web-port=6066 $ myproj --datadir=/var/faust/w2 worker -l info --web-port=6067 $ myproj --datadir=/var/faust/w3 worker -l info --web-port=6068 $ myproj --datadir=/var/faust/w4 worker -l info --web-port=6069 ``` -------------------------------- ### Count Stream Elements with Stream.enumerate() in Python Source: https://faust-streaming.github.io/faust/_sources/userguide/streams The Stream.enumerate() method functions similarly to Python's built-in enumerate, providing a count for each value processed in an asynchronous stream. It yields tuples of (index, value), starting the count from zero by default. An optional starting point argument can be provided to alter the initial count. ```python @app.agent() async def process(stream): async for i, value in stream.enumerate(): pass @app.agent() async def process_with_start(stream): async for i, value in stream.enumerate(start=1): pass ``` -------------------------------- ### Get Faust Version Source: https://faust-streaming.github.io/faust/_modules/faust/types/settings/settings Retrieves the installed version of the Faust library. This is useful for checking compatibility or reporting issues. It relies on the `symbol_by_name` utility function. ```python faust_version: str = symbol_by_name("faust:__version__") ``` -------------------------------- ### Installing Faust Documentation Dependencies Source: https://faust-streaming.github.io/faust/_sources/contributing Commands to install dependencies required for building Faust documentation using pip. Ensures all necessary packages are present for successful documentation generation. ```console $ pip install -U -r requirements/docs.txt ``` -------------------------------- ### Define an Asynchronous Task (Python) Source: https://faust-streaming.github.io/faust/_sources/userguide/application Illustrates how to define a new asynchronous task using the `@app.task()` decorator in Faust. This task will be started once the Faust application is operational. ```python @app.task() async def mytask(): print('APP STARTED AND OPERATIONAL') ``` -------------------------------- ### Faust Application (`faust.app.base.App`) Source: https://faust-streaming.github.io/faust/reference/faust.app Represents an instance of the Faust library, serving as the entry point for all Faust applications. It manages configuration, event loops, and other core components. ```APIDOC ## `faust.app.base.App` ### Description Faust Application. An app is an instance of the Faust library. Everything starts here. ### Class Definition ```python class App(_id : str_, _*_ , _monitor : Optional[Monitor] = None_, _config_source : Optional[Any] = None_, _loop : Optional[AbstractEventLoop] = None_, _beacon : Optional[NodeT] = None_, _** options: Any_) ``` ### Parameters #### Path Parameters - **id** (str) - Required - Application ID. #### Keyword Arguments - **monitor** (Optional[Monitor]) - Optional monitor instance. - **config_source** (Optional[Any]) - Optional configuration source. - **loop** (Optional[asyncio.AbstractEventLoop]) - Optional event loop to use. - **beacon** (Optional[NodeT]) - Optional beacon node. - **options** (Any) - Additional options for application configuration. ### Class Variables - **SCAN_CATEGORIES** (ClassVar[List[str]]) - List of categories to scan for Faust components. ``` -------------------------------- ### Install development version of Faust using pip Source: https://faust-streaming.github.io/faust/userguide/installation This command installs the latest development snapshot of Faust directly from its GitHub repository. It's useful for testing the newest features or contributing to Faust's development. ```bash $ pip install https://github.com/faust-streaming/faust/zipball/master#egg=faust ``` -------------------------------- ### Signal Before App Configuration (Python) Source: https://faust-streaming.github.io/faust/_sources/userguide/application The on_before_configured signal is a synchronous signal dispatched before the application begins reading its configuration. It can be used for initial setup or logging before configuration is applied. ```python @app.on_before_configured.connect def before_configuration(app, **kwargs): print(f'App {app} is being configured') ``` -------------------------------- ### Install Faust with Eventlet Support Source: https://faust-streaming.github.io/faust/faq Installs the Faust streaming library with optional support for the eventlet asynchronous I/O library. This is necessary for integrating Faust with blocking Python libraries and frameworks that can work with eventlet. ```bash pip install -U faust-streaming[eventlet] ``` -------------------------------- ### Faust Stream with Custom Model and Task Source: https://faust-streaming.github.io/faust/_sources/userguide/streams Illustrates creating a stream with a custom record model for deserialization and processing it within an asyncio task. This is a more practical example for real-world applications. ```python import faust import asyncio class Withdrawal(faust.Record): account: str amount: float app = faust.App('withdrawal-processor', broker='kafka://localhost:9092') withdrawals_topic = app.topic('withdrawals', value_type=Withdrawal) @app.task async def process_withdrawals(): async for w in withdrawals_topic.stream(): print(f"Withdrawal from account {w.account}: Amount {w.amount}") if __name__ == '__main__': # To run this, you'd typically use 'faust -A your_module_name worker -l info' # app.main() is used here for demonstration purposes if running as a script directly # In a real Faust app, you'd rely on the CLI pass # In a real app, you'd run using Faust CLI: faust -A your_script_name worker -l info ``` -------------------------------- ### Running Faust Application Entry Point Source: https://faust-streaming.github.io/faust/_sources/userguide/cli Demonstrates how to set up a Python script to be executable as a Faust application using `app.main()`. This is the standard way to make a Faust project runnable from the command line. ```python if __name__ == '__main__': app.main() ``` ```python # package/__main__.py from package.app import app app.main() ``` -------------------------------- ### Python Web Framework Blueprint Example Source: https://faust-streaming.github.io/faust/_sources/history/changelog-1.1 An example of a simple web route using a Python framework's blueprint, demonstrating a basic 'Hello world' response. This showcases a minimal web abstraction suitable for rapid experimentation. ```python @blueprint.route('/', name='index') async def hello(self, request): return self.text('Hello world') ``` -------------------------------- ### Core Coordinator Assignment Logic Source: https://faust-streaming.github.io/faust/_modules/faust/assignor/partition_assignor Executes the main assignment process, interacting with sensors for start, error, and completion events. It calls _perform_assignment to get the actual topic assignments. ```Python def _assign( self, cluster: ClusterMetadata, member_metadata: MemberMetadataMapping ) -> MemberAssignmentMapping: sensor_state = self.app.sensors.on_assignment_start(self) try: assignment = self._perform_assignment(cluster, member_metadata) except MemoryError: raise except Exception as exc: self.app.sensors.on_assignment_error(self, sensor_state, exc) raise else: self.app.sensors.on_assignment_completed(self, sensor_state) return assignment ``` -------------------------------- ### Attach Custom Headers on Message Produce (Python) Source: https://faust-streaming.github.io/faust/_sources/userguide/application The on_produce_message signal is a synchronous signal called before messages are produced. This example demonstrates how to attach custom headers, such as tracing information, to Kafka messages. ```python from typing import Any, List, Tuple from faust.types import AppT from mode.utils.compat import want_bytes @app.on_produce_message.connect() def on_produce_attach_trace_headers( self, sender: AppT, key: bytes = None, value: bytes = None, partition: Optional[int] = None, timestamp: Optional[float] = None, headers: List[Tuple[str, bytes]] = None, **kwargs: Any) -> None: test = current_test() if test is not None: # Headers at this point is a list of ``(key, value)`` pairs. # Note2: values in headers must be :class:`bytes`. headers.extend([ (k, want_bytes(v)) for k, v in test.as_headers().items() ]) ```