### LoggingMiddleware Example Implementation Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/api-reference/middleware.md An example implementation of ServerMiddleware that logs job start and finish events, including execution time and success/failure status. ```rust use async_trait::async_trait; use sidekiq::{ChainIter, Job, Result, ServerMiddleware, WorkerRef, RedisPool}; use std::sync::Arc; use std::time::Instant; struct LoggingMiddleware; #[async_trait] impl ServerMiddleware for LoggingMiddleware { async fn call( &self, iter: ChainIter, job: &Job, worker: Arc, redis: RedisPool, ) -> Result<()> { let start = Instant::now(); tracing::info!( class = &job.class, jid = &job.jid, "Job started" ); let result = iter.next(job, worker, redis).await; let elapsed = start.elapsed(); if result.is_ok() { tracing::info!( class = &job.class, jid = &job.jid, elapsed_ms = elapsed.as_millis(), "Job completed successfully" ); } else { tracing::warn!( class = &job.class, jid = &job.jid, elapsed_ms = elapsed.as_millis(), error = ?result, "Job failed" ); } result } } ``` -------------------------------- ### Start the Sidekiq Server Source: https://github.com/film42/sidekiq-rs/blob/master/README.md Initializes the Redis connection pool, creates a Processor, registers workers, adds custom middlewares, and starts the Sidekiq server. This is the main entry point for running the job processor. ```rust // Redis let manager = sidekiq::RedisConnectionManager::new("redis://127.0.0.1/").unwrap(); let mut redis = bb8::Pool::builder().build(manager).await.unwrap(); // Sidekiq server let mut p = Processor::new( redis, vec!["yolo".to_string(), "brolo".to_string()], ); // Add known workers p.register(PaymentReportWorker::new()); // Custom Middlewares p.using(FilterExpiredUsersMiddleware::new()) .await; // Start the server p.run().await; ``` -------------------------------- ### ValidationMiddleware Example Implementation Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/api-reference/middleware.md An example implementation of ServerMiddleware that performs job validation before passing control to the next middleware in the chain. It returns an error if the job is invalid. ```rust use async_trait::async_trait; use sidekiq::{ChainIter, Job, Result, ServerMiddleware, WorkerRef, RedisPool}; use std::sync::Arc; #[async_trait] impl ServerMiddleware for ValidationMiddleware { async fn call( &self, iter: ChainIter, job: &Job, worker: Arc, redis: RedisPool, ) -> Result<()> { // Validate before processing if !is_valid_job(job) { return Err(Error::Message("Invalid job".into())); } // Continue to next middleware iter.next(job, worker, redis).await } } ``` -------------------------------- ### run Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/api-reference/processor.md Starts the processor server, spawning worker tasks and initiating background routines for job processing and monitoring. ```APIDOC ## run ### Description Starts the processor server, spawning worker tasks and starting background routines. This method blocks until all tasks are shut down. ### Method `run(self) -> impl Future` ### Parameters None ### Return * `impl Future` - A future that completes when all tasks exit. ### Example ```rust processor.run().await; // Blocks until shutdown ``` ``` -------------------------------- ### Enqueue and Schedule Jobs Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/INDEX.md Provides examples for enqueuing jobs immediately, with custom options, and scheduling them for a future time. Requires the `redis` client and job arguments. ```rust // Enqueue immediately MyWorker::perform_async(&redis, args).await?; // Enqueue with custom options MyWorker::opts() .queue("priority") .retry(3) .unique_for(Duration::from_secs(3600)) .perform_async(&redis, args) .await?; // Schedule for later MyWorker::perform_in(&redis, Duration::from_secs(3600), args).await?; ``` -------------------------------- ### Initializing Redis Connection Pool Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/errors.md Demonstrates the initialization of a Redis connection pool, including error handling for connection failures. It shows how to recover from `Error::Redis` during setup. ```rust let redis = match RedisConnectionManager::new("redis://127.0.0.1/") { Ok(mgr) => Pool::builder().build(mgr).await?, Err(Error::Redis(e)) => { eprintln!("Cannot connect to Redis: {}", e); return Err(e.into()); } Err(e) => return Err(e), }; ``` -------------------------------- ### Small App (Development) Configuration Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/configuration.md A complete example for a small application, using a single Redis pool and simple processor configuration, suitable for development environments. ```rust use sidekiq::{Processor, ProcessorConfig, RedisConnectionManager}; use bb8::Pool; #[tokio::main] async fn main() -> Result<()> { // Single pool, simple config let manager = RedisConnectionManager::new("redis://localhost/")?; let redis = Pool::builder().build(manager).await?; let processor = Processor::new( redis, vec!["default".to_string()], ); processor.run().await; Ok(()) } ``` -------------------------------- ### Configure and Run Processor Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/INDEX.md Sets up a job processor with custom configuration for worker count and balancing strategy, then registers a worker and middleware. The `run` method starts job processing. ```rust use rusty_sidekiq::prelude::*; use rusty_sidekiq::redis::RedisConnectionManager; use bb8::Pool; // Assuming EmailWorker is defined elsewhere and implements Worker // struct EmailWorker; // Assuming LoggingMiddleware is defined elsewhere and implements Middleware // struct LoggingMiddleware; // impl LoggingMiddleware { fn new() -> Self { LoggingMiddleware } } // async fn setup_processor() -> Result<()> // { // let manager = RedisConnectionManager::new("redis://localhost/")?; // let redis = Pool::builder().build(manager).await?; // let config = ProcessorConfig::default() // .num_workers(50) // .balance_strategy(BalanceStrategy::RoundRobin); // let mut processor = Processor::new( // redis, // vec!["default".to_string(), "emails".to_string()], // ) // .with_config(config); // processor.register(EmailWorker); // processor.using(LoggingMiddleware::new()).await; // processor.run().await; // Ok(()) // } ``` -------------------------------- ### Run the Processor Server Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/api-reference/processor.md Starts the Sidekiq-rs processor server. This method blocks until the server is shut down and spawns various background tasks. ```rust processor.run().await; // Blocks until shutdown ``` -------------------------------- ### Database Transaction Middleware Implementation Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/api-reference/middleware.md Implement custom middleware to wrap job execution within a database transaction. This example demonstrates starting, committing, and rolling back transactions based on job success or failure. Requires a custom DbConnection and Transaction implementation. ```rust use async_trait::async_trait; use sidekiq::{ChainIter, Job, Result, ServerMiddleware, WorkerRef, RedisPool}; use std::sync::Arc; #[derive(Clone)] struct DbConnection { // Your database connection } struct TransactionMiddleware { db: DbConnection, } impl TransactionMiddleware { fn new(db: DbConnection) -> Self { Self { db } } } #[async_trait] impl ServerMiddleware for TransactionMiddleware { async fn call( &self, iter: ChainIter, job: &Job, worker: Arc, redis: RedisPool, ) -> Result<()> { // Start transaction let tx = self.db.begin_transaction().await?; let result = iter.next(job, worker, redis).await; if result.is_ok() { tx.commit().await?; } else { tx.rollback().await?; } result } } impl DbConnection { async fn begin_transaction(&self) -> Result { // Implementation Err(Error::Message("Not implemented".into())) } } struct Transaction; impl Transaction { async fn commit(self) -> Result<()> { Ok(()) } async fn rollback(self) -> Result<()> { Ok(()) } } ``` -------------------------------- ### Medium App (Production) Configuration Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/configuration.md A comprehensive example for a medium-sized production application, featuring separate enqueue and fetch pools, custom worker count, and a round-robin balance strategy. ```rust use sidekiq::{Processor, ProcessorConfig, BalanceStrategy, RedisConnectionManager}; use bb8::Pool; #[tokio::main] async fn main() -> Result<()> { let manager = RedisConnectionManager::new("redis://localhost/")?; let enqueue = Pool::builder().max_size(10).build(manager.clone()).await?; let fetch = Pool::builder().max_size(30).build(manager).await?; let config = ProcessorConfig::default() .num_workers(50) .balance_strategy(BalanceStrategy::RoundRobin); let processor = Processor::new( fetch, vec![ "default".to_string(), "emails".to_string(), "batch".to_string(), ], ) .with_config(config); processor.run().await; Ok(()) } ``` -------------------------------- ### Schedule a Job to Run Later Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/api-reference/enqueue-opts.md Schedule a job to be performed after a specified duration. This example schedules a reminder for a user. ```rust use std::time::Duration; // Schedule a reminder for 1 hour from now sidekiq::opts() .queue("reminders") .retry(1) .perform_in( &redis, Duration::from_secs(3600), "ReminderWorker".into(), json!({ "user_id": 123, "reminder_type": "follow_up" }) ) .await?; ``` -------------------------------- ### WorkerOpts::new Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/api-reference/worker.md Creates a new `WorkerOpts` instance with default values. This is the starting point for configuring custom job options. ```APIDOC ## `WorkerOpts::new` ### Description Creates a new `WorkerOpts` with default values. ### Method Constructor ### Endpoint N/A (SDK Method) ### Parameters None ### Return - `WorkerOpts` ### Example ```rust WorkerOpts::::new() ``` ``` -------------------------------- ### Schedule a Daily Report Job Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/INDEX.md This example demonstrates how to schedule a periodic job to run daily at 8:00 AM. It includes destroying existing periodic jobs, building a new periodic job with custom arguments and queue, and registering it with the processor. ```rust periodic::destroy_all(redis.clone()).await?; periodic::builder("0 0 8 * * *")? .name("Daily report at 8am") .queue("reports") .args(ReportArgs { report_type: "summary".into(), })? .register(&mut processor, ReportWorker) .await?; ``` -------------------------------- ### Implement Logging Middleware for Jobs Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/INDEX.md This middleware logs the start and end of job processing. It requires the `async_trait` crate and the `ServerMiddleware` trait. ```rust struct LoggingMiddleware; #[async_trait] impl ServerMiddleware for LoggingMiddleware { async fn call( &self, iter: ChainIter, job: &Job, worker: Arc, redis: RedisPool, ) -> Result<()> { tracing::info!("Starting job: {}", job.jid); let result = iter.next(job, worker, redis).await; tracing::info!("Finished job: {}", job.jid); result } } processor.using(LoggingMiddleware).await; ``` -------------------------------- ### Counter new Method Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/types.md Creates a new Counter instance initialized with a specified starting value. This is the constructor for the Counter struct. ```rust new(n: usize) -> Self ``` -------------------------------- ### Create New WorkerOpts Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/api-reference/worker.md Instantiates a new WorkerOpts struct with default values for queue and retry behavior. Used as a starting point for configuring job options. ```rust WorkerOpts::::new() ``` -------------------------------- ### Redis Pool Usage Examples Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/api-reference/redis.md Shows common operations performed using a RedisPool. This includes enqueuing a job using a worker and fetching a connection for direct Redis commands like sadd. ```rust // Enqueue a job MyWorker::perform_async(&redis, args).await?; // Fetch a connection for custom operations let mut conn = redis.get().await?; conn.sadd("my_set", "value").await?; ``` -------------------------------- ### Implement Server Middleware for Job Filtering in Rust Source: https://github.com/film42/sidekiq-rs/blob/master/README.md Create a custom server middleware by implementing the `ServerMiddleware` trait. This example filters jobs based on a `user_guid` parameter, preventing execution for expired users. It leverages `serde` for conditional type-checking and parameter validation. ```rust use tracing::info; struct FilterExpiredUsersMiddleware {} impl FilterExpiredUsersMiddleware { fn new() -> Self { Self { } } } #[derive(Deserialize)] struct FiltereExpiredUsersArgs { user_guid: String, } impl FiltereExpiredUsersArgs { fn is_expired(&self) -> bool { self.user_guid == "USR-123-EXPIRED" } } #[async_trait] impl ServerMiddleware for FilterExpiredUsersMiddleware { async fn call( &self, chain: ChainIter, job: &Job, worker: Arc, redis: RedisPool, ) -> ServerResult { // Use serde to check if a user_guid is part of the job args. let args: Result<(FiltereExpiredUsersArgs,), serde_json::Error> = serde_json::from_value(job.args.clone()); // If we can safely deserialize then attempt to filter based on user guid. if let Ok((filter,)) = args { if filter.is_expired() { error!({ "class" = job.class, "jid" = job.jid, "user_guid" = filter.user_guid }, "Detected an expired user, skipping this job" ); return Ok(()); } } // This customer is not expired, so we may continue. chain.next(job, worker, redis).await } } ``` -------------------------------- ### Default QueueConfig Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/api-reference/processor.md Creates a default queue configuration with zero dedicated workers for a specific queue. This is useful when starting with a base configuration. ```rust let config = QueueConfig::default(); ``` -------------------------------- ### Set Redis Connection Timeout Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/configuration.md Configure a connection timeout for Redis to prevent hanging connections. This example sets a 5-second timeout. ```rust let redis = Pool::builder() .connection_timeout(std::time::Duration::from_secs(5)) .build(manager) .await?; ``` -------------------------------- ### Registering Custom Middleware in Processor Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/api-reference/middleware.md Register custom middleware with a Sidekiq-rs Processor instance before starting its execution. Middleware is applied in a Last-In, First-Out (LIFO) order. ```rust let mut processor = Processor::new(redis, queues); processor.register(MyWorker); // Register custom middleware processor.using(LoggingMiddleware).await; processor.using(RateLimitMiddleware::new(10)).await; processor.using(TimeoutMiddleware::new(Duration::from_secs(60))).await; processor.run().await; ``` -------------------------------- ### Configure Worker Options with `opts()` Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/api-reference/worker.md Override the `opts()` method to customize worker behavior such as queue name, retry count, and unique job duration. This example configures an `EmailWorker` to use the 'emails' queue and retry up to 5 times. ```rust #[async_trait] impl Worker for EmailWorker { fn opts() -> sidekiq::WorkerOpts { sidekiq::WorkerOpts::new() .queue("emails") .retry(5) .unique_for(std::time::Duration::from_secs(3600)) } async fn perform(&self, args: EmailArgs) -> Result<()> { // ... Ok(()) } } ``` -------------------------------- ### Register and Parse Periodic Jobs with Cron Expressions Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/errors.md Validate a cron expression using `periodic::parse` and register a periodic job with a processor using `periodic::builder`. The cron expression must be valid, and the job must be registered before the processor starts. ```rust // Validate cron expression periodic::parse("0 0 8 * * *")?; // Must call register before running processor periodic::builder("0 0 8 * * *")? .register(&mut processor, MyWorker) .await?; processor.run().await; ``` -------------------------------- ### Basic Redis Connection Manager Usage Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/configuration.md Demonstrates various ways to initialize `RedisConnectionManager` with different Redis connection string formats, including host, port, database selection, and authentication. ```rust use sidekiq::RedisConnectionManager; use bb8::Pool; // Simple connection let manager = RedisConnectionManager::new("redis://localhost/")?; let redis = Pool::builder().build(manager).await?; // With port let manager = RedisConnectionManager::new("redis://localhost:6380/")?; // With database selection let manager = RedisConnectionManager::new("redis://localhost/1")?; // With authentication let manager = RedisConnectionManager::new("redis://user:password@localhost/")?; // Full URL let manager = RedisConnectionManager::new("redis://user:password@host:6380/2")?; ``` -------------------------------- ### Initialize Redis Connection Pool Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/INDEX.md Creates a Redis connection pool using `bb8` and `RedisConnectionManager`. Demonstrates setting up a custom namespace for Redis keys. ```rust use rusty_sidekiq::redis::{RedisConnectionManager, with_custom_namespace}; use bb8::Pool; // async fn setup_redis_pool() -> Result> // { // let manager = RedisConnectionManager::new("redis://localhost:6379/0")?; // let redis = Pool::builder().max_size(32).build(manager).await?; // Ok(redis) // } // async fn setup_redis_with_namespace() -> Result> // { // let manager = RedisConnectionManager::new("redis://localhost:6379/0")?; // let customizer = with_custom_namespace("my_app".to_string()); // let redis = Pool::builder() // .connection_customizer(customizer) // .build(manager) // .await?; // Ok(redis) // } ``` -------------------------------- ### periodic::builder Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/api-reference/periodic.md Creates a new periodic job builder initialized with a cron expression. This is the starting point for configuring a scheduled job. ```APIDOC ## periodic::builder ### Description Creates a new periodic job builder with a cron expression. This is the starting point for configuring a scheduled job. ### Method `periodic::builder(cron_str: &str) -> Result` ### Parameters #### Path Parameters - **cron_str** (string) - Required - Valid cron expression (6 fields: `second minute hour day month day_of_week`) ### Return - **Builder** - Builder for fluent configuration ### Errors - **CronClock error** - If the cron string is invalid ### Example ```rust let builder = periodic::builder("0 0 8 * * *")?; ``` ``` -------------------------------- ### Register Workers with Processor Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/INDEX.md Demonstrates how to initialize a `Processor` and register different types of workers with it. Workers can be registered directly or with custom dependencies. ```rust let mut processor = Processor::new(redis, queues); processor.register(EmailWorker); processor.register(PaymentWorker::new(db)); processor.register(ReportWorker::new(analytics)); ``` -------------------------------- ### Default Processor Configuration Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/configuration.md Creates a Processor with default settings. This is the simplest way to initialize the processor. ```rust let config = ProcessorConfig::default(); ``` -------------------------------- ### Get Sorted Set Members by Index Range (ZRANGE) Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/api-reference/redis.md Retrieves members from a sorted set within a specified index range (inclusive). ```rust let members = conn.zrange("periodic".to_string(), 0, -1).await?; ``` -------------------------------- ### Rust Periodic Job Registration and Processing Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/api-reference/periodic.md This snippet shows how to set up a Redis connection, define a worker, register two periodic jobs (daily and weekly summaries) with different schedules and arguments, and then run the Sidekiq processor. Ensure you have a Redis instance running and the `sidekiq-rs` and `bb8` crates added to your `Cargo.toml`. ```rust use async_trait::async_trait; use bb8::Pool; use serde::{Deserialize, Serialize}; use sidekiq::{periodic, Processor, RedisConnectionManager, Result, Worker}; #[derive(Clone, Serialize, Deserialize)] struct EmailArgs { email: String, report_type: String, } #[derive(Clone)] struct EmailWorker; #[async_trait] impl Worker for EmailWorker { fn opts() -> sidekiq::WorkerOpts { sidekiq::WorkerOpts::new().queue("emails") } async fn perform(&self, args: EmailArgs) -> Result<()> { println!("Sending {} report to {}", args.report_type, args.email); Ok(()) } } #[tokio::main] async fn main() -> Result<()> { // Setup let manager = RedisConnectionManager::new("redis://127.0.0.1/")?; let redis = Pool::builder().build(manager).await?; let mut processor = Processor::new(redis.clone(), vec!["default".to_string()]); // Clear old periodic jobs periodic::destroy_all(redis.clone()).await?; // Register periodic jobs periodic::builder("0 0 8 * * *")? // Daily at 8 AM .name("Daily summary at 8am") .queue("emails") .args(EmailArgs { email: "reports@example.com".into(), report_type: "daily".into(), })? .register(&mut processor, EmailWorker) .await?; periodic::builder("0 0 * * 0")? // Weekly at midnight Sunday .name("Weekly summary at midnight Monday") .queue("emails") .retry(3) .args(EmailArgs { email: "reports@example.com".into(), report_type: "weekly".into(), })? .register(&mut processor, EmailWorker) .await?; // Run processor processor.run().await; Ok(()) } ``` -------------------------------- ### RedisConnection::new Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/api-reference/redis.md Initializes a new `RedisConnection` wrapper around a `bb8` multiplexed connection without any namespace. ```APIDOC ## RedisConnection::new ### Description Creates a new `RedisConnection` instance, wrapping a raw Redis connection. This constructor does not apply any namespace. ### Method Rust function call ### Signature `new(connection: Connection) -> Self` ### Parameters - `connection` (Connection): A multiplexed Redis connection obtained from a pool. ### Return - `Self`: A new `RedisConnection` instance. ### Example ```rust let mut conn = redis.get().await?; // Assuming 'redis' is a RedisPool ``` ``` -------------------------------- ### Create New Processor Instance Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/api-reference/processor.md Instantiates a new Processor, specifying the Redis connection pool and the queues it should monitor. Requires a Redis connection manager and an async context. ```rust let manager = sidekiq::RedisConnectionManager::new("redis://127.0.0.1/")?; let redis = bb8::Pool::builder().build(manager).await?; let processor = Processor::new( redis, vec!["default".to_string(), "low_priority".to_string()], ); ``` -------------------------------- ### RetryOpts Direct Construction with WorkerOpts Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/types.md Shows how to configure retry options directly when creating WorkerOpts. ```rust // Direct construction WorkerOpts::new().retry(RetryOpts::Never) WorkerOpts::new().retry(5) WorkerOpts::new().retry(true) ``` -------------------------------- ### ProcessorConfig with QueueConfig Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/api-reference/processor.md Demonstrates how to configure a processor with a base number of workers and dedicated workers for specific queues. The total workers spawned is the sum of base workers and all dedicated queue workers. ```APIDOC ## ProcessorConfig with QueueConfig ### Description Configures the processor with a base number of workers and dedicated workers for specific queues. The total workers spawned is the sum of base workers and all dedicated queue workers. ### Example ```rust let config = ProcessorConfig::default() .num_workers(5) // 5 shared workers .queue_config("urgent".to_string(), QueueConfig::default().num_workers(20)) .queue_config("batch".to_string(), QueueConfig::default().num_workers(3)); // Total workers spawned: 5 + 20 + 3 = 28 ``` ``` -------------------------------- ### Counter Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/types.md A thread-safe atomic counter used for tracking concurrent job execution. It provides methods to initialize, get the current value, increment, and decrement the count. ```APIDOC ## Counter Struct Thread-safe atomic counter for tracking concurrent job execution. ### Definition ```rust pub struct Counter { count: Arc, } ``` ### Methods #### `new(n: usize) -> Self` Creates a counter initialized to `n`. #### `value(&self) -> usize` Gets the current count. #### `incrby(&self, n: usize)` Increments by n. #### `decrby(&self, n: usize)` Decrements by n. ``` -------------------------------- ### Use Separate Redis Connection Pools Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/INDEX.md Illustrates setting up distinct Redis connection pools for enqueuing and fetching jobs. This can optimize performance by dedicating pools to specific operations. ```rust let enqueue = Pool::builder().max_size(10).build(manager.clone()).await?; let fetch = Pool::builder().max_size(50).build(manager).await?; // Fast enqueueing MyWorker::perform_async(&enqueue, args).await?; // Separate processing pool let processor = Processor::new(fetch, queues).run().await; ``` -------------------------------- ### Configure Redis Connection Manager Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/errors.md Create a `RedisConnectionManager` instance using the correct connection string for your Redis server. Ensure the Redis server is running and accessible. ```rust // Check Redis is running: redis-cli ping // Check connection string is correct: redis://localhost:6379/ let manager = RedisConnectionManager::new("redis://localhost:6379/")?; ``` -------------------------------- ### Disable Argument Coercion for Tuple Arguments Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/api-reference/worker.md Set `disable_argument_coercion()` to `true` when your worker expects a tuple or vector that should not be automatically unwrapped. This example ensures `TupleWorker` receives its arguments as a tuple `(ItemArgs,)`. ```rust #[async_trait] impl Worker<(ItemArgs,)> for TupleWorker { fn disable_argument_coercion(&self) -> bool { true // Keep the tuple wrapper } async fn perform(&self, args: (ItemArgs,)) -> Result<()> { // ... Ok(()) } } ``` -------------------------------- ### Processor::new Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/api-reference/processor.md Creates a new Processor instance. It takes a Redis connection pool and a list of queue names to monitor. ```APIDOC ## Processor::new ### Description Creates a new `Processor` instance with the specified Redis pool and queues to monitor. ### Method `new(redis: RedisPool, queues: Vec) -> Self` ### Parameters - `redis` (RedisPool) - bb8 connection pool - `queues` (Vec) - List of queue names to fetch from ### Return `Self` - New Processor instance ### Default config CPU count workers, RoundRobin queue balancing ### Example ```rust let manager = sidekiq::RedisConnectionManager::new("redis://127.0.0.1/")?; let redis = bb8::Pool::builder().build(manager).await?; let processor = Processor::new( redis, vec!["default".to_string(), "low_priority".to_string()], ); ``` ``` -------------------------------- ### ProcessorConfig builder methods Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/api-reference/processor.md Methods for configuring the Processor, including setting worker count, balancing strategy, and per-queue configurations. ```APIDOC ## ProcessorConfig Builder Methods ### num_workers Sets the number of worker tasks to spawn. #### Method `num_workers(mut self, num_workers: usize) -> Self` #### Parameters * `num_workers` (usize) - Worker count. Recommended to match CPU count for CPU-bound tasks, or higher for IO-bound tasks. #### Return * `Self` - For method chaining. #### Example ```rust ProcessorConfig::default().num_workers(50) ``` ### balance_strategy Sets the queue balancing strategy. #### Method `balance_strategy(mut self, balance_strategy: BalanceStrategy) -> Self` #### Parameters * `balance_strategy` (BalanceStrategy) - Balancing method (e.g., `RoundRobin`, `None`). #### Return * `Self` - For method chaining. #### Example ```rust ProcessorConfig::default() .balance_strategy(BalanceStrategy::None) ``` ### queue_config Sets per-queue configuration, allowing dedicated workers for specific queues. #### Method `queue_config(mut self, queue: String, config: QueueConfig) -> Self` #### Parameters * `queue` (String) - Queue name. * `config` (QueueConfig) - Queue-specific settings. #### Return * `Self` - For method chaining. #### Example ```rust ProcessorConfig::default() .num_workers(10) .queue_config("urgent".to_string(), QueueConfig::default().num_workers(20)) .queue_config("batch".to_string(), QueueConfig::default().num_workers(5)) ``` ``` -------------------------------- ### Set Custom `max_retries` for a Worker Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/api-reference/worker.md Implement the `max_retries()` method to control the number of times a failed job will be retried before being moved to the dead set. This example limits retries to 10 for `ProcessDataWorker`. ```rust #[async_trait] impl Worker for ProcessDataWorker { fn max_retries(&self) -> usize { 10 // Only retry 10 times instead of default 25 } async fn perform(&self, args: DataArgs) -> Result<()> { // ... Ok(()) } } ``` -------------------------------- ### Separate Pools for Enqueue and Fetch Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/configuration.md Illustrates the best practice of using separate `bb8::Pool` instances for enqueueing and fetching operations to prevent blocking and allow independent sizing. ```rust // Both from same manager let manager = RedisConnectionManager::new("redis://localhost/")?; // Enqueue pool - smaller, optimized for speed let enqueue_pool = Pool::builder() .max_size(10) .build(manager.clone()) .await?; // Fetch pool - larger, handles blocking operations let fetch_pool = Pool::builder() .max_size(32) .build(manager) .await?; // Use in code // MyWorker::perform_async(&enqueue_pool, args).await?; // let processor = Processor::new(fetch_pool, queues); // processor.run().await; ``` -------------------------------- ### Implement Worker Trait for Payment Processing Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/api-reference/worker.md Example of implementing the `perform` method for a `PaymentWorker`. This method contains the core logic for processing a job and should return `Ok(())` on success or an error for retries. ```rust #[async_trait] impl Worker for PaymentWorker { async fn perform(&self, args: PaymentArgs) -> Result<()> { println!("Processing payment for user: {}", args.user_id); // ... actual work Ok(()) } } ``` -------------------------------- ### Customizing bb8 Connection Pool Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/configuration.md Shows how to configure the `bb8::Pool` with custom settings like maximum pool size and connection timeout duration. ```rust use bb8::Pool; use std::time::Duration; let manager = RedisConnectionManager::new("redis://localhost/")?; let redis = Pool::builder() .max_size(32) // Maximum pool size .connection_timeout(Duration::from_secs(5)) .build(manager) .await?; ``` -------------------------------- ### Calculate Connection Pool Size for Performance Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/configuration.md Determines the optimal Redis connection pool size based on the number of worker tasks. A formula is provided as a starting point for performance tuning. ```rust use bb8::Pool; // Formula: (num_workers * 1.25) is a good starting point let num_workers = 50; let pool_size = (num_workers as f64 * 1.25) as u32 as usize; let redis = Pool::builder() .max_size(pool_size) .build(manager) .await?; ``` -------------------------------- ### StatsPublisher new Method Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/types.md Constructs a new StatsPublisher instance with essential details like hostname, queues, a counter for busy jobs, and the processor's concurrency limit. This is the constructor for the StatsPublisher. ```rust new(hostname: String, queues: Vec, busy_jobs: Counter, concurrency: usize) -> Self ``` -------------------------------- ### Get Raw Redis Connection Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/api-reference/redis.md Borrows the raw Redis connection without namespacing, allowing direct use of the underlying redis-rs crate API. Useful for operations not covered by the RedisConnection wrapper. ```rust use redis::AsyncCommands; let mut conn = redis.get().await?; let count: i64 = conn .unnamespaced_borrow_mut() .incr("counter", 1) .await?; ``` -------------------------------- ### Add Middleware to Processor Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/INDEX.md Shows how to add various middleware components to the processor pipeline. Each middleware can be configured with specific parameters like timeouts or rate limits. ```rust processor.using(LoggingMiddleware::new()).await; processor.using(TimeoutMiddleware::new(Duration::from_secs(60))).await; processor.using(RateLimitMiddleware::new(10)).await; ``` -------------------------------- ### Get Sorted Set Members by Score Range with Limit (ZRANGEBYSCORE_LIMIT) Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/api-reference/redis.md Fetches members from a sorted set based on a score range, with options for pagination (offset and limit). Ideal for fetching ready jobs. ```rust let ready_jobs = conn.zrangebyscore_limit( "schedule".to_string(), "-inf", now.timestamp(), 0, 10 ).await?; ``` -------------------------------- ### Override `class_name` for Nested Workers Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/api-reference/worker.md Override the `class_name()` method to explicitly set the Sidekiq class name, useful when working with Ruby Sidekiq workers in nested modules. This example sets the class name to `Workers::MyWorker`. ```rust #[async_trait] impl Worker<()> for MyWorker { fn class_name() -> String { "Workers::MyWorker".to_string() } async fn perform(&self, _args: ()) -> Result<()> { Ok(()) } } ``` -------------------------------- ### Enqueue Job Immediately with Options Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/api-reference/enqueue-opts.md Use `perform_async` to enqueue a job immediately with specified queue, retry count, and arguments. Ensure the `serde_json` crate is imported for argument serialization. ```rust use serde_json::json; sidekiq::opts() .queue("emails") .retry(3) .perform_async(&redis, "EmailWorker".into(), json!({ "email": "user@example.com", "subject": "Welcome!", "template": "welcome" })) .await?; ``` -------------------------------- ### Namespace Configuration for Redis Keys Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/configuration.md Demonstrates how to add a custom namespace prefix to all Redis keys managed by `sidekiq-rs` using `with_custom_namespace`. ```rust use sidekiq::with_custom_namespace; use bb8::Pool; let manager = RedisConnectionManager::new("redis://localhost/")?; let customizer = with_custom_namespace("my_app".to_string()); let redis = Pool::builder() .connection_customizer(customizer) .build(manager) .await?; // All keys are now prefixed: "my_app:queue:default", "my_app:scheduled", etc. ``` -------------------------------- ### Configure Job Retry Behavior Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/api-reference/enqueue-opts.md Sets the retry strategy for a job using boolean, count, or RetryOpts enum. Examples show disabling retries, setting a maximum retry count, and using a specific RetryOpts value. ```rust sidekiq::opts() .retry(false) // Don't retry .perform_async(&redis, "MyWorker".into(), json!(args)) .await?; ``` ```rust sidekiq::opts() .retry(5) // Max 5 retries .perform_async(&redis, "MyWorker".into(), json!(args)) .await?; ``` ```rust sidekiq::opts() .retry(RetryOpts::Never) .perform_async(&redis, "MyWorker".into(), json!(args)) .await?; ``` -------------------------------- ### EnqueueOpts Builder Methods Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/types.md Provides builder methods for configuring EnqueueOpts. These methods allow chaining to set the queue name, retry options, unique duration, and retry queue. ```rust - `queue>(self, queue: S) -> Self` - `retry>(self, retry: RO) -> Self` - `unique_for(self, duration: Duration) -> Self` - `retry_queue(self, queue: String) -> Self` ``` -------------------------------- ### EnqueueOpts Builder Methods Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/api-reference/enqueue-opts.md Methods to configure EnqueueOpts for job enqueueing, supporting method chaining. ```APIDOC ## queue>(self, queue: S) -> Self Sets the queue name for this job. ### Method `queue(queue: S)` ### Parameters - **queue** (S): Queue name ### Return `Self` - For chaining ### Example ```rust sidekiq::opts() .queue("emails") .queue("high_priority") .queue("batch") ``` ``` ```APIDOC ## retry>(self, retry: RO) -> Self Sets the retry behavior for the job. ### Method `retry(retry: RO)` ### Parameters - **retry** (RO): Boolean, number (usize), or `RetryOpts` enum. - `true` or `false` (bool) → `RetryOpts::Yes` or `RetryOpts::Never` - Number (usize) → `RetryOpts::Max(n)` - `RetryOpts` enum directly ### Return `Self` - For chaining ### Example ```rust // Don't retry sidekiq::opts() .retry(false) .perform_async(&redis, "MyWorker".into(), json!(args)) .await?; // Max 5 retries sidekiq::opts() .retry(5) .perform_async(&redis, "MyWorker".into(), json!(args)) .await?; // Use RetryOpts enum sidekiq::opts() .retry(RetryOpts::Never) .perform_async(&redis, "MyWorker".into(), json!(args)) .await?; ``` ``` ```APIDOC ## unique_for(self, duration: Duration) -> Self Makes this job unique for a specified duration. Only one job with the same class, queue, and argument hash can be enqueued within this period. ### Method `unique_for(duration: Duration)` ### Parameters - **duration** (Duration): The uniqueness time-to-live (TTL). ### Return `Self` - For chaining ### Notes Uniqueness is determined by a SHA256 hash of the job arguments. This uses Redis SET with NX and EX flags. ### Example ```rust use std::time::Duration; sidekiq::opts() .queue("emails") .unique_for(Duration::from_secs(3600)) .perform_async(&redis, "EmailWorker".into(), json!({ "email": "user@example.com", "type": "welcome" })) .await?; // If called again within 1 hour with the same arguments, the job will not be enqueued. ``` ``` ```APIDOC ## retry_queue(self, queue: String) -> Self Specifies an alternative queue for retried jobs. Failed jobs will be moved to this queue instead of being retried on the original queue. ### Method `retry_queue(queue: String)` ### Parameters - **queue** (String): The name of the queue for retried jobs. ### Return `Self` - For chaining ### Example ```rust sidekiq::opts() .queue("primary") .retry_queue("dlq") // Dead letter queue .perform_async(&redis, "MyWorker".into(), json!(args)) .await?; ``` ``` -------------------------------- ### opts() -> WorkerOpts Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/api-reference/worker.md Returns the default options for enqueueing this worker's jobs. Override to set queue, retry behavior, and unique job handling. Defaults to `WorkerOpts::new()` with `queue = "default"` and `retry = Yes`. ```APIDOC ## `opts() -> WorkerOpts` ### Description Returns the default options for enqueueing this worker's jobs. Override to set queue, retry behavior, and unique job handling. ### Parameters None ### Return - `WorkerOpts`: Configurable options ### Default `WorkerOpts::new()` with `queue = "default"` and `retry = Yes` ### Location `src/lib.rs:315-320` ### Example ```rust #[async_trait] impl Worker for EmailWorker { fn opts() -> sidekiq::WorkerOpts { sidekiq::WorkerOpts::new() .queue("emails") .retry(5) .unique_for(std::time::Duration::from_secs(3600)) } async fn perform(&self, args: EmailArgs) -> Result<()> { // ... Ok(()) } } ``` ``` -------------------------------- ### perform_async (static) Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/api-reference/worker.md Static method to enqueue this worker with default options and arguments. This is a direct way to enqueue a job without configuring specific options. ```APIDOC ## `perform_async` (static) ### Description Static method to enqueue this worker with default options and arguments. ### Method POST (conceptual, as this is an SDK method) ### Endpoint N/A (SDK Method) ### Parameters - **redis** (`&RedisPool`) - Redis connection pool - **args** (`Args`) - Job arguments ### Return - `Result<()>` - Returns error if enqueueing fails ### Example ```rust MyWorker::perform_async(&redis, MyArgs { user_id: 123 }).await?; ``` ``` -------------------------------- ### Enqueue Job with Configured Options Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/api-reference/worker.md Enqueues a job immediately using the options configured via the WorkerOpts builder methods. Requires a Redis connection pool and job arguments. ```rust MyWorker::opts() .queue("emails") .retry(3) .perform_async(&redis, MyArgs { email: "user@example.com" }) .await?; ``` -------------------------------- ### Create Job Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/api-reference/enqueue-opts.md Creates a `Job` struct from the configured `EnqueueOpts` and provided arguments without immediately enqueueing it. ```APIDOC ## create_job(&self, class: String, args: impl Serialize) -> Result Creates a Job from the options and arguments without enqueueing. ### Method `create_job(class: String, args: impl Serialize)` ### Parameters - **class** (String): The name of the worker class. - **args** (impl Serialize): The arguments for the job, which must be serializable. ### Return `Result` - A new `Job` instance with a generated JID and timestamp, or an error. ### Use Cases Useful for inspecting a job before enqueueing or for storing it for later processing. ### Example ```rust let job = sidekiq::opts() .queue("emails") .create_job("EmailWorker".into(), json!({ "email": "user@example.com" }))?; println!("Job ID: {}", job.jid); println!("Queue: {}", job.queue); println!("Args: {}", job.args); // To enqueue the created job: // UnitOfWork::from_job(job).enqueue(&redis).await?; ``` ``` -------------------------------- ### Create Default EnqueueOpts Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/api-reference/enqueue-opts.md Initializes a new EnqueueOpts instance with default values for queue and retry behavior. ```rust let opts = sidekiq::opts(); ``` -------------------------------- ### Configure Per-Queue Settings Source: https://github.com/film42/sidekiq-rs/blob/master/_autodocs/api-reference/processor.md Sets specific configurations for individual queues, allowing for dedicated workers or custom settings per queue. ```rust ProcessorConfig::default() .num_workers(10) .queue_config("urgent".to_string(), QueueConfig::default().num_workers(20)) .queue_config("batch".to_string(), QueueConfig::default().num_workers(5)) ``` -------------------------------- ### Create a Job using Crate Level Method Source: https://github.com/film42/sidekiq-rs/blob/master/README.md Inserts a job using the crate-level `perform_async` function, providing the worker name, queue name, and arguments. This offers more control over job creation. ```rust sidekiq::perform_async( &mut redis, "PaymentReportWorker".into(), "yolo".into(), PaymentReportArgs { user_guid: "USR-123".to_string(), }, ) .await?; ```