### Usage Examples Source: https://github.com/mysorf-9239/mysorf-base/blob/main/src/mysorf_base/runtime/README.md Examples demonstrating how to use the runtime subsystem. ```APIDOC ## Usage Examples ### Context manager (recommended) ```python from mysorf_base.runtime import bootstrap with bootstrap(["logging=rich", "tracking=wandb"]) as ctx: ctx.logger.info("started") ctx.tracker.start_run(run_name="exp-01") # teardown() is called automatically on exit ``` ### Manual teardown ```python from mysorf_base.runtime import bootstrap, teardown ctx = bootstrap() ctx.logger.info("started") ctx.tracker.start_run(run_name="exp-01") teardown(ctx) ``` ### Hydra overrides ```python with bootstrap(["runtime.seed=42", "logging=structlog", "tracking=disabled"]) as ctx: ctx.logger.info(f"seed={ctx.cfg.runtime.seed}") ``` ### Disabled backends (for tests) ```python ctx = bootstrap(["logging=disabled", "tracking=disabled", "profiling=disabled"]) # ctx.logger → NullLogger # ctx.tracker → NullTracker # ctx.profiler → NullProfiler ``` ``` -------------------------------- ### Install mysorf-base with optional extras Source: https://github.com/mysorf-9239/mysorf-base/blob/main/README.md Install with optional dependencies for specific subsystems like rich logging or Weights & Biases tracking. 'all' installs all optional extras. ```bash pip install "mysorf-base[logging-rich,tracking-wandb] @ git+https://github.com/mysorf-9239/mysorf-base.git@v0.1.0" pip install "mysorf-base[all] @ git+https://github.com/mysorf-9239/mysorf-base.git@v0.1.0" ``` -------------------------------- ### mysorf-base Installation from GitHub Source: https://context7.com/mysorf-9239/mysorf-base/llms.txt Install mysorf-base from GitHub using pip. Supports installation with optional backends for extended functionality. ```bash # Install from GitHub pip install git+https://github.com/mysorf-9239/mysorf-base.git@v0.1.0 # With optional backends pip install "mysorf-base[logging-rich,logging-structlog,tracking-wandb,profiling-pandas] @ git+https://..." pip install "mysorf-base[all] @ git+https://github.com/mysorf-9239/mysorf-base.git@v0.1.0" ``` -------------------------------- ### Install Wandb Backend Source: https://github.com/mysorf-9239/mysorf-base/blob/main/src/mysorf_base/tracking/README.md Install the tracking subsystem with the Weights & Biases backend support. This command installs the necessary extras for wandb integration. ```bash pip install -e ".[tracking-wandb]" ``` -------------------------------- ### Set Up Development Environment Source: https://github.com/mysorf-9239/mysorf-base/blob/main/CONTRIBUTING.md Clone the repository, create and activate a conda environment, and install the project in editable mode with development dependencies. Finally, install pre-commit hooks. ```bash git clone https://github.com/mysorf-9239/mysorf-base.git cd mysorf-base conda env create -f conda-recipes/dev.yaml conda activate mysorf-base pip install -e ".[dev]" pre-commit install ``` -------------------------------- ### Install mysorf-base with pip Source: https://github.com/mysorf-9239/mysorf-base/blob/main/README.md Install the library from GitHub using pip. Specify a version for reproducibility. ```bash pip install git+https://github.com/mysorf-9239/mysorf-base.git@v0.1.0 ``` -------------------------------- ### CLI Configuration Entrypoint Source: https://github.com/mysorf-9239/mysorf-base/blob/main/src/mysorf_base/config/README.md Example of how to use the mysorf_base.cli to configure the application. This allows setting configuration options directly from the command line. ```bash python -m mysorf_base.cli logging=console tracking=disabled runtime.seed=42 ``` -------------------------------- ### Pandas Profiling Installation Source: https://github.com/mysorf-9239/mysorf-base/blob/main/src/mysorf_base/profiling/README.md Command to install the pandas profiling backend, which requires the `profiling-pandas` extra. ```bash pip install -e ".[profiling-pandas"]" ``` -------------------------------- ### Grid Strategy Example Source: https://github.com/mysorf-9239/mysorf-base/blob/main/src/mysorf_base/sweeps/README.md Example demonstrating the grid strategy for generating all Cartesian product combinations of parameters. ```APIDOC from mysorf_base.sweeps import SearchSpace, CategoricalParam space = SearchSpace(params=[ CategoricalParam(name="lr", values=[0.001, 0.01]), CategoricalParam(name="dropout", values=[0.1, 0.3]), ]) # This configuration will produce 4 trials: (0.001, 0.1), (0.001, 0.3), (0.01, 0.1), (0.01, 0.3) ``` -------------------------------- ### Artifact Path Convention Example Source: https://github.com/mysorf-9239/mysorf-base/blob/main/src/mysorf_base/artifacts/README.md Illustrates the standard directory structure for storing artifacts, including checkpoints and datasets. ```text {base_dir}/{artifact_type}/{name}/{version}/{filename} ``` ```text artifacts/checkpoint/best_model/run_abc123/model.pt artifacts/dataset/train_split/epoch_0005/train.csv ``` -------------------------------- ### Install Structlog Logging Backend Source: https://github.com/mysorf-9239/mysorf-base/blob/main/src/mysorf_base/logging/README.md Installs the 'logging-structlog' extra for structured and JSON logging. Execute this command within your project's environment. ```bash pip install -e ".[logging-structlog]" ``` -------------------------------- ### mysorf-base Local Editable Install Source: https://context7.com/mysorf-9239/mysorf-base/llms.txt Perform a local editable install of mysorf-base for development purposes. Includes setting up pre-commit hooks. ```bash # Local editable install for development git clone https://github.com/mysorf-9239/mysorf-base.git && cd mysorf-base pip install -e ".[dev]" && pre-commit install ``` -------------------------------- ### Install Rich Logging Backend Source: https://github.com/mysorf-9239/mysorf-base/blob/main/src/mysorf_base/logging/README.md Installs the 'logging-rich' extra for enhanced terminal output. This command should be run in your project's environment. ```bash pip install -e ".[logging-rich]" ``` -------------------------------- ### Direct Construction Source: https://github.com/mysorf-9239/mysorf-base/blob/main/src/mysorf_base/profiling/README.md Example of directly constructing and using a profiler instance. ```APIDOC ## Direct construction ```python from mysorf_base.profiling import build_profiler, ProfilingConfig cfg = ProfilingConfig(backend="basic", top_k=3) profiler = build_profiler(cfg) summary = profiler.profile_records(records) ``` ``` -------------------------------- ### Grid Strategy Example Source: https://github.com/mysorf-9239/mysorf-base/blob/main/src/mysorf_base/sweeps/README.md Illustrates the Grid strategy for defining a search space that generates all Cartesian product combinations of parameters. This example uses two categorical parameters. ```python from mysorf_base.sweeps import SearchSpace, CategoricalParam space = SearchSpace(params=[ CategoricalParam(name="lr", values=[0.001, 0.01]), CategoricalParam(name="dropout", values=[0.1, 0.3]), ]) # Produces 4 trials: (0.001, 0.1), (0.001, 0.3), (0.01, 0.1), (0.01, 0.3) ``` -------------------------------- ### Direct Construction Usage Source: https://github.com/mysorf-9239/mysorf-base/blob/main/src/mysorf_base/tracking/README.md Example of directly constructing and using a tracker instance. ```APIDOC ## Direct Construction Usage ### Description Demonstrates how to manually create and use a tracker instance when not using the `RuntimeContext`. ### Example ```python from mysorf_base.tracking import build_tracker, TrackingConfig cfg = TrackingConfig(backend="disabled") tracker = build_tracker(cfg) tracker.start_run() ``` ``` -------------------------------- ### Tracking via RuntimeContext Source: https://github.com/mysorf-9239/mysorf-base/blob/main/src/mysorf_base/tracking/README.md Recommended usage for the tracking subsystem, integrating with the RuntimeContext for automatic setup and teardown. ```python from mysorf_base.runtime import bootstrap with bootstrap(["tracking=wandb"]) as ctx: ctx.tracker.start_run(run_name="baseline") ctx.tracker.log_params({"lr": 0.001, "epochs": 100}) ctx.tracker.log_metrics({"loss": 0.42, "auroc": 0.91}, step=1) # finish() is called automatically on context exit ``` -------------------------------- ### Local editable install of mysorf-base Source: https://github.com/mysorf-9239/mysorf-base/blob/main/README.md Clone the repository and install in editable mode for local development. Includes dev dependencies and sets up pre-commit hooks. ```bash git clone https://github.com/mysorf-9239/mysorf-base.git cd mysorf-base pip install -e ".[dev]" pre-commit install ``` -------------------------------- ### Wandb Configuration in YAML Source: https://github.com/mysorf-9239/mysorf-base/blob/main/src/mysorf_base/tracking/README.md Example of how Wandb backend configuration reads credentials and settings from environment variables using OmegaConf. ```yaml wandb: api_key: ${oc.env:WANDB_API_KEY,null} mode: ${oc.env:WANDB_MODE,offline} project: ${oc.env:WANDB_PROJECT,mysorf-base} entity: ${oc.env:WANDB_ENTITY,null} ``` -------------------------------- ### Backend: WandB Source: https://github.com/mysorf-9239/mysorf-base/blob/main/src/mysorf_base/sweeps/README.md The 'wandb' backend delegates sweep execution to Weights & Biases, utilizing `wandb.sweep()` and `wandb.agent()`. It requires the `tracking-wandb` extra to be installed. ```APIDOC ## WandB Backend ### Description Delegates sweep execution to Weights & Biases (`wandb.sweep` and `wandb.agent`). Requires the `tracking-wandb` extra to be installed. ### Installation ```bash pip install -e ".[tracking-wandb]" ``` ### Configuration - Requires `search_space` to be passed to `build_sweep_runner()`. - Maps `strategy="grid"` to a wandb grid sweep. - Maps `strategy="random"` to a wandb random sweep, using `n_trials` as the agent count. ``` -------------------------------- ### WandB Tracking Credentials Configuration Source: https://github.com/mysorf-9239/mysorf-base/blob/main/conf/README.md Example of how to configure Weights & Biases tracking credentials using environment variables, either in a .env file or exported directly. This is used by the tracking configuration groups. ```bash # .env (never commit this file) WANDB_PROJECT=my-project WANDB_API_KEY=your-key WANDB_MODE=online ``` ```bash export WANDB_PROJECT=my-project export WANDB_API_KEY=your-key export WANDB_MODE=online ``` -------------------------------- ### bootstrap Source: https://github.com/mysorf-9239/mysorf-base/blob/main/src/mysorf_base/runtime/README.md Compose config and build all subsystem instances. ```APIDOC ## bootstrap(overrides) ### Description Compose config and build all subsystem instances. ### Parameters #### Path Parameters - **overrides** (list[str]) - Optional - List of Hydra configuration overrides. ``` -------------------------------- ### Backend Directory Structure Source: https://github.com/mysorf-9239/mysorf-base/blob/main/src/mysorf_base/artifacts/backends/README.md Illustrates the recommended directory structure for implementing a new artifact backend. ```text mysorf_base/artifacts/backends/ ├── local.py ├── null.py └── s3.py ``` -------------------------------- ### bootstrap() — Runtime Entry Point Source: https://context7.com/mysorf-9239/mysorf-base/llms.txt The primary entry point for initializing the runtime environment. It composes configuration, constructs subsystem instances, and returns a RuntimeContext. Supports context manager protocol for automatic teardown. ```APIDOC ## bootstrap() ### Description Composes configuration and constructs all subsystem instances. Accepts a list of Hydra override strings and returns a frozen `RuntimeContext`. ### Method `bootstrap(overrides: list[str]) -> RuntimeContext` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Parameters - **overrides** (list[str]) - Required - A list of Hydra override strings to customize configuration. ### Request Example ```python from mysorf_base.runtime import bootstrap with bootstrap(["logging=rich", "tracking=wandb", "runtime.seed=42"]) as ctx: ctx.logger.info(f"run_id={ctx.run_id} config_hash={ctx.config_hash[:8]}") ctx.tracker.start_run(run_name="baseline-v1") ``` ### Response #### Success Response (RuntimeContext) - **ctx** (RuntimeContext) - An immutable container holding all initialized subsystem instances. #### Response Example ```python # Example of accessing context attributes print(ctx.run_id) print(ctx.cfg.logging.backend) ctx.logger.info("Hello, world!") ``` ### Teardown Supports the context manager protocol for automatic `teardown()`. Manual teardown is also possible. ``` -------------------------------- ### Bootstrap Logging via RuntimeContext Source: https://github.com/mysorf-9239/mysorf-base/blob/main/src/mysorf_base/logging/README.md Recommended method for initializing logging. Uses the bootstrap function to configure logging based on provided settings. ```python from mysorf_base.runtime import bootstrap with bootstrap(["logging=structlog"]) as ctx: ctx.logger.info("experiment started") ctx.logger.warning("low memory") ``` -------------------------------- ### Bootstrap with Disabled Backends for Testing Source: https://context7.com/mysorf-9239/mysorf-base/llms.txt Initialize `bootstrap()` with disabled backends for testing purposes. Subsystems will return null objects, preventing errors and output. ```python ctx = bootstrap(["logging=disabled", "tracking=disabled", "profiling=disabled"]) ctx.logger.info("this is a no-op") # NullLogger — no output, no crash ctx.tracker.log_metrics({"x": 1.0}) # NullTracker — no-op ``` -------------------------------- ### Backward-Compatible Configuration Override Source: https://github.com/mysorf-9239/mysorf-base/blob/main/README.md For backward compatibility, bare overrides are mapped to the `config show` command, allowing for quick configuration checks. ```bash mysorf-base logging=structlog ``` -------------------------------- ### Bootstrap RuntimeContext Source: https://context7.com/mysorf-9239/mysorf-base/llms.txt Initializes the runtime context with specified configurations. Use this to access profiler, logger, and tracker services. ```python from mysorf_base.runtime import bootstrap with bootstrap(["profiling=basic"]) as ctx: s = ctx.profiler.profile_records(records) ctx.logger.info(f"profiled {s.row_count} rows") ctx.tracker.log_metrics({"data/rows": float(s.row_count)}) ``` -------------------------------- ### Bootstrap runtime context and log info Source: https://github.com/mysorf-9239/mysorf-base/blob/main/README.md Initialize the runtime context with specified subsystems and log basic run information. The context provides access to logger, run ID, and config hash. ```python from mysorf_base.runtime import bootstrap with bootstrap(["logging=rich", "tracking=wandb"]) as ctx: ctx.logger.info(f"run_id={ctx.run_id} config_hash={ctx.config_hash}") ctx.tracker.start_run(run_name="baseline") ctx.tracker.log_metrics({"loss": 0.42}, step=1) # checkpoint events are emitted automatically: ctx.event_bus.subscribe("checkpoint.saved", lambda e: ctx.logger.info(e.name)) ``` -------------------------------- ### Build Disabled Tracker Source: https://context7.com/mysorf-9239/mysorf-base/llms.txt Instantiates a tracker with the 'disabled' backend, which performs no operations. This is useful for environments where tracking is not needed or for initial setup. ```python from mysorf_base.tracking import build_tracker, TrackingConfig, NullTracker # Disabled tracker — no-op, no extra dependencies tracker = build_tracker(TrackingConfig(backend="disabled")) tracker.start_run(run_name="local-test") tracker.log_params({"lr": 0.001}) tracker.log_metrics({"loss": 0.5}, step=0) tracker.finish() # no-op ``` -------------------------------- ### Bootstrap Runtime with Context Manager Source: https://context7.com/mysorf-9239/mysorf-base/llms.txt Use the `bootstrap()` function as a context manager for automatic teardown. This is the recommended approach for managing runtime resources. ```python from mysorf_base.runtime import bootstrap # Context manager (recommended) — teardown() is automatic with bootstrap(["logging=rich", "tracking=wandb", "runtime.seed=42"]) as ctx: ctx.logger.info(f"run_id={ctx.run_id} config_hash={ctx.config_hash[:8]}") ctx.tracker.start_run(run_name="baseline-v1") ctx.tracker.log_params({"lr": 0.001, "epochs": 50}) ctx.tracker.log_metrics({"loss": 0.42, "auroc": 0.91}, step=1) # tracker.finish(), artifact_manager.finalize(), logger cleanup on exit ``` -------------------------------- ### Profiling via RuntimeContext Source: https://github.com/mysorf-9239/mysorf-base/blob/main/src/mysorf_base/profiling/README.md Demonstrates recommended usage of the profiler through the RuntimeContext, including profiling records, logging, and timing operations. ```python from mysorf_base.runtime import bootstrap with bootstrap(["profiling=basic"]) as ctx: summary = ctx.profiler.profile_records([ {"drug": "A", "score": 1.2}, {"drug": "B", "score": 0.9}, ]) ctx.logger.info(f"rows={summary.row_count}, cols={summary.column_count}") ctx.tracker.log_metrics({"profile_rows": float(summary.row_count)}) with ctx.profiler.time("stage.encode"): _ = [record["drug"] for record in [{"drug": "A"}, {"drug": "B"}]] ctx.profiler.reset_timing() ``` -------------------------------- ### Direct Logger Construction Source: https://github.com/mysorf-9239/mysorf-base/blob/main/src/mysorf_base/logging/README.md Constructs a logger instance directly by parsing a configuration and using the build_logger function. Useful for specific setups or testing. ```python from mysorf_base.logging import build_logger, parse_logging_config cfg = parse_logging_config({"backend": "console", "level": "DEBUG"}) logger = build_logger(cfg, name="my-app") logger.info("hello") ``` -------------------------------- ### Commit Message Format Source: https://github.com/mysorf-9239/mysorf-base/blob/main/CONTRIBUTING.md Defines the standard format for commit messages, including a type, short summary, and an optional body. Provides examples for common types. ```git : ``` ```git feat: add sweeps subsystem with grid and random strategies fix: resolve config path when installed via pip docs: standardize subsystem README format ``` -------------------------------- ### Primary Defaults List in config.yaml Source: https://github.com/mysorf-9239/mysorf-base/blob/main/conf/README.md Defines the default configuration group members for each top-level section in the Hydra setup. This file is essential for bootstrapping the application with the correct settings. ```yaml defaults: - _self_ - env: local - paths: default - runtime: default - logging: console - tracking: disabled - profiling: basic - artifacts: default - sweeps: default - hydra: default app: name: mysorf-base subsystem: config version: 0.1.0 ``` -------------------------------- ### Build Logger Directly Source: https://github.com/mysorf-9239/mysorf-base/blob/main/src/mysorf_base/logging/README.md Demonstrates how to directly construct a logger instance by parsing a configuration and using the build_logger function. This is useful for scenarios outside of the standard runtime context. ```APIDOC ## Direct construction ```python from mysorf_base.logging import build_logger, parse_logging_config cfg = parse_logging_config({"backend": "console", "level": "DEBUG"}) logger = build_logger(cfg, name="my-app") logger.info("hello") ``` ``` -------------------------------- ### Tracker Protocol Source: https://github.com/mysorf-9239/mysorf-base/blob/main/src/mysorf_base/tracking/README.md The core Tracker protocol defines the interface for all tracking backends. Implementations must provide methods for starting and finishing runs, logging parameters, metrics, and artifacts. ```APIDOC ## Tracker Protocol ### Description Defines the interface for tracking operations. ### Methods - `start_run(run_name: str | None = None)`: Starts a new tracking run. - `log_params(params: Mapping[str, Any])`: Logs parameters for the current run. - `log_metrics(metrics: Mapping[str, float], *, step: int | None = None)`: Logs metrics for the current run, optionally at a specific step. - `log_artifact(path: str, *, name: str | None = None)`: Logs an artifact from a given path. - `finish()`: Finishes the current tracking run. ``` -------------------------------- ### Configure Wandb via Environment Variables Source: https://github.com/mysorf-9239/mysorf-base/blob/main/src/mysorf_base/tracking/README.md Set Weights & Biases credentials and configuration by exporting environment variables. ```bash export WANDB_PROJECT=my-project export WANDB_ENTITY=my-team export WANDB_API_KEY=your-key export WANDB_MODE=online ``` -------------------------------- ### Define Search Space with Typed Parameters Source: https://github.com/mysorf-9239/mysorf-base/blob/main/src/mysorf_base/sweeps/README.md Define a hyperparameter search space using typed parameter descriptors like CategoricalParam, FloatParam, and IntegerParam. This example includes categorical, log-scaled float, and integer parameters. ```python from mysorf_base.sweeps import SearchSpace, CategoricalParam, IntegerParam, FloatParam space = SearchSpace(params=[ CategoricalParam(name="optimizer", values=["adam", "sgd"]), FloatParam(name="lr", low=1e-4, high=1e-2, log_scale=True, n_points=5), IntegerParam(name="batch_size", low=16, high=128, step=16), ]) ``` -------------------------------- ### Direct Profiler Construction Source: https://github.com/mysorf-9239/mysorf-base/blob/main/src/mysorf_base/profiling/README.md Shows how to directly construct a profiler using `build_profiler` with a `ProfilingConfig` object. ```python from mysorf_base.profiling import build_profiler, ProfilingConfig cfg = ProfilingConfig(backend="basic", top_k=3) profiler = build_profiler(cfg) summary = profiler.profile_records(records) ``` -------------------------------- ### Build Basic Profiler and Profile Records Source: https://context7.com/mysorf-9239/mysorf-base/llms.txt Constructs a profiler with the 'basic' backend and uses it to analyze a list of records. Outputs summary statistics and column details. ```python from mysorf_base.profiling import build_profiler, ProfilingConfig cfg = ProfilingConfig(backend="basic", top_k=3, numeric_stats=True) profiler = build_profiler(cfg) records = [ {"drug": "aspirin", "dose_mg": 500.0, "response": 1}, {"drug": "ibuprofen", "dose_mg": 200.0, "response": 0}, {"drug": "aspirin", "dose_mg": 250.0, "response": 1}, {"drug": "placebo", "dose_mg": None, "response": 0}, ] summary = profiler.profile_records(records) print(f"rows={summary.row_count}, cols={summary.column_count}") # rows=4, cols=3 for col in summary.columns: print(f" {col.name}: non_null={col.non_null_count}, unique={col.unique_count}") if col.numeric: print(f" min={col.numeric.minimum}, max={col.numeric.maximum}, mean={col.numeric.mean}") ``` -------------------------------- ### mysorf-base CLI: Backward-compatible overrides Source: https://context7.com/mysorf-9239/mysorf-base/llms.txt Bare overrides passed to the mysorf-base CLI are backward-compatible and map to `config show`. ```bash # Backward-compatible: bare overrides map to `config show` mysorf-base logging=structlog tracking=wandb # equivalent to: mysorf-base config show logging=structlog tracking=wandb ``` -------------------------------- ### Save and Load Artifacts via RuntimeContext Source: https://github.com/mysorf-9239/mysorf-base/blob/main/src/mysorf_base/artifacts/README.md Demonstrates saving and loading artifacts using the recommended RuntimeContext, which automatically handles run IDs and logging. ```python from mysorf_base.runtime import bootstrap from mysorf_base.artifacts import ArtifactType with bootstrap() as ctx: ctx.logger.info(f"run_id={ctx.run_id}") record = ctx.artifact_manager.save( "model.pt", name="best_model", artifact_type=ArtifactType.CHECKPOINT, ) ctx.logger.info(f"Saved to {record.path}") path = ctx.artifact_manager.load("best_model", ArtifactType.CHECKPOINT) ``` -------------------------------- ### Runtime Bootstrap with Context Manager Source: https://github.com/mysorf-9239/mysorf-base/blob/main/src/mysorf_base/runtime/README.md Use the bootstrap function as a context manager for automatic resource teardown. Pass Hydra overrides as a list of strings. ```python from mysorf_base.runtime import bootstrap with bootstrap(["logging=rich", "tracking=wandb"]) as ctx: ctx.logger.info("started") ctx.tracker.start_run(run_name="exp-01") # teardown() is called automatically on exit ``` -------------------------------- ### Show Effective Configuration Source: https://github.com/mysorf-9239/mysorf-base/blob/main/README.md Use this command to display the combined configuration, including redacted secrets. You can also specify overrides for specific parameters. ```bash mysorf-base config show ``` ```bash mysorf-base config show logging=rich tracking=wandb runtime.seed=42 ``` -------------------------------- ### Configure Wandb via .env File Source: https://github.com/mysorf-9239/mysorf-base/blob/main/src/mysorf_base/tracking/README.md Set Weights & Biases credentials and configuration using a .env file. This file should not be committed to version control. ```bash # .env (never commit this file) WANDB_PROJECT=my-project WANDB_ENTITY=my-team WANDB_API_KEY=your-key WANDB_MODE=online ``` -------------------------------- ### Runtime Bootstrap with Manual Teardown Source: https://github.com/mysorf-9239/mysorf-base/blob/main/src/mysorf_base/runtime/README.md Manually bootstrap the runtime context and explicitly call the teardown function to release resources. ```python from mysorf_base.runtime import bootstrap, teardown ctx = bootstrap() ctx.logger.info("started") ctx.tracker.start_run(run_name="exp-01") teardown(ctx) ``` -------------------------------- ### mysorf-base CLI: Full bootstrap smoke test Source: https://context7.com/mysorf-9239/mysorf-base/llms.txt Perform a full bootstrap smoke test with the mysorf-base CLI, which prints a RuntimeContext summary. Hydra overrides can be applied. ```bash # Full bootstrap smoke test — prints RuntimeContext summary mysorf-base bootstrap smoke mysorf-base bootstrap smoke logging=rich tracking=disabled profiling=basic ``` -------------------------------- ### Build and Use Rich Logger with Tracebacks Source: https://context7.com/mysorf-9239/mysorf-base/llms.txt Instantiates a logger with the 'rich' backend for pretty terminal output, including syntax highlighting and tracebacks. Requires the 'mysorf-base[logging-rich]' extra. ```python # Rich backend — install extra: pip install "mysorf-base[logging-rich]" cfg_rich = parse_logging_config({"backend": "rich", "level": "DEBUG", "rich_tracebacks": True}) rich_logger = build_logger(cfg_rich, name="train") rich_logger.info("pretty output with syntax highlighting") ``` -------------------------------- ### Use Logger via RuntimeContext Source: https://context7.com/mysorf-9239/mysorf-base/llms.txt Demonstrates accessing a logger configured via the RuntimeContext, specifically using the 'rich' backend. This is the recommended approach for integrating logging. ```python # Via RuntimeContext (recommended) with __import__("mysorf_base.runtime", fromlist=["bootstrap"]).bootstrap(["logging=rich"]) as ctx: ctx.logger.info("all five methods available on ctx.logger") ``` -------------------------------- ### Runtime Bootstrap with Disabled Backends Source: https://github.com/mysorf-9239/mysorf-base/blob/main/src/mysorf_base/runtime/README.md Bootstrap the runtime with specific backends disabled, useful for testing. This results in NullLogger, NullTracker, and NullProfiler instances. ```python ctx = bootstrap(["logging=disabled", "tracking=disabled", "profiling=disabled"]) # ctx.logger → NullLogger # ctx.tracker → NullTracker # ctx.profiler → NullProfiler ``` -------------------------------- ### Running a Sweep via `run_sweep` Source: https://github.com/mysorf-9239/mysorf-base/blob/main/src/mysorf_base/sweeps/README.md The recommended way to run a sweep is by using the `run_sweep` function. This function abstracts away the complexity of setting up the sweep runner and executing trials. ```APIDOC ## run_sweep ### Description Executes a hyperparameter sweep using the provided search space, trial function, runtime context, and sweep configuration. ### Usage ```python from mysorf_base.runtime import bootstrap from mysorf_base.sweeps import SearchSpace, CategoricalParam, SweepsConfig, run_sweep with bootstrap() as ctx: space = SearchSpace(params=[ CategoricalParam(name="lr", values=[0.001, 0.01, 0.1]), ]) cfg = SweepsConfig(strategy="grid") def trial_fn(ctx, params): loss = train_model(lr=float(params["lr"])) return {"loss": loss} summary = run_sweep(space, trial_fn, ctx, cfg) best = summary.best_trial("loss", mode="min") ctx.logger.info(f"Best: {best.override_set}, loss={best.metrics['loss']}") ``` ### Parameters - **space** (SearchSpace): Defines the hyperparameter search space. - **trial_fn** (Callable): The function to execute for each trial. It receives the runtime context and a dictionary of parameters. - **ctx** (RuntimeContext): The runtime context for the sweep. - **cfg** (SweepsConfig): Configuration for the sweep, including strategy and backend. ### Returns - SweepSummary: An object containing the results of all trials. ``` -------------------------------- ### Use W&B Tracker via RuntimeContext Source: https://context7.com/mysorf-9239/mysorf-base/llms.txt Integrates the Weights & Biases (W&B) tracker using the RuntimeContext. Requires 'mysorf-base[tracking-wandb]' and W&B credentials configured. Supports online/offline modes. ```python # W&B tracker via RuntimeContext — pip install "mysorf-base[tracking-wandb]" # Set credentials in .env (never commit): # WANDB_PROJECT=my-project WANDB_ENTITY=my-team WANDB_API_KEY=xxx WANDB_MODE=online from mysorf_base.runtime import bootstrap with bootstrap(["tracking=wandb"]) as ctx: ctx.tracker.start_run(run_name="exp-grid-search-01") ctx.tracker.log_params({"lr": 0.001, "batch_size": 32, "optimizer": "adam"}) for step in range(10): loss = 1.0 / (step + 1) ctx.tracker.log_metrics({"train/loss": loss, "train/auroc": 0.8 + step * 0.01}, step=step) ctx.tracker.log_artifact("/tmp/model.pt", name="best_model") # tracker.finish() called automatically on __exit__ ``` -------------------------------- ### Direct Artifact Manager Construction Source: https://github.com/mysorf-9239/mysorf-base/blob/main/src/mysorf_base/artifacts/README.md Shows how to manually construct an ArtifactManager with a specific configuration, including versioning strategy and run ID. ```python from mysorf_base.artifacts import build_artifact_manager, ArtifactsConfig from mysorf_base.config import compose_typed_config cfg = compose_typed_config() mgr = build_artifact_manager( ArtifactsConfig(backend="local", versioning_strategy="manual"), cfg.paths, run_id="exp-01", ) record = mgr.save("model.pt", "best_model", ArtifactType.CHECKPOINT, version="v1") ``` -------------------------------- ### Import Public API Source: https://github.com/mysorf-9239/mysorf-base/blob/main/src/mysorf_base/runtime/README.md Import the necessary components from the runtime subsystem. ```python from mysorf_base.runtime import bootstrap, teardown, RuntimeContext ``` -------------------------------- ### Usage via RuntimeContext Source: https://github.com/mysorf-9239/mysorf-base/blob/main/src/mysorf_base/tracking/README.md Recommended way to use the tracking subsystem within the application's runtime context. ```APIDOC ## Usage via RuntimeContext ### Description Integrates tracking seamlessly with the application's runtime context, automatically handling tracker initialization and cleanup. ### Example ```python from mysorf_base.runtime import bootstrap with bootstrap(["tracking=wandb"]) as ctx: ctx.tracker.start_run(run_name="baseline") ctx.tracker.log_params({"lr": 0.001, "epochs": 100}) ctx.tracker.log_metrics({"loss": 0.42, "auroc": 0.91}, step=1) # finish() is called automatically on context exit ``` ``` -------------------------------- ### Run Bootstrap Smoke Test Source: https://github.com/mysorf-9239/mysorf-base/blob/main/README.md Execute a full bootstrap smoke test to verify the system's core functionality and print the RuntimeContext summary. ```bash mysorf-base bootstrap smoke ``` -------------------------------- ### Usage via RuntimeContext Source: https://github.com/mysorf-9239/mysorf-base/blob/main/src/mysorf_base/profiling/README.md Recommended usage pattern for the profiling subsystem integrated with the RuntimeContext. ```APIDOC ## Usage via RuntimeContext (recommended) ```python from mysorf_base.runtime import bootstrap with bootstrap(["profiling=basic"]) as ctx: summary = ctx.profiler.profile_records([ {"drug": "A", "score": 1.2}, {"drug": "B", "score": 0.9}, ]) ctx.logger.info(f"rows={summary.row_count}, cols={summary.column_count}") ctx.tracker.log_metrics({"profile_rows": float(summary.row_count)}) with ctx.profiler.time("stage.encode"): _ = [record["drug"] for record in [{"drug": "A"}, {"drug": "B"}]] ctx.profiler.reset_timing() ``` ``` -------------------------------- ### W&B Credentials in .env file Source: https://context7.com/mysorf-9239/mysorf-base/llms.txt Configure Weights & Biases (W&B) credentials in a .env file for tracking experiments. Ensure this file is never committed to version control. ```bash # W&B credentials in .env (never commit) WANDB_PROJECT=my-project WANDB_ENTITY=my-team WANDB_API_KEY=your-key WANDB_MODE=online ``` -------------------------------- ### RuntimeContext Source: https://github.com/mysorf-9239/mysorf-base/blob/main/src/mysorf_base/runtime/README.md Immutable container with all subsystem instances. ```APIDOC ## RuntimeContext ### Description Immutable container with all subsystem instances. ### Fields - **cfg** (AppConfig) - Configuration object. - **run_id** (str) - Unique identifier for the run. - **config_hash** (str) - SHA256 hash of the configuration. - **event_bus** (EventBus) - Lightweight synchronous runtime event bus. - **logger** (Logger Protocol) - Logger instance. - **tracker** (Tracker Protocol) - Tracker instance. - **profiler** (Profiler Protocol) - Profiler instance. - **artifact_manager** (ArtifactManager Protocol) - Artifact manager instance. ### Context Manager Support `RuntimeContext` supports the context manager protocol. `__exit__` calls `teardown()` automatically. ``` -------------------------------- ### Render Configuration as YAML Source: https://github.com/mysorf-9239/mysorf-base/blob/main/src/mysorf_base/config/README.md Renders the composed configuration as a YAML string. Useful for debugging or inspecting the final configuration. ```python from mysorf_base.config import to_yaml print(to_yaml(["tracking=wandb"])) ``` -------------------------------- ### Event Subscription Source: https://github.com/mysorf-9239/mysorf-base/blob/main/src/mysorf_base/checkpoints/README.md Demonstrates how to subscribe to checkpoint save and load events emitted by the CheckpointManager. ```APIDOC ## EventBus Subscription Example ### Description This example shows how to subscribe to `checkpoint.saved` and `checkpoint.loaded` events using the EventBus. ### Example ```python from mysorf_base.checkpoints import build_checkpoint_manager from mysorf_base.runtime import bootstrap with bootstrap() as ctx: manager = build_checkpoint_manager(ctx.artifact_manager, ctx.event_bus) ctx.event_bus.subscribe( "checkpoint.saved", lambda e: ctx.logger.info(f"saved {e.payload['name']} → {e.payload['path']}"), ) ctx.event_bus.subscribe( "checkpoint.loaded", lambda e: ctx.logger.info(f"loaded {e.payload['name']} v{e.payload['version']}"), ) ``` ### Events Emitted - **`checkpoint.saved`**: Emitted when `save_checkpoint()` or `save_checkpoint_file()` is called. Payload keys: `name`, `version`, `epoch`, `path`. - **`checkpoint.loaded`**: Emitted when `load_checkpoint()` is called. Payload keys: `name`, `version`, `epoch`, `path`. ``` -------------------------------- ### Bootstrap Runtime with Manual Teardown Source: https://context7.com/mysorf-9239/mysorf-base/llms.txt Manually manage the `RuntimeContext` lifecycle using `bootstrap()` and `teardown()` when the context manager cannot be used. Ensure `teardown()` is called in a `finally` block. ```python from mysorf_base.runtime import bootstrap, teardown ctx = bootstrap(["logging=console"]) try: ctx.logger.info("started") finally: teardown(ctx) ``` -------------------------------- ### Run Quality Checks Source: https://github.com/mysorf-9239/mysorf-base/blob/main/CONTRIBUTING.md Execute various quality checks including linting, type checking, testing, and building a wheel. The `check` command runs all standard checks. ```bash make lint # ruff check + format check make typecheck # mypy make test # pytest make check # all of the above make smoke-wheel # build wheel + import from an isolated target dir pre-commit run --all-files # full pre-commit pipeline ``` -------------------------------- ### Load Environment Files Source: https://github.com/mysorf-9239/mysorf-base/blob/main/src/mysorf_base/config/README.md Loads environment variables from discovered `.env` files into `os.environ`. ```APIDOC ## load_env_files() ### Description Loads environment variables from `.env` files found in standard locations. This function prioritizes existing OS environment variables, meaning values from `.env` files will not override them. ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```python from mysorf_base.config import load_env_files load_env_files() ``` ### Response #### Success Response (200) This function does not return a value; it modifies `os.environ` in place. #### Response Example None ``` -------------------------------- ### build_tracker() - Tracker Backends Source: https://context7.com/mysorf-9239/mysorf-base/llms.txt Constructs a Tracker protocol instance. Supports 'disabled' and 'wandb' backends. ```APIDOC ## `build_tracker(cfg)` Constructs a `Tracker` protocol instance. The `disabled` backend is a no-op. The `wandb` backend (extra: `tracking-wandb`) wraps the W&B SDK and supports `online`, `offline`, and `disabled` modes. ### Parameters - **cfg** (TrackingConfig) - The tracking configuration. ### Backends - **disabled**: A no-operation tracker. - **wandb**: Wraps the Weights & Biases SDK. Requires `mysorf-base[tracking-wandb]`. Supports `online`, `offline`, and `disabled` modes. ### Example Usage ```python from mysorf_base.tracking import build_tracker, TrackingConfig, NullTracker # Disabled tracker tracker = build_tracker(TrackingConfig(backend="disabled")) tracker.start_run(run_name="local-test") tracker.log_params({"lr": 0.001}) tracker.log_metrics({"loss": 0.5}, step=0) tracker.finish() # no-op # W&B tracker via RuntimeContext # Ensure WANDB_PROJECT, WANDB_ENTITY, WANDB_API_KEY are set in environment variables or .env file from mysorf_base.runtime import bootstrap with bootstrap(["tracking=wandb"]) as ctx: ctx.tracker.start_run(run_name="exp-grid-search-01") ctx.tracker.log_params({"lr": 0.001, "batch_size": 32, "optimizer": "adam"}) for step in range(10): loss = 1.0 / (step + 1) ctx.tracker.log_metrics({"train/loss": loss, "train/auroc": 0.8 + step * 0.01}, step=step) ctx.tracker.log_artifact("/tmp/model.pt", name="best_model") # tracker.finish() called automatically on __exit__ ``` ``` -------------------------------- ### Import Artifacts API Source: https://github.com/mysorf-9239/mysorf-base/blob/main/src/mysorf_base/artifacts/README.md Import necessary components from the mysorf_base.artifacts library to manage artifacts. ```python from mysorf_base.artifacts import ( ArtifactManager, ArtifactNotFoundError, ArtifactRecord, ArtifactType, ArtifactsConfig, NullArtifactManager, VersioningStrategy, build_artifact_manager, parse_artifacts_config, ) ``` -------------------------------- ### mysorf-base CLI: Show effective config Source: https://context7.com/mysorf-9239/mysorf-base/llms.txt Use the mysorf-base CLI to display the effective configuration, with options for secret redaction and applying Hydra overrides. ```bash # Show effective config with secret redaction mysorf-base config show mysorf-base config show logging=rich tracking=wandb runtime.seed=99 mysorf-base config show env=ci logging=structlog ``` -------------------------------- ### Accessing RuntimeContext Fields and Subsystems Source: https://context7.com/mysorf-9239/mysorf-base/llms.txt Demonstrates accessing identity fields, typed configuration attributes, and subsystem instances from the `RuntimeContext` object within a `bootstrap()` context. ```python from mysorf_base.runtime import bootstrap with bootstrap(["logging=console", "tracking=disabled"]) as ctx: # Identity fields print(ctx.run_id) # e.g. "run_20260101_120000_a3f2b1c4" print(ctx.config_hash) # full SHA-256 of the composed config # Typed config — attribute access, never dict subscripting print(ctx.cfg.app.name) # "mysorf-base" print(ctx.cfg.runtime.seed) # 7 (default) print(ctx.cfg.logging.backend) # "console" print(ctx.cfg.tracking.backend) # "disabled" print(ctx.cfg.artifacts.versioning_strategy) # "run_id" # Subsystem instances (Protocol types — backend-agnostic) ctx.logger.info("Logger protocol") ctx.tracker.log_metrics({"loss": 0.1}, step=0) ctx.artifact_manager.save("output.csv", "results", artifact_type=...) ctx.event_bus.subscribe("my.event", lambda e: print(e.payload)) ctx.event_bus.publish("my.event", {"value": 42}) ``` -------------------------------- ### Direct Tracker Construction Source: https://github.com/mysorf-9239/mysorf-base/blob/main/src/mysorf_base/tracking/README.md Manually construct a tracker instance using a TrackingConfig object and the build_tracker function. ```python from mysorf_base.tracking import build_tracker, TrackingConfig cfg = TrackingConfig(backend="disabled") tracker = build_tracker(cfg) tracker.start_run() ``` -------------------------------- ### Subsystem Directory Structure Source: https://github.com/mysorf-9239/mysorf-base/blob/main/CONTRIBUTING.md Illustrates the conventional layout for a subsystem within the mysorf-base project, including core components and backends. ```text src/mysorf_base// ├── __init__.py # Shallow public API exports only ├── README.md # Usage and API reference ├── core/ │ ├── schema.py # @dataclass config schema and data models │ ├── interfaces.py # Protocol definitions │ ├── factory.py # build_xxx() + parse_xxx_config() │ └── validate.py # validate_xxx_config() └── backends/ # Concrete implementations ``` -------------------------------- ### Profiling Subsystem Public API Source: https://github.com/mysorf-9239/mysorf-base/blob/main/src/mysorf_base/profiling/README.md Import necessary components from the profiling subsystem. These include configuration models, summary objects, and factory functions. ```python from mysorf_base.profiling import ( ProfilingConfig, ProfileSummary, NullProfiler, build_profiler, parse_profiling_config, validate_profiling_config, ) ``` -------------------------------- ### Public API Imports Source: https://github.com/mysorf-9239/mysorf-base/blob/main/src/mysorf_base/logging/README.md Import necessary components from the logging module. Ensure these are available in your project. ```python from mysorf_base.logging import ( LoggingConfig, NullLogger, build_logger, parse_logging_config, validate_logging_config, ) ``` -------------------------------- ### build_tracker Source: https://github.com/mysorf-9239/mysorf-base/blob/main/src/mysorf_base/tracking/README.md Constructs a tracker implementation based on the provided TrackingConfig. ```APIDOC ## build_tracker ### Description Builds a tracker instance from a configuration object. ### Parameters #### Request Body - **cfg** (TrackingConfig) - Required - The configuration object specifying the backend and its settings. ``` -------------------------------- ### Load Environment Files Source: https://github.com/mysorf-9239/mysorf-base/blob/main/src/mysorf_base/config/README.md Loads environment variables from discovered .env files into os.environ. Existing OS environment variables take precedence. ```python from mysorf_base.config import load_env_files load_env_files() ``` -------------------------------- ### Backend: Local Source: https://github.com/mysorf-9239/mysorf-base/blob/main/src/mysorf_base/sweeps/README.md The 'local' backend executes sweep trials sequentially within the current process using only standard Python libraries. It integrates with `Tracker` and `ArtifactManager` from `RuntimeContext`. ```APIDOC ## Local Backend ### Description Executes sweep trials sequentially in the current process without external dependencies. Integrates with `RuntimeContext`'s `Tracker` and `ArtifactManager`. ### Behavior When sweeps are disabled via configuration, `run_sweep(...)` returns an empty `SweepSummary` with `skip_reason="sweeps disabled by configuration"` and logs a warning. ```