ztxexp

https://github.com/ztxtech/ztxexp
Admin
Experiment framework for ML/LLM workflows with reproducible run artifacts, streamlining parameter...
Tokens:3,310
Snippets:50
Trust Score:7.4
License:MIT
Update:4 months ago
Show doc for...
Tokens:
Raw
### Initialize Experiment Template

Source: https://github.com/ztxtech/ztxexp/blob/main/CHANGELOG.md

Use this command to start an interactive wizard for generating an experiment script scaffold. It guides you through a 7-question process to assemble the experiment structure.

```bash
ztxexp init-template
```

--------------------------------

### Preview Skill Setup

Source: https://github.com/ztxtech/ztxexp/blob/main/docs_src/vibe-coding.zh.md

Previews the skill setup, allowing specification of language and inclusion of OpenAI related configurations.

```bash
ztxexp show-skill --language bilingual
```

```bash
ztxexp show-skill --language zh --with-openai
```

--------------------------------

### Preview Vibe Coding Setup

Source: https://github.com/ztxtech/ztxexp/blob/main/docs_src/vibe-coding.zh.md

Previews the Vibe Coding setup without making changes to the file system. Useful for verifying configurations before applying them.

```bash
ztxexp show-vibe --profile webcoding --language bilingual
```

--------------------------------

### Generate Template with Defaults

Source: https://github.com/ztxtech/ztxexp/blob/main/docs_src/vibe-coding.zh.md

Generates an experiment template and accepts all recommended default configurations during the interactive setup.

```bash
ztxexp init-template --name my_experiment --yes
```

--------------------------------

### Initialize Skill in Specific Target Directory

Source: https://github.com/ztxtech/ztxexp/blob/main/docs_src/vibe-coding.zh.md

Installs the built-in skill, targeting a specific directory ('skills/', '.codex/skills/', or 'both') without interactive prompts.

```bash
ztxexp init-skill --target skills
```

```bash
ztxexp init-skill --target codex
```

```bash
ztxexp init-skill --target both
```

--------------------------------

### Upgrade ZTXEXP Package

Source: https://github.com/ztxtech/ztxexp/blob/main/CHANGELOG.md

Installs or upgrades the ZTXEXP package to the latest version from pip.

```bash
pip install -U ztxexp
```

--------------------------------

### Force Skill Removal

Source: https://github.com/ztxtech/ztxexp/blob/main/docs_src/vibe-coding.zh.md

Removes the installed skills, including unmanaged directories. Use with caution as it can delete user-created content.

```bash
ztxexp remove-skill --force --target both
```

--------------------------------

### Remove Skill

Source: https://github.com/ztxtech/ztxexp/blob/main/CHANGELOG.md

Removes a managed skill from your project. By default, it only deletes managed installs and skips unmanaged directories unless --force is used.

```bash
ztxexp remove-skill
```

--------------------------------

### Initialize Skill Management

Source: https://github.com/ztxtech/ztxexp/blob/main/CHANGELOG.md

Use this command to initialize skill management within your project. It can be used interactively or non-interactively.

```bash
ztxexp init-skill
```

--------------------------------

### Initialize Vibe Agent Integration

Source: https://github.com/ztxtech/ztxexp/blob/main/CHANGELOG.md

Initializes the Vibe agent integration within your project. This command helps set up agent guidance and persistence modules.

```bash
ztxexp init-vibe
```

--------------------------------

### Perform Safe Initialization (Dry Run)

Source: https://github.com/ztxtech/ztxexp/blob/main/docs_src/vibe-coding.zh.md

Executes initialization commands in a dry-run mode, simulating the actions without modifying the file system. This is useful for safety checks before applying changes.

```bash
ztxexp init-vibe --dry-run
```

```bash
ztxexp init-skill --dry-run --target both
```

```bash
ztxexp remove-skill --dry-run --target both
```

```bash
ztxexp init-template --name my_experiment --no-interactive --dry-run
```

--------------------------------

### Preview Template Generation (Dry Run)

Source: https://github.com/ztxtech/ztxexp/blob/main/docs_src/vibe-coding.zh.md

Simulates the generation of an experiment template without writing any files. This is useful for checking the planned output and configuration in a non-interactive environment.

```bash
ztxexp init-template --name my_experiment --no-interactive --dry-run
```

--------------------------------

### Show Skill Information

Source: https://github.com/ztxtech/ztxexp/blob/main/CHANGELOG.md

Displays information about the managed skills in your project.

```bash
ztxexp show-skill
```

--------------------------------

### 记录过程指标

Source: https://github.com/ztxtech/ztxexp/blob/main/docs_src/user-manual.zh.md

使用 ctx.log_metric 记录 step 级曲线，例如每个 epoch 的 loss/acc。这会写入 metrics.jsonl 并触发 tracker 回调。

```python
ctx.log_metric(step=1, metrics={"loss": 0.92, "acc": 0.71}, split="train", phase="fit")
```

--------------------------------

### 构建实验配置

Source: https://github.com/ztxtech/ztxexp/blob/main/docs_src/user-manual.zh.md

使用 ExperimentPipeline 构建参数空间，包括网格搜索、变体配置和排除已完成的实验。

```python
from ztxexp import ExperimentPipeline

pipeline = (
    ExperimentPipeline("./results_demo", base_config={"seed": 42})
    .grid({"lr": [1e-3, 1e-2]})
    .variants([{"model": "tiny"}, {"model": "base"}])
    .exclude_completed()
)
```

--------------------------------

### Generate Experiment Skeleton Protocol

Source: https://github.com/ztxtech/ztxexp/blob/main/CHANGELOG.md

This command generates a basic structure for your experiment scripts, including configuration files, main script, and module stubs.

```bash
ztxexp init-template --name my_experiment --no-interactive
```

--------------------------------

### Build Project Distribution

Source: https://github.com/ztxtech/ztxexp/blob/main/CHANGELOG.md

Builds the project distribution packages without isolation.

```bash
python -m build --no-isolation
```

--------------------------------

### Check Distribution Packages

Source: https://github.com/ztxtech/ztxexp/blob/main/CHANGELOG.md

Verifies the integrity and format of the built distribution packages.

```bash
twine check dist/*
```

--------------------------------

### 主动跳过实验

Source: https://github.com/ztxtech/ztxexp/blob/main/docs_src/user-manual.zh.md

使用 raise SkipRun 抛出异常，主动跳过当前 run。这会记录 skip 事件到 events.jsonl。

```python
raise SkipRun("reason")
```

--------------------------------

### Run Skill CLI and Behavior Tests

Source: https://github.com/ztxtech/ztxexp/blob/main/CHANGELOG.md

Executes automated tests for the CLI and behavior of skill management workflows.

```bash
tests/test_cli_skill.py
```

--------------------------------

### 返回最终指标

Source: https://github.com/ztxtech/ztxexp/blob/main/docs_src/user-manual.zh.md

在 exp_fn 中返回一个字典，包含最终指标，例如 score。这会自动落到 metrics.json。

```python
return {"score": 0.93}
```

--------------------------------

### Run Template Smoke Tests

Source: https://github.com/ztxtech/ztxexp/blob/main/CHANGELOG.md

Executes a suite of smoke tests for the template generation workflow.

```bash
tests/test_templates_smoke.py
```

--------------------------------

### ztxexp 单次实验函数签名

Source: https://github.com/ztxtech/ztxexp/blob/main/docs_src/user-manual.zh.md

ztxexp 的单次实验函数固定签名，接收 RunContext 并返回字典或 None。

```python
def exp_fn(ctx: RunContext) -> dict | None:
    ...
```

--------------------------------

### 编写 exp_fn 示例

Source: https://github.com/ztxtech/ztxexp/blob/main/docs_src/user-manual.zh.md

一个完整的 exp_fn 示例，演示如何获取配置、记录过程指标、写入业务产物以及返回最终指标。

```python
from pathlib import Path
from ztxexp import RunContext


def exp_fn(ctx: RunContext) -> dict | None:
    lr = float(ctx.config["lr"])
    model = str(ctx.config["model"])

    # 过程指标
    ctx.log_metric(step=1, metrics={"loss": 0.8}, split="train", phase="fit")

    # 业务产物
    artifact = Path(ctx.run_dir) / "artifacts" / "summary.txt"
    artifact.write_text(f"run={ctx.run_id}, model={model}, lr={lr}\n", encoding="utf-8")

    # 最终指标
    return {"score": round(1.0 - lr, 4)}
```

--------------------------------

### Show Vibe Agent Integration Status

Source: https://github.com/ztxtech/ztxexp/blob/main/CHANGELOG.md

Displays the current status and configuration of the Vibe agent integration.

```bash
ztxexp show-vibe
```

--------------------------------

### 分析实验结果

Source: https://github.com/ztxtech/ztxexp/blob/main/docs_src/user-manual.zh.md

使用 ResultAnalyzer 将实验结果转换为 DataFrame，包括成功运行的实验和曲线指标。

```python
from ztxexp import ResultAnalyzer

analyzer = ResultAnalyzer("./results_demo")
df = analyzer.to_dataframe(statuses=("succeeded",))
curve_df = analyzer.to_curve_dataframe(metric_key="loss")
```

--------------------------------

### Run CLI Template Tests

Source: https://github.com/ztxtech/ztxexp/blob/main/CHANGELOG.md

Executes automated tests for the CLI workflow of template generation.

```bash
tests/test_cli_template.py
```

--------------------------------

### Run CI Checks

Source: https://github.com/ztxtech/ztxexp/blob/main/CHANGELOG.md

Executes a series of checks for the CI workflow, including linting, testing, building, and distribution checks.

```bash
ruff check .
```

```bash
pytest
```

```bash
mkdocs build --strict
```

```bash
python -m build --no-isolation
```

```bash
twine check dist/*
```

--------------------------------

### Remove Vibe Agent Integration

Source: https://github.com/ztxtech/ztxexp/blob/main/CHANGELOG.md

Removes the Vibe agent integration from your project.

```bash
ztxexp remove-vibe
```
Tokens:
Raw