### TOML Configuration File Example for qlib-data

Source: https://context7.com/qibinliang/qlib-data/llms.txt

This TOML file defines the configuration for the qlib-data tool, covering pipeline settings, download parameters, normalization options, dump configurations, and provider-specific settings like tushare token.

```toml
# config.toml - Full configuration example

[pipeline]
provider = "akshare"          # Data provider: akshare, baostock, tushare
market = "cn"                 # Market: cn, us, hk
freq = "day"                  # Frequency: day, 5min, 1min
start_date = "2000-01-01"     # Download start date
output_dir = "~/.qlib/qlib_data/cn_data"  # Output directory
work_dir = "~/.qlib_data/work"            # Temporary work directory

[download]
max_workers = 1               # Parallel download workers (keep 1 for rate limiting)
delay = 0.5                   # Delay between API requests in seconds
max_retry_rounds = 3          # Retry rounds for failed symbols

[normalize]
max_workers = 16              # Parallel normalization workers
adjust_type = "hfq"           # Price adjustment: hfq (backward), qfq (forward), none

[dump]
max_workers = 16              # Parallel dump workers
exclude_fields = ["symbol", "date"]  # Fields to exclude from binary output

[tushare]
token = ""                    # Tushare API token (or use TUSHARE_TOKEN env var)

```

--------------------------------

### Install Development Dependencies

Source: https://github.com/qibinliang/qlib-data/blob/main/README.md

Installs the qlib-data package with development dependencies, enabling features like testing and linting. This command is essential for developers contributing to the project.

```bash
pip install -e ".[dev]"
```

--------------------------------

### TOML Configuration File

Source: https://context7.com/qibinliang/qlib-data/llms.txt

The tool supports TOML configuration files for complex setups. CLI arguments always override config file values.

```APIDOC
## TOML Configuration File

### Description
The tool supports TOML configuration files for complex setups. CLI arguments always override config file values.

### Method
Configuration File

### Endpoint
N/A (Configuration File)

### Parameters
#### Configuration Fields

**[pipeline]**
- **provider** (string) - Data provider: akshare, baostock, tushare.
- **market** (string) - Market: cn, us, hk.
- **freq** (string) - Frequency: day, 5min, 1min.
- **start_date** (string) - Download start date (YYYY-MM-DD).
- **output_dir** (string) - Output directory for Qlib binary data.
- **work_dir** (string) - Temporary work directory.

**[download]**
- **max_workers** (integer) - Parallel download workers (keep 1 for rate limiting).
- **delay** (float) - Delay between API requests in seconds.
- **max_retry_rounds** (integer) - Retry rounds for failed symbols.

**[normalize]**
- **max_workers** (integer) - Parallel normalization workers.
- **adjust_type** (string) - Price adjustment: hfq (backward), qfq (forward), none.

**[dump]**
- **max_workers** (integer) - Parallel dump workers.
- **exclude_fields** (array of strings) - Fields to exclude from binary output.

**[tushare]**
- **token** (string) - Tushare API token (or use TUSHARE_TOKEN env var).

### Request Example
```toml
# config.toml - Full configuration example

[pipeline]
provider = "akshare"
market = "cn"
freq = "day"
start_date = "2000-01-01"
output_dir = "~/.qlib/qlib_data/cn_data"
work_dir = "~/.qlib_data/work"

[download]
max_workers = 1
delay = 0.5
max_retry_rounds = 3

[normalize]
max_workers = 16
adjust_type = "hfq"

[dump]
max_workers = 16
exclude_fields = ["symbol", "date"]

[tushare]
token = ""
```

### Response
N/A (Configuration File)
```

--------------------------------

### Install qlib-data CLI Tool

Source: https://github.com/qibinliang/qlib-data/blob/main/README.md

Installs the qlib-data command-line interface tool from PyPI or from source for development. Requires Python version 3.10 or higher.

```bash
pip install qlib-data

# From source (development)
git clone https://github.com/QibinLiang/qlib-data.git
cd qlib-data
pip install -e ".[dev]"
```

--------------------------------

### Implement Custom Data Collector

Source: https://context7.com/qibinliang/qlib-data/llms.txt

Shows how to extend `BaseCollector` to create a custom data provider. This involves implementing `get_instrument_list` to specify symbols and `fetch_symbol_data` to retrieve OHLCV data for each symbol. The example demonstrates initializing and running the custom collector.

```python
from pathlib import Path
import pandas as pd
from qlib_data.core.collector import BaseCollector
from qlib_data.core.types import Market, Freq, AdjustType

class CustomCollector(BaseCollector):
    """Custom data collector implementation."""

    def get_instrument_list(self) -> list[str]:
        """Return list of symbols to download."""
        # Implement your symbol list retrieval logic
        return ["SH600000", "SH600001", "SZ000001"]

    def fetch_symbol_data(self, symbol: str) -> pd.DataFrame | None:
        """Fetch OHLCV data for a single symbol."""
        # Implement your data fetching logic
        # Must return DataFrame with columns: date, open, close, high, low, volume
        data = {
            "date": ["2024-01-02", "2024-01-03"],
            "open": [10.0, 10.5],
            "close": [10.5, 11.0],
            "high": [10.8, 11.2],
            "low": [9.9, 10.4],
            "volume": [1000000, 1200000],
            "symbol": [symbol, symbol],
        }
        return pd.DataFrame(data)

# Use the collector
collector = CustomCollector(
    source_dir=Path("./work/source"),
    market=Market.CN,
    freq=Freq.DAY,
    start="2024-01-01",
    end="2024-12-31",
    adjust=AdjustType.HFQ,
    delay=0.5,
    max_retry_rounds=3,
)

# Run collection (returns list of failed symbols)
failed = collector.collect()
print(f"Failed symbols: {failed}")
```

--------------------------------

### Get Index Constituents and Save to CSV

Source: https://context7.com/qibinliang/qlib-data/llms.txt

Demonstrates how to retrieve constituents for CSI300, CSI500, and CSI100 indices using a data manager. It also shows how to save the CSI300 constituents to a CSV file.

```python
from qlib_data.manager import Manager

mgr = Manager()

# Get CSI300 constituents
csi300 = mgr.get_constituents("csi300")
print(f"CSI300 has {len(csi300)} stocks")
print(csi300.head())

# Get CSI500 constituents
csi500 = mgr.get_constituents("csi500")

# Get CSI100 constituents
csi100 = mgr.get_constituents("csi100")

# Save to CSV
csi300.to_csv("csi300_constituents.csv", index=False)
```

--------------------------------

### Convert Stock Symbols Between Formats

Source: https://context7.com/qibinliang/qlib-data/llms.txt

This snippet shows how to use the symbol format conversion utilities in `qlib-data`. It covers converting symbols to Qlib format (e.g., SH600000), converting from Qlib format to Tushare, Baostock, or plain formats, and converting from Tushare or Baostock formats back to Qlib format, including batch conversion examples.

```python
from qlib_data.symbols.formats import (
    to_qlib,
    to_tushare,
    to_baostock,
    to_plain,
    from_tushare,
    from_baostock,
)

# Convert to Qlib format (SH600000)
qlib_sym = to_qlib("600000")        # Auto-infers exchange -> "SH600000"
qlib_sym = to_qlib("000001")        # Auto-infers exchange -> "SZ000001"
qlib_sym = to_qlib("600000", "SH")  # Explicit exchange -> "SH600000"

# Convert from Qlib to other formats
ts_sym = to_tushare("SH600000")     # "600000.SH"
bs_sym = to_baostock("SH600000")    # "sh.600000"
plain = to_plain("SH600000")        # "600000"

# Convert from other formats to Qlib
qlib1 = from_tushare("600000.SH")   # "SH600000"
qlib2 = from_baostock("sh.600000")  # "SH600000"

# Batch conversion example
symbols = ["600000", "000001", "300001"]
qlib_symbols = [to_qlib(s) for s in symbols]
# Result: ["SH600000", "SZ000001", "SZ300001"]
```

--------------------------------

### Implement Custom Data Normalizer

Source: https://context7.com/qibinliang/qlib-data/llms.txt

Illustrates how to extend `BaseNormalizer` for custom data cleaning. The `normalize_symbol` method is implemented to rename columns, convert date formats, sort data, handle missing values, and select required columns. The example shows initialization and execution of the normalizer.

```python
from pathlib import Path
import pandas as pd
from qlib_data.core.normalizer import BaseNormalizer
from qlib_data.core.types import Freq

class CustomNormalizer(BaseNormalizer):
    """Custom data normalizer implementation."""

    def normalize_symbol(self, df: pd.DataFrame, symbol: str) -> pd.DataFrame:
        """Normalize raw data for a single symbol.

        Must return DataFrame with columns:
        date, open, close, high, low, volume, amount
        """
        df = df.copy()

        # Rename columns to standard names
        df = df.rename(columns={
            "trade_date": "date",
            "opening": "open",
            "closing": "close",
            "highest": "high",
            "lowest": "low",
            "vol": "volume",
            "amt": "amount",
        })

        # Convert date format
        df["date"] = pd.to_datetime(df["date"])

        # Sort by date
        df = df.sort_values("date").reset_index(drop=True)

        # Handle missing values
        df = df.dropna(subset=["open", "close", "high", "low"])

        # Select required columns
        cols = ["date", "open", "close", "high", "low", "volume", "amount"]
        df = df[[c for c in cols if c in df.columns]]

        return df

# Use the normalizer
normalizer = CustomNormalizer(
    source_dir=Path("./work/source"),
    normalize_dir=Path("./work/normalize"),
    freq=Freq.DAY,
    max_workers=8,
)

# Run normalization
normalizer.normalize()
```

--------------------------------

### Download Specific Symbols with qlib-data

Source: https://github.com/qibinliang/qlib-data/blob/main/README.md

Downloads OHLCV data for specified stock symbols (e.g., SH600000, SZ000001) starting from a particular date. This command allows targeted data retrieval.

```bash
qlib-data download --provider akshare --symbols SH600000,SZ000001 --start 2020-01-01
```

--------------------------------

### Load and Analyze Data with Qlib

Source: https://context7.com/qibinliang/qlib-data/llms.txt

Demonstrates how to initialize Qlib with a data directory and load financial data. It shows how to fetch daily features for specific instruments and fields, load data for all instruments, and perform basic calculations like daily returns.

```python
import qlib
from qlib.data import D

# Initialize Qlib with your data directory
qlib.init(provider_uri="~/.qlib/qlib_data/cn_data")

# Load daily features for specific stocks
df = D.features(
    instruments=["SH600000", "SZ000001"],
    fields=["$open", "$close", "$high", "$low", "$volume"],
    start_time="2023-01-01",
    end_time="2024-01-01",
    freq="day",
)
print(df.head())

# Load data for all instruments
all_instruments = D.instruments(market="all")
df_all = D.features(
    instruments=all_instruments,
    fields=["$close", "$volume"],
    start_time="2024-01-01",
    end_time="2024-01-31",
)

# Calculate returns
df["returns"] = df["$close"].pct_change()
```

--------------------------------

### Configure and Run Data Download Pipeline

Source: https://context7.com/qibinliang/qlib-data/llms.txt

This snippet demonstrates how to configure and execute a full or incremental data download pipeline using the `qlib-data` library. It covers setting up pipeline parameters like provider, market, date range, and output directories, as well as running the pipeline in full or incremental modes, or executing individual stages like download, normalize, and dump.

```python
from qlib_data.config.models import AppConfig, PipelineConfig, DownloadConfig
from qlib_data.core.types import Market, Freq
from qlib_data.pipeline import Pipeline

config = AppConfig(
    pipeline=PipelineConfig(
        provider="akshare",
        market=Market.CN,
        freq=Freq.DAY,
        start_date="2020-01-01",
        end_date="2024-01-01",
        output_dir="~/.qlib/qlib_data/cn_data",
        work_dir="~/.qlib_data/work",
    ),
    download=DownloadConfig(
        delay=0.5,
        max_retry_rounds=3,
        symbols=["SH600000", "SZ000001"],  # Optional: specific symbols
    ),
)

pipeline = Pipeline(config)

# Run full download
pipeline.run(incremental=False)

# Or run incremental update
pipeline.run(incremental=True)

# Run individual stages
failed_symbols = pipeline.run_download()  # Returns list of failed symbols
pipeline.run_normalize()
pipeline.run_dump(incremental=False)
```

--------------------------------

### Load and Manage Configuration

Source: https://context7.com/qibinliang/qlib-data/llms.txt

This section details how to load application configurations using `qlib-data`. It covers loading from TOML files with automatic path expansion and Pydantic validation, accessing configuration values, loading default configurations, and manually creating and resolving configurations.

```python
from qlib_data.config.loader import load_config
from qlib_data.config.models import AppConfig, PipelineConfig, NormalizeConfig
from qlib_data.core.types import AdjustType

# Load from TOML file
config = load_config("./config.toml")

# Access configuration values
print(f"Provider: {config.pipeline.provider}")
print(f"Output: {config.pipeline.output_dir}")
print(f"Adjust type: {config.normalize.adjust_type}")

# Load default configuration (no file)
default_config = load_config(None)

# Manually create configuration
config = AppConfig(
    pipeline=PipelineConfig(
        provider="baostock",
        start_date="2015-01-01",
    ),
    normalize=NormalizeConfig(
        adjust_type=AdjustType.QFQ,  # Forward-adjusted prices
        max_workers=8,
    ),
)
config.resolve_paths()  # Expand ~ and env vars
```

--------------------------------

### Initialize Qlib Data Provider

Source: https://github.com/qibinliang/qlib-data/blob/main/README.md

Initializes the Qlib data provider with a specified URI and demonstrates fetching features for a given symbol. This is a fundamental step for using Qlib for data analysis.

```python
import qlib
from qlib.data import D

qlib.init(provider_uri="~/.qlib/qlib_data/cn_data")
df = D.features(["SH600000"], fields=["$close", "$volume"], freq="day")
```

--------------------------------

### Download Command - Full Data Download

Source: https://context7.com/qibinliang/qlib-data/llms.txt

The primary command downloads A-share market data from a specified provider and converts it to Qlib binary format. Supports custom date ranges, specific symbols, and multiple data providers.

```APIDOC
## Download Command - Full Data Download

### Description
The primary command downloads A-share market data from a specified provider and converts it to Qlib binary format. Supports custom date ranges, specific symbols, and multiple data providers.

### Method
CLI Command

### Endpoint
N/A (CLI Tool)

### Parameters
#### Query Parameters
- **--provider** (string) - Required - Data provider (e.g., akshare, baostock, tushare).
- **--market** (string) - Optional - Market identifier (e.g., cn, us, hk). Defaults to 'cn'.
- **--freq** (string) - Optional - Data frequency (e.g., day, 5min, 1min). Defaults to 'day'.
- **--symbols** (string) - Optional - Comma-separated list of stock symbols.
- **--start** (string) - Optional - Start date for data download (YYYY-MM-DD).
- **--end** (string) - Optional - End date for data download (YYYY-MM-DD).
- **--tushare-token** (string) - Optional - Tushare API token. Can also be set via environment variable TUSHARE_TOKEN.
- **--output** (string) - Optional - Output directory for Qlib binary data.
- **--config** (string) - Optional - Path to a TOML configuration file.
- **--delay** (float) - Optional - Delay in seconds between API requests.

### Request Example
```bash
# Download all A-share daily data using free akshare provider
qlib-data download --provider akshare --market cn --freq day

# Download specific symbols with custom date range
qlib-data download --provider akshare --symbols SH600000,SZ000001 --start 2020-01-01 --end 2024-01-01

# Download using tushare with API token
qlib-data download --provider tushare --tushare-token YOUR_API_TOKEN --output ~/.qlib/qlib_data/cn_data

# Download using baostock provider
qlib-data download --provider baostock --market cn --freq day --delay 1.0

# Download with custom configuration file
qlib-data download --config ./config.toml
```

### Response
#### Success Response (0)
Data is downloaded and converted to Qlib binary format in the specified output directory.

#### Response Example
N/A (CLI Output)
```

--------------------------------

### Download Data with Custom Configuration File

Source: https://context7.com/qibinliang/qlib-data/llms.txt

This command enables downloading data using a custom TOML configuration file. CLI arguments take precedence over settings in the configuration file.

```bash
qlib-data download --config ./config.toml
```

--------------------------------

### Download Data using baostock Provider

Source: https://context7.com/qibinliang/qlib-data/llms.txt

This command utilizes the baostock provider to download A-share market data. It specifies the market as 'cn' and frequency as 'day', with an optional delay parameter to control request frequency.

```bash
qlib-data download --provider baostock --market cn --freq day --delay 1.0
```

--------------------------------

### Download A-share Daily Data with akshare

Source: https://github.com/qibinliang/qlib-data/blob/main/README.md

Downloads all A-share daily OHLCV data using the free akshare provider. This command fetches data for the Chinese market ('cn') at a daily frequency ('day').

```bash
qlib-data download --provider akshare --market cn --freq day
```

--------------------------------

### Download Data using tushare with API Token

Source: https://context7.com/qibinliang/qlib-data/llms.txt

This command downloads A-share market data using the tushare provider, which requires an API token for access. It also allows specifying an output directory for the Qlib-formatted data.

```bash
qlib-data download --provider tushare --tushare-token YOUR_API_TOKEN --output ~/.qlib/qlib_data/cn_data
```

--------------------------------

### Python Pipeline Class for Data Orchestration

Source: https://context7.com/qibinliang/qlib-data/llms.txt

This Python code snippet demonstrates the import of necessary classes for orchestrating the data pipeline using qlib-data. It includes AppConfig, PipelineConfig, Pipeline, and type definitions for Market and Freq.

```python
from pathlib import Path
from qlib_data.config.models import AppConfig, PipelineConfig, DownloadConfig
from qlib_data.core.pipeline import Pipeline
from qlib_data.core.types import Market, Freq

```

--------------------------------

### Pipeline Class - Orchestrating Data Downloads

Source: https://context7.com/qibinliang/qlib-data/llms.txt

The Pipeline class orchestrates the full data pipeline: download, normalize, and dump stages. It dynamically loads the correct provider implementation.

```APIDOC
## Pipeline Class - Orchestrating Data Downloads

### Description
The Pipeline class orchestrates the full data pipeline: download, normalize, and dump stages. It dynamically loads the correct provider implementation.

### Method
Python API

### Endpoint
N/A (Python Class)

### Parameters
#### Initialization Parameters
- **config** (AppConfig) - Application configuration object.
- **pipeline_config** (PipelineConfig) - Pipeline specific configuration.
- **download_config** (DownloadConfig) - Download stage configuration.

### Request Example
```python
from pathlib import Path
from qlib_data.config.models import AppConfig, PipelineConfig, DownloadConfig
from qlib_data.core.pipeline import Pipeline
from qlib_data.core.types import Market, Freq

# Example configuration (replace with actual configuration)
app_config = AppConfig(
    work_dir=Path("~/.qlib_data/work").expanduser(),
    output_dir=Path("~/.qlib/qlib_data/cn_data").expanduser()
)

pipeline_config = PipelineConfig(
    provider="akshare",
    market=Market.CN,
    freq=Freq.DAY
)

download_config = DownloadConfig(
    start_date="2020-01-01",
    end_date="2024-01-01"
)

# Instantiate the pipeline
pipeline = Pipeline(
    config=app_config,
    pipeline_config=pipeline_config,
    download_config=download_config
)

# Run the pipeline (download, normalize, dump)
pipeline.run()

print("Data pipeline completed.")
```

### Response
#### Success Response
Data is processed through the download, normalize, and dump stages, resulting in Qlib-compatible binary files.

#### Response Example
```
Data pipeline completed.
```
```

--------------------------------

### Lint and Format Code with Ruff

Source: https://github.com/qibinliang/qlib-data/blob/main/README.md

Applies linting and formatting rules to the source code and tests using the Ruff tool. This ensures code consistency and adherence to project standards.

```bash
ruff check src/ tests/
ruff format src/ tests/
```

--------------------------------

### Index Command - Download Index Constituents

Source: https://context7.com/qibinliang/qlib-data/llms.txt

Fetches constituent lists for major Chinese stock indices (CSI300, CSI500, CSI100) and outputs them in CSV format.

```APIDOC
## Index Command - Download Index Constituents

### Description
Fetches constituent lists for major Chinese stock indices (CSI300, CSI500, CSI100) and outputs them in CSV format.

### Method
CLI Command

### Endpoint
N/A (CLI Tool)

### Parameters
#### Query Parameters
- **--name** (string) - Required - Name of the index (e.g., csi300, csi500, csi100).
- **--output** (string) - Optional - Path to save the output CSV file. If not provided, output goes to stdout.

### Request Example
```bash
# Download CSI300 index constituents
qlib-data index --name csi300 --output csi300.csv

# Download CSI500 index constituents to stdout
qlib-data index --name csi500

# Download CSI100 index constituents
qlib-data index --name csi100 --output csi100_constituents.csv
```

### Response
#### Success Response (0)
Index constituents are downloaded and saved to a CSV file or printed to stdout.

#### Response Example
```csv
# Example output for CSI300
code,name,start_date,end_date
600000.SH,浦发银行,2005-01-01,2005-03-31
600004.SH,白云机场,2005-01-01,2005-03-31
...
```
```

--------------------------------

### Download Specific Symbols with Date Range using akshare

Source: https://context7.com/qibinliang/qlib-data/llms.txt

This command allows for targeted data downloads by specifying particular stock symbols and a custom date range. It uses the akshare provider and is useful for downloading historical data for specific assets.

```bash
qlib-data download --provider akshare --symbols SH600000,SZ000001 --start 2020-01-01 --end 2024-01-01
```

--------------------------------

### Configure qlib-data with TOML File

Source: https://github.com/qibinliang/qlib-data/blob/main/README.md

Defines download and processing parameters in a TOML configuration file. This allows for persistent settings that can be overridden by CLI arguments.

```toml
[pipeline]
provider = "akshare"
market = "cn"
freq = "day"
start_date = "2000-01-01"
output_dir = "~/.qlib/qlib_data/cn_data"
work_dir = "~/.qlib_data/work"

[download]
max_workers = 1
delay = 0.5
max_retry_rounds = 3

[normalize]
max_workers = 16
adjust_type = "hfq"    # hfq = backward-adjusted, qfq = forward-adjusted

[dump]
max_workers = 16
exclude_fields = ["symbol", "date"]

[tushare]
token = ""  # or set TUSHARE_TOKEN env var

```

--------------------------------

### Download Data using Tushare with API Token

Source: https://github.com/qibinliang/qlib-data/blob/main/README.md

Downloads A-share data using the tushare provider, which requires an API token. The token can be provided directly via the command line or set as an environment variable.

```bash
qlib-data download --provider tushare --tushare-token YOUR_TOKEN
```

--------------------------------

### Use Configuration File with CLI

Source: https://github.com/qibinliang/qlib-data/blob/main/README.md

Applies settings defined in a TOML configuration file to a qlib-data command. CLI arguments provided alongside the --config flag will override the file settings.

```bash
qlib-data download --config path/to/config.toml
```

--------------------------------

### Update Dataset with Custom Delay using akshare

Source: https://context7.com/qibinliang/qlib-data/llms.txt

This command incrementally updates a Qlib dataset using the akshare provider, with a custom delay set between API requests to manage rate limits. It also specifies the output directory.

```bash
qlib-data update --provider akshare --delay 0.8 --output ~/.qlib/qlib_data/cn_data
```

--------------------------------

### Download Data

Source: https://github.com/qibinliang/qlib-data/blob/main/README.md

Downloads A-share data from specified providers and converts it into Qlib's binary format. Supports various markets, frequencies, and date ranges.

```APIDOC
## POST /qlib-data download

### Description
Downloads A-share data and converts to Qlib binary format.

### Method
CLI Command

### Endpoint
`qlib-data download`

### Parameters
#### Query Parameters
- **--provider, -p** (string) - Optional - Data provider (`akshare`, `baostock`, `tushare`). Default: `akshare`
- **--market, -m** (string) - Optional - Market (`cn`, `us`, `hk`). Default: `cn`
- **--freq, -f** (string) - Optional - Frequency (`day`, `5min`, `1min`). Default: `day`
- **--start** (string) - Optional - Start date (YYYY-MM-DD). Default: `2000-01-01`
- **--end** (string) - Optional - End date (YYYY-MM-DD). Default: `today`
- **--output, -o** (string) - Optional - Output directory. Default: `~/.qlib/qlib_data/cn_data`
- **--symbols** (string) - Optional - Comma-separated symbols (e.g. `SH600000,SZ000001`). Default: `all`
- **--config** (string) - Optional - Path to TOML config file.
- **--delay** (float) - Optional - Delay between API requests (seconds). Default: `0.5`
- **--tushare-token** (string) - Optional - Tushare API token. Default: env `TUSHARE_TOKEN`

### Request Example
```bash
qlib-data download --provider akshare --market cn --freq day
qlib-data download --provider tushare --tushare-token YOUR_TOKEN --symbols SH600000,SZ000001 --start 2020-01-01
```

### Response
#### Success Response (0)
Data is downloaded and converted to Qlib binary format in the specified output directory.

#### Response Example
(No specific JSON response, output is files in the file system)
```

--------------------------------

### Update Command - Incremental Updates

Source: https://context7.com/qibinliang/qlib-data/llms.txt

The update command detects the latest date in an existing Qlib dataset and only fetches new data since that date, saving time and API requests.

```APIDOC
## Update Command - Incremental Updates

### Description
The update command detects the latest date in an existing Qlib dataset and only fetches new data since that date, saving time and API requests.

### Method
CLI Command

### Endpoint
N/A (CLI Tool)

### Parameters
#### Query Parameters
- **--provider** (string) - Required - Data provider (e.g., akshare, baostock, tushare).
- **--output** (string) - Optional - Path to the existing Qlib data directory to update. Defaults to a standard location if not specified.
- **--tushare-token** (string) - Optional - Tushare API token. Can also be set via environment variable TUSHARE_TOKEN.
- **--delay** (float) - Optional - Delay in seconds between API requests.

### Request Example
```bash
# Incrementally update existing dataset
qlib-data update --provider akshare --output ~/.qlib/qlib_data/cn_data

# Update with tushare provider
qlib-data update --provider tushare --tushare-token YOUR_TOKEN --output ~/.qlib/qlib_data/cn_data

# Update with custom delay between requests
qlib-data update --provider akshare --delay 0.8 --output ~/.qlib/qlib_data/cn_data
```

### Response
#### Success Response (0)
New data since the last update is fetched and added to the Qlib dataset.

#### Response Example
N/A (CLI Output)
```

--------------------------------

### Update Dataset using tushare Provider

Source: https://context7.com/qibinliang/qlib-data/llms.txt

This command incrementally updates a Qlib dataset using the tushare provider, requiring an API token. It specifies the output directory for the updated data.

```bash
qlib-data update --provider tushare --tushare-token YOUR_TOKEN --output ~/.qlib/qlib_data/cn_data
```

--------------------------------

### Check Command - Data Integrity Validation

Source: https://context7.com/qibinliang/qlib-data/llms.txt

Validates the structure and integrity of a Qlib data directory, checking calendars, instruments, and binary feature files.

```APIDOC
## Check Command - Data Integrity Validation

### Description
Validates the structure and integrity of a Qlib data directory, checking calendars, instruments, and binary feature files.

### Method
CLI Command

### Endpoint
N/A (CLI Tool)

### Parameters
#### Query Parameters
- **--path** (string) - Optional - Path to the Qlib data directory to check. Defaults to a standard location if not specified.

### Request Example
```bash
# Check default data directory
qlib-data check

# Check specific data directory
qlib-data check --path ~/.qlib/qlib_data/cn_data

# Check custom data location
qlib-data check --path /data/qlib/custom_dataset
```

### Response
#### Success Response (0)
Indicates that the data directory is valid and all checks passed.

#### Response Example
```
Data directory /path/to/qlib_data is valid.
- Trading calendars checked.
- Instrument lists checked.
- Feature files integrity verified.
```
```

--------------------------------

### Index Constituents

Source: https://github.com/qibinliang/qlib-data/blob/main/README.md

Downloads lists of index constituents for specified stock market indices.

```APIDOC
## POST /qlib-data index

### Description
Download index constituent lists.

### Method
CLI Command

### Endpoint
`qlib-data index`

### Parameters
#### Query Parameters
- **--name, -n** (string) - Required - Index name (`csi300`, `csi500`, `csi100`)
- **--output, -o** (string) - Optional - Output CSV file path. Default: `stdout`

### Request Example
```bash
qlib-data index --name csi300 --output csi300.csv
```

### Response
#### Success Response (0)
Index constituent list is output to the specified file or standard output.

#### Response Example
(CSV formatted data)
```

--------------------------------

### Run Tests with Pytest

Source: https://github.com/qibinliang/qlib-data/blob/main/README.md

Executes the test suite for the qlib-data project using the pytest framework. This command verifies the integrity and correctness of the codebase.

```bash
python -m pytest
```

--------------------------------

### Download CSI500 Index Constituents to stdout

Source: https://context7.com/qibinliang/qlib-data/llms.txt

This command fetches the constituent list for the CSI500 index and prints it directly to standard output. This is useful for piping the output to other commands or for quick inspection.

```bash
qlib-data index --name csi500
```

--------------------------------

### Trading Calendar Operations with CalendarManager

Source: https://context7.com/qibinliang/qlib-data/llms.txt

This code snippet illustrates how to use the `CalendarManager` in `qlib-data` to manage trading calendars. It covers initializing the manager, reading existing calendars, retrieving the latest trading date, writing new calendar dates, and merging new dates with existing calendars for different frequencies.

```python
from pathlib import Path
import pandas as pd
from qlib_data.calendar.manager import CalendarManager
from qlib_data.core.types import Freq

# Initialize with Qlib data directory
cal_mgr = CalendarManager(Path("~/.qlib/qlib_data/cn_data").expanduser())

# Read existing calendar
calendar = cal_mgr.read_calendar(Freq.DAY)
print(f"Calendar has {len(calendar)} trading days")
print(f"First date: {calendar[0]}")
print(f"Last date: {calendar[-1]}")

# Get the latest date (useful for incremental updates)
latest = cal_mgr.get_latest_date(Freq.DAY)
if latest:
    print(f"Latest date in calendar: {latest}")

# Write new calendar
new_dates = [pd.Timestamp("2024-01-02"), pd.Timestamp("2024-01-03")]
cal_path = cal_mgr.write_calendar(new_dates, Freq.DAY)

# Merge new dates with existing calendar
merged = cal_mgr.merge_calendar(new_dates, Freq.DAY)
print(f"Merged calendar: {len(merged)} dates")
```

--------------------------------

### Manage Symbol Lists with SymbolManager

Source: https://context7.com/qibinliang/qlib-data/llms.txt

This code snippet demonstrates the usage of `SymbolManager` for managing stock symbol lists, including caching. It shows how to initialize the manager with a cache directory, load symbols from cache, save symbols to cache, and filter symbol lists based on inclusion and exclusion criteria for a specified market.

```python
from pathlib import Path
from qlib_data.symbols.manager import SymbolManager
from qlib_data.core.types import Market

# Initialize with cache directory
mgr = SymbolManager(cache_dir=Path("~/.qlib_data/cache").expanduser())

# Load cached symbols
cached = mgr.load_cached(Market.CN)
if cached:
    print(f"Loaded {len(cached)} cached symbols")

# Save symbols to cache
symbols = ["SH600000", "SH600001", "SZ000001"]
mgr.save_cache(symbols, Market.CN)

# Filter symbols
filtered = mgr.filter_symbols(
    symbols=["SH600000", "SH600001", "SZ000001", "SZ000002"],
    include=["SH600000", "SZ000001"],  # Only keep these
)
# Result: ["SH600000", "SZ000001"]

# Exclude specific symbols
filtered = mgr.filter_symbols(
    symbols=["SH600000", "SH600001", "SZ000001"],
    exclude=["SH600001"],
)
# Result: ["SH600000", "SZ000001"]
```

--------------------------------

### Fetch Index Constituents with CnIndexManager

Source: https://context7.com/qibinliang/qlib-data/llms.txt

This snippet shows how to fetch index constituents for major Chinese stock indices (CSI300, CSI500, CSI100) using the `CnIndexManager` from `qlib-data`, which utilizes the `akshare` provider for data retrieval.

```python
from qlib_data.index.cn_index import CnIndexManager

mgr = CnIndexManager()

# Example usage (assuming you want to fetch CSI300 constituents):
# csi300_constituents = mgr.fetch_constituents(index_name="csi300")
# print(csi300_constituents)
```

--------------------------------

### Check Default Data Directory Integrity

Source: https://context7.com/qibinliang/qlib-data/llms.txt

This command validates the integrity and structure of the default Qlib data directory. It checks for consistency in calendars, instruments, and feature files.

```bash
qlib-data check
```

--------------------------------

### Download Index Constituent Lists

Source: https://github.com/qibinliang/qlib-data/blob/main/README.md

Downloads membership lists for specified stock indices (e.g., CSI300, CSI500, CSI100) and outputs them to a CSV file or standard output.

```bash
qlib-data index --name csi300 --output csi300.csv
```

--------------------------------

### Perform Incremental Data Update

Source: https://github.com/qibinliang/qlib-data/blob/main/README.md

Updates an existing Qlib data directory by fetching only new data since the last recorded date. This is efficient for keeping datasets up-to-date.

```bash
qlib-data update --provider akshare --output ~/.qlib/qlib_data/cn_data
```

--------------------------------

### Download CSI100 Index Constituents

Source: https://context7.com/qibinliang/qlib-data/llms.txt

This command downloads the constituent list for the CSI100 index and saves it to a specified CSV file. It allows for easy management of index membership data.

```bash
qlib-data index --name csi100 --output csi100_constituents.csv
```

--------------------------------

### Check Custom Data Location Integrity

Source: https://context7.com/qibinliang/qlib-data/llms.txt

This command validates the integrity of a Qlib data directory located at a custom path. It ensures that the specified dataset is correctly structured for Qlib.

```bash
qlib-data check --path /data/qlib/custom_dataset
```

--------------------------------

### Update Data

Source: https://github.com/qibinliang/qlib-data/blob/main/README.md

Incrementally updates an existing Qlib dataset by downloading only new data since the last recorded date.

```APIDOC
## POST /qlib-data update

### Description
Incrementally update an existing Qlib dataset. Detects the latest date in the existing calendar and only downloads new data since that date.

### Method
CLI Command

### Endpoint
`qlib-data update`

### Parameters
#### Query Parameters
- **--provider, -p** (string) - Optional - Data provider. Default: `akshare`
- **--output, -o** (string) - Optional - Existing Qlib data directory. Default: `~/.qlib/qlib_data/cn_data`
- **--config** (string) - Optional - Path to TOML config file.
- **--delay** (float) - Optional - Delay between API requests (seconds). Default: `0.5`
- **--tushare-token** (string) - Optional - Tushare API token. Default: env `TUSHARE_TOKEN`

### Request Example
```bash
qlib-data update --provider akshare --output ~/.qlib/qlib_data/cn_data
```

### Response
#### Success Response (0)
Existing Qlib data is updated with new data in the specified output directory.

#### Response Example
(No specific JSON response, output is files in the file system)
```

--------------------------------

### Type Check Code with MyPy

Source: https://github.com/qibinliang/qlib-data/blob/main/README.md

Performs static type checking on the qlib-data project's source code using MyPy. This helps catch type-related errors before runtime.

```bash
mypy src/qlib_data/
```

--------------------------------

### Check Data Integrity

Source: https://github.com/qibinliang/qlib-data/blob/main/README.md

Validates the integrity of a Qlib binary data directory.

```APIDOC
## POST /qlib-data check

### Description
Validate the integrity of a Qlib binary data directory.

### Method
CLI Command

### Endpoint
`qlib-data check`

### Parameters
#### Query Parameters
- **--path, -p** (string) - Optional - Qlib data directory to check. Default: `~/.qlib/qlib_data/cn_data`

### Request Example
```bash
qlib-data check --path ~/.qlib/qlib_data/cn_data
```

### Response
#### Success Response (0)
Indicates that the data directory integrity check passed.

#### Response Example
(Console output indicating success or specific errors)
```

--------------------------------

### Check Qlib Data Integrity

Source: https://github.com/qibinliang/qlib-data/blob/main/README.md

Validates the integrity of a Qlib binary data directory. This command checks for consistency in calendars, instruments, and file alignment.

```bash
qlib-data check --path ~/.qlib/qlib_data/cn_data
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.