### Setup Environment Source: https://github.com/yandex-research/tab-ddpm/blob/main/README.md Commands to create and configure the Conda environment for TabDDPM. ```bash export REPO_DIR=/path/to/the/code cd $REPO_DIR conda create -n tddpm python=3.9.7 conda activate tddpm pip install torch==1.10.1+cu111 -f https://download.pytorch.org/whl/torch_stable.html pip install -r requirements.txt # if the following commands do not succeed, update conda conda env config vars set PYTHONPATH=${PYTHONPATH}:${REPO_DIR} conda env config vars set PROJECT_DIR=${REPO_DIR} conda deactivate conda activate tddpm ``` -------------------------------- ### Set up local development environment Source: https://github.com/yandex-research/tab-ddpm/blob/main/CTGAN/CTGAN/CONTRIBUTING.rst Commands to create a virtual environment and install the package for development. ```shell $ mkvirtualenv CTGAN $ cd CTGAN/ $ make install-develop ``` -------------------------------- ### Run TabDDPM Tuning Source: https://github.com/yandex-research/tab-ddpm/blob/main/README.md Template and example for tuning TabDDPM hyperparameters. ```bash python scripts/tune_ddpm.py [ds_name] [train_size] synthetic [catboost|mlp] [exp_name] --eval_seeds python scripts/tune_ddpm.py churn2 6500 synthetic catboost ddpm_tune --eval_seeds ``` -------------------------------- ### Run TabDDPM Pipeline Source: https://github.com/yandex-research/tab-ddpm/blob/main/README.md Template and example for executing the training, sampling, and evaluation pipeline. ```bash python scripts/pipeline.py --config [path_to_your_config] --train --sample --eval python scripts/pipeline.py --config exp/churn2/ddpm_cb_best/config.toml --train --sample ``` -------------------------------- ### Install CTGAN via pip Source: https://github.com/yandex-research/tab-ddpm/blob/main/CTGAN/CTGAN/README.md Use this command to install the standalone CTGAN library using the pip package manager. ```bash pip install ctgan ``` -------------------------------- ### TabDDPM Configuration Example Source: https://github.com/yandex-research/tab-ddpm/blob/main/CONFIG_DESCRIPTION.md A complete TOML configuration file defining model parameters, diffusion settings, training hyperparameters, and preprocessing policies. ```toml seed = 0 parent_dir = "exp/abalone/check" real_data_path = "data/abalone/" model_type = "mlp" num_numerical_features = 7 device = "cuda:0" [model_params] is_y_cond = false d_in = 11 num_classes = 0 [model_params.rtdl_params] d_layers = [ 256, 256, ] dropout = 0.0 [diffusion_params] num_timesteps = 1000 gaussian_loss_type = "mse" scheduler = "cosine" [train.main] steps = 1000 lr = 0.001 weight_decay = 1e-05 batch_size = 4096 [train.T] seed = 0 normalization = "quantile" num_nan_policy = "__none__" cat_nan_policy = "__none__" cat_min_frequency = "__none__" cat_encoding = "__none__" y_policy = "default" [sample] num_samples = 20800 batch_size = 10000 seed = 0 [eval.type] eval_model = "catboost" eval_type = "synthetic" [eval.T] seed = 0 normalization = "__none__" num_nan_policy = "__none__" cat_nan_policy = "__none__" cat_min_frequency = "__none__" cat_encoding = "__none__" y_policy = "default" ``` -------------------------------- ### Specify development dependencies Source: https://github.com/yandex-research/tab-ddpm/blob/main/CTGAN/CTGAN/CONTRIBUTING.rst Examples of how to include development versions in project dependencies. ```python install_requires = [ ... 'ctgan>=X.Y.Z.dev', ... ] ``` ```bash pip install 'ctgan>=X.Y.Z.dev' ``` -------------------------------- ### Install CTGAN via conda Source: https://github.com/yandex-research/tab-ddpm/blob/main/CTGAN/CTGAN/README.md Use this command to install the standalone CTGAN library using the conda package manager with the required channels. ```bash conda install -c pytorch -c conda-forge ctgan ``` -------------------------------- ### Run Evaluation Over Seeds Source: https://github.com/yandex-research/tab-ddpm/blob/main/README.md Template and example for evaluating models using multiple sampling and evaluation seeds. ```bash python scripts/eval_seeds.py --config [path_to_your_config] [n_eval_seeds] [ddpm|smote|ctabgan|ctabgan-plus|tvae] synthetic [catboost|mlp] [n_sample_seeds] python scripts/eval_seeds.py --config exp/churn2/ddpm_cb_best/config.toml 10 ddpm synthetic catboost 5 ``` -------------------------------- ### Generate and view documentation Source: https://github.com/yandex-research/tab-ddpm/blob/main/CTGAN/CTGAN/CONTRIBUTING.rst Command to build and preview the project documentation locally. ```shell $ make view-docs ``` -------------------------------- ### Train and Sample with CTGANSynthesizer Source: https://github.com/yandex-research/tab-ddpm/blob/main/CTGAN/CTGAN/README.md Demonstrates loading a demo dataset, configuring discrete columns, fitting the model, and generating synthetic data. ```python3 from ctgan import CTGANSynthesizer from ctgan import load_demo data = load_demo() # Names of the columns that are discrete discrete_columns = [ 'workclass', 'education', 'marital-status', 'occupation', 'relationship', 'race', 'sex', 'native-country', 'income' ] ctgan = CTGANSynthesizer(epochs=10) ctgan.fit(data, discrete_columns) # Synthetic copy samples = ctgan.sample(1000) ``` -------------------------------- ### Run full test suite Source: https://github.com/yandex-research/tab-ddpm/blob/main/CTGAN/CTGAN/CONTRIBUTING.rst Command to verify changes across all supported Python versions. ```shell $ make test-all ``` -------------------------------- ### Upload release candidate Source: https://github.com/yandex-research/tab-ddpm/blob/main/CTGAN/CTGAN/CONTRIBUTING.rst Command to build and upload a pre-release version to PyPi. ```makefile make release-candidate ``` -------------------------------- ### Execute release commands Source: https://github.com/yandex-research/tab-ddpm/blob/main/CTGAN/CTGAN/CONTRIBUTING.rst Makefile commands to trigger different types of project releases. ```makefile make release ``` ```makefile make release-minor ``` ```makefile make release-major ``` -------------------------------- ### Download Datasets Source: https://github.com/yandex-research/tab-ddpm/blob/main/README.md Commands to download and extract the datasets used in the paper. ```bash conda activate tddpm cd $PROJECT_DIR wget "https://www.dropbox.com/s/rpckvcs3vx7j605/data.tar?dl=0" -O data.tar tar -xvf data.tar ``` -------------------------------- ### Run quality assurance and testing commands Source: https://github.com/yandex-research/tab-ddpm/blob/main/CTGAN/CTGAN/CONTRIBUTING.rst Commands to lint code, run tests, and generate coverage reports. ```shell $ make lint # Check code styling $ make test # Run the tests $ make coverage # Get the coverage report ``` -------------------------------- ### Build the Conda Package Source: https://github.com/yandex-research/tab-ddpm/blob/main/CTGAN/CTGAN/conda/README.md Builds the package using the specified channels after checking out the stable branch. ```bash git checkout stable cd conda conda build -c sdv-dev -c pytorch -c conda-forge . ``` -------------------------------- ### Upload Package to Anaconda Source: https://github.com/yandex-research/tab-ddpm/blob/main/CTGAN/CTGAN/conda/README.md Authenticates with the Anaconda server and uploads the built package to the sdv-dev channel. ```bash anaconda login anaconda upload -u sdv-dev ``` -------------------------------- ### Configure Problem Type Source: https://github.com/yandex-research/tab-ddpm/blob/main/CTAB-GAN-Plus/README.md Configuration syntax for specifying the dataset problem type, or setting it to None. ```python problem_type= {None: None} ``` -------------------------------- ### Create a development branch Source: https://github.com/yandex-research/tab-ddpm/blob/main/CTGAN/CTGAN/CONTRIBUTING.rst Command to create a new branch for your specific bugfix or feature. ```shell $ git checkout -b name-of-your-bugfix-or-feature ``` -------------------------------- ### Commit and push changes Source: https://github.com/yandex-research/tab-ddpm/blob/main/CTGAN/CTGAN/CONTRIBUTING.rst Standard git commands to stage, commit, and push your local changes to the remote repository. ```shell $ git add . $ git commit -m "Your detailed description of your changes." $ git push origin name-of-your-bugfix-or-feature ``` -------------------------------- ### Clone the CTGAN repository Source: https://github.com/yandex-research/tab-ddpm/blob/main/CTGAN/CTGAN/CONTRIBUTING.rst Use this command to clone your fork of the CTGAN repository locally. ```shell $ git clone git@github.com:your_name_here/CTGAN.git ``` -------------------------------- ### Required Package Versions Source: https://github.com/yandex-research/tab-ddpm/blob/main/CTAB-GAN-Plus/README.md List of specific package versions required to ensure compatibility with the model's implementation. ```text numpy==1.21.0 torch==1.9.1 pandas==1.2.4 sklearn==0.24.1 dython==0.6.4.post1 scipy==1.4.1 ``` -------------------------------- ### Adjust Image Side Size for Large Datasets Source: https://github.com/yandex-research/tab-ddpm/blob/main/CTAB-GAN-Plus/README.md Configuration list defining the side sizes of the image-like format used for encoding data. Enlarge this list to accommodate datasets with a larger number of columns. ```python sides = [4, 8, 16, 24, 32] ``` -------------------------------- ### Aggregate experiment results into a DataFrame Source: https://github.com/yandex-research/tab-ddpm/blob/main/agg_results.ipynb Iterates through defined datasets and algorithm paths to load JSON metrics and populate a pandas DataFrame. Requires the lib module and specific directory structures for experiment outputs. ```python import os import lib import numpy as np import pandas as pd DATASETS = [ "abalone", "adult", "buddy", "california", "cardio", "churn2", "default", "diabetes", "fb-comments", "gesture", "higgs-small", "house", "insurance", "king", "miniboone", "wilt" ] _REGRESSION = [ "abalone", "california", "fb-comments", "house", "insurance", "king", ] method2exp = { "real": "exp/{}/ddpm_cb_best/", "tab-ddpm": "exp/{}/ddpm_cb_best/", "smote": "exp/{}/smote/", "ctabgan+": "exp/{}/ctabgan-plus/", "ctabgan": "exp/{}/ctabgan/", "tvae": "exp/{}/tvae/" } eval_file = "eval_catboost.json" show_std = False df = pd.DataFrame(columns=["method"] + [_[:3].upper() for _ in DATASETS]) for algo in method2exp: algo_res = [] for ds in DATASETS: if not os.path.exists(os.path.join(method2exp[algo].format(ds), eval_file)): algo_res.append("--") continue metric = "r2" if ds in _REGRESSION else "f1" res_dict = lib.load_json(os.path.join(method2exp[algo].format(ds), eval_file)) if algo == "real": res = f'{res_dict["real"]["test"][metric + "-mean"]:.4f}' if show_std: res += f'+-{res_dict["real"]["test"][metric + "-std"]:.4f}' else: res = f'{res_dict["synthetic"]["test"][metric + "-mean"]:.4f}' if show_std: res += f'+-{res_dict["synthetic"]["test"][metric + "-std"]:.4f}' algo_res.append(res) df.loc[len(df)] = [algo] + algo_res ``` ```python df ``` -------------------------------- ### Run test subsets with pytest Source: https://github.com/yandex-research/tab-ddpm/blob/main/CTGAN/CTGAN/CONTRIBUTING.rst Commands to execute specific test modules or filter tests by keyword. ```bash $ python -m pytest tests.test_ctgan $ python -m pytest -k 'foo' ``` -------------------------------- ### Bibtex citation Source: https://github.com/yandex-research/tab-ddpm/blob/main/CTAB-GAN/README.md Bibtex entry for citing the CTAB-GAN paper. ```bibtex @InProceedings{zhao21, title = {CTAB-GAN: Effective Table Data Synthesizing}, author = {Zhao, Zilong and Kunar, Aditya and Birke, Robert and Chen, Lydia Y.}, booktitle = {Proceedings of The 13th Asian Conference on Machine Learning}, pages = {97--112}, year = {2021}, editor = {Balasubramanian, Vineeth N. and Tsang, Ivor}, volume = {157}, series = {Proceedings of Machine Learning Research}, month = {17--19 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v157/zhao21a/zhao21a.pdf}, url = {https://proceedings.mlr.press/v157/zhao21a.html} } ``` -------------------------------- ### Bibtex Citation Source: https://github.com/yandex-research/tab-ddpm/blob/main/CTAB-GAN-Plus/README.md Bibtex entry for citing the CTAB-GAN+ paper. ```bibtex @article{zhao2022ctab, title={CTAB-GAN+: Enhancing Tabular Data Synthesis}, author={Zhao, Zilong and Kunar, Aditya and Birke, Robert and Chen, Lydia Y}, journal={arXiv preprint arXiv:2204.00401}, year={2022} } ``` -------------------------------- ### Cite CTGAN in LaTeX Source: https://github.com/yandex-research/tab-ddpm/blob/main/CTGAN/CTGAN/README.md Use this BibTeX entry to cite the CTGAN research paper in academic publications. ```LaTeX @inproceedings{xu2019modeling, title={Modeling Tabular data using Conditional GAN}, author={Xu, Lei and Skoularidou, Maria and Cuesta-Infante, Alfredo and Veeramachaneni, Kalyan}, booktitle={Advances in Neural Information Processing Systems}, year={2019} } ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.