### Install SDMetrics from source Source: https://github.com/sdv-dev/sdmetrics/blob/main/INSTALL.md Clones the repository and installs the stable branch on Unix-based systems. ```bash git clone https://github.com/sdv-dev/SDMetrics cd SDMetrics git checkout stable make install ``` -------------------------------- ### Install SDMetrics Source: https://github.com/sdv-dev/sdmetrics/blob/main/README.md Installation commands for pip and conda environments. ```bash pip install sdmetrics ``` ```bash conda install -c conda-forge sdmetrics ``` -------------------------------- ### Install SDMetrics from Source Source: https://github.com/sdv-dev/sdmetrics/blob/main/RELEASE.md Clone the repository and install development requirements. Ensure your virtual environment is activated before running. ```bash git clone https://github.com/sdv-dev/SDMetrics.git cd SDMetrics git checkout main make install-develop ``` -------------------------------- ### Install SDMetrics with pip Source: https://github.com/sdv-dev/sdmetrics/blob/main/INSTALL.md Installs the latest stable release from PyPi. ```bash pip install sdmetrics ``` -------------------------------- ### Set Up Local Development Environment Source: https://github.com/sdv-dev/sdmetrics/blob/main/CONTRIBUTING.rst Install the project in a virtual environment for local development. ```shell $ mkvirtualenv SDMetrics $ cd SDMetrics/ $ make install-develop ``` -------------------------------- ### Install SDMetrics for development Source: https://github.com/sdv-dev/sdmetrics/blob/main/INSTALL.md Clones the repository and installs the package in development mode from the main branch. ```bash git clone git@github.com:sdv-dev/SDMetrics cd SDMetrics git checkout main git checkout -b make install-develp ``` -------------------------------- ### Manage release candidates Source: https://github.com/sdv-dev/sdmetrics/blob/main/CONTRIBUTING.rst Commands to build, upload, and install pre-release versions. ```bash make release-candidate ``` ```bash pip install 'sdmetrics>=X.Y.Z.dev' ``` -------------------------------- ### Install SDMetrics with conda Source: https://github.com/sdv-dev/sdmetrics/blob/main/INSTALL.md Installs the latest stable release from the sdv-dev and conda-forge channels. ```bash conda install -c sdv-dev -c conda-forge sdmetrics ``` -------------------------------- ### Get Available Single Column Metrics Source: https://github.com/sdv-dev/sdmetrics/blob/main/sdmetrics/single_column/README.md Use the `get_subclasses` method from the `SingleColumnMetric` base class to retrieve a dictionary of all implemented single column metrics. ```python3 In [1]: from sdmetrics.single_column import SingleColumnMetric In [2]: SingleColumnMetric.get_subclasses() Out[2]: {'CSTest': sdmetrics.single_column.statistical.cstest.CSTest, 'KSComplement': sdmetrics.single_column.statistical.kscomplement.KSComplement} ``` -------------------------------- ### GET /sdmetrics/single_table/subclasses Source: https://github.com/sdv-dev/sdmetrics/blob/main/sdmetrics/single_table/README.md Retrieves a list of all available SingleTableMetric subclasses, which include various efficacy, detection, and privacy metrics. ```APIDOC ## GET /sdmetrics/single_table/subclasses ### Description Retrieves a dictionary mapping the names of all available SingleTableMetric subclasses to their respective class paths. ### Method GET ### Endpoint /sdmetrics/single_table/subclasses ### Response #### Success Response (200) - **subclasses** (object) - A dictionary where keys are metric names and values are the corresponding class paths. #### Response Example { "BNLogLikelihood": "sdmetrics.single_table.bayesian_network.BNLogLikelihood", "KSComplement": "sdmetrics.single_table.multi_single_column.KSComplement", "ContinuousKLDivergence": "sdmetrics.single_table.multi_column_pairs.ContinuousKLDivergence" } ``` -------------------------------- ### Get All Single Table Metrics Source: https://github.com/sdv-dev/sdmetrics/blob/main/sdmetrics/single_table/README.md Use `SingleTableMetric.get_subclasses()` to retrieve a dictionary of all available single table metrics in the SDMetrics library. This is useful for discovering and iterating over available metrics. ```python In [1]: from sdmetrics.single_table import SingleTableMetric In [2]: SingleTableMetric.get_subclasses() Out[2]: {'BNLogLikelihood': sdmetrics.single_table.bayesian_network.BNLogLikelihood, 'LogisticDetection': sdmetrics.single_table.detection.sklearn.LogisticDetection, 'SVCDetection': sdmetrics.single_table.detection.sklearn.SVCDetection, 'BinaryDecisionTreeClassifier': sdmetrics.single_table.efficacy.binary.BinaryDecisionTreeClassifier, 'BinaryAdaBoostClassifier': sdmetrics.single_table.efficacy.binary.BinaryAdaBoostClassifier, 'BinaryLogisticRegression': sdmetrics.single_table.efficacy.binary.BinaryLogisticRegression, 'BinaryMLPClassifier': sdmetrics.single_table.efficacy.binary.BinaryMLPClassifier, 'MulticlassDecisionTreeClassifier': sdmetrics.single_table.efficacy.multiclass.MulticlassDecisionTreeClassifier, 'MulticlassMLPClassifier': sdmetrics.single_table.efficacy.multiclass.MulticlassMLPClassifier, 'LinearRegression': sdmetrics.single_table.efficacy.regression.LinearRegression, 'MLPRegressor': sdmetrics.single_table.efficacy.regression.MLPRegressor, 'GMLogLikelihood': sdmetrics.single_table.gaussian_mixture.GMLogLikelihood, 'CSTest': sdmetrics.single_table.multi_single_column.CSTest, 'KSComplement': sdmetrics.single_table.multi_single_column.KSComplement, 'ContinuousKLDivergence': sdmetrics.single_table.multi_column_pairs.ContinuousKLDivergence, 'DiscreteKLDivergence': sdmetrics.single_table.multi_column_pairs.DiscreteKLDivergence, 'CategoricalCAP': sdmetrics.single_table.privacy.cap, 'CategoricalGeneralizedCAP': sdmetrics.single_table.privacy.cap, 'CategoricalZeroCAP': sdmetrics.single_table.privacy.cap, 'CategoricalKNN': sdmetrics.single_table.privacy.cap, 'CategoricalNB': sdmetrics.single_table.privacy.cap, 'CategoricalRF': sdmetrics.single_table.privacy.cap, 'CategoricalEnsemble': sdmetrics.single_table.privacy.ensemble, 'NumericalLR': sdmetrics.single_table.privacy.numerical_sklearn, 'NumericalMLP': sdmetrics.single_table.privacy.numerical_sklearn, 'NumericalSVR': sdmetrics.single_table.privacy.numerical_sklearn, 'NumericalRadiusNearestNeighbor': sdmetrics.single_table.privacy.radius_nearest_neighbor} ``` -------------------------------- ### View Documentation Locally Source: https://github.com/sdv-dev/sdmetrics/blob/main/CONTRIBUTING.rst Generate and view the project documentation locally to check its appearance. ```shell $ make view-docs ``` -------------------------------- ### Execute release commands Source: https://github.com/sdv-dev/sdmetrics/blob/main/CONTRIBUTING.rst Use make commands to automate the release process for different version types. ```bash make release ``` ```bash make release-minor ``` ```bash make release-major ``` -------------------------------- ### Create Release Candidate Source: https://github.com/sdv-dev/sdmetrics/blob/main/RELEASE.md Use the GitHub Actions 'Release' workflow to create a release candidate. Ensure 'Release candidate' is checked and 'Test PyPI' is not. -------------------------------- ### Run Tests and Linting Source: https://github.com/sdv-dev/sdmetrics/blob/main/RELEASE.md Execute the test suite and linting checks. The tests must complete without errors. Warnings are acceptable. ```bash make test && make lint ``` -------------------------------- ### Run All Tests Source: https://github.com/sdv-dev/sdmetrics/blob/main/CONTRIBUTING.rst Ensure your changes pass all styling checks and tests across supported Python versions. ```shell $ make test-all ``` -------------------------------- ### Clone SDMetrics Repository Source: https://github.com/sdv-dev/sdmetrics/blob/main/CONTRIBUTING.rst Clone your forked repository locally to begin development. ```shell $ git clone git@github.com:your_name_here/SDMetrics.git ``` -------------------------------- ### Compute KSComplement Metric on Demo Data Source: https://github.com/sdv-dev/sdmetrics/blob/main/sdmetrics/single_column/README.md Demonstrates how to load demo data, extract specific columns as numpy arrays, and compute the KSComplement metric using the `compute` method. Ensure data is in 1D numpy array format. ```python3 In [3]: from sdmetrics import load_demo In [4]: real_data, synthetic_data, metadata = load_demo() In [5]: from sdmetrics.single_column import KSComplement In [6]: real_column = real_data['users']['age'].to_numpy() In [7]: synthetic_column = synthetic_data['users']['age'].to_numpy() In [8]: KSComplement.compute(real_column, synthetic_column) Out[8]: 0.8 ``` -------------------------------- ### Load Demo Datasets Source: https://context7.com/sdv-dev/sdmetrics/llms.txt Use load_demo to retrieve sample real and synthetic datasets for single-table, multi-table, or time series testing. ```python from sdmetrics import load_demo # Load single-table demo (student placements data) real_data, synthetic_data, metadata = load_demo(modality='single_table') print(f"Real data shape: {real_data.shape}") print(f"Synthetic data shape: {synthetic_data.shape}") print(f"Columns: {list(real_data.columns)}") # Real data shape: (215, 17) # Synthetic data shape: (215, 17) # Columns: ['student_id', 'gender', 'second_perc', 'high_perc', 'high_spec', ...] # Load multi-table demo (users, sessions, transactions) real_tables, synthetic_tables, multi_metadata = load_demo(modality='multi_table') print(f"Tables: {list(real_tables.keys())}") # Tables: ['users', 'sessions', 'transactions'] # Load time series demo real_ts, synthetic_ts, ts_metadata = load_demo(modality='timeseries') ``` -------------------------------- ### Update HISTORY.md Source: https://github.com/sdv-dev/sdmetrics/blob/main/RELEASE.md Run the 'Release Prep' workflow to automatically update HISTORY.md. Manually review and ensure all issues from the milestone are included. ```markdown # History ## X.Y.Z (YYYY-MM-DD) ### New Features * - [Issue #](https://github.com/sdv-dev/SDMetrics/issues/) by @resolver ### General Improvements * - [Issue #](https://github.com/sdv-dev/SDMetrics/issues/) by @resolver ### Bug Fixed * - [Issue #](https://github.com/sdv-dev/SDMetrics/issues/) by @resolver ``` -------------------------------- ### Generate Quality Report Source: https://github.com/sdv-dev/sdmetrics/blob/main/README.md Load demo data and generate a quality report for single-table synthetic data. ```python from sdmetrics import load_demo from sdmetrics.reports.single_table import QualityReport real_data, synthetic_data, metadata = load_demo(modality='single_table') my_report = QualityReport() my_report.generate(real_data, synthetic_data, metadata) ``` -------------------------------- ### Check Release Readiness Source: https://github.com/sdv-dev/sdmetrics/blob/main/RELEASE.md Verify that the release can be made after updating HISTORY.md. This command checks for any potential issues before final release. ```bash make check-release ``` -------------------------------- ### Create SDV Test Branch Source: https://github.com/sdv-dev/sdmetrics/blob/main/RELEASE.md Create a new branch in the SDV repository to test the SDMetrics release candidate. Update pyproject.toml to specify the candidate version. ```bash git checkout -b test-sdmetrics-X.Y.Z ``` ```toml 'sdmetrics>=X.Y.Z.dev0' ``` ```bash git push --set-upstream origin test-sdmetrics-X.Y.Z ``` -------------------------------- ### Compute Metric with Metadata Source: https://github.com/sdv-dev/sdmetrics/blob/main/sdmetrics/single_table/README.md Demonstrates providing a metadata dictionary to define the target column for the metric calculation. ```python In[11]: users_metadata = metadata['tables']['users'].copy() In[12]: users_metadata['target'] = 'country' In[13]: MulticlassDecisionTreeClassifier.compute( real_table, synthetic_table, metadata=users_metadata ) Out[13]: (0.05555555555555555,) ``` -------------------------------- ### Run Development Checks Source: https://github.com/sdv-dev/sdmetrics/blob/main/CONTRIBUTING.rst Execute linting, tests, and coverage reports to ensure code quality and test coverage. ```shell $ make lint # Check code styling $ make test # Run the tests $ make coverage # Get the coverage report ``` -------------------------------- ### Create a Feature Branch Source: https://github.com/sdv-dev/sdmetrics/blob/main/CONTRIBUTING.rst Create a new branch for your bug fix or feature development, following a specific naming convention. ```shell $ git checkout -b name-of-your-bugfix-or-feature ``` -------------------------------- ### Try Except Pass in SDMetrics Visualization Source: https://github.com/sdv-dev/sdmetrics/blob/main/static_code_analysis.txt This snippet demonstrates a 'try, except, pass' block. This pattern can hide exceptions, making debugging difficult. Consider logging or re-raising exceptions if appropriate. ```python except Exception: pass ``` -------------------------------- ### Commit and Push Changes Source: https://github.com/sdv-dev/sdmetrics/blob/main/CONTRIBUTING.rst Stage, commit, and push your local changes to your feature branch on GitHub. ```shell $ git add . $ git commit -m "Your detailed description of your changes." $ git push origin name-of-your-bugfix-or-feature ``` -------------------------------- ### Compute LogisticDetection Metric Source: https://github.com/sdv-dev/sdmetrics/blob/main/sdmetrics/single_table/README.md Demonstrates basic usage of the LogisticDetection metric on a specific table from demo data. ```python3 In [3]: from sdmetrics.single_table import LogisticDetection In [4]: from sdmetrics import load_demo In [5]: real_data, synthetic_data, metadata = load_demo() In [6]: real_table = real_data['users'] In [7]: synthetic_table = synthetic_data['users'] In [8]: LogisticDetection.compute(real_table, synthetic_table) Out[8]: 1.0 ``` -------------------------------- ### KSComplement Metric Source: https://context7.com/sdv-dev/sdmetrics/llms.txt Compares continuous distributions using the Kolmogorov-Smirnov test. Returns 1 minus the KS statistic. ```python from sdmetrics.single_column import KSComplement import numpy as np import pandas as pd # Generate sample continuous data np.random.seed(42) real_values = pd.Series(np.random.normal(100, 15, 1000)) synthetic_good = pd.Series(np.random.normal(100, 15, 1000)) # Similar distribution synthetic_bad = pd.Series(np.random.normal(120, 30, 1000)) # Different distribution # Compare distributions good_score = KSComplement.compute(real_values, synthetic_good) bad_score = KSComplement.compute(real_values, synthetic_bad) print(f"Similar distribution score: {good_score:.4f}") print(f"Different distribution score: {bad_score:.4f}") # Similar distribution score: 0.9650 # Different distribution score: 0.5120 # Normalize the score (already normalized for this metric) normalized = KSComplement.normalize(good_score) print(f"Normalized score: {normalized:.4f}") ``` -------------------------------- ### Execute Multiple Metrics Simultaneously Source: https://context7.com/sdv-dev/sdmetrics/llms.txt Runs a batch of metrics on the provided data using the compute_metrics function. ```python from sdmetrics import compute_metrics, load_demo from sdmetrics.single_column import ( BoundaryAdherence, KSComplement, CategoryCoverage, TVComplement ) real_data, synthetic_data, metadata = load_demo(modality='single_table') ``` -------------------------------- ### Compute Boundary Adherence Metric Source: https://github.com/sdv-dev/sdmetrics/blob/main/README.md Calculate the boundary adherence metric for a single column. ```python # calculate whether the synthetic data respects the min/max bounds # set by the real data from sdmetrics.single_column import BoundaryAdherence BoundaryAdherence.compute(real_data['start_date'], synthetic_data['start_date']) ``` -------------------------------- ### Generate Multi-Table Quality Report Source: https://context7.com/sdv-dev/sdmetrics/llms.txt Use the QualityReport class to evaluate synthetic data against real data, retrieve property scores, table details, and visualizations. ```python report = QualityReport() report.generate(real_data, synthetic_data, metadata, verbose=True) # Output: # (1/4) Evaluating Column Shapes: 100% # Column Shapes Score: 85.2% # (2/4) Evaluating Column Pair Trends: 100% # Column Pair Trends Score: 78.5% # (3/4) Evaluating Cardinality: 100% # Cardinality Score: 92.1% # (4/4) Evaluating Intertable Trends: 100% # Intertable Trends Score: 80.3% # Overall Score (Average): 84.03% # Get scores by property print(report.get_properties()) # Get details for a specific table users_details = report.get_details(property_name='Column Shapes', table_name='users') print(users_details) # Get visualization for a specific table fig = report.get_visualization(property_name='Column Shapes', table_name='transactions') fig.show() ``` -------------------------------- ### Retrieve Time Series Metric Subclasses Source: https://github.com/sdv-dev/sdmetrics/blob/main/sdmetrics/timeseries/README.md Use the base class to discover all available time series metric implementations. ```python3 In [1]: from sdmetrics.timeseries import TimeSeriesMetric In [2]: TimeSeriesMetric.get_subclasses() Out[2]: {'LSTMDetection': sdmetrics.timeseries.detection.LSTMDetection} ``` -------------------------------- ### Save and Load Reports Source: https://github.com/sdv-dev/sdmetrics/blob/main/README.md Persist a generated report to a file and reload it for later use. ```python my_report.save(filepath='demo_data_quality_report.pkl') # load it at any point in the future my_report = QualityReport.load(filepath='demo_data_quality_report.pkl') ``` -------------------------------- ### Compute Multiple Metrics for a Single Column Source: https://context7.com/sdv-dev/sdmetrics/llms.txt Use this to compute all defined metrics at once for a single column of real and synthetic data. Ensure metrics are defined in a dictionary. ```python metrics = { 'boundary': BoundaryAdherence, 'ks_complement': KSComplement, 'category_coverage': CategoryCoverage, 'tv_complement': TVComplement } real_column = real_data['high_perc'] synthetic_column = synthetic_data['high_perc'] results = compute_metrics( metrics=metrics, real_data=real_column, synthetic_data=synthetic_column ) print(results[['metric', 'name', 'normalized_score', 'goal']]) ``` -------------------------------- ### Evaluate Synthetic Data with Detection Metrics Source: https://context7.com/sdv-dev/sdmetrics/llms.txt Uses classifiers to determine if synthetic data can be distinguished from real data. Scores near 1.0 indicate high quality. ```python from sdmetrics import load_demo from sdmetrics.single_table import LogisticDetection, SVCDetection real_data, synthetic_data, metadata = load_demo(modality='single_table') # Compute detection score using Logistic Regression logistic_score = LogisticDetection.compute(real_data, synthetic_data, metadata) print(f"Logistic Detection Score: {logistic_score:.4f}") # Score close to 1.0 means synthetic is hard to distinguish (good) # Score close to 0.0 means synthetic is easily distinguished (bad) # Alternative: Use SVM classifier svc_score = SVCDetection.compute(real_data, synthetic_data, metadata) print(f"SVC Detection Score: {svc_score:.4f}") ``` -------------------------------- ### Locate MultiTableMetric subclasses Source: https://github.com/sdv-dev/sdmetrics/blob/main/sdmetrics/multi_table/README.md Retrieve all available multi-table metric classes using the get_subclasses method. ```python3 In [1]: from sdmetrics.multi_table import MultiTableMetric In [2]: MultiTableMetric.get_subclasses() Out[2]: {'CSTest': sdmetrics.multi_table.multi_single_table.CSTest, 'KSComplement': sdmetrics.multi_table.multi_single_table.KSComplement, 'LogisticDetection': sdmetrics.multi_table.multi_single_table.LogisticDetection, 'SVCDetection': sdmetrics.multi_table.multi_single_table.SVCDetection, 'BNLikelihood': sdmetrics.multi_table.multi_single_table.BNLikelihood, 'BNLogLikelihood': sdmetrics.multi_table.multi_single_table.BNLogLikelihood, 'LogisticParentChildDetection': sdmetrics.multi_table.detection.parent_child.LogisticParentChildDetection, 'SVCParentChildDetection': sdmetrics.multi_table.detection.parent_child.SVCParentChildDetection} ``` -------------------------------- ### Compute LSTMDetection Metric with Metadata Source: https://github.com/sdv-dev/sdmetrics/blob/main/sdmetrics/timeseries/README.md Execute the metric by passing a metadata dictionary containing the sequence key configuration. ```python In[7]: metadata['sequence_key'] = 'store_id' In[8]: LSTMDetection.compute(real_data, synthetic_data, metadata=metadata) Out[8]: 0.5 ``` -------------------------------- ### TVComplement Metric Source: https://context7.com/sdv-dev/sdmetrics/llms.txt Compares discrete or categorical distributions using 1 minus half the L1 distance. ```python from sdmetrics.single_column import TVComplement import pandas as pd # Sample categorical data with frequency distributions real_data = pd.Series(['A']*40 + ['B']*30 + ['C']*20 + ['D']*10) synthetic_similar = pd.Series(['A']*38 + ['B']*32 + ['C']*18 + ['D']*12) synthetic_different = pd.Series(['A']*10 + ['B']*10 + ['C']*40 + ['D']*40) # Compute TV Complement similar_score = TVComplement.compute(real_data, synthetic_similar) different_score = TVComplement.compute(real_data, synthetic_different) print(f"Similar distribution: {similar_score:.4f}") # ~0.96 print(f"Different distribution: {different_score:.4f}") # ~0.40 ``` -------------------------------- ### Compare Multi-table Cardinality Distributions Source: https://context7.com/sdv-dev/sdmetrics/llms.txt Measures similarity between parent-child relationships in multi-table datasets. ```python from sdmetrics import load_demo from sdmetrics.multi_table import CardinalityShapeSimilarity # Load multi-table data real_data, synthetic_data, metadata = load_demo(modality='multi_table') # Compute cardinality shape similarity score = CardinalityShapeSimilarity.compute(real_data, synthetic_data, metadata) print(f"Cardinality Shape Similarity: {score:.4f}") # Get breakdown by relationship breakdown = CardinalityShapeSimilarity.compute_breakdown( real_data, synthetic_data, metadata ) for relationship, details in breakdown.items(): if isinstance(relationship, tuple): parent, child = relationship print(f"{parent} -> {child}: {details['score']:.4f}") ``` -------------------------------- ### Compute LogisticParentChildDetection with metadata Source: https://github.com/sdv-dev/sdmetrics/blob/main/sdmetrics/multi_table/README.md Calculate the LogisticParentChildDetection metric by passing a metadata dictionary instead of explicit foreign keys. ```python In[10]: LogisticParentChildDetection.compute(real_data, synthetic_data, metadata) Out[10]: 0.7569444444444444 ``` -------------------------------- ### Visualize Single Column Distribution Comparison Source: https://context7.com/sdv-dev/sdmetrics/llms.txt Generates a plot comparing the distribution of a single real column against its synthetic counterpart. Specify 'bar' for categorical or 'distplot' for continuous data. ```python from sdmetrics import load_demo from sdmetrics.visualization import ( get_column_plot, get_column_pair_plot ) real_data, synthetic_data, metadata = load_demo(modality='single_table') fig = get_column_plot( real_data['salary'], synthetic_data['salary'], plot_type='bar' # or 'distplot' for continuous ) fig.update_layout(title='Salary Distribution: Real vs Synthetic') fig.show() ``` -------------------------------- ### Compute LogisticParentChildDetection with foreign keys Source: https://github.com/sdv-dev/sdmetrics/blob/main/sdmetrics/multi_table/README.md Calculate the LogisticParentChildDetection metric by explicitly defining foreign key relationships. ```python3 In [7]: from sdmetrics.multi_table import LogisticParentChildDetection In [8]: foreign_keys = [ ...: ('users', 'user_id', 'sessions', 'user_id'), ...: ('sessions', 'session_id', 'transactions', 'session_id') ...: ] In [9]: LogisticParentChildDetection.compute(real_data, synthetic_data, foreign_keys=foreign_keys) Out[9]: 0.7569444444444444 ``` -------------------------------- ### Run test subsets with pytest Source: https://github.com/sdv-dev/sdmetrics/blob/main/CONTRIBUTING.rst Execute specific test modules or filter tests by keyword using pytest. ```bash $ python -m pytest tests.test_sdmetrics $ python -m pytest -k 'foo' ``` -------------------------------- ### Compute MulticlassDecisionTreeClassifier Metric Source: https://github.com/sdv-dev/sdmetrics/blob/main/sdmetrics/single_table/README.md Shows how to specify a target column directly when computing the MulticlassDecisionTreeClassifier metric. ```python3 In [9]: from sdmetrics.single_table import MulticlassDecisionTreeClassifier In [10]: MulticlassDecisionTreeClassifier.compute(real_table, synthetic_table, target='country') Out[10]: (0.05555555555555555,) ``` -------------------------------- ### Visualize Report Results Source: https://github.com/sdv-dev/sdmetrics/blob/main/README.md Retrieve a visualization for a specific property within a generated report. ```python my_report.get_visualization(property_name='Column Pair Trends') ``` -------------------------------- ### CorrelationSimilarity Metric Source: https://context7.com/sdv-dev/sdmetrics/llms.txt Compares correlation coefficients between column pairs in real and synthetic datasets. ```python from sdmetrics.column_pairs import CorrelationSimilarity import pandas as pd import numpy as np np.random.seed(42) ``` -------------------------------- ### Generate Multi-Table Quality Report Source: https://context7.com/sdv-dev/sdmetrics/llms.txt The multi-table QualityReport evaluates relational data quality, including cardinality and inter-table relationships. ```python from sdmetrics import load_demo from sdmetrics.reports.multi_table import QualityReport # Load multi-table demo real_data, synthetic_data, metadata = load_demo(modality='multi_table') ``` -------------------------------- ### Compute KSComplement metric Source: https://github.com/sdv-dev/sdmetrics/blob/main/sdmetrics/multi_table/README.md Calculate the KSComplement metric score for real and synthetic datasets. ```python3 In [3]: from sdmetrics.multi_table import KSComplement In [4]: from sdmetrics import load_demo In [5]: real_data, synthetic_data, metadata = load_demo() In [6]: KSComplement.compute(real_data, synthetic_data) Out[6]: 0.8194444444444443 ``` -------------------------------- ### Assert Used in SDMetrics Source: https://github.com/sdv-dev/sdmetrics/blob/main/static_code_analysis.txt This snippet shows an assertion check for 'cls.min_value == 0.0'. Ensure assertions are used appropriately for debugging and not for runtime error handling. ```python assert cls.min_value == 0.0 ``` -------------------------------- ### Compute LSTMDetection Metric with Sequence Key Source: https://github.com/sdv-dev/sdmetrics/blob/main/sdmetrics/timeseries/README.md Calculate the detection score by providing real and synthetic data along with the sequence key. ```python3 In [3]: from sdmetrics.timeseries import LSTMDetection In [4]: from sdmetrics.demos import load_timeseries_demo In [5]: real_data, synthetic_data, metadata = load_timeseries_demo() In [6]: LSTMDetection.compute(real_data, synthetic_data, sequence_key=['store_id']) Out[6]: 0.5 ``` -------------------------------- ### Visualize Column Pair Relationship Source: https://context7.com/sdv-dev/sdmetrics/llms.txt Creates a plot to visualize the relationship between two columns across real and synthetic datasets. Supported plot types include 'scatter', 'heatmap', and 'box'. ```python fig = get_column_pair_plot( real_data[['salary', 'duration']], synthetic_data[['salary', 'duration']], plot_type='scatter' # or 'heatmap', 'box' ) fig.show() ``` -------------------------------- ### CategoryCoverage Metric Source: https://context7.com/sdv-dev/sdmetrics/llms.txt Measures the fraction of categories from real data present in synthetic data. Provides a breakdown of coverage. ```python from sdmetrics.single_column import CategoryCoverage import pandas as pd # Sample categorical data real_categories = pd.Series(['A', 'B', 'C', 'D', 'E', 'A', 'B', 'C']) synthetic_partial = pd.Series(['A', 'B', 'C', 'A', 'B', 'C']) # Missing D, E synthetic_full = pd.Series(['A', 'B', 'C', 'D', 'E', 'A']) # All categories # Compute coverage scores partial_score = CategoryCoverage.compute(real_categories, synthetic_partial) full_score = CategoryCoverage.compute(real_categories, synthetic_full) print(f"Partial coverage score: {partial_score:.2%}") # 60% (3/5 categories) print(f"Full coverage score: {full_score:.2%}") # 100% (5/5 categories) # Get detailed breakdown breakdown = CategoryCoverage.compute_breakdown(real_categories, synthetic_partial) print(f"Real categories: {breakdown['real']}") print(f"Synthetic covers: {breakdown['synthetic']}") # Real categories: 5 # Synthetic covers: 3 ``` -------------------------------- ### BoundaryAdherence Metric Source: https://context7.com/sdv-dev/sdmetrics/llms.txt Measures the fraction of synthetic values within the min/max bounds of real data. Supports both numeric and datetime columns. ```python from sdmetrics.single_column import BoundaryAdherence import pandas as pd # Sample data real_ages = pd.Series([25, 30, 35, 40, 45, 50, 55, 60]) synthetic_ages = pd.Series([28, 32, 38, 42, 48, 52, 58, 65]) # 65 exceeds max # Compute boundary adherence score = BoundaryAdherence.compute(real_ages, synthetic_ages) print(f"Boundary Adherence Score: {score:.2%}") # Boundary Adherence Score: 87.50% # (7 out of 8 synthetic values are within real data bounds) # Works with datetime columns too real_dates = pd.to_datetime(['2023-01-01', '2023-06-15', '2023-12-31']) synthetic_dates = pd.to_datetime(['2023-03-01', '2023-07-20', '2024-01-15']) date_score = BoundaryAdherence.compute(real_dates, synthetic_dates) print(f"Date Boundary Adherence: {date_score:.2%}") ``` -------------------------------- ### Compute New Row Synthesis Metric Source: https://github.com/sdv-dev/sdmetrics/blob/main/README.md Calculate the new row synthesis metric to check for exact data copies. ```python # calculate whether the synthetic data is new or whether it's an exact copy of the real data from sdmetrics.single_table import NewRowSynthesis NewRowSynthesis.compute(real_data, synthetic_data, metadata) ``` -------------------------------- ### Generate Single Table Quality Report Source: https://context7.com/sdv-dev/sdmetrics/llms.txt QualityReport assesses column shapes and pair trends, providing scores and visualization capabilities. ```python from sdmetrics import load_demo from sdmetrics.reports.single_table import QualityReport # Load demo data real_data, synthetic_data, metadata = load_demo(modality='single_table') # Create and generate the quality report report = QualityReport() report.generate(real_data, synthetic_data, metadata, verbose=True) # Output: # Generating report ... # (1/2) Evaluating Column Shapes: 100% # Column Shapes Score: 82.78% # (2/2) Evaluating Column Pair Trends: 100% # Column Pair Trends Score: 82.9% # Overall Score (Average): 82.84% # Get overall quality score overall_score = report.get_score() print(f"Overall Quality Score: {overall_score:.2%}") # Overall Quality Score: 82.84% # Get property-level scores properties_df = report.get_properties() print(properties_df) # Property Score # 0 Column Shapes 0.827778 # 1 Column Pair Trends 0.829000 # Get detailed breakdown for a specific property details = report.get_details(property_name='Column Shapes') print(details.head()) # Column Metric Score # 0 gender TVComplement 0.95 # 1 high_perc KSComplement 0.87 # 2 high_spec TVComplement 0.92 # Visualize the column pair trends fig = report.get_visualization(property_name='Column Pair Trends') fig.show() # Save and load reports report.save('quality_report.pkl') loaded_report = QualityReport.load('quality_report.pkl') ``` -------------------------------- ### Compute Contingency Similarity Source: https://context7.com/sdv-dev/sdmetrics/llms.txt Use ContingencySimilarity to compare the relationships between categorical columns using contingency tables. Requires pandas DataFrames for real and synthetic data. ```python from sdmetrics.column_pairs import ContingencySimilarity import pandas as pd real_data = pd.DataFrame({ 'category1': ['A', 'A', 'A', 'B', 'B', 'B', 'C', 'C', 'C'] * 10, 'category2': ['X', 'X', 'Y', 'X', 'Y', 'Y', 'Y', 'Z', 'Z'] * 10 }) synthetic_similar = pd.DataFrame({ 'category1': ['A', 'A', 'A', 'B', 'B', 'B', 'C', 'C', 'C'] * 10, 'category2': ['X', 'X', 'Y', 'X', 'Y', 'Y', 'Y', 'Z', 'Z'] * 10 }) synthetic_different = pd.DataFrame({ 'category1': ['A', 'B', 'C'] * 30, 'category2': ['X', 'Y', 'Z'] * 30 }) similar_score = ContingencySimilarity.compute(real_data, synthetic_similar) different_score = ContingencySimilarity.compute(real_data, synthetic_different) print(f"Similar contingency score: {similar_score:.4f}") print(f"Different contingency score: {different_score:.4f}") ``` -------------------------------- ### Compute Disclosure Protection Scores Source: https://context7.com/sdv-dev/sdmetrics/llms.txt Calculates privacy risk scores for synthetic data. Use the estimate version for large datasets to improve performance. ```python np.random.seed(42) n = 500 real_data = pd.DataFrame({ 'age': np.random.randint(18, 80, n), 'zipcode': np.random.choice(['10001', '10002', '10003', '10004'], n), 'gender': np.random.choice(['M', 'F'], n), 'salary': np.random.randint(30000, 150000, n), 'diagnosis': np.random.choice(['healthy', 'diabetes', 'heart_disease'], n) }) synthetic_data = pd.DataFrame({ 'age': np.random.randint(18, 80, n), 'zipcode': np.random.choice(['10001', '10002', '10003', '10004'], n), 'gender': np.random.choice(['M', 'F'], n), 'salary': np.random.randint(30000, 150000, n), 'diagnosis': np.random.choice(['healthy', 'diabetes', 'heart_disease'], n) }) # Compute disclosure protection # Attacker knows: age, zipcode, gender # Attacker tries to infer: diagnosis score = DisclosureProtection.compute( real_data, synthetic_data, known_column_names=['age', 'zipcode', 'gender'], sensitive_column_names=['diagnosis'], computation_method='cap', # Options: 'cap', 'zero_cap', 'generalized_cap' continuous_column_names=['age'], # Discretize continuous columns num_discrete_bins=10 ) print(f"Disclosure Protection Score: {score:.4f}") # For large datasets, use the estimate version estimate_score = DisclosureProtectionEstimate.compute( real_data, synthetic_data, known_column_names=['age', 'zipcode', 'gender'], sensitive_column_names=['diagnosis'], num_rows_subsample=200, num_iterations=5, verbose=True ) ``` -------------------------------- ### Generate Single Table Diagnostic Report Source: https://context7.com/sdv-dev/sdmetrics/llms.txt DiagnosticReport identifies fundamental data validity and structure issues in synthetic datasets. ```python from sdmetrics import load_demo from sdmetrics.reports.single_table import DiagnosticReport real_data, synthetic_data, metadata = load_demo(modality='single_table') # Generate diagnostic report diagnostic = DiagnosticReport() diagnostic.generate(real_data, synthetic_data, metadata, verbose=True) # Output: # (1/2) Evaluating Data Validity: 100% # Data Validity Score: 98.5% # (2/2) Evaluating Data Structure: 100% # Data Structure Score: 100.0% # Overall Score (Average): 99.25% # Get diagnostic score score = diagnostic.get_score() print(f"Diagnostic Score: {score:.2%}") # Get validity details validity_details = diagnostic.get_details(property_name='Data Validity') print(validity_details) # Column Metric Score # 0 salary BoundaryAdherence 0.95 # 1 duration BoundaryAdherence 0.98 # 2 high_perc BoundaryAdherence 1.00 ``` -------------------------------- ### Measure Predictive Efficacy with Decision Trees Source: https://context7.com/sdv-dev/sdmetrics/llms.txt Evaluates if synthetic data preserves predictive relationships by training a decision tree classifier. ```python from sdmetrics.single_table import BinaryDecisionTreeClassifier import pandas as pd import numpy as np np.random.seed(42) n = 1000 # Create data with predictive relationship def generate_data(n): x1 = np.random.normal(0, 1, n) x2 = np.random.normal(0, 1, n) y = (x1 + x2 > 0).astype(int) # Binary target return pd.DataFrame({'x1': x1, 'x2': x2, 'target': y}) real_data = generate_data(n) synthetic_good = generate_data(n) # Preserves relationship # Create synthetic that doesn't preserve relationship synthetic_bad = generate_data(n) synthetic_bad['target'] = np.random.randint(0, 2, n) # Random target metadata = { 'columns': { 'x1': {'sdtype': 'numerical'}, 'x2': {'sdtype': 'numerical'}, 'target': {'sdtype': 'categorical'} } } # Compute efficacy scores good_score = BinaryDecisionTreeClassifier.compute( real_data, synthetic_good, metadata, target='target' ) bad_score = BinaryDecisionTreeClassifier.compute( real_data, synthetic_bad, metadata, target='target' ) print(f"Good synthetic efficacy: {good_score:.4f}") print(f"Bad synthetic efficacy: {bad_score:.4f}") ``` -------------------------------- ### Compute Correlation Similarity Source: https://context7.com/sdv-dev/sdmetrics/llms.txt Use CorrelationSimilarity to compare the correlation between columns in real and synthetic data. Supports Pearson and Spearman coefficients. Ensure data is in pandas DataFrames. ```python n = 500 x = np.random.normal(0, 1, n) y_real = 0.8 * x + 0.6 * np.random.normal(0, 1, n) y_synth_good = 0.75 * x + 0.65 * np.random.normal(0, 1, n) real_data = pd.DataFrame({'col1': x, 'col2': y_real}) synthetic_good = pd.DataFrame({'col1': x, 'col2': y_synth_good}) good_score = CorrelationSimilarity.compute(real_data, synthetic_good) print(f"Similar correlation score: {good_score:.4f}") breakdown = CorrelationSimilarity.compute_breakdown( real_data, synthetic_good, coefficient='Pearson' ) print(f"Real correlation: {breakdown['real']:.4f}") print(f"Synthetic correlation: {breakdown['synthetic']:.4f}") spearman_score = CorrelationSimilarity.compute( real_data, synthetic_good, coefficient='Spearman' ) ``` -------------------------------- ### Compute New Row Synthesis Score Source: https://context7.com/sdv-dev/sdmetrics/llms.txt Use NewRowSynthesis to measure if synthetic rows are new or copies of real data. Requires real data, synthetic data, and metadata. A numerical_match_tolerance can be specified. ```python from sdmetrics import load_demo from sdmetrics.single_table import NewRowSynthesis real_data, synthetic_data, metadata = load_demo(modality='single_table') score = NewRowSynthesis.compute( real_data, synthetic_data, metadata, numerical_match_tolerance=0.01 ) print(f"New Row Synthesis Score: {score:.2%}") breakdown = NewRowSynthesis.compute_breakdown( real_data, synthetic_data, metadata, numerical_match_tolerance=0.01, synthetic_sample_size=100 ) print(f"New rows: {breakdown['num_new_rows']}") print(f"Matched rows: {breakdown['num_matched_rows']}") ``` -------------------------------- ### Compute Referential Integrity Source: https://context7.com/sdv-dev/sdmetrics/llms.txt Use ReferentialIntegrity to check if foreign key values in synthetic data reference valid primary keys. Input data should be tuples of (primary_key_df, foreign_key_df). ```python from sdmetrics.column_pairs import ReferentialIntegrity import pandas as pd real_pk = pd.DataFrame({'user_id': [1, 2, 3, 4, 5]}) synth_pk = pd.DataFrame({'user_id': [1, 2, 3, 4, 5]}) real_fk = pd.DataFrame({'user_id': [1, 1, 2, 3, 4, 5]}) synth_fk_good = pd.DataFrame({'user_id': [1, 2, 2, 3, 4, 5]}) synth_fk_bad = pd.DataFrame({'user_id': [1, 2, 6, 7, 8, 9]}) good_score = ReferentialIntegrity.compute( real_data=(real_pk, real_fk), synthetic_data=(synth_pk, synth_fk_good) ) bad_score = ReferentialIntegrity.compute( real_data=(real_pk, real_fk), synthetic_data=(synth_pk, synth_fk_bad) ) print(f"Good referential integrity: {good_score:.2%}") print(f"Bad referential integrity: {bad_score:.2%}") ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.