### Prepare Training with Sample Size (Python)

Source: https://docs.dedupe.io/en/latest/API-documentation

Example of preparing training data using a specified sample size. This is a common use case when starting a new deduplication project without pre-existing training data.

```python
matcher.prepare_training(data_1, data_2, 150000)
```

--------------------------------

### Install Dedupe Library using pip

Source: https://docs.dedupe.io/en/latest/_sources/index

This command installs the Dedupe library using pip, the Python package installer. Ensure you have Python and pip installed on your system before running this command. It fetches the latest stable version of the library from the Python Package Index.

```bash
pip install dedupe
```

--------------------------------

### Prepare Training with Existing File (Python)

Source: https://docs.dedupe.io/en/latest/API-documentation

Example of preparing training data by loading from an existing training file. This is useful for resuming training or using previously generated training data.

```python
with open('training_file.json') as f:
    matcher.prepare_training(data_1, data_2, training_file=f)
```

--------------------------------

### Example Record Comparison for Deduplication

Source: https://docs.dedupe.io/en/latest/how-it-works/How-it-works

Illustrates a common scenario in data deduplication where two records are compared to determine if they refer to the same entity, highlighting variations in fields like name and address. This example demonstrates the need for fuzzy matching.

```text
first name | last name | address                   | phone   | 
--------------------------------------------------------------
bob        | roberts   | 1600 pennsylvania ave.   | 555-0123 | 
Robert     | Roberts   | 1600 Pensylvannia Avenue |          | 

```

--------------------------------

### write_training

Source: https://docs.dedupe.io/en/latest/API-documentation

Writes the labeled examples to a file object in JSON format.

```APIDOC
## write_training

### Description
Write a JSON file that contains labeled examples.

### Method
`write_training(file_obj)`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **file_obj** (TextIO) - file object to write training data to

### Request Example
```python
with open('training.json', 'w') as f:
    matcher.write_training(f)
```

### Response
#### Success Response (200)
None (Writes data to the provided file object).

#### Response Example
None
```

--------------------------------

### Write Training Data

Source: https://docs.dedupe.io/en/latest/API-documentation

Writes the labeled examples to a JSON file.

```APIDOC
## write_training

### Description
Write a JSON file that contains labeled examples.

### Method
`write_training(_file_obj_)

### Parameters
#### File Object
* **file_obj** (`TextIO`) – file object to write training data to

### Request Example
```python
with open('training.json', 'w') as f:
    matcher.write_training(f)
```
```

--------------------------------

### write_training

Source: https://docs.dedupe.io/en/latest/API-documentation

Writes the current labeled training examples to a file object in JSON format.

```APIDOC
## write_training

### Description
Write a JSON file that contains labeled examples.

### Method
GET

### Endpoint
/write_training

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **file_obj** (TextIO) - Required - File object to write training data to.

### Request Example
```json
{
  "file_obj": "training.json"
}
```

### Response
#### Success Response (200)
Indicates that the training data has been written to the specified file object.

#### Response Example
```json
{
  "message": "Training data written to file."
}
```
```

--------------------------------

### Dedupe.io Matcher Join Example

Source: https://docs.dedupe.io/en/latest/API-documentation

Example demonstrating how to use the `matcher.join` function to find duplicate records between two datasets. It takes two data dictionaries, a similarity threshold, and a constraint type as input. The function returns an iterable of matched record pairs and their similarity scores.

```python
>>> links = matcher.join(data_1, data_2, threshold=0.5)
>>> list(links)
[
    ((1, 2), 0.790),
    ((4, 5), 0.720),
    ((10, 11), 0.899)
]
```

--------------------------------

### Example Pairwise Probabilities for Duplicates

Source: https://docs.dedupe.io/en/latest/how-it-works/Grouping-duplicates

Illustrates how pairwise probabilities between records A, B, and C might be represented. This serves as an input to the clustering algorithm.

```text
A -- 0.6 -- B -- 0.6 -- C
```

--------------------------------

### Write Training Data to File

Source: https://docs.dedupe.io/en/latest/_modules/dedupe/api

Serializes the labeled training examples to a JSON file. This allows for saving the current state of training data for later use.

```python
def write_training(self, file_obj: TextIO) -> None:  # pragma: no cover
    """
    Write a JSON file that contains labeled examples

    Args:
        file_obj: file object to write training data to

    Examples:
        >>> with open('training.json', 'w') as f:
        >>>     matcher.write_training(f)
    """
    serializer.write_training(self.training_pairs, file_obj)
```

--------------------------------

### Write Training Data to File

Source: https://docs.dedupe.io/en/latest/API-documentation

Writes the current labeled training examples to a JSON file object. This allows for saving the training progress and reusing it later or for manual inspection.

```python
with open('training.json', 'w') as f:
    matcher.write_training(f)
```

--------------------------------

### Read Training Data from File

Source: https://docs.dedupe.io/en/latest/_modules/dedupe/api

Reads previously saved training data from a file object. This method deserializes labeled examples to continue training or analysis.

```python
def _read_training(self, training_file: TextIO) -> None:
    """
    Read training from previously built training data file object

    Args:
        training_file: file object containing the training data
    """

    logger.info("reading training from file")
    training_pairs = serializer.read_training(training_file)
    self.mark_pairs(training_pairs)
```

--------------------------------

### Initialize Gazetteer with String Variables

Source: https://docs.dedupe.io/en/latest/API-documentation

Initializes the Gazetteer object with a defined set of string variables, specifying field names and whether they can have missing values. This setup is crucial for defining the data structure Dedupe will use for comparison.

```python
import dedupe

variables = [
    dedupe.variables.String("Site name"),
    dedupe.variables.String("Address"),
    dedupe.variables.String("Zip", has_missing=True),
    dedupe.variables.String("Phone", has_missing=True),
]
matcher = dedupe.Gazetteer(variables)
```

--------------------------------

### Predicate Block Example Output

Source: https://docs.dedupe.io/en/latest/_sources/how-it-works/Making-smart-comparisons

This code snippet represents the output of a predicate blocking strategy. It shows how records are grouped into blocks based on a shared feature (e.g., the first three characters of an address). The keys are the shared features, and the values are tuples of record IDs belonging to that block.

```python
{
  '160': (1, 2),
  '123': (3, 4)
}
```

--------------------------------

### Dedupe.pairs() Method Example (Python)

Source: https://docs.dedupe.io/en/latest/API-documentation

Demonstrates how to use the `pairs` method of the `Dedupe` class to yield pairs of records that share common fingerprints. This method takes a dictionary of records as input and returns an iterator of record pairs.

```Python
>>> pairs = matcher.pairs(data)
>>> list(pairs)
[
    (
        (1, {"name": "Pat", "address": "123 Main"}),
        (2, {"name": "Pat", "address": "123 Main"}),
    ),
    (
        (1, {"name": "Pat", "address": "123 Main"}),
        (3, {"name": "Sam", "address": "123 Main"}),
    ),
]
```

--------------------------------

### Initialize Dedupe with String Variables

Source: https://docs.dedupe.io/en/latest/API-documentation

Initializes the Dedupe object with a predefined set of string variables. This setup is crucial for defining the fields that will be used for duplicate detection. The 'has_missing' parameter indicates fields that may contain null values.

```python
import
dedupe

variables = [
    dedupe.variables.String("Site name"),
    dedupe.variables.String("Address"),
    dedupe.variables.String("Zip", has_missing=True),
    dedupe.variables.String("Phone", has_missing=True),
]
dedup er = dedupe.Dedupe(variables)
```

--------------------------------

### Dedupe.cluster() Method Example (Python)

Source: https://docs.dedupe.io/en/latest/API-documentation

Illustrates the usage of the `cluster` method from the `Dedupe` class. This method takes similarity scores of record pairs and a threshold to group records referring to the same entity. It yields tuples of record IDs and their corresponding confidence scores.

```Python
>>> pairs = matcher.pairs(data)
>>> scores = matcher.scores(pairs)
>>> clusters = matcher.cluster(scores)
>>> list(clusters)
[
    ((1, 2, 3), (0.790, 0.860, 0.790)),
    ((4, 5), (0.720, 0.720)),
    ((10, 11), (0.899, 0.899)),
]
```

--------------------------------

### Predicate Blocking Example (Python)

Source: https://docs.dedupe.io/en/latest/how-it-works/Making-smart-comparisons

This snippet demonstrates how predicate blocking works by grouping records based on a common feature extracted from a specific field. Records sharing the same feature are placed into the same block. This reduces the number of comparisons needed by only comparing records within the same block.

```python
{
 '160' : (1,2) # tuple of record_ids
 '123' : (3,4)
 }
```

--------------------------------

### Prepare Training Data

Source: https://docs.dedupe.io/en/latest/API-documentation

Initializes the active learner with your data and, optionally, existing training data. Sets up the learner for the training process.

```APIDOC
## prepare_training

### Description
Initialize the active learner with your data and, optionally, existing training data. Sets up the learner.

### Method
`prepare_training(_data_ , _training_file =None_, _sample_size =1500_, _blocked_proportion =0.9_)

### Parameters
#### Data
* **data** (`Union`[`Mapping`[`int`, `Mapping`[`str`, `Any`]], `Mapping`[`str`, `Mapping`[`str`, `Any`]]) – Dictionary of records, where the keys are record_ids and the values are dictionaries with the keys being field names
* **training_file** (`TextIO` | `None`) – file object containing training data
* **sample_size** (`int`) – Size of the sample to draw. Defaults to 1500.
* **blocked_proportion** (`float`) – The proportion of record pairs to be sampled from similar records, as opposed to randomly selected pairs. Defaults to 0.9.

### Request Example
```python
# Example 1: Preparing training with data and sample size
matcher.prepare_training(data_d, 150000, .5)

# Example 2: Preparing training with a training file
with open('training_file.json') as f:
    matcher.prepare_training(data_d, training_file=f)
```
```

--------------------------------

### prepare_training

Source: https://docs.dedupe.io/en/latest/API-documentation

Initializes the active learner with your data and, optionally, existing training data. This method is crucial for setting up the deduplication model with the datasets to be compared.

```APIDOC
## prepare_training

### Description
Initialize the active learner with your data and, optionally, existing training data.

### Method
`prepare_training(data_1, data_2, training_file=None, sample_size=1500, blocked_proportion=0.9)`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **data_1** (Union[Mapping[int, Mapping[str, Any]], Mapping[str, Mapping[str, Any]]]) - Dictionary of records from first dataset, where the keys are record_ids and the values are dictionaries with the keys being field names
- **data_2** (Union[Mapping[int, Mapping[str, Any]], Mapping[str, Mapping[str, Any]]]) - Dictionary of records from second dataset, same form as data_1
- **training_file** (TextIO | None) - file object containing training data
- **sample_size** (int) - The size of the sample to draw.
- **blocked_proportion** (float) - The proportion of record pairs to be sampled from similar records, as opposed to randomly selected pairs.

### Request Example
```python
# Example 1: Using sample size
matcher.prepare_training(data_1, data_2, 150000)

# Example 2: Using an existing training file
with open('training_file.json') as f:
    matcher.prepare_training(data_1, data_2, training_file=f)
```

### Response
#### Success Response (200)
None (This method modifies the internal state of the matcher object).

#### Response Example
None
```

--------------------------------

### prepare_training

Source: https://docs.dedupe.io/en/latest/API-documentation

Initializes the active learner with your data and, optionally, existing training data. This is a crucial step before training a matching model.

```APIDOC
## prepare_training

### Description
Initialize the active learner with your data and, optionally, existing training data.

### Method
POST

### Endpoint
/prepare_training

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **data_1** (Union[Mapping[int, Mapping[str, Any]], Mapping[str, Mapping[str, Any]]]) - Required - Dictionary of records from the first dataset.
- **data_2** (Union[Mapping[int, Mapping[str, Any]], Mapping[str, Mapping[str, Any]]]) - Required - Dictionary of records from the second dataset.
- **training_file** (TextIO | None) - Optional - File object containing training data.
- **sample_size** (int) - Optional - The size of the sample to draw. Defaults to 1500.
- **blocked_proportion** (float) - Optional - The proportion of record pairs to be sampled from similar records, as opposed to randomly selected pairs. Defaults to 0.9.

### Request Example
```json
{
  "data_1": {
    "1": {"name": "John Doe", "address": "123 Main St"}
  },
  "data_2": {
    "2": {"name": "Jane Doe", "address": "123 Main St"}
  },
  "sample_size": 150000
}
```

### Response
#### Success Response (200)
Indicates successful initialization. The response body is typically empty or a confirmation message.

#### Response Example
```json
{
  "message": "Training preparation successful."
}
```
```

--------------------------------

### Prepare Training for Dedupe IO Gazetteer Matching

Source: https://docs.dedupe.io/en/latest/_modules/dedupe/api

Initializes the active learner for gazetteer matching with provided data and optional existing training data. It takes a single dictionary of records and can load training from a file. Parameters control the sample size and the proportion of blocked pairs for sampling. It includes a check to ensure the input data dictionary is not empty.

```python
class Gazetteer(Link, GazetteerMatching):
    """
    Class for active learning gazetteer matching.

    Gazetteer matching is for matching a messy data set against a
    'canonical dataset'.  This class is useful for such tasks as matching messy
    addresses against a clean list
    """

    def prepare_training(
        self,
        data: Data,
        sample_size: int = 1500,
        blocked_proportion: float = 0.9,
        training_file: TextIO | None = None,
    ) -> None:
        """
        Initialize the active learner with your data and, optionally,
        existing training data.

        Args:
            data: Dictionary of records, where the keys are record_ids
                  and the values are dictionaries with the keys being field names.
            sample_size: The size of the sample to draw.
            blocked_proportion: The proportion of record pairs to be sampled
                                from similar records, as opposed to
                                randomly selected pairs.
            training_file: file object containing training data

        Examples:
            >>> matcher.prepare_training(data_d, 150000, .5)

            >>> with open('training_file.json') as f:
            >>>     matcher.prepare_training(data_d, training_file=f)
        """
        self._checkData(data)

        # Reset active learner
        self.active_learner = None

        if training_file:
            self._read_training(training_file)

        # We need the active learner to know about all our
        # existing training data, so add them to data dictionary
        examples, y = flatten_training(self.training_pairs)

        self.active_learner = labeler.DedupeDisagreementLearner(
            self.data_model.predicates,
            self.data_model.distances,
            data,
            index_include=examples,
        )

        self.active_learner.mark(examples, y)


    def _checkData(self, data: Data) -> None:
        if len(data) == 0:
            raise ValueError("Dictionary of records is empty.")

        self.data_model.check(next(iter(data.values())))

```

--------------------------------

### Initialize Active Learner for Deduplication

Source: https://docs.dedupe.io/en/latest/_modules/dedupe/api

Initializes the active learner with provided data and an optional existing training file. This method sets up the learner for active learning-based deduplication. It accepts a dictionary of records, a path to a training file, a sample size, and a blocked proportion. The data should be structured as a dictionary where keys are record IDs and values are dictionaries of field names.

```python
def prepare_training(
        self,
        data: Data,
        training_file: TextIO | None = None,
        sample_size: int = 1500,
        blocked_proportion: float = 0.9,
    ) -> None:
        """
        Initialize the active learner with your data and, optionally,
        existing training data.

        Sets up the learner.

        Args:
            data: Dictionary of records, where the keys are
                  record_ids and the values are dictionaries
                  with the keys being field names

```

--------------------------------

### Load StaticRecordLink Settings from File

Source: https://docs.dedupe.io/en/latest/API-documentation

Demonstrates how to load pre-trained settings for the StaticRecordLink class from a binary file. This is useful for performing record linkage using previously saved configurations.

```python
with open('learned_settings', 'rb') as f:
    matcher = StaticRecordLink(f)
```

--------------------------------

### Get Blocks of Pairs

Source: https://docs.dedupe.io/en/latest/API-documentation

Yields groups of record pairs that share fingerprints, used for efficient comparison.

```APIDOC
## Get Blocks of Pairs

### Description
Yields groups of pairs of records that share fingerprints. Each group contains one record from `data` paired with records from the indexed data that share a fingerprint. Each pair within and among blocks will occur at most once. If this method is overridden, ensure that every pair of records is compared no more than once, as downstream methods assume this condition.

### Method
`blocks(data)`

### Parameters
- **data** - Required - A dictionary of records, where the keys are record_ids and the values are dictionaries with the keys being field names.

### Request Example
```python
data_to_block = {
    'record1': {'name': 'Pat', 'address': '123 Main'},
    'record2': {'name': 'Sam', 'address': '123 Main'}
}
blocks_generator = matcher.blocks(data_to_block)
for block in blocks_generator:
    print(block)
```

### Response
#### Success Response (200)
- **blocks** (Iterator[list[tuple[tuple[record_id, record_data], tuple[record_id, record_data]]]]) - An iterator yielding lists of record pairs. Each pair is a tuple containing two records, where each record is a tuple of (record_id, record_data).

#### Response Example
```python
[
    [
        (
            ('record1', {'name': 'Pat', 'address': '123 Main'}),
            ('indexed_record_A', {'name': 'Pat', 'address': '123 Main'})
        ),
        (
            ('record1', {'name': 'Pat', 'address': '123 Main'}),
            ('indexed_record_B', {'name': 'Sam', 'address': '123 Main'})
        )
    ]
]
```
```

--------------------------------

### Initialize StaticDedupe from Learned Settings

Source: https://docs.dedupe.io/en/latest/_sources/API-documentation

Initializes a StaticDedupe object by loading learned settings from a file. This object is used for static deduplication, applying pre-learned rules. It requires a file path to the settings.

```python
from dedupe import StaticDedupe

with open('learned_settings', 'rb') as f:
    matcher = StaticDedupe(f)
```

--------------------------------

### StaticGazetteer Initialization

Source: https://docs.dedupe.io/en/latest/API-documentation

Initializes the StaticGazetteer with saved settings. It can optionally use multiple cores for processing and perform operations in memory.

```APIDOC
## StaticGazetteer Initialization

### Description
Initializes the `StaticGazetteer` class using a saved settings file. It allows for specifying the number of CPU cores to use and whether to keep data in memory.

### Method
`__init__`

### Parameters
- **settings_file** (BinaryIO) - Required - A file object containing settings info produced from the `write_settings()` method.
- **num_cores** (int | None) - Optional - The number of cpus to use for parallel processing. Defaults to the number of available CPUs. If set to 0, multiprocessing is disabled.
- **in_memory** (bool) - Optional - If True, `dedupe.Dedupe.pairs()` will generate pairs in RAM using the sqlite3 ‘:memory:’ option rather than writing to disk. May be faster if sufficient memory is available.

### Request Example
```python
with open('learned_settings', 'rb') as f:
    matcher = StaticGazetteer(f)
```

### Response
This is a constructor and does not return a value directly, but initializes an instance of `StaticGazetteer`.
```

--------------------------------

### Python: Define Dedupe Variables

Source: https://docs.dedupe.io/en/latest/_sources/Variable-definition

Example of defining a collection of variable objects for record matching in Dedupe. Includes String, ShortString, and String with missing values.

```python
import dedupe.variables

[
    dedupe.variables.String("Site Name"),
    dedupe.variables.String("Address"),
    dedupe.variables.ShortString("Zip", has_missing=True),
    dedupe.variables.String("Phone", has_missing=True)
]
```

--------------------------------

### Initialize StaticGazetteer from Learned Settings

Source: https://docs.dedupe.io/en/latest/_sources/API-documentation

Initializes a StaticGazetteer object by loading learned settings from a file. This object is used for static gazetteer operations, applying pre-learned rules. It requires a file path to the settings.

```python
from dedupe import StaticGazetteer

with open('learned_settings', 'rb') as f:
    matcher = StaticGazetteer(f)
```

--------------------------------

### Initialize StaticGazetteer with Saved Settings

Source: https://docs.dedupe.io/en/latest/API-documentation

Loads pre-trained settings for StaticGazetteer from a file. This is useful for reusing trained models without retraining. Ensure the settings file is opened in binary read mode ('rb').

```Python
with open('learned_settings', 'rb') as f:
    matcher = StaticGazetteer(f)
```

--------------------------------

### Get Uncertain Pairs for Labeling

Source: https://docs.dedupe.io/en/latest/_modules/dedupe/api

Returns a list of record pairs that the Dedupe model is most uncertain about. This is primarily useful for building user interfaces for active learning.

```python
def uncertain_pairs(self) -> TrainingExamples:
    """
     Returns a list of pairs of records from the sample of record pairs
     tuples that Dedupe is most curious to have labeled.

     This method is mainly useful for building a user interface for training
     a matching model.

    Examples:
        >>> pair = matcher.uncertain_pairs()
        >>> print(pair)
        [({'name' : 'Georgie Porgie'}, {'name' : 'Georgette Porgette'})]
    """
    assert (
        self.active_learner is not None
```

--------------------------------

### Initialize StaticRecordLink from Learned Settings

Source: https://docs.dedupe.io/en/latest/_sources/API-documentation

Initializes a StaticRecordLink object by loading learned settings from a file. This object is used for static record linkage, applying pre-learned rules. It requires a file path to the settings.

```python
from dedupe import StaticRecordLink

with open('learned_settings', 'rb') as f:
    matcher = StaticRecordLink(f)
```

--------------------------------

### GazetteerMatching Class Initialization

Source: https://docs.dedupe.io/en/latest/_modules/dedupe/api

Initializes the GazetteerMatching class, which is a subclass of Matching. It configures the number of cores to use for processing and whether to use in-memory storage for the database. The database path is set to an in-memory SQLite database if `in_memory` is True, otherwise, a temporary directory is created for the database file.

```python
class GazetteerMatching(Matching):
    def __init__(
        self,
        num_cores: int | None,
        in_memory: bool = False,
        **kwargs
    ) -> None:
        super().__init__(num_cores, in_memory, **kwargs)

        self.db: PathLike
        if self.in_memory:
            self.db = ":memory:"
        else:
            self.temp_dir = tempfile.TemporaryDirectory()
            self.db = self.temp_dir.name + "/blocks.db"

        self.indexed_data: (
            MutableMapping[int, RecordDict] | MutableMapping[str, RecordDict]
        )

```

--------------------------------

### Python: Text Variable Definition with Corpus

Source: https://docs.dedupe.io/en/latest/_sources/Variable-definition

Defines a Text variable in Dedupe for comparing blocks of text using cosine similarity. Includes an example with a corpus to learn weights.

```python
dedupe.variables.Text("Product description",
                        corpus=[
                                'this product is great',
                                'this product is great and blue'
                               ])
```

--------------------------------

### Initialize StaticDedupe with Saved Settings

Source: https://docs.dedupe.io/en/latest/API-documentation

Loads saved deduplication settings from a file to initialize a StaticDedupe object. This allows for applying previously trained models to new data without retraining. Requires a file object containing the settings.

```python
with open('learned_settings', 'rb') as f:
    matcher = StaticDedupe(f)

```

--------------------------------

### Get Uncertain Pairs

Source: https://docs.dedupe.io/en/latest/API-documentation

Returns a list of pairs of records from the sample of record pairs tuples that Dedupe is most curious to have labeled. This method is mainly useful for building a user interface for training a matching model.

```APIDOC
## uncertain_pairs

### Description
Returns a list of pairs of records from the sample of record pairs tuples that Dedupe is most curious to have labeled. This method is mainly useful for building a user interface for training a matching model.

### Method
`uncertain_pairs()`

### Response Example
```python
pair = matcher.uncertain_pairs()
print(pair)
# Expected output: [({'name' : 'Georgie Porgie'}, {'name' : 'Georgette Porgette'})]
```
```

--------------------------------

### prepare_training Gazetteer

Source: https://docs.dedupe.io/en/latest/_modules/dedupe/api

Initializes the active learner for gazetteer matching, optionally using existing training data. This method is used for matching a messy dataset against a canonical dataset.

```APIDOC
## POST /prepare_training (Gazetteer)

### Description
Initializes the active learner with your data and, optionally, existing training data for gazetteer matching. This is useful for tasks like matching messy addresses against a clean list.

### Method
POST

### Endpoint
`/prepare_training`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **data** (Data) - Required - Dictionary of records representing the messy dataset.
- **training_file** (TextIO | None) - Optional - File object containing existing training data.
- **sample_size** (int) - Optional - The size of the sample to draw (default: 1500).
- **blocked_proportion** (float) - Optional - The proportion of record pairs to be sampled from similar records, as opposed to randomly selected pairs (default: 0.5).

### Request Example
```json
{
  "data": { "record_id_1": { "field1": "value1", "field2": "value2" } },
  "sample_size": 150000,
  "blocked_proportion": 0.5
}
```

### Response
#### Success Response (200)
This method does not return a specific JSON response; it initializes the internal state of the matcher.

#### Response Example
N/A
```

--------------------------------

### Get Uncertain Pairs for Labeling

Source: https://docs.dedupe.io/en/latest/API-documentation

Retrieves a list of record pairs that the Dedupe model is most uncertain about. This is useful for building interactive user interfaces where users can label these ambiguous pairs to improve the training data.

```python
pair = matcher.uncertain_pairs()
print(pair)
```

--------------------------------

### Prepare Training Data for Dedupe (Python)

Source: https://docs.dedupe.io/en/latest/API-documentation

Initializes the active learner with your data and optionally existing training data. This method is crucial for training the matching model. It accepts two dictionaries of records, an optional training file, sample size, and blocked proportion.

```python
deduper.prepare_training(_data_1_, _data_2_, _training_file=None_, _sample_size=1500_, _blocked_proportion=0.9_)
```

--------------------------------

### Initializing StaticMatching for Dedupe

Source: https://docs.dedupe.io/en/latest/_modules/dedupe/api

This Python code initializes the StaticMatching class, designed for loading dedupe settings from a file. It handles loading the data model, classifier, and predicates, with specific error handling for incompatible settings files or missing libraries like 'rlr'. The constructor also logs the predicates and initializes a fingerprinter.

```python
class StaticMatching(Matching):
    """
    Class for initializing a dedupe object from a settings file.
    """

    def __init__(
        self,
        settings_file: BinaryIO,
        num_cores: int | None = None,
        in_memory: bool = False,
        **kwargs,
    ) -> None:  # pragma: no cover
        """Args:
            settings_file: A file object containing settings
                           info produced from the
                           :func:`~dedupe.api.ActiveMatching.write_settings` method.

            num_cores: The number of cpus to use for parallel
                       processing, defaults to the number of cpus
                       available on the machine. If set to 0, then
                       multiprocessing will be disabled.

            in_memory: If True, :meth:`dedupe.Dedupe.pairs` will generate
                       pairs in RAM with the sqlite3 ':memory:' option
                       rather than writing to disk. May be faster if
                       sufficient memory is available.

        .. warning::

            If using multiprocessing on Windows or Mac OS X, then
            you must protect calls to the Dedupe methods with a
            `if __name__ == '__main__'` in your main module, see
            https://docs.python.org/3/library/multiprocessing.html#the-spawn-and-forkserver-start-methods
        """
        super().__init__(num_cores, in_memory, **kwargs)

        try:
            self.data_model = pickle.load(settings_file)
            self.classifier = pickle.load(settings_file)
            self.predicates = pickle.load(settings_file)
        except (KeyError, AttributeError):
            raise SettingsFileLoadingException(
                "This settings file is not compatible with "
                "the current version of dedupe. This can happen "
                "if you have recently upgraded dedupe."
            )
        except ModuleNotFoundError as exc:
            if "No module named 'rlr'" in str(exc):
                raise SettingsFileLoadingException(
                    "This settings file was created with a previous "
                    "version of dedupe that used the 'rlr' library. "
                    "To continue to use this settings file, you need "
                    "install that library: `pip install rlr`"
                )
            else:
                raise SettingsFileLoadingException(
                    "Something has gone wrong with loading the settings file. "
                    "Try deleting the file"
                ) from exc
        except:  # noqa: E722
            raise SettingsFileLoadingException(
                "Something has gone wrong with loading the settings file. "
                "Try deleting the file"
            )

        logger.info("Predicate set:")
        for predicate in self.predicates:
            logger.info(predicate)

        self._fingerprinter = blocking.Fingerprinter(self.predicates)

```

--------------------------------

### Prepare Gazetteer Training Data

Source: https://docs.dedupe.io/en/latest/API-documentation

Prepares the Gazetteer for active learning by initializing it with datasets and optional existing training data. It allows specifying sample size and the proportion of blocked pairs to draw from similar records.

```python
matcher.prepare_training(_data_1_ , _data_2_ , _training_file =None_, _sample_size =1500_, _blocked_proportion =0.9_)
```

```python
matcher.prepare_training(data_1, data_2, 150000)
```

```python
with open('training_file.json') as f:
    matcher.prepare_training(data_1, data_2, training_file=f)
```

--------------------------------

### Install and use USAddress variable for US address comparison in dedupe

Source: https://docs.dedupe.io/en/latest/_sources/Variable-definition

The USAddress variable type is specifically for United States addresses. It parses addresses into components using the 'usaddress' package for component-wise comparison. Requires the 'dedupe-variable-address' package.

```python
import addressvariable

addressvariable.USAddress("address")
```

```console
pip install dedupe-variable-address
```

--------------------------------

### Install and use WesternName variable for American name comparison in dedupe

Source: https://docs.dedupe.io/en/latest/_sources/Variable-definition

The WesternName variable type is for American names, corporations, and households, using the 'probablepeople' package. It parses names into components like given name, surname, etc. Requires the 'dedupe-variable-name' package.

```python
import namevariable

namevariable.WesternName("field")
```

```console
pip install dedupe-variable-name
```

--------------------------------

### Gazetteer and StaticGazetteer API

Source: https://docs.dedupe.io/en/latest/API-documentation

Documentation for the Gazetteer and StaticGazetteer classes used for gazetteer matching.

```APIDOC
## Gazetteer Class

### Description
Handles gazetteer matching, allowing for efficient blocking and scoring of records within a dataset.

### Properties

#### `fingerprinter`
Instance of `dedupe.blocking.Fingerprinter` class if `train()` has been run, otherwise `None`.

### Methods

#### `blocks(_data_)`
Yields groups of pairs of records that share fingerprints. Each group contains one record from `data_1` paired with records from the indexed records that share a fingerprint. Ensures each pair is compared at most once.

**Parameters:**
- **data** (dict) - Dictionary of records, where keys are record_ids and values are dictionaries of field names.

**Example:**
```python
>>> blocks = matcher.pairs(data)
>>> print(list(blocks))
[
    [
        (
            (1, {"name": "Pat", "address": "123 Main"}),
            (8, {"name": "Pat", "address": "123 Main"}),
        ),
        (
            (1, {"name": "Pat", "address": "123 Main"}),
            (9, {"name": "Sam", "address": "123 Main"}),
        ),
    ],
    [
        (
            (2, {"name": "Sam", "address": "2600 State"}),
            (5, {"name": "Pam", "address": "2600 Stat"}),
        ),
        (
            (2, {"name": "Sam", "address": "123 State"}),
            (7, {"name": "Sammy", "address": "123 Main"}),
        ),
    ],
]
```

#### `score(_blocks_)`
Scores groups of record pairs. Yields structured numpy arrays representing pairs of records and their associated match probability.

**Parameters:**
- **blocks** (iterator) - Iterator of blocks of records.

#### `many_to_n(_score_blocks_, _threshold_=0.0, _n_matches_=1_)`
For each group of scored pairs, yields the highest scoring `n_matches` pairs.

**Parameters:**
- **score_blocks** (Iterable[Union[memmap, ndarray]]) - Iterator of numpy structured arrays with dtype `[('pairs', id_type, 2), ('score', 'f4')]`.
- **threshold** (float) - Number between 0 and 1. Records with a probability above this threshold are considered potential duplicates. Lowering increases recall, raising increases precision.
- **n_matches** (int) - The number of top-scoring pairs to select per group.

## StaticGazetteer Class

### Description
Provides methods similar to `Gazetteer` but with pre-computed fingerprints for static datasets.

### Properties

#### `fingerprinter`
Instance of `dedupe.blocking.Fingerprinter` class.

### Methods

#### `blocks(_data_)`
Same as `dedupe.Gazetteer.blocks()`.

#### `score(_blocks_)`
Same as `dedupe.Gazetteer.score()`.

#### `many_to_n(_score_blocks_, _threshold_=0.0, _n_matches_=1_)`
Same as `dedupe.Gazetteer.many_to_n()`.
```

--------------------------------

### Dedupe Console Labeling Utility

Source: https://docs.dedupe.io/en/latest/API-documentation

Trains a dedupe matcher instance from the command line. This function is useful for interactive training sessions where users label record pairs directly via the console. It requires a pre-initialized dedupe.Dedupe instance and prepared training data.

```python
import dedupe

# Assuming 'variables' is defined with field information
# and 'data' is prepared training data
deducer = dedupe.Dedupe(variables)
deducer.prepare_training(data)
dedupe.console_label(deducer)
```

--------------------------------

### Install and use DateTime variable for date comparison in dedupe

Source: https://docs.dedupe.io/en/latest/_sources/Variable-definition

The DateTime variable type compares dates and timestamps, accepting strings or datetime objects. It offers fuzzy parsing and options for day/year precedence to handle various date formats. Requires the 'dedupe-variable-datetime' package.

```python
import datetimetype

datetimetype.DateTime("field")
```

```console
pip install dedupe-variable-datetime
```

--------------------------------

### Prepare Training for Dedupe IO Record Linkage

Source: https://docs.dedupe.io/en/latest/_modules/dedupe/api

Prepares the active learner for record linkage by loading training data. It accepts dictionaries of records from two datasets, and optionally a file object for existing training data. Parameters control the sample size and the proportion of blocked record pairs to sample. It raises a ValueError if input dictionaries are empty.

```python
class Link(ActiveMatching):
    """
    Mixin Class for Active Learning Record Linkage
    """

    def prepare_training(
        self,
        data_1: Data,
        data_2: Data,
        training_file: TextIO | None = None,
        sample_size: int = 1500,
        blocked_proportion: float = 0.9,
    ) -> None:
        """
        Initialize the active learner with your data and, optionally,
        existing training data.

        Args:
            data_1: Dictionary of records from first dataset, where the
                    keys are record_ids and the values are dictionaries
                    with the keys being field names
            data_2: Dictionary of records from second dataset, same
                    form as data_1
            training_file: file object containing training data

            sample_size: The size of the sample to draw.

            blocked_proportion: The proportion of record pairs to
                                be sampled from similar records,
                                as opposed to randomly selected
                                pairs.

        Examples:
            >>> matcher.prepare_training(data_1, data_2, 150000)

            or

            >>> with open('training_file.json') as f:
            >>>     matcher.prepare_training(data_1, data_2, training_file=f)
        """
        self._checkData(data_1, data_2)

        # Reset active learner
        self.active_learner = None

        if training_file:
            self._read_training(training_file)

        # We need the active learner to know about all our
        # existing training data, so add them to data dictionaries
        examples, y = flatten_training(self.training_pairs)

        self.active_learner = labeler.RecordLinkDisagreementLearner(
            self.data_model.predicates,
            self.data_model.distances,
            data_1,
            data_2,
            index_include=examples,
        )

        self.active_learner.mark(examples, y)

    def _checkData(self, data_1: Data, data_2: Data) -> None:
        if len(data_1) == 0:
            raise ValueError("Dictionary of records from first dataset is empty.")
        elif len(data_2) == 0:
            raise ValueError("Dictionary of records from second dataset is empty.")

        self.data_model.check(next(iter(data_1.values())))
        self.data_model.check(next(iter(data_2.values())))

```

--------------------------------

### Adjust Recall Parameter in Dedupe Training

Source: https://docs.dedupe.io/en/latest/_sources/Troubleshooting

To mitigate slow blocking times caused by too many or too complex blocking rules, you can reduce the number of blocking rules dedupe learns. This can be done by adjusting the `recall` parameter in `dedupe.Dedupe.train` or by providing fewer positive examples during training. Lowering recall can increase speed at the cost of potentially missing some true positives.

```python
from dedupe import Dedupe

dedupe_classifier = Dedupe(fields, **kwargs)
dedupe_classifier.train(recall=0.1)  # Example: setting recall to 0.1
```

--------------------------------

### Initializing ActiveMatching for Dedupe Training

Source: https://docs.dedupe.io/en/latest/_modules/dedupe/api

This Python code initializes the ActiveMatching class, which is used for training a dedupe model. The constructor takes a variable definition, which describes the fields to be used for training, along with optional parameters for the number of CPU cores and memory usage. It sets up the necessary components for active learning and model training.

```python
class ActiveMatching(Matching):
    """
    Class for training a matcher.
    """

    active_learner: labeler.DisagreementLearner | None
    training_pairs: TrainingData

    def __init__(
        self,
        variable_definition: Collection[Variable],
        num_cores: int | None = None,
        in_memory: bool = False,
        **kwargs,
    ) -> None:
        """Args:
            variable_definition: A list of Variable objects describing
                                 the variables will be used for
                                 training a model. See :ref:`variable_definitions`

            num_cores: The number of cpus to use for parallel

```

--------------------------------

### Gazetteer and StaticGazetteer

Source: https://docs.dedupe.io/en/latest/_sources/API-documentation

Classes and methods for gazetteer matching and operations.

```APIDOC
## Gazetteer and StaticGazetteer Overview

### Description
These classes provide functionality for gazetteer matching, which involves finding entities in a dataset that correspond to a set of known entities.

### Methods
- **blocks(data)**: Generates blocking candidates from input data.
- **score(blocks)**: Scores the similarity between blocks of records.
- **many_to_n(score_blocks, threshold, n_matches)**: Finds the top N matches for each record from a set of scored blocks.

### Attributes
- **fingerprinter**: An instance of `dedupe.blocking.Fingerprinter` used for blocking, available after training.

### Endpoint
N/A (Library classes and methods)

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body (for methods like `blocks`, `score`, `many_to_n`)
- **data** (list of dicts or similar) - The input data for blocking.
- **blocks** (dict) - Scored blocks for scoring or matching.
- **score_blocks** (dict) - Scored blocks for `many_to_n`.
- **threshold** (float) - Minimum score for a match.
- **n_matches** (int) - The number of top matches to return.

### Request Example (Example for `many_to_n`)
```json
{
  "score_blocks": {
    "block_id1": [{"record_id": "recA", "score": 0.9},
                  {"record_id": "recB", "score": 0.8}],
    "block_id2": [{"record_id": "recC", "score": 0.95}]
  },
  "threshold": 0.7,
  "n_matches": 2
}
```

### Response
#### Success Response (200)
- **matches** (dict) - A dictionary containing the results of gazetteer operations, such as matched entities or scores.

#### Response Example (Example for `many_to_n`)
```json
{
  "matches": {
    "recA": [{"record_id": "recB", "score": 0.8},
               {"record_id": "recC", "score": 0.95}],
    "recC": [{"record_id": "recA", "score": 0.9}]
  }
}
```
```

--------------------------------

### write_settings

Source: https://docs.dedupe.io/en/latest/API-documentation

Writes the learned settings, including the data model and predicates, to a file object.

```APIDOC
## write_settings

### Description
Write a settings file containing the data model and predicates to a file object.

### Method
GET

### Endpoint
/write_settings

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **file_obj** (BinaryIO) - Required - File object to write settings data into.

### Request Example
```json
{
  "file_obj": "learned_settings"
}
```

### Response
#### Success Response (200)
Indicates that the settings have been written to the specified file object.

#### Response Example
```json
{
  "message": "Settings written to file."
}
```
```