Python Record Linkage Toolkit

https://recordlinkage.readthedocs.io/en/latest/
Admin
A Python library for linking records in or between data sources with indexing methods, comparison...
Tokens:70,600
Snippets:498
Trust Score:9.6
Update:5 days ago
Show doc for...
Tokens:
Raw
### Install recordlinkage with all dependencies

Source: https://recordlinkage.readthedocs.io/en/latest/installation.html

Installs the package along with all recommended and optional dependencies.

```bash
pip install recordlinkage['all']

```

--------------------------------

### Install pytest

Source: https://recordlinkage.readthedocs.io/en/latest/contributing.html

Install the pytest testing framework using pip.

```bash
pip install pytest

```

--------------------------------

### Install Airspeed Velocity (asv)

Source: https://recordlinkage.readthedocs.io/en/latest/contributing.html

Install the Airspeed Velocity benchmarking tool using pip.

```bash
pip install asv

```

--------------------------------

### Install recordlinkage with pip

Source: https://recordlinkage.readthedocs.io/en/latest/installation.html

Basic installation command for the Python Record Linkage Toolkit. Requires Python 3.6 or higher.

```bash
pip install recordlinkage

```

--------------------------------

### Add Exact comparison method

Source: https://recordlinkage.readthedocs.io/en/latest/ref-compare.html

Example from the exact method documentation showing how to import the Exact class and add it to a Compare object.

```python
from recordlinkage.compare import Exact

indexer = recordlinkage.Compare()
indexer.add(Exact())
```

--------------------------------

### Full record linkage example with recordlinkage

Source: https://recordlinkage.readthedocs.io/en/latest/guides/link_two_dataframes.html

Complete example showing the record linkage workflow: load FEBRL4 data, block index on 'given_name', compare fields with exact and string methods (jarowinkler for surname), compute features, and classify matches where the sum of features exceeds 3. The output prints the number of matches, which is 3241.

```python
import recordlinkage
from recordlinkage.datasets import load_febrl4

dfA, dfB = load_febrl4()

# Indexation step
indexer = recordlinkage.Index()
indexer.block("given_name")
candidate_links = indexer.index(dfA, dfB)

# Comparison step
compare_cl = recordlinkage.Compare()

compare_cl.exact("given_name", "given_name", label="given_name")
compare_cl.string(
    "surname", "surname", method="jarowinkler", threshold=0.85, label="surname"
)
compare_cl.exact("date_of_birth", "date_of_birth", label="date_of_birth")
compare_cl.exact("suburb", "suburb", label="suburb")
compare_cl.exact("state", "state", label="state")
compare_cl.string("address_1", "address_1", threshold=0.85, label="address_1")

features = compare_cl.compute(candidate_links, dfA, dfB)

# Classification step
matches = features[features.sum(axis=1) > 3]
print(len(matches))
```

```output
3241
```

--------------------------------

### Read links from annotation file (interactive example)

Source: https://recordlinkage.readthedocs.io/en/latest/annotation.html

Shows an interactive session reading an annotation file and printing its links. The output is not shown in the source.

```python
> annotation = read_annotation_file("result.json")
> print(annotation.links)

```

--------------------------------

### Initialize Compare and compute feature vectors

Source: https://recordlinkage.readthedocs.io/en/latest/ref-compare.html

Example from the Compare class documentation showing how to initialize the Compare object, add comparison methods for string, exact, and numeric attributes, and compute the feature vectors for candidate pairs.

```python
# initialise class
comp = recordlinkage.Compare()

# initialise similarity measurement algorithms
comp.string('first_name', 'name', method='jarowinkler')
comp.string('lastname', 'lastname', method='jarowinkler')
comp.exact('dateofbirth', 'dob')
comp.exact('sex', 'sex')
comp.string('address', 'address', method='levenshtein')
comp.exact('place', 'place')
comp.numeric('income', 'income')

# the method .compute() returns the DataFrame with the feature vectors.
comp.compute(candidate_pairs, census_data_1980, census_data_1990)
```

--------------------------------

### compare_vectorized with custom callable

Source: https://recordlinkage.readthedocs.io/en/latest/ref-compare.html

Example from the compare_vectorized method documentation showing how to initialize a Compare object and use compare_vectorized with a custom callable, then compute the comparison.

```python
>>> comp = recordlinkage.Compare()
>>> comp.compare_vectorized(custom_callable, 'first_name', 'name')
>>> comp.compare(PAIRS, DATAFRAME1, DATAFRAME2)
```

--------------------------------

### Calling the custom index class

Source: https://recordlinkage.readthedocs.io/en/latest/ref-index.html

Instantiate the custom index class and call its index() method to generate record pairs. The example shows the usage pattern for the custom algorithm.

```python
custom_index = CustomIndex():
custom_index.index()

```

--------------------------------

### recordlinkage.describe_option(pat, _print_desc=False)

Source: https://recordlinkage.readthedocs.io/en/latest/ref-misc.html

Prints the description for one or more registered options. Call with no arguments to get a listing for all registered options.

```APIDOC
## recordlinkage.describe_option(pat, _print_desc=False)

### Description
Prints the description for one or more registered options. Call with no arguments to get a listing for all registered options.

### Parameters
- **pat** - Required - Pattern to match option names.
- **_print_desc** (bool) - Optional - Internal flag for printing description.

### Returns
- **None**
```

--------------------------------

### Example: clean() function usage

Source: https://recordlinkage.readthedocs.io/en/latest/ref-preprocessing.html

Demonstrates cleaning a pandas Series with the clean() function. The example shows how to clean a list of names, converting to lowercase, removing punctuation and brackets, and handling missing values (None becomes NaN).

```python
>>> import pandas
>>> from recordlinkage.preprocessing import clean
>>>
>>> names = ['Mary-ann',
            'Bob :)',
            'Angel',
            'Bob (alias Billy)',
            None]
>>> s = pandas.Series(names)
>>> print(clean(s))
0    mary ann
1         bob
2       angel
3         bob
4         NaN
dtype: object
```

--------------------------------

### SKLearnAdapter: RandomForest classifier for record pair classification

Source: https://recordlinkage.readthedocs.io/en/latest/ref-classifiers.html

Shows how to wrap a Scikit-learn RandomForestClassifier with the SKLearnAdapter and BaseClassifier. The example creates a sample dataset with binary_vectors, initializes the classifier, fits it to the features and links, and predicts matches.

```python
# import ScitKit-Learn classifier
from sklearn.ensemble import RandomForestClassifier

# import BaseClassifier from recordlinkage.base
from recordlinkage.base import BaseClassifier
from recordlinkage.adapters import SKLearnClassifier
from recordlinkage.datasets import binary_vectors

class RandomForest(SKLearnClassifier, BaseClassifier):

    def __init__(*args, **kwargs):
        super(self, RandomForest).__init__()

        # set the kernel
        kernel = RandomForestClassifier(*args, **kwargs)


# make a sample dataset
features, links = binary_vectors(10000, 2000, return_links=True)

# initialise the random forest
cl = RandomForest(n_estimators=20)
cl.fit(features, links)

# predict the matches
cl.predict(...)

```

--------------------------------

### Import recordlinkage and load Krebs register dataset

Source: https://recordlinkage.readthedocs.io/en/latest/guides/classifiers.html

Imports pandas, recordlinkage, and the load_krebsregister function. This is the required setup before using the Krebs register dataset.

```python
In [1]: import pandas
   ...: import recordlinkage as rl
   ...: from recordlinkage.datasets import load_krebsregister
   ...: 
```

--------------------------------

### Define and use a custom multi-column comparison algorithm (CompareAddress)

Source: https://recordlinkage.readthedocs.io/en/latest/ref-compare.html

Defines a custom BaseCompareFeature subclass that compares two address columns across datasets, handling swapped values. The example then uses it with recordlinkage.Compare to compute similarity features, showing that the cross-over comparison yields a higher mean.

```python
import recordlinkage as rl
from recordlinkage.base import BaseCompareFeature

class CompareAddress(BaseCompareFeature):

    def _compute_vectorized(self, s1_1, s1_2, s2_1, s2_2):
        """Compare addresses.

        Compare addresses. Compare address_1 of file A with
        address_1 and address_2 of file B. The same for address_2
        of dataset 1.

        """

        return ((s1_1 == s2_1) | (s1_2 == s2_2) | (s1_1 == s2_2) | (s1_2 == s2_1)).astype(float)

comparer = rl.Compare()

# naive
comparer.add(CompareAddress('address_1', 'address_1', label='sim_address_1'))
comparer.add(CompareAddress('address_2', 'address_2', label='sim_address_2'))

# better
comparer.add(CompareAddress(('address_1', 'address_2'),
                            ('address_1', 'address_2'),
                            label='sim_address'
)

features = comparer.compute(pairs, dfA, dfB)
features.mean()

```

```text
sim_address_1    0.02488
sim_address_2    0.02025
sim_address      0.03566
dtype: float64

```

--------------------------------

### KerasAdapter: Neural network classifier for record pair classification

Source: https://recordlinkage.readthedocs.io/en/latest/ref-classifiers.html

Shows how to wrap a Keras sequential model with the KerasAdapter and BaseClassifier. The example builds a neural network with dense layers, compiles it with Adam optimizer and binary crossentropy loss, then fits and predicts on data.

```python
from tensorflow.keras import layers
from recordlinkage.base import BaseClassifier
from recordlinkage.adapters import KerasAdapter

class NNClassifier(KerasAdapter, BaseClassifier):
    """Neural network classifier."""
    def __init__(self):
        super(NNClassifier, self).__init__()

        model = tf.keras.Sequential()
        model.add(layers.Dense(16, input_dim=8, activation='relu'))
        model.add(layers.Dense(8, activation='relu'))
        model.add(layers.Dense(1, activation='sigmoid'))
        model.compile(
            optimizer=tf.train.AdamOptimizer(0.001),
            loss='binary_crossentropy',
            metrics=['accuracy']
        )

        self.kernel = model

# initialise the model
cl = NNClassifier()
# fit the model to the data
cl.fit(X_train, links_true)
# predict the class of the data
cl.predict(X_pred)

```

--------------------------------

### Block on multiple columns with Block indexer

Source: https://recordlinkage.readthedocs.io/en/latest/performance.html

Use the Block indexer to create record pairs that agree on multiple variables, reducing the number of pairs and improving performance. This example blocks on 'first_name' and 'surname'.

```python
from recordlinkage.index import Block
indexer = Block(left_on=['first_name', 'surname'],
                             right_on=['name', 'surname'])
pairs = indexer.index(dfA, dfB)

```

--------------------------------

### Block on multiple variables with Index and block method

Source: https://recordlinkage.readthedocs.io/en/latest/performance.html

Use the Index class with multiple block calls to create pairs that agree on multiple variables. This example blocks on 'first_name' and 'surname', then on 'first_name' and 'age'.

```python
indexer = recordlinkage.Index()
indexer.block(left_on=['first_name', 'surname'],
              right_on=['name', 'surname'])
indexer.block(left_on=['first_name', 'age'],
              right_on=['name', 'age'])
pairs = indexer.index(dfA, dfB)

```

--------------------------------

### Custom indexing algorithm for names starting with 'W'

Source: https://recordlinkage.readthedocs.io/en/latest/ref-index.html

Defines a custom indexing algorithm that returns all record pairs where the given name starts with 'W'. It uses pandas.MultiIndex.from_product to create the pairs.

```python
import recordlinkage
from recordlinkage.datasets import load_febrl4

df_a, df_b = load_febrl4()

from recordlinkage.base import BaseIndexAlgorithm

class FirstLetterWIndex(BaseIndexAlgorithm):
    """Custom class for indexing"""

    def _link_index(self, df_a, df_b):
        """Make pairs with given names starting with the letter 'w'."""

        # Select records with names starting with a w.
        name_a_w = df_a[df_a['given_name'].str.startswith('w') == True]
        name_b_w = df_b[df_b['given_name'].str.startswith('w') == True]

        # Make a product of the two numpy arrays
        return pandas.MultiIndex.from_product(
            [name_a_w.index.values, name_b_w.index.values],
            names=[df_a.index.name, df_b.index.name]
        )

indexer = FirstLetterWIndex()
candidate_pairs = indexer.index(df_a, df_b)

print ('Returns a', type(candidate_pairs).__name__)
print ('Number of candidate record pairs starting with the letter w:', len(candidate_pairs))

```

--------------------------------

### Run performance tests for versions since v0.6.0

Source: https://recordlinkage.readthedocs.io/en/latest/contributing.html

Run performance benchmarks for all versions since tag v0.6.0 using asv.

```bash
asv run --skip-existing-commits v0.6.0..master

```

--------------------------------

### Run performance tests with asv

Source: https://recordlinkage.readthedocs.io/en/latest/contributing.html

Run performance benchmarks for the current version of the package using asv.

```bash
asv run

```

--------------------------------

### display_config_options(pat, _print_desc=True)

Source: https://recordlinkage.readthedocs.io/en/latest/ref-misc.html

Function that displays descriptions of configuration options matching a regex pattern. If _print_desc is False, returns the descriptions as a unicode string.

```APIDOC
## display_config_options(pat, _print_desc=True)

### Description
Displays descriptions of configuration options that match the given regex pattern. If _print_desc is True (default), the descriptions are printed to stdout. Otherwise, the descriptions are returned as a unicode string.

### Parameters
- **pat** (_str_) - Required - Regexp pattern. All matching keys will have their description displayed.
- **_print_desc** (_bool_, default True) - Optional - If True (default) the description(s) will be printed to stdout. Otherwise, the description(s) will be returned as a unicode string (for testing).

### Returns
- None by default, the description(s) as a unicode string if _print_desc is False.

```

--------------------------------

### Add Full index algorithm

Source: https://recordlinkage.readthedocs.io/en/latest/ref-index.html

Shows the shortcut for adding a Full index algorithm using the add method.

```python
from recordlinkage.index import Full

indexer = recordlinkage.Index()
indexer.add(Full())
```

--------------------------------

### Custom indexing algorithm with configurable letter

Source: https://recordlinkage.readthedocs.io/en/latest/ref-index.html

Defines a custom indexing algorithm that takes a letter as an argument and returns record pairs where the given name starts with that letter.

```python
class FirstLetterIndex(BaseIndexAlgorithm):
    """Custom class for indexing"""

    def __init__(self, letter):
        super(FirstLetterIndex, self).__init__()

        # the letter to save
        self.letter = letter

    def _link_index(self, df_a, df_b):
        """Make record pairs that agree on the first letter of the given name."""

        # Select records with names starting with a 'letter'.
        a_startswith_w = df_a[df_a['given_name'].str.startswith(self.letter) == True]
        b_startswith_w = df_b[df_b['given_name'].str.startswith(self.letter) == True]

        # Make a product of the two numpy arrays
        return pandas.MultiIndex.from_product(
            [a_startswith_w.index.values, b_startswith_w.index.values],
            names=[df_a.index.name, df_b.index.name]
        )

```

--------------------------------

### Create record pairs with BlockIndex on 'first_name'

Source: https://recordlinkage.readthedocs.io/en/latest/ref-index.html

Example of using recordlinkage.BlockIndex to create record pairs from two census datasets, blocking on the 'first_name' column. The indexer is applied to the two DataFrames.

```python
>>> indexer = recordlinkage.BlockIndex(on='first_name')
>>> indexer.index(census_data_1980, census_data_1990)

```

--------------------------------

### Import Block algorithm from recordlinkage.index

Source: https://recordlinkage.readthedocs.io/en/latest/ref-index.html

Import the Block indexing algorithm from the recordlinkage.index submodule. This is the recommended way to access the indexing algorithms.

```python
from recordlinkage.index import Block

```

--------------------------------

### Import recordlinkage and load_febrl4

Source: https://recordlinkage.readthedocs.io/en/latest/guides/link_two_dataframes.html

Imports the recordlinkage module and the load_febrl4 function from recordlinkage.datasets. This is the first step before loading the Febrl datasets.

```python
import recordlinkage
from recordlinkage.datasets import load_febrl4
```

--------------------------------

### Create a full index with recordlinkage.Index

Source: https://recordlinkage.readthedocs.io/en/latest/guides/link_two_dataframes.html

Loads the Index class, calls the .full method, and generates all possible record pairs between dfA and dfB. A performance warning is issued because a full index can result in a large number of pairs.

```python
indexer = recordlinkage.Index()
indexer.full()
pairs = indexer.index(dfA, dfB)
```

--------------------------------

### Add SortedNeighbourhood index algorithm

Source: https://recordlinkage.readthedocs.io/en/latest/ref-index.html

Shows the shortcut for adding a SortedNeighbourhood index algorithm using the add method.

```python
from recordlinkage.index import SortedNeighbourhood

indexer = recordlinkage.Index()
indexer.add(SortedNeighbourhood())
```

--------------------------------

### User-defined algorithm 2: CompareZipCodes with partial_sim_value

Source: https://recordlinkage.readthedocs.io/en/latest/ref-compare.html

Extends the custom algorithm to accept a partial_sim_value parameter via __init__. The warning states not to change the order of pairs in the MultiIndex. The example uses the same typo 'extact' and passes the parameter as a keyword argument.

```python
import recordlinkage as rl
from recordlinkage.base import BaseCompareFeature

class CompareZipCodes(BaseCompareFeature):

    def __init__(self, left_on, right_on, partial_sim_value, *args, **kwargs):
        super(CompareZipCodes, self).__init__(left_on, right_on, *args, **kwargs)

        self.partial_sim_value = partial_sim_value

    def _compute_vectorized(self, s1, s2):
        """Compare zipcodes.

        If the zipcodes in both records are identical, the similarity
        is 0. If the first two values agree and the last two don't, then
        the similarity is 0.5. Otherwise, the similarity is 0.
        """

        # check if the zipcode are identical (return 1 or 0)
        sim = (s1 == s2).astype(float)

        # check the first 2 numbers of the distinct comparisons
        sim[(sim == 0) & (s1.str[0:2] == s2.str[0:2])] = self.partial_sim_value

        return sim

comparer = rl.Compare()
comparer.extact('given_name', 'given_name', 'y_name')
comparer.string('surname', 'surname', 'y_surname')
comparer.add(CompareZipCodes('postcode', 'postcode',
                             'partial_sim_value'=0.5, label='y_postcode'))
comparer.compute(pairs, dfA, dfB)

```

--------------------------------

### Set up and compute record comparisons with recordlinkage.Compare

Source: https://recordlinkage.readthedocs.io/en/latest/guides/data_deduplication.html

Creates a Compare object, adds exact and string comparisons for attributes, and computes the feature DataFrame for candidate links. The compute method triggers the comparisons and returns a DataFrame with features as columns and record pairs as rows.

```python
compare_cl = recordlinkage.Compare()
compare_cl.exact("given_name", "given_name", label="given_name")
compare_cl.string(
    "surname", "surname", method="jarowinkler", threshold=0.85, label="surname"
)
compare_cl.exact("date_of_birth", "date_of_birth", label="date_of_birth")
compare_cl.exact("suburb", "suburb", label="suburb")
compare_cl.exact("state", "state", label="state")
compare_cl.string("address_1", "address_1", threshold=0.85, label="address_1")
features = compare_cl.compute(candidate_links, dfA)
```

--------------------------------

### recordlinkage.Index()

Source: https://recordlinkage.readthedocs.io/en/latest/ref-index.html

Constructor for the Index class. Initializes an indexer with an optional list of index algorithm classes.

```APIDOC
## recordlinkage.Index()

### Description
Class to make an index of record pairs.

### Parameters
- **algorithms** (list) - Optional - A list of index algorithm classes. The classes are based on `recordlinkage.base.BaseIndexAlgorithm`.

### Example
```python
indexer = recordlinkage.Index()
indexer.block(left_on='first_name', right_on='givenname')
indexer.index(census_data_1980, census_data_1990)
```
```

--------------------------------

### Import recordlinkage and load_febrl1

Source: https://recordlinkage.readthedocs.io/en/latest/guides/data_deduplication.html

Imports the recordlinkage module and the load_febrl1 function from the recordlinkage.datasets submodule. This is the first step before loading the Febrl dataset 1.

```python
import recordlinkage
from recordlinkage.datasets import load_febrl1

```

--------------------------------

### User-defined algorithm 1: CompareZipCodes

Source: https://recordlinkage.readthedocs.io/en/latest/ref-compare.html

Defines a custom comparison algorithm by subclassing BaseCompareFeature and implementing _compute_vectorized. The algorithm returns 1.0 for identical zipcodes, 0.5 if the first two digits match, and 0 otherwise. The example then adds it to a Compare instance and computes. Note: the source contains a typo 'extact' for 'exact'.

```python
import recordlinkage as rl
from recordlinkage.base import BaseCompareFeature

class CompareZipCodes(BaseCompareFeature):

    def _compute_vectorized(self, s1, s2):
        """Compare zipcodes.

        If the zipcodes in both records are identical, the similarity
        is 1. If the first two values agree and the last two don't, then
        the similarity is 0.5. Otherwise, the similarity is 0.
        """

        # check if the zipcode are identical (return 1 or 0)
        sim = (s1 == s2).astype(float)

        # check the first 2 numbers of the distinct comparisons
        sim[(sim == 0) & (s1.str[0:2] == s2.str[0:2])] = 0.5

        return sim

comparer = rl.Compare()
comparer.extact('given_name', 'given_name', 'y_name')
comparer.string('surname', 'surname', 'y_surname')
comparer.add(CompareZipCodes('postcode', 'postcode', label='y_postcode'))
comparer.compute(pairs, dfA, dfB)

```

```python
0.0    71229
0.5     3166
1.0     2854
Name: sim_postcode, dtype: int64

```

--------------------------------

### Import preprocessing functions

Source: https://recordlinkage.readthedocs.io/en/latest/ref-preprocessing.html

Import the clean and phonetic functions from the recordlinkage.preprocessing submodule.

```python
from recordlinkage.preprocessing import clean, phonetic
```

--------------------------------

### Index candidate links with blocking

Source: https://recordlinkage.readthedocs.io/en/latest/about.html

Create a set of candidate links using the blocking indexing technique, which only includes record pairs that agree on the surname.

```python
indexer = recordlinkage.Index()
indexer.block('surname')
candidate_links = indexer.index(df_a, df_b)

```

--------------------------------

### Add Block index algorithm

Source: https://recordlinkage.readthedocs.io/en/latest/ref-index.html

Shows the shortcut for adding a Block index algorithm using the add method.

```python
from recordlinkage.index import Block

indexer = recordlinkage.Index()
indexer.add(Block())
```

--------------------------------

### Create full index with recordlinkage.Index().full()

Source: https://recordlinkage.readthedocs.io/en/latest/guides/data_deduplication.html

Loads the recordlinkage.Index class, calls the .full() method, and generates a full index on a single DataFrame. This creates all possible unique record pairs, which can be large; a performance warning is shown.

```python
indexer = recordlinkage.Index()
indexer.full()
candidate_links = indexer.index(dfA)
```

--------------------------------

### Generate annotation file for linking records

Source: https://recordlinkage.readthedocs.io/en/latest/annotation.html

Creates an annotation file for a linking task. Loads the Febrl4 dataset, blocks on 'surname', and writes the first 50 pairs to 'annotation_demo_linking.json' with dataset names.

```python
import recordlinkage as rl
from recordlinkage.index import Block
from recordlinkage.datasets import load_febrl4

df_a, df_b = load_febrl4()

blocker = Block("surname", "surname")
pairs = blocker.index(df_a, df_b)

rl.write_annotation_file(
    "annotation_demo_linking.json",
    pairs[0:50],
    df_a,
    df_b,
    dataset_a_name="Febrl4 A",
    dataset_b_name="Febrl4 B"
)

```

--------------------------------

### Date comparison shortcut usage

Source: https://recordlinkage.readthedocs.io/en/latest/ref-compare.html

Shows how to use the date() shortcut, which is a shortcut for recordlinkage.compare.Date. It creates a Compare object and adds a Date comparison.

```python
from recordlinkage.compare import Date

indexer = recordlinkage.Compare()
indexer.add(Date())

```

--------------------------------

### String comparison shortcut usage

Source: https://recordlinkage.readthedocs.io/en/latest/ref-compare.html

Shows how to use the string() shortcut, which is a shortcut for recordlinkage.compare.String. It creates a Compare object and adds a String comparison.

```python
from recordlinkage.compare import String

indexer = recordlinkage.Compare()
indexer.add(String())

```

--------------------------------

### Add Random index algorithm

Source: https://recordlinkage.readthedocs.io/en/latest/ref-index.html

Shows the shortcut for adding a Random index algorithm using the add method.

```python
from recordlinkage.index import Random

indexer = recordlinkage.Index()
indexer.add(Random())
```

--------------------------------

### Inspect first comparison vectors with features.head(10)

Source: https://recordlinkage.readthedocs.io/en/latest/guides/data_deduplication.html

Displays the first 10 rows of the comparison feature DataFrame. The output shows the comparison scores for each attribute for the first candidate pairs.

```python
features.head(10)
```

--------------------------------

### Summarize comparison features with features.describe()

Source: https://recordlinkage.readthedocs.io/en/latest/guides/data_deduplication.html

Computes descriptive statistics (count, mean, std, min, quartiles, max) for each comparison feature column. The output shows the distribution of similarity scores across all candidate pairs.

```python
features.describe()
```

--------------------------------

### recordlinkage.Compare constructor

Source: https://recordlinkage.readthedocs.io/en/latest/ref-compare.html

Initializes a Compare object for comparing record pairs. Accepts optional features, n_jobs, and indexing_type parameters.

```APIDOC
## recordlinkage.Compare

### Description
Class to compare record pairs with efficiently. Class to compare the attributes of candidate record pairs. The `Compare` class has methods like `string`, `exact` and `numeric` to initialise the comparing of the records. The `compute` method is used to start the actual comparing.

### Constructor
`recordlinkage.Compare(features=[], n_jobs=1, indexing_type='label', **kwargs)`

### Parameters
- **features** (list) - Optional - List of compare algorithms.
- **n_jobs** (integer, optional, default=1) - The number of jobs to run in parallel for comparing of record pairs. If -1, then the number of jobs is set to the number of cores.
- **indexing_type** (string, optional, default='label') - The indexing type. The MultiIndex is used to index the DataFrame(s). This can be done with pandas `.loc` or with `.iloc`. Use the value 'label' to make use of `.loc` and 'position' to make use of `.iloc`. The value 'position' is only available when the MultiIndex consists of integers. The value 'position' is much faster.

### Example
```python
# initialise class
comp = recordlinkage.Compare()

# initialise similarity measurement algorithms
comp.string('first_name', 'name', method='jarowinkler')
comp.string('lastname', 'lastname', method='jarowinkler')
comp.exact('dateofbirth', 'dob')
comp.exact('sex', 'sex')
comp.string('address', 'address', method='levenshtein')
comp.exact('place', 'place')
comp.numeric('income', 'income')

# the method .compute() returns the DataFrame with the feature vectors.
comp.compute(candidate_pairs, census_data_1980, census_data_1990)
```
```

--------------------------------

### Run tests with pytest

Source: https://recordlinkage.readthedocs.io/en/latest/contributing.html

Run the test suite for the package using pytest.

```bash
python -m pytest tests/

```

--------------------------------

### Index with block method

Source: https://recordlinkage.readthedocs.io/en/latest/ref-index.html

Creates an Index object, adds a block index on 'first_name' and 'givenname', and generates record pairs between two census datasets.

```python
indexer = recordlinkage.Index()
indexer.block(left_on='first_name', right_on='givenname')
indexer.index(census_data_1980, census_data_1990)
```

--------------------------------

### Import recordlinkage and pandas

Source: https://recordlinkage.readthedocs.io/en/latest/about.html

Import the recordlinkage module and the pandas data manipulation framework.

```python
import recordlinkage
import pandas

```

--------------------------------

### Blocking with Block indexer

Source: https://recordlinkage.readthedocs.io/en/latest/ref-index.html

Creates an indexer and adds Block rules on 'given_name' and 'surname' to generate candidate record pairs from the FEBRL4 dataset.

```python
import recordlinkage as rl
from recordlinkage.datasets import load_febrl4
from recordlinkage.index import Block

df_a, df_b = load_febrl4()

indexer = rl.Index()
indexer.add(Block('given_name', 'given_name'))
indexer.add(Block('surname', 'surname'))
indexer.index(df_a, df_b)

```

--------------------------------

### recordlinkage.datasets.load_febrl3(return_links=False)

Source: https://recordlinkage.readthedocs.io/en/latest/ref-datasets.html

Loads the FEBRL 3 dataset as a pandas DataFrame. Optionally returns the true links when return_links is True.

```APIDOC
## recordlinkage.datasets.load_febrl3(return_links=False)

### Description
Loads the FEBRL 3 dataset as a pandas DataFrame. The dataset contains 5000 records with duplicates. Optionally returns the true links when return_links is True.

### Method
Function call

### Endpoint
recordlinkage.datasets.load_febrl3

### Parameters
#### Parameters
- **return_links** (bool) - Optional - When True, the function returns also the true links.

### Returns
- **pandas.DataFrame** - A pandas DataFrame with Febrl dataset3.csv. When return_links is True, the function returns also the true links.
```

--------------------------------

### recordlinkage.datasets.load_febrl2(return_links=False)

Source: https://recordlinkage.readthedocs.io/en/latest/ref-datasets.html

Loads the FEBRL 2 dataset as a pandas DataFrame. Optionally returns the true links when return_links is True.

```APIDOC
## recordlinkage.datasets.load_febrl2(return_links=False)

### Description
Loads the FEBRL 2 dataset as a pandas DataFrame. The true links are all links in the lower triangular part of the matrix.

### Method
Function call

### Endpoint
recordlinkage.datasets.load_febrl2

### Parameters
#### Parameters
- **return_links** (bool) - Optional - When True, the function returns also the true links.

### Returns
- **pandas.DataFrame** - A pandas DataFrame with Febrl dataset2.csv. When return_links is True, the function returns also the true links.
```
Tokens:
Raw