### Match Example Instruments

Source: https://github.com/harmonydata/harmony/blob/main/README.md

This Python code demonstrates how to load example instruments and use Harmony to match them. It retrieves questions and similarity scores from the matching response.

```python
instruments = harmony.example_instruments["CES_D English"], harmony.example_instruments["GAD-7 Portuguese"]
match_response = harmony.match_instruments(instruments)

questions = match_response.questions
similarity = match_response.similarity_with_polarity
```

--------------------------------

### Install Harmony Python Library

Source: https://github.com/harmonydata/harmony/blob/main/Harmony_example_walkthrough.ipynb

Use this command to install the harmonydata library from PyPI. Ensure you have pip installed and configured.

```python
!pip install harmonydata
```

--------------------------------

### Install and Run Tox Locally

Source: https://github.com/harmonydata/harmony/blob/main/README.md

Install tox using pip and then run tox to execute local checks, including source distribution checks, setuptools checks, and unit tests with pytest. Tox manages environment setup and dependency installation.

```bash
pip install tox
tox
```

--------------------------------

### Install Harmony Python Library

Source: https://github.com/harmonydata/harmony/blob/main/README.md

Install the Harmony Python library using pip. This command installs the latest version from PyPI.

```bash
pip install harmonydata
```

--------------------------------

### Prepare Instruments for Matching

Source: https://github.com/harmonydata/harmony/blob/main/Harmony_example_walkthrough.ipynb

Assemble a list of instruments to be used in the matching process. This includes both pre-defined example instruments and custom-created ones.

```python
instruments = [harmony.example_instruments["CES_D English"],
               harmony.example_instruments["GAD-7 Portuguese"],
               gad_7_norwegian]
```

--------------------------------

### Install Matplotlib

Source: https://github.com/harmonydata/harmony/blob/main/Harmony_example_walkthrough.ipynb

Installs the matplotlib library, which is often used for data visualization with Harmony. This command ensures that matplotlib and its dependencies are up-to-date.

```bash
!pip install matplotlib
```

--------------------------------

### Import Harmony and Check Version

Source: https://github.com/harmonydata/harmony/blob/main/Harmony_example_walkthrough.ipynb

Import the Harmony library and verify its installed version. This is a standard first step when using the library.

```python
import harmony
print(harmony.__version__)
```

--------------------------------

### Run Apache Tika Server

Source: https://github.com/harmonydata/harmony/blob/main/README.md

This command starts the Apache Tika server, which is required for extracting text from PDF documents.

```bash
java -jar tika-server-standard-2.3.0.jar
```

--------------------------------

### Check Harmony Version

Source: https://github.com/harmonydata/harmony/blob/main/Harmony_example_walkthrough.ipynb

Retrieve the installed version of the Harmony library. Useful for ensuring compatibility or tracking library updates.

```python
harmony.__version__
```

--------------------------------

### Create a Custom Instrument

Source: https://github.com/harmonydata/harmony/blob/main/Harmony_example_walkthrough.ipynb

Create a new instrument from a list of question strings. This example shows how to define a Norwegian version of the GAD-7 instrument.

```python
from harmony import create_instrument_from_list

gad_7_norwegian = create_instrument_from_list(["Følt deg nervøs, engstelig eller veldig stresset",
                   "Ikke klart å slutte å bekymre deg eller kontrolleren bekymringene dine"],
                  instrument_name="GAD-7 Norwegian")
```

--------------------------------

### Running Harmony API with Docker

Source: https://github.com/harmonydata/harmony/blob/main/README.md

Starts the Harmony API service using a Docker container. This exposes the API on port 8000 for local access.

```bash
docker run -p 8000:80 harmonydata/harmonyapi
```

--------------------------------

### Re-release Harmony Package to PyPI

Source: https://github.com/harmonydata/harmony/blob/main/README.md

Manually re-release the Harmony package to PyPI. This involves activating a Python environment, installing twine, removing existing distribution files, creating a source distribution, and uploading it using twine.

```bash
source activate py311
pip install twine
rm -rf dist
python setup.py sdist
twine upload dist/*
```

--------------------------------

### Using OpenAI Embeddings for Vectorization

Source: https://github.com/harmonydata/harmony/blob/main/README.md

Customizes Harmony to use OpenAI's text-embedding-ada-002 model for vectorization. This requires an OpenAI account and client setup.

```python
import numpy as np
from harmony import match_instruments_with_function, example_instruments
from openai import OpenAI

client = OpenAI()
model_name = "text-embedding-ada-002"
def convert_texts_to_vector(texts):
    vectors = client.embeddings.create(input = texts, model=model_name).data
    return np.asarray([vectors[i].embedding for i in range(len(vectors))])
instruments = example_instruments["CES_D English"], example_instruments["GAD-7 Portuguese"]
match_response = match_instruments_with_function(instruments, None, convert_texts_to_vector)
```

--------------------------------

### Download spaCy Models for Harmony

Source: https://github.com/harmonydata/harmony/blob/main/README.md

This Python code snippet downloads the necessary spaCy models required by Harmony for text extraction from PDFs. Ensure you have the Harmony library installed.

```python
import harmony
harmony.download_models()
```

--------------------------------

### Matching Instruments

Source: https://github.com/harmonydata/harmony/blob/main/README.md

Initiates the instrument matching process using the harmony library. Requires a list of instruments as input.

```python
from harmony import match_instruments
match_response = match_instruments(instruments)
```

--------------------------------

### Create Instruments from List of Strings

Source: https://github.com/harmonydata/harmony/blob/main/README.md

This Python code shows how to create Harmony instruments directly from lists of strings and then match these instruments.

```python
from harmony import create_instrument_from_list, match_instruments
instrument1 = create_instrument_from_list(["I feel anxious", "I feel nervous"])
instrument2 = create_instrument_from_list(["I feel afraid", "I feel worried"])

match_response = match_instruments([instrument1, instrument2])
```

--------------------------------

### Import Harmony Library

Source: https://github.com/harmonydata/harmony/blob/main/Harmony_example_walkthrough.ipynb

Import the Harmony library to access its functionalities. This is the first step before using any other Harmony features.

```python
import harmony
```

--------------------------------

### Read __init__.py

Source: https://github.com/harmonydata/harmony/blob/main/update.ipynb

Reads the content of the '__init__.py' file to extract version information.

```python
with open("src/harmony/__init__.py", "r", encoding="utf-8") as f:
    text = f.read()
```

--------------------------------

### Load Instruments from Local File

Source: https://github.com/harmonydata/harmony/blob/main/README.md

This Python code imports the necessary function to load instruments from a local file, such as a PDF. The function returns a list of Instrument instances.

```python
from harmony import load_instruments_from_local_file
instruments = load_instruments_from_local_file("gad-7.pdf")
```

--------------------------------

### Update Version in __init__.py

Source: https://github.com/harmonydata/harmony/blob/main/update.ipynb

Parses the '__init__.py' file, finds the version string, increments the patch version, and updates the line in memory. Prints old and new versions.

```python
init_py_lines = text.split("\n")
for idx, line in list(enumerate(init_py_lines)):
    if "__version__" in line:
        old_version = re.sub(r'.+= \
```

```python
        version_bits = old_version.split(".")
        old_version_regex = r"\.".join(version_bits)
        version_bits[-1] = str(int(version_bits[-1]) + 1)
        new_version = ".".join(version_bits)
        init_py_lines[idx] = re.sub(old_version, new_version, line)
        
        print ("Old version", old_version)
        print ("New version", new_version)
```

--------------------------------

### Import Matplotlib for Plotting

Source: https://github.com/harmonydata/harmony/blob/main/Harmony_example_walkthrough.ipynb

Import the matplotlib.pyplot library for creating visualizations. This is a standard first step for plotting in Python.

```python
import matplotlib.pyplot as plt
```

--------------------------------

### Write Updated __init__.py

Source: https://github.com/harmonydata/harmony/blob/main/update.ipynb

Writes the modified lines back to the '__init__.py' file, saving the new version number.

```python
with open("src/harmony/__init__.py", "w", encoding="utf-8") as f:
    f.write("\n".join(init_py_lines))
```

--------------------------------

### Load Instrument from PDF File

Source: https://github.com/harmonydata/harmony/blob/main/README.md

This Python function loads instrument data from a local PDF file. Ensure the PDF file is accessible in the specified path.

```python
harmony.load_instruments_from_local_file("gad-7.pdf")
```

--------------------------------

### Import Instruments from Google Forms

Source: https://github.com/harmonydata/harmony/blob/main/README.md

This Python code demonstrates how to import questionnaires from Google Forms using a URL. It utilizes the `convert_files_to_instruments` function with a `RawFile` object specifying the Google Forms URL and type.

```python
from harmony import convert_files_to_instruments
from harmony.schemas.requests.text import RawFile
from harmony.schemas.enums.file_types import FileType

# Create a RawFile with the Google Forms URL
file = RawFile(
    file_name="Customer Satisfaction Survey",
    file_type=FileType.google_forms,
    content="https://docs.google.com/forms/d/e/1FAIpQLSc.../viewform"
)

# Convert to Harmony instruments
instruments = convert_files_to_instruments([file])
```

--------------------------------

### Accessing Instrument Questions

Source: https://github.com/harmonydata/harmony/blob/main/README.md

Iterates through instruments and their questions to print details. Useful for inspecting survey structures.

```python
for instrument in instruments:
    print(f"Form: {instrument.instrument_name}")
    for question in instrument.questions:
        print(f"{question.question_no}. {question.question_text}")
        if question.options:
            print(f"   Options: {', '.join(question.options)}")
```

--------------------------------

### Calculate Similarity Matrix

Source: https://github.com/harmonydata/harmony/blob/main/Harmony_example_walkthrough.ipynb

Calculates and displays the similarity matrix for response options. This is useful for understanding how different options relate to each other.

```python
response_options_similarity
```

--------------------------------

### Update Version in README.md

Source: https://github.com/harmonydata/harmony/blob/main/update.ipynb

Parses the 'README.md' file and updates any lines containing 'Version ' followed by the old version string.

```python
readme_lines = text.split("\n")
for idx, line in list(enumerate(readme_lines)):
    if "Version " in line:
        readme_lines[idx] = re.sub("Version " + old_version_regex, "Version " + new_version, line)
```

--------------------------------

### Display Matched Questions and Topics

Source: https://github.com/harmonydata/harmony/blob/main/Harmony_example_walkthrough.ipynb

Iterate through the matched questions and print their text along with any associated topics. This helps in reviewing the results of the matching process.

```python
for q in questions:
    print (q.question_text)
    print("\t", "Topics:", q.topics)
```

--------------------------------

### Read README.md

Source: https://github.com/harmonydata/harmony/blob/main/update.ipynb

Reads the content of the 'README.md' file to update the displayed version information.

```python
with open("README.md", "r", encoding="utf-8") as f:
    text = f.read()
```

--------------------------------

### Match Instruments to Topics

Source: https://github.com/harmonydata/harmony/blob/main/Harmony_example_walkthrough.ipynb

Match a list of instruments against a set of topics to identify relevant questions. This function returns questions, similarity scores, and response option similarities.

```python
match_response = harmony.match_instruments(instruments, topics=[
    "anxiety", "nervous", "difficulty", "scared", "unhappy", "sleep", "eating"
    ])

questions = match_response.questions
similarity = match_response.similarity_with_polarity
response_options_similarity = match_response.response_options_similarity
```

--------------------------------

### Import Harmony Crosswalk Function

Source: https://github.com/harmonydata/harmony/blob/main/Harmony_example_walkthrough.ipynb

Import the generate_crosswalk_table function from the harmony.matching module. This function is used to create a table of matched question pairs.

```python
from harmony.matching.generate_crosswalk_table import generate_crosswalk_table
```

--------------------------------

### Write Updated README.md

Source: https://github.com/harmonydata/harmony/blob/main/update.ipynb

Writes the modified lines back to the 'README.md' file, saving the updated version information.

```python
with open("README.md", "w", encoding="utf-8") as f:
    f.write("\n".join(readme_lines))
```

--------------------------------

### Read CITATION.cff

Source: https://github.com/harmonydata/harmony/blob/main/update.ipynb

Reads the content of the 'CITATION.cff' file to prepare for version updates.

```python
with open("CITATION.cff", "r", encoding="utf-8") as f:
    text = f.read()
```

--------------------------------

### Cluster Questions and Display Results

Source: https://github.com/harmonydata/harmony/blob/main/Harmony_example_walkthrough.ipynb

Iterate through the clusters generated by Harmony and print the cluster ID, description, keywords, and associated questions. This helps in understanding the thematic groupings of questions.

```python
for cluster in match_response.clusters:
    print (f"Cluster #{cluster.cluster_id}: {cluster.text_description}")
    print (f"Keywords: {cluster.keywords}")
    for question in cluster.items:
        print ("\t", question.question_text)
    print ("\n")
```

--------------------------------

### Add Updated Files to Git Staging

Source: https://github.com/harmonydata/harmony/blob/main/update.ipynb

Stages the modified version files using Git, preparing them for commit.

```bash
!git add src/harmony/__init__.py
!git add CITATION.cff README.md pyproject.toml
```

--------------------------------

### Set Google Forms API Key Environment Variable

Source: https://github.com/harmonydata/harmony/blob/main/README.md

This bash command sets the Google Forms API key as an environment variable. This is a prerequisite for using Harmony's Google Forms integration.

```bash
export GOOGLE_FORMS_API_KEY="your-api-key-here"
```

--------------------------------

### Generate Crosswalk Table

Source: https://github.com/harmonydata/harmony/blob/main/Harmony_example_walkthrough.ipynb

Generate the crosswalk table using the imported function. This requires instruments, similarity scores, and the defined threshold. Options to allow within-instrument matches and enforce one-to-one matching are available.

```python
df_crosswalk_table = generate_crosswalk_table(instruments, similarity, threshold, is_allow_within_instrument_matches = True, is_enforce_one_to_one = True)
```

--------------------------------

### Setting Sentence Transformer Path

Source: https://github.com/harmonydata/harmony/blob/main/README.md

Configures the environment variable to specify a custom sentence transformer model for vectorization. This affects how text is converted into numerical representations.

```bash
export HARMONY_SENTENCE_TRANSFORMER_PATH=sentence-transformers/distiluse-base-multilingual-cased-v2
```

--------------------------------

### Cluster Questions using K-Means

Source: https://github.com/harmonydata/harmony/blob/main/Harmony_example_walkthrough.ipynb

Clusters a list of questions using the k-means algorithm. It takes a list of questions, the desired number of clusters, and an option to display a graph. Returns a DataFrame with cluster assignments and a score indicating clustering quality.

```python
from harmony import cluster_questions
df, score = cluster_questions(match_response.questions, num_clusters = 5, is_show_graph = True)

print (f"Score = {score}")

df
```

--------------------------------

### Creating a RawFile for Google Forms

Source: https://github.com/harmonydata/harmony/blob/main/README.md

Defines a RawFile object for a Google Form using its ID. This is used to represent survey data sources.

```python
file = RawFile(
    file_name="Survey",
    file_type=FileType.google_forms,
    content="1FAIpQLSc_form_id_here"
)
```

--------------------------------

### Update Version in CITATION.cff

Source: https://github.com/harmonydata/harmony/blob/main/update.ipynb

Parses the 'CITATION.cff' file and updates the version string using the previously determined new version.

```python
citation_lines = text.split("\n")
for idx, line in list(enumerate(citation_lines)):
    if line.startswith("version:"):
        citation_lines[idx] = re.sub(old_version_regex, new_version, line)
```

--------------------------------

### Update Version in pyproject.toml

Source: https://github.com/harmonydata/harmony/blob/main/update.ipynb

Reads, updates, and writes back the 'pyproject.toml' file with the new version number.

```python
with open("pyproject.toml", "r", encoding="utf-8") as f:
    text = f.read()
pyproject_lines = text.split("\n")
for idx, line in list(enumerate(pyproject_lines)):
    if "version " in line:
        pyproject_lines[idx] = re.sub(old_version_regex, new_version, line)
with open("pyproject.toml", "w", encoding="utf-8") as f:
    f.write("\n".join(pyproject_lines))
```

--------------------------------

### Import Regular Expression Module

Source: https://github.com/harmonydata/harmony/blob/main/update.ipynb

Imports the 're' module for regular expression operations, which are used for version manipulation.

```python
import re
```

--------------------------------

### Display Similarities Between Instruments

Source: https://github.com/harmonydata/harmony/blob/main/Harmony_example_walkthrough.ipynb

Iterates through instrument-to-instrument similarities and prints the F1 similarity score between pairs of instruments. This is useful for understanding how similar different assessment instruments are to each other.

```python
for similarity in match_response.instrument_to_instrument_similarities:
    print (f"F1 similarity of {similarity.instrument_1_name} to {similarity.instrument_2_name}:	{similarity.f1}")
```

--------------------------------

### Write Updated CITATION.cff

Source: https://github.com/harmonydata/harmony/blob/main/update.ipynb

Writes the modified lines back to the 'CITATION.cff' file, saving the new version number.

```python
with open("CITATION.cff", "w", encoding="utf-8") as f:
    f.write("\n".join(citation_lines))
```

--------------------------------

### View Similarity Scores

Source: https://github.com/harmonydata/harmony/blob/main/Harmony_example_walkthrough.ipynb

Display the calculated similarity scores between instruments and topics. These scores indicate the degree of relevance.

```python
similarity
```

--------------------------------

### Run Tox for a Specific Python Version

Source: https://github.com/harmonydata/harmony/blob/main/README.md

Execute tox tests for a specific Python version, such as Python 3.9, by using the -e flag. This is useful when you want to test against a particular environment.

```bash
tox -e py39
```

--------------------------------

### Commit Version Update

Source: https://github.com/harmonydata/harmony/blob/main/update.ipynb

Commits the staged changes to the Git repository with a descriptive message.

```bash
!git commit -m "Update version"
```

--------------------------------

### Display Crosswalk Table

Source: https://github.com/harmonydata/harmony/blob/main/Harmony_example_walkthrough.ipynb

Print the generated crosswalk table DataFrame to view the matched question pairs, their texts, and match scores.

```python
df_crosswalk_table
```

--------------------------------

### Plot Similarity Matrix

Source: https://github.com/harmonydata/harmony/blob/main/Harmony_example_walkthrough.ipynb

Display the similarity matrix using matplotlib's imshow function. Use a 'hot' colormap and 'nearest' interpolation for visualization.

```python
plt.imshow(similarity, cmap='hot', interpolation='nearest')
```

--------------------------------

### Set Crosswalk Threshold

Source: https://github.com/harmonydata/harmony/blob/main/Harmony_example_walkthrough.ipynb

Define the similarity threshold for generating the crosswalk table. Matches with a score below this threshold will be excluded.

```python
threshold = 0.6
```

--------------------------------

### Push Changes to Remote Repository

Source: https://github.com/harmonydata/harmony/blob/main/update.ipynb

Pushes the committed changes to the remote Git repository.

```bash
!git push
```

--------------------------------

### BibTeX Entry for Harmony Citation

Source: https://github.com/harmonydata/harmony/blob/main/README.md

This is a BibTeX entry for citing the Harmony validation paper in LaTeX documents. It includes all necessary bibliographic details.

```bibtex
@article{mcelroy2024using,
  title={Using natural language processing to facilitate the harmonisation of mental health questionnaires: a validation study using real-world data},
  author={McElroy, Eoin and Wood, Thomas and Bond, Raymond and Mulvenna, Maurice and Shevlin, Mark and Ploubidis, George B and Hoffmann, Mauricio Scopel and Moltrecht, Bettina},
  journal={BMC psychiatry},
  volume={24},
  number={1},
  pages={530},
  year={2024},
  publisher={Springer}
}
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.