### Setup ThreatExchange Access Token

Source: https://github.com/facebook/threatexchange/blob/main/api-reference-examples/python-notebook/Getting Started with Sharing.ipynb

This Python snippet shows how to set up an access token for ThreatExchange using the pytx library. It demonstrates calling the access_token function, which handles token retrieval.

```python
from pytx.access_token import access_token

# Specify the location of your token via one of several ways:
# https://pytx.readthedocs.org/en/latest/pytx.access_token.html
access_token()
```

--------------------------------

### Install golang-threatexchange Library

Source: https://github.com/facebook/threatexchange/blob/main/api-reference-examples/go/README.md

Installs the golang-threatexchange library using the go get command. This is the standard way to fetch Go packages.

```bash
go get github.com/Facebook/ThreatExchange/api-reference-examples/go/threatexchange
```

--------------------------------

### Enable Debug Logging

Source: https://github.com/facebook/threatexchange/blob/main/api-reference-examples/python-notebook/Getting Started with Sharing.ipynb

This Python code snippet demonstrates how to enable debug level logging for the pytx library. It uses the setup_logger function, which can be configured to write logs to a file.

```python
from pytx.logger import setup_logger

# Uncomment this, if you want debug logging enabled
# setup_logger(log_file="pytx.log")
```

--------------------------------

### Install pytx using pip

Source: https://github.com/facebook/threatexchange/blob/main/api-reference-examples/python/pytx/docs/installation.rst

This command installs the pytx library using the pip package installer. It is the recommended method for setting up the library.

```bash
$ pip install pytx
```

--------------------------------

### Setup pytx Logger

Source: https://github.com/facebook/threatexchange/blob/main/api-reference-examples/python/pytx/docs/quickstart.rst

Configures the pytx logger to write debug information to a specified log file. If no file path is provided, logging will not occur. This is crucial for troubleshooting pytx operations.

```python
from pytx import setup_logger
setup_logger('/path/to/my/log/file.log')
```

--------------------------------

### Direct Broker Usage for ThreatExchange API

Source: https://github.com/facebook/threatexchange/blob/main/api-reference-examples/python/pytx/docs/quickstart.rst

Demonstrates how to use the pytx Broker directly to make GET requests to the ThreatExchange API. This approach bypasses the convenience classes and allows for manual URL and parameter construction.

```python
from pytx.request import Broker
from pytx.vocabulary import ThreatExchange as te

b = Broker()
url = te.URL + te.THREAT_DESCRIPTORS
params = {te.TEXT: "www.facebook.com"}
response = b.get(url, params)
```

--------------------------------

### Confirm Data Shared to ThreatExchange

Source: https://github.com/facebook/threatexchange/blob/main/api-reference-examples/python-notebook/Getting Started with Sharing.ipynb

This Python snippet demonstrates how to confirm data has been shared to ThreatExchange by performing a search. It imports necessary libraries for datetime, time, pandas, and pytx.

```python
from datetime import datetime, timedelta
from time import strftime
import pandas as pd
from pytx import ThreatDescriptor
from pytx.vocabulary import ThreatExchange as te
```

--------------------------------

### Placeholder for Sharing Malicious Data

Source: https://github.com/facebook/threatexchange/blob/main/api-reference-examples/python-notebook/Getting Started with Sharing.ipynb

This is a placeholder for writing Python code to share lists of malicious URLs or domain names with other members or privacy groups within the ThreatExchange platform.

```python
# Put your Python code here!
```

--------------------------------

### Configure SSL Context for ThreatExchange RTUListener

Source: https://github.com/facebook/threatexchange/blob/main/api-reference-examples/python/pytx/docs/quickstart.rst

This example shows how to create a custom SSLContext using Python's 'ssl' module to enable HTTPS for the RTUListener, ensuring secure communication. It requires certificate and key files.

```python
import ssl
ssl_context = ssl.SSLContext(ssl.PROTOCOL_TLSv1_2)ssl_context.load_cert_chain(certfile='<your_cert_file.pem',
                               keyfile='<your_key_file.key')
```

--------------------------------

### Query ThreatDescriptor Objects and Create DataFrame

Source: https://github.com/facebook/threatexchange/blob/main/api-reference-examples/python-notebook/Getting Started with Sharing.ipynb

This snippet demonstrates how to query ThreatDescriptor objects with specific parameters like owner, time range, and field limits. It then converts the results into a pandas DataFrame for easier analysis and displays the first 10 rows.

```python
from datetime import datetime, timedelta
from time import strftime
import pandas as pd

# Assuming ThreatDescriptor, get_app_id, and te are defined elsewhere
# Example placeholder definitions:
class ThreatDescriptor:
    _default_fields = ['id', 'type', 'value', 'description']
    def __init__(self, **kwargs):
        self.__dict__.update(kwargs)
    def to_dict(self):
        return self.__dict__

def get_app_id():
    return '12345'

class te:
    LIMIT = 'limit'

# Define your search string and other params, see 
# https://pytx.readthedocs.org/en/latest/pytx.common.html#pytx.common.Common.objects
# for the full list of options
results = ThreatDescriptor.objects(
    fields=ThreatDescriptor._default_fields,
    limit=100, # Example limit
    owner=str(get_app_id()),
    since=strftime('%Y-%m-%d %H:%M:%S +0000', (datetime.utcnow() + timedelta(hours=(-1))).timetuple()), 
    until=strftime('%Y-%m-%d %H:%M:%S +0000', datetime.utcnow().timetuple())
)

data_frame = pd.DataFrame([result.to_dict() for result in results])
print(data_frame.head(n=10))
```

--------------------------------

### Install pre-commit hooks

Source: https://github.com/facebook/threatexchange/blob/main/api-reference-examples/python/pytx/CONTRIBUTING.md

Installs the pre-commit framework for the pytx project. This ensures code adheres to PEP-8 and passes basic linting checks before committing.

```bash
pre-commit install
```

--------------------------------

### Create ThreatDescriptor using .new()

Source: https://github.com/facebook/threatexchange/blob/main/api-reference-examples/python/pytx/docs/quickstart.rst

Shows how to create a ThreatDescriptor via the class method `.new()`, passing parameters in a dictionary. This method is an alternative to instantiating and saving manually.

```python
from pytx import ThreatDescriptor
from pytx.vocabulary import PrivacyType as pt
from pytx.vocabulary import ThreatDescriptor as td

result = ThreatDescriptor.new(params={td.INDICATOR: 'foo',
                                      td.PRIVACY_TYPE: pt.VISIBLE
                                     })
```

--------------------------------

### Install or Upgrade pytx using pip

Source: https://github.com/facebook/threatexchange/blob/main/api-reference-examples/python/pytx/README.rst

This command demonstrates how to install the pytx library using pip. The optional [--upgrade] flag can be used to update the library to the latest version if it's already installed.

```bash
pip install pytx [--upgrade]
```

--------------------------------

### Install and Activate Virtualenv for Python Development

Source: https://github.com/facebook/threatexchange/blob/main/python-threatexchange/CONTRIBUTING.md

Steps to install virtualenv using pip, create a virtual environment in the current directory, activate it, and install the project with development dependencies.

```shell
pip install virtualenv
virtualenv .venv
source .venv/bin/activate
pip install --editable '.[dev]'
```

--------------------------------

### Create ThreatDescriptor using .send()

Source: https://github.com/facebook/threatexchange/blob/main/api-reference-examples/python/pytx/docs/quickstart.rst

Illustrates creating a ThreatDescriptor using the `.send()` method, specifying the HTTP method and parameters. This provides flexibility for different API operations.

```python
from pytx import ThreatDescriptor
from pytx.vocabulary import PrivacyType as pt
from pytx.vocabulary import ThreatDescriptor as td

result = ThreatDescriptor.send(params={td.INDICATOR: 'foo',
                                      td.PRIVACY_TYPE: pt.VISIBLE
                                     },
                                  type_='POST'
                                 )
```

--------------------------------

### Install python-threatexchange

Source: https://github.com/facebook/threatexchange/blob/main/python-threatexchange/README.md

Installs the python-threatexchange library using pip. Ensure you have pip installed or follow the provided link for instructions.

```bash
python3 -m pip install threatexchange --upgrade
```

--------------------------------

### Configure ThreatExchange Privacy Settings

Source: https://github.com/facebook/threatexchange/blob/main/api-reference-examples/python-notebook/Getting Started with Sharing.ipynb

This Python snippet configures privacy settings for ThreatExchange API defaults. It uses PrivacyType from pytx.vocabulary to set the privacy level and defines members for sharing.

```python
from pytx.access_token import get_app_id
from pytx.vocabulary import PrivacyType as pt

# Choose the privacy level from 
# https://pytx.readthedocs.org/en/latest/pytx.vocabulary.html#pytx.vocabulary.PrivacyType
privacy_type = pt.HAS_WHITELIST 

# Populate this with strings of app IDs or privacy groups.  If using pt.VISIBLE, set to None
privacy_members=[str(get_app_id())] # Will also take other member or privacy group IDs as strings
```

--------------------------------

### Install vpdq Python Package

Source: https://github.com/facebook/threatexchange/blob/main/vpdq/python/README.md

Installs the vpdq Python package from PyPI using pip. This is the recommended way to install the library for general use.

```sh
python -m pip install vpdq --upgrade
```

--------------------------------

### Install Node Modules

Source: https://github.com/facebook/threatexchange/blob/main/pdq/wasm/README.md

Installs the necessary Node.js modules for the web server used in testing. This is typically run in the Web Assembly module's directory.

```bash
npm install
```

--------------------------------

### Run Python API Examples

Source: https://github.com/facebook/threatexchange/blob/main/api-reference-examples/python/te-tag-query/README.md

Execute example Python scripts to interact with the ThreatExchange API. These scripts demonstrate common API operations.

```python
python api-example-submit.py
```

```python
python api-example-update.py
```

```python
python api-example-copy.py
```

--------------------------------

### Local Installation of ThreatExchange

Source: https://github.com/facebook/threatexchange/blob/main/python-threatexchange/CONTRIBUTING.md

Commands to install the ThreatExchange Python library locally for development. This involves navigating to the project directory, packaging the library, and performing a local installation.

```shell
cd ThreatExchange/python-threatexchange
make package
make local_install
threatexchange --help
```

--------------------------------

### Create and Listen with ThreatExchange RTUListener

Source: https://github.com/facebook/threatexchange/blob/main/api-reference-examples/python/pytx/docs/quickstart.rst

This snippet demonstrates how to define a custom callback function to process incoming POST requests from ThreatExchange and initiate the RTUListener. It requires the 'pytx' library.

```python
from pytx import RTUListener

def my_callback(request):
    print "POST Data: {}".format(request)
    return "200 OK"

my_listener = RTUListener(host='0.0.0.0',
                            port=1337,
                            listener_url='/threatexchange/',
                            get_response="hello",
                            callback=my_callback)
my_listener.listen()
```

--------------------------------

### Start HMA Demo Instance with Docker Compose

Source: https://github.com/facebook/threatexchange/blob/main/hasher-matcher-actioner/README.md

This command uses Docker Compose to start a demo instance of the HMA application, including a PostgreSQL database, with all necessary configurations ready for testing.

```sh
docker compose up
```

--------------------------------

### Install Packaging Requirements for vPDQ

Source: https://github.com/facebook/threatexchange/blob/main/vpdq/CONTRIBUTING.md

Installs the necessary Python dependencies required for packaging the vPDQ Python bindings, as listed in 'packaging-requirements.txt'.

```sh
python -m pip install -r packaging-requirements.txt
```

--------------------------------

### Run Node Server

Source: https://github.com/facebook/threatexchange/blob/main/pdq/wasm/README.md

Starts a local Node.js server to serve the Web Assembly files. The port number is specified in the server.js file.

```bash
node server.js
```

--------------------------------

### Query ThreatDescriptors by Type and Text

Source: https://github.com/facebook/threatexchange/blob/main/api-reference-examples/python/pytx/docs/quickstart.rst

Shows how to query ThreatDescriptors by specifying both the type (e.g., IP_ADDRESS) and text content. It iterates through results and prints the indicator field.

```python
from pytx import ThreatDescriptor
from pytx.vocabulary import ThreatDescriptor as td
from pytx.vocabulary import Types as t

# type is type_ because type is a reserved word.
results = ThreatDescriptor.objects(type_=t.IP_ADDRESS,
                                   text='127.0.0.1')
for result in results:
    print result.get(td.INDICATOR)
```

--------------------------------

### Query ThreatDescriptors by Text

Source: https://github.com/facebook/threatexchange/blob/main/api-reference-examples/python/pytx/docs/quickstart.rst

Demonstrates querying for ThreatDescriptor objects based on text content. It shows how to retrieve a summary of results and access specific fields like confidence.

```python
from pytx import ThreatDescriptor
from pytx.vocabulary import ThreatDescriptor as td
from pytx.vocabulary import Types as t

results = ThreatDescriptor.objects(text='www.facebook.com')
for result in results:
    print result.get(td.CONFIDENCE)
```

--------------------------------

### Install Python Packages using Pip

Source: https://github.com/facebook/threatexchange/blob/main/api-reference-examples/python-notebook/README.md

This command installs Python packages using pip, a package installer for Python. It's a common method for managing project dependencies.

```shell
pip install <package_name>
```

--------------------------------

### Install vPDQ Dev Tools with Pip

Source: https://github.com/facebook/threatexchange/blob/main/vpdq/CONTRIBUTING.md

Installs clang-format and other necessary development tools for the vPDQ Python binding using pip with the 'dev' optional dependency.

```sh
python -m pip install -e .[dev]
```

--------------------------------

### Start ThreatExchange Wiki Local Editor

Source: https://github.com/facebook/threatexchange/wiki/README

Runs the Docker container to start the local wiki editor. It maps the host's current directory to the container's wiki volume and exposes the editor on port 9999.

```shell
ThreatExchange.wiki$ docker run -p 9999:80 -v`pwd`:/var/wiki threatexchange.wiki:latest
```

--------------------------------

### Install libav* Dependencies (Ubuntu)

Source: https://github.com/facebook/threatexchange/blob/main/vpdq/README.md

Installs essential development libraries for libav* on Ubuntu systems using apt-get. These are required for building the vpdq C++ library.

```sh
sudo apt-get install -y libavdevice-dev libavfilter-dev libavformat-dev libavcodec-dev libswresample-dev libswscale-dev libavutil-dev
```

--------------------------------

### Submit Example (Ruby)

Source: https://github.com/facebook/threatexchange/blob/main/api-reference-examples/ruby/te-tag-query/README.md

Demonstrates submitting data to the ThreatExchange API using a Ruby script. This script likely handles authentication and data formatting for submission.

```ruby
ruby api-example-submit.rb
```

--------------------------------

### Install vPDQ Python Bindings (Editable Mode)

Source: https://github.com/facebook/threatexchange/blob/main/vpdq/CONTRIBUTING.md

Installs the vPDQ Python bindings in editable mode from the vpdq/ directory. This allows for development and testing of the Python components.

```sh
# vpdq/
python -m pip install --editable .
```

--------------------------------

### CLI Command Example: Fetching Datasets

Source: https://github.com/facebook/threatexchange/blob/main/python-threatexchange/README.md

Demonstrates how to use the `threatexchange` CLI to fetch datasets, specifically mentioning the `-P` flag which is part of the public interface.

```shell
threatexchange dataset -P
```

--------------------------------

### vstr File Format Example

Source: https://github.com/facebook/threatexchange/blob/main/tmk/README-more.md

Example of the binary structure of a .vstr file, which stores decoded video stream data. The output shows the initial header containing magic numbers and dimensions, followed by the RGB frame-raster data.

```bash
# The header includes the dimensions of the data
$ hex ~/wideband/tmk/vstr/misc-shelf-sd.vstr | head
00000000: 54 4d 4b 31  56 53 54 52  90 01 00 00  e0 00 00 00 |TMK1VSTR........|
00000010: 0f 00 00 00  00 00 00 00  00 00 00 00  00 00 00 00 |................|
00000020: 3d 47 61 3d  47 61 3d 47  61 3d 47 61  3d 47 61 3d |=Ga=Ga=Ga=Ga=Ga=|
00000030: 47 61 3d 47  61 3d 47 61  3d 47 61 3d  47 61 3d 47 |Ga=Ga=Ga=Ga=Ga=G|
00000040: 61 3d 47 61  3d 47 61 3d  47 61 3d 47  61 3d 47 61 |a=Ga=Ga=Ga=Ga=Ga|
00000050: 3e 48 62 3e  48 62 3d 47  61 3d 47 61  3d 47 61 3b |>Hb>Hb=Ga=Ga=Ga;|
00000060: 45 5f 3a 44  5e 3a 44 5e  38 42 5c 38  42 5c 37 41 |E_:D^:D^8B\8B\7A|
00000070: 5b 36 40 5a  36 40 5a 34  3e 58 34 3e  58 34 3e 58 |[6@Z6@Z4>X4>X4>X|
00000080: 33 3d 57 33  3d 57 33 3d  57 33 3d 57  33 3d 57 33 |3=W3=W3=W3=W3=W3|
00000090: 3d 57 32 3c  56 32 3c 56  30 3a 54 30  3a 54 2f 39 |=W2<V2<V0:T0:T/9|
```

--------------------------------

### React to a Threat Descriptor

Source: https://github.com/facebook/threatexchange/blob/main/api-reference-examples/curl/te-tag-query/README.md

React to another application's threat descriptor with predefined reactions like HELPFUL or SAW_THIS_TOO.

```APIDOC
## POST /{application_id}/

### Description
Allows an application to react to another application's threat descriptor.

### Method
POST

### Endpoint
`https://graph.facebook.com/v6.0/{application_id}/`

### Parameters
#### Path Parameters
- **application_id** (string) - Required - The ID of the application whose threat descriptor is being reacted to.

#### Query Parameters
- **access_token** (string) - Required - The access token for authentication.
- **reactions** (string) - Required - A comma-separated list of reactions (e.g., HELPFUL, SAW_THIS_TOO).

### Request Example
```json
{
  "access_token": "$TX_ACCESS_TOKEN",
  "reactions": "HELPFUL,SAW_THIS_TOO"
}
```

### Response
#### Success Response (200)
- **success** (boolean) - Indicates if the operation was successful.

#### Response Example
```json
{
  "success": true
}
```
```

--------------------------------

### Get Threat Descriptor Details

Source: https://github.com/facebook/threatexchange/blob/main/api-reference-examples/curl/te-tag-query/README.md

Retrieves detailed information for one or more threat descriptor IDs. This is the final step to get all metadata associated with hashes found by tag.

```APIDOC
## GET /

### Description
Fetches detailed information for specified threat descriptor IDs. This endpoint allows you to retrieve comprehensive metadata for multiple hashes in a single request.

### Method
GET

### Endpoint
`https://graph.facebook.com/v4.0/`

### Query Parameters
- **access_token** (string) - Required - Your Facebook App's access token.
- **ids** (string) - Required - A comma-separated or URL-encoded array of threat descriptor IDs (e.g., `[ID1,ID2]`).
- **fields** (string) - Required - A comma-separated list of fields to retrieve for each descriptor (e.g., `raw_indicator,type,added_on,last_updated,confidence,owner,privacy_type,review_status,status,severity,share_level,tags,description`).

### Request Example
```bash
curl -s 'https://graph.facebook.com/v4.0/'\\
"?access_token=$TX_ACCESS_TOKEN"\\
'&ids=%5B2556006484495859,2356527701137016%5D'\\
'&fields=raw_indicator,type,added_on,last_updated,confidence,owner,privacy_type,review_status,status,severity,share_level,tags,description'
```

### Response
#### Success Response (200)
- Returns a JSON object where each key is a threat descriptor ID and the value is an object containing the requested fields (e.g., `raw_indicator`, `type`, `added_on`, `tags`, `description`).

#### Response Example
```json
{
   "2556006484495859": {
      "raw_indicator": "ginuwine1551474919abc-2.evilevillabs.com",
      "type": "DOMAIN",
      "added_on": "2019-11-09T02:58:41+0000",
      "last_updated": "2019-11-09T02:58:42+0000",
      "confidence": 80,
      "owner": {
         "id": "494491891138576",
         "email": "redacted@redacted.com",
         "name": "Media Hash Sharing RF Test"
      },
      "privacy_type": "HAS_PRIVACY_GROUP",
      "review_status": "REVIEWED_MANUALLY",
      "status": "UNKNOWN",
      "severity": "INFO",
      "share_level": "AMBER",
      "tags": {
         "data": [
            {
               "id": "1283621968426798",
               "text": "pwny"
            },
            {
               "id": "1928945133884049",
               "text": "ignore"
            },
            {
               "id": "884078131700721",
               "text": "testing"
            }
         ]
      },
      "description": "Testing camel-casing codemod",
      "id": "2556006484495859"
   }
}
```
```

--------------------------------

### Get Tagged Objects by Tag ID

Source: https://github.com/facebook/threatexchange/blob/main/api-reference-examples/curl/te-tag-query/README.md

Retrieves a list of objects (e.g., hashes) associated with a specific tag ID. This endpoint is used to get the IDs of threat descriptors linked to a tag.

```APIDOC
## GET /{tag_id}/tagged_objects/

### Description
Fetches objects associated with a given tag ID. This is the second step in retrieving hashes by tag, providing IDs of threat descriptors.

### Method
GET

### Endpoint
`https://graph.facebook.com/v4.0/{tag_id}/tagged_objects/`

### Parameters
#### Path Parameters
- **tag_id** (string) - Required - The ID of the threat tag.

#### Query Parameters
- **access_token** (string) - Required - Your Facebook App's access token.
- **limit** (integer) - Optional - The maximum number of results to return per page.

### Request Example
```bash
curl -s 'https://graph.facebook.com/v4.0/1283621968426798/tagged_objects/'\\
"?access_token=$TX_ACCESS_TOKEN"\\
'&limit=100'
```

### Response
#### Success Response (200)
- **data** (array) - An array of tagged objects, each with an `id`, `type`, and `name` (which can be the hash).
- **paging** (object) - Pagination information, including a `next` URL if more results are available.

#### Response Example
```json
{
   "data": [
      {
         "id": "2556006484495859",
         "type": "THREAT_DESCRIPTOR",
         "name": "ginuwine1551474919abc-2.evilevillabs.com"
      },
      {
         "id": "2356527701137016",
         "type": "THREAT_DESCRIPTOR",
         "name": "e8b19da37825a3056e84c522f05ee081"
      }
  ],
  "paging": {
    "cursors": {
      "before": "REDACTED",
      "after": "REDACTED"
    },
   "next": "https://graph.facebook.com/v6.0/2733125556794397/tagged_objects?access_token=REDACTED&limit=1000&after=..."
  }
}
```
```

--------------------------------

### Copy Example (Ruby)

Source: https://github.com/facebook/threatexchange/blob/main/api-reference-examples/ruby/te-tag-query/README.md

Shows how to copy data between different entities or locations within the ThreatExchange API using a Ruby script. This is useful for data migration or sharing.

```ruby
ruby api-example-copy.rb
```

--------------------------------

### Querying hashes by tag (Ruby CLI)

Source: https://github.com/facebook/threatexchange/blob/main/api-reference-examples/ruby/te-tag-query/README.md

Examples demonstrating how to query ThreatExchange for hashes associated with specific tags. This includes basic queries, verbose output, and filtering by date.

```bash
te-tag-query-ruby --help
```

```bash
te-tag-query-ruby tag-to-details media_type_photo
te-tag-query-ruby -v tag-to-details media_type_photo
te-tag-query-ruby tag-to-details media_type_video
```

```bash
te-tag-query-ruby tag-to-details --tagged-since -1day media_type_photo
te-tag-query-ruby tag-to-details --tagged-since -1week media_type_video
```

--------------------------------

### Authenticate pytx with Access Token

Source: https://github.com/facebook/threatexchange/blob/main/api-reference-examples/python/pytx/docs/quickstart.rst

Demonstrates multiple ways to provide an access token to pytx for authentication. This includes using environment variables, a token file, or directly passing app ID and secret. It also shows how to retrieve the currently used access token.

```python
from pytx.access_token import access_token

# Use environment variables to build the access token.
# 1. Use the value of the 'TX_ACCESS_TOKEN' environment variable.
# 2. Use the concatenation of the 'TX_APP_ID' and 'TX_APP_SECRET' environment variables.
# There is no need to call access_token() if the environment variables are set.

# 3. Use a .pytx file which contains your app-id and app-secret.
# File should be: 'app-id|app-secret' on one line
# pytx will use either '$PWD/.pytx' or '~/.pytx' if they are found.
# There is no need to call access_token() if the environment variables are set.

# 4. Use the concatenation of the app_id and app_secret parameters
access_token(app_id='<app-id>', app_secret='<app-secret>')

# 5. Use the first line of the file 'token_file'
access_token(token_file='/path/to/token/file')
```

```python
from pytx.access_token import get_access_token
print get_access_token()
```

--------------------------------

### Match Text Content with threatexchange CLI

Source: https://github.com/facebook/threatexchange/blob/main/python-threatexchange/README.md

Demonstrates matching text content against available signals using the threatexchange CLI. It shows example outputs for different matching types.

```bash
$ threatexchange match text -- 'bball now?'
raw_text - (Sample Signals) INVESTIGATION_SEED
trend_query - (Sample Signals) INVESTIGATION_SEED
```

--------------------------------

### Submitting Batch Requests with pytx

Source: https://github.com/facebook/threatexchange/blob/main/api-reference-examples/python/pytx/docs/quickstart.rst

Illustrates how to create and submit batch requests to the ThreatExchange graph. This allows multiple operations to be combined into a single API call for efficiency. It shows how to reference results from one request within another.

```python
import json
from pytx import ThreatDescriptor, Batch
from pytx.errors import pytxFetchError

a = ThreatDescriptor.objects(text='foo',
                                request_dict=True)
b = {'type': 'GET',
     'url': '{result=foo:$.data.0.id}'}
try:
    result = Batch.submit(foo=a,
                          bar=b)
    print json.dumps(result, indent=4, sort_keys=True)
except pytxFetchError, e:
    print e.message
```

--------------------------------

### Share Data from CSV to ThreatExchange

Source: https://github.com/facebook/threatexchange/blob/main/api-reference-examples/python-notebook/Getting Started with Sharing.ipynb

This Python script reads data from a CSV file and publishes it to ThreatExchange. It iterates through rows, updates default fields with row data, and creates new ThreatDescriptors. Error handling is included for upload failures.

```python
import csv
import pytx.errors
from pytx import ThreatDescriptor

# The file to upload
file = 'test_share.csv'

# Load the CSV and serially publish it
ind_count = 0
fail_count = 0
with open(file, 'rb') as csvfile:
    reader = csv.DictReader(csvfile, delimiter=',', quotechar='"')
    for row in reader:
        try:
            fields = default_fields.copy()
            fields.update(row)
            result = ThreatDescriptor.new(params=fields)
        except Exception, e:
            print 'Unable to upload' + row['indicator'] + 'due to ' + result['message'] + "\n"
            fail_count = fail_count + 1
        else:
            ind_count = ind_count + 1
print "Done publishing %d indicators with %d failures!" % (ind_count, fail_count)
```

--------------------------------

### PDQ Benchmark Help Command

Source: https://github.com/facebook/threatexchange/blob/main/pdq/cpp/index/README.md

Displays the available command-line options for the benchmark-query executable. This helps in understanding how to customize the benchmark runs.

```bash
$ ./benchmark-query -h
Usage: ./benchmark-query [options]
Options:
  -v               Verbose output
  --seed N         Random seed (default: 41)
  -q N             Number of queries to run (default: 1000)
  -b N             Number of PDQ hashes to query against (default: 10000)
  -d N             Maximum Hamming distance for matches (default: 31)
  -m               Method for querying (default: linear), Available: linear, mih
```

--------------------------------

### Define Default Fields for ThreatExchange Sharing

Source: https://github.com/facebook/threatexchange/blob/main/api-reference-examples/python-notebook/Getting Started with Sharing.ipynb

This Python code defines default fields for ThreatExchange descriptors, which are used when sharing data. It utilizes vocabulary constants from the pytx library to set attributes like threat type, severity, and privacy.

```python
from pytx.vocabulary import Attack as a
from pytx.vocabulary import ReviewStatus as rs
from pytx.vocabulary import Severity as s
from pytx.vocabulary import ShareLevel as sl
from pytx.vocabulary import Status as st
from pytx.vocabulary import ThreatDescriptor as td
from pytx.vocabulary import ThreatType as tt
from pytx.vocabulary import Types as t

# See: https://pytx.readthedocs.org/en/latest/pytx.vocabulary.html#pytx.vocabulary.ThreatDescriptor
default_fields = {
    #td.ATTACK_TYPE: a.MALWARE, # TODO uncomment when PR #120 gets added to Pytx in pip
    td.CONFIDENCE: 75,
    #td.EXPIRED_ON: '2016-02-25 00:00:00+0000',
    td.PRIVACY_TYPE: privacy_type,
    td.REVIEW_STATUS: rs.REVIEWED_MANUALLY,
    td.SHARE_LEVEL: sl.AMBER,
    td.SEVERITY: s.SEVERE,
    td.STATUS: st.MALICIOUS,
    td.THREAT_TYPE: tt.MALICIOUS_IP,
    td.TYPE: t.IP_ADDRESS,
    td.DESCRIPTION: '[example][tags] Test description'
}

# Add in privacy members, as needed
if privacy_members is not None:
    default_fields[td.PRIVACY_MEMBERS] = ','.join(privacy_members)
```

--------------------------------

### Get Threat Tag IDs

Source: https://github.com/facebook/threatexchange/blob/main/api-reference-examples/curl/te-tag-query/README.md

Retrieves threat tag IDs based on a text query. This is the first step in fetching hashes associated with a specific tag.

```APIDOC
## GET /threat_tags/

### Description
Fetches threat tag IDs that match a given text query. Useful for finding the internal ID for a tag you want to use for searching.

### Method
GET

### Endpoint
`https://graph.facebook.com/v4.0/threat_tags/`

### Query Parameters
- **access_token** (string) - Required - Your Facebook App's access token.
- **text** (string) - Required - The text to search for among threat tags.
- **limit** (integer) - Optional - The maximum number of results to return.

### Request Example
```bash
curl -s 'https://graph.facebook.com/v4.0/threat_tags/'\\
"?access_token=$TX_ACCESS_TOKEN"\\
'&text=pwny&limit=10'
```

### Response
#### Success Response (200)
- **data** (array) - An array of threat tag objects, each containing an `id` and `text`.
- **paging** (object) - Pagination information.

#### Response Example
```json
{
   "data": [
      {
         "id": "2722789434440042",
         "text": "pwny2"
      },
      {
         "id": "1283621968426798",
         "text": "pwny"
      }
   ],
   "paging": {
      "cursors": {
         "before": "MAZDZD",
         "after": "MQZDZD"
      }
   }
}
```
```

--------------------------------

### Create ThreatDescriptor using .save()

Source: https://github.com/facebook/threatexchange/blob/main/api-reference-examples/python/pytx/docs/quickstart.rst

Demonstrates creating a new ThreatDescriptor object and saving it to ThreatExchange using the `.save()` method. This involves instantiating the class, setting indicator and privacy type, and then calling save.

```python
from pytx import ThreatDescriptor
from pytx.vocabulary import PrivacyType as pt

td = ThreatDescriptor()
td.indicator = "foo"
td.privacy_type = pt.VISIBLE
td.save()
```

--------------------------------

### Get Descriptor Details by Object IDs

Source: https://github.com/facebook/threatexchange/blob/main/api-reference-examples/curl/te-tag-query/README.md

Retrieves detailed information for a list of tagged object IDs. This step requires specifying the desired fields for each object.

```shell
curl -s 'https://graph.facebook.com/v4.0/'\
"?access_token=$TX_ACCESS_TOKEN"\
'&ids=%5B2556006484495859,2356527701137016%5D'\
'&fields=raw_indicator,type,added_on,last_updated,confidence,owner,privacy_type,review_status,status,severity,share_level,tags,description'
```

--------------------------------

### Configure pytx Connection Settings

Source: https://github.com/facebook/threatexchange/blob/main/api-reference-examples/python/pytx/docs/quickstart.rst

Allows customization of connection parameters for pytx, including custom headers, proxies, and SSL verification settings. These settings are passed to the underlying requests library for fine-grained control.

```python
from pytx import connection()
connection(headers=<your stuff here>,
              proxies=<your stuff here>,
              verify=<your stuff here>)
```

--------------------------------

### Get Tagged Object IDs by Tag ID

Source: https://github.com/facebook/threatexchange/blob/main/api-reference-examples/curl/te-tag-query/README.md

Fetches a list of object IDs associated with a specific tag ID. This query can be paginated if more results are available.

```shell
curl -s 'https://graph.facebook.com/v4.0/1283621968426798/tagged_objects/'\
"?access_token=$TX_ACCESS_TOKEN"\
'&limit=100'
```

--------------------------------

### Build and Run PDQ Benchmark (CPP)

Source: https://github.com/facebook/threatexchange/blob/main/pdq/cpp/index/README.md

Instructions to build the benchmark executable using make and then run it. This is the primary command for executing the performance tests.

```bash
cd pdq/cpp
make bin/benchmark-query
./benchmark-query
```

--------------------------------

### Get Tag ID by Text

Source: https://github.com/facebook/threatexchange/blob/main/api-reference-examples/curl/te-tag-query/README.md

Retrieves the ThreatExchange tag ID for a given text string using a prefix match. This is the first step in querying for tagged objects.

```shell
curl -s 'https://graph.facebook.com/v4.0/threat_tags/'\
"?access_token=$TX_ACCESS_TOKEN"\
'&text=pwny&limit=10'
```

--------------------------------

### Initialize ThreatExchange Client (Go)

Source: https://github.com/facebook/threatexchange/blob/main/api-reference-examples/go/README.md

Demonstrates how to create a new ThreatExchange client instance. Requires an appID, appSecret, and a logger.

```go
client, err = threatexchange.New(
	appID,
	appSecert,
	log.New(logrus.StandardLogger().Writer(), "ThreatExchange: ", log.Lshortfile),
)
```

--------------------------------

### Build and run PDQ hashing benchmark

Source: https://github.com/facebook/threatexchange/blob/main/pdq/cpp/hashing/README.md

This snippet shows how to navigate to the C++ benchmark directory, compile the photo hasher benchmark, and then execute it. It's the primary way to initiate the performance testing.

```bash
cd pdq/cpp
make bin/benchmark-photo-hasher
./benchmark-photo-hasher
```

--------------------------------

### Retrieve Full ThreatDescriptor Details

Source: https://github.com/facebook/threatexchange/blob/main/api-reference-examples/python/pytx/docs/quickstart.rst

Explains how to fetch the complete details for a ThreatDescriptor object after obtaining a summary. This requires an additional API call using the `.details()` method, followed by converting the object to a dictionary.

```python
from pytx import ThreatDescriptor

results = ThreatDescriptor.objects(text='www.facebook.com')
for result in results:
    # Make another API call to get all of the fields
    result.details()
    print result.to_dict()
```

--------------------------------

### Python Logs: Sample API Fetching Simulation

Source: https://github.com/facebook/threatexchange/blob/main/hasher-matcher-actioner/docs/user-interface.md

Demonstrates the logging output when the 'Setup Sample API' is used, showing the process of fetching simulated signals for the SEED_SAMPLE exchange. This helps in understanding the data fetching mechanism in a development environment.

```python
INFO in fetcher: SEED_SAMPLE[sample] Fetching signals for SEED_SAMPLE from sample
INFO in fetcher: SEED_SAMPLE[sample] No checkpoint, should be the first fetch.
DEBUG in fetcher: SEED_SAMPLE[sample] fetch_iter() with 18 new records
INFO in fetcher: SEED_SAMPLE[sample] Fetched all data! Up to date!
```

--------------------------------

### PDQ hashing benchmark help command

Source: https://github.com/facebook/threatexchange/blob/main/pdq/cpp/hashing/README.md

Displays the usage and available options for the `benchmark-photo-hasher` executable. This includes flags for verbose output, specifying the number of hashes, and enabling dihedral hash computation.

```bash
$ ./benchmark-query -h
Usage: ./benchmark-photo-hasher [options] folder_path
Options:
  -v               Verbose output
  -n N             Total number of hashes to generate, can be more or less than the number of images in the folder
                           (default: 0, meaning generate one hash for each image in the folder)
  --dihedral       Compute dihedral versions of the hashes (default: false)
```

--------------------------------

### Build .NET Test Project

Source: https://github.com/facebook/threatexchange/blob/main/pdq/wasm/README.md

Compiles the .NET project that contains the Selenium WebDriver tests. This command creates executables in the bin/Debug/net6.0 directory.

```bash
dotnet build
```

--------------------------------

### FFmpeg Installation Check

Source: https://github.com/facebook/threatexchange/blob/main/vpdq/README.md

This command verifies if FFmpeg is installed on the system by displaying its version information. If FFmpeg is not installed or not in the system's PATH, it will output a 'command not found' error.

```sh
$ ffmpeg
       ffmpeg version 4.4.2 Copyright (c) 2000-2023 the FFmpeg developers
       ...
```

--------------------------------

### Manually Run Flask Startup Script (Bash)

Source: https://github.com/facebook/threatexchange/blob/main/hasher-matcher-actioner/CONTRIBUTING.md

Manually executes the startup script for the Flask server, which is useful if automatic startup fails due to syntax errors. This script is also run automatically when the Dev Container starts.

```bash
cd /workspace
.devcontainer/startup.sh
```

--------------------------------

### Install vpdq Locally

Source: https://github.com/facebook/threatexchange/blob/main/vpdq/python/README.md

Installs the vpdq Python package locally from the root vpdq folder. This is useful for development or testing modifications.

```sh
python -m pip install -e .
```

--------------------------------

### Get Threat Exchange Members

Source: https://github.com/facebook/threatexchange/blob/main/api-reference-examples/js/node/README.md

Retrieves a list of threat exchange members. This function makes a GET request to the /threat_exchange_members endpoint.

```javascript
api.getThreatExchangeMembers(function(err,data) {
    // process data from response here
});
```

--------------------------------

### Building PHP 7 with GD Support

Source: https://github.com/facebook/threatexchange/blob/main/pdq/php/README.md

Commands and configuration steps to compile PHP 7 from source, ensuring the GD library is enabled and correctly linked with JPEG and PNG support.

```bash
$ tar zvxf php-7.1.11.tar.gz
$ cd php-7.1.11
$ ./configure \
 --prefix=/usr/local/php7 \
 --with-gd \
 --with-jpeg-dir=/usr/local/opt/jpeg \
 --with-png-dir=/usr/local/opt/libpng \
 --with-zlib-dir=/usr

$ make
$ make install
```

--------------------------------

### Install vpdq Python Binding

Source: https://github.com/facebook/threatexchange/blob/main/vpdq/README.md

Installs the vpdq Python binding using pip. Requires all C++ implementation dependencies to be met.

```sh
pip install vpdq
```

--------------------------------

### Generate documentation locally

Source: https://github.com/facebook/threatexchange/blob/main/api-reference-examples/python/pytx/CONTRIBUTING.md

Builds the project documentation locally using Sphinx and reStructuredText (rst) format. The generated documentation will be in the 'build/docs' directory.

```bash
make docs
```

--------------------------------

### PHP PDQ Hashing Usage Example

Source: https://github.com/facebook/threatexchange/blob/main/pdq/php/README.md

Demonstrates how to execute the PDQ hashing tool from the command line using a PHP script, showing the expected output for both pure-PHP and Zend extension hashing.

```bash
$ cat php.ini
memory_limit = -1;
include_path = ".";
extension=ext/pdq/modules/pdq.so;

$ php -c . pdqhashertest.php b.jpg
d8f8f0cec0f4a84f0637022a278f67f0b36e2ed596621e1d33e6339c4e9c9b22,100,purephp,b.jpg
d8f8f0cec0f4a84f0637022a278f67f0b36e2ed596621e1d33e6339c4e9c9b22,100,extnphp,b.jpg
```

--------------------------------

### Get Threat Indicator Object by ID

Source: https://github.com/facebook/threatexchange/blob/main/api-reference-examples/js/node/README.md

Fetches details for a specific threat indicator using its ID. Allows for optional fields to customize the GET request.

```javascript
var fields = {}; // optional fields here as shown in Facebook documentation
var id = 12345; // unique ID to threat indicator object
api.getThreatIndicatorObject(id,fields,function(err,data) {
    // process data here
});
```

--------------------------------

### Get Malware Object by ID

Source: https://github.com/facebook/threatexchange/blob/main/api-reference-examples/js/node/README.md

Retrieves detailed information about a specific malware object using its unique ID. Supports optional fields for the GET request.

```javascript
var fields = {}; // optional fields here as shown in Facebook documentation
var id = 12345; // unique ID to malware object
api.getMalwareObject(id,fields,function(err,data) {
    // process data here
});
```

--------------------------------

### Get Threat Indicators

Source: https://github.com/facebook/threatexchange/blob/main/api-reference-examples/js/node/README.md

Retrieves threat indicators. This function supports optional parameters for filtering or specifying data, making a GET request to the /threat_indicators endpoint.

```javascript
var options = {}; // optional values here as shown in Facebook documentation
api.getThreatIndicators(options,function(err,data) {
    // process data from response here
});
```

--------------------------------

### Get Malware Analyses

Source: https://github.com/facebook/threatexchange/blob/main/api-reference-examples/js/node/README.md

Fetches malware analyses from the Threat Exchange. Allows for optional parameters to filter or specify the data to retrieve via a GET request.

```javascript
var options = {}; // optional values as shown in Facebook documentation
api.getMalwareAnalyses(options,function(err,data) {
    // process data from response here
});
```

--------------------------------

### Build and Verify with Maven

Source: https://github.com/facebook/threatexchange/blob/main/pdq/java/README.md

Compiles, tests, and installs the PDQ hashing library using Maven. This process creates a JAR file in the target directory. It's an alternative to using the Makefile for building.

```shell
mvn clean test install
cp target/pdq-0.0.1-SNAPSHOT.jar pdq.jar
```

--------------------------------

### Upgrade Database Schema using Flask CLI

Source: https://github.com/facebook/threatexchange/blob/main/hasher-matcher-actioner/CONTRIBUTING.md

Applies generated migration files to the local database. Run this command from within the 'src/OpenMediaMatch' folder.

```bash
flask db upgrade
```

--------------------------------

### Install PDQ Hashing Dependencies

Source: https://github.com/facebook/threatexchange/blob/main/pdq/python/README.md

Installs the necessary Python libraries, PIL (Python Image Library) and Pillow, required for the PDQ hashing implementation.

```shell
sudo pip3 install Image
sudo pip3 install pillow
```

--------------------------------

### Threatexchange CLI for Extension Management

Source: https://github.com/facebook/threatexchange/blob/main/python-threatexchange/threatexchange/extensions/README.md

Shows how to use the `threatexchange` CLI to manage project extensions. This includes adding new extensions, listing installed extensions, removing extensions, and performing a factory reset of the configuration.

```bash
# Provide the path to the module with TX_MANIFEST
$ threatexchange config extensions add threatexchangecontributions.myproject
Added my.module.name:
  Signals:
    foo - FooSignal
    bar - BarSignal
  Contents:
    baz - BazContent
  APIs:
    zed - ZedAPI

# What do I have?
$ threatexchange config extensions list
threatexchangecontributions.myproject

# I don't want this anymore!
$ threatexchange config extensions remove threatexchangecontributions.myproject

# Oh no I broke everything and am filled with sadness and regret!
$ rm ~/.threatexchange  # factory reset
```

--------------------------------

### Compile Parallel TMK Code (C++)

Source: https://github.com/facebook/threatexchange/blob/main/tmk/README.md

Compiles parallelized variants of TMK scoring binaries. Requires OpenMP runtime library and FAISS. On macOS, install with 'brew install libomp faiss'.

```bash
make parallel
```

--------------------------------

### Run PDQ hashing benchmark with different hash counts

Source: https://github.com/facebook/threatexchange/blob/main/pdq/cpp/hashing/README.md

These examples demonstrate running the `benchmark-photo-hasher` with varying numbers of hashes to generate, showcasing the impact on performance metrics like hashing speed and read speed.

```bash
$ ./benchmark-photo-hasher ../data/reg-test-input/dih/
PHOTO COUNT:               8
ERROR COUNT:               0
TIME SPENT HASHING PHOTOS (SECONDS):     0.015171
PHOTOS HASHED PER SECOND:   527.324158
TIME SPENT READING PHOTOS (SECONDS):        0.299410
PHOTOS READ PER SECOND:     26.719229
```

```bash
$ ./benchmark-photo-hasher -n 10 ../data/reg-test-input/dih/
PHOTO COUNT:               10
ERROR COUNT:               0
TIME SPENT HASHING PHOTOS (SECONDS):     0.018181
PHOTOS HASHED PER SECOND:   550.028442
TIME SPENT READING PHOTOS (SECONDS):        0.334852
PHOTOS READ PER SECOND:     29.863914
```

```bash
$ ./benchmark-photo-hasher -n 100 ../data/reg-test-input/dih/
PHOTO COUNT:               100
ERROR COUNT:               0
TIME SPENT HASHING PHOTOS (SECONDS):     0.182545
PHOTOS HASHED PER SECOND:   547.810364
TIME SPENT READING PHOTOS (SECONDS):        2.841218
PHOTOS READ PER SECOND:     35.196167
```

```bash
$ ./benchmark-photo-hasher -n 1000 ../data/reg-test-input/dih/
PHOTO COUNT:               1000
ERROR COUNT:               0
TIME SPENT HASHING PHOTOS (SECONDS):     1.767847
PHOTOS HASHED PER SECOND:   565.659729
TIME SPENT READING PHOTOS (SECONDS):        27.135609
PHOTOS READ PER SECOND:     36.851948
```