### Run Octopii from Command Line

Source: https://context7.com/redhuntlabs/octopii/llms.txt

Octopii can be executed from the command line to scan local directories, individual files, S3 buckets, or Apache open directory listings. Results are typically saved to `output.json`, and webhook notifications can be configured for real-time alerts. Installation requires Python dependencies, Tesseract OCR, and spaCy language models.

```bash
# Install dependencies
pip install -r requirements.txt
sudo apt install tesseract-ocr -y
python -m spacy download en_core_web_sm

# Scan a local directory
python3 octopii.py dummy-pii/

# Scan a single file
python3 octopii.py dummy-pii/dummy-PAN-India.jpg

# Scan an S3 bucket
python3 octopii.py https://s3.amazonaws.com/bucket-name/

# Scan Apache open directory
python3 octopii.py https://example.com/open-directory/

# Scan with webhook notifications
python3 octopii.py dummy-pii/ --notify https://hooks.slack.com/services/XXX/YYY/ZZZ

# Example output:
# {
#     "file_path": "dummy-pii/dummy-PAN-India.jpg",
#     "pii_class": "Permanent Account Number",
#     "score": 8,
#     "country_of_origin": "India",
#     "faces": 0,
#     "identifiers": ["ABCDE1234F"],
#     "emails": [],
#     "phone_numbers": [],
#     "addresses": ["INDIA"]
# }
```

--------------------------------

### Load and Access PII Definitions using Python

Source: https://context7.com/redhuntlabs/octopii/llms.txt

Loads all PII definitions from the `definitions.json` file using the `text_utils.get_regexes()` function. It demonstrates how to access specific document definitions, such as the 'Aadhaar Card', to retrieve its regex pattern, keywords, and region. This allows for programmatic access and utilization of the defined PII rules.

```python
# Supported document types include:
# - Email, Phone Number
# - Banking, Payment Card
# - Aadhaar Card, PAN (India)
# - Social Security Number (USA)
# - Driver's Licenses (USA, India, Russia)
# - Passports (UK, USA, India, Ukraine, Russia, Singapore, etc.)
# - ID Cards (Hong Kong, China, South Africa, Ireland)
# - Saudi documents (Iqama, Driver's License, Visa, Health Insurance)
# - Professional Resume
# - UUID4

import text_utils

# Load all definitions
rules = text_utils.get_regexes()

# Access specific document definition
aadhaar_regex = rules["Aadhaar Card"]["regex"]
aadhaar_keywords = rules["Aadhaar Card"]["keywords"]
aadhaar_region = rules["Aadhaar Card"]["region"]

print(f"Regex: {aadhaar_regex}")
print(f"Keywords: {aadhaar_keywords}")
print(f"Region: {aadhaar_region}")
```

--------------------------------

### Scan File for PII using Python

Source: https://context7.com/redhuntlabs/octopii/llms.txt

The `search_pii` function is the main entry point for scanning a single file. It handles file type detection, face detection, OCR text extraction, and returns a dictionary containing all detected PII elements, including file path, PII class, score, country of origin, and extracted identifiers.

```python
import octopii
import text_utils

# Load the regex rules from definitions.json
rules = text_utils.get_regexes()

# Scan a single file for PII
result = octopii.search_pii("dummy-pii/dummy-passport-britain.jpg")

# Result structure:
# {
#     "file_path": "dummy-pii/dummy-passport-britain.jpg",
#     "pii_class": "British Passport",
#     "score": 12,
#     "country_of_origin": "United Kingdom",
#     "faces": 1,
#     "identifiers": ["P<GBRSPECIMEN<<ANGELA<ZOE<<<<<<<<<<<<<<<<<<"],
#     "emails": [],
#     "phone_numbers": [],
#     "addresses": ["United Kingdom", "London"]
# }
```

--------------------------------

### List S3 Bucket Files with Python

Source: https://context7.com/redhuntlabs/octopii/llms.txt

Parses S3 bucket XML listings to extract and return all file URLs. This utility function is available in the 'file_utils' library.

```python
import file_utils

# List files in public S3 bucket
s3_url = "https://s3.amazonaws.com/public-bucket/"
files = file_utils.list_s3_files(s3_url)

for f in files:
    print(f)
# Output: Full URLs to each file in the bucket
```

--------------------------------

### Configure PII Definitions using JSON

Source: https://context7.com/redhuntlabs/octopii/llms.txt

Defines regular expressions, regions, and keywords for identifying various PII document types. This JSON structure allows for the addition of custom document types to enhance the tool's detection capabilities. Each custom type requires a regex pattern, a region, and a list of relevant keywords.

```json
{
  "Custom Document Type": {
    "regex": "[A-Z]{2}[0-9]{8}",
    "region": "Country Name",
    "keywords": [
      "keyword1",
      "keyword2",
      "document",
      "specific",
      "terms"
    ]
  }
}
```

--------------------------------

### Classify Documents by Keywords with Python

Source: https://context7.com/redhuntlabs/octopii/llms.txt

Classifies document types by matching extracted words against keyword lists using Gestalt pattern matching. It requires 'text_utils' and regex rules, and returns classification scores.

```python
import text_utils

rules = text_utils.get_regexes()

# Tokenized words extracted from an Aadhaar card image
words = ['Aadhaar', 'UID', 'Government', 'India', 'Unique', 'Identification', 'Authority']

scores = text_utils.keywords_classify_pii(rules, words)

# Get highest scoring classification
max_score = max(scores.values())
pii_class = [k for k, v in scores.items() if v == max_score][0]

print(f"Detected: {pii_class} (score: {max_score})")
# Output: Detected: Aadhaar Card (score: 7)

# Threshold: score >= 5 is considered a valid match
if max_score >= 5:
    print(f"Confidence: High - document classified as {pii_class}")
```

--------------------------------

### Detect Image and PDF Files with Python

Source: https://context7.com/redhuntlabs/octopii/llms.txt

Checks if a given file path corresponds to an image or a PDF file. These helper functions are provided by the 'file_utils' library.

```python
import file_utils

# Check file types
print(file_utils.is_image("document.jpg"))   # True
print(file_utils.is_image("document.pdf"))   # False
print(file_utils.is_pdf("document.pdf"))     # True
print(file_utils.is_pdf("document.jpg"))     # False
```

--------------------------------

### List Directory Files from URL with Python

Source: https://context7.com/redhuntlabs/octopii/llms.txt

Extracts file URLs from HTML pages that use Apache-style open directory listings. This function is part of the 'file_utils' library.

```python
import file_utils

# List files from Apache directory listing
url = "https://example.com/public-files/"
files = file_utils.list_directory_files(url)

for f in files:
    print(f)
# Output: Full URLs to each file in the directory
```

--------------------------------

### List Local Files Recursively with Python

Source: https://context7.com/redhuntlabs/octopii/llms.txt

Recursively lists all files within a specified local directory path. This function is part of the 'file_utils' library.

```python
import file_utils

# List all files in a directory
files = file_utils.list_local_files("dummy-pii/")
for f in files:
    print(f)

# Output:
dummy-pii/dummy-PAN-India.jpg
dummy-pii/dummy-aadhaar.png
dummy-pii/dummy-debit-card.jpg
dummy-pii/dummy-drivers-license-maharashtra.jpg
# ...
```

--------------------------------

### Send Scan Results to Webhook using Python

Source: https://context7.com/redhuntlabs/octopii/llms.txt

Sends scan results to webhook endpoints like Slack, Discord, or custom URLs. The `push_data` function automatically formats the payload based on the endpoint. It takes the JSON-stringified result and the webhook URL as input. Success is indicated by a confirmation message, while failure provides an error reason.

```python
import webhook
import json

result = {
    "file_path": "sensitive-doc.jpg",
    "pii_class": "Social Security Number",
    "score": 6,
    "country_of_origin": "United States",
    "identifiers": ["123-45-6789"]
}

# Send to Slack
webhook.push_data(json.dumps(result), "https://hooks.slack.com/services/XXX/YYY/ZZZ")

# Send to Discord (auto-detects and uses {"content": ...} format)
webhook.push_data(json.dumps(result), "https://discord.com/api/webhooks/123/abc")

# Send to custom endpoint (uses {"text": ...} format)
webhook.push_data(json.dumps(result), "https://api.example.com/webhook")

# Output on success: "Scan results sent to webhook."
# Output on failure: "Couldn't send scan results to webhook. Reason: <error>"
```

--------------------------------

### Extract Emails with Python

Source: https://context7.com/redhuntlabs/octopii/llms.txt

Extracts email addresses from a given text using regular expressions. It requires the 'text_utils' library and a set of predefined regex rules loaded via `get_regexes()`.

```python
import text_utils

rules = text_utils.get_regexes()
text = "Contact John at john.doe@example.com or support@company.org for assistance."

emails = text_utils.email_pii(text, rules)
print(emails)  # Output: ['john.doe@example.com', 'support@company.org']
```

--------------------------------

### Append Scan Results to JSON File with Python

Source: https://context7.com/redhuntlabs/octopii/llms.txt

Appends scan results, formatted as a dictionary, to a specified JSON output file. The file is created if it does not already exist. This function is part of the 'file_utils' library.

```python
import file_utils

result = {
    "file_path": "test.jpg",
    "pii_class": "Aadhaar Card",
    "score": 8,
    "country_of_origin": "India",
    "faces": 1,
    "identifiers": ["1234 5678 9012"],
    "emails": [],
    "phone_numbers": [],
    "addresses": ["Mumbai"]
}

file_utils.append_to_output_file(result, "output.json")
# Creates/appends to output.json with formatted JSON
```

--------------------------------

### Extract Text from Image using Python (OCR)

Source: https://context7.com/redhuntlabs/octopii/llms.txt

The `scan_image_for_text` function performs Optical Character Recognition (OCR) on an image using Tesseract. It applies several preprocessing stages, including auto-rotation, grayscaling, thresholding, and deskewing, to enhance OCR accuracy. The function returns both the raw OCR output and a list of cleaned, tokenized words suitable for further analysis.

```python
import cv2
import image_utils

# Load and process image
image = cv2.imread("dummy-pii/dummy-aadhaar.png")

# Extract text with multiple preprocessing passes:
# 1. Original image
# 2. Auto-rotated
# 3. Grayscaled
# 4. Monochrome (Otsu threshold)
# 5. Mean threshold
# 6. Gaussian threshold
# 7-9. Three deskew passes
original_text, tokenized_words = image_utils.scan_image_for_text(image)

# original_text: Complete raw OCR output from all passes
# tokenized_words: List of cleaned words (>= 2 chars) for classification
print(f"Extracted {len(tokenized_words)} words")
print(f"Sample words: {tokenized_words[:10]}")
```

--------------------------------

### Extract ID Numbers with Python

Source: https://context7.com/redhuntlabs/octopii/llms.txt

Extracts various government ID numbers such as Aadhaar, PAN, SSN, passports, and driver's licenses using region-specific regex patterns. It depends on 'text_utils' and regex rules from `get_regexes()`.

```python
import text_utils

rules = text_utils.get_regexes()
text = """
Aadhaar: 1234 5678 9012
PAN: ABCDE1234F
SSN: 123-45-6789
Passport MRZ: P<GBRSPECIMEN<<ANGELA<ZOE<<<<<<<<<<<<<<<<<<
"""

identifiers = text_utils.id_card_numbers_pii(text, rules)
for item in identifiers:
    print(f"{item['identifier_class']}: {item['result']}")

# Output:
Aadhaar Card: ['1234 5678 9012']
Permanent Account Number: ['ABCDE1234F']
Social Security Number: ['123-45-678']
British Passport: ['P<GBRSPECIMEN<<ANGELA<ZOE<<<<<<<<<<<<<<<<<<']
```

--------------------------------

### Detect Faces in Image using Python

Source: https://context7.com/redhuntlabs/octopii/llms.txt

The `scan_image_for_people` function utilizes OpenCV's Haar cascade classifier to detect human faces within an image. It accepts images loaded by OpenCV or PIL and returns the total count of detected faces. This is particularly useful for identifying documents that contain photographs.

```python
import cv2
import image_utils
import numpy

# Load image with OpenCV
image = cv2.imread("dummy-pii/dummy-passport-britain.jpg")

# Detect faces in the image
face_count = image_utils.scan_image_for_people(image)
print(f"Detected {face_count} face(s)")  # Output: Detected 1 face(s)

# Works with PIL images too (converts internally)
from PIL import Image
pil_image = Image.open("dummy-pii/dummy-drivers-license-maharashtra.jpg")
face_count = image_utils.scan_image_for_people(numpy.array(pil_image))
```

--------------------------------

### Extract Locations with Python (NLTK)

Source: https://context7.com/redhuntlabs/octopii/llms.txt

Extracts geographic locations from text using NLTK's Named Entity Recognition. It identifies GPE, GSP, LOCATION, and FACILITY tags. This function is part of the 'text_utils' library.

```python
import text_utils

text = """
The document was issued in Mumbai, India.
The holder resides in California, United States.
Office located at 123 Main Street, New York.
"""

locations = text_utils.regional_pii(text)
print(locations)  # Output: ['Mumbai', 'India', 'California', 'United States', 'New York']
```

--------------------------------

### Extract Phone Numbers with Python

Source: https://context7.com/redhuntlabs/octopii/llms.txt

Extracts phone numbers from text, supporting various formats including international numbers with country codes. It utilizes the 'text_utils' library and regex rules obtained from `get_regexes()`.

```python
import text_utils

rules = text_utils.get_regexes()
text = """
Contact us at +1 555-123-4567 or (800) 555-1234.
International: +91 98765 43210
Local: 4000002170
"""

phone_numbers = text_utils.phone_pii(text, rules)
print(phone_numbers)
# Output: ['+1 555-123-4567', '800', '555', '1234', '+91 98765 43210', '4000002170']
```

--------------------------------

### Octopii PII Definition Structure

Source: https://github.com/redhuntlabs/octopii/blob/master/CONTRIBUTING.md

This JSON structure defines a Personally Identifiable Information (PII) class for the Octopii project. It includes a regular expression for pattern matching, the country of origin, and a list of keywords to aid in OCR detection. This structure is crucial for defining new PII types or updating existing ones.

```json
"<pii class name>" : {
      "regex" : "<a regular expression or null>",
      "region" : <the country this PII originates from or null>,
      "keywords" : [
         "<keyword 1>",
         "<keyword 2>",
         ...
      ]
   },
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.