ContentShield

https://github.com/zachhandley/contentshield
Admin
Ultra-fast content moderation with SymSpell fuzzy matching. Multi-language profanity detection with...
Tokens:9,674
Snippets:154
Trust Score:8.8
License:MIT
Update:3 months ago
Show doc for...
Tokens:
Raw
### Set up and run ContentShield Hono API

Source: https://github.com/zachhandley/contentshield/blob/main/README.md

Navigate to the Hono API example directory, install dependencies, and run the development server. This API can be called from fetch, curl, or other services.

```bash
cd examples/hono-api
pnpm install
pnpm run dev   # Node — POST http://localhost:3000/detect
# or `pnpm run deploy:cf` for Cloudflare Workers
```

--------------------------------

### Install Dependencies

Source: https://github.com/zachhandley/contentshield/blob/main/README.md

Use npm to install project dependencies before building or testing.

```bash
npm install
```

--------------------------------

### Install and Run (Bun)

Source: https://github.com/zachhandley/contentshield/blob/main/examples/hono-api/README.md

Installs dependencies and runs the Hono API using Bun.

```bash
bun install
bun run src/server-bun.ts
```

--------------------------------

### Install Dependencies

Source: https://github.com/zachhandley/contentshield/blob/main/CONTRIBUTING.md

Install project dependencies using npm or pnpm. pnpm is recommended for better performance and disk space efficiency.

```bash
# Install dependencies
npm install
```

```bash
# Or with pnpm (recommended)
pnpm install
```

--------------------------------

### Install Dependencies (Node/Bun)

Source: https://github.com/zachhandley/contentshield/blob/main/examples/hono-api/README.md

Installs project dependencies using pnpm for Node.js environments.

```bash
pnpm install
```

--------------------------------

### Run Development Server (Node)

Source: https://github.com/zachhandley/contentshield/blob/main/examples/hono-api/README.md

Starts the Hono API development server using pnpm for Node.js.

```bash
pnpm run dev
# POST http://localhost:3000/detect
```

--------------------------------

### Install ContentShield with npm

Source: https://github.com/zachhandley/contentshield/blob/main/README.md

Use this command to add ContentShield to your project via npm.

```bash
npm install content-shield
```

--------------------------------

### Install ContentShield with pnpm

Source: https://github.com/zachhandley/contentshield/blob/main/README.md

Use this command to add ContentShield to your project via pnpm.

```bash
pnpm add content-shield
```

--------------------------------

### Language Metadata Example

Source: https://github.com/zachhandley/contentshield/blob/main/CONTRIBUTING.md

Example JSON structure for language metadata. Includes name, code, version, word count, update date, and contributors.

```json
// metadata.json
{
  "name": "English",
  "code": "en",
  "version": "1.0.0",
  "wordCount": 1500,
  "lastUpdated": "2023-01-01",
  "contributors": ["contributor1", "contributor2"]
}
```

--------------------------------

### Install ContentShield with yarn

Source: https://github.com/zachhandley/contentshield/blob/main/README.md

Use this command to add ContentShield to your project via yarn.

```bash
yarn add content-shield
```

--------------------------------

### Conventional Commit Message Example (Feature)

Source: https://github.com/zachhandley/contentshield/blob/main/CONTRIBUTING.md

Example of a conventional commit message for a new feature. Follows the '<type>(<scope>): <description>' format with an optional body and footer.

```git commit
feat(detector): add batch processing support

Add batchAnalyze method to process multiple texts efficiently
with controlled concurrency and memory optimization.

Closes #123
```

--------------------------------

### GET /

Source: https://github.com/zachhandley/contentshield/blob/main/examples/hono-api/README.md

Retrieves metadata about the ContentShield API.

```APIDOC
## GET /

### Description
Retrieves metadata about the ContentShield API.

### Method
GET

### Endpoint
/

### Response
#### Success Response (200)
- **metadata** (object) - API metadata
```

--------------------------------

### Profanity Data Example

Source: https://github.com/zachhandley/contentshield/blob/main/CONTRIBUTING.md

Example JSON structure for profanity word lists. Each entry includes the word, its severity, associated categories, and variations.

```json
// profanity.json
{
  "words": [
    {
      "word": "example",
      "severity": "medium",
      "categories": ["profanity"],
      "variations": ["ex4mple", "exampl3"]
    }
  ]
}
```

--------------------------------

### Conventional Commit Message Example (Fix)

Source: https://github.com/zachhandley/contentshield/blob/main/CONTRIBUTING.md

Example of a conventional commit message for a bug fix. Demonstrates the structure for reporting and resolving issues.

```git commit
fix(trie): resolve memory leak in cache eviction

Fix issue where cache entries were not properly cleaned up
during eviction, leading to memory growth over time.

Fixes #456
```

--------------------------------

### Deploy to Cloudflare Workers

Source: https://github.com/zachhandley/contentshield/blob/main/examples/hono-api/README.md

Installs dependencies and deploys the Hono API to Cloudflare Workers using pnpm and wrangler.

```bash
pnpm install
# one-time: `npx wrangler login`
pnpm run deploy:cf
```

--------------------------------

### JSDoc Comment for Public API

Source: https://github.com/zachhandley/contentshield/blob/main/CONTRIBUTING.md

All public APIs must be documented using JSDoc comments. This example shows how to document an asynchronous analyze function, including its parameters, return type, and an example usage.

```typescript
/**
 * Analyze text for profanity with comprehensive results
 *
 * @param text - The text to analyze
 * @param options - Analysis options
 * @returns Promise that resolves to detection results
 *
 * @example
 * ```typescript
 * const result = await detector.analyze('text to check')
 * console.log(result.severity)
 * ```
 */
async analyze(text: string, options?: AnalysisOptions): Promise<DetectionResult> {
  // Implementation
}
```

--------------------------------

### Filter User-Generated Content by Severity

Source: https://github.com/zachhandley/contentshield/blob/main/README.md

Implement content filtering based on severity levels for user-generated content. This example uses a ContentShieldDetector instance configured for specific languages and minimum severity.

```typescript
import { ContentShieldDetector, SeverityLevel } from 'content-shield'
import { EN, ES } from 'content-shield/languages'

const detector = new ContentShieldDetector({
  minSeverity: SeverityLevel.HIGH, // Only catch the naughtiest words
  languages: ['en', 'es'],
  languageData: { en: EN, es: ES }
})

async function filterUserContent(content: string) {
  const result = await detector.analyze(content)

  if (result.maxSeverity >= SeverityLevel.SEVERE) {
    // Block content entirely
    return null
  } else if (result.hasProfanity) {
    // Filter but allow
    return await detector.filter(content)
  }

  return content
}
```

--------------------------------

### Simple Text Detection and Filtering

Source: https://github.com/zachhandley/contentshield/blob/main/README.md

Use this snippet for a quick check of profanity in text and to get detailed analysis results. Ensure ContentShield is configured with language data before use.

```typescript
import { detect, filter, isClean, configure } from 'content-shield'
import { EN } from 'content-shield/languages/en'

// Configure once with language data
await configure({ languageData: { en: EN } })

// Quick naughty word check
const isProfane = !(await isClean('Your text here'))

// Get detailed analysis
const result = await detect('Your text here')
console.log(result.hasProfanity, result.matches)

// Filter naughty content
const cleanText = await filter('Your text here')
```

--------------------------------

### Build the Library

Source: https://github.com/zachhandley/contentshield/blob/main/README.md

Compile the ContentShield library for distribution or development use.

```bash
npm run build
```

--------------------------------

### Importing Configuration Utilities

Source: https://github.com/zachhandley/contentshield/blob/main/CHANGELOG.md

Import utilities for managing and retrieving configuration presets, and for dynamic configuration management.

```typescript
import { getConfigPreset, ConfigManager } from 'naughty-words'
```

--------------------------------

### Run with Deno

Source: https://github.com/zachhandley/contentshield/blob/main/examples/hono-api/README.md

Executes the Hono API using Deno, allowing network, environment, and file access.

```bash
deno run --allow-net --allow-env --allow-read npm:tsx src/server-node.ts
```

--------------------------------

### Fork and Branch

Source: https://github.com/zachhandley/contentshield/blob/main/README.md

Initiate contribution by forking the repository and creating a new feature branch.

```bash
git checkout -b feature/amazing-feature
```

--------------------------------

### Development Scripts

Source: https://github.com/zachhandley/contentshield/blob/main/CONTRIBUTING.md

Scripts for running the development server, building the library, and watching for changes.

```bash
# Development
npm run build          # Build the library
npm run build:watch    # Watch mode for development
npm run dev           # Development server
```

--------------------------------

### Format Code

Source: https://github.com/zachhandley/contentshield/blob/main/README.md

Automatically format the code according to project style guidelines.

```bash
npm run format
```

--------------------------------

### Clone ContentShield Repository

Source: https://github.com/zachhandley/contentshield/blob/main/CONTRIBUTING.md

Clone your forked repository locally and navigate into the project directory.

```bash
git clone https://github.com/yourusername/ContentShield.git
cd ContentShield
```

--------------------------------

### Project Structure Overview

Source: https://github.com/zachhandley/contentshield/blob/main/CONTRIBUTING.md

Illustrates the directory structure of the contentshield project, detailing the purpose of each main directory.

```tree
src/
├── core/             # Core detection engine
│   ├── detector.ts   # Main detector class
│   ├── trie.ts       # Trie data structure
│   ├── filter.ts     # Text filtering
│   └── ...
├── config/           # Configuration system
│   ├── default-config.ts
│   ├── config-validator.ts
│   └── config-manager.ts
├── languages/        # Language integration
│   ├── factory.ts    # Factory functions
│   ├── language-loader.ts
│   └── multi-language-detector.ts
├── types/           # TypeScript types
├── utils/           # Utility functions
└── index.ts         # Main entry point

tests/
├── unit/            # Unit tests
├── integration/     # Integration tests
└── performance/     # Performance tests

data/
└── languages/       # Language data files
    ├── en/
    ├── es/
    └── ...

```

--------------------------------

### Configure Multi-language Support

Source: https://github.com/zachhandley/contentshield/blob/main/examples/hono-api/README.md

Demonstrates how to import and configure ContentShield for multiple languages, including English, Spanish, and German.

```typescript
import { ES } from 'content-shield/languages/es'
import { DE } from 'content-shield/languages/de'
await configure({
  languageData: { en: EN, es: ES, de: DE },
  languages: ['en', 'es', 'de'],
})
```

--------------------------------

### Run Tests

Source: https://github.com/zachhandley/contentshield/blob/main/README.md

Execute the test suite to ensure code integrity and functionality.

```bash
npm test
```

--------------------------------

### Run ContentShield Tests with npm Scripts

Source: https://github.com/zachhandley/contentshield/blob/main/CONTRIBUTING.md

Utilize npm scripts for running tests, including options for running all tests, specific files, generating coverage reports, or filtering for performance tests.

```bash
# Run all tests
npm run test:run

# Run specific test file
npm run test:run -- detector.test.ts

# Run with coverage
npm run test:coverage

# Run performance tests only
npm run test:run -- performance/
```

--------------------------------

### Run Content Detection and Filtering

Source: https://github.com/zachhandley/contentshield/blob/main/site/index.html

Handles the 'click' event on the run button to initiate content analysis. It takes user input, ensures the selected language is loaded, calls ContentShield.detect and ContentShield.filter, and updates the UI with the results, including profanity status, match count, severity, filtered text, and individual matches.

```javascript
runBtn.addEventListener('click', async () => {
  const text = input.value.trim()
  if (!text) {
    status.textContent = 'Type something to analyse first.'
    return
  }
  runBtn.disabled = true
  try {
    await ensureLanguageLoaded(langSelect.value)
    const result = await ContentShield.detect(text)
    const filtered = await ContentShield.filter(text)
    resultsCard.hidden = false
    summary.innerHTML = ` <span>Profanity: <strong>${result.hasProfanity ? 'yes' : 'no'}</strong></span> <span>Matches: <strong>${result.totalMatches || 0}</strong></span> <span>Max severity: <strong>${result.maxSeverity || 0}</strong></span> `
    filteredEl.textContent = filtered || text
    matchesEl.innerHTML = ''
    for (const m of (result.matches || [])) {
      matchesEl.appendChild(renderMatch(m))
    }
    if (!result.matches?.length) {
      matchesEl.innerHTML = '<div class="status">No matches.</div>'
    }
  } catch (err) {
    status.textContent = `Error: ${err.message}`
    console.error(err)
  } finally {
    runBtn.disabled = false
  }
})
```

--------------------------------

### Initialize ContentShield Detector with Custom Words and Whitelist

Source: https://github.com/zachhandley/contentshield/blob/main/README.md

Configure the ContentShield detector with specific languages, custom profanity, and a whitelist of words to ignore. Use 'auto' for language detection or specify exact languages for better performance.

```typescript
import { ContentShieldDetector, SeverityLevel, ProfanityCategory } from 'content-shield'
import { EN } from 'content-shield/languages/en'

const detector = new ContentShieldDetector({
  languages: ['en'],
  languageData: { en: EN },
  customWords: [
    {
      word: 'frack', // Custom sci-fi profanity
      language: 'en',
      severity: SeverityLevel.MODERATE,
      categories: [ProfanityCategory.GENERAL],
      variations: ['fr@ck', 'fr4ck'],
      caseSensitive: false
    }
  ],
  whitelist: ['hello', 'world'] // Never flag these words
})
```

--------------------------------

### Run Tests with Coverage

Source: https://github.com/zachhandley/contentshield/blob/main/README.md

Execute tests and generate a code coverage report to identify untested areas.

```bash
npm run test:coverage
```

--------------------------------

### Push to Branch

Source: https://github.com/zachhandley/contentshield/blob/main/README.md

Upload your local feature branch to the remote repository.

```bash
git push origin feature/amazing-feature
```

--------------------------------

### Cleanup Script

Source: https://github.com/zachhandley/contentshield/blob/main/CONTRIBUTING.md

Script to clean build artifacts from the project.

```bash
# Cleanup
npm run clean         # Clean build artifacts
```

--------------------------------

### Initialize and Load Language Data

Source: https://github.com/zachhandley/contentshield/blob/main/site/index.html

Sets up event listeners for UI elements and ensures the necessary language data is loaded before performing analysis. Handles user input, language selection, and triggers the detection process.

```javascript
const input = document.getElementById('input')
const langSelect = document.getElementById('lang')
const runBtn = document.getElementById('run')
const status = document.getElementById('status')
const resultsCard = document.getElementById('resultsCard')
const summary = document.getElementById('summary')
const filteredEl = document.getElementById('filtered')
const matchesEl = document.getElementById('matches')
const loadedLanguages = new Set(['en'])
const baseUrl = window.location.origin + window.location.pathname.replace(/\/[^/]*$/, '')

async function ensureLanguageLoaded(code) {
  if (loadedLanguages.has(code)) return
  status.textContent = `Loading ${code} data...`
  await ContentShield.loadLanguage(code, baseUrl)
  loadedLanguages.add(code)
  status.textContent = `Loaded ${[...loadedLanguages].join(', ')}.`
}
```

--------------------------------

### Importing Core Functions

Source: https://github.com/zachhandley/contentshield/blob/main/CHANGELOG.md

Import the primary detection, filtering, and cleaning functions for quick integration.

```typescript
import { detect, filter, isClean } from 'naughty-words'
```

--------------------------------

### Importing Utility Functions

Source: https://github.com/zachhandley/contentshield/blob/main/CHANGELOG.md

Import utility functions for text manipulation, fuzzy matching, and similarity calculations.

```typescript
import { fuzzyMatch, normalizeText, calculateSimilarity } from 'naughty-words'
```

--------------------------------

### Load Language and Detect Profanity (JavaScript)

Source: https://github.com/zachhandley/contentshield/blob/main/site/index.html

This snippet shows how to load a specific language pack and then use ContentShield to detect profanity in a given text. Ensure the script is loaded before execution.

```javascript
<script src="https://zachhandley.github.io/ContentShield/content-shield.global.js"></script>
    <script>
      // English is bundled. For other languages:
      await ContentShield.loadLanguage('es', 'https://zachhandley.github.io/ContentShield')
    
      const result = await ContentShield.detect("some text to check")
      console.log(result)
    </script>
```

--------------------------------

### POST /detect

Source: https://github.com/zachhandley/contentshield/blob/main/examples/hono-api/README.md

Performs content detection on the provided text and returns the full DetectionResult from ContentShield.

```APIDOC
## POST /detect

### Description
Performs content detection on the provided text and returns the full DetectionResult from ContentShield.

### Method
POST

### Endpoint
/detect

### Parameters
#### Request Body
- **text** (string) - Required - The text to analyze.

### Request Example
{
  "text": "..."
}

### Response
#### Success Response (200)
- **DetectionResult** (object) - The full detection result from ContentShield.
```

--------------------------------

### Create Language Detectors

Source: https://github.com/zachhandley/contentshield/blob/main/README.md

Instantiate English-only, Spanish-only, or multi-language detectors using the create functions. Ensure the correct language data is provided for each detector.

```typescript
import {
  createEnglishDetector,
  createSpanishDetector,
  createMultiLanguageDetector
} from 'content-shield'
import { EN, ES, FR } from 'content-shield/languages'

const englishOnly = createEnglishDetector({ languageData: { en: EN } })
const spanishOnly = createSpanishDetector({ languageData: { es: ES } })
const multiLang = createMultiLanguageDetector(
  ['en', 'es', 'fr'],
  { languageData: { en: EN, es: ES, fr: FR } }
)
```

--------------------------------

### Add Upstream Repository

Source: https://github.com/zachhandley/contentshield/blob/main/CONTRIBUTING.md

Add the official ContentShield repository as an upstream remote to keep your fork updated.

```bash
git remote add upstream https://github.com/zachhandley/ContentShield.git
```

--------------------------------

### Commit Changes

Source: https://github.com/zachhandley/contentshield/blob/main/README.md

Record your changes with a descriptive commit message.

```bash
git commit -m 'Add amazing feature'
```

--------------------------------

### Render Match Details

Source: https://github.com/zachhandley/contentshield/blob/main/site/index.html

Creates and formats HTML elements to display individual match details, including severity and categories. This function is used to render each detected match in a user-friendly format.

```javascript
function renderMatch(match) {
  const sev = Math.min(4, Math.max(1, match.severity || 1))
  const cats = (match.categories || []).join(', ') || '—'
  const wrap = document.createElement('div')
  wrap.className = 'match'
  wrap.innerHTML = ` <span class="badge sev-${sev}">Severity ${sev}</span> <span class="word"></span> <span class="cats"></span> `
  wrap.querySelector('.word').textContent = match.word
  wrap.querySelector('.cats').textContent = cats
  return wrap
}
```

--------------------------------

### Testing Scripts

Source: https://github.com/zachhandley/contentshield/blob/main/CONTRIBUTING.md

Scripts for running various types of tests, including watch mode, single runs, coverage, and UI tests.

```bash
# Testing
npm run test          # Run tests in watch mode
npm run test:run      # Run tests once
npm run test:coverage # Run tests with coverage
npm run test:ui       # Run tests with UI
```

--------------------------------

### Bug Report Template

Source: https://github.com/zachhandley/contentshield/blob/main/CONTRIBUTING.md

Markdown template for reporting bugs. Includes sections for bug description, reproduction steps, expected behavior, environment details, and additional context.

```markdown
**Describe the Bug**
Clear description of the bug.

**To Reproduce**
Steps to reproduce the behavior:
1. Initialize with config...
2. Call method...
3. See error

**Expected Behavior**
What you expected to happen.

**Environment**
- OS: [e.g., Windows 10, macOS 12.0, Ubuntu 20.04]
- Node.js version: [e.g., 16.14.0]
- Package version: [e.g., 1.0.0]

**Additional Context**
Any other context about the problem.
```

--------------------------------

### Feature Request Template

Source: https://github.com/zachhandley/contentshield/blob/main/CONTRIBUTING.md

Markdown template for submitting feature requests. Includes sections for feature description, use case, proposed solution, alternatives considered, and additional context.

```markdown
**Feature Description**
Clear description of the feature you'd like to see.

**Use Case**
Explain the use case and why this feature would be valuable.

**Proposed Solution**
Describe how you envision this feature working.

**Alternatives Considered**
Any alternative solutions or features you've considered.

**Additional Context**
Any other context or screenshots about the feature request.
```

--------------------------------

### Importing Advanced Types and Classes

Source: https://github.com/zachhandley/contentshield/blob/main/CHANGELOG.md

Import specific types and classes for more advanced usage scenarios, such as custom detector configurations.

```typescript
import { NaughtyWordsDetector, SeverityLevel, FilterMode } from 'naughty-words'
```

--------------------------------

### Pull Request Template

Source: https://github.com/zachhandley/contentshield/blob/main/CONTRIBUTING.md

Use this template when creating a pull request to ensure all necessary information is provided. It covers the description, type of change, testing, performance impact, and related issues.

```markdown
## Description
Brief description of changes and motivation.

## Type of Change
- [ ] Bug fix (non-breaking change fixing an issue)
- [ ] New feature (non-breaking change adding functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
- [ ] Documentation update

## Testing
- [ ] New tests added
- [ ] Existing tests updated
- [ ] Performance tests included (if applicable)

## Performance Impact
Describe any performance implications of the changes.

## Checklist
- [ ] Code follows style guidelines
- [ ] Self-review completed
- [ ] Tests pass locally
- [ ] Documentation updated
- [ ] No breaking changes (or properly documented)

## Related Issues
Closes #issue_number
```

--------------------------------

### Use ContentShield in the browser

Source: https://github.com/zachhandley/contentshield/blob/main/README.md

Include the ContentShield script in your HTML and use its functions for text analysis. Supports loading different languages.

```html
<script src="https://zachhandley.github.io/ContentShield/content-shield.global.js"></script>
<script>
  // English is bundled with the IIFE. For other languages:
  await ContentShield.loadLanguage('es', 'https://zachhandley.github.io/ContentShield')

  const result = await ContentShield.detect("some user-submitted text")
  console.log(result.hasProfanity, result.matches)
</script>
```

--------------------------------

### Lint Code

Source: https://github.com/zachhandley/contentshield/blob/main/README.md

Check the code for stylistic errors and potential issues using the configured linter.

```bash
npm run lint
```

--------------------------------

### Advanced Content Analysis and Filtering

Source: https://github.com/zachhandley/contentshield/blob/main/README.md

This snippet demonstrates creating a custom ContentShieldDetector for multi-language analysis, specifying severity levels, categories, and fuzzy matching. It also shows different filtering modes: censor, remove, and replace.

```typescript
import {
  ContentShieldDetector,
  SeverityLevel,
  ProfanityCategory,
  FilterMode
} from 'content-shield'
import { EN, ES, FR } from 'content-shield/languages'

// Create a custom multi-language detector
const detector = new ContentShieldDetector({
  languages: ['en', 'es', 'fr'],
  languageData: { en: EN, es: ES, fr: FR },
  minSeverity: SeverityLevel.MODERATE,
  categories: [
    ProfanityCategory.HATE_SPEECH,
    ProfanityCategory.VIOLENCE
  ],
  fuzzyMatching: true,
  fuzzyThreshold: 0.8
})

// Analyze text
const analysis = await detector.analyze('Text to analyze')

// Filter with different modes
const censored = await detector.filter(text, FilterMode.CENSOR)  // "f***"
const removed = await detector.filter(text, FilterMode.REMOVE)   // ""
const replaced = await detector.filter(text, FilterMode.REPLACE) // "[filtered]"
```

--------------------------------

### Importing Factory Functions

Source: https://github.com/zachhandley/contentshield/blob/main/CHANGELOG.md

Import factory functions to create specialized detectors, such as English-only or multi-language detectors.

```typescript
import { createEnglishDetector, createMultiLanguageDetector } from 'naughty-words'
```

--------------------------------

### Code Quality Scripts

Source: https://github.com/zachhandley/contentshield/blob/main/CONTRIBUTING.md

Scripts for linting, fixing linting issues, formatting code with Prettier, and performing TypeScript type checking.

```bash
# Code Quality
npm run lint          # Lint code
npm run lint:fix      # Fix linting issues
npm run format        # Format code with Prettier
npm run typecheck     # TypeScript type checking
```

--------------------------------

### POST /is-clean

Source: https://github.com/zachhandley/contentshield/blob/main/examples/hono-api/README.md

Checks if the provided text is clean, returning a boolean indicating its status.

```APIDOC
## POST /is-clean

### Description
Checks if the provided text is clean, returning a boolean indicating its status.

### Method
POST

### Endpoint
/is-clean

### Parameters
#### Request Body
- **text** (string) - Required - The text to check.

### Request Example
{
  "text": "..."
}

### Response
#### Success Response (200)
- **clean** (boolean) - True if the text is clean, false otherwise.
```

--------------------------------

### Write Unit Tests for ContentShieldDetector

Source: https://github.com/zachhandley/contentshield/blob/main/CONTRIBUTING.md

Use Vitest to write unit tests for individual functions and classes within ContentShieldDetector. Ensure tests cover expected behavior for profanity detection and handling of empty input.

```typescript
import { describe, it, expect, beforeEach } from 'vitest'
import { ContentShieldDetector } from '../src/core/detector.js'

describe('ContentShieldDetector', () => {
  let detector: ContentShieldDetector

  beforeEach(async () => {
    detector = new ContentShieldDetector()
    await detector.initialize()
  })

  it('should detect profanity correctly', async () => {
    const result = await detector.analyze('test text with profanity')

    expect(result.matches).toHaveLength(1)
    expect(result.severity).toBeGreaterThan(0)
    expect(result.confidence).toBeGreaterThan(0.5)
  })

  it('should handle empty text gracefully', async () => {
    const result = await detector.analyze('')

    expect(result.matches).toHaveLength(0)
    expect(result.severity).toBe(0)
  })
})
```

--------------------------------

### Benchmark Performance Changes

Source: https://github.com/zachhandley/contentshield/blob/main/CONTRIBUTING.md

Use this code to benchmark performance changes by measuring the execution time of a function over multiple iterations and calculating the average time.

```typescript
// Benchmark your changes
const times = []
for (let i = 0; i < 100; i++) {
  const start = performance.now()
  await yourFunction()
  const end = performance.now()
  times.push(end - start)
}

const avgTime = times.reduce((a, b) => a + b, 0) / times.length
console.log(`Average time: ${avgTime}ms`)
```

--------------------------------

### POST /filter

Source: https://github.com/zachhandley/contentshield/blob/main/examples/hono-api/README.md

Filters the provided text, returning the filtered result.

```APIDOC
## POST /filter

### Description
Filters the provided text, returning the filtered result.

### Method
POST

### Endpoint
/filter

### Parameters
#### Request Body
- **text** (string) - Required - The text to filter.

### Request Example
{
  "text": "..."
}

### Response
#### Success Response (200)
- **filtered** (string) - The filtered text.
```

--------------------------------

### Implement Performance Tests for ContentShield

Source: https://github.com/zachhandley/contentshield/blob/main/CONTRIBUTING.md

Add performance tests to ensure code meets performance requirements, such as processing speed and memory usage. These tests are crucial for performance-critical code sections.

```typescript
describe('Performance Tests', () => {
  it('should process 1000+ texts per second', async () => {
    const texts = generateTestTexts(1000)
    const start = performance.now()

    await detector.batchAnalyze(texts)

    const end = performance.now()
    const textsPerSecond = 1000 / ((end - start) / 1000)

    expect(textsPerSecond).toBeGreaterThan(1000)
  })

  it('should maintain memory usage under 200MB', async () => {
    const initialMemory = process.memoryUsage().heapUsed

    // Perform memory-intensive operations
    await performLargeOperations()

    const finalMemory = process.memoryUsage().heapUsed
    const memoryIncrease = finalMemory - initialMemory

    expect(memoryIncrease).toBeLessThan(200 * 1024 * 1024) // 200MB
  })
})
```

--------------------------------

### Moderate Chat Messages with Profanity Detection

Source: https://github.com/zachhandley/contentshield/blob/main/README.md

Use this snippet to detect and censor profanity in chat messages. Configure ContentShield once at application startup with language data.

```typescript
import { detect, filter, FilterMode, configure } from 'content-shield'
import { EN } from 'content-shield/languages/en'

// Configure once at app startup
await configure({ languageData: { en: EN } })

async function moderateMessage(message: string) {
  const result = await detect(message)

  if (result.hasProfanity) {
    // Uh oh, someone's been naughty!
    console.log(`Naughty words detected: ${result.totalMatches} matches`)

    // Return filtered message
    return await filter(message, FilterMode.CENSOR)
  }

  return message
}
```

--------------------------------

### Validate User Input for Cleanliness

Source: https://github.com/zachhandley/contentshield/blob/main/README.md

This snippet checks if user-provided input, like usernames, is free of prohibited content. Ensure ContentShield is configured with language data.

```typescript
import { isClean, configure } from 'content-shield'
import { EN } from 'content-shield/languages/en'

// Configure once at app startup
await configure({ languageData: { en: EN } })

async function validateUsername(username: string) {
  const clean = await isClean(username)

  if (!clean) {
    throw new Error('Username contains naughty content')
  }

  return username
}
```

--------------------------------

### ContentShield Filter Modes

Source: https://github.com/zachhandley/contentshield/blob/main/README.md

Defines the different modes for handling detected profanity: CENSOR, REMOVE, REPLACE, or DETECT_ONLY.

```typescript
enum FilterMode {
  CENSOR = 'censor',           // Replace with * (f***)
  REMOVE = 'remove',           // Remove entirely (poof!)
  REPLACE = 'replace',         // Replace with [filtered]
  DETECT_ONLY = 'detect_only'  // Just detect, don't modify
}
```
Tokens:
Raw