Datadog (datadog/documentation)

Datadog

https://github.com/datadog/documentation
Admin
Datadog is a monitoring and analytics platform for cloud-scale applications, providing comprehensive...

Tokens:4,214,352
Snippets:26,050
Trust Score:9.4
Update:4 months ago
Show doc for...
Context Summary (auto-generated)
Raw
# Datadog Documentation Repository

## Project Overview

The Datadog Documentation repository is a comprehensive static site generator system built with Hugo that powers the official Datadog documentation at docs.datadoghq.com. This repository manages multi-language technical documentation covering 67+ product categories including monitoring, observability, security, and cloud integrations. The system uses Hugo as the static site generator (version 0.148.0), enhanced with Node.js build automation, Python content processing scripts, AlpineJS for reactive components, and extensive JavaScript components for interactive features. The project utilizes Go 1.24.4 for module management with the websites-modules dependency (v1.4.269).

The architecture follows a distributed content model where documentation is sourced from multiple external repositories (integrations-core, integrations-extras, marketplace, and the websites-sources Go module), processed through automated build pipelines, and published with region-specific content delivery. The system supports five languages (English, French, Spanish, Japanese, Korean), generates API and reference documentation from OpenAPI specifications and schema files in six programming languages, and provides over 430 Hugo shortcodes for consistent content presentation. Branch-based preview environments, Typesense-powered search with automated synchronization, CDOCS component integration, and extensive automation make this a production-grade documentation platform handling thousands of pages across multiple products.

## Build System and Development Commands

### Starting the Development Server

```bash
# Full build including external dependencies
make start

# Expected output:
# Setting up build scripts...
# Installing Node.js dependencies...
# Fetching integration data from S3...
# Compiling .mdoc files...
# Hugo server running on http://localhost:1313

# Quick start without external dependencies
make start-no-pre-build

# Start with Docker container
make start-docker

# Using Yarn directly (basic development server)
yarn start
# Server runs on port 1313
```

### Building for Production

```bash
# Production build with minification
yarn build

# Output structure:
# ./public/
#   ├── index.html
#   ├── api/
#   │   ├── v1/
#   │   └── v2/
#   ├── integrations/
#   ├── static/
#   └── [additional content directories]

# Preview environment build (includes drafts and future content)
yarn run build:preview

# Live/production build
yarn run build:live

# Build API documentation pages from OpenAPI specs
yarn run build:apiPages
# Processes ./data/api/v1/full_spec.yaml and ./data/api/v2/full_spec.yaml
# Generates content in ./content/en/api/v1/ and ./content/en/api/v2/

# Build reference documentation pages from schema files
# Uses build-reference-pages.js to generate reference docs from data/reference/
# Processes schema.json, functions.json, errors.json, and schema.tables.json
```

### Dependency Management

```bash
# Install all dependencies (Node.js + Python + external repos)
make dependencies

# Expected behavior:
# 1. Creates Python virtual environment in ./hugpython
# 2. Installs yarn packages from package.json (managed by Yarn 4.10.3)
# 3. Clones API client example repositories
# 4. Downloads integration data from S3
# 5. Compiles .mdoc files to HTML via cdocs-hugo-integration
# 6. Builds llms.txt file for LLM consumption

# Install only Node.js dependencies
yarn install --immutable

# Set up Python virtual environment
make hugpython
# Creates ./hugpython/ directory with Python 3 virtual environment
```

### Working with External Content

```bash
# Download integration data from websites-sources S3 bucket
make websites_sources_data

# Downloads to: ./_vendor/
# Contains: integrations, layouts, data files, assets

# Preview specific feature branch from S3
make start-sources FF_S3_PATH=feature-branch-name
# Default path is "staging" if not specified

# Find source file for a specific integration
make find-int int=postgres
# Output: Path to integration README.md in external repository

# Compile .mdoc files to HTML (CDOCS components)
make build-cdocs
# Processes .mdoc.md files and converts them to HTML for Hugo

# Watch for CDOCS file changes during development
make watch-cdocs
# Automatically recompiles .mdoc files when they change

# Build llms.txt file for LLM consumption
make build-llms-txt
# Generates optimized documentation format for AI assistants
```

## Hugo Shortcodes

### Code Display Shortcodes

```markdown
<!-- Basic code block with syntax highlighting -->
{{< code-block lang="python" filename="example.py" >}}
import datadog_api_client
from datadog_api_client.v2 import ApiClient

configuration = datadog_api_client.Configuration(
    api_key={"apiKeyAuth": "YOUR_API_KEY"}
)

with ApiClient(configuration) as api_client:
    # Your code here
    pass
{{< /code-block >}}

<!-- Code block with line wrapping enabled -->
{{< code-block lang="bash" wrap="true" >}}
curl -X POST "https://api.datadoghq.com/api/v2/metrics" \
  -H "DD-API-KEY: ${DD_API_KEY}" \
  -H "Content-Type: application/json" \
  -d @- << EOF
{
  "series": [
    {
      "metric": "system.load.1",
      "type": 0,
      "points": [{"timestamp": 1636629071, "value": 0.7}]
    }
  ]
}
EOF
{{< /code-block >}}

<!-- Collapsible code block -->
{{< code-block lang="yaml" filename="config.yaml" collapsible="true" >}}
init_config:
instances:
  - host: localhost
    port: 5432
    username: datadog
    password: ${POSTGRES_PASSWORD}
{{< /code-block >}}

<!-- Disable copy button -->
{{< code-block lang="javascript" disable_copy="true" >}}
console.log('This code cannot be copied');
{{< /code-block >}}
```

### Multi-Language Code Tabs

```markdown
<!-- Create tabbed code examples -->
{{< tabs >}}
{{% tab "Python" %}}
```python
from datadog import api, initialize

options = {
    'api_key': '<YOUR_API_KEY>',
    'app_key': '<YOUR_APP_KEY>'
}

initialize(**options)

api.Monitor.create(
    type="metric alert",
    query="avg(last_5m):sum:system.net.bytes_rcvd{*} > 100",
    name="Bytes received on host0",
    message="We may need to add web hosts"
)
```
{{% /tab %}}
{{% tab "Ruby" %}}
```ruby
require 'dogapi'

api_key = '<YOUR_API_KEY>'
app_key = '<YOUR_APP_KEY>'

dog = Dogapi::Client.new(api_key, app_key)

dog.monitor(
  'metric alert',
  'avg(last_5m):sum:system.net.bytes_rcvd{*} > 100',
  name: 'Bytes received on host0',
  message: 'We may need to add web hosts'
)
```
{{% /tab %}}
{{% tab "cURL" %}}
```bash
curl -X POST "https://api.datadoghq.com/api/v1/monitor" \
  -H "DD-API-KEY: ${DD_API_KEY}" \
  -H "DD-APPLICATION-KEY: ${DD_APP_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "type": "metric alert",
    "query": "avg(last_5m):sum:system.net.bytes_rcvd{*} > 100",
    "name": "Bytes received on host0",
    "message": "We may need to add web hosts"
  }'
```
{{% /tab %}}
{{< /tabs >}}
```

### Region-Specific Content

```markdown
<!-- Display content based on Datadog region -->
{{< site-region region="us" >}}
Your Datadog site is: https://app.datadoghq.com
API endpoint: https://api.datadoghq.com
{{< /site-region >}}

{{< site-region region="eu" >}}
Your Datadog site is: https://app.datadoghq.eu
API endpoint: https://api.datadoghq.eu
{{< /site-region >}}

{{< site-region region="us3,us5,ap1,gov" >}}
Contact support for region-specific endpoints.
{{< /site-region >}}

<!-- Regions: us, us3, us5, eu, ap1, gov -->
```

### SDK and Version Display

```markdown
<!-- Display current SDK version from data/sdk_versions.json -->
{{< sdk-version datadog-api-client-python >}}
<!-- Outputs: 2.44.0 -->

{{< sdk-version datadog-api-client-go >}}
<!-- Outputs: v2.47.0 -->

<!-- Display latest Lambda layer version -->
{{< latest-lambda-layer-version layer="python" >}}

<!-- Display Private Action Runner version -->
{{< private-action-runner-version >}}

<!-- Display Synthetics Worker version -->
{{< synthetics-worker-version >}}
```

### Integration Shortcodes

```markdown
<!-- Fetch metrics from integration metadata -->
{{< get-metrics-from-git "postgres" >}}
<!-- Fetches from: integrations_data or metadata JSON -->

<!-- Fetch service checks -->
{{< get-service-checks-from-git "postgres" >}}

<!-- Display integration tile gallery with category filtering -->
{{< integrations >}}
<!-- Renders filterable grid of all integrations -->

<!-- Display specific integration category -->
{{< integrations category="aws" >}}
{{< integrations category="marketplace" >}}
{{< integrations category="ccm" >}}  <!-- Cloud Cost Management -->
```

### Callouts and Notifications

```markdown
<!-- Beta feature callout -->
{{< beta-callout url="https://forms.gle/example" >}}
This feature is in public beta. To request access, complete the form.
{{< /beta-callout >}}

<!-- Private beta callout -->
{{< beta-callout-private >}}
This feature is in private beta.
{{< /beta-callout-private >}}

<!-- Custom callout with CTA button -->
{{< callout header="Get Started" url="https://app.datadoghq.com/signup" >}}
Sign up for a free Datadog trial to get started.
{{< /callout >}}

<!-- Learning center callout -->
{{< learning-center-callout >}}
```

### Collapsible Content

```markdown
<!-- Collapsible section -->
{{< collapse title="Advanced Configuration" >}}
This content is hidden by default and can be expanded by clicking.

```yaml
advanced_config:
  enable_profiling: true
  sample_rate: 0.5
```

{{< /collapse >}}

<!-- Nested collapsible sections work -->
{{< collapse title="Troubleshooting" >}}
Common issues and solutions...

{{< collapse title="Connection Errors" >}}
Check your network configuration.
{{< /collapse >}}

{{< /collapse >}}
```

### Image and Media

```markdown
<!-- Responsive image with imgix processing -->
{{< img src="screenshots/dashboard-example.png" alt="Dashboard example" >}}

<!-- Video embeds -->
{{< wistia video-id="abc123xyz" >}}
{{< vimeo video-id="123456789" >}}
```

### Navigation Helpers

```markdown
<!-- What's Next section (related content) -->
{{< whatsnext desc="Explore related topics:" >}}
  {{< nextlink href="/logs/" >}}Log Management{{< /nextlink >}}
  {{< nextlink href="/tracing/" >}}APM & Distributed Tracing{{< /nextlink >}}
  {{< nextlink href="/metrics/" >}}Metrics{{< /nextlink >}}
{{< /whatsnext >}}

<!-- Tile-based navigation -->
{{< tile-nav >}}

<!-- Absolute language-specific URL -->
{{< absLangUrl href="/getting_started/" >}}
<!-- In French: /fr/getting_started/ -->
<!-- In English: /getting_started/ -->
```

## API and Reference Documentation Generation

### Processing OpenAPI Specifications (API Documentation)

```javascript
// build-api-pages.js - Main API documentation generator
const init = require('./assets/scripts/build-api-pages.js').init;

// Processes OpenAPI specs and generates:
// 1. API section pages (content/en/api/{version}/{tag}/_index.md)
// 2. Resource examples (content/en/api/{version}/{tag}/examples.json)
// 3. Menu structure (config/_default/menus/api.en.yaml)
// 4. Translation strings (data/api/{version}/translate_*.json)

// Run the build:
node -e 'require("./assets/scripts/build-api-pages.js").init()'
```

### API Documentation Structure

```yaml
# Example generated API section: content/en/api/v2/monitors/_index.md
---
title: Monitors
headless: true
---

# examples.json structure for each endpoint
{
  "operationId": "CreateMonitor",
  "tag": "Monitors",
  "summary": "Create a monitor",
  "path": "/api/v2/monitors",
  "method": "POST",
  "parameters": {
    "body": {
      "type": "object",
      "properties": {
        "name": {"type": "string", "example": "Example Monitor"},
        "type": {"type": "string", "example": "metric alert"},
        "query": {"type": "string", "example": "avg(last_5m):sum:system.load.1{*} > 0.5"}
      }
    }
  },
  "examples": {
    "go": "examples/go/monitors/CreateMonitor.go",
    "python": "examples/python/monitors/CreateMonitor.py",
    "ruby": "examples/ruby/monitors/CreateMonitor.rb",
    "typescript": "examples/typescript/monitors/CreateMonitor.ts",
    "java": "examples/java/monitors/CreateMonitor.java",
    "rust": "examples/rust/monitors/create_monitor.rs"
  },
  "response": {
    "200": {
      "description": "OK",
      "schema": {"$ref": "#/components/schemas/Monitor"}
    }
  }
}
```

### Processing Reference Documentation (Schema-based)

```javascript
// build-reference-pages.js - Reference documentation generator
// Processes schema files from data/reference/ directory
const init = require('./assets/scripts/build-reference-pages.js').init;

// Processes reference documentation from:
// 1. data/reference/schema.json - Main schema definitions
// 2. data/reference/functions.json - Function references
// 3. data/reference/errors.json - Error code documentation
// 4. data/reference/schema.tables.json - Table schema documentation
// 5. data/reference/schema.deref.json - Dereferenced schema

// Generates structured reference pages similar to API documentation
// Uses lodash, js-yaml, marked, and slugify for content transformation
```

### Dereferencing OpenAPI Specs

```javascript
// dereference.js - Resolve $ref pointers in OpenAPI specs
const $RefParser = require('@apidevtools/json-schema-ref-parser');
const fs = require('fs');

async function dereferenceSpec(specPath, outputPath) {
    const schema = await $RefParser.dereference(specPath);
    fs.writeFileSync(outputPath, JSON.stringify(schema, null, 2));
}

// Usage in build pipeline:
// yarn run dereference
// Input: ./data/api/v1/full_spec.yaml
// Output: ./data/api/v1/full_spec_deref.json
```

### API Code Examples Configuration

```json
// data/sdk_versions.json - Tracks SDK versions for examples
[
  {
    "client": "datadog-api-client-go",
    "version": "v2.47.0"
  },
  {
    "client": "datadog-api-client-python",
    "version": "2.44.0"
  },
  {
    "client": "datadog-api-client-ruby",
    "version": "v2.43.0"
  },
  {
    "client": "datadog-api-client-typescript",
    "version": "v1.45.0"
  },
  {
    "client": "datadog-api-client-java",
    "version": "datadog-api-client-2.44.0"
  },
  {
    "client": "datadog-api-client-rust",
    "version": "0.20.0"
  },
  {
    "client": "integrations-core",
    "version": "ddev-v13.0.0"
  }
]
```

### Fetching API Client Examples

```makefile
# Makefile - Clone API client repositories for examples
EXAMPLES_REPOS := datadog-api-client-go datadog-api-client-java \
                  datadog-api-client-python datadog-api-client-ruby \
                  datadog-api-client-typescript datadog-api-client-rust

all-examples:
	@for repo in $(EXAMPLES_REPOS); do \
		git clone https://github.com/DataDog/$$repo.git examples/$$repo; \
	done

# Examples structure:
# examples/
#   ├── datadog-api-client-go/examples/
#   ├── datadog-api-client-python/examples/
#   ├── datadog-api-client-ruby/examples/
#   ├── datadog-api-client-typescript/examples/
#   ├── datadog-api-client-java/examples/
#   └── datadog-api-client-rust/examples/
```

## JavaScript Components and Interactive Features

### AlpineJS Reactive Components

```javascript
// assets/scripts/alpine.js - AlpineJS integration (v3.13.7)
// Provides reactive, declarative JavaScript framework for interactive UI components
import Alpine from 'alpinejs';

// AlpineJS is used throughout the documentation for:
// - Interactive dropdowns and toggles
// - Dynamic content filtering
// - State management in UI components
// - Region selector functionality
// - Cookie banner interactions

// Example usage in HTML templates:
// <div x-data="{ open: false }">
//   <button @click="open = !open">Toggle</button>
//   <div x-show="open">Content</div>
// </div>

window.Alpine = Alpine;
Alpine.start();
```

### Multi-Language Code Tabs

```javascript
// assets/scripts/components/codetabs.js
import { getQueryParameterByName } from '../helpers/browser';
import regionConfig from '../config/regions.config';

const initCodeTabs = () => {
    const tabQueryParameter = getQueryParameterByName('tab');

    // Detect all code tab containers
    const codeTabsList = document.querySelectorAll('.code-tabs');

    // Render tab navigation
    const renderCodeTabElements = () => {
        codeTabsList.forEach((tabContainer) => {
            const tabs = tabContainer.querySelectorAll('.tab-pane');
            const navTabs = tabContainer.querySelector('.nav-tabs');

            tabs.forEach((tab, index) => {
                const lang = tab.getAttribute('data-lang');
                const li = document.createElement('li');
                li.className = index === 0 ? 'active' : '';
                li.innerHTML = `<a data-lang="${lang}">${lang}</a>`;
                navTabs.appendChild(li);
            });
        });
    };

    // Activate tab and sync across all code blocks
    const activateCodeTab = (tabAnchorElement) => {
        const activeLang = tabAnchorElement.getAttribute('data-lang');
        document.querySelectorAll(`[data-lang="${activeLang}"]`).forEach((el) => {
            el.closest('li').classList.add('active');
            const tabPane = el.closest('.code-tabs').querySelector(`.tab-pane[data-lang="${activeLang}"]`);
            tabPane?.classList.add('active');
        });
        updateUrl(activeLang);
    };

    // Persist tab selection to URL
    const updateUrl = (activeLang) => {
        const url = new URL(window.location);
        url.searchParams.set('tab', activeLang);
        window.history.replaceState({}, '', url);
    };
};

// Usage: Automatically syncs language selection across all code examples on page
```

### Typesense Search Integration

```javascript
// assets/scripts/components/instantsearch.js
import TypesenseInstantSearchAdapter from 'typesense-instantsearch-adapter';
import instantsearch from 'instantsearch.js';

const typesenseAdapter = new TypesenseInstantSearchAdapter({
    server: {
        nodes: [
            { host: 'search1.datadoghq.com', port: '443', protocol: 'https' },
            { host: 'search2.datadoghq.com', port: '443', protocol: 'https' },
            { host: 'search3.datadoghq.com', port: '443', protocol: 'https' }
        ],
        apiKey: 'PUBLIC_SEARCH_API_KEY'
    },
    collectionSpecificSearchParameters: {
        docs_alias_view: {
            query_by: 'title,content,tags',
            per_page: 20
        }
    }
});

const search = instantsearch({
    indexName: 'docs_alias_view',
    searchClient: typesenseAdapter.searchClient,
    routing: true
});

// Add search widgets
search.addWidgets([
    instantsearch.widgets.searchBox({
        container: '#search-box',
        placeholder: 'Search documentation...'
    }),
    instantsearch.widgets.hits({
        container: '#hits',
        templates: {
            item: (hit) => `
                <a href="${hit.url}">
                    <h3>${hit._highlightResult.title.value}</h3>
                    <p>${hit._highlightResult.content.value}</p>
                </a>
            `
        }
    }),
    instantsearch.widgets.pagination({
        container: '#pagination'
    })
]);

search.start();

// Automated search index synchronization
// yarn run typesense:sync:preview - Sync preview environment
// yarn run typesense:sync:production - Sync production environment
// Uses typesense-sync package and typesense.config.json configuration
```

### Copy Code Functionality

```javascript
// assets/scripts/components/copy-code.js
const copyToClipboard = (text) => {
    const textarea = document.createElement('textarea');
    textarea.value = text;
    textarea.style.position = 'fixed';
    textarea.style.opacity = '0';
    document.body.appendChild(textarea);
    textarea.select();
    document.execCommand('copy');
    document.body.removeChild(textarea);
};

// Attach to all copy buttons
document.querySelectorAll('.js-copy-button').forEach((button) => {
    button.addEventListener('click', (e) => {
        const codeBlock = e.target.closest('.code-snippet').querySelector('code');
        const code = codeBlock.textContent;
        copyToClipboard(code);

        // Show feedback
        button.textContent = 'Copied!';
        setTimeout(() => {
            button.textContent = 'Copy';
        }, 2000);
    });
});
```

### Integration Filtering

```javascript
// assets/scripts/components/integrations.js
import mixitup from 'mixitup';

const initIntegrationFilters = () => {
    const container = document.querySelector('.integration-tiles');

    const mixer = mixitup(container, {
        selectors: {
            target: '.integration-tile'
        },
        animation: {
            duration: 300
        },
        callbacks: {
            onMixEnd: function(state) {
                console.log(`Showing ${state.totalShow} of ${state.totalTargets} integrations`);
            }
        }
    });

    // Category filters
    document.querySelectorAll('.filter-button').forEach((button) => {
        button.addEventListener('click', (e) => {
            const category = e.target.getAttribute('data-filter');
            mixer.filter(category);
        });
    });

    // Search functionality
    const searchInput = document.querySelector('.integration-search');
    searchInput.addEventListener('input', (e) => {
        const searchTerm = e.target.value.toLowerCase();
        mixer.filter((el) => {
            const title = el.querySelector('.integration-title').textContent.toLowerCase();
            return title.includes(searchTerm);
        });
    });
};

// Categories: .aws, .azure, .gcp, .marketplace, .ccm, .dsm, .dbm, .ndm, .llm
```

### Dynamic Table of Contents

```javascript
// assets/scripts/components/table-of-contents.js
const generateTOC = () => {
    const headers = document.querySelectorAll('h2, h3');
    const tocContainer = document.querySelector('#table-of-contents');

    headers.forEach((header) => {
        const id = header.getAttribute('id');
        const text = header.textContent;
        const level = header.tagName.toLowerCase();

        const link = document.createElement('a');
        link.href = `#${id}`;
        link.textContent = text;
        link.className = `toc-${level}`;

        tocContainer.appendChild(link);
    });

    // Highlight active section on scroll
    const observer = new IntersectionObserver((entries) => {
        entries.forEach((entry) => {
            if (entry.isIntersecting) {
                const id = entry.target.getAttribute('id');
                document.querySelectorAll('.toc a').forEach((link) => {
                    link.classList.remove('active');
                });
                document.querySelector(`.toc a[href="#${id}"]`)?.classList.add('active');
            }
        });
    }, { rootMargin: '-100px 0px -80% 0px' });

    headers.forEach((header) => observer.observe(header));
};
```

## Python Build Automation Scripts

### Integration Finder

```python
# local/bin/py/integration-finder.py
import os
import sys
import subprocess

INTEGRATION_REPOS = [
    'integrations-core',
    'integrations-extras',
    'integrations-internal-core',
    'marketplace'
]

def update_repos():
    """Clone or update integration repositories."""
    for repo in INTEGRATION_REPOS:
        repo_path = f'./integrations_data/{repo}'
        if os.path.exists(repo_path):
            print(f'Updating {repo}...')
            subprocess.run(['git', 'pull'], cwd=repo_path)
        else:
            print(f'Cloning {repo}...')
            url = f'https://github.com/DataDog/{repo}.git'
            subprocess.run(['git', 'clone', url, repo_path])

def find_integration(integration_name):
    """Find the source file for a specific integration."""
    for repo in INTEGRATION_REPOS:
        integration_path = f'./integrations_data/{repo}/{integration_name}'
        readme_path = f'{integration_path}/README.md'

        if os.path.exists(readme_path):
            print(f'Found: {readme_path}')
            return readme_path

    print(f'Integration "{integration_name}" not found in any repository.')
    return None

if __name__ == '__main__':
    if len(sys.argv) < 2:
        print('Usage: python integration-finder.py <integration_name>')
        sys.exit(1)

    integration_name = sys.argv[1]
    update_repos()
    find_integration(integration_name)

# Usage:
# make find-int int=postgres
# Output: Found: ./integrations_data/integrations-core/postgres/README.md
```

### Downloading Integration Data from S3

```python
# local/bin/py/build/get_websites_sources_data.py
import boto3
import os
import tarfile
import shutil

def download_from_s3():
    """Download pre-built integration data from S3."""
    s3_path = os.environ.get('FF_S3_PATH', 'staging')
    bucket = 'datadog-websites-sources'
    key = f'{s3_path}/data.tar.gz'

    print(f'Downloading from s3://{bucket}/{key}...')

    s3 = boto3.client('s3')
    local_file = '/tmp/websites_sources_data.tar.gz'

    s3.download_file(bucket, key, local_file)

    # Extract to _vendor directory
    print('Extracting to _vendor/...')
    with tarfile.open(local_file, 'r:gz') as tar:
        tar.extractall('./_vendor')

    os.remove(local_file)
    print('Done!')

if __name__ == '__main__':
    download_from_s3()

# Downloads:
# _vendor/content/en/integrations/
# _vendor/data/
# _vendor/layouts/
# _vendor/assets/
```

### Placeholder Translation Management

```python
# local/bin/py/placehold_translations.py
import os
import re
import shutil

def placehold_translations():
    """Replace translated content with placeholders during development."""
    translated_dirs = ['content/fr', 'content/ja', 'content/es', 'content/ko']

    for lang_dir in translated_dirs:
        if os.path.exists(lang_dir):
            print(f'Processing {lang_dir}...')

            for root, dirs, files in os.walk(lang_dir):
                for file in files:
                    if file.endswith('.md'):
                        file_path = os.path.join(root, file)

                        with open(file_path, 'r', encoding='utf-8') as f:
                            content = f.read()

                        # Replace body with placeholder
                        content = re.sub(
                            r'(---.*?---\n)(.*)',
                            r'\1[Translation placeholder]',
                            content,
                            flags=re.DOTALL
                        )

                        with open(file_path, 'w', encoding='utf-8') as f:
                            f.write(content)

    print('Translation placeholders created.')

# Reduces build time and repository size during development
```

## Configuration Files

### Hugo Configuration

```yaml
# config/_default/config.yaml
title: "Docs"
canonifyURLs: true
pygmentsCodeFences: true
enableRobotsTXT: true
publishDir: "./public"
defaultContentLanguage: "en"
enableGitInfo: true  # For sitemap lastmod based on git

# Module imports for shared components
module:
  imports:
   - path: github.com/DataDog/websites-modules
     mounts:
     - source: layouts/partials/nav/main-nav.html
       target: layouts/partials/nav/main-nav.html
     - source: data
       target: data

# Content mounts for multi-language support
mounts:
  - source: content/en
    target: content
    lang: en
  - source: content/fr
    target: content
    lang: fr
  - source: content/ja
    target: content
    lang: ja
  - source: _vendor/content/en
    target: content
    lang: en

# Output formats for search and redirects
outputFormats:
  Search:
    baseName: "search"
    isPlainText: true
    mediaType: "application/json"
  Redirects:
    baseName: "redirects"
    isPlainText: true
    mediaType: "text/plain"

# CDN cache control
deployment:
  matchers:
    - pattern: "^.+\\.(js|css|svg)$"
      cacheControl: "max-age=31536000, no-transform, public"
      gzip: true
    - pattern: "^.+\\.html$"
      cacheControl: "public, must-revalidate, proxy-revalidate, max-age=0"
      gzip: true

timeout: 300s
```

### Language and Region Configuration

```yaml
# config/_default/params.yaml
code_languages:
  py:
    name: "Python"
    extension: "py"
    syntax: "python"
    example_file: "examples/v2/monitors/CreateMonitor.py"
  rb:
    name: "Ruby"
    extension: "rb"
    syntax: "ruby"
    example_file: "examples/v2/monitors/CreateMonitor.rb"
  go:
    name: "Go"
    extension: "go"
    syntax: "go"
    example_file: "examples/v2/monitors/CreateMonitor.go"
  java:
    name: "Java"
    extension: "java"
    syntax: "java"
    example_file: "examples/v2/monitors/CreateMonitor.java"
  sh:
    name: "cURL"
    extension: "sh"
    syntax: "bash"

allowed_regions:
  - us      # datadoghq.com
  - us3     # us3.datadoghq.com
  - us5     # us5.datadoghq.com
  - eu      # datadoghq.eu
  - ap1     # ap1.datadoghq.com
  - gov     # ddog-gov.com
```

### Package Configuration

```json
// package.json
{
  "name": "docs",
  "version": "1.0.0",
  "description": "Datadog Docs built on HUGO",
  "scripts": {
    "start": "hugo server -D --port 1313 --navigateToChanged",
    "build": "yarn run prebuild && yarn run build:hugo",
    "build:hugo": "hugo -d ./public -s ./ --minify",
    "build:apiPages": "node -e 'require(\"./assets/scripts/build-api-pages.js\").init()'",
    "dereference": "node ./assets/scripts/dereference.js",
    "typesense:sync:preview": "NODE_ENV=preview node ./local/bin/js/typesense_sync.cjs",
    "typesense:sync:production": "NODE_ENV=production node ./local/bin/js/typesense_sync.cjs"
  },
  "engines": {
    "node": ">= 20.11.0"
  },
  "dependencies": {
    "@apidevtools/json-schema-ref-parser": "^8.0.0",
    "@datadog/browser-logs": "^5.27.0",
    "@datadog/browser-rum": "^5.27.0",
    "alpinejs": "^3.13.7",
    "bootstrap": "^5.2",
    "cdocs-hugo-integration": "https://s3.amazonaws.com/origin-static-assets/corp-node-packages/master/cdocs-hugo-integration-v2.2.1.tgz",
    "docs-llms-txt": "https://s3.amazonaws.com/origin-static-assets/corp-node-packages/master/docs-llms-txt-v1.0.0.tgz",
    "geo-locate": "https://s3.amazonaws.com/origin-static-assets/corp-node-packages/master/geo-locate-v1.0.2.tgz",
    "hugo-bin": "0.148.0",
    "instantsearch.js": "^4.74.1",
    "js-yaml": "^3.14.1",
    "marked": "^1.2.9",
    "mixitup": "^3.3.1",
    "signup-failover": "https://s3.amazonaws.com/origin-static-assets/corp-node-packages/master/signup-failover-v1.0.0.tgz",
    "slugify": "^1.6.6",
    "typesense": "^1.8.2",
    "typesense-instantsearch-adapter": "^2.8.0",
    "typesense-sync": "https://s3.amazonaws.com/origin-static-assets/corp-node-packages/master/typesense-sync-v1.1.0.tgz"
  },
  "hugo-bin": {
    "buildTags": "extended,withdeploy"
  },
  "packageManager": "yarn@4.10.3"
}
```

## Testing and Quality Assurance

### Running Tests

```bash
# Run Jest tests for JavaScript components
yarn run jest-test

# Test files in: assets/scripts/tests/
# - build-api-pages.test.js
# - copy-code.test.js
# - expression-language-evaluator.test.js
# - browser-detection.test.js
```

### Example Jest Test

```javascript
// assets/scripts/tests/copy-code.test.js
const { copyToClipboard } = require('../components/copy-code');

describe('Copy Code Functionality', () => {
    test('copies code to clipboard', () => {
        document.body.innerHTML = `
            <div class="code-snippet">
                <button class="js-copy-button">Copy</button>
                <code>console.log('test');</code>
            </div>
        `;

        const button = document.querySelector('.js-copy-button');
        button.click();

        expect(button.textContent).toBe('Copied!');
    });
});
```

### Link Validation

```yaml
# .htmltest.yml - HTML link checker configuration
DirectoryPath: "./public"
CheckDoctype: true
CheckExternal: true
IgnoreDirectoryMissingTrailingSlash: true
IgnoreURLs:
  - "https://github.com/DataDog/.*"
  - "https://app.datadoghq.com/.*"
ExternalTimeout: 30
```

## Git Workflow and Branch Previews

### Branch Naming Convention

```bash
# REQUIRED: Branch names must include forward slash
git checkout -b yourname/feature-description

# GitLab CI will:
# 1. Run build pipeline
# 2. Deploy to preview environment
# 3. Post preview URL in PR comments

# Preview URL format:
# https://docs-staging.datadoghq.com/yourname-feature-description/

# Invalid branch names (pipeline won't run):
git checkout -b feature-description  # ❌ No slash
git checkout -b yourname_feature     # ❌ Underscore instead of slash
```

### Pull Request Workflow

```bash
# Create feature branch
git checkout -b john/add-api-docs

# Make changes
vim content/en/api/custom-endpoint.md

# Commit changes
git add .
git commit -m "Add documentation for custom API endpoint"

# Push to GitHub
git push origin john/add-api-docs

# Create pull request
# GitHub bot will comment with preview URL after build completes:
# "Preview: https://docs-staging.datadoghq.com/john-add-api-docs/"

# Merge to master triggers production deployment
# Production URL: https://docs.datadoghq.com/
```

## Summary and Integration Patterns

The Datadog Documentation repository demonstrates a sophisticated approach to large-scale technical documentation management. Its primary use cases include generating API and reference documentation from OpenAPI specifications and schema files across six programming languages, managing integrations sourced from multiple external repositories, providing multi-language content delivery across five languages with region-specific customization, and enabling interactive documentation features through client-side JavaScript components powered by AlpineJS and Bootstrap 5. The build system orchestrates complex workflows including external content fetching, OpenAPI dereferencing, CDOCS component compilation, llms.txt generation for AI assistants, and automated Typesense search index synchronization.

Integration patterns emphasize distributed content management where documentation lives close to the code in external repositories (integrations-core, API client repos, websites-sources module) and is assembled during the build process using Yarn 4.10.3. Hugo's module system (version 0.148.0) imports shared components from websites-modules, while the _vendor directory holds dynamically fetched content from S3 buckets. The JavaScript layer provides enhanced interactivity including AlpineJS reactive components for state management, synchronized multi-language code tabs, Typesense-powered instant search with automated sync capabilities, integration filtering with mixitup, region-aware content display with geo-locate, and cookie banner management. Specialized build scripts handle both API documentation generation (build-api-pages.js) and reference documentation (build-reference-pages.js) from schema files containing functions, errors, and table structures. This architecture enables teams to maintain documentation alongside code while providing a unified, searchable, and highly interactive documentation experience across the entire Datadog platform. Branch-based preview deployments with customizable S3 paths, comprehensive testing, CDOCS integration for component documentation, and automated build pipelines with llms.txt support ensure documentation quality and rapid iteration while serving both human users and AI assistants.