### Display pgedge-anonymizer Help Information Source: https://github.com/pgedge/pgedge-anonymizer/blob/main/docs/installation.md After installation, run this command to view the help information and available options for the pgedge-anonymizer binary. ```bash pgedge-anonymizer help ``` -------------------------------- ### Example pgEdge Anonymizer Configuration File Structure Source: https://context7.com/pgedge/pgedge-anonymizer/llms.txt Illustrates the basic structure of the `pgedge-anonymizer.yaml` configuration file, including database connection settings and placeholders for patterns and columns. ```yaml # pgedge-anonymizer.yaml # PostgreSQL connection settings database: host: localhost # Default: localhost port: 5432 # Default: 5432 database: myapp # Required (or set PGDATABASE) user: anonymizer # Required (or set PGUSER) password: "" # Optional; prompts if empty, or use PGPASSWORD sslmode: prefer # disable | prefer | require | verify-ca | verify-full # SSL certificate paths (only needed for verify-ca / verify-full) # database: ``` -------------------------------- ### Build Anonymizer from Source Source: https://github.com/pgedge/pgedge-anonymizer/blob/main/docs/installation.md Use the make build command to compile the pgedge-anonymizer binary. This command handles dependency installation and places the binary in the /bin directory. ```bash make build ``` -------------------------------- ### Run Anonymization with Custom Configuration File Source: https://context7.com/pgedge/pgedge-anonymizer/llms.txt Specify an alternative configuration file path for the anonymization process. This is useful for managing different anonymization setups. ```bash pgedge-anonymizer run --config /etc/myapp/anonymizer.yaml ``` -------------------------------- ### Build and Development Commands (Bash) Source: https://context7.com/pgedge/pgedge-anonymizer/llms.txt Clone the repository, build the pgedge-anonymizer binary using `make build`, and run tests or linting with `make test` or `make lint`. Verify the installation by running `./bin/pgedge-anonymizer version`. ```bash # Clone and build git clone https://github.com/pgedge/pgedge-anonymizer.git cd pgedge-anonymizer make build # Produces ./bin/pgedge-anonymizer # Development commands make test # Run test suite (requires PostgreSQL) make lint # Run golangci-lint make fmt # Format Go source # Verify installation ./bin/pgedge-anonymizer version # pgedge-anonymizer 1.0.0 (built unknown) ``` -------------------------------- ### Run pgEdge Anonymizer Source: https://github.com/pgedge/pgedge-anonymizer/blob/main/docs/usage.md Invoke `pgedge-anonymizer run` to start the anonymization process. Command-line options like `--config`, `--quiet`, `--host`, `--port`, `--database`, `--user`, `--password`, and `--sslmode` can override configuration file settings. Use `--config` to specify the path to the configuration file, defaulting to `pgedge-anonymizer.yaml`. ```bash pgedge-anonymizer run [flags] ``` -------------------------------- ### Override Database Settings with Command-Line Flags Source: https://github.com/pgedge/pgedge-anonymizer/blob/main/docs/configuration.md Command-line options for database settings override values set in the configuration file. This example shows how to override host, port, database name, user, password, and SSL mode. ```bash pgedge-anonymizer run \ --config config.yaml \ --host production-db.example.com \ --port 5433 \ --database myapp_staging \ --user admin \ --password secret \ --sslmode require \ --quiet ``` -------------------------------- ### pgEdge Anonymizer Configuration Example Source: https://github.com/pgedge/pgedge-anonymizer/blob/main/docs/quickstart.md Define database connection details and column anonymization patterns in this YAML configuration file. Specify the host, port, database name, user, and the columns to anonymize with their corresponding patterns. ```yaml database: host: localhost port: 5432 database: myapp user: anonymizer columns: - column: public.users.email pattern: EMAIL - column: public.users.phone pattern: US_PHONE - column: public.users.ssn pattern: US_SSN ``` -------------------------------- ### Configure Columns for Anonymization Source: https://context7.com/pgedge/pgedge-anonymizer/llms.txt Specify columns and their corresponding anonymization patterns in the configuration file. This example shows common PII fields like names, emails, and addresses. ```yaml columns: - column: public.users.first_name pattern: PERSON_FIRST_NAME - column: public.users.last_name pattern: PERSON_LAST_NAME - column: public.users.email pattern: EMAIL - column: public.users.phone pattern: US_PHONE - column: public.users.date_of_birth pattern: DOB_OVER_18 - column: public.users.ssn pattern: US_SSN - column: public.addresses.street_address pattern: ADDRESS - column: public.addresses.city pattern: CITY - column: public.addresses.postal_code pattern: US_ZIP - column: public.payments.card_number pattern: CREDIT_CARD - column: public.payments.card_expiry pattern: CREDIT_CARD_EXPIRY - column: public.payments.card_cvv pattern: CREDIT_CARD_CVV - column: public.support_tickets.description pattern: LOREMIPSUM - column: public.audit_log.ip_address pattern: IPV4_ADDRESS ``` -------------------------------- ### pgEdge Anonymizer Output Example Source: https://github.com/pgedge/pgedge-anonymizer/blob/main/docs/quickstart.md Observe the anonymization progress and statistics displayed by the pgEdge Anonymizer during execution. This output provides insights into processed rows, anonymized values, and overall performance. ```text Processing public.users.email (est. 50000 rows)... 10000 rows processed 20000 rows processed 30000 rows processed 40000 rows processed 50000 rows processed Completed: 50000 rows, 48234 values anonymized === Anonymization Statistics === Total columns processed: 1 Total rows processed: 50000 Total values anonymized: 48234 Total duration: 2.34s Throughput: 21367 rows/sec ``` -------------------------------- ### Define Custom Mask Patterns Source: https://github.com/pgedge/pgedge-anonymizer/blob/main/docs/custom_pattern.md Use the 'patterns' configuration to define custom formats for masking data. Each pattern includes a name, format string, and type. The 'note' field provides an example or description. ```yaml patterns: - name: PRODUCT_SKU format: "SKU-AA-####" type: mask note: Product SKU (SKU-AB-1234) - name: LICENSE_PLATE format: "AAA-####" type: mask note: License plate (ABC-1234) - name: EMPLOYEE_ID format: "EMP-AAA-###" type: mask note: Employee ID (EMP-ABC-123) - name: SERIAL_NUMBER format: "SN\-AAAA\-####\-XX" type: mask note: Serial number with escaped hyphens ``` -------------------------------- ### Anonymize UK Customer Data Source: https://github.com/pgedge/pgedge-anonymizer/blob/main/docs/patterns.md Example configuration for anonymizing various customer data fields using UK-specific patterns. Ensure the patterns match the data types in your columns. ```yaml columns: - column: customers.full_name pattern: UK_NAME - column: customers.phone pattern: UK_PHONE - column: customers.address pattern: UK_ADDRESS - column: customers.postcode pattern: UK_POSTCODE - column: customers.ni_number pattern: UK_NI ``` -------------------------------- ### Show pgEdge Anonymizer Usage Help Source: https://context7.com/pgedge/pgedge-anonymizer/llms.txt Display general usage help for the pgEdge Anonymizer or specific help for individual commands like `run` or `validate`. ```bash pgedge-anonymizer help ``` ```bash pgedge-anonymizer help run ``` ```bash pgedge-anonymizer help validate ``` -------------------------------- ### Specify Configuration File Path Source: https://github.com/pgedge/pgedge-anonymizer/blob/main/docs/configuration.md Use the --config option to specify an alternative path to the pgedge-anonymizer.yaml configuration file. ```bash pgedge-anonymizer run --config /path/to/config.yaml ``` -------------------------------- ### Configure Database Connection Properties Source: https://github.com/pgedge/pgedge-anonymizer/blob/main/docs/configuration.md Specify PostgreSQL connection settings within the 'database' section of your configuration file. Ensure all required fields are provided. ```yaml database: host: localhost port: 5432 database: myapp user: anonymizer password: "" sslmode: prefer ``` -------------------------------- ### Run pgEdge Anonymizer Command Source: https://github.com/pgedge/pgedge-anonymizer/blob/main/docs/quickstart.md Execute the pgEdge Anonymizer tool using the 'run' command after creating your configuration file. This command initiates the data anonymization process based on the specified settings. ```bash pgedge-anonymizer run ``` -------------------------------- ### Configure Pattern File Paths Source: https://github.com/pgedge/pgedge-anonymizer/blob/main/docs/configuration.md Define paths to default and user-defined pattern files in the 'patterns' section of the configuration. Set 'disable_defaults' to true to skip loading built-in patterns. ```yaml patterns: default_path: /etc/pgedge/pgedge-anonymizer-patterns.yaml user_path: "./my-patterns.yaml" disable_defaults: false ``` -------------------------------- ### Reference Custom Patterns in Configuration Source: https://github.com/pgedge/pgedge-anonymizer/blob/main/docs/custom_pattern.md Specify the path to your custom pattern file in the configuration. This can be done in the main configuration file or via the command line. ```yaml patterns: user_path: ./custom-patterns.yaml ``` -------------------------------- ### Run Go Linter for pgEdge Anonymizer Source: https://github.com/pgedge/pgedge-anonymizer/blob/main/README.md Applies the Go Linter to check code quality and style for the pgEdge Anonymizer project. ```bash make lint ``` -------------------------------- ### Run Anonymization with Runtime Database Overrides and Quiet Mode Source: https://context7.com/pgedge/pgedge-anonymizer/llms.txt Override database connection parameters at runtime and suppress verbose output for a cleaner execution. Useful for CI/CD or automated processes. ```bash pgedge-anonymizer run \ --host staging-db.internal \ --port 5432 \ --database myapp_clone \ --user anonymizer \ --password secret \ --sslmode require \ --quiet ``` -------------------------------- ### Reference Custom Patterns on Command Line Source: https://github.com/pgedge/pgedge-anonymizer/blob/main/docs/custom_pattern.md Provide the path to your custom pattern file using the --patterns flag when running the pgedge-anonymizer command. ```bash pgedge-anonymizer run --config config.yaml --patterns custom-patterns.yaml ``` -------------------------------- ### pgedge-anonymizer CLI Commands Source: https://context7.com/pgedge/pgedge-anonymizer/llms.txt These commands demonstrate how to validate the anonymizer configuration before running it and how to execute the anonymization process. The `--quiet` flag is useful for CI/CD pipelines. ```bash # Step 1: Validate before running pgedge-anonymizer validate --config pgedge-anonymizer.yaml ``` ```bash # Step 2: Run anonymization pgedge-anonymizer run --config pgedge-anonymizer.yaml ``` ```bash # Step 3: Run quietly for CI/CD pipelines pgedge-anonymizer run --config pgedge-anonymizer.yaml --quiet ``` -------------------------------- ### Configure SSL Certificate Properties Source: https://github.com/pgedge/pgedge-anonymizer/blob/main/docs/configuration.md If using an SSL mode that requires certificates, include supporting properties like sslcert, sslkey, and sslrootcert in the 'database' section. ```yaml database: sslmode: verify-full sslcert: /path/to/client-cert.pem sslkey: /path/to/client-key.pem sslrootcert: /path/to/ca-cert.pem ``` -------------------------------- ### Database Connection via Environment Variables Source: https://context7.com/pgedge/pgedge-anonymizer/llms.txt Use standard PostgreSQL environment variables for database connection details when they are not provided in the config file or CLI flags. Ensure these variables are exported before running the anonymizer. ```bash export PGHOST=db.internal export PGPORT=5432 export PGDATABASE=myapp_staging export PGUSER=anonymizer export PGPASSWORD=secret export PGSSLMODE=require pgedge-anonymizer run --config config.yaml ``` -------------------------------- ### Clone pgedge-anonymizer Repository Source: https://github.com/pgedge/pgedge-anonymizer/blob/main/docs/installation.md Clone the official pgedge-anonymizer repository to your local machine. This is the first step before building from source. ```bash git clone https://github.com/pgedge/pgedge-anonymizer.git cd pgedge-anonymizer ``` -------------------------------- ### Run pgEdge Anonymizer Test Suite Source: https://github.com/pgedge/pgedge-anonymizer/blob/main/README.md Executes the integrated test suite for the pgEdge Anonymizer. Requires PostgreSQL for integration tests. ```bash make test ``` -------------------------------- ### Print pgEdge Anonymizer Version Information Source: https://context7.com/pgedge/pgedge-anonymizer/llms.txt Display the current version of the pgEdge Anonymizer tool, including the build date and time. This is useful for troubleshooting and compatibility checks. ```bash pgedge-anonymizer version ``` -------------------------------- ### Run Anonymization with Custom Patterns File Source: https://context7.com/pgedge/pgedge-anonymizer/llms.txt Combine built-in anonymization patterns with custom patterns defined in a separate file. This allows for extended or specialized anonymization rules. ```bash pgedge-anonymizer run --config config.yaml --patterns ./my-patterns.yaml ``` -------------------------------- ### Complete pgEdge Anonymizer Configuration Source: https://github.com/pgedge/pgedge-anonymizer/blob/main/docs/sample_config.md This configuration file includes all sections for database connection, pattern loading, and column anonymization. Ensure sensitive fields like passwords are handled securely, either via environment variables (PGPASSWORD) or prompting. ```yaml # pgEdge Anonymizer Configuration # Database connection database: host: db.example.com port: 5432 database: production_copy user: anonymizer password: "" # Will prompt or use PGPASSWORD sslmode: require # Pattern configuration patterns: user_path: ./custom-patterns.yaml disable_defaults: false # Columns to anonymize columns: # Customer PII - column: public.customers.name pattern: PERSON_NAME - column: public.customers.email pattern: EMAIL - column: public.customers.phone pattern: US_PHONE - column: public.customers.dob pattern: DOB_OVER_18 # Employee data - column: hr.employees.ssn pattern: US_SSN - column: hr.employees.passport_number pattern: PASSPORT # UK-specific data - column: uk.customers.ni_number pattern: UK_NI - column: uk.customers.nhs_number pattern: UK_NHS # Notes and free text - column: public.support_tickets.description pattern: LOREMIPSUM ``` -------------------------------- ### Define Custom Patterns Source: https://github.com/pgedge/pgedge-anonymizer/blob/main/docs/configuration.md Define your own anonymization patterns in a YAML file. Reference this file in your main configuration. ```yaml patterns: - name: EMPLOYEE_ID replacement: "EMP-XXXXXX" note: "Internal employee identifiers" - name: CUSTOMER_REF replacement: "CUST-XXXXXXXX" note: "Customer reference numbers" ``` ```yaml patterns: user_path: ./my-patterns.yaml ``` -------------------------------- ### Validate Configuration with a Specific File Source: https://context7.com/pgedge/pgedge-anonymizer/llms.txt Validate a specific configuration file without executing the anonymization process. This is useful for testing configuration changes. ```bash pgedge-anonymizer validate --config /path/to/config.yaml ``` -------------------------------- ### Referencing Custom Patterns in Main Config (YAML) Source: https://context7.com/pgedge/pgedge-anonymizer/llms.txt Specify the path to your custom patterns file in the main configuration using `patterns.user_path`. Then, reference your custom pattern names in the `columns` section for specific fields. ```yaml # Reference in main config patterns: user_path: ./custom-patterns.yaml columns: - column: public.orders.order_number pattern: ORDER_NUMBER - column: public.employees.employee_id pattern: EMPLOYEE_ID - column: public.inventory.sku pattern: PRODUCT_SKU ``` -------------------------------- ### Run Anonymization Using Only Custom Patterns Source: https://context7.com/pgedge/pgedge-anonymizer/llms.txt Disable all built-in patterns and use only the custom patterns provided in the specified file. This offers maximum control over the anonymization process. ```bash pgedge-anonymizer run --patterns ./my-patterns.yaml --no-defaults ``` -------------------------------- ### Validate pgEdge Anonymizer Configuration Source: https://github.com/pgedge/pgedge-anonymizer/blob/main/docs/usage.md Use the `validate` keyword to check configuration file syntax, database connectivity, column existence, and pattern validity before running the anonymizer. Include the `--config` flag to specify the configuration file path and any necessary database connection flags. ```bash pgedge-anonymizer validate [flags] ``` -------------------------------- ### Define Custom Number Patterns Source: https://github.com/pgedge/pgedge-anonymizer/blob/main/docs/custom_pattern.md Define custom number patterns using printf-style format codes. Specify a name, format, type, and optional min/max values. ```yaml patterns: - name: ORDER_NUMBER format: "ORD-%08d" type: number min: 1 max: 99999999 note: Order number (ORD-00000001) ``` ```yaml - name: INVOICE_NUMBER format: "INV-%06d" type: number min: 100000 max: 999999 note: Invoice number (INV-100000) ``` ```yaml - name: ACCOUNT_ID format: "%010d" type: number min: 1000000000 max: 9999999999 note: 10-digit account ID ```