### Install Dependencies Source: https://github.com/clerk/migration-tool/blob/main/prompts/migration-prompt.md Run this command to install project dependencies using Bun. This should be done before proceeding with any migration steps. ```bash bun install ``` -------------------------------- ### Run the Migration Tool Source: https://github.com/clerk/migration-tool/blob/main/README.md Execute the migration tool to start processing users. The tool automatically handles rate limits and logs errors. ```bash bun migrate ``` -------------------------------- ### Clone Repository and Install Dependencies Source: https://github.com/clerk/migration-tool/blob/main/README.md Clone the Clerk User Migration Tool repository and install its dependencies using Bun. ```bash git clone git@github.com:clerk/migration-tool cd migration-tool bun install ``` -------------------------------- ### Example CLI Commands for Validation Source: https://github.com/clerk/migration-tool/blob/main/prompts/transformer-prompt.md Use these commands to test the transformer registration and check for linting errors. The `bun migrate` command should list your newly added platform. ```bash # Run tests to verify transformer is properly registered bun run test # Check for lint errors bun lint # Test the migration CLI (should show your new platform in the list) bun migrate ``` -------------------------------- ### Install Dependencies and Configure Clerk Secret Key Source: https://context7.com/clerk/migration-tool/llms.txt Clone the repository, install dependencies using Bun, and configure your Clerk secret key either via a `.env` file, a CLI flag, or an environment variable. ```bash git clone git@github.com:clerk/migration-tool cd migration-tool bun install # Option 1: .env file (recommended) echo "CLERK_SECRET_KEY=sk_test_xxx" > .env # Option 2: CLI flag (no .env needed) bun migrate --clerk-secret-key sk_test_xxx # Option 3: environment variable export CLERK_SECRET_KEY=sk_test_xxx bun migrate ``` -------------------------------- ### Initiate Supabase Data Migration Source: https://github.com/clerk/migration-tool/blob/main/prompts/migration-prompt.md Command to start the migration process using the 'supabase' transformer and a specified JSON file. Requires a Clerk secret key, typically provided in a .env file. ```bash bun migrate --transformer supabase --file exports/users.json ``` -------------------------------- ### Basic Transformer Example Source: https://github.com/clerk/migration-tool/blob/main/docs/creating-transformers.md A simple transformer configuration for a fictional platform named 'My Platform'. It maps standard user fields and sets 'bcrypt' as the default password hasher. ```typescript // src/transformers/myplatform.ts const myPlatformTransformer = { key: 'myplatform', value: 'myplatform', label: 'My Platform', description: 'Use this transformer when migrating from My Platform. It handles standard user fields and bcrypt passwords.', transformer: { user_id: 'userId', email_address: 'email', first: 'firstName', last: 'lastName', phone_number: 'phone', hashed_password: 'password', }, defaults: { passwordHasher: 'bcrypt', }, }; export default myPlatformTransformer; ``` -------------------------------- ### Converted JSON Log Example Source: https://github.com/clerk/migration-tool/blob/main/README.md Example of the converted JSON array format, suitable for loading into spreadsheets or databases. ```json [ { "userId": "user_1", "status": "success", "clerkUserId": "clerk_abc123" }, { "userId": "user_2", "status": "error", "error": "Email already exists" }, { "userId": "user_3", "status": "fail", "error": "invalid_type for required field.", "path": ["email"], "row": 5 } ] ``` -------------------------------- ### NDJSON Log Example Source: https://github.com/clerk/migration-tool/blob/main/README.md Example of the NDJSON format used for log files, where each line is a valid JSON object. ```json {"userId":"user_1","status":"success","clerkUserId":"clerk_abc123"} {"userId":"user_2","status":"error","error":"Email already exists"} {"userId":"user_3","status":"fail","error":"invalid_type for required field.","path":["email"],"row":5} ``` -------------------------------- ### Example User Data for Transformation Source: https://github.com/clerk/migration-tool/blob/main/prompts/transformer-prompt.md Sample JSON structure representing user data that needs to be transformed for Clerk. Pay attention to fields like `email_confirmed` and `app_metadata` which require specific handling. ```json { "users": [ { "_id": { "$oid": "507f1f77bcf86cd799439011" }, "email": "user@example.com", "email_confirmed": false, "password_digest": "$2a$10$...", "full_name": "John Doe", "phone": "+1234567890", "created_at": "2024-01-15T10:30:00Z", "app_metadata": { "role": "admin" } } ] } ``` -------------------------------- ### Firebase preTransform Example Source: https://github.com/clerk/migration-tool/blob/main/docs/creating-transformers.md An example of a `preTransform` function tailored for Firebase exports. It handles CSV files by adding headers and JSON files by extracting user arrays from a wrapper object. ```typescript preTransform: (filePath: string, fileType: string): PreTransformResult => { if (fileType === 'text/csv') { // Firebase CSV exports don't have headers - create temp file with headers const originalContent = fs.readFileSync(filePath, 'utf-8'); const newFilePath = path.join('tmp', 'users-with-headers.csv'); fs.writeFileSync(newFilePath, `${CSV_HEADERS}\n${originalContent}`); return { filePath: newFilePath }; } if (fileType === 'application/json') { // Firebase JSON wraps users in { users: [...] } const parsed = JSON.parse(fs.readFileSync(filePath, 'utf-8')); if (parsed.users && Array.isArray(parsed.users)) { return { filePath, data: parsed.users }; } } return { filePath }; }; ``` -------------------------------- ### Advanced Transformer with Nested Fields Source: https://github.com/clerk/migration-tool/blob/main/docs/creating-transformers.md An example of an advanced transformer designed for platforms with nested user data structures. It demonstrates the use of dot notation to extract deeply nested fields. ```typescript const advancedTransformer = { key: 'advanced', value: 'advanced', label: 'Advanced Platform', description: 'Use this for platforms with nested user data structures. Supports dot notation for extracting nested fields.', transformer: { 'user._id.$oid': 'userId', 'profile.email': 'email', 'profile.name.first': 'firstName', 'profile.name.last': 'lastName', 'auth.passwordHash': 'password', 'metadata.public': 'publicMetadata', }, defaults: { passwordHasher: 'bcrypt', }, }; export default advancedTransformer; ``` -------------------------------- ### Data Flow in Migration Tool Source: https://github.com/clerk/migration-tool/blob/main/CODEX.md Illustrates the step-by-step process of user data transformation and import, from file loading to user creation. ```text User File (CSV/JSON) ↓ loadUsersFromFile (functions.ts) ↓ Run preTransform (if defined) ↓ Parse file ↓ Apply transformer defaults ↓ transformUsers (functions.ts) ↓ Transform field names via transformer ↓ Apply transformer postTransform ↓ Validate with Zod schema ↓ Log validation errors ↓ importUsers (import-users.ts) ↓ Process sequentially with rate limiting ↓ createUser (import-users.ts) ↓ Create user with primary email/phone ↓ Add additional emails/phones ↓ Handle errors and logging ``` -------------------------------- ### Run Migration Tool with Transformer and File Options Source: https://github.com/clerk/migration-tool/blob/main/README.md Execute the migration tool specifying a custom transformer and the path to the user data file. ```bash bun migrate -t clerk -f path/to/users.json ``` -------------------------------- ### Development Commands Source: https://github.com/clerk/migration-tool/blob/main/AGENTS.md Common commands for developing and managing the migration tool. ```bash bun migrate ``` ```bash bun delete ``` ```bash bun clean-logs ``` ```bash bun convert-logs ``` ```bash bun run test ``` ```bash bun lint ``` ```bash bun lint:fix ``` ```bash bun format ``` ```bash bun format:test ``` -------------------------------- ### Testing Transformer with CLI Commands Source: https://github.com/clerk/migration-tool/blob/main/docs/creating-transformers.md Commands to run the migration CLI, execute tests, and check for linting issues after creating a transformer. ```bash # Run the migration CLI with your new transformer bun migrate # Run tests to ensure validation still passes bun run test # Check for linting issues bun lint ``` -------------------------------- ### Development Commands Source: https://github.com/clerk/migration-tool/blob/main/CODEX.md Common CLI commands for developing and managing the migration tool, including running the migration, cleaning logs, and testing. ```bash bun migrate bun delete bun clean-logs bun convert-logs bun run test bun lint bun lint:fix bun format bun format:test ``` -------------------------------- ### Run Migration with Transformer Source: https://github.com/clerk/migration-tool/blob/main/prompts/migration-prompt.md Use this command to run the migration process with a specified transformer and file path. Ensure the transformer key and file path are correctly provided. ```bash bun migrate -y --transformer [transformer-key] --file [file-path] ``` -------------------------------- ### Project Structure Overview Source: https://github.com/clerk/migration-tool/blob/main/CODEX.md Provides a hierarchical view of the migration tool's source code directories and key files. ```tree src/ ├── clean-logs/ # Log cleanup utility ├── convert-logs/ # NDJSON to JSON converter ├── delete/ # User deletion functionality ├── migrate/ # Main migration logic │ ├── cli.ts # Interactive CLI │ ├── functions.ts # Data loading and transformation │ ├── import-users.ts # User creation with Clerk API │ ├── index.ts # Entry point │ └── validator.ts # Zod schema validation ├── transformers/ # Platform-specific transformers │ ├── auth0.ts │ ├── authjs.ts │ ├── clerk.ts │ ├── firebase.ts │ ├── supabase.ts │ └── index.ts ├── envs-constants.ts # Environment configuration ├── logger.ts # NDJSON logging ├── types.ts # TypeScript types └── utils.ts # Shared utilities ``` -------------------------------- ### Transformer System Overview Source: https://github.com/clerk/migration-tool/blob/main/CLAUDE.md Illustrates the data flow and components involved in the transformer system for migrating users. It shows how data moves from user files through transformation and validation to user import. ```mermaid graph TD A[User File (CSV/JSON)] --> B(loadUsersFromFile (functions.ts)) B --> C{Run preTransform (if defined)} C --> D[Parse file] D --> E[Apply transformer defaults] E --> F[transformUsers (functions.ts)] F --> G[Transform field names via transformer] G --> H[Apply transformer postTransform] H --> I[Validate with Zod schema] I --> J[Log validation errors] J --> K[importUsers (import-users.ts)] K --> L[Process sequentially with rate limiting] L --> M[createUser (import-users.ts)] M --> N[Create user with primary email/phone] N --> O[Add additional emails/phones] O --> P[Handle errors and logging] ``` -------------------------------- ### Run Interactive Migration CLI Source: https://context7.com/clerk/migration-tool/llms.txt Launches the interactive migration wizard. Use flags for non-interactive mode, specifying the transformer, file path, and optional resume point. The `--require-password` flag filters users, and specific flags are available for Firebase hash parameters. ```bash # Interactive mode — guided step-by-step bun migrate # Non-interactive mode — all options via flags bun migrate -y -t auth0 -f users.json # Non-interactive with inline secret key (no .env required) bun migrate -y -t clerk -f users.json --clerk-secret-key sk_test_xxx # Resume a migration that was interrupted at a known user ID bun migrate -y -t clerk -f users.json --resume-after="user_2abc123" # Migrate only users who have passwords bun migrate -y -t supabase -f users.json --require-password # Firebase migration with scrypt hash parameters bun migrate -y -t firebase -f users.csv \ --firebase-signer-key "jxspr8Ki0RYycVU8zykbdLGjFQ3McFUH..." \ --firebase-salt-separator "Bw==" \ --firebase-rounds 8 \ --firebase-mem-cost 14 ``` -------------------------------- ### Supabase Export with Custom Options Source: https://github.com/clerk/migration-tool/blob/main/docs/exporting-users.md Run the Supabase user export command, specifying a custom database connection string via the --db-url flag and an output file path using the --output flag. ```bash bun export:supabase --db-url "postgres://..." --output users.json ``` -------------------------------- ### Testing Commands Source: https://github.com/clerk/migration-tool/blob/main/AGENTS.md Commands for running tests, including specific files and watch mode. ```bash bun run test ``` ```bash bun run test ``` ```bash bun run test --watch ``` -------------------------------- ### Analyze NDJSON Logs with grep Source: https://github.com/clerk/migration-tool/blob/main/README.md Demonstrates how to use command-line tools like `grep` and `wc` to analyze NDJSON log files directly. ```bash # Count successful imports grep '"status":"success"' logs/migration-2026-01-27T12:00:00.log | wc -l # Find all errors grep '"status":"error"' logs/migration-2026-01-27T12:00:00.log # Get specific user grep '"userId":"user_123"' logs/migration-2026-01-27T12:00:00.log ``` -------------------------------- ### Run Supabase User Export Source: https://github.com/clerk/migration-tool/blob/main/docs/exporting-users.md Execute the Supabase user export command using Bun. If the SUPABASE_DB_URL environment variable is not set, you will be prompted to enter the connection string interactively. ```bash bun export:supabase ``` -------------------------------- ### Configure Migration Tool for Non-Interactive Mode Source: https://github.com/clerk/migration-tool/blob/main/README.md Set up the migration tool for non-interactive execution by providing the transformer, file path, and Clerk secret key. This is essential for automation and AI agent usage. ```bash bun migrate -y \ --transformer clerk \ --file users.json \ --clerk-secret-key sk_test_xxx ``` -------------------------------- ### Post-Change Workflow Source: https://github.com/clerk/migration-tool/blob/main/CODEX.md Essential commands to run after making code changes to ensure code quality and test coverage. ```bash bun lint:fix bun format bun run test ``` -------------------------------- ### Run Migration Tool with Clerk Secret Key Source: https://github.com/clerk/migration-tool/blob/main/README.md Specify the Clerk secret key directly via a command-line argument when running the migration tool in non-interactive mode. This avoids the need for a .env file. ```bash bun migrate -y -t clerk -f users.json --clerk-secret-key sk_test_xxx ``` -------------------------------- ### Testing Commands Source: https://github.com/clerk/migration-tool/blob/main/CODEX.md Commands for executing tests, including running all tests, specific files, or in watch mode. ```bash bun run test bun run test bun run test --watch ``` -------------------------------- ### Provide Clerk Secret Key via Command Line Source: https://github.com/clerk/migration-tool/blob/main/README.md Pass your Clerk secret key directly as a command-line argument. Useful for automation and AI agents. ```bash bun migrate --clerk-secret-key sk_test_xxx ``` -------------------------------- ### Run Migration Tool in Non-Interactive Mode Source: https://github.com/clerk/migration-tool/blob/main/README.md Use the --yes flag to run the migration tool in non-interactive mode, skipping all confirmation prompts. ```bash bun migrate -y ``` -------------------------------- ### Provide Clerk Secret Key via .env File Source: https://github.com/clerk/migration-tool/blob/main/README.md Store your Clerk secret key in a .env file for repeated use. This is the recommended method for authentication. ```bash CLERK_SECRET_KEY=your-secret-key ``` -------------------------------- ### Analyze JSON Logs in Python Source: https://github.com/clerk/migration-tool/blob/main/README.md Demonstrates loading and analyzing converted JSON log files using Python, including counting entries by status. ```python # Load in Python import json with open('logs/migration-2026-01-27T12:00:00.json') as f: logs = json.load(f) # Count by status from collections import Counter status_counts = Counter(entry['status'] for entry in logs) ``` -------------------------------- ### Migrate Users from Supabase Export Source: https://github.com/clerk/migration-tool/blob/main/docs/exporting-users.md After exporting users from Supabase, use this command to initiate the migration process, specifying the transformer and the exported file. ```bash bun migrate --transformer supabase --file supabase-export.json ``` -------------------------------- ### Test Transformer Registration Source: https://github.com/clerk/migration-tool/blob/main/prompts/migration-prompt.md Execute this command to run tests and verify that a custom transformer is properly registered within the project. This is a crucial step before running the migration with a new transformer. ```bash bun run test ``` -------------------------------- ### Analyze JSON Logs in Node.js Source: https://github.com/clerk/migration-tool/blob/main/README.md Shows how to load and analyze converted JSON log files using Node.js, including filtering and aggregation. ```javascript // Load in Node.js/JavaScript const logs = require('./logs/migration-2026-01-27T12:00:00.json'); // Filter successful imports const successful = logs.filter((entry) => entry.status === 'success'); // Count errors by type const errorCounts = logs .filter((entry) => entry.status === 'error') .reduce((acc, entry) => { acc[entry.error] = (acc[entry.error] || 0) + 1; return acc; }, {}); ``` -------------------------------- ### Supabase Pooler Connection String Source: https://github.com/clerk/migration-tool/blob/main/docs/exporting-users.md Use this connection string for pooler connections to your Supabase Postgres database. Ensure to replace placeholders with your actual project details. ```bash postgres://postgres.[REF]:[PASSWORD]@aws-0-[REGION].pooler.supabase.com:6543/postgres ``` -------------------------------- ### Set Supabase DB URL Environment Variable Source: https://github.com/clerk/migration-tool/blob/main/docs/exporting-users.md Configure the SUPABASE_DB_URL environment variable in your .env file to automatically provide the connection string for the export command, bypassing interactive prompts. ```bash SUPABASE_DB_URL=postgres://postgres.ref:password@aws-0-us-east-1.pooler.supabase.com:6543/postgres ``` -------------------------------- ### Environment Variables for Clerk Migration Tool Source: https://context7.com/clerk/migration-tool/llms.txt Configure the Clerk migration tool using these environment variables. CLERK_SECRET_KEY is required, while RATE_LIMIT, CONCURRENCY_LIMIT, CLERK_PUBLISHABLE_KEY, NEXT_PUBLIC_SUPABASE_URL, and NEXT_PUBLIC_SUPABASE_ANON_KEY are optional for specific functionalities. ```env CLERK_SECRET_KEY=sk_test_xxx # Required. Determines prod/dev instance type automatically. RATE_LIMIT=100 # Optional. Override requests/second (default: 100 prod, 10 dev). CONCURRENCY_LIMIT=9 # Optional. Override concurrent requests (default: ~95% of RATE_LIMIT). # Optional for automatic Clerk Dashboard cross-reference in interactive mode: CLERK_PUBLISHABLE_KEY=pk_test_xxx # Optional for Supabase OAuth provider checking: NEXT_PUBLIC_SUPABASE_URL=https://xxx.supabase.co NEXT_PUBLIC_SUPABASE_ANON_KEY=eyJ... ``` -------------------------------- ### Post-Change Checks Source: https://github.com/clerk/migration-tool/blob/main/AGENTS.md Essential commands to run after making any code changes to ensure code quality and test coverage. ```bash bun lint:fix ``` ```bash bun format ``` ```bash bun run test ``` -------------------------------- ### Resume a Failed Migration Source: https://github.com/clerk/migration-tool/blob/main/README.md Use this command to resume a migration that was previously interrupted or failed. Provide the migration run ID to continue from the last successful state. ```bash bun migrate -y -t clerk -f users.json -r user_abc123 ``` -------------------------------- ### Run Migration Tool Non-Interactively Source: https://github.com/clerk/migration-tool/blob/main/README.md Execute the migration tool in non-interactive mode, providing all necessary options as command-line arguments. This is useful for automation. ```bash bun migrate -y -t auth0 -f users.json ``` -------------------------------- ### AuthJS (Next-Auth) Field Mapping for Migration Source: https://github.com/clerk/migration-tool/blob/main/prompts/migration-prompt.md Maps fields from AuthJS (Next-Auth) to Clerk, assuming a basic SQL export format. Customize the transformer if your export format differs. ```text id → userId email → email (routed by email_verified) name → firstName + lastName (split) created_at → createdAt updated_at → updatedAt ``` -------------------------------- ### Load and Transform User Data from File Source: https://context7.com/clerk/migration-tool/llms.txt Reads user data from JSON or CSV files, applies transformations, and validates records. Use this as the main entry point for loading user data before importing. ```typescript import { loadUsersFromFile } from './src/migrate/functions'; // Load Auth0 export const { users, validationFailed } = await loadUsersFromFile('users.json', 'auth0'); console.log(`Loaded ${users.length} valid users, ${validationFailed} failed validation`); // users[0] shape after transformation: // { // userId: 'auth0|64c7f1a2b3d', // email: 'alice@example.com', // verified // firstName: 'Alice', // lastName: 'Smith', // password: '$2b$10$...', // passwordHasher: 'bcrypt', // privateMetadata: { plan: 'pro' }, // createdAt: '2023-08-01T12:00:00.000Z' // } // Load Supabase export (CSV) const { users: supaUsers } = await loadUsersFromFile('supabase-export.json', 'supabase'); // Supabase postTransform converts created_at to ISO 8601 and routes // unconfirmed emails to unverifiedEmailAddresses // Load Firebase CSV (preTransform adds headers automatically) const { users: fbUsers } = await loadUsersFromFile('firebase-users.csv', 'firebase'); // Firebase postTransform builds scrypt password string and splits displayName ``` -------------------------------- ### loadUsersFromFile Source: https://context7.com/clerk/migration-tool/llms.txt Loads and transforms user data from a specified file (JSON or CSV) using predefined transformers. It applies pre-transformations, default fields, field mapping, and post-transformations, then validates each record. Returns successfully validated users and a count of validation failures. ```APIDOC ## `loadUsersFromFile(file, key)` — Load and Transform User Data Main data loading entry point. Reads a JSON or CSV file, applies the named transformer's `preTransform`, default fields, field-name mapping, and `postTransform`, then validates every record with Zod. Returns only the users that passed validation along with a count of validation failures. ### Parameters - **file** (string) - Required - The path to the user data file (JSON or CSV). - **key** (string) - Required - The key identifying the transformer to use (e.g., 'auth0', 'supabase', 'firebase'). ### Returns An object containing: - **users** (array) - An array of user objects that passed validation after transformation. - **validationFailed** (number) - The count of user records that failed validation. ``` -------------------------------- ### Convert NDJSON Logs to JSON Arrays with `bun convert-logs` Source: https://context7.com/clerk/migration-tool/llms.txt Convert timestamped NDJSON log files in the `./logs/` directory to standard JSON arrays for easier analysis. This command is interactive, prompting for file selection and writing JSON files alongside the originals. ```bash # Interactive: lists .log files, prompts selection, writes .json files alongside originals bun convert-logs ``` ```json # Input (./logs/migration-2026-01-27T12-00-00.log): # {"userId":"user_1","status":"success","clerkUserId":"user_clerk_abc"} # {"userId":"user_2","status":"error","error":"Email already exists","code":"422"} # {"userId":"user_3","status":"fail","error":"invalid_type","path":["email"],"row":5} # Output (./logs/migration-2026-01-27T12-00-00.json): # [ # {"userId":"user_1","status":"success","clerkUserId":"user_clerk_abc"}, # {"userId":"user_2","status":"error","error":"Email already exists","code":"422"}, # {"userId":"user_3","status":"fail","error":"invalid_type","path":["email"],"row":5} # ] ``` -------------------------------- ### Supabase Direct Connection String Source: https://github.com/clerk/migration-tool/blob/main/docs/exporting-users.md This connection string is for direct connections to your Supabase Postgres database. It requires the IPv4 add-on to be enabled. ```bash postgresql://postgres:[PASSWORD]@db.[REF].supabase.co:5432/postgres ``` -------------------------------- ### Migrate from Firebase with Hash Configuration Source: https://github.com/clerk/migration-tool/blob/main/README.md Perform a migration from Firebase, providing specific hash configuration options such as signer key, salt separator, rounds, and memory cost. This is required for Firebase migrations. ```bash bun migrate -y -t firebase -f users.csv \ --firebase-signer-key "abc123..." \ --firebase-salt-separator "Bw==" \ --firebase-rounds 8 \ --firebase-mem-cost 14 ``` -------------------------------- ### Provide Clerk Secret Key via Environment Variable Source: https://github.com/clerk/migration-tool/blob/main/README.md Set the CLERK_SECRET_KEY environment variable before running the migration tool. ```bash export CLERK_SECRET_KEY=sk_test_xxx bun migrate ``` -------------------------------- ### Clerk Instance-to-Instance Field Mapping Source: https://github.com/clerk/migration-tool/blob/main/prompts/migration-prompt.md Maps fields from an existing Clerk instance to a new one during migration. Useful for direct instance-to-instance data transfers. ```text id → userId primary_email_address → email verified_email_addresses → emailAddresses unverified_email_addresses → unverifiedEmailAddresses first_name → firstName last_name → lastName password_digest → password password_hasher → passwordHasher primary_phone_number → phone verified_phone_numbers → phoneNumbers unverified_phone_numbers → unverifiedPhoneNumbers username → username totp_secret → totpSecret backup_codes_enabled → backupCodesEnabled backup_codes → backupCodes public_metadata → publicMetadata unsafe_metadata → unsafeMetadata private_metadata → privateMetadata ``` -------------------------------- ### Registering a Custom Transformer Source: https://github.com/clerk/migration-tool/blob/main/prompts/transformer-prompt.md After creating a custom transformer file, it must be imported and added to the `transformers` array in `src/transformers/index.ts` for the CLI commands to recognize it. ```typescript // Add import at the top import customTransformer from './custom'; // Add to the transformers array export const transformers = [ // ... existing transformers customTransformer, // ADD YOUR TRANSFORMER HERE ]; ``` -------------------------------- ### Delete Migrated Users with `bun delete` Source: https://context7.com/clerk/migration-tool/llms.txt Use the `bun delete` command to remove all users from your Clerk instance that were created by the migration tool. This command prompts for confirmation and logs results. Use with extreme caution on production instances. ```bash bun delete # Prompts for confirmation, then deletes all users where externalId is set. # Logs results to ./logs/{timestamp}-user-deletion.log (NDJSON format). # DO NOT run on a production instance that has pre-existing users. ``` -------------------------------- ### Analyze Migration Fields with `analyzeFields` Source: https://context7.com/clerk/migration-tool/llms.txt Use this function to examine user data for identifier coverage and optional field presence before migration. It generates a report to help configure Clerk settings. ```typescript import { analyzeFields } from './src/migrate/cli'; const users = [ { userId: '1', email: 'a@example.com', firstName: 'Alice', password: 'hash' }, { userId: '2', email: 'b@example.com', firstName: 'Bob' }, { userId: '3', phone: '+15559876543' }, ]; const analysis = analyzeFields(users); // { // totalUsers: 3, // identifiers: { // verifiedEmails: 2, // unverifiedEmails: 0, // verifiedPhones: 1, // unverifiedPhones: 0, // username: 0, // hasAnyIdentifier: 3 // all users have at least one identifier // }, // fieldCounts: { firstName: 2, password: 1 }, // presentOnAll: [], // presentOnSome: ['First Name', 'Password'] // } // Check how many users will be skipped const skippable = analysis.totalUsers - analysis.identifiers.hasAnyIdentifier; console.log(`${skippable} users have no identifier and will be skipped`); ``` -------------------------------- ### Transformer File Structure Source: https://github.com/clerk/migration-tool/blob/main/prompts/transformer-prompt.md This is the basic structure for a custom transformer file. You will need to fill in the specific mappings and potentially add preTransform or postTransform functions based on your data. ```typescript // src/transformers/[platform-name].ts const [platformName]Transformer = { key: '[platform-key]', value: '[platform-key]', label: '[Platform Name]', description: '[Description of what this transformer handles]', preTransform?: (filePath, fileType) => PreTransformResult, // if needed transformer: { // field mappings }, postTransform?: (user) => void, // if needed defaults?: { // default values }, }; export default [platformName]Transformer; ``` -------------------------------- ### Create Custom Platform Transformer Source: https://context7.com/clerk/migration-tool/llms.txt Add support for a new authentication platform by creating a transformer object and registering it. The transformer pattern is composable with optional hooks like `preTransform`, `postTransform`, and `defaults`. ```typescript // src/transformers/myplatform.ts import type { PreTransformResult } from '../types'; const myPlatformTransformer = { key: 'myplatform', label: 'My Platform', description: 'Migrates users from My Platform with bcrypt passwords.', // Required: map source field names → Clerk field names // Supports dot notation for nested objects: 'user._id.$oid': 'userId' transformer: { user_id: 'userId', email_address: 'email', email_verified: 'emailVerified', // temporary — cleaned in postTransform first: 'firstName', last: 'lastName', phone_number: 'phone', hashed_password: 'password', meta: 'publicMetadata', }, // Optional: runs before field mapping; useful for adding CSV headers // or extracting user arrays from JSON wrapper objects preTransform: (filePath: string, fileType: string): PreTransformResult => { if (fileType === 'application/json') { const parsed = JSON.parse(require('fs').readFileSync(filePath, 'utf-8')); if (parsed.data?.users) { return { filePath, data: parsed.data.users }; } } return { filePath }; }, // Optional: runs after field mapping; route verification, clean temp fields postTransform: (user: Record) => { const emailVerified = user.emailVerified as boolean | undefined; if (user.email && !emailVerified) { user.unverifiedEmailAddresses = user.email; delete user.email; } delete user.emailVerified; }, // Optional: applied to every user before validation defaults: { passwordHasher: 'bcrypt' as const, }, }; export default myPlatformTransformer; // src/transformers/index.ts — register the transformer: // import myPlatformTransformer from './myplatform'; // export const transformers = [...existingTransformers, myPlatformTransformer]; // The CLI will automatically include it in the platform selection menu. ``` -------------------------------- ### Convert NDJSON Logs to JSON Source: https://github.com/clerk/migration-tool/blob/main/README.md Initiates the utility to convert NDJSON log files to standard JSON array format for easier analysis. ```bash bun convert-logs ``` -------------------------------- ### Resume Migration After Specific User ID Source: https://github.com/clerk/migration-tool/blob/main/README.md If a migration is interrupted, resume the process from a specific user ID using the --resume-after flag. ```bash bun migrate --resume-after="user_xxx" ``` -------------------------------- ### Handle Promise Errors with `tryCatch` Source: https://context7.com/clerk/migration-tool/llms.txt This utility wraps promises to provide Go-style error handling, returning a tuple `[result, error]`. Use it to avoid repetitive try/catch blocks for asynchronous operations. ```typescript import { tryCatch } from './src/utils'; // Basic usage const [users, err] = await tryCatch(loadUsersFromFile('users.json', 'clerk')); if (err) { console.error('Failed to load file:', err.message); process.exit(1); } console.log(`Loaded ${users.length} users`); // Non-fatal additional operations const [, emailErr] = await tryCatch( clerk.emailAddresses.createEmailAddress({ userId: 'user_abc', emailAddress: 'secondary@example.com', primary: false, }) ); if (emailErr) { // Log but don't fail — additional emails are non-fatal console.warn('Could not add secondary email:', emailErr.message); } ``` -------------------------------- ### Analyze NDJSON Logs Directly Source: https://context7.com/clerk/migration-tool/llms.txt Analyze NDJSON log files directly using command-line tools like `grep` and `wc`, or by loading them into JavaScript for programmatic analysis. ```bash # Count successes grep '"status":"success"' logs/migration-*.log | wc -l # Find all errors grep '"status":"error"' logs/migration-2026-01-27T12-00-00.log ``` ```javascript # After conversion — JavaScript const logs = require('./logs/migration-2026-01-27T12-00-00.json'); const errors = logs.filter(e => e.status === 'error'); const errorCounts = errors.reduce((acc, e) => { acc[e.error] = (acc[e.error] || 0) + 1; return acc; }, {}); ``` -------------------------------- ### Apply Field Mapping with `transformKeys` Source: https://context7.com/clerk/migration-tool/llms.txt Maps source platform field names to Clerk schema field names using a transformer configuration. Handles nested objects and filters out empty/null values. Use this to standardize user data before importing. ```typescript import { transformKeys } from './src/utils'; import auth0Transformer from './src/transformers/auth0'; const auth0User = { user_id: 'auth0|64c7f1a2b3d', email: 'alice@example.com', email_verified: true, given_name: 'Alice', family_name: 'Smith', user_metadata: { theme: 'dark' }, app_metadata: { plan: 'pro' }, created_at: '2023-08-01T12:00:00.000Z', }; const transformed = transformKeys(auth0User, auth0Transformer); // { // userId: 'auth0|64c7f1a2b3d', // email: 'alice@example.com', // emailVerified: true, // firstName: 'Alice', // lastName: 'Smith', // publicMetadata: { theme: 'dark' }, // privateMetadata: { plan: 'pro' }, // createdAt: '2023-08-01T12:00:00.000Z' // } // Nested field example (dot-notation in transformer config) const nestedTransformer = { transformer: { '_id.$oid': 'userId', 'profile.email': 'email', }, }; const mongoUser = { _id: { $oid: 'abc123' }, profile: { email: 'bob@example.com' } }; transformKeys(mongoUser, nestedTransformer); // { userId: 'abc123', email: 'bob@example.com' } ``` -------------------------------- ### Clean Migration Logs Source: https://github.com/clerk/migration-tool/blob/main/README.md Removes all log files generated by the migration and deletion processes from the ./logs folder. ```bash bun clean-logs ``` -------------------------------- ### Import Users to Clerk API Source: https://context7.com/clerk/migration-tool/llms.txt Processes an array of validated users concurrently against the Clerk API. Handles rate limiting, retries, logging, and provides a summary. Call this after loading and validating users with `loadUsersFromFile`. ```typescript import { loadUsersFromFile } from './src/migrate/functions'; import { importUsers } from './src/migrate/import-users'; const { users, validationFailed } = await loadUsersFromFile('users.json', 'auth0'); // Import all users (including those without passwords) await importUsers(users, true, validationFailed); // Import only users with passwords const usersWithPasswords = users.filter(u => u.password); await importUsers(usersWithPasswords, false, validationFailed); // Console output during migration: // ◐ Migrating users: [47/200] (45 successful, 2 failed) // // ┌ Migration Summary ──────────────────────────────────┐ // │ Total users in file: 200 │ // │ Successfully imported: 198 │ // │ Failed with errors: 2 │ // │ │ // │ Error Breakdown: │ // │ • 2 users: That email address is already in use. │ // └─────────────────────────────────────────────────────┘ ``` -------------------------------- ### Set Custom Session Claims for Foreign Key Constraints Source: https://context7.com/clerk/migration-tool/llms.txt Preserve backward compatibility for foreign key constraints by setting custom session claims in the Clerk Dashboard. This ensures that migrated users retain their external IDs. ```json { "userId": "{{user.externalId || user.id}}" } ``` -------------------------------- ### Detect Clerk Instance Type Source: https://context7.com/clerk/migration-tool/llms.txt Use `detectInstanceType` to determine if a Clerk secret key is for a production or development instance. This helps in auto-configuring rate limits and concurrency. ```typescript import { detectInstanceType } from './src/envs-constants'; detectInstanceType('sk_live_abc123'); // 'prod' detectInstanceType('sk_test_xyz789'); // 'dev' // Auto-configured limits per instance type: // Production: RATE_LIMIT=100, CONCURRENCY_LIMIT=9 // Development: RATE_LIMIT=10, CONCURRENCY_LIMIT=1 // Override via .env: // RATE_LIMIT=50 // CONCURRENCY_LIMIT=5 ``` -------------------------------- ### importUsers Source: https://context7.com/clerk/migration-tool/llms.txt Imports an array of validated user objects into Clerk concurrently. This function manages API concurrency, retries on rate limiting (429 errors), logs migration progress and errors to NDJSON files, and provides a summary of the import process. It should be called after `loadUsersFromFile`. ```APIDOC ## `importUsers(users, skipPasswordRequirement, validationFailed)` — Import to Clerk Processes an array of validated users concurrently against the Clerk API. Manages a shared `p-limit` concurrency limiter, tracks progress, handles 429 retries (up to 5 times), writes NDJSON logs, and prints a summary. Call this after `loadUsersFromFile`. ### Parameters - **users** (array) - Required - An array of user objects to import. - **skipPasswordRequirement** (boolean) - Required - If true, allows importing users even if they do not have a password defined. If false, only users with passwords will be imported. - **validationFailed** (number) - Required - The count of validation failures from `loadUsersFromFile`, used for reporting. ### Behavior - Imports users concurrently. - Manages API rate limiting with retries (up to 5 times for 429 errors). - Writes NDJSON logs for migration events. - Prints a summary of the import process to the console. ``` -------------------------------- ### transformKeys Source: https://context7.com/clerk/migration-tool/llms.txt Applies field name mapping from a source platform's schema to Clerk's schema using a provided transformer configuration. It supports flattening nested objects using dot-notation and filters out null or empty string values. ```APIDOC ## `transformKeys(data, transformerConfig)` — Apply Field Mapping Maps source platform field names to Clerk schema field names using a transformer's `transformer` configuration object. Handles selective flattening of nested objects via dot-notation keys. Filters out empty strings and nulls. ### Parameters - **data** (object) - Required - The source data object to transform. - **transformerConfig** (object) - Required - An object containing the transformation configuration. Expected to have a `transformer` property which is an object mapping source keys to target keys. ### Returns An object with keys mapped according to the `transformerConfig`. ```