### Initial Setup and Dependencies Source: https://github.com/post-crawl/node-sdk/blob/main/DEVELOPMENT.md This snippet covers cloning the Node.js SDK repository, installing dependencies using Bun, and setting up a local environment variable for API key testing. ```bash # Clone the repository git clone https://github.com/post-crawl/node-sdk.git cd node-sdk # Install dependencies bun install # Create .env file for testing echo "POSTCRAWL_API_KEY=sk_your_test_key_here" > .env ``` -------------------------------- ### Node.js SDK: Run All Examples Source: https://github.com/post-crawl/node-sdk/blob/main/examples/README.md A command to execute all available examples in sequence using the PostCrawl Node.js SDK. ```bash bun run examples ``` -------------------------------- ### PostCrawl SDK Development Setup with Bun Source: https://github.com/post-crawl/node-sdk/blob/main/README.md Outlines the steps for setting up the PostCrawl Node.js SDK development environment using Bun. It includes cloning the repository, installing dependencies, running tests, performing checks, and building the package. ```bash # Clone the repository git clone https://github.com/post-crawl/node-sdk.git cd node-sdk # Install dependencies bun install # Run tests make test # Run all checks (format, lint, test) make check # Build the package make build ``` -------------------------------- ### Testing Against Local API Source: https://github.com/post-crawl/node-sdk/blob/main/DEVELOPMENT.md Provides an example of how to instantiate the PostCrawlClient to connect to a local API instance by specifying the baseUrl. ```typescript const pc = new PostCrawlClient({ apiKey: 'sk_test', baseUrl: 'http://localhost:8787' }); ``` -------------------------------- ### Run PostCrawl examples with bun Source: https://github.com/post-crawl/node-sdk/blob/main/README.md Provides instructions on how to run the provided examples for the PostCrawl SDK using the bun runtime. ```bash # Using bun (recommended) bun run examples # Or run individual examples bun run example:search bun run example:extract bun run example:sne ``` -------------------------------- ### Run PostCrawl examples with Node.js and tsx Source: https://github.com/post-crawl/node-sdk/blob/main/README.md Shows how to execute the PostCrawl SDK examples using Node.js and the tsx command-line tool. ```bash # Using Node.js with tsx npx tsx examples/search_101.ts ``` -------------------------------- ### Install PostCrawl SDK using bun Source: https://github.com/post-crawl/node-sdk/blob/main/README.md Installs the PostCrawl SDK using the bun package manager. ```bash bun add postcrawl ``` -------------------------------- ### Install PostCrawl SDK using pnpm Source: https://github.com/post-crawl/node-sdk/blob/main/README.md Installs the PostCrawl SDK using the pnpm package manager. ```bash pnpm add postcrawl ``` -------------------------------- ### Install PostCrawl SDK using yarn Source: https://github.com/post-crawl/node-sdk/blob/main/README.md Installs the PostCrawl SDK using the yarn package manager. ```bash yarn add postcrawl ``` -------------------------------- ### Install PostCrawl SDK using npm Source: https://github.com/post-crawl/node-sdk/blob/main/README.md Installs the PostCrawl SDK using the npm package manager. ```bash npm install postcrawl ``` -------------------------------- ### Publishing to npm Source: https://github.com/post-crawl/node-sdk/blob/main/DEVELOPMENT.md Outlines the process for publishing the Node.js SDK package to npm, including a dry run to verify the contents before the actual release. ```bash # Dry run to check what will be published npm publish --dry-run # Publish to npm npm publish ``` -------------------------------- ### Install dotenv for environment variables Source: https://github.com/post-crawl/node-sdk/blob/main/README.md Installs the dotenv package to load API keys from .env files. ```bash npm install dotenv # or bun add dotenv ``` -------------------------------- ### Development Commands with Bun Source: https://github.com/post-crawl/node-sdk/blob/main/DEVELOPMENT.md Provides essential commands for running tests, type checking, linting, formatting, building, and development watch mode for the Node.js SDK using Bun. ```bash # Run tests in watch mode bun run test:watch # Run tests once bun run test # Run tests with coverage bun run test:coverage # Type checking bun run typecheck # Linting and formatting bun run check:biome # Full quality check bun run check # Build the package bun run build # Watch mode for development bun run dev ``` -------------------------------- ### Node.js SDK: Basic Search Example Source: https://github.com/post-crawl/node-sdk/blob/main/examples/README.md Demonstrates how to perform a basic search across multiple platforms like Reddit and TikTok using the PostCrawl Node.js SDK. It shows how to handle search results and access rate limit information. ```bash bun run example:search ``` -------------------------------- ### Development Shortcuts with Make Source: https://github.com/post-crawl/node-sdk/blob/main/DEVELOPMENT.md This section details Makefile commands for managing the Node.js SDK development workflow, including running tests, formatting code, performing checks, cleaning artifacts, and building the distribution. ```makefile # Show all available commands make help # Run all tests make test # Run specific test file make test-types make test-client # Format code make format # Run all checks make check # Clean build artifacts make clean # Build distribution make build ``` -------------------------------- ### Contributing Workflow Source: https://github.com/post-crawl/node-sdk/blob/main/DEVELOPMENT.md Details the steps for contributing to the Node.js SDK, including forking the repository, creating feature branches, making changes, running tests and checks, committing, and submitting a pull request. ```bash 1. Fork the repository 2. Create a feature branch: git checkout -b feature-name 3. Make your changes 4. Run tests: bun run test 5. Run checks: bun run check 6. Commit with clear message 7. Push and create Pull Request ``` -------------------------------- ### Node.js SDK: Content Extraction Example Source: https://github.com/post-crawl/node-sdk/blob/main/examples/README.md Illustrates how to extract full content from social media URLs using the PostCrawl Node.js SDK. This example covers extracting from multiple URLs, including comments, handling platform-specific data, and managing errors for failed extractions. ```bash bun run example:extract ``` -------------------------------- ### Enabling Debug Logging Source: https://github.com/post-crawl/node-sdk/blob/main/DEVELOPMENT.md Illustrates how to enable debug logging for the PostCrawl SDK by setting the DEBUG environment variable when running tests. ```bash DEBUG=postcrawl:* bun run test ``` -------------------------------- ### Node.js SDK: Combined Search and Extract Example Source: https://github.com/post-crawl/node-sdk/blob/main/examples/README.md Shows how to perform a combined search and extraction operation in a single API call with the PostCrawl Node.js SDK. It highlights features like markdown response mode, credit usage estimation, and processing extracted content. ```bash bun run example:sne ``` -------------------------------- ### PostCrawl SDK Available Development Commands Source: https://github.com/post-crawl/node-sdk/blob/main/README.md Lists the available `make` commands for managing the PostCrawl Node.js SDK development workflow. These commands cover tasks such as viewing help, formatting code, linting, testing, running all checks, building the package, and executing examples. ```bash make help # Show all available commands make format # Format code with biome make lint # Run linting and type checking make test # Run test suite make check # Run format, lint, and tests make build # Build distribution packages make examples # Run all examples ``` -------------------------------- ### Testing with Vitest Source: https://github.com/post-crawl/node-sdk/blob/main/DEVELOPMENT.md Demonstrates how to write unit tests using Vitest for the Node.js SDK. It includes a basic test structure with describe, it, and expect for arranging, acting, and asserting test outcomes. ```typescript import { describe, it, expect } from 'vitest'; describe('Feature', () => { it('should do something', () => { // Arrange const input = 'test'; // Act const result = someFunction(input); // Assert expect(result).toBe('expected'); }); }); ``` -------------------------------- ### Troubleshooting TypeScript Errors Source: https://github.com/post-crawl/node-sdk/blob/main/DEVELOPMENT.md A sequence of commands to resolve TypeScript errors, typically by clearing the cache, reinstalling dependencies, and rebuilding the project. ```bash # Clear TypeScript cache rm -rf node_modules/.cache rm -rf *.tsbuildinfo # Reinstall dependencies bun install # Rebuild bun run build ``` -------------------------------- ### Check and Update Node.js Dependencies Source: https://github.com/post-crawl/node-sdk/blob/main/DEVELOPMENT.md Commands to check for outdated Node.js dependencies and update them using npm-check-updates and bun update. This is crucial for maintaining the SDK's health and security. ```bash # Check for updates bunx npm-check-updates # Update dependencies bun update ``` -------------------------------- ### Type Generation Command Source: https://github.com/post-crawl/node-sdk/blob/main/DEVELOPMENT.md Shows the command to regenerate TypeScript types from the PostCrawl OpenAPI specification using a Makefile target. It emphasizes not to manually edit the generated types file. ```bash # Regenerate types from OpenAPI make generate-types ``` -------------------------------- ### PostCrawl search and extract usage Source: https://github.com/post-crawl/node-sdk/blob/main/README.md Demonstrates how to perform a combined search and extract operation using the PostCrawl SDK. This example shows how to specify search parameters, include comments, and set the response mode. ```typescript const posts = await pc.searchAndExtract({ socialPlatforms: ['reddit'], query: 'search query', results: 5, page: 1, includeComments: false, responseMode: 'markdown' }); ``` -------------------------------- ### Handle PostCrawl SDK Errors Source: https://github.com/post-crawl/node-sdk/blob/main/README.md Provides an example of how to catch and handle various PostCrawl SDK errors, including `AuthenticationError`, `InsufficientCreditsError`, `RateLimitError`, and `ValidationError`. It demonstrates checking the error type and logging specific details for each error. ```typescript import { AuthenticationError, InsufficientCreditsError, RateLimitError, ValidationError } from 'postcrawl'; try { const results = await pc.search({ ... }); } catch (error) { if (error instanceof AuthenticationError) { console.error('Invalid API key'); } else if (error instanceof InsufficientCreditsError) { console.error('Insufficient credits:', error.requiredCredits); } else if (error instanceof RateLimitError) { console.error(`Rate limited. Retry after ${error.retryAfter}s`); } else if (error instanceof ValidationError) { console.error('Validation errors:', error.details); } } ``` -------------------------------- ### PostCrawl Client Initialization Source: https://github.com/post-crawl/node-sdk/blob/main/README.md Illustrates the parameters for initializing the PostCrawl client, including the required API key and optional settings for timeout, retries, and retry delay. ```typescript const pc = new PostCrawlClient({ apiKey: string, // Required: Your PostCrawl API key (starts with 'sk_') timeout?: number, // Optional: Request timeout in ms (default: 90000) maxRetries?: number, // Optional: Max retry attempts (default: 3) retryDelay?: number, // Optional: Delay between retries in ms (default: 1000) }) ``` -------------------------------- ### Basic PostCrawl search usage Source: https://github.com/post-crawl/node-sdk/blob/main/README.md Demonstrates how to initialize the PostCrawl client and perform a basic search for content on Reddit. It shows how to process the search results, logging the title, URL, date, and a snippet of each post. ```typescript import { PostCrawlClient } from 'postcrawl'; // Initialize the client with your API key const pc = new PostCrawlClient({ apiKey: 'sk_your_api_key_here' }); // Search for content const results = await pc.search({ socialPlatforms: ['reddit'], query: 'machine learning', results: 10, page: 1 }); // Process results for (const post of results) { console.log(`${post.title} - ${post.url}`); console.log(` Date: ${post.date}`); console.log(` Snippet: ${post.snippet.substring(0, 100)}...`); } ``` -------------------------------- ### Manage PostCrawl API Key with Environment Variables Source: https://github.com/post-crawl/node-sdk/blob/main/README.md Explains how to manage the PostCrawl API key using environment variables for secure storage. It shows how to export the key directly or use a `.env` file and then load it into the PostCrawl client in TypeScript code. ```bash # Example for bash/zsh export POSTCRAWL_API_KEY="sk_your_api_key_here" # Example for .env file # .env # POSTCRAWL_API_KEY=sk_your_api_key_here ``` ```typescript import { config } from 'dotenv'; import { PostCrawlClient } from 'postcrawl'; config(); const pc = new PostCrawlClient({ apiKey: process.env.POSTCRAWL_API_KEY! }); ``` -------------------------------- ### PostCrawl Search API Parameters Source: https://github.com/post-crawl/node-sdk/blob/main/README.md Details the parameters for the PostCrawl search API, including social platforms, query, number of results, and pagination. ```typescript const results = await pc.search({ socialPlatforms: ['reddit', 'tiktok'], query: 'your search query', results: 10, // 1-100 page: 1 // pagination }); ``` -------------------------------- ### PostCrawl Extract API Parameters Source: https://github.com/post-crawl/node-sdk/blob/main/README.md Outlines the parameters for the PostCrawl extract API, specifying the URLs to extract from, whether to include comments, and the desired response mode. ```typescript const posts = await pc.extract({ urls: ['https://reddit.com/...', 'https://tiktok.com/...'], includeComments: true, responseMode: 'raw' // or 'markdown' }); ``` -------------------------------- ### PostCrawl extract content usage Source: https://github.com/post-crawl/node-sdk/blob/main/README.md Shows how to use the PostCrawl SDK to extract content from a list of social media URLs, including an option to include comments and specify the response mode. It also includes error handling for failed extractions. ```typescript // Extract content from URLs const posts = await pc.extract({ urls: [ 'https://reddit.com/r/...', 'https://tiktok.com/@...' ], includeComments: true, responseMode: 'raw' }); // Process extracted posts for (const post of posts) { if (post.error) { console.error(`Failed to extract ${post.url}: ${post.error}`); } else { console.log(`Extracted from ${post.source}: ${post.url}`); } } ``` -------------------------------- ### Retrieve PostCrawl Rate Limit Information Source: https://github.com/post-crawl/node-sdk/blob/main/README.md Demonstrates how to access and log the rate limit information provided by the PostCrawl client after making an API request. It shows how to access properties like `limit`, `remaining`, and `reset` from the `rateLimitInfo` object. ```typescript const pc = new PostCrawlClient({apiKey: 'sk_...'}); const results = await pc.search({ ... }); console.log(`Rate limit: ${pc.rateLimitInfo.limit}`); console.log(`Remaining: ${pc.rateLimitInfo.remaining}`); console.log(`Reset at: ${pc.rateLimitInfo.reset}`); ``` -------------------------------- ### PostCrawl Search and Extract API Parameters Source: https://github.com/post-crawl/node-sdk/blob/main/README.md Specifies the parameters for the combined search and extract API operation, including social platforms, query, results, page, include comments, and response mode. ```typescript const posts = await pc.searchAndExtract({ socialPlatforms: ['reddit'], query: 'search query', results: 5, page: 1, includeComments: false, responseMode: 'markdown' }); ``` -------------------------------- ### Access Platform-Specific Data with PostCrawl SDK Source: https://github.com/post-crawl/node-sdk/blob/main/README.md Demonstrates how to extract data using the PostCrawl SDK and conditionally access platform-specific fields for Reddit and TikTok posts. It utilizes type guards like `isRedditPost` and `isTiktokPost` for safe data access and handles potential errors during extraction. ```typescript import { PostCrawlClient, isRedditPost, isTiktokPost } from 'postcrawl'; // Extract content with proper type handling const posts = await pc.extract({ urls: ['https://reddit.com/...'] }); for (const post of posts) { if (post.error) { console.error(`Error: ${post.error}`); } else if (isRedditPost(post.raw)) { // Access Reddit-specific fields with camelCase console.log(`Subreddit: r/${post.raw.subredditName}`); console.log(`Score: ${post.raw.score}`); console.log(`Title: ${post.raw.title}`); console.log(`Upvotes: ${post.raw.upvotes}`); console.log(`Created: ${post.raw.createdAt}`); if (post.raw.comments) { console.log(`Comments: ${post.raw.comments.length}`); } } else if (isTiktokPost(post.raw)) { // Access TikTok-specific fields with camelCase console.log(`Username: @${post.raw.username}`); console.log(`Likes: ${post.raw.likes}`); console.log(`Total Comments: ${post.raw.totalComments}`); console.log(`Created: ${post.raw.createdAt}`); if (post.raw.hashtags) { console.log(`Hashtags: ${post.raw.hashtags.join(', ')}`); } } } ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.