### Quick Setup for PostCrawl Python SDK Source: https://github.com/post-crawl/python-sdk/blob/main/DEVELOPMENT.md Clones the PostCrawl Python SDK repository, navigates into the directory, installs all dependencies using uv sync, and runs tests to verify the setup. ```bash git clone https://github.com/post-crawl/python-sdk.git cd python-sdk uv sync make test ``` -------------------------------- ### Install PostCrawl SDK Source: https://github.com/post-crawl/python-sdk/blob/main/examples/README.md Installs the PostCrawl Python SDK using pip or uv. ```bash pip install postcrawl # or with uv: uv add postcrawl ``` -------------------------------- ### Building and Publishing the SDK Source: https://github.com/post-crawl/python-sdk/blob/main/DEVELOPMENT.md Commands for building distribution packages, verifying the package installation, and publishing the SDK to TestPyPI for testing or to PyPI for production release. ```bash # Build distribution packages make build # Verify package (builds and tests installation) make verify # Publish to TestPyPI for testing make publish-test # Publish to PyPI (production) make publish ``` -------------------------------- ### Running SDK Examples Source: https://github.com/post-crawl/python-sdk/blob/main/DEVELOPMENT.md Executes example scripts to test the functionality of the PostCrawl SDK, specifically for search, extract, and combined search and extract operations. ```bash make search # Run search example make extract # Run extract example make sne # Run search & extract example ``` -------------------------------- ### Run Example with .env File Source: https://github.com/post-crawl/python-sdk/blob/main/examples/README.md Demonstrates running a PostCrawl SDK example using a .env file for API key configuration. ```bash python search_101.py ``` -------------------------------- ### Install uv Package Manager Source: https://github.com/post-crawl/python-sdk/blob/main/DEVELOPMENT.md Installs the uv package manager, a fast Python package manager, using a curl script. ```bash curl -LsSf https://astral.sh/uv/install.sh | sh ``` -------------------------------- ### Makefile Commands for Development Source: https://github.com/post-crawl/python-sdk/blob/main/DEVELOPMENT.md Provides a comprehensive list of Makefile commands for common development tasks in the PostCrawl Python SDK, including running development servers, tests, formatting, linting, type checking, and executing examples. ```bash # Development commands make dev # Run development server (if applicable) make test # Run all tests make format # Format code with black and ruff make lint # Run linting checks make typecheck # Run mypy type checking make check # Run all checks (format, lint, tests) # Example commands make examples # Run all examples make search # Run search example make extract # Run extract example make sne # Run search and extract example # Build and release make build # Build distribution packages make verify # Verify package installation make clean # Clean build artifacts make publish-test # Publish to TestPyPI (requires ~/.pypirc) make publish # Publish to PyPI (requires ~/.pypirc) # Type generation make generate-types # Regenerate types from TypeScript schemas ``` -------------------------------- ### Run Example Scripts Source: https://github.com/post-crawl/python-sdk/blob/main/README.md Provides instructions on how to run example Python scripts for the PostCrawl SDK using either uv or the standard Python interpreter. ```bash # Using uv (recommended) uv run python examples/search_101.py # Or with standard Python cd examples python search_101.py ``` -------------------------------- ### Run Search Example Source: https://github.com/post-crawl/python-sdk/blob/main/examples/README.md Executes the search_101.py script to search content across Reddit and TikTok, demonstrating pagination and result parsing. ```bash python search_101.py ``` -------------------------------- ### Run Search and Extract Combined Example Source: https://github.com/post-crawl/python-sdk/blob/main/examples/README.md Executes search_and_extract_101.py to perform combined search and extraction operations with markdown formatting. ```bash python search_and_extract_101.py ``` -------------------------------- ### Install PostCrawl SDK with uv Source: https://github.com/post-crawl/python-sdk/blob/main/README.md Installs the PostCrawl SDK using the uv package manager. It first ensures uv is installed and then adds the postcrawl package to the project. ```bash # Install uv if you haven't already curl -LsSf https://astral.sh/uv/install.sh | sh # Add postcrawl to your project uv add postcrawl ``` -------------------------------- ### Running Tests with Pytest Source: https://github.com/post-crawl/python-sdk/blob/main/DEVELOPMENT.md Demonstrates how to run tests using pytest, including running all tests, all checks, specific test files, and tests with code coverage. ```bash # Run all tests make test # Run all checks (format, lint, tests) make check # Run specific test file uv run pytest tests/test_client.py # Run with coverage uv run pytest --cov=postcrawl ``` -------------------------------- ### Manage Python Versions (Bash) Source: https://github.com/post-crawl/python-sdk/blob/main/DEVELOPMENT.md Provides commands to check available Python versions and install a specific version using 'uv'. This is crucial for managing Python environment compatibility. ```bash # Check available Python versions uv python list # Install specific version uv python install 3.12 ``` -------------------------------- ### Managing Dependencies with uv Source: https://github.com/post-crawl/python-sdk/blob/main/DEVELOPMENT.md Shows how to manage project dependencies using the uv package manager, including adding new packages, adding development dependencies, and updating all dependencies. ```bash # Add a dependency uv add package-name # Add a dev dependency uv add --dev package-name # Update all dependencies uv sync --upgrade ``` -------------------------------- ### Install python-dotenv with uv Source: https://github.com/post-crawl/python-sdk/blob/main/README.md Installs the python-dotenv package using uv for managing environment variables. ```bash uv add python-dotenv ``` -------------------------------- ### Reset Virtual Environment (Bash) Source: https://github.com/post-crawl/python-sdk/blob/main/DEVELOPMENT.md Resets the virtual environment by removing the existing '.venv' directory and then synchronizing dependencies using 'uv'. This is useful for starting with a clean environment. ```bash rm -rf .venv uv sync ``` -------------------------------- ### Install python-dotenv with pip Source: https://github.com/post-crawl/python-sdk/blob/main/README.md Installs the python-dotenv package using pip for managing environment variables. ```bash pip install python-dotenv ``` -------------------------------- ### Run Extract Example Source: https://github.com/post-crawl/python-sdk/blob/main/examples/README.md Executes the extract_101.py script to extract full content from social media URLs, handling platform-specific data. ```bash python extract_101.py ``` -------------------------------- ### Install PostCrawl SDK with pip Source: https://github.com/post-crawl/python-sdk/blob/main/README.md Installs the PostCrawl SDK using the pip package manager. ```bash pip install postcrawl ``` -------------------------------- ### Clone Repository and Install Dependencies (Bash) Source: https://github.com/post-crawl/python-sdk/blob/main/README.md Provides bash commands to clone the PostCrawl Python SDK repository and install its dependencies using uv. ```bash # Clone the repository git clone https://github.com/post-crawl/python-sdk.git cd python-sdk # Install dependencies uv sync ``` -------------------------------- ### Building and Publishing Commands (Bash) Source: https://github.com/post-crawl/python-sdk/blob/main/CLAUDE.md Contains bash commands for building distribution packages, verifying installations, and publishing the Post-Crawl Python SDK to TestPyPI and PyPI. ```bash # Build distribution packages make build ``` ```bash # Verify package installation make verify ``` ```bash # Publish to TestPyPI make publish-test ``` ```bash # Publish to PyPI make publish ``` -------------------------------- ### Contributing to the SDK Source: https://github.com/post-crawl/python-sdk/blob/main/DEVELOPMENT.md Outlines the steps for contributing to the PostCrawl Python SDK, including forking the repository, creating a feature branch, making changes, running quality checks, and submitting a pull request. ```bash 1. Fork the repository 2. Create a feature branch 3. Make your changes 4. Run `make check` to ensure code quality 5. Submit a pull request ``` -------------------------------- ### Set PostCrawl API Key Source: https://github.com/post-crawl/python-sdk/blob/main/examples/README.md Sets the PostCrawl API key as an environment variable. ```bash export POSTCRAWL_API_KEY=sk_your_api_key_here ``` -------------------------------- ### Code Quality Checks Source: https://github.com/post-crawl/python-sdk/blob/main/DEVELOPMENT.md Commands to ensure code quality by formatting code with black and ruff, running linting and type checking, and executing all checks before committing. ```bash # Format code with black and ruff make format # Run linting and type checking make lint # Run everything (recommended before commits) make check ``` -------------------------------- ### Regenerate SDK Types (Bash) Source: https://github.com/post-crawl/python-sdk/blob/main/DEVELOPMENT.md Demonstrates how to regenerate the SDK's types. This can be done from the SDK directory using 'make generate-types' or manually by navigating to the 'typegen' directory and running the generation script. ```bash # From the SDK directory make generate-types # Or manually from typegen cd ../typegen python generate.py ``` -------------------------------- ### Update Lock File (Bash) Source: https://github.com/post-crawl/python-sdk/blob/main/DEVELOPMENT.md Updates the lock file by removing the existing 'uv.lock' and then synchronizing dependencies. This ensures that dependencies are up-to-date according to the project's requirements. ```bash rm uv.lock uv sync ``` -------------------------------- ### Available Make Commands (Bash) Source: https://github.com/post-crawl/python-sdk/blob/main/README.md Lists the various make commands available for managing the PostCrawl Python SDK development workflow, including formatting, linting, testing, building, and publishing. ```bash make help # Show all available commands make format # Format code with black and ruff make lint # Run linting and type checking make test # Run test suite make check # Run format, lint, and tests make build # Build distribution packages make verify # Verify package installation make publish-test # Publish to TestPyPI ``` -------------------------------- ### Run Tests and Checks (Bash) Source: https://github.com/post-crawl/python-sdk/blob/main/README.md Illustrates how to execute the test suite and run all development checks (formatting, linting, testing) using make commands. ```bash # Run tests make test # Run all checks (format, lint, test) make check ``` -------------------------------- ### Load API Key from .env File (Python) Source: https://github.com/post-crawl/python-sdk/blob/main/README.md Shows how to load the PostCrawl API key from a .env file using python-dotenv and initialize the PostCrawlClient. ```python import os from dotenv import load_dotenv from postcrawl import PostCrawlClient load_dotenv() pc = PostCrawlClient(api_key=os.getenv("POSTCRAWL_API_KEY")) ``` -------------------------------- ### Async Search with PostCrawlClient Source: https://github.com/post-crawl/python-sdk/blob/main/README.md Demonstrates asynchronous searching for content on Reddit using the PostCrawl Python SDK. It initializes the client, performs a search for 'machine learning', and prints details of the results. ```python import asyncio from postcrawl import PostCrawlClient async def main(): # Initialize the client with your API key async with PostCrawlClient(api_key="sk_your_api_key_here") as pc: # Search for content results = await pc.search( social_platforms=["reddit"], query="machine learning", results=10, page=1 ) # Process results for post in results: print(f"{post.title} - {post.url}") print(f" Date: {post.date}") print(f" Snippet: {post.snippet[:100]}...") # Run the async function asyncio.run(main()) ``` -------------------------------- ### Core Development Commands (Bash) Source: https://github.com/post-crawl/python-sdk/blob/main/CLAUDE.md Provides essential bash commands for core development tasks within the Post-Crawl Python SDK project. Includes running tests, formatting code, linting, type checking, and pre-commit checks. ```bash # Run tests make test ``` ```bash # Run a single test file uv run pytest tests/test_client.py ``` ```bash # Run with coverage uv run pytest --cov=postcrawl ``` ```bash # Format code make format ``` ```bash # Lint and type check make lint make typecheck ``` ```bash # Run all checks before commit make check ``` -------------------------------- ### Build Package (Bash) Source: https://github.com/post-crawl/python-sdk/blob/main/README.md Command to build the distribution packages for the PostCrawl Python SDK. ```bash # Build the package make build ``` -------------------------------- ### Synchronous Search and Extract with PostCrawlClient Source: https://github.com/post-crawl/python-sdk/blob/main/README.md Shows synchronous usage of the PostCrawl Python SDK. It initializes the client and performs both a search for 'artificial intelligence' and extracts content from provided URLs, including comments. ```python from postcrawl import PostCrawlClient # Initialize the client pc = PostCrawlClient(api_key="sk_your_api_key_here") # Search synchronously results = pc.search_sync( social_platforms=["reddit", "tiktok"], query="artificial intelligence", results=5 ) # Extract content from URLs posts = pc.extract_sync( urls=["https://reddit.com/r/...", "https://tiktok.com/@..."], include_comments=True ) ``` -------------------------------- ### API Reference: Synchronous Methods Source: https://github.com/post-crawl/python-sdk/blob/main/README.md Highlights that all asynchronous methods in the PostCrawl Python SDK have corresponding synchronous versions available for direct use. ```python # All methods have synchronous versions results = pc.search_sync(...) posts = pc.extract_sync(...) combined = pc.search_and_extract_sync(...) ``` -------------------------------- ### API Reference: Async Search Source: https://github.com/post-crawl/python-sdk/blob/main/README.md Illustrates the parameters for the asynchronous search method in the PostCrawl Python SDK. It specifies social platforms, query, number of results, and pagination. ```python results = await pc.search( social_platforms=["reddit", "tiktok"], query="your search query", results=10, # 1-100 page=1 # pagination ) ``` -------------------------------- ### Import PostCrawl Exceptions (Python) Source: https://github.com/post-crawl/python-sdk/blob/main/README.md Shows how to import specific exception classes from the postcrawl library for robust error handling in your applications. ```python from postcrawl.exceptions import ( AuthenticationError, InsufficientCreditsError, RateLimitError, ValidationError ) ``` -------------------------------- ### Set API Key via Environment Variable (Bash) Source: https://github.com/post-crawl/python-sdk/blob/main/README.md Demonstrates how to set the PostCrawl API key as an environment variable for secure access. ```bash export POSTCRAWL_API_KEY="sk_your_api_key_here" ``` -------------------------------- ### API Reference: Async Extract Source: https://github.com/post-crawl/python-sdk/blob/main/README.md Demonstrates the parameters for the asynchronous extract method in the PostCrawl Python SDK. It takes a list of URLs, an option to include comments, and a response mode. ```python posts = await pc.extract( urls=["https://reddit.com/...", "https://tiktok.com/..."], include_comments=True, response_mode="raw" # or "markdown" ) ``` -------------------------------- ### Extract Content and Handle Data Types (Python) Source: https://github.com/post-crawl/python-sdk/blob/main/README.md Demonstrates how to extract web content using the PostCrawl SDK and process different post types (Reddit, TikTok) with specific attribute access. Includes basic error checking for extraction failures. ```python posts = await pc.extract(urls=["https://reddit.com/..."]) for post in posts: if post.error: print(f"Error: {post.error}") elif isinstance(post.raw, RedditPost): # Access Reddit-specific fields with snake_case attributes print(f"Subreddit: r/{post.raw.subreddit_name}") print(f"Score: {post.raw.score}") print(f"Title: {post.raw.title}") print(f"Upvotes: {post.raw.upvotes}") print(f"Created: {post.raw.created_at}") if post.raw.comments: print(f"Comments: {len(post.raw.comments)}") elif isinstance(post.raw, TiktokPost): # Access TikTok-specific fields with snake_case attributes print(f"Username: @{post.raw.username}") print(f"Likes: {post.raw.likes}") print(f"Total Comments: {post.raw.total_comments}") print(f"Created: {post.raw.created_at}") if post.raw.hashtags: print(f"Hashtags: {', '.join(post.raw.hashtags)}") ``` -------------------------------- ### Access Rate Limit Information (Python) Source: https://github.com/post-crawl/python-sdk/blob/main/README.md Illustrates how to retrieve and print rate limit details (limit, remaining, reset time) after making a request with the PostCrawl client. ```python pc = PostCrawlClient(api_key="sk_...") results = await pc.search(...) print(f"Rate limit: {pc.rate_limit_info['limit']}") print(f"Remaining: {pc.rate_limit_info['remaining']}") print(f"Reset at: {pc.rate_limit_info['reset']}") ``` -------------------------------- ### API Reference: Async Search and Extract Source: https://github.com/post-crawl/python-sdk/blob/main/README.md Shows the parameters for the combined asynchronous search and extract method in the PostCrawl Python SDK. It includes options for social platforms, query, results, pagination, comments, and response mode. ```python posts = await pc.search_and_extract( social_platforms=["reddit"], query="search query", results=5, page=1, include_comments=False, response_mode="markdown" ) ``` -------------------------------- ### Type Generation Command (Bash) Source: https://github.com/post-crawl/python-sdk/blob/main/CLAUDE.md A bash command to regenerate types for the Post-Crawl Python SDK from TypeScript schemas. ```bash # Regenerate types from TypeScript schemas make generate-types ``` -------------------------------- ### Working with Platform-Specific Types Source: https://github.com/post-crawl/python-sdk/blob/main/README.md Demonstrates importing platform-specific types like RedditPost and TiktokPost from the PostCrawl SDK for type-safe data handling. ```python from postcrawl import PostCrawlClient, RedditPost, TiktokPost ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.