### Install snscrape development version

Source: https://github.com/justanotherarchivist/snscrape/blob/master/README.md

Installs the latest development version of snscrape directly from its GitHub repository. This requires Git to be installed.

```bash
pip3 install git+https://github.com/JustAnotherArchivist/snscrape.git
```

--------------------------------

### Install snscrape using pip

Source: https://github.com/justanotherarchivist/snscrape/blob/master/README.md

Installs the snscrape library using pip. Ensure Python 3.8 or higher is installed. Dependencies like lxml, libxml2, and libxslt may also need to be installed.

```bash
pip3 install snscrape
```

--------------------------------

### Install snscrape using pip

Source: https://context7.com/justanotherarchivist/snscrape/llms.txt

Installs the snscrape library using pip. This is the standard method for adding the package to your Python environment.

```bash
pip3 install snscrape
```

```bash
pip3 install git+https://github.com/JustAnotherArchivist/snscrape.git
```

--------------------------------

### Scrape Reddit User Submissions and Comments (CLI)

Source: https://context7.com/justanotherarchivist/snscrape/llms.txt

Examples of using the Reddit User Scraper CLI to extract content from a Reddit user's profile via the Pushshift API. Supports filtering to get only submissions or comments, and by timestamp.

```bash
# Get all submissions and comments from a user
snscrape reddit-user spez
```

```bash
# Get only submissions (no comments)
snscrape reddit-user --no-comments spez
```

```bash
# Get only comments
snscrape reddit-user --no-submissions spez
```

```bash
# Filter by timestamp
snscrape reddit-user --before 1609459200 --after 1577836800 spez
```

--------------------------------

### Scrape Twitter Hashtags (CLI)

Source: https://context7.com/justanotherarchivist/snscrape/llms.txt

Examples for using the Twitter Hashtag Scraper CLI to retrieve tweets associated with a specific hashtag. Allows for limiting results and specifying output format.

```bash
# Get tweets with a hashtag
snscrape twitter-hashtag archiveteam
```

```bash
# Get first 100 tweets as JSONL
snscrape --jsonl --max-results 100 twitter-hashtag python
```

--------------------------------

### Scrape Instagram User Posts (CLI)

Source: https://context7.com/justanotherarchivist/snscrape/llms.txt

Examples of using the Instagram User Scraper CLI to extract posts from a user's profile. Includes options for retrieving user entity information and limiting results.

```bash
# Get all posts from a user
snscrape instagram-user natgeo
```

```bash
# Get user entity info
snscrape --with-entity --max-results 0 instagram-user natgeo
```

```bash
# Get first 50 posts as JSONL
snscrape --jsonl --max-results 50 instagram-user nasa
```

--------------------------------

### Scrape Facebook User Pages, Community Posts, and Groups with Python

Source: https://context7.com/justanotherarchivist/snscrape/llms.txt

This Python code snippet demonstrates scraping Facebook data using snscrape. It includes examples for scraping user/page posts, community posts, and group information. The code fetches page entity details and iterates through posts, extracting content and outlinks. Ensure snscrape is installed.

```Python
import snscrape.modules.facebook as snfacebook

# Scrape user/page posts
scraper = snfacebook.FacebookUserScraper('Meta')

# Get page entity
page = scraper.entity
if page:
    print(f"Page: {page.name}")
    print(f"Username: {page.username}")
    print(f"Likes: {page.likes}")
    print(f"Followers: {page.followers}")
    print(f"Verified: {page.verified}")

# Get posts
for i, post in enumerate(scraper.get_items()):
    if i >= 20:
        break
    print(f"URL: {post.cleanUrl}")
    print(f"Date: {post.date}")
    print(f"Content: {post.content[:100] if post.content else 'No text'}...")
    print(f"Outlinks: {post.outlinks}")
    print("---")

# Scrape community/visitor posts
community_scraper = snfacebook.FacebookCommunityScraper('Meta')

# Scrape group
group_scraper = snfacebook.FacebookGroupScraper('123456789')
```

--------------------------------

### Get Twitter Trends (CLI)

Source: https://context7.com/justanotherarchivist/snscrape/llms.txt

Command to fetch currently trending topics on Twitter using the Twitter Trends Scraper CLI. Outputs results in JSONL format.

```bash
# Get current Twitter trends
snscrape --jsonl twitter-trends
```

--------------------------------

### Scrape Twitter User Timeline (CLI)

Source: https://context7.com/justanotherarchivist/snscrape/llms.txt

Examples of using the Twitter User Scraper CLI to retrieve tweets from a specific user's timeline. Supports various output formats and filtering options.

```bash
# Get all tweets from a user (outputs URLs by default)
snscrape twitter-user textfiles
```

```bash
# Get first 100 tweets with JSONL output
snscrape --jsonl --max-results 100 twitter-user textfiles
```

```bash
# Save output to a file
snscrape twitter-user textfiles > twitter-@textfiles.txt
```

```bash
# Get user entity info only (no tweets)
snscrape --with-entity --max-results 0 twitter-user textfiles
```

```bash
# Scrape by user ID instead of username
snscrape twitter-user --user-id 12345678
```

--------------------------------

### Scrape Reddit Users, Subreddits, and Searches with Python

Source: https://context7.com/justanotherarchivist/snscrape/llms.txt

This Python code snippet shows how to use snscrape to scrape Reddit data, including user submissions and comments, subreddit posts, and search results. It utilizes the Pushshift API for data retrieval. Ensure the snscrape library is installed.

```Python
import snscrape.modules.reddit as snreddit

# Scrape user submissions and comments
scraper = snreddit.RedditUserScraper(
    'spez',
    submissions=True,
    comments=True
)

for i, item in enumerate(scraper.get_items()):
    if i >= 50:
        break

    if isinstance(item, snreddit.Submission):
        print(f"[Submission] {item.title}")
        print(f"  Subreddit: r/{item.subreddit}")
        print(f"  URL: {item.url}")
    elif isinstance(item, snreddit.Comment):
        print(f"[Comment] {item.body[:100]}...")
        print(f"  Subreddit: r/{item.subreddit}")

# Scrape subreddit
subreddit_scraper = snreddit.RedditSubredditScraper('python')

# Search Reddit
search_scraper = snreddit.RedditSearchScraper('machine learning')

# Get specific submission with all comments
submission_scraper = snreddit.RedditSubmissionScraper('abc123')
for item in submission_scraper.get_items():
    print(item)
```

--------------------------------

### Filtering Tweets by Date (CLI - Bash)

Source: https://context7.com/justanotherarchivist/snscrape/llms.txt

Details how to filter scraped tweets based on date using the snscrape command-line interface. It covers using the --since option for a specific start date and employing Twitter's native search operators like 'since:' and 'until:' for defining date ranges. Supported date formats are also listed.

```Bash
# Get tweets since a specific datetime
snscrape --since 2023-01-01 twitter-user textfiles

# Using Twitter search operators for date range
snscrape twitter-search "from:textfiles since:2023-01-01 until:2023-06-30"

# Datetime formats supported:
# 2023-01-01
# 2023-01-01 +0000 (with timezone)
# 2023-01-01 12:00:00
# 2023-01-01 12:00:00 +0000
# 1672531200 (Unix timestamp)
```

--------------------------------

### snscrape CLI Basic Syntax and Help

Source: https://context7.com/justanotherarchivist/snscrape/llms.txt

Demonstrates the general command-line interface syntax for snscrape and how to access help information for global options or specific scrapers.

```bash
snscrape [GLOBAL-OPTIONS] SCRAPER-NAME [SCRAPER-OPTIONS] [SCRAPER-ARGUMENTS...]
```

```bash
snscrape --help
```

```bash
snscrape twitter-user --help
```

--------------------------------

### CLI Verbose Output and Debugging Options (Bash)

Source: https://context7.com/justanotherarchivist/snscrape/llms.txt

Demonstrates various command-line interface options for snscrape, including enabling verbose output (-v for INFO, -vv for DEBUG), dumping local variables on errors for debugging, configuring retry behavior, and enabling progress reporting.

```Bash
# Verbose output (-v for INFO, -vv for DEBUG)
snscrape -vv twitter-user textfiles

# Dump local variables on warnings/errors for debugging
snscrape --dump-locals twitter-user textfiles

# Configure retry behavior
snscrape --retries 5 twitter-user textfiles

# Progress reporting
snscrape --progress --max-results 1000 twitter-search "python"
```

--------------------------------

### CLI Scraping and Output Formatting

Source: https://context7.com/justanotherarchivist/snscrape/llms.txt

Documentation for using the snscrape CLI, including custom output formats, JSONL serialization, and date filtering.

```APIDOC
## CLI Scraping Operations

### Description
Perform scraping tasks via the command line with support for custom output formatting and date-based filtering.

### Common Commands
- **Custom Format**: `snscrape -f "{date} - {rawContent}" twitter-user [username]`
- **JSONL Output**: `snscrape --jsonl twitter-user [username] > output.jsonl`
- **Date Filtering**: `snscrape --since 2023-01-01 twitter-search "python"`

### Debugging Options
- **Verbose**: `-v` (INFO), `-vv` (DEBUG)
- **Dump Locals**: `--dump-locals` (Dumps variables on error)
- **Retries**: `--retries [n]` (Configure retry attempts)
```

--------------------------------

### Scrape Reddit Subreddit Content (CLI)

Source: https://context7.com/justanotherarchivist/snscrape/llms.txt

Demonstrates the Reddit Subreddit Scraper CLI for extracting all submissions and comments from a specified subreddit.

```bash
# Get all content from a subreddit
snscrape reddit-subreddit programming
```

--------------------------------

### Custom Output Formats (CLI - Bash)

Source: https://context7.com/justanotherarchivist/snscrape/llms.txt

Explains how to customize the output format of snscrape when using the command-line interface. This includes using Python format strings to specify desired fields, outputting in JSONL format, and using a specific JSONL format for compatibility with buggy integer parsers.

```Bash
# Custom format string (use {field} syntax)
snscrape -f "{date} - {rawContent}" twitter-user textfiles

# Output specific fields
snscrape -f "{id},{user.username},{likeCount},{retweetCount}" twitter-search "python"

# JSONL output (recommended for data processing)
snscrape --jsonl twitter-user textfiles > tweets.jsonl

# JSONL for buggy int parsers (adds string versions of large integers)
snscrape --jsonl-for-buggy-int-parser twitter-user textfiles
```

--------------------------------

### Scrape Instagram Hashtag Posts (CLI)

Source: https://context7.com/justanotherarchivist/snscrape/llms.txt

Demonstrates the Instagram Hashtag Scraper CLI for finding posts associated with a specific hashtag. Allows for limiting the number of results returned.

```bash
# Get posts with a hashtag
snscrape instagram-hashtag photography
```

```bash
# Get first 100 posts
snscrape --max-results 100 instagram-hashtag travel
```

--------------------------------

### Scrape Instagram Location Posts (CLI)

Source: https://context7.com/justanotherarchivist/snscrape/llms.txt

Shows how to use the Instagram Location Scraper CLI to retrieve posts from a specific location, identified by its location ID.

```bash
# Get posts from a location
snscrape instagram-location 212988663
```

--------------------------------

### Search Twitter for Tweets (CLI)

Source: https://context7.com/justanotherarchivist/snscrape/llms.txt

Demonstrates how to use the Twitter Search Scraper CLI to find tweets based on keywords, phrases, and advanced search operators. Supports filtering by engagement and date ranges.

```bash
# Search for tweets containing a phrase
snscrape twitter-search "python programming"
```

```bash
# Get top tweets instead of live/chronological
snscrape --jsonl twitter-search --top "machine learning"
```

```bash
# Search with Twitter advanced operators
snscrape --max-results 500 twitter-search "from:elonmusk since:2023-01-01 until:2023-12-31"
```

```bash
# Search tweets mentioning a user
snscrape twitter-search "@openai"
```

```bash
# Search with min engagement filters
snscrape twitter-search "AI min_faves:1000"
```

--------------------------------

### Data Serialization and Structures

Source: https://context7.com/justanotherarchivist/snscrape/llms.txt

Explains how to serialize Tweet and User objects to JSON and outlines the available fields for each data type.

```APIDOC
## Data Serialization

### Description
Both `Tweet` and `User` objects in snscrape provide a `.json()` method to serialize the object data into a JSON string format.

### Usage
```python
# Serialize a Tweet object
json_str = tweet.json()

# Serialize a User object
user_json = user.json()
```

### Tweet Fields
- **inReplyToUser** (User) - User being replied to
- **mentionedUsers** (list) - List of mentioned Users
- **hashtags** (list) - List of hashtag strings
- **viewCount** (int) - View count if available
- **bookmarkCount** (int) - Bookmark count if available

### User Fields
- **username** (string) - Twitter handle
- **id** (int) - Numeric user ID
- **followersCount** (int) - Follower count
- **statusesCount** (int) - Total tweet count
```

--------------------------------

### CLI Reddit Search Scraper

Source: https://context7.com/justanotherarchivist/snscrape/llms.txt

Scrape Reddit submissions and comments using the command line interface.

```APIDOC
## CLI reddit-search

### Description
Searches Reddit for submissions and comments matching a specific query.

### Method
CLI Command

### Endpoint
snscrape reddit-search "[query]"

### Parameters
#### Query Parameters
- **query** (string) - Required - The search term to look for on Reddit.

### Request Example
snscrape reddit-search "machine learning"

### Response
#### Success Response (200)
- **output** (stream) - Returns a stream of Reddit submission/comment objects.
```

--------------------------------

### User Object Fields and JSON Conversion (Python)

Source: https://context7.com/justanotherarchivist/snscrape/llms.txt

Illustrates how to access detailed information about a Twitter user, including their username, ID, display name, bio, verification status, account creation date, follower/friend counts, tweet counts, location, and profile image/banner URLs. It also shows the conversion of the user object to a JSON string.

```Python
from snscrape.modules.twitter import User

# User object fields:
# user.username - Twitter handle (without @)
# user.id - numeric user ID
# user.displayname - display name
# user.rawDescription - raw bio text
# user.renderedDescription - bio with URLs expanded
# user.descriptionLinks - list of TextLink in bio
# user.verified - legacy verification status
# user.created - account creation datetime
# user.followersCount - follower count
# user.friendsCount - following count
# user.statusesCount - tweet count
# user.favouritesCount - likes count
# user.listedCount - list membership count
# user.mediaCount - media count
# user.location - location string
# user.protected - whether account is protected
# user.link - TextLink in profile
# user.profileImageUrl - profile image URL
# user.profileBannerUrl - banner image URL
# user.blue - Twitter Blue status
# user.blueType - type of blue verification

# Convert to JSON
json_str = user.json()
```

--------------------------------

### Telegram Channel Scraper (CLI)

Source: https://context7.com/justanotherarchivist/snscrape/llms.txt

Scrapes posts from public Telegram channels. Options include retrieving all posts, channel entity info, or a limited number of posts in JSONL format.

```bash
# Get all posts from a channel
snscrape telegram-channel duaborovik

# Get channel entity info
snscrape --with-entity --max-results 0 telegram-channel duaborovik

# Get first 100 posts as JSONL
snscrape --jsonl --max-results 100 telegram-channel telegram
```

--------------------------------

### Tweet Object Fields and JSON Conversion (Python)

Source: https://context7.com/justanotherarchivist/snscrape/llms.txt

Demonstrates accessing various attributes of a tweet object, such as in-reply-to user, mentioned users, hashtags, cashtags, geotagging information, view counts, and bookmark counts. It also shows how to convert the tweet object into a JSON string.

```Python
# tweet.inReplyToUser - User being replied to
# tweet.mentionedUsers - list of mentioned Users
# tweet.hashtags - list of hashtag strings
# tweet.cashtags - list of cashtag strings
# tweet.coordinates - Coordinates object (if geotagged)
# tweet.place - Place object (if location tagged)
# tweet.card - Card object (if contains card)
# tweet.viewCount - view count (if available)
# tweet.bookmarkCount - bookmark count (if available)
# tweet.pinned - whether tweet is pinned

# Convert to JSON
json_str = tweet.json()
```

--------------------------------

### Enable debug logging in snscrape (Python)

Source: https://github.com/justanotherarchivist/snscrape/blob/master/README.md

Enables debug-level logging for snscrape when used as a Python module. This is useful for diagnosing issues and should be called before any snscrape functions.

```python
import logging
logging.basicConfig(level = logging.DEBUG)
```

--------------------------------

### Scrape Instagram User Profiles, Hashtags, and Locations with Python

Source: https://context7.com/justanotherarchivist/snscrape/llms.txt

This Python code snippet demonstrates how to use the snscrape library to scrape data from Instagram. It covers fetching user profile information, iterating through user posts, scraping posts by hashtag, and scraping posts by location ID. Dependencies include the snscrape library.

```Python
import snscrape.modules.instagram as sninstagram

# Scrape user profile
scraper = sninstagram.InstagramUserScraper('natgeo')

# Get user entity
user = scraper.entity
if user:
    print(f"Username: {user.username}")
    print(f"Name: {user.name}")
    print(f"Followers: {user.followers}")
    print(f"Following: {user.following}")
    print(f"Posts: {user.posts}")

# Get posts
for i, post in enumerate(scraper.get_items()):
    if i >= 20:
        break
    print(f"URL: {post.url}")
    print(f"Date: {post.date}")
    print(f"Content: {post.content[:50] if post.content else 'No caption'}...")
    print(f"Likes: {post.likes}")
    print(f"Comments: {post.comments}")
    print(f"Is Video: {post.isVideo}")
    print("---")

# Scrape hashtag
hashtag_scraper = sninstagram.InstagramHashtagScraper('photography')
for i, post in enumerate(hashtag_scraper.get_items()):
    if i >= 10:
        break
    print(post.url)

# Scrape location
location_scraper = sninstagram.InstagramLocationScraper(212988663)
```

--------------------------------

### Reddit Search Scraper (CLI)

Source: https://context7.com/justanotherarchivist/snscrape/llms.txt

Searches Reddit for submissions and comments matching a query. Supports JSONL output for structured data.

```bash
# Search Reddit
snscrape reddit-search "machine learning"

# Search with JSON output
snscrape --jsonl reddit-search "python tutorial"
```

--------------------------------

### Scrape Twitter User Profile Timeline (CLI)

Source: https://context7.com/justanotherarchivist/snscrape/llms.txt

Utilizes the Twitter Profile Scraper CLI to gather a comprehensive timeline of a user's activity, including retweets and replies, which is more detailed than the standard user scraper.

```bash
# Get full profile timeline
snscrape twitter-profile elonmusk
```

```bash
# With JSON output
snscrape --jsonl twitter-profile elonmusk
```

--------------------------------

### Reddit Subreddit Scraper (CLI)

Source: https://context7.com/justanotherarchivist/snscrape/llms.txt

Scrapes posts from a specific Reddit subreddit. It can output results in JSONL format and exclude comments.

```bash
snscrape --jsonl reddit-subreddit --no-comments python
```

--------------------------------

### Scrape Telegram Channels and Posts with Python

Source: https://context7.com/justanotherarchivist/snscrape/llms.txt

This Python code snippet illustrates how to scrape public Telegram channels using the snscrape library. It covers fetching channel metadata (title, members, verification status) and iterating through posts, extracting content, outlinks, and link previews. Requires the snscrape library.

```Python
import snscrape.modules.telegram as sntelegram

# Create channel scraper
scraper = sntelegram.TelegramChannelScraper('duaborovik')

# Get channel entity
channel = scraper.entity
if channel:
    print(f"Channel: {channel.title}")
    print(f"Username: @{channel.username}")
    print(f"Members: {channel.members}")
    print(f"Verified: {channel.verified}")
    print(f"Description: {channel.description}")

# Get posts
for i, post in enumerate(scraper.get_items()):
    if i >= 30:
        break
    print(f"URL: {post.url}")
    print(f"Date: {post.date}")
    print(f"Content: {post.content[:100] if post.content else 'No text'}...")
    print(f"Outlinks: {post.outlinks}")

    if post.linkPreview:
        print(f"Link Preview: {post.linkPreview.title} - {post.linkPreview.href}")
    print("---")
```

--------------------------------

### Reddit Scraper

Source: https://context7.com/justanotherarchivist/snscrape/llms.txt

Scrapes Reddit users, subreddits, and searches via the Pushshift API. Supports fetching submissions and comments.

```APIDOC
## Reddit Scraper

Scrapes Reddit users, subreddits, and searches via the Pushshift API.

### Usage

```python
import snscrape.modules.reddit as snreddit

# Scrape user submissions and comments
scraper = snreddit.RedditUserScraper(
    'spez',
    submissions=True,
    comments=True
)

for i, item in enumerate(scraper.get_items()):
    if i >= 50:
        break

    if isinstance(item, snreddit.Submission):
        print(f"[Submission] {item.title}")
        print(f"  Subreddit: r/{item.subreddit}")
        print(f"  URL: {item.url}")
    elif isinstance(item, snreddit.Comment):
        print(f"[Comment] {item.body[:100]}...")
        print(f"  Subreddit: r/{item.subreddit}")

# Scrape subreddit
subreddit_scraper = snreddit.RedditSubredditScraper('python')

# Search Reddit
search_scraper = snreddit.RedditSearchScraper('machine learning')

# Get specific submission with all comments
submission_scraper = snreddit.RedditSubmissionScraper('abc123')
for item in submission_scraper.get_items():
    print(item)
```
```

--------------------------------

### Scrape Single Twitter Tweet and Conversation (CLI)

Source: https://context7.com/justanotherarchivist/snscrape/llms.txt

Shows how to use the Twitter Tweet Scraper CLI to fetch a single tweet by its ID, including options to retrieve its full conversation thread or recursively all connected tweets.

```bash
# Get a single tweet by ID
snscrape twitter-tweet 1234567890123456789
```

```bash
# Get tweet with full conversation thread (scroll both directions)
snscrape --jsonl twitter-tweet --scroll 1234567890123456789
```

```bash
# Recursively get all connected tweets (warning: slow)
snscrape twitter-tweet --recurse 1234567890123456789
```

--------------------------------

### Telegram Scraper

Source: https://context7.com/justanotherarchivist/snscrape/llms.txt

Scrapes public Telegram channels, including posts and channel metadata such as title, members, and description.

```APIDOC
## Telegram Scraper

Scrapes public Telegram channels including posts and channel metadata.

### Usage

```python
import snscrape.modules.telegram as sntelegram

# Create channel scraper
scraper = sntelegram.TelegramChannelScraper('duaborovik')

# Get channel entity
channel = scraper.entity
if channel:
    print(f"Channel: {channel.title}")
    print(f"Username: @{channel.username}")
    print(f"Members: {channel.members}")
    print(f"Verified: {channel.verified}")
    print(f"Description: {channel.description}")

# Get posts
for i, post in enumerate(scraper.get_items()):
    if i >= 30:
        break
    print(f"URL: {post.url}")
    print(f"Date: {post.date}")
    print(f"Content: {post.content[:100] if post.content else 'No text'}...")
    print(f"Outlinks: {post.outlinks}")

    if post.linkPreview:
        print(f"Link Preview: {post.linkPreview.title} - {post.linkPreview.href}")
    print("---")
```
```

--------------------------------

### Python Twitter User Scraper

Source: https://context7.com/justanotherarchivist/snscrape/llms.txt

Programmatically scrape Twitter user tweets and profile information using the Python library.

```APIDOC
## Python TwitterUserScraper

### Description
Retrieves user entity information and iterates through a user's recent tweets.

### Method
Python Class

### Endpoint
snscrape.modules.twitter.TwitterUserScraper('[username]')

### Parameters
#### Path Parameters
- **username** (string) - Required - The Twitter handle of the user to scrape.

### Request Example
scraper = sntwitter.TwitterUserScraper('textfiles')

### Response
#### Success Response (200)
- **entity** (object) - User profile metadata (followers, following, etc.)
- **items** (iterator) - A stream of tweet objects.
```

--------------------------------

### Collect latest tweets by hashtag (CLI)

Source: https://github.com/justanotherarchivist/snscrape/blob/master/README.md

Fetches the latest 100 tweets associated with a specific hashtag using the snscrape CLI. The `--max-results` option limits the number of returned tweets.

```bash
snscrape --max-results 100 twitter-hashtag archiveteam
```

--------------------------------

### Facebook Group Scraper (CLI)

Source: https://context7.com/justanotherarchivist/snscrape/llms.txt

Scrapes posts from a Facebook group using its name or ID.

```bash
# Get posts from a group (by name or ID)
snscrape facebook-group 123456789
```

--------------------------------

### Facebook User Scraper (CLI)

Source: https://context7.com/justanotherarchivist/snscrape/llms.txt

Scrapes posts from a Facebook user or page. Can also retrieve page entity information.

```bash
# Get posts from a Facebook page
snscrape facebook-user Meta

# Get page entity info
snscrape --with-entity --max-results 0 facebook-user zuck
```

--------------------------------

### Python Twitter Profile Scraper

Source: https://context7.com/justanotherarchivist/snscrape/llms.txt

Scrapes a user's complete profile timeline, including pinned tweets, retweets, and replies. Offers more comprehensive results than TwitterUserScraper.

```python
import snscrape.modules.twitter as sntwitter

# TwitterProfileScraper provides more comprehensive results than TwitterUserScraper
scraper = sntwitter.TwitterProfileScraper('elonmusk')

# Get entity (user profile)
user = scraper.entity

# Iterate through profile timeline
for i, tweet in enumerate(scraper.get_items()):
    if i >= 50:
        break

    # Check if tweet is pinned
    if tweet.pinned:
        print(f"[PINNED] {tweet.rawContent[:50]}...")

    # Check if it's a retweet
    if tweet.retweetedTweet:
        print(f"RT @{tweet.retweetedTweet.user.username}: {tweet.retweetedTweet.rawContent[:50]}...")
    else:
        print(f"{tweet.rawContent[:50]}...")
```

--------------------------------

### Python Twitter Search Scraper

Source: https://context7.com/justanotherarchivist/snscrape/llms.txt

Searches Twitter programmatically for tweets based on a query. Supports different search modes (live/chronological, top tweets) and collects results into a list.

```python
import snscrape.modules.twitter as sntwitter
from snscrape.modules.twitter import TwitterSearchScraperMode

# Search for live/chronological tweets (default)
scraper = sntwitter.TwitterSearchScraper('python programming')

# Search for top tweets instead
scraper = sntwitter.TwitterSearchScraper(
    'machine learning',
    mode=TwitterSearchScraperMode.TOP
)

# Collect tweets into a list
tweets = []
for i, tweet in enumerate(scraper.get_items()):
    if i >= 500:
        break
    tweets.append({
        'id': tweet.id,
        'date': tweet.date,
        'content': tweet.rawContent,
        'user': tweet.user.username,
        'likes': tweet.likeCount,
        'retweets': tweet.retweetCount,
        'url': tweet.url,
        'hashtags': tweet.hashtags,
        'media': tweet.media,
    })

print(f"Collected {len(tweets)} tweets")
```

--------------------------------

### Tweet Data Structure

Source: https://context7.com/justanotherarchivist/snscrape/llms.txt

Details the fields available in the Tweet data structure, including content, engagement metrics, media, and metadata.

```APIDOC
## Tweet Data Structure

The Tweet dataclass contains comprehensive tweet information including content, engagement metrics, media, and metadata.

### Fields

```python
from snscrape.modules.twitter import Tweet

# Tweet object fields:
tweet.url - URL of the tweet
tweet.date - datetime of tweet creation
tweet.rawContent - raw tweet text
tweet.renderedContent - rendered text with URLs expanded
tweet.id - tweet ID
tweet.user - User object
tweet.replyCount - number of replies
tweet.retweetCount - number of retweets
tweet.likeCount - number of likes
tweet.quoteCount - number of quote tweets
tweet.conversationId - conversation thread ID
tweet.lang - language code
tweet.source - source application
tweet.links - list of TextLink objects
tweet.media - list of Photo/Video/Gif objects
tweet.retweetedTweet - retweeted Tweet object (if retweet)
tweet.quotedTweet - quoted Tweet object (if quote tweet)
tweet.inReplyToTweetId - ID of tweet being replied to
```
```

--------------------------------

### Collect tweets by a specific user (CLI)

Source: https://github.com/justanotherarchivist/snscrape/blob/master/README.md

Uses the snscrape command-line interface to collect all tweets from a specified Twitter user. The output is redirected to a file for further processing.

```bash
snscrape twitter-user textfiles >twitter-@textfiles
```

--------------------------------

### Facebook Scraper

Source: https://context7.com/justanotherarchivist/snscrape/llms.txt

Scrapes Facebook user pages, community posts, and groups. Provides access to page entities and posts.

```APIDOC
## Facebook Scraper

Scrapes Facebook user pages, community posts, and groups.

### Usage

```python
import snscrape.modules.facebook as snfacebook

# Scrape user/page posts
scraper = snfacebook.FacebookUserScraper('Meta')

# Get page entity
page = scraper.entity
if page:
    print(f"Page: {page.name}")
    print(f"Username: {page.username}")
    print(f"Likes: {page.likes}")
    print(f"Followers: {page.followers}")
    print(f"Verified: {page.verified}")

# Get posts
for i, post in enumerate(scraper.get_items()):
    if i >= 20:
        break
    print(f"URL: {post.cleanUrl}")
    print(f"Date: {post.date}")
    print(f"Content: {post.content[:100] if post.content else 'No text'}...")
    print(f"Outlinks: {post.outlinks}")
    print("---")

# Scrape community/visitor posts
community_scraper = snfacebook.FacebookCommunityScraper('Meta')

# Scrape group
group_scraper = snfacebook.FacebookGroupScraper('123456789')
```
```

--------------------------------

### Instagram Scraper

Source: https://context7.com/justanotherarchivist/snscrape/llms.txt

Scrapes Instagram user profiles, hashtags, and locations. Provides access to user entities, posts, and associated metadata.

```APIDOC
## Instagram Scraper

Scrapes Instagram user profiles, hashtags, and locations.

### Usage

```python
import snscrape.modules.instagram as sninstagram

# Scrape user profile
scraper = sninstagram.InstagramUserScraper('natgeo')

# Get user entity
user = scraper.entity
if user:
    print(f"Username: {user.username}")
    print(f"Name: {user.name}")
    print(f"Followers: {user.followers}")
    print(f"Following: {user.following}")
    print(f"Posts: {user.posts}")

# Get posts
for i, post in enumerate(scraper.get_items()):
    if i >= 20:
        break
    print(f"URL: {post.url}")
    print(f"Date: {post.date}")
    print(f"Content: {post.content[:50] if post.content else 'No caption'}...")
    print(f"Likes: {post.likes}")
    print(f"Comments: {post.comments}")
    print(f"Is Video: {post.isVideo}")
    print("---")

# Scrape hashtag
hashtag_scraper = sninstagram.InstagramHashtagScraper('photography')
for i, post in enumerate(hashtag_scraper.get_items()):
    if i >= 10:
        break
    print(post.url)

# Scrape location
location_scraper = sninstagram.InstagramLocationScraper(212988663)
```
```

--------------------------------

### Python Twitter Tweet Scraper

Source: https://context7.com/justanotherarchivist/snscrape/llms.txt

Retrieves a single tweet or entire conversation threads with configurable scraping modes.

```APIDOC
## Python TwitterTweetScraper

### Description
Retrieves a single tweet or conversation thread based on the provided Tweet ID.

### Method
Python Class

### Endpoint
snscrape.modules.twitter.TwitterTweetScraper([tweet_id], mode=[mode])

### Parameters
#### Path Parameters
- **tweet_id** (integer) - Required - The unique ID of the tweet.
- **mode** (enum) - Required - The scraping mode: SINGLE, SCROLL, or RECURSE.

### Request Example
scraper = sntwitter.TwitterTweetScraper(1234567890123456789, mode=TwitterTweetScraperMode.SINGLE)

### Response
#### Success Response (200)
- **items** (iterator) - Returns the tweet object or conversation thread.
```

--------------------------------

### Python Twitter Tweet Scraper

Source: https://context7.com/justanotherarchivist/snscrape/llms.txt

Retrieves a single tweet or an entire conversation thread using different scraping modes (SINGLE, SCROLL, RECURSE).

```python
import snscrape.modules.twitter as sntwitter
from snscrape.modules.twitter import TwitterTweetScraperMode

# Get single tweet
scraper = sntwitter.TwitterTweetScraper(
    1234567890123456789,
    mode=TwitterTweetScraperMode.SINGLE
)

for tweet in scraper.get_items():
    print(f"Tweet: {tweet.rawContent}")
    print(f"Likes: {tweet.likeCount}")
    print(f"Quote tweet: {tweet.quotedTweet}")

# Get full conversation thread (scroll mode)
scraper = sntwitter.TwitterTweetScraper(
    1234567890123456789,
    mode=TwitterTweetScraperMode.SCROLL
)

conversation = list(scraper.get_items())
print(f"Found {len(conversation)} tweets in conversation")

# Recursive mode - gets all connected tweets (slow but comprehensive)
scraper = sntwitter.TwitterTweetScraper(
    1234567890123456789,
    mode=TwitterTweetScraperMode.RECURSE
)
```

--------------------------------

### Python Twitter User Scraper

Source: https://context7.com/justanotherarchivist/snscrape/llms.txt

Scrapes tweets from a specific Twitter user programmatically using the snscrape Python library. It provides access to user entity information and individual tweet details.

```python
import snscrape.modules.twitter as sntwitter

# Create scraper for a user
scraper = sntwitter.TwitterUserScraper('textfiles')

# Get user entity information
user = scraper.entity
print(f"Username: {user.username}")
print(f"Display name: {user.displayname}")
print(f"Followers: {user.followersCount}")
print(f"Following: {user.friendsCount}")
print(f"Tweets: {user.statusesCount}")
print(f"Verified: {user.verified}")
print(f"Created: {user.created}")

# Iterate through tweets
for i, tweet in enumerate(scraper.get_items()):
    if i >= 100:  # Limit to 100 tweets
        break
    print(f"Tweet ID: {tweet.id}")
    print(f"Date: {tweet.date}")
    print(f"Content: {tweet.rawContent[:100]}...")
    print(f"Likes: {tweet.likeCount}")
    print(f"Retweets: {tweet.retweetCount}")
    print(f"URL: {tweet.url}")
    print("---")
```

--------------------------------

### Tweet Data Structure Definition in Python

Source: https://context7.com/justanotherarchivist/snscrape/llms.txt

This Python snippet defines the structure of a Tweet object as used by the snscrape library. It lists various fields available within the Tweet object, such as URL, date, content, engagement metrics (likes, replies, retweets), media, and metadata. This is a reference for understanding the output of Twitter scraping.

```Python
from snscrape.modules.twitter import Tweet

# Tweet object fields:
# tweet.url - URL of the tweet
# tweet.date - datetime of tweet creation
# tweet.rawContent - raw tweet text
# tweet.renderedContent - rendered text with URLs expanded
# tweet.id - tweet ID
# tweet.user - User object
# tweet.replyCount - number of replies
# tweet.retweetCount - number of retweets
# tweet.likeCount - number of likes
# tweet.quoteCount - number of quote tweets
# tweet.conversationId - conversation thread ID
# tweet.lang - language code
# tweet.source - source application
# tweet.links - list of TextLink objects
# tweet.media - list of Photo/Video/Gif objects
# tweet.retweetedTweet - retweeted Tweet object (if retweet)
# tweet.quotedTweet - quoted Tweet object (if quote tweet)
# tweet.inReplyToTweetId - ID of tweet being replied to
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.