### Install facebook-scraper Python Package Source: https://github.com/kritsanan1/facebook-scraper/blob/master/README.md Instructions to install the facebook-scraper package from PyPI or the master branch using pip. ```shell pip install facebook-scraper ``` ```shell pip install git+https://github.com/kevinzg/facebook-scraper.git ``` -------------------------------- ### Facebook Scraper CLI Usage Source: https://github.com/kritsanan1/facebook-scraper/blob/master/README.md Example of how to use the facebook-scraper tool from the command line to scrape posts from a specified page and save them to a CSV file. It includes options for specifying the output filename and the number of pages to scrape. ```shell $ facebook-scraper --filename nintendo_page_posts.csv --pages 10 nintendo ``` -------------------------------- ### Facebook Post Example Data Structure Source: https://github.com/kritsanan1/facebook-scraper/blob/master/README.md An example of the data structure returned when scraping a Facebook post. This includes fields like availability, like counts, image URLs, post text, and reaction data. Note that some fields like 'comments_full' might be None. ```python { 'available': True, 'comments': 459, 'comments_full': None, 'factcheck': None, 'fetched_time': datetime.datetime(2021, 4, 20, 13, 39, 53, 651417), 'image': 'https://scontent.fhlz2-1.fna.fbcdn.net/v/t1.6435-9/fr/cp0/e15/q65/58745049_2257182057699568_1761478225390731264_n.jpg?_nc_cat=111&ccb=1-3&_nc_sid=8024bb&_nc_ohc=ygH2fPmfQpAAX92ABYY&_nc_ht=scontent.fhlz2-1.fna&tp=14&oh=7a8a7b4904deb55ec696ae255fff97dd&oe=60A36717', 'images': ['https://scontent.fhlz2-1.fna.fbcdn.net/v/t1.6435-9/fr/cp0/e15/q65/58745049_2257182057699568_1761478225390731264_n.jpg?_nc_cat=111&ccb=1-3&_nc_sid=8024bb&_nc_ohc=ygH2fPmfQpAAX92ABYY&_nc_ht=scontent.fhlz2-1.fna&tp=14&oh=7a8a7b4904deb55ec696ae255fff97dd&oe=60A36717'], 'is_live': False, 'likes': 3509, 'link': 'https://www.nintendo.com/amiibo/line-up/', 'post_id': '2257188721032235', 'post_text': 'Don’t let this diminutive version of the Hero of Time fool you, ', 'post_url': 'https://www.facebook.com/story.php?story_fbid=2257188721032235&id=119240841493711', 'reactions': {'haha': 22, 'like': 2657, 'love': 706, 'sorry': 1, 'wow': 123}, 'reactors': None, 'shared_post_id': None, 'shared_post_url': None, 'shared_text': '', 'shared_time': None, 'shared_user_id': None, 'shared_username': None, 'shares': 441, 'text': 'Don’t let this diminutive version of the Hero of Time fool you, ', 'time': datetime.datetime(2019, 4, 30, 5, 0, 1), 'user_id': '119240841493711', 'username': 'Nintendo', 'video': None, 'video_id': None, 'video_thumbnail': None, 'w3_fb_url': 'https://www.facebook.com/Nintendo/posts/2257188721032235' } ``` -------------------------------- ### Download Comments of a Facebook Post Source: https://github.com/kritsanan1/facebook-scraper/blob/master/README.md A practical example demonstrating how to download comments for a specific Facebook post using the `get_posts` function. It utilizes the `post_urls` parameter to specify the post and the `options` dictionary to enable comment extraction with a limit and progress bar. It also shows how to extract the post data from the generator. ```python """ Download comments for a public Facebook post. """ import facebook_scraper as fs # get POST_ID from the URL of the post which can have the following structure: # https://www.facebook.com/USER/posts/POST_ID # https://www.facebook.com/groups/GROUP_ID/posts/POST_ID POST_ID = "pfbid02NsuAiBU9o1ouwBrw1vYAQ7khcVXvz8F8zMvkVat9UJ6uiwdgojgddQRLpXcVBqYbl" # number of comments to download -- set this to True to download all comments MAX_COMMENTS = 100 # get the post (this gives a generator) gen = fs.get_posts( post_urls=[POST_ID], options={"comments": MAX_COMMENTS, "progress": True} ) # take 1st element of the generator which is the post we requested post = next(gen) ``` -------------------------------- ### Facebook Scraper Error Handling and Exceptions Source: https://context7.com/kritsanan1/facebook-scraper/llms.txt Provides a comprehensive example of how to handle various exceptions that can occur during Facebook scraping. It demonstrates using try-except blocks to catch specific errors like `NotFound`, `TemporarilyBanned`, `AccountDisabled`, `InvalidCookies`, `LoginRequired`, `LoginError`, and `UnexpectedResponse`, along with a general exception handler for other issues. ```python from facebook_scraper import get_posts, exceptions import time try: for post in get_posts('page_name', pages=10, cookies='cookies.txt'): print(post['text']) except exceptions.NotFound: print("Error: Page, post, or profile not found") except exceptions.TemporarilyBanned: print("Error: Your IP has been temporarily banned by Facebook") print("Wait a few hours or use a different IP/proxy") except exceptions.AccountDisabled: print("Error: Your Facebook account has been disabled") except exceptions.InvalidCookies: print("Error: The provided cookies are invalid or expired") print("Make sure cookies include 'c_user' and 'xs'") except exceptions.LoginRequired: print("Error: This content requires authentication") print("Please provide valid cookies or credentials") except exceptions.LoginError: print("Error: Failed to log in with provided credentials") except exceptions.UnexpectedResponse: print("Error: Facebook returned an unexpected response") except Exception as e: print(f"Unexpected error: {e}") # Implement retry logic time.sleep(10) ``` -------------------------------- ### Basic Usage of get_posts Function Source: https://github.com/kritsanan1/facebook-scraper/blob/master/README.md Example of how to use the `get_posts` function to scrape posts from a given Facebook page name. It demonstrates iterating through the returned posts and printing a snippet of the post text. The `pages` parameter controls the number of pages to scrape. ```python from facebook_scraper import get_posts for post in get_posts('nintendo', pages=1): print(post['text'][:50]) ``` -------------------------------- ### Facebook Comment and Reply Example Data Structure Source: https://github.com/kritsanan1/facebook-scraper/blob/master/README.md This Python snippet illustrates the data structure for a Facebook comment, including details about the commenter, the comment text, time, image, and reactions. It also shows the nested structure for replies, each with similar comment-level details. ```python { 'comment_id': '1417925635669547', 'comment_url': 'https://facebook.com/1417925635669547', 'commenter_id': '100009665948953', 'commenter_url': 'https://facebook.com/tw0311?eav=AfZuEAOAat6KRX5WFplL0SNA4ZW78Z2O7W_sjwMApq67hZxXDwXh2WF2ezhICX1LCT4&fref=nf&rc=p&refid=52&__tn__=R&paipv=0', 'commenter_name': 'someone', 'commenter_meta': None, 'comment_text': 'something', 'comment_time': datetime.datetime(2023, 6, 23, 0, 0), 'comment_image': 'https://scontent.ftpe8-2.fna.fbcdn.net/m1/v/t6/An_UvxJXg9tdnLU3Y5qjPi0200MLilhzPXUgxzGjQzUMaNcmjdZA6anyrngvkdub33NZzZhd51fpCAEzNHFhko5aKRFP5fS1w_lKwYrzcNLupv27.png?ccb=10-5&oh=00_AfCdlpCwAg-SHhniMQ16uElFHh-OG8kGGmLAzvOY5_WZgw&oe=64BE3279&_nc_sid=7da55a', 'comment_reactors': [ {'name': 'Tom', 'link': 'https://facebook.com/ryan.dwayne?eav=AfaxdKIITTXyZj4H-eanXQgoxzOa8Vag6XkGXXDisGzh_W74RYZSXxlFZBofR4jUIOg&fref=pb&paipv=0', 'type': 'like'}, {'name': 'Macy', 'link': 'https://facebook.com/profile.php?id=100000112053053&eav=AfZ5iWlNN-EjjSwVNQl7E2HiVp25AUZMqfoPvLRZGnbUAQxuLeN8nl6xnnQTJB3uxDM&fref=pb&paipv=0', 'type': 'like'} ], 'comment_reactions': {'like': 2}, 'comment_reaction_count': 2, 'replies': [ { 'comment_id': '793761608817229', 'comment_url': 'https://facebook.com/793761608817229', 'commenter_id': '100022377272712', 'commenter_url': 'https://facebook.com/brizanne.torres?eav=Afab9uP4ByIMn1xaYK0UDd1SRU8e5Zu7faKEx6qTzLKD2vp_bB1xLDGvTwEd6u8A7jY&fref=nf&rc=p&__tn__=R&paipv=0', 'commenter_name': 'David', 'commenter_meta': None, 'comment_text': 'something', 'comment_time': datetime.datetime(2023, 6, 23, 18, 0), 'comment_image': None, 'comment_reactors': [], 'comment_reactions': {'love': 2}, 'comment_reaction_count': None } ] } ``` -------------------------------- ### Get Facebook Group Information Source: https://github.com/kritsanan1/facebook-scraper/blob/master/README.md Retrieves information about a Facebook group using its name or ID. To view the list of admins, a login session (via cookies) is required. The function returns details such as the group's name, ID, member count, and type. ```python from facebook_scraper import get_group_info # Get information for a specific group group_data = get_group_info("makeupartistsgroup") # Get group information with cookies to see admins group_data_with_cookies = get_group_info("makeupartistsgroup", cookies="cookies.txt") ``` -------------------------------- ### Get Facebook Profile Information Source: https://github.com/kritsanan1/facebook-scraper/blob/master/README.md Extracts information from a Facebook profile's about section. It takes the account name or ID as input and can optionally use cookies for accessing logged-in specific data like Date of birth and Gender. Friends data can also be retrieved by setting the `friends` argument. ```python from facebook_scraper import get_profile # Get profile information for a user profile_data = get_profile("zuck") # Get profile information with cookies for potentially more data profile_data_with_cookies = get_profile("zuck", cookies="cookies.txt") # Get profile information including friends list profile_data_with_friends = get_profile("zuck", friends=True) # Get profile information with a limited number of friends profile_data_limited_friends = get_profile("zuck", friends=50) ``` -------------------------------- ### Configure HTTP/HTTPS Proxy for Facebook Scraper Source: https://context7.com/kritsanan1/facebook-scraper/llms.txt Shows how to set up proxy servers for scraping requests. This is useful for rotating IP addresses or routing traffic through specific network locations. The `set_proxy` function accepts a proxy URL and an optional SSL verification flag. It supports both HTTP/HTTPS and SOCKS proxies. ```python from facebook_scraper import set_proxy, get_posts # Configure proxy settings set_proxy( proxy='http://username:password@proxy.example.com:8080', verify=True # Set to False to disable SSL verification ) # Or with SOCKS proxy set_proxy( proxy='socks5://127.0.0.1:9050', # Tor proxy example verify=True ) # Now all requests will go through the proxy for post in get_posts('nintendo', pages=3): print(post['text'][:100]) ``` -------------------------------- ### Use Persistent Session with Cookies for Facebook Scraping Source: https://context7.com/kritsanan1/facebook-scraper/llms.txt Demonstrates how to save and reuse session cookies for persistent scraping. This method improves efficiency by avoiding repeated logins. It requires the `facebook_scraper` library and takes email, password, and a file path for cookies as input. ```python from facebook_scraper import use_persistent_session, get_posts use_persistent_session( email='your_email@example.com', password='your_password', cookies_file_path='.fb-cookies.pkl' ) # Subsequent calls will reuse the saved session for post in get_posts('any_page', pages=5): print(post['text']) ``` -------------------------------- ### Set Custom User Agent for Facebook Scraping Source: https://context7.com/kritsanan1/facebook-scraper/llms.txt Explains how to configure a custom user agent string. This allows the scraper to mimic different browsers or devices, potentially avoiding detection or accessing content restricted to specific user agents. The `set_user_agent` function takes the desired user agent string as input. ```python from facebook_scraper import set_user_agent, get_posts # Set custom user agent set_user_agent( 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) ' 'AppleWebKit/537.36 (KHTML, like Gecko) ' 'Chrome/120.0.0.0 Safari/537.36' ) # Scrape with custom user agent for post in get_posts('page_name', pages=5): print(post['post_id']) ``` -------------------------------- ### Write Facebook Posts to CSV with Resume Source: https://context7.com/kritsanan1/facebook-scraper/llms.txt Saves scraped posts directly to a CSV file, supporting automatic pagination and resume functionality for large-scale scraping. Can filter posts by regex and specify output keys. ```python import facebook_scraper as fs # Save posts to CSV with resume capability fs.write_posts_to_csv( group='676845025728409', page_limit=100, timeout=60, options={ 'allow_extra_requests': False, 'comments': 50 }, filename='./facebook_posts.csv', resume_file='next_page.txt', # Saves pagination state for resuming matching='.+', # Regex to filter posts (match all) not_matching='^Warning', # Exclude posts starting with "Warning" keys=[ 'post_id', 'text', 'time', 'likes', 'comments', 'shares', 'user_id', 'username', 'post_url' ], format='csv', # Can be 'csv' or 'json' days_limit=365 # Only fetch posts from last 365 days ) # For incremental scraping across multiple files for i in range(1, 11): try: fs.write_posts_to_csv( account='nintendo', page_limit=100, filename=f'./data/nintendo_batch_{i}.csv', resume_file='nintendo_cursor.txt', keys=['post_id', 'text', 'time', 'likes'], format='csv' ) print(f"Completed batch {i}") except Exception as e: print(f"Error in batch {i}: {e}") break ``` -------------------------------- ### Scrape Posts by Hashtag on Facebook Source: https://context7.com/kritsanan1/facebook-scraper/llms.txt Illustrates how to search for and retrieve posts containing a specific hashtag across Facebook. The `get_posts` function can be used with the `hashtag` parameter. It allows for specifying the number of pages to scrape, using cookies, and enabling extra options like `allow_extra_requests`. ```python from facebook_scraper import get_posts # Search for posts with a specific hashtag for post in get_posts( hashtag='gaming', pages=10, cookies='cookies.txt', options={'allow_extra_requests': True} ): print(f"Author: {post.get('username')}") print(f"Text: {post['text']}") print(f"Post URL: {post['post_url']}") print(f"Hashtags found: {post.get('hashtags', [])}") print("-" * 60) ``` -------------------------------- ### Search Facebook Posts by Keyword Source: https://context7.com/kritsanan1/facebook-scraper/llms.txt Performs a global search across Facebook to find posts that match specified keywords. Allows pagination and configuration of extra requests. ```python from facebook_scraper import get_posts_by_search # Search for posts containing specific keywords search_term = "nintendo switch" for post in get_posts_by_search( search_term, pages=5, cookies='cookies.txt', options={'allow_extra_requests': True} ): print(f"Found post by: {post.get('username', 'Unknown')}") print(f"Text: {post['text'][:200]}") print(f"URL: {post['post_url']}") print(f"Engagement: {post['likes']} likes") print("-" * 60) ``` -------------------------------- ### Manage Facebook Authentication Cookies Source: https://context7.com/kritsanan1/facebook-scraper/llms.txt Handles Facebook authentication using cookies loaded from files, extracted directly from browsers, or provided as dictionaries. Allows scraping of restricted content and clearing cookies. ```python from facebook_scraper import set_cookies, unset_cookies, get_posts import browser_cookie3 # Method 1: Load cookies from a file (Netscape format) set_cookies('cookies.txt') # Method 2: Extract cookies directly from browser set_cookies('from_browser') # Requires browser-cookie3 package # Method 3: Use cookie dictionary cookies_dict = { 'c_user': '100000123456789', 'xs': 'your_xs_cookie_value_here', 'datr': 'your_datr_value' } set_cookies(cookies_dict) # Use authenticated session to scrape restricted content for post in get_posts('private_page', pages=5): print(post['text']) # Clear cookies when done unset_cookies() ``` -------------------------------- ### Extract Reactors for a Facebook Post Source: https://context7.com/kritsanan1/facebook-scraper/llms.txt Retrieves a list of users who reacted to a specific Facebook post, including their names, profile links, and the type of reaction (e.g., like, love). ```python from facebook_scraper import get_reactors # Get all reactors for a specific post post_id = '3065154550235644' for reactor in get_reactors( post_id, cookies='cookies.txt' ): print(f"User: {reactor.get('name')}") print(f"Profile: {reactor.get('link')}") print(f"Reaction type: {reactor.get('type')}") # like, love, haha, wow, sad, angry ``` -------------------------------- ### Write Facebook Posts to CSV with Resume Source: https://github.com/kritsanan1/facebook-scraper/blob/master/README.md Saves scraped posts directly to CSV files, supporting resuming interrupted scraping sessions. This function is ideal for large-scale scraping as data is saved continuously. It allows specifying page limits, timeouts, filtering options (matching/not_matching regex), desired keys, file format, and a date limit for posts. ```python import facebook_scraper as fs GROUP_ID = "YOUR_GROUP_ID" # Example: Saves the first 100 pages of posts from a group to separate CSV files for i in range(1, 101): fs.write_posts_to_csv( group=GROUP_ID, page_limit=100, timeout=60, options={ 'allow_extra_requests': False }, filename=f'./data/messages_{i}.csv', resume_file='next_page.txt', matching='.+', not_matching='^Warning', keys=[ 'post_id', 'text', 'timestamp', 'time', 'user_id' ], format='csv', days_limit=3650 ) ``` -------------------------------- ### Scrape Photos from a Facebook Page Source: https://context7.com/kritsanan1/facebook-scraper/llms.txt Demonstrates how to extract photo posts from a specific Facebook page. The `get_photos` function retrieves photo-related data, including post IDs, image URLs, descriptions, timestamps, and likes. It can optionally use cookies and fetch extra information. ```python from facebook_scraper import get_photos # Get photos from a page for photo_post in get_photos( 'nintendo', pages=10, cookies='cookies.txt', extra_info=True ): print(f"Photo Post ID: {photo_post['post_id']}") # Access all images in the post if photo_post.get('images'): for img_url in photo_post['images']: print(f" Image URL: {img_url}") # Image descriptions if photo_post.get('images_description'): for desc in photo_post['images_description']: print(f" Description: {desc}") print(f"Posted: {photo_post['time']}") print(f"Likes: {photo_post['likes']}") ``` -------------------------------- ### Extract Facebook User Profile Information Source: https://context7.com/kritsanan1/facebook-scraper/llms.txt Retrieves detailed information from a Facebook user's profile such as name, bio, education, and work history. Requires cookies for full data access and can optionally fetch a limited number of friends. ```python from facebook_scraper import get_profile # Get profile information (requires cookies for full data) profile = get_profile('zuck', cookies='cookies.txt') print(f"Name: {profile.get('Name')}") print(f"About: {profile.get('About')}") print(f"Work: {profile.get('Work')}") print(f"Education: {profile.get('Education')}") print(f"Places lived: {profile.get('Places lived')}") print(f"Quotes: {profile.get('Favourite Quotes')}") # Extract friends list (limited to 100 friends) profile_with_friends = get_profile( 'username', cookies='cookies.txt', friends=100 ) if profile_with_friends.get('Friends'): for friend in profile_with_friends['Friends']: print(f"Friend: {friend['name']} - {friend['link']}") ``` -------------------------------- ### Extract Facebook Group Information Source: https://context7.com/kritsanan1/facebook-scraper/llms.txt Retrieves metadata about a Facebook group, including its name, member count, type, and administrators. Authentication might be required to list group administrators. ```python from facebook_scraper import get_group_info # Get group metadata group_info = get_group_info('makeupartistsgroup', cookies='cookies.txt') print(f"Group Name: {group_info['name']}") print(f"Group ID: {group_info['id']}") print(f"Members: {group_info['members']:,}") print(f"Type: {group_info['type']}") # List group administrators (requires authentication) if group_info.get('admins'): print("\nAdministrators:") for admin in group_info['admins']: print(f" - {admin['name']}: {admin['link']}") ``` -------------------------------- ### Extract and Process Facebook Post Comments Source: https://github.com/kritsanan1/facebook-scraper/blob/master/README.md This Python snippet demonstrates how to extract comments from a Facebook post object and iterate through them. It also shows how to access and process replies to each comment. Assumes a 'post' object with a 'comments_full' key containing a list of comments. ```python # extract the comments part comments = post['comments_full'] # process comments as you want... for comment in comments: # e.g. ...print them print(comment) # e.g. ...get the replies for them for reply in comment['replies']: print(' ', reply) ``` -------------------------------- ### Scrape Facebook Page Posts with Python Source: https://context7.com/kritsanan1/facebook-scraper/llms.txt Extracts posts from a public Facebook page using the page name or ID. Supports pagination, detailed reactions, comments, reactors, and progress display. It can optionally use cookies for authentication and specify the number of posts per page and comments per post. ```python from facebook_scraper import get_posts # Basic usage - get posts from a page for post in get_posts('nintendo', pages=5): print(f"Post ID: {post['post_id']}") print(f"Text: {post['text'][:100]}") print(f"Likes: {post['likes']}") print(f"Comments: {post['comments']}") print(f"Shares: {post['shares']}") print(f"Posted at: {post['time']}") print("-" * 50) # Advanced usage with reactions and comments for post in get_posts( 'nintendo', pages=10, timeout=60, cookies='cookies.txt', extra_info=True, # Get detailed reactions options={ 'comments': 50, # Get up to 50 comments per post 'reactors': True, # Get people who reacted 'progress': True, # Show progress bar 'allow_extra_requests': True, 'posts_per_page': 100 } ): # Access detailed reaction counts if post.get('reactions'): print(f"Reactions: {post['reactions']}") # Output: {'like': 2657, 'love': 706, 'haha': 22, 'wow': 123, 'sorry': 1} # Access comments if post.get('comments_full'): for comment in post['comments_full']: print(f"Comment by {comment['commenter_name']}: {comment['comment_text']}") # Access replies to comments for reply in comment.get('replies', []): print(f" Reply: {reply['comment_text']}") ``` -------------------------------- ### Extract Facebook Posts by URL or ID with Python Source: https://context7.com/kritsanan1/facebook-scraper/llms.txt Retrieves specific Facebook posts using their URLs or post IDs. This function allows for targeted data extraction, including images, videos, comments, and associated metadata. It supports specifying comment limits, progress display, and cookie-based authentication. ```python from facebook_scraper import get_posts # Extract specific posts by ID post_ids = [ 'pfbid02NsuAiBU9o1ouwBrw1vYAQ7khcVXvz8F8zMvkVat9UJ6uiwdgojgddQRLpXcVBqYbl', '3065154550235644' ] for post in get_posts( post_urls=post_ids, options={'comments': 100, 'progress': True}, cookies='cookies.txt' ): print(f"Post URL: {post['post_url']}") print(f"Author: {post['username']}") print(f"Text: {post['text']}") # Extract images if post.get('images'): for idx, image_url in enumerate(post['images']): print(f"Image {idx + 1}: {image_url}") # Extract video information if post.get('video'): print(f"Video URL: {post['video']}") print(f"Video thumbnail: {post['video_thumbnail']}") print(f"Duration: {post['video_duration_seconds']} seconds") # Process all comments if post.get('comments_full'): for comment in post['comments_full']: print(f"\nCommenter: {comment['commenter_name']}") print(f"Comment: {comment['comment_text']}") print(f"Time: {comment['comment_time']}") print(f"Reactions: {comment.get('comment_reactions', {})}") ``` -------------------------------- ### Scrape Facebook Group Posts with Python Source: https://context7.com/kritsanan1/facebook-scraper/llms.txt Extracts posts from public Facebook groups using the group ID. This function supports pagination and allows for the retrieval of comments and other post details. Cookie-based authentication is often necessary for accessing group content. ```python from facebook_scraper import get_posts # Scrape posts from a Facebook group group_id = '676845025728409' for post in get_posts( group=group_id, pages=10, cookies='cookies.txt', # Authentication often required for groups options={ 'comments': True, 'allow_extra_requests': True } ): print(f"Post ID: {post['post_id']}") print(f"Author: {post.get('username', 'Unknown')}") print(f"Content: {post['text']}") print(f"Engagement: {post['likes']} likes, {post['comments']} comments") # Handle shared posts if post.get('shared_post_id'): print(f"Shared from: {post['shared_username']}") print(f"Shared text: {post['shared_text']}") ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.