### CLI Usage Example Source: https://github.com/kevinzg/facebook-scraper/blob/master/README.md Example of how to use the facebook-scraper command-line interface to download posts from a specific page. Use `--help` for more options. ```sh $ facebook-scraper --filename nintendo_page_posts.csv --pages 10 nintendo ``` -------------------------------- ### Install Latest Release Source: https://github.com/kevinzg/facebook-scraper/blob/master/README.md Install the latest stable version of the facebook-scraper library from PyPI. ```sh pip install facebook-scraper ``` -------------------------------- ### Python Example: Basic Profile Info Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/api-reference-main.md Demonstrates how to retrieve basic profile information for a given Facebook account. ```python from facebook_scraper import get_profile # Basic profile info profile = get_profile('zuck') print(profile['Name'], profile['About']) ``` -------------------------------- ### Python Example: Get All Followers Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/api-reference-main.md Illustrates how to retrieve all followers for a specified Facebook page. ```python # Get all followers profile = get_profile('page_name', followers=True) ``` -------------------------------- ### Install Latest Master Branch Source: https://github.com/kevinzg/facebook-scraper/blob/master/README.md Install the most recent version directly from the master branch of the GitHub repository. ```sh pip install git+https://github.com/kevinzg/facebook-scraper.git ``` -------------------------------- ### Example Usage of Reaction Summary Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/types.md Iterates through posts to retrieve and print the total number of reactions and their distribution. ```python for post in get_posts('page', extra_info=True): reactions: dict = post.get('reactions', {}) if reactions: total = sum(reactions.values()) print(f"Post has {total} reactions: {reactions}") ``` -------------------------------- ### Python Example: Authenticated Profile with Friends Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/api-reference-main.md Shows how to fetch an authenticated user's profile and extract a limited number of friends. ```python # Authenticated profile with friends profile = get_profile('zuck', cookies='cookies.txt', friends=100) for friend in profile['Friends']: print(friend['name'], friend['id']) ``` -------------------------------- ### Get Posts from a Page Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/README.md Fetches posts from a specified page and prints their text and like counts. Use this to get basic post information. ```python from facebook_scraper import get_posts for post in get_posts('nintendo', pages=10): print(post['text']) print(f"Likes: {post['likes']}") ``` -------------------------------- ### Iterate Through Post Reactors Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/types.md Example of how to retrieve and print the name and reaction type for each reactor on a given post. ```python for reactor in get_reactors('post_id'): print(f"{reactor['name']} reacted with {reactor['type']}") ``` -------------------------------- ### Example Usage of Review Type Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/types.md Fetches page information including reviews and prints the username, rating, and text for each review. ```python info = get_page_info('restaurant', reviews=True) for review in info.get('reviews', []): print(f"{review['username']}: {review['rating']} stars") print(f" {review['text']}") ``` -------------------------------- ### Facebook Post Data Structure Example Source: https://github.com/kevinzg/facebook-scraper/blob/master/README.md This is an example of the data structure returned for a Facebook post. Note that some fields might be `None` depending on the post and scraping options. ```python {'available': True, 'comments': 459, 'comments_full': None, 'factcheck': None, 'fetched_time': datetime.datetime(2021, 4, 20, 13, 39, 53, 651417), 'image': 'https://scontent.fhlz2-1.fna.fbcdn.net/v/t1.6435-9/fr/cp0/e15/q65/58745049_2257182057699568_1761478225390731264_n.jpg?_nc_cat=111&ccb=1-3&_nc_sid=8024bb&_nc_ohc=ygH2fPmfQpAAX92ABYY&_nc_ht=scontent.fhlz2-1.fna&tp=14&oh=7a8a7b4904deb55ec696ae255fff97dd&oe=60A36717', 'images': ['https://scontent.fhlz2-1.fna.fbcdn.net/v/t1.6435-9/fr/cp0/e15/q65/58745049_2257182057699568_1761478225390731264_n.jpg?_nc_cat=111&ccb=1-3&_nc_sid=8024bb&_nc_ohc=ygH2fPmfQpAAX92ABYY&_nc_ht=scontent.fhlz2-1.fna&tp=14&oh=7a8a7b4904deb55ec696ae255fff97dd&oe=60A36717'], 'is_live': False, 'likes': 3509, 'link': 'https://www.nintendo.com/amiibo/line-up/', 'post_id': '2257188721032235', 'post_text': 'Don’t let this diminutive version of the Hero of Time fool you, ' 'Young Link is just as heroic as his fully grown version! Young ' 'Link joins the Super Smash Bros. series of amiibo figures!\n' '\n' 'https://www.nintendo.com/amiibo/line-up/', 'post_url': 'https://www.facebook.com/story.php?story_fbid=2257188721032235&id=119240841493711', 'reactions': {'haha': 22, 'like': 2657, 'love': 706, 'sorry': 1, 'wow': 123}, # if `extra_info` was set 'reactors': None, 'shared_post_id': None, 'shared_post_url': None, 'shared_text': '', 'shared_time': None, 'shared_user_id': None, 'shared_username': None, 'shares': 441, 'text': 'Don’t let this diminutive version of the Hero of Time fool you, ' 'Young Link is just as heroic as his fully grown version! Young Link ' 'joins the Super Smash Bros. series of amiibo figures!\n' '\n' 'https://www.nintendo.com/amiibo/line-up/', 'time': datetime.datetime(2019, 4, 30, 5, 0, 1), 'user_id': '119240841493711', 'username': 'Nintendo', 'video': None, 'video_id': None, 'video_thumbnail': None, 'w3_fb_url': 'https://www.facebook.com/Nintendo/posts/2257188721032235'} ``` -------------------------------- ### Python Examples for get_posts Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/api-reference-main.md Demonstrates various ways to use the `get_posts` function, including fetching from a page, using authentication, retrieving specific posts by URL, and extracting posts with comments. ```python from facebook_scraper import get_posts # Get posts from page for post in get_posts('nintendo', pages=1): print(post['text'], post['likes']) ``` ```python # Get posts with authentication for post in get_posts('example_page', cookies='cookies.txt', page_limit=5): print(post['post_id']) ``` ```python # Get specific posts by URL for post in get_posts(post_urls=['12345', '67890']): print(post) ``` ```python # Get posts with comments for post in get_posts('page', options={'comments': True}): for comment in post.get('comments_full', []): print(comment['comment_text']) ``` -------------------------------- ### Python Example: Simple CSV Output Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/api-reference-main.md Demonstrates how to use write_posts_to_csv to save a specified number of posts from a Facebook page to a CSV file. ```python from facebook_scraper import write_posts_to_csv # Simple CSV output write_posts_to_csv('nintendo', filename='posts.csv', page_limit=10) ``` -------------------------------- ### Python Example: JSON Output with Resume Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/api-reference-main.md Shows how to write posts from a Facebook group to a JSON file, enabling resume capability for pagination and specifying which post fields to include. ```python write_posts_to_csv( group=123456, filename='group_posts.json', format='json', resume_file='next_page.txt', page_limit=100, keys=['post_id', 'text', 'likes', 'time'] ) ``` -------------------------------- ### Get Posts with Comments and Reactions Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/README.md Fetches posts from a page, including comments and reactor information. Use this for detailed post analysis. ```python from facebook_scraper import get_posts options = { 'comments': True, 'reactors': 50, 'progress': True } for post in get_posts('page', options=options, page_limit=5): print(f"{post['username']}: {post['text'][:100]}") for comment in post.get('comments_full', []): print(f ``` -------------------------------- ### Define Options Type Alias with Example Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/types.md Defines a type alias for configuration options dictionaries, controlling advanced behavior in post extraction. Common keys include comments, reactors, progress, and posts_per_page. ```python Options = Dict[str, Any] ``` ```python options: Options = { 'comments': 50, 'reactors': True, 'progress': True, 'allow_extra_requests': False } for post in get_posts('page', options=options): print(post) ``` -------------------------------- ### get() Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/api-reference-facebook-scraper-class.md Handles internal HTTP GET/POST requests to specified URLs. It normalizes URLs, manages redirects, and applies proxy/timeout settings, returning a response object with HTML and text content. ```APIDOC ## get() ### Description Internal HTTP GET/POST request handler. Normalizes URLs, handles redirects, detects locale, removes comment markers, and applies configured proxy and timeout settings. ### Method `get(self, url: str, **kwargs)` ### Parameters #### Path Parameters - **url** (str) - Required - Absolute or relative URL #### Query Parameters - **post** (bool) - Optional - Use POST instead of GET - **kwargs** (various) - Optional - Additional request parameters ### Return type Response object with `.html` and `.text` attributes ### Behavior: - Normalizes relative URLs to mobile Facebook base URL - Handles video redirects - Detects and warns about non-English locale - Removes comment markers from HTML - Increments request counter - Applies configured proxy and timeout settings ### Raises: #### RequestException - Condition: HTTP error (including rate limiting) ``` -------------------------------- ### Export Posts to CSV with Resume Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/configuration.md This example demonstrates how to export scraped posts to a CSV file with the capability to resume the process from a progress file. It specifies the group ID, filename, format, page limit, resume file, and encoding. ```python from facebook_scraper import write_posts_to_csv # CSV export with resume capability write_posts_to_csv( group='123456789', filename='group_posts.csv', format='csv', page_limit=50, resume_file='progress.txt', encoding='utf-8', ) ``` -------------------------------- ### Catch LoginError Exception Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/errors.md Shows how to catch the LoginError exception, which occurs during login attempts. This example prints the error message and suggests checking credentials or retrying. ```python from facebook_scraper import use_persistent_session, LoginError try: use_persistent_session('email@example.com', 'password') except LoginError as e: print(f"Login failed: {e}") print("Check credentials or try again later") ``` -------------------------------- ### Netscape Cookie File Format Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/configuration.md Example of a Netscape HTTP Cookie file format, commonly used for storing browser cookies. ```text # Netscape HTTP Cookie File .facebook.com TRUE / TRUE 9999999999 c_user 123456789 .facebook.com TRUE / TRUE 9999999999 xs abcdef1234567890 ``` -------------------------------- ### Credentials Type Alias and Usage Example Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/types.md Defines a type alias for login credentials as an email and password tuple. Used for authenticated scraping. ```python Credentials = Tuple[str, str] ``` ```python credentials: Credentials = ('email@example.com', 'password') for post in get_posts('page', credentials=credentials): print(post) ``` -------------------------------- ### Python Example: Filtered CSV Output Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/api-reference-main.md Illustrates filtering Facebook posts using regex patterns for both inclusion and exclusion before writing them to a CSV file. ```python # Filter posts by regex write_posts_to_csv( 'page', filename='filtered.csv', matching='.*important.*', not_matching='^Sponsored', page_limit=50 ) ``` -------------------------------- ### Get Products from a Page Shop Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/api-reference-facebook-scraper-class.md Fetches products listed on a Facebook page's shop. Returns a list of dictionaries, each containing product details like name, link, image, and price. ```python def get_shop(self, page: str, **kwargs) -> list ``` ```python products = scraper.get_shop('apple') for product in products: print(product['name'], product['price']) ``` -------------------------------- ### Get User Profile Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/api-reference-facebook-scraper-class.md Fetches profile information for a given user account, identified by username or profile ID. Supports fetching friends, followers, etc. ```python profile = scraper.get_profile('zuck') print(profile['Name'], profile['Friend_count']) ``` -------------------------------- ### Export Posts to JSON with Subset of Fields Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/configuration.md This example shows how to export scraped posts to a JSON file, selecting only specific fields like post ID, text, likes, and time. It sets the page limit for the export. ```python from facebook_scraper import write_posts_to_csv # JSON export with subset of fields write_posts_to_csv( 'page', filename='posts.json', format='json', keys=['post_id', 'text', 'likes', 'time'], page_limit=20, ) ``` -------------------------------- ### Enable Debug Logging for Facebook Scraper Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/errors.md Enable detailed logging to troubleshoot errors. This setup allows you to view request/response details, status codes, and more in the logs. ```python from facebook_scraper import enable_logging, get_posts import logging # Enable debug logging enable_logging(logging.DEBUG) # Now all requests will be logged for post in get_posts('page'): print(post) ``` -------------------------------- ### Get Paginated Collection Items Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/api-reference-facebook-scraper-class.md An internal method to extract paginated lists such as likes or followers. Requires a starting URL and optionally accepts a limit. ```python def get_collection( self, more_url: str, limit: Optional[int] = None, **kwargs ) -> Iterator[Profile] ``` -------------------------------- ### Configure Custom Logger Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/configuration.md Set up a custom logging configuration, directing output to a file with a specified format and level. Ensure the custom logger is enabled. ```python import logging from facebook_scraper import enable_logging # Setup logging to file logging.basicConfig( filename='scraper.log', level=logging.DEBUG, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s' ) enable_logging(logging.DEBUG) ``` -------------------------------- ### Configure Proxy Settings Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/configuration.md Set up HTTP or HTTPS proxies for requests, including options for authentication and SSL verification. ```python from facebook_scraper import set_proxy # HTTP proxy set_proxy('http://proxy.example.com:8080') # HTTPS proxy set_proxy('https://proxy.example.com:443') # With credentials set_proxy('http://user:pass@proxy.example.com:8080') # Skip SSL verification set_proxy('http://proxy.example.com:8080', verify=False) ``` -------------------------------- ### Get Post Reactors Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/api-reference-facebook-scraper-class.md Retrieves users who have reacted to a specific Facebook post. Requires the post ID. ```python for reactor in scraper.get_reactors(2257188721032235): print(reactor['name'], reactor['type']) ``` -------------------------------- ### Get Page Information Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/api-reference-facebook-scraper-class.md Retrieves metadata, follower counts, and ratings for a Facebook page. Accepts page name or ID. ```python page = scraper.get_page_info('nintendo') print(page['followers'], page['likes']) ``` -------------------------------- ### Configure FacebookScraper Session Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/api-reference-facebook-scraper-class.md Demonstrates how to initialize FacebookScraper and directly configure its underlying HTMLSession's headers and request timeouts. ```python from facebook_scraper.facebook_scraper import FacebookScraper scraper = FacebookScraper() # Configure session directly scraper.session.headers['Custom-Header'] = 'value' scraper.requests_kwargs['timeout'] = 60 scraper.request_count # Number of requests made ``` -------------------------------- ### Get Page Reviews Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/api-reference-facebook-scraper-class.md Extracts reviews for a specified Facebook page. Each review includes reviewer details, text, rating, and timestamp. ```python for review in scraper.get_page_reviews('restaurant'): print(review['username'], review['text']) ``` -------------------------------- ### Configure Proxy Settings Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/api-reference-facebook-scraper-class.md Sets HTTP and HTTPS proxy for all outgoing requests. Includes an option to verify SSL certificates. The proxy connection is tested by fetching an IP address. ```python def set_proxy(self, proxy: str, verify: bool = True) ``` ```python scraper.set_proxy("http://proxy.example.com:8080") ``` -------------------------------- ### Product Object Structure Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/types.md Represents a product found in a shop. Includes name, product page link, image URL, and formatted price. ```python { 'name': str, # Product name 'link': str, # URL to product page 'image': str, # URL to product image 'price': str, # Price as formatted string } ``` -------------------------------- ### Get Group Information Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/api-reference-facebook-scraper-class.md Fetches information about a Facebook group, including name, member count, and potentially admins/members. Requires group ID. ```python group = scraper.get_group_info('123456789') print(group['name'], group['members']) ``` -------------------------------- ### Get Photo Posts from a Page Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/api-reference-facebook-scraper-class.md Extracts photo posts from a specified Facebook page or account. Supports additional arguments for pagination and other options. ```python def get_photos(self, account: str, **kwargs) -> Iterator[Post] ``` -------------------------------- ### Login and Persist Session Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/api-reference-main.md Log in with email and password, then persist authentication to disk for reuse. Attempts to load existing cookies first; if none are found, it logs in and saves new cookies. ```python from facebook_scraper import use_persistent_session, get_posts # First run: saves cookies use_persistent_session('email@example.com', 'password') # Subsequent runs: uses saved cookies for post in get_posts('page'): print(post) ``` -------------------------------- ### Basic Usage: Get Posts from a Page Source: https://github.com/kevinzg/facebook-scraper/blob/master/README.md Retrieve posts from a specified Facebook page. The `pages` parameter controls the number of pages to fetch. ```python from facebook_scraper import get_posts for post in get_posts('nintendo', pages=1): print(post['text'][:50]) ``` -------------------------------- ### Enable Basic Logging Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/configuration.md Enable the library's built-in logging. Choose the desired verbosity level from DEBUG (most verbose) to WARNING (least verbose). ```python from facebook_scraper import enable_logging import logging # Debug level (most verbose) enable_logging(logging.DEBUG) # Info level enable_logging(logging.INFO) # Warning level (least verbose) enable_logging(logging.WARNING) ``` -------------------------------- ### Extract Reactors from a Post Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/api-reference-main.md Use this function to get a list of users who reacted to a specific Facebook post. You can specify a timeout and provide cookies for authentication. ```python from facebook_scraper import get_reactors for reactor in get_reactors('2257188721032235'): print(reactor['name'], reactor['type']) ``` -------------------------------- ### Get Friends from User Profile Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/api-reference-facebook-scraper-class.md Retrieves a list of friends from a specified user profile. Supports limiting the number of friends fetched via kwargs. ```python def get_friends(self, account: str, **kwargs) -> Iterator[Profile] ``` -------------------------------- ### Get Facebook Group Information Source: https://github.com/kevinzg/facebook-scraper/blob/master/README.md Extracts information about a Facebook group using its name or ID. Login with cookies is required to see the list of admins. ```python from facebook_scraper import get_group_info get_group_info("makeupartistsgroup") # or get_group_info("makeupartistsgroup", cookies="cookies.txt") ``` -------------------------------- ### Iterate and Print Products Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/types.md Iterates through products obtained from get_shop() and prints their name and price. Ensure get_shop() is imported and called with a valid identifier. ```python for product in get_shop('apple'): print(f"{product['name']}: {product['price']}") ``` -------------------------------- ### Extract Reactors and Reactions Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/api-reference-extractors.md Shows how to retrieve reactor information, including their name and the type of reaction they gave. Requires the 'reactors': True option and limits to the first 5 reactors for brevity. ```python from facebook_scraper import get_posts options = {'reactors': True} for post in get_posts('page', options=options): if post.get('reactors'): print(f"Post has {len(post['reactors'])} reactions:") for reactor in post['reactors'][:5]: # Show first 5 print(f" {reactor['name']} reacted with {reactor['type']}") ``` -------------------------------- ### Configure Proxy for Requests Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/api-reference-main.md Configure HTTP/HTTPS proxy for requests. Specify the proxy URL and optionally whether to verify SSL certificates. ```python def set_proxy(proxy, verify=True) ``` -------------------------------- ### Get Posts by Hashtag Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/api-reference-facebook-scraper-class.md Retrieves posts that match a specific hashtag. The hashtag should be provided as a string without the '#' symbol. Various options can be passed via kwargs. ```python def get_posts_by_hashtag(self, hashtag: str, **kwargs) -> Iterator[Post] ``` -------------------------------- ### Get Posts from a Facebook Page Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/api-reference-facebook-scraper-class.md Retrieves posts from a specified Facebook page or account. Accepts various keyword arguments that are passed to internal iterators and extractors. ```python def get_posts(self, account: str, **kwargs) -> Iterator[Post] ``` ```python for post in scraper.get_posts('nintendo', page_limit=5): print(post) ``` -------------------------------- ### Catch NotFound Exception Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/errors.md Demonstrates how to catch the NotFound exception when using the get_posts function. This is useful for handling cases where the requested content does not exist. ```python from facebook_scraper import get_posts, NotFound try: for post in get_posts('nonexistent_page'): print(post) except NotFound as e: print(f"Page not found: {e}") ``` -------------------------------- ### Iterate Through Post Comments and Replies Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/types.md Demonstrates fetching posts with comments and then iterating through each comment and its replies to print commenter names and comment text. ```python for post in get_posts('page', options={'comments': True}): for comment in post.get('comments_full', []): print(f"{comment['commenter_name']}: {comment['comment_text']}") for reply in comment['replies']: print(f ``` -------------------------------- ### Fast Pagination with Minimal Extraction Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/api-reference-pagination.md Use minimal extraction options to speed up pagination when only post IDs are needed. Set 'allow_extra_requests', 'comments', and 'reactors' to False for maximum speed. ```python from facebook_scraper import get_posts # Minimal extraction for speed options = { 'allow_extra_requests': False, 'comments': False, 'reactors': False, } for post in get_posts('page', options=options, page_limit=5): print(post['post_id']) # Fast: skips detailed extraction ``` -------------------------------- ### Fast Scraping Configuration Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/configuration.md Use this configuration for quick scraping when minimal data is required. It disables extra requests, comments, and reactors, and sets a short timeout. ```python from facebook_scraper import get_posts options = { 'allow_extra_requests': False, 'comments': False, 'reactors': False, } for post in get_posts('page', options=options, page_limit=5, timeout=15): print(post['text']) ``` -------------------------------- ### Get Page Information Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/api-reference-main.md Extracts basic information about a Facebook page. Use this function to retrieve details like name, ID, follower count, and contact information. ```python from facebook_scraper import get_page_info # Basic page info info = get_page_info('nintendo') print(f"Page: {info['name']}, Followers: {info['followers']}") # With reviews info = get_page_info('restaurant_page', reviews=True) for review in info['reviews']: print(review['username'], review['rating']) ``` -------------------------------- ### get_shop() Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/api-reference-facebook-scraper-class.md Fetches product information from a Facebook page's shop section. Returns a list of dictionaries, where each dictionary represents a product with its name, link, image, and price. ```APIDOC ## get_shop() ### Description Get shop/products from a page. Returns a list of dictionaries, where each product contains its name, link, image URL, and price string. ### Method `get_shop(self, page: str, **kwargs)` ### Parameters #### Path Parameters - **page** (str) - Required - Page name or ID ### Return type `list` – List of product dictionaries **Each product contains:** - `name` (str): Product name - `link` (str): Product URL - `image` (str): Product image URL - `price` (str): Price string ### Request Example ```python products = scraper.get_shop('apple') for product in products: print(product['name'], product['price']) ``` ``` -------------------------------- ### Get Posts from a Facebook Group Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/api-reference-facebook-scraper-class.md Fetches posts from a given Facebook group ID. Additional parameters like page limit and other options can be passed via kwargs. ```python def get_group_posts(self, group: Union[str, int], **kwargs) -> Iterator[Post] ``` -------------------------------- ### Extract Friends List Source: https://github.com/kevinzg/facebook-scraper/blob/master/README.md Retrieves a list of friends for a given Facebook profile. Set the `friends` argument to `True` to get all friends, or to an integer to limit the number of friends retrieved. ```python get_profile("zuck", friends=True) # Or get_profile("zuck", friends=50) ``` -------------------------------- ### FacebookScraper Constructor Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/api-reference-facebook-scraper-class.md Initializes the FacebookScraper. You can provide an existing HTMLSession or additional keyword arguments for requests. ```python def __init__(self, session=None, requests_kwargs=None) ``` ```python from facebook_scraper.facebook_scraper import FacebookScraper scraper = FacebookScraper() # or with custom session scraper = FacebookScraper(requests_kwargs={'timeout': 60}) ``` -------------------------------- ### PostExtractor Constructor Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/api-reference-extractors.md Initializes the PostExtractor with an HTML element, extraction options, an HTTP request function, and optional full post HTML for context. ```python def __init__( self, element: Element, options: dict, request_fn: RequestFunction, full_post_html=None ) ``` -------------------------------- ### Download Comments for a Public Facebook Post Source: https://github.com/kevinzg/facebook-scraper/blob/master/README.md This script downloads comments for a given Facebook post ID. Ensure you have the `facebook_scraper` library installed. Set `MAX_COMMENTS` to `True` to download all comments. ```python """ Download comments for a public Facebook post. """ import facebook_scraper as fs # get POST_ID from the URL of the post which can have the following structure: # https://www.facebook.com/USER/posts/POST_ID # https://www.facebook.com/groups/GROUP_ID/posts/POST_ID POST_ID = "pfbid02NsuAiBU9o1ouwBrw1vYAQ7khcVXvz8F8zMvkVat9UJ6uiwdgojgddQRLpXcVBqYbl" # number of comments to download -- set this to True to download all comments MAX_COMMENTS = 100 # get the post (this gives a generator) gen = fs.get_posts( post_urls=[POST_ID], options={\"comments\": MAX_COMMENTS, \"progress\": True} ) # take 1st element of the generator which is the post we requested post = next(gen) # extract the comments part comments = post['comments_full'] # process comments as you want... for comment in comments: # e.g. ...print them print(comment) # e.g. ...get the replies for them for reply in comment['replies']: print(' ', reply) ``` -------------------------------- ### Fast Extraction (Minimal Data) Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/api-reference-extractors.md Use this method for quick retrieval of basic post information with minimal HTTP requests. It skips comments, reactors, and extra requests. ```python from facebook_scraper import get_posts options = { 'allow_extra_requests': False, # Skip extra requests 'comments': False, # Skip comments 'reactors': False, # Skip reactors } for post in get_posts('page', options=options): print(post['text'], post['likes']) ``` -------------------------------- ### Minimal Facebook Scraper Options Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/configuration.md Shows how to configure the scraper with minimal options for faster execution by disabling comments and extra requests, combined with a page limit. ```python # Minimal options lean_options = { 'comments': False, 'allow_extra_requests': False, # Faster but less data } for post in get_posts('page', options=lean_options, page_limit=5): print(post) ``` -------------------------------- ### Comprehensive Extraction (Full Data) Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/api-reference-extractors.md Employ this method to retrieve complete post details, including all comments, reactors, and high-quality videos. This approach involves more HTTP requests and is slower. ```python from facebook_scraper import get_posts options = { 'allow_extra_requests': True, # Get detailed info 'comments': True, # Get all comments 'reactors': True, # Get all reactors 'progress': True, # Show progress 'youtube_dl': True, # High-quality videos } for post in get_posts('page', options=options): print(post) ``` -------------------------------- ### Comprehensive Scraping Configuration Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/configuration.md This configuration is for maximum data extraction, enabling all available features including extra requests, comments, reactors, progress tracking, and YouTube downloads. It also sets a longer timeout. ```python from facebook_scraper import get_posts options = { 'allow_extra_requests': True, 'comments': True, 'reactors': True, 'progress': True, 'youtube_dl': True, } for post in get_posts('page', options=options, page_limit=None, timeout=60): print(post) ``` -------------------------------- ### Balanced Scraping Configuration Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/configuration.md This configuration offers a balance between data comprehensiveness and scraping speed. It enables comments and reactors up to a specified limit and includes progress tracking. ```python from facebook_scraper import get_posts options = { 'comments': 50, 'reactors': 20, 'progress': True, } for post in get_posts('page', options=options, page_limit=10, timeout=30): print(post['text'], post['likes']) ``` -------------------------------- ### Define Post Type Alias with Example Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/types.md Defines a type alias for post dictionaries returned by scraping functions. It includes standard fields like post_id, text, time, likes, comments, and shares. ```python Post = Dict[str, Any] ``` ```python for post in get_posts('page'): p: Post = post print(p['username'], p['likes'], p['text'][:50]) ``` -------------------------------- ### Resume Scraping from a Checkpoint Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/api-reference-pagination.md Utilize `resume_file` with `write_posts_to_csv` to save and later resume scraping progress. The first run saves the checkpoint, and subsequent runs continue from where they left off. ```python from facebook_scraper import write_posts_to_csv # First run: saves checkpoint write_posts_to_csv( 'page', filename='posts.csv', resume_file='checkpoint.txt', page_limit=10 ) # Second run: continues from checkpoint write_posts_to_csv( 'page', filename='posts2.csv', resume_file='checkpoint.txt', page_limit=10 ) ``` -------------------------------- ### Internal HTTP GET/POST Request Handler Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/api-reference-facebook-scraper-class.md Handles internal HTTP GET or POST requests to Facebook. Normalizes URLs, manages redirects, checks locale, and applies proxy/timeout settings. Raises RequestException on HTTP errors. ```python def get(self, url: str, **kwargs) ``` -------------------------------- ### Basic Facebook Scraper Options Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/configuration.md Demonstrates setting basic options for comment extraction, reactor limits, and progress bar display when fetching posts from a Facebook page. ```python from facebook_scraper import get_posts # Basic options options = { 'comments': True, # Get all comments 'reactors': 50, # Get max 50 reactors 'progress': True, # Show progress bar } for post in get_posts('page', options=options): print(post) ``` -------------------------------- ### Use Shorter Timeouts Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/README.md Adjust the 'timeout' parameter in get_posts for faster networks to reduce waiting times. ```python for post in get_posts('page', timeout=15): print(post) ``` -------------------------------- ### Public API Entry-Point Functions Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/MANIFEST.txt This section covers the main functions users can call to interact with the facebook-scraper library. It includes functions for fetching various types of content and performing searches. ```APIDOC ## Public API Functions ### Description Provides access to core data scraping functionalities for Facebook. ### Functions - **get_posts()**: Retrieves posts from a profile, group, or page. - **get_profile()**: Fetches detailed information about a user profile. - **get_group_info()**: Retrieves information about a Facebook group. - **get_page_info()**: Fetches details about a Facebook page. - **get_photos()**: Retrieves photos associated with a profile, group, or page. - **get_shop()**: Fetches product information from a Facebook shop. - **search()**: Performs searches within Facebook. ### Parameters Detailed parameter tables and examples for each function are available in the `api-reference-main.md` file. ### Return Value Each function returns data structures specific to the requested content (e.g., list of posts, profile details, group info, page info, photos, shop products, search results). ### Examples Refer to `api-reference-main.md` for complete parameter tables and examples for each function. ``` -------------------------------- ### Get Profile Information Source: https://github.com/kevinzg/facebook-scraper/blob/master/README.md Extracts basic profile information from a Facebook account. Pass the account name or ID as the first parameter. Optionally, provide a `cookies.txt` file for access to more detailed information like date of birth and gender. ```python from facebook_scraper import get_profile get_profile("zuck") # Or get_profile("zuck", cookies="cookies.txt") ``` -------------------------------- ### Debug UnexpectedResponse with User Agent and Logging Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/errors.md Shows how to debug UnexpectedResponse errors by setting a different user agent and enabling debug logging to inspect the actual response. ```python from facebook_scraper.facebook_scraper import FacebookScraper scraper = FacebookScraper() scraper.set_user_agent("Mozilla/5.0...") # Different user agent enable_logging() # Enable debug logging to see actual response try: info = scraper.get_group_info('group_id') except UnexpectedResponse as e: # Check console output for debug details pass ``` -------------------------------- ### FacebookScraper Constructor Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/api-reference-facebook-scraper-class.md Initializes the FacebookScraper class. It can optionally accept a pre-configured HTMLSession or additional keyword arguments for HTTP requests. ```APIDOC ## FacebookScraper Constructor ### Description Initializes the FacebookScraper class. It can optionally accept a pre-configured HTMLSession or additional keyword arguments for HTTP requests. ### Method __init__ ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```python from facebook_scraper.facebook_scraper import FacebookScraper scraper = FacebookScraper() # or with custom session scraper = FacebookScraper(requests_kwargs={'timeout': 60}) ``` ### Response #### Success Response None #### Response Example None ``` -------------------------------- ### Configure Posts Per Page Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/configuration.md Adjust the number of posts fetched per page request. The maximum allowed value is 200. ```python from facebook_scraper import get_posts options = { 'posts_per_page': 200 # Maximum allowed } for post in get_posts('page', options=options): print(post) ``` -------------------------------- ### Resume Scraping from Checkpoint Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/configuration.md Save and resume scraping progress using a checkpoint file. The first run saves the checkpoint, and subsequent runs resume from it. ```python from facebook_scraper import write_posts_to_csv # First run: fetches and saves checkpoint write_posts_to_csv( 'page', filename='posts.csv', resume_file='checkpoint.txt', page_limit=100 ) # Second run: resumes from saved checkpoint write_posts_to_csv( 'page', filename='posts.csv', # Different file resume_file='checkpoint.txt', # Same checkpoint page_limit=100 ) ``` -------------------------------- ### set_proxy() Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/api-reference-facebook-scraper-class.md Configures HTTP and HTTPS proxy settings for the scraper. It also tests the proxy connection. ```APIDOC ## set_proxy() ### Description Configures HTTP and HTTPS proxy settings for the scraper. It also tests the proxy connection. ### Method set_proxy ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```python scraper.set_proxy("http://proxy.example.com:8080") ``` ### Response #### Success Response None #### Response Example None ``` -------------------------------- ### Set Cookies for Authentication Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/README.md Use the 'set_cookies' function with a 'cookies.txt' file to authenticate using exported browser cookies. ```python from facebook_scraper import set_cookies, get_posts set_cookies('cookies.txt') for post in get_posts('private_page'): print(post) ``` -------------------------------- ### Execute Callback for Each Page Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/api-reference-pagination.md Provide a `request_url_callback` function to execute custom logic for each page fetched. This is useful for logging or implementing rate limiting. ```python from facebook_scraper import get_posts import time def log_and_delay(url): print(f"Fetching: {url}") time.sleep(1) # Rate limiting for post in get_posts('page', request_url_callback=log_and_delay): print(post['post_id']) ``` -------------------------------- ### Use Persistent Session for Authentication Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/README.md Employ 'use_persistent_session' with email and password for authenticated scraping. ```python from facebook_scraper import use_persistent_session, get_posts use_persistent_session('email@example.com', 'password') for post in get_posts('page'): print(post) ``` -------------------------------- ### Set Custom User-Agent Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/api-reference-facebook-scraper-class.md Allows setting a custom User-Agent string for HTTP requests. This can be useful for mimicking different browsers or devices. ```python def set_user_agent(self, user_agent: str) ``` ```python scraper.set_user_agent("Mozilla/5.0 (iPhone; CPU iPhone OS 14_0 like Mac OS X)") ``` -------------------------------- ### Comprehensive Error Handling for get_posts Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/errors.md A robust function to fetch posts with handling for NotFound, TemporarilyBanned (with exponential backoff), LoginRequired (with cookie retry), and UnexpectedResponse. Use this for critical scraping tasks. ```python from facebook_scraper import ( get_posts, NotFound, TemporarilyBanned, LoginRequired, UnexpectedResponse, InvalidCookies, set_cookies, enable_logging ) import time enable_logging() def safe_get_posts(page_name, max_retries=3): """Get posts with comprehensive error handling""" for attempt in range(max_retries): try: return get_posts(page_name, page_limit=10) except NotFound: print(f"Page '{page_name}' not found") return None except TemporarilyBanned: if attempt < max_retries - 1: wait_time = 60 * (2 ** attempt) # Exponential backoff print(f"Rate limited. Waiting {wait_time}s...") time.sleep(wait_time) else: print("Too many rate limit attempts") raise except LoginRequired: print("Login required. Setting cookies...") try: set_cookies('cookies.txt') continue # Retry with cookies except InvalidCookies: print("Invalid cookies file") return None except UnexpectedResponse as e: print(f"Unexpected response: {e}") print("Facebook may have changed its structure") return None return None # Usage posts_gen = safe_get_posts('nintendo') if posts_gen: for post in posts_gen: print(post['text']) ``` -------------------------------- ### Set Cookies from Browser Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/configuration.md Loads cookies directly from the default browser using the `set_cookies` function. Requires the `browser-cookie3` package. ```python from facebook_scraper import set_cookies # Automatically extract from default browser set_cookies("from_browser") ``` -------------------------------- ### Enable Debug Logging Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/api-reference-main.md Enable debug logging output to the console. You can specify the log level, with DEBUG being the default. ```python from facebook_scraper import enable_logging import logging enable_logging(logging.INFO) ``` -------------------------------- ### Retry Logic for Pagination Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/api-reference-pagination.md Illustrates the exponential backoff and mode switching strategy used during retries for failed HTTP requests. This logic is applied when fetching pages. ```text Attempt 1: Wait 2s, retry Attempt 2: Wait 4s, retry Attempt 3: Wait 6s, switch to noscript mode, retry Attempt 4: Wait 8s, retry Attempt 5: Wait 10s, retry Attempt 6: Wait 12s, retry Attempt 7: Give up, raise error ``` -------------------------------- ### get_profile Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/api-reference-facebook-scraper-class.md Fetches profile information for a given user account, identified by username or profile ID. Returns a dictionary containing profile fields. ```APIDOC ## get_profile() ### Description Get profile information for a user. ### Method Signature ```python def get_profile(self, account: str, **kwargs) -> Profile ``` ### Parameters #### Path Parameters * **account** (str) - Required - Username or profile ID * **kwargs** (various) - Optional - friends, followers, following, likes params ### Return type `Profile` – Dictionary with profile fields ### Example ```python profile = scraper.get_profile('zuck') print(profile['Name'], profile['Friend_count']) ``` ``` -------------------------------- ### Select Specific Fields for Extraction Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/configuration.md Specify which fields to save when writing posts to a CSV file. Use the 'keys' parameter with a list of desired field names. ```python from facebook_scraper import write_posts_to_csv # Save only specific fields write_posts_to_csv( 'page', filename='posts.csv', keys=[ 'post_id', 'username', 'text', 'likes', 'time' ] ) ``` -------------------------------- ### Use Persistent Session for Scraping Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/configuration.md Establishes a persistent session using email and password, saving cookies for future use. Subsequent runs will automatically use these saved cookies. ```python from facebook_scraper import use_persistent_session, get_posts # First run use_persistent_session('email@example.com', 'password') # Subsequent runs use saved cookies for post in get_posts('page'): print(post) ``` -------------------------------- ### Progressive Fetching with Rate Limiting Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/api-reference-pagination.md Fetches posts progressively from a page, yielding each post and implementing a rate-limiting delay after a certain number of posts are fetched. ```python from facebook_scraper import get_posts import time def progressive_scrape(page, posts_per_batch=50, total_posts=1000): """Fetch posts progressively with rate limiting""" posts_fetched = 0 batch_count = 0 for post in get_posts(page, page_limit=None): yield post posts_fetched += 1 batch_count += 1 if batch_count % 50 == 0: print(f"Fetched {posts_fetched} posts, rate limiting...") time.sleep(5) if posts_fetched >= total_posts: break for post in progressive_scrape('page', total_posts=500): print(post['post_id']) ``` -------------------------------- ### Catch LoginRequired Exception Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/errors.md Demonstrates how to catch the LoginRequired exception and attempt to retry the operation after setting authentication cookies. Use this when content access is blocked due to missing login. ```python from facebook_scraper import get_posts, LoginRequired, set_cookies try: for post in get_posts('private_page'): print(post) except LoginRequired: print("Login required for this content") set_cookies('cookies.txt') # Retry with authentication for post in get_posts('private_page'): print(post) ``` -------------------------------- ### Extract Comments and Replies Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/api-reference-extractors.md Demonstrates how to fetch and iterate through comments and nested replies for each post. Requires the 'comments': True option. ```python from facebook_scraper import get_posts options = {'comments': True} for post in get_posts('page', options=options): if post.get('comments_full'): print(f"Post has {len(post['comments_full'])} comments:") for comment in post['comments_full']: print(f" {comment['commenter_name']}: {comment['comment_text']}") # Nested replies for reply in comment.get('replies', []): print(f" └ {reply['commenter_name']}: {reply['comment_text']}") ``` -------------------------------- ### Resume Interrupted Scraping with Checkpoint File Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/README.md Continue a scraping process from where it left off by specifying a resume_file. This is useful for long-running scraping tasks. ```python from facebook_scraper import write_posts_to_csv # Interrupted? Resume with checkpoint file write_posts_to_csv( 'page', filename='posts.csv', resume_file='checkpoint.txt', page_limit=100 ) ``` -------------------------------- ### get_posts() Source: https://github.com/kevinzg/facebook-scraper/blob/master/_autodocs/configuration.md Fetches posts from a specified account, group, hashtag, or by direct post URLs. Supports various options for filtering, pagination, and authentication. ```APIDOC ## get_posts() ### Description Fetches posts from a specified account, group, hashtag, or by direct post URLs. Supports various options for filtering, pagination, and authentication. ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Function Signature `get_posts(account: str = None, group: int/str = None, post_urls: list[str] = None, hashtag: str = None, timeout: int = 30, page_limit: int = 10, credentials: tuple[str, str] = None, cookies: str/dict/CookieJar = None, extra_info: bool = False, youtube_dl: bool = False, start_url: str = None, request_url_callback: callable = None, options: dict = {})` ### Parameters Details - **account** (str) - Page name, profile, or ID (one required) - **group** (int/str) - Group ID (alternative to account) - **post_urls** (list[str]) - Direct post URLs (alternative to account/group) - **hashtag** (str) - Hashtag search (alternative to account/group) - **timeout** (int) - Request timeout in seconds - **page_limit** (int) - Number of pages to fetch; None for all - **credentials** (tuple[str, str]) - (email, password) for login - **cookies** (str/dict/CookieJar) - Authentication cookies - **extra_info** (bool) - Extract reactions (deprecated, use options['reactions']) - **youtube_dl** (bool) - Use youtube-dl for videos - **start_url** (str) - Resume from specific pagination URL - **request_url_callback** (callable) - Callback for each page URL - **options** (dict) - Advanced extraction options ```