### Perform GET Requests with Crawlbase API in C# Source: https://github.com/crawlbase/crawlbase-net/blob/main/README.md Illustrates various ways to make GET requests using the Crawlbase API client in C#. Examples include basic usage, handling responses, passing custom options like user agent or format, and enabling cloud storage for responses. ```C# api.Get(url, options); ``` ```C# try { api.Get("https://www.facebook.com/britneyspears"); Console.WriteLine(api.StatusCode); Console.WriteLine(api.OriginalStatus); Console.WriteLine(api.CrawlbaseStatus); Console.WriteLine(api.Body); } catch(Exception ex) { Console.WriteLine(ex.ToString()); } ``` ```C# api.Get("https://www.reddit.com/r/pics/comments/5bx4bx/thanks_obama/", new Dictionary() { {"user_agent", "Mozilla/5.0 (Windows NT 6.2; rv:20.0) Gecko/20121202 Firefox/30.0"}, {"format", "json"}, }); Console.WriteLine(api.StatusCode); Console.WriteLine(api.Body); ``` ```C# api.Get("https://www.reddit.com/r/pics/comments/5bx4bx/thanks_obama/", new Dictionary() { {"store", "true"}, }); Console.WriteLine(api.StorageURL); Console.WriteLine(api.StorageRID); ``` -------------------------------- ### Utilize Crawlbase Scraper API in C# Source: https://github.com/crawlbase/crawlbase-net/blob/main/README.md Explains how to initialize and use the Crawlbase Scraper API client in C# for structured data extraction. Includes examples for making GET requests and handling responses. ```C# Crawlbase.ScraperAPI scraper_api = new Crawlbase.ScraperAPI("YOUR_TOKEN"); ``` ```C# scraper_api.Get(url, options); ``` ```C# try { scraper_api.Get("https://www.amazon.com/Halo-SleepSack-Swaddle-Triangle-Neutral/dp/B01LAG1TOS"); Console.WriteLine(scraper_api.StatusCode); Console.WriteLine(scraper_api.Body); } catch(Exception ex) { Console.WriteLine(ex.ToString()); } ``` -------------------------------- ### Perform Javascript-Enabled Requests with Crawlbase API in C# Source: https://github.com/crawlbase/crawlbase-net/blob/main/README.md Details how to use the Crawlbase API client with a Javascript token to scrape dynamic websites. Includes examples for initializing the client and making GET requests with and without specific rendering options like `page_wait`. ```C# Crawlbase.API api = new Crawlbase.API("YOUR_JAVASCRIPT_TOKEN"); ``` ```C# api.Get("https://www.nfl.com"); Console.WriteLine(api.StatusCode); Console.WriteLine(api.Body); ``` ```C# api.Get("https://www.freelancer.com", new Dictionary() { {"page_wait", "5000"}, }); Console.WriteLine(api.StatusCode); ``` -------------------------------- ### Initialize Crawlbase Storage API Client Source: https://github.com/crawlbase/crawlbase-net/blob/main/README.md This snippet shows the basic initialization of the Crawlbase Storage API client using a private token. This setup is a prerequisite for making any subsequent calls to retrieve, delete, or bulk process stored data. ```csharp Crawlbase.StorageAPI storage_api = new Crawlbase.StorageAPI("YOUR_TOKEN"); ``` -------------------------------- ### Crawlbase .NET API Client Reference Source: https://github.com/crawlbase/crawlbase-net/blob/main/README.md Comprehensive reference for the `Crawlbase.API` and `Crawlbase.ScraperAPI` classes, detailing their constructors, methods for making GET and POST requests, and available response properties. Includes common request options. ```APIDOC Crawlbase.API(token: string) - Initializes the Crawlbase API client. - Parameters: - token: Your Crawlbase API token (normal or Javascript). Crawlbase.API.Get(url: string, options?: Dictionary) - Makes a GET request to the specified URL. - Parameters: - url: The URL to scrape. - options: Optional dictionary of request parameters (e.g., user_agent, format, store, page_wait). - Async version: GetAsync Crawlbase.API.Post(url: string, data: Dictionary | string, options?: Dictionary) - Makes a POST request to the specified URL. - Parameters: - url: The URL to post to. - data: The data payload (dictionary or string). - options: Optional dictionary of request parameters (e.g., post_content_type). - Async version: PostAsync Crawlbase.API.StatusCode: int - The HTTP status code of the Crawlbase response. Crawlbase.API.OriginalStatus: int - The original HTTP status code returned by the target website. Crawlbase.API.CrawlbaseStatus: int - The Crawlbase-specific status code. Crawlbase.API.Body: string - The body of the response from the target website. Crawlbase.API.StorageURL: string - URL to the stored response in Crawlbase Cloud Storage (if 'store' option is true). Crawlbase.API.StorageRID: string - Request ID for the stored response in Crawlbase Cloud Storage. Crawlbase.ScraperAPI(token: string) - Initializes the Crawlbase Scraper API client. - Parameters: - token: Your Crawlbase API token. Crawlbase.ScraperAPI.Get(url: string, options?: Dictionary) - Makes a GET request using the Scraper API for structured data extraction. - Parameters: - url: The URL to scrape. - options: Optional dictionary of request parameters. - Async version: GetAsync Crawlbase.ScraperAPI.StatusCode: int - The HTTP status code of the Scraper API response. Crawlbase.ScraperAPI.Body: string - The body of the response from the Scraper API. Common Options: - user_agent: string (e.g., "Mozilla/5.0...") - format: string (e.g., "json") - store: "true" | "false" - post_content_type: "json" | "x-www-form-urlencoded" - page_wait: int (milliseconds) ``` -------------------------------- ### Retrieve Original and Crawlbase Status Codes in C# Source: https://github.com/crawlbase/crawlbase-net/blob/main/README.md Shows how to access the `OriginalStatus` and `CrawlbaseStatus` properties from the API response after a GET request. These provide detailed information about the request's outcome. ```C# api.Get("https://sfbay.craigslist.org/"); Console.WriteLine(api.OriginalStatus); Console.WriteLine(api.CrawlbaseStatus); ``` -------------------------------- ### Perform POST Requests with Crawlbase API in C# Source: https://github.com/crawlbase/crawlbase-net/blob/main/README.md Covers various ways to make POST requests using the Crawlbase API client in C#. Examples include basic usage with dictionary data and specifying the content type for JSON payloads. ```C# api.Post(url, data, options); ``` ```C# api.Post("https://producthunt.com/search", new Dictionary() { {"text", "example search"}, }); Console.WriteLine(api.StatusCode); Console.WriteLine(api.Body); ``` ```C# api.Post("https://httpbin.org/post", new Dictionary() { {"some_json", "with some value"}, }, new Dictionary() { {"post_content_type", "json"}, }); Console.WriteLine(api.StatusCode); Console.WriteLine(api.Body); ``` -------------------------------- ### Initialize Crawlbase API Client in C# Source: https://github.com/crawlbase/crawlbase-net/blob/main/README.md Demonstrates how to instantiate the Crawlbase API client in C# using your API token. This client is used for making subsequent scraping and crawling requests. ```C# Crawlbase.API api = new Crawlbase.API("YOUR_TOKEN"); ``` -------------------------------- ### Integrate Leads API with Crawlbase .NET Client Source: https://github.com/crawlbase/crawlbase-net/blob/main/README.md This snippet demonstrates how to initialize the Crawlbase Leads API client with a token and retrieve lead information for a given domain. It shows how to access status codes, raw body, success status, remaining requests, and iterate through extracted leads and their sources, including basic error handling. ```csharp Crawlbase.LeadsAPI leads_api = new Crawlbase.LeadsAPI("YOUR_TOKEN"); try { leads_api.Get("stripe.com"); Console.WriteLine(leads_api.StatusCode); Console.WriteLine(leads_api.Body); Console.WriteLine(leads_api.Success); Console.WriteLine(leads_api.RemainingRequests); foreach (var lead in leads_api.Leads) { Console.WriteLine(lead.Email); foreach (var source in lead.Sources) { Console.WriteLine(source); } } } catch(Exception ex) { Console.WriteLine(ex.ToString()); } ``` -------------------------------- ### Capture Screenshots with Crawlbase .NET Client Source: https://github.com/crawlbase/crawlbase-net/blob/main/README.md This snippet illustrates how to initialize the Crawlbase Screenshots API client and capture a screenshot of a given URL. It covers both basic retrieval and saving the screenshot to a specified local file path using options, demonstrating how to access the HTTP status code and the path where the screenshot is saved. Additionally, it shows how to convert the Base64 encoded image body to a byte array. ```csharp Crawlbase.ScreenshotsAPI screenshots_api = new Crawlbase.ScreenshotsAPI("YOUR_TOKEN"); try { screenshots_api.Get("https://www.apple.com"); Console.WriteLine(screenshots_api.StatusCode); Console.WriteLine(screenshots_api.ScreenshotPath); } catch(Exception ex) { Console.WriteLine(ex.ToString()); } ``` ```csharp Crawlbase.ScreenshotsAPI screenshots_api = new Crawlbase.ScreenshotsAPI("YOUR_TOKEN"); try { screenshots_api.Get("https://www.apple.com", new Dictionary() { {"save_to_path", @"C:\Users\Default\Documents\apple.jpg"}, }); Console.WriteLine(screenshots_api.StatusCode); Console.WriteLine(screenshots_api.ScreenshotPath); } catch(Exception ex) { Console.WriteLine(ex.ToString()); } ``` ```csharp byte[] bytes = Convert.FromBase64String(screenshots_api.Body); ``` -------------------------------- ### Perform Bulk Requests with Crawlbase Storage API Source: https://github.com/crawlbase/crawlbase-net/blob/main/README.md This snippet demonstrates how to perform a bulk request to retrieve multiple items from Crawlbase Storage using a list of RIDs. It iterates through the responses, printing details for each retrieved item such as status, URL, RID, and body content. ```csharp try { var list = new List(); list.Add(RID1); list.Add(RID2); list.Add(RIDn); var responses = storage_api.Bulk(list); Console.WriteLine(storage_api.StatusCode); foreach (var response in responses) { Console.WriteLine(response.OriginalStatus); Console.WriteLine(response.CrawlbaseStatus); Console.WriteLine(response.URL); Console.WriteLine(response.RID); Console.WriteLine(response.StoredAt); Console.WriteLine(response.Body); } } catch(Exception ex) { Console.WriteLine(ex.ToString()); } ``` -------------------------------- ### List RIDs from Crawlbase Storage API Source: https://github.com/crawlbase/crawlbase-net/blob/main/README.md This snippet shows how to retrieve a list of all Request IDs (RIDs) stored in your Crawlbase Storage area. It also demonstrates how to apply a limit to the number of RIDs returned, allowing for paginated or partial retrieval of stored item identifiers. ```csharp try { var rids = storage_api.RIDs(); foreach (var rid in rids) { Console.WriteLine(rid); } } catch(Exception ex) { Console.WriteLine(ex.ToString()); } ``` ```csharp var rids = storage_api.RIDs(100); ``` -------------------------------- ### Retrieve Total Document Count with Crawlbase .NET Storage API Source: https://github.com/crawlbase/crawlbase-net/blob/main/README.md This C# code snippet demonstrates how to retrieve the total number of documents stored in your Crawlbase account using the `storage_api.TotalCount()` method. It includes basic error handling to catch and print any exceptions that may occur during the API call, ensuring robust application behavior. ```csharp try { var totalCount = storage_api.TotalCount(); Console.WriteLine(totalCount); } catch(Exception ex) { Console.WriteLine(ex.ToString()); } ``` -------------------------------- ### Retrieve Data from Crawlbase Storage API Source: https://github.com/crawlbase/crawlbase-net/blob/main/README.md This snippet demonstrates how to retrieve stored data from Crawlbase Storage using either a URL or a Request ID (RID). It shows how to access various response properties like status codes, original status, Crawlbase status, URL, RID, and stored timestamp, along with the body content. ```csharp try { var response = storage_api.GetByUrl("https://www.apple.com"); Console.WriteLine(storage_api.StatusCode); Console.WriteLine(storage_api.Body); Console.WriteLine(response.OriginalStatus); Console.WriteLine(response.CrawlbaseStatus); Console.WriteLine(response.URL); Console.WriteLine(response.RID); Console.WriteLine(response.StoredAt); } catch(Exception ex) { Console.WriteLine(ex.ToString()); } ``` ```csharp try { var response = storage_api.GetByRID(RID); Console.WriteLine(storage_api.StatusCode); Console.WriteLine(storage_api.Body); Console.WriteLine(response.OriginalStatus); Console.WriteLine(response.CrawlbaseStatus); Console.WriteLine(response.URL); Console.WriteLine(response.RID); Console.WriteLine(response.StoredAt); } catch(Exception ex) { Console.WriteLine(ex.ToString()); } ``` -------------------------------- ### Delete Data from Crawlbase Storage API Source: https://github.com/crawlbase/crawlbase-net/blob/main/README.md This snippet shows how to delete a specific item from your Crawlbase Storage area using its Request ID (RID). It demonstrates checking the boolean success result of the deletion operation and includes basic error handling. ```csharp try { bool success = storage_api.Delete(RID); Console.WriteLine(success); } catch(Exception ex) { Console.WriteLine(ex.ToString()); } ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.