### Install ScrAPI via Package Manager Console Source: https://github.com/deventerprisesoftware/scrapi-sdk-dotnet/blob/master/README.md Use this command in the Package Manager Console within Visual Studio to install the ScrAPI NuGet package. ```powershell Install-Package ScrAPI ``` -------------------------------- ### Quick Start Basic Scraping Source: https://github.com/deventerprisesoftware/scrapi-sdk-dotnet/blob/master/README.md Initialize the ScrapiClient with your API key and create a ScrapeRequest to fetch website content. An empty API key enables limited free mode. ```csharp var client = new ScrapiClient("YOUR_API_KEY"); // "" for limited free mode. var request = new ScrapeRequest("https://deventerprise.com"); var response = await client.ScrapeAsync(request); // The result will contain the content and other information about the operation. Console.WriteLine(response?.Content); ``` -------------------------------- ### Install ScrAPI via .NET CLI Source: https://github.com/deventerprisesoftware/scrapi-sdk-dotnet/blob/master/README.md Use this command in the command line interface to add the ScrAPI package to your .NET Core project. ```bash dotnet add package ScrAPI ``` -------------------------------- ### Set API Key and Run Basic Scrape Source: https://github.com/deventerprisesoftware/scrapi-sdk-dotnet/blob/master/examples/basic_scrape/README.md Sets the SCRAPI_API_KEY environment variable and executes the basic scrape C# script. Ensure you replace 'your_api_key_here_or_blank' with your actual API key or leave it blank for unauthenticated requests. ```powershell $env:SCRAPI_API_KEY="your_api_key_here_or_or_blank" dotnet run ./BasicScrape.cs ``` -------------------------------- ### Listing Supported Cities Source: https://github.com/deventerprisesoftware/scrapi-sdk-dotnet/blob/master/README.md Fetches a list of supported cities for a given country, which can be used with the ProxyCity request option. The 'Key' property of each city object is the value to use. ```csharp var supportedCities = await client.GetSupportedCitiesAsync("USA"); // Use the Key value in the ProxyCity request property. foreach (var city in supportedCities) { Console.WriteLine($"{city.Key}: {city.Name}"); } ``` -------------------------------- ### Control Browser with Commands Source: https://github.com/deventerprisesoftware/scrapi-sdk-dotnet/blob/master/README.md Utilize browser commands to interact with a webpage before capturing its state when `UseBrowser` is enabled. This includes inputting text, selecting options, waiting, and scrolling. ```csharp var request = new ScrapeRequest("https://www.roboform.com/filling-test-all-fields") { UseBrowser = true, AcceptDialogs = true }; // Example of chaining commands to control the website. request.BrowserCommands .Input("input[name='01___title']", "Mr") .Input("input[name='02frstname']", "Werner") .Input("input[name='04lastname']", "van Deventer") .Select("select[name='40cc__type']", "Discover") .Wait(TimeSpan.FromSeconds(3)) .WaitFor("input[type='reset']") .Click("input[type='reset']") .Wait(TimeSpan.FromSeconds(1)) .Scroll(1000) .Evaluate("console.log('any valid code...')"); ``` -------------------------------- ### Listing Supported Countries Source: https://github.com/deventerprisesoftware/scrapi-sdk-dotnet/blob/master/README.md Fetches a list of supported countries for proxy selection. The 'Key' property of each country object can be used with the ProxyCountry request option. ```csharp var supportedCountries = await client.GetSupportedCountriesAsync(); // Use the Key value in the ProxyCountry request property. foreach (var country in supportedCountries) { Console.WriteLine($"{country.Key}: {country.Name}"); } ``` -------------------------------- ### Register ScrapiClient with Dependency Injection Source: https://github.com/deventerprisesoftware/scrapi-sdk-dotnet/blob/master/README.md Register the IScrapiClient as a singleton in your IServiceCollection for use with dependency injection and mocking. ```csharp // Add singleton to IServiceCollection services.AddSingleton(_ => new ScrapiClient("YOUR_API_KEY")); ``` -------------------------------- ### Configure Scrape Request Options Source: https://github.com/deventerprisesoftware/scrapi-sdk-dotnet/blob/master/README.md Set various options for a ScrapeRequest, including cookies, headers, proxy details, browser usage, and response format. ```csharp var request = new ScrapeRequest("https://deventerprise.com") { Cookies = new Dictionary { { "cookie1", "value1" }, { "cookie2", "value2" }, }, Headers = new Dictionary { { "header1", "value1" }, { "header2", "value2" }, }, ProxyCountry = "USA", ProxyCity = "NewYork", ProxyType = ProxyType.Residential, UseBrowser = true, SolveCaptchas = true, IncludeScreenshot = true, IncludePdf = true, IncludeVideo = true, RequestMethod = "GET", ResponseFormat = ResponseFormat.Html, ResponseSelector = "//div[@class='content']", CustomProxyUrl = "https://user:password@local.proxy:8080", SessionId = Guid.NewGuid().ToString(), CallbackUrl = new Uri("https://webhook.site/"), }; ``` -------------------------------- ### Handling Scrapi Exceptions Source: https://github.com/deventerprisesoftware/scrapi-sdk-dotnet/blob/master/README.md Demonstrates how to catch ScrapiException, which is thrown for API errors. The exception includes an HTTP status code property to aid in implementing retry logic. ```csharp var client = new ScrapiClient("YOUR_API_KEY"); // "" for limited free mode. var request = new ScrapeRequest("https://deventerprise.com"); try { var result = await client.ScrapeAsync(request); Console.WriteLine(result?.Content); } catch (ScrapiException ex) when (ex.StatusCode == System.Net.HttpStatusCode.InternalServerError) { // Error messages from the server aim to be as helpful as possible. Console.WriteLine(ex.Message); throw; } // The result will contain the content and other information about the operation. Console.WriteLine(result?.Content); ``` -------------------------------- ### Accessing Scrape Response Data Source: https://github.com/deventerprisesoftware/scrapi-sdk-dotnet/blob/master/README.md Demonstrates how to access various properties of the ScrapeResponse object, including URLs, timing, status, content, and parsed data. It also shows how to iterate through solved captchas, headers, cookies, and error messages. ```csharp var response = await client.ScrapeAsync(request); Console.WriteLine(response.RequestUrl); // The requested URL. Console.WriteLine(response.ResponseUrl); // The final URL of the page. Console.WriteLine(response.Duration); // The amount of time the operation took. Console.WriteLine(response.Attempts); // The number of attempts to scrape the page. Console.WriteLine(response.CreditsUsed); // The number of credits used for this request. Console.WriteLine(response.StatusCode); // The response status code from the request. Console.WriteLine(response.ScreenshotUrl); // The URL of the screenshot file if included. Console.WriteLine(response.PdfUrl); // The URL of the PDF file if included. Console.WriteLine(response.VideoUrl); // The URL of the video file if included. Console.WriteLine(response.Content); // The final page content. Console.WriteLine(response.ContentHash); // SHA1 hash of the content. Console.WriteLine(response.Html); // Html Agility Pack parsed HTML content. foreach (var captchaSolved in response.CaptchasSolved) { Console.WriteLine($"{captchaSolved.Value} occurrences of {captchaSolved.Key} solved"); } foreach (var header in response.Headers) { Console.WriteLine($"{header.Key}: {header.Value}"); } foreach (var cookie in response.Cookies) { Console.WriteLine($"{cookie.Key}: {cookie.Value}"); } foreach (var errorMessage in response.ErrorMessages ?? []) { Console.WriteLine(errorMessage); // Any errors that occurred during the request. } ``` -------------------------------- ### Checking Credit Balance Source: https://github.com/deventerprisesoftware/scrapi-sdk-dotnet/blob/master/README.md Retrieves the remaining credit balance for the API key using the GetCreditBalanceAsync method. ```csharp var balance = await client.GetCreditBalanceAsync(); ``` -------------------------------- ### Setting Scrape Request Defaults Source: https://github.com/deventerprisesoftware/scrapi-sdk-dotnet/blob/master/README.md Configures default settings for ScrapeRequest objects, such as proxy type, browser usage, and captcha solving. These defaults apply to all new requests unless explicitly overridden. ```csharp // Set default that will apply to all new `ScrapeRequest` object (unless overridden). ScrapeRequestDefaults.ProxyType = ProxyType.Residential; ScrapeRequestDefaults.UseBrowser = true; ScrapeRequestDefaults.SolveCaptchas = true; ScrapeRequestDefaults.Headers.Add("Sample", "Custom-Value"); // Any new request will have the corresponding values automatically applied. var request = new ScrapeRequest("https://deventerprise.com") { ProxyType = ProxyType.Tor }; Debug.Assert(request.ProxyType == ProxyType.Tor); // Overridden Debug.Assert(request.UseBrowser); Debug.Assert(request.SolveCaptchas); Debug.Assert(request.Headers.ContainsKey("Sample")); ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.