### Define Data for Single Form Field Source: https://github.com/scrapy/form2request/blob/main/docs/usage.md Use the `data` parameter to specify values for form fields. This example shows how to set a value for a single text input. ```python >>> html = b"""
""" >>> selector = Selector(body=html, base_url="https://example.com") >>> form = selector.css("form") >>> form2request(form, {"foo": "bar"}) Request(url='https://example.com?foo=bar', method='GET', headers=[], body=b'') ``` -------------------------------- ### Finding Form by Attribute using XPath Source: https://github.com/scrapy/form2request/blob/main/docs/usage.md Provides an example XPath expression to locate a form element based on one of its attributes, such as 'id' or 'name'. This is useful for targeting specific forms in complex HTML. ```xpath //form[@id="foo"] ``` -------------------------------- ### Extracting HTML Form with Scrapy Response Source: https://github.com/scrapy/form2request/blob/main/docs/usage.md Shows how to get an HTML form object directly from a Scrapy TextResponse object using CSS selectors. This is common when working within a Scrapy project. ```python from scrapy.http import TextResponse html = b"" response = TextResponse("https://example.com", body=html) form = response.css("form") ``` -------------------------------- ### Converting to Requests Library Format Source: https://github.com/scrapy/form2request/blob/main/docs/usage.md Illustrates how to convert the generated Request object into a format compatible with the Python requests library. This allows for easy sending of the form submission request. ```pycon import requests request = request_data.to_requests() requests.send(request) ``` -------------------------------- ### Extracting HTML Form with lxml Source: https://github.com/scrapy/form2request/blob/main/docs/usage.md Demonstrates how to use the lxml library to parse an HTML string and obtain a form element. This is an alternative to using parsel for form extraction. ```pycon from lxml.html import fromstring html = b"" root = fromstring(html, base_url="https://example.com") form = root.xpath("//form")[0] ``` -------------------------------- ### Extracting HTML Form with Parsel Source: https://github.com/scrapy/form2request/blob/main/docs/usage.md Demonstrates how to use the parsel library to select an HTML form element from a given HTML body. This is the first step before using form2request. ```pycon from parsel import Selector html = b"" selector = Selector(body=html, base_url="https://example.com") form = selector.css("form") ``` -------------------------------- ### Basic Form to Request Conversion Source: https://github.com/scrapy/form2request/blob/main/docs/usage.md Convert an HTML form into a basic Request object. The output includes the URL, method, headers, and body of the request. ```python >>> request_data = form2request(form) >>> request_data Request(url='https://example.com?foo=bar', method='GET', headers=[], body=b'') ``` -------------------------------- ### Finding Form by Index using XPath Source: https://github.com/scrapy/form2request/blob/main/docs/usage.md Illustrates how to use an XPath expression to select a form element based on its order of appearance in the HTML document. This is useful when forms lack unique identifiers. ```xpath (//form)[2] ``` -------------------------------- ### Submitting Form with Default Submit Button Source: https://github.com/scrapy/form2request/blob/main/docs/usage.md Submits a form by default, including the name and value of the first submit button found. This is the standard behavior when a submit button is present. ```python >>> html = b"" >>> selector = Selector(body=html, base_url="https://example.com") >>> form = selector.css("form") >>> form2request(form) Request(url='https://example.com?foo=bar', method='GET', headers=[], body='') ``` -------------------------------- ### Uploading Files with FileField Source: https://github.com/scrapy/form2request/blob/main/docs/usage.md Demonstrates how to upload files using the FileField class within form2request. Ensure the form has enctype="multipart/form-data". ```python >>> from form2request import FileField, form2request >>> html = b""" ... """ >>> selector = Selector(body=html, base_url="https://example.com") >>> form = selector.css("form") >>> request_data = form2request(form, { ... "description": "quarterly report", ... "attachment": FileField( ... content=b"col1,col2\n1,2\n", ... filename="report.csv", ... content_type="text/csv", ... ), ... }) >>> request_data.method 'POST' >>> request_data.url 'https://example.com' >>> request_data.headers[0][1].startswith("multipart/form-data") True ``` -------------------------------- ### Finding Form by Traversing Parent Elements Source: https://github.com/scrapy/form2request/blob/main/docs/usage.md Demonstrates a Python code snippet that finds a form element by first locating an input element within it and then traversing up the DOM tree to reach the parent form. This is a fallback method when direct form selection is difficult. ```python from lxml.html import fromstring html = b"" root = fromstring(html, base_url="https://example.com") element = root.xpath('//input[@name="zip_code"]')[0] while True: if element.tag == "form": break element = element.getparent() form = element ``` -------------------------------- ### Define Data for Multiple Same-Name Form Fields Source: https://github.com/scrapy/form2request/blob/main/docs/usage.md When a form has multiple fields with the same `name`, provide an iterable of key-value tuples to `form2request` to specify values for all of them. ```python >>> html = b"""""" >>> selector = Selector(body=html, base_url="https://example.com") >>> form = selector.css("form") >>> form2request(form, (("foo", "bar"), ("foo", "baz"))) Request(url='https://example.com?foo=bar&foo=baz', method='GET', headers=[], body=b'') ``` -------------------------------- ### Submitting Form with Specific Submit Button Source: https://github.com/scrapy/form2request/blob/main/docs/usage.md Submits a form by explicitly selecting which submit button to use. This is useful when a form has multiple submit buttons and you need to control which one is activated. ```python >>> html = b"" >>> selector = Selector(body=html, base_url="https://example.com") >>> form = selector.css("form") >>> submit_baz = form.css("[value=baz]") >>> form2request(form, click=submit_baz) Request(url='https://example.com?foo=baz', method='GET', headers=[], body='') ``` -------------------------------- ### Convert Request to Scrapy Request Source: https://github.com/scrapy/form2request/blob/main/docs/usage.md Convert the Request object to a Scrapy Request object, suitable for use with the Scrapy framework (version 1.7.1+). Allows specifying a callback function. ```python >>> request_data.to_scrapy(callback=self.parse)