### Full Mechanize Browser Configuration Example Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/08-configuration.md A comprehensive example demonstrating how to set up a Mechanize Browser instance with persistent cookies, authentication, proxy settings, custom headers, and other features before making a request. ```python from mechanize import Browser, CookieJar, LWPCookieJar, HTTPPasswordMgrWithDefaultRealm # Create browser br = Browser() # Configure cookies (persistent) jar = LWPCookieJar("cookies.txt") try: jar.load() except: pass br.set_cookiejar(jar) # Configure authentication pm = HTTPPasswordMgrWithDefaultRealm() pm.add_password(None, "http://example.com", "user", "password") br.set_password_manager(pm) # Configure features br.set_handle_robots(True) br.set_handle_redirect(True) br.set_handle_refresh((True, 3)) # Max 3 second refresh br.set_handle_equiv(True) br.set_handle_gzip(True) br.set_handle_referer(True) # Configure proxies br.set_proxies({ "http": "http://proxy.example.com:8080", "https": "https://proxy.example.com:8080" }) # Configure headers br.addheaders = [ ("User-Agent", "Mozilla/5.0 (Custom)"), ("Accept-Language", "en-US,en;q=0.9"), ] # Now use browser br.open("http://example.com") br.select_form(nr=0) br["field"] = "value" response = br.submit() ``` -------------------------------- ### Install Mechanize for Development Source: https://github.com/python-mechanize/mechanize/blob/master/README.rst Install the mechanize library from a git clone for development purposes. ```bash git clone https://github.com/python-mechanize/mechanize.git cd mechanize pip3 install -e . ``` -------------------------------- ### Install Global Opener Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/05-utilities.md Installs an opener as the global default. Subsequent calls to urlopen will use this opener. ```python from mechanize import install_opener, build_opener opener = build_opener() install_opener(opener) # Now use the global urlopen from mechanize import urlopen response = urlopen("http://example.com") ``` -------------------------------- ### Install Mechanize using pip Source: https://github.com/python-mechanize/mechanize/blob/master/README.rst Install the mechanize library for normal usage with pip. ```bash pip3 install mechanize ``` -------------------------------- ### install_opener Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/05-utilities.md Installs a global default opener. This allows subsequent calls to urlopen to use the specified opener. ```APIDOC ## install_opener ### Description Install a global default opener. ### Method Signature ```python def install_opener(opener): ``` ### Parameters * **opener**: The OpenerDirector object to install as the default. ### Example ```python from mechanize import install_opener, build_opener opener = build_opener() install_opener(opener) # Now use the global urlopen from mechanize import urlopen response = urlopen("http://example.com") ``` ``` -------------------------------- ### HTTPPasswordMgrWithDefaultRealm Example Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/04-handlers.md Demonstrates using HTTPPasswordMgrWithDefaultRealm to manage credentials with a default realm fallback. ```APIDOC ## HTTPPasswordMgrWithDefaultRealm Password manager with default realm fallback. **Module:** `mechanize._urllib2` **Signature:** ```python class HTTPPasswordMgrWithDefaultRealm(HTTPPasswordMgr): ``` If no credentials match a specific realm, uses credentials added with realm=None. **Example:** ```python pm = mechanize.HTTPPasswordMgrWithDefaultRealm() pm.add_password(None, "http://example.com", "user", "pass") # Default pm.add_password("Special Realm", "http://example.com", "admin", "admin_pass") ``` ``` -------------------------------- ### MimeWriter startbody Method Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/02-forms.md Starts the MIME body. Can specify content type and other parameters. ```python def startbody(self, ctype=None, plist=[], prefix=1, add_to_http_hdrs=0, content_type=1): ``` -------------------------------- ### LWPCookieJar Example Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/03-cookies.md Create and use LWPCookieJar for persistent cookie storage in libwww-perl format. Cookies are loaded at initialization and saved before the program exits. ```python from mechanize import LWPCookieJar, Browser # Create persistent cookie storage jar = LWPCookieJar("cookies.txt") try: jar.load() except: pass br = Browser() br.set_cookiejar(jar) # ... do stuff ... # Save cookies for next session jar.save() ``` -------------------------------- ### Open URL with Default Opener Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/05-utilities.md Opens a URL using the currently installed default opener. Can send POST data and set timeouts. ```python from mechanize import urlopen response = urlopen("http://example.com") html = response.read() ``` -------------------------------- ### MimeWriter startmultipartbody Method Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/02-forms.md Starts a multipart MIME body. Allows specifying subtype and boundary. ```python def startmultipartbody(self, subtype, boundary=None, plist=[], prefix=1, add_to_http_hdrs=0, content_type=1): ``` -------------------------------- ### MozillaCookieJar Example Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/03-cookies.md Load cookies from a Mozilla/Netscape format file, typically used by browsers like Firefox. Handles potential loading errors. ```python from mechanize import MozillaCookieJar, Browser import os # Load from Firefox cookies jar = MozillaCookieJar(os.path.expanduser("~/.mozilla/firefox/cookies.txt")) try: jar.load() except: pass br = Browser() br.set_cookiejar(jar) ``` -------------------------------- ### Access Form Controls Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/00-index.md Provides examples of accessing and manipulating various form control types in Mechanize. ```python # Text, Password, Hidden, Textarea, Select, Checkbox, Radio br["field_name"] = "value" # Multi-select br["field_name"] = ["opt1", "opt2"] # File upload (requires a file object 'f') # form.find_control(name="file").add_file(f) ``` -------------------------------- ### Create and Add a Persistent Cookie Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/03-cookies.md Example of creating a persistent cookie with specific attributes like expiration time and domain, then adding it to the browser's cookie jar. Ensure 'time' is imported. ```python from mechanize import Cookie import time # Create a persistent cookie cookie = Cookie( version=0, name="sessionid", value="abc123def456", port=None, port_specified=False, domain=".example.com", domain_specified=True, domain_initial_dot=True, path="/", path_specified=True, secure=True, expires=int(time.time()) + 3600, # 1 hour from now discard=False, comment=None, comment_url=None, rfc2109=False ) br.cookiejar.add_cookie(cookie) ``` -------------------------------- ### Basic Mechanize Browser Initialization and Navigation Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/00-index.md Demonstrates the fundamental steps to create a Mechanize browser instance, open a webpage, and select the first form on the page. ```python import mechanize # Create browser br = mechanize.Browser() # Open page br.open("http://example.com") # Select form br.select_form(nr=0) ``` -------------------------------- ### Access CookieJar Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/01-browser.md Get the underlying CookieJar object to iterate over stored cookies. ```python jar = br.cookiejar if jar: for cookie in jar: print(cookie.name, cookie.value) ``` -------------------------------- ### encoding Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/05-utilities.md Gets the character encoding of the response. Returns the encoding name as a string. ```APIDOC ## encoding ### Description Get response character encoding. ### Method Signature ```python @property def encoding(self): ``` ### Returns str - encoding name ``` -------------------------------- ### Configure SSL Client Certificates Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/08-configuration.md Set up SSL client certificates for secure connections. Requires specifying the host, certificate path, and key path. ```python browser.set_client_cert_manager(cert_mgr) ``` ```python from mechanize import HTTPSClientCertMgr, Browser cm = HTTPSClientCertMgr() cm.add_cert("api.example.com", "/path/to/client.crt", "/path/to/client.key") br = Browser() br.set_client_cert_manager(cm) ``` -------------------------------- ### request_host Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/03-cookies.md Gets the hostname from a request object, normalized to lowercase for cookie processing. ```APIDOC ### request_host Get request hostname for cookie purposes. ```python def request_host(request): ``` **Returns:** str - Lowercase hostname **Module:** `mechanize._clientcookie` ``` -------------------------------- ### Initialize Browser and Open URL Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/01-browser.md Instantiate the Browser class and open a URL to retrieve its HTML content. The response object can then be used to read the page's HTML. ```python br = Browser() response = br.open("http://example.com") html_content = response.read() ``` -------------------------------- ### title Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/05-utilities.md Gets the title of the current HTML page. Returns the page title as a string. ```APIDOC ## title ### Description Get page title. ### Method Signature ```python @property def title(self): ``` ### Returns str - page title ``` -------------------------------- ### Configuring Browser Proxies and Authentication Source: https://github.com/python-mechanize/mechanize/blob/master/docs/index.md Sets up proxy servers, including authentication, and configures website-specific authentication credentials. Use this to manage network access and secure resource retrieval. ```python br = mechanize.Browser() # Explicitly configure proxies (Browser will attempt to set good defaults). # Note the userinfo ("joe:password@") and port number (":3128") are optional. br.set_proxies({"http": "joe:password@myproxy.example.com:3128", "ftp": "proxy.example.com", }) # Add HTTP Basic/Digest auth username and password for HTTP proxy access. # (equivalent to using "joe:password@..." form above) br.add_proxy_password("joe", "password") # Add HTTP Basic/Digest auth username and password for website access. br.add_password("http://example.com/protected/", "joe", "password") ``` -------------------------------- ### Build Custom Opener with Handlers Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/00-index.md Creates a custom opener with specified handlers for authentication, cookies, and proxies. ```python import mechanize # Build custom opener pm = mechanize.HTTPPasswordMgr() pm.add_password("Realm", "http://example.com", "user", "pass") opener = mechanize.build_opener( mechanize.HTTPBasicAuthHandler(pm), mechanize.HTTPCookieProcessor(), mechanize.ProxyHandler({"http": "http://proxy:8080"}) ) # Use it response = opener.open("http://example.com") ``` -------------------------------- ### OpenerDirector Key Methods Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/06-types.md Chains handlers for request/response processing. Key methods include opening URLs and adding handlers. ```python def open(fullurl: str | Request, data=None, timeout=_GLOBAL_DEFAULT_TIMEOUT) -> Response def add_handler(handler: BaseHandler) -> None ``` -------------------------------- ### OpenerFactory.build_opener Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/05-utilities.md Builds an OpenerDirector instance with the specified handlers. ```APIDOC ## OpenerFactory.build_opener ### Description Builds an opener chain (OpenerDirector) with the provided handlers. ### Method build_opener ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body - **\*handlers** (BaseHandler) - Required - Handler instances to include ### Request Example ```python from mechanize import OpenerFactory, HTTPBasicAuthHandler, HTTPPasswordMgr factory = OpenerFactory() pm = HTTPPasswordMgr() pm.add_password("Realm", "http://example.com", "user", "pass") opener = factory.build_opener( HTTPBasicAuthHandler(pm), mechanize.HTTPCookieProcessor() ) response = opener.open("http://example.com") ``` ### Response #### Success Response - **OpenerDirector** - An instance of OpenerDirector configured with the provided handlers. #### Response Example ```python # Example usage of the returned opener response = opener.open("http://example.com") ``` ``` -------------------------------- ### Request Host Type Hint Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/06-types.md Type signature for the request_host function, used to get the host of a request. ```python def request_host(request: Request) -> str ``` -------------------------------- ### Open URL with UserAgentBase Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/05-utilities.md Demonstrates opening a URL and reading its content using the UserAgentBase class. This is a low-level interface for URL opening. ```python ua = mechanize.UserAgentBase() response = ua.open("http://example.com") content = response.read() ``` -------------------------------- ### Get Page Title Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/01-browser.md Retrieve the title of the current HTML page. Raises BrowserStateError if not viewing HTML. ```python print(br.title()) ``` -------------------------------- ### Configure HTTP Proxy Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/00-index.md Sets up HTTP and HTTPS proxies for the browser instance. ```python br = mechanize.Browser() br.set_proxies({ "http": "http://proxy.example.com:8080", "https": "https://proxy.example.com:8080" }) ``` -------------------------------- ### Handle LinkNotFoundError Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/07-errors.md Catch LinkNotFoundError when a specified link cannot be found on the page. The example demonstrates printing available links. ```python br = mechanize.Browser() br.open("http://example.com") try: link = br.find_link(text="Download") except mechanize.LinkNotFoundError: print("Download link not found") print("Available links:") for link in br.links(): print(f" {link.text} -> {link.url}") ``` -------------------------------- ### Effective Request Host Type Hint Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/06-types.md Type signature for the effective_request_host function, used to get the host of a request. ```python def effective_request_host(request: Request) -> str ``` -------------------------------- ### Get Page Encoding Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/01-browser.md Retrieve the character encoding of the current HTML page. Raises BrowserStateError if no document is loaded. ```python print(br.encoding()) ``` -------------------------------- ### Browser Constructor Options Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/08-configuration.md Instantiate the Browser class with custom options for history, request handling, content parsing, and factory class. Defaults are used if not specified. ```python Browser( history=None, request_class=None, content_parser=None, factory_class=Factory, allow_xhtml=False ) ``` -------------------------------- ### Get Current URL Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/01-browser.md Retrieve the URL of the document currently loaded in the browser. Raises BrowserStateError if no document is loaded. ```python url = br.geturl() ``` -------------------------------- ### Password and Authentication Configuration Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/08-configuration.md Methods for setting up HTTP and proxy authentication credentials, as well as configuring SSL client certificates. ```APIDOC ## Password and Authentication Configuration ### `set_password_manager(password_mgr)` Set HTTP authentication credentials. **Setup:** ```python from mechanize import HTTPPasswordMgr, HTTPPasswordMgrWithDefaultRealm, Browser # Basic setup pm = HTTPPasswordMgrWithDefaultRealm() pm.add_password(None, "http://example.com", "user", "password") br = Browser() br.set_password_manager(pm) # Per-realm passwords pm = HTTPPasswordMgr() pm.add_password("Realm Name", "http://example.com", "user", "pass") br.set_password_manager(pm) ``` ### `set_proxy_password_manager(password_mgr)` Set proxy authentication credentials. **Setup:** ```python from mechanize import HTTPProxyPasswordMgr, Browser ppm = HTTPProxyPasswordMgr() ppm.add_password("Proxy Realm", "proxy.example.com", "proxyuser", "proxypass") br = Browser() br.set_proxy_password_manager(ppm) ``` ### `set_client_cert_manager(cert_mgr)` Configure SSL client certificates. **Setup:** ```python from mechanize import HTTPSClientCertMgr, Browser cm = HTTPSClientCertMgr() cm.add_cert("api.example.com", "/path/to/client.crt", "/path/to/client.key") br = Browser() br.set_client_cert_manager(cm) ``` ``` -------------------------------- ### Get Possible Values for List Controls Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/02-forms.md Retrieve a list of all available option values for a given list control. ```python checkbox_group = form.find_control(name="interests") print(checkbox_group.possible_values()) # ["option1", "option2", "option3"] ``` -------------------------------- ### Configure Proxy Servers Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/05-utilities.md Sets up proxy servers for the UserAgentBase instance, including options for custom proxy bypass logic. Specify a dictionary mapping schemes to proxy URLs. ```python ua.set_proxies({ "http": "http://proxy.example.com:8080", "https": "https://proxy.example.com:8080" }) ``` ```python def bypass_proxy(hostname): return hostname in ["localhost", "internal.company.com"] ua.set_proxies({...}, proxy_bypass=bypass_proxy) ``` -------------------------------- ### Required Dependency for Mechanize Source: https://github.com/python-mechanize/mechanize/blob/master/docs/faq.md Mechanize requires the html5lib library for parsing HTML. Ensure this dependency is installed before using mechanize. ```default html5lib ``` -------------------------------- ### Factory Key Methods Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/06-types.md Provides methods to set the response, retrieve forms and links, and configure request classes. ```python def set_request_class(request_class: type) -> None def set_response(response: Response) -> None def forms() -> tuple[HTMLForm, ...] def links() -> tuple[Link, ...] ``` -------------------------------- ### Handle ControlNotFoundError Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/07-errors.md Catch ControlNotFoundError when a form control cannot be found by the specified criteria. This example shows how to list available controls. ```python form = br.forms()[0] try: field = form.find_control(name="email") except mechanize.ControlNotFoundError: print("Email field not found") print("Available controls:") for ctrl in form.controls: print(f" {ctrl.name} ({ctrl.type})") ``` -------------------------------- ### Handle FormNotFoundError Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/07-errors.md Catch FormNotFoundError when attempting to select a form that does not exist on the page. This example shows how to print available forms. ```python br = mechanize.Browser() br.open("http://example.com") try: br.select_form(name="login") except mechanize.FormNotFoundError as e: print(f"Form error: {e}") print(f"Available forms: {[f.name for f in br.forms()]}") ``` -------------------------------- ### __copy__ Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/01-browser.md Creates a shallow copy of the browser instance. The copy shares the CookieJar and handlers but has an independent navigation history and state. ```APIDOC ## __copy__ ### Description Clone the browser instance. ### Method `__copy__()` ### Returns New Browser instance: This new instance is thread-safe for use in another thread. ### Example ```python br2 = copy.copy(br) # Share cookies and config, independent history ``` ``` -------------------------------- ### Get All Links Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/01-browser.md Retrieves all links present on the current page. Can be used to iterate through all links or to filter them using criteria similar to find_link. ```python # All links for link in br.links(): print(link.text, link.url) ``` ```python # Filtered links for link in br.links(url_regex=r"\.pdf"): print(link.text) ``` -------------------------------- ### Load Cookies with Error Handling Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/03-cookies.md Demonstrates loading cookies from a file using LWPCookieJar and catching LoadError if the file cannot be loaded. ```python from mechanize import LWPCookieJar, LoadError jar = LWPCookieJar("cookies.txt") try: jar.load() except LoadError as e: print(f"Failed to load cookies: {e}") ``` -------------------------------- ### Get Effective Request Host Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/03-cookies.md Determine the effective host for a given request, as defined by RFC 2965, which is crucial for cookie matching. ```python from mechanize import effective_request_host req = mechanize.Request("http://www.example.com/path") host = effective_request_host(req) # "www.example.com" ``` -------------------------------- ### AbstractBasicAuthHandler Initialization Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/04-handlers.md Base class for Basic authentication handlers, requiring a password manager. ```APIDOC ## AbstractBasicAuthHandler Base class for Basic authentication handlers. **Module:** `mechanize._urllib2` **Signature:** ```python class AbstractBasicAuthHandler(BaseHandler): def __init__(self, password_mgr): ``` | Parameter | Type | Description | |-----------|------|-------------| | password_mgr | HTTPPasswordMgr | Password manager with credentials | ``` -------------------------------- ### Navigate Back Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/00-index.md Illustrates how to go back to the previous page in the browser history. ```python br.back() ``` -------------------------------- ### Configure Proxy with Bypass Function Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/00-index.md Sets up a proxy and defines a function to determine which hosts should bypass the proxy. ```python def bypass_proxy(host): return host.endswith(".internal") br.set_proxies( {"http": "http://proxy:8080"}, proxy_bypass=bypass_proxy ) ``` -------------------------------- ### Fill and Submit Form Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/00-index.md Demonstrates how to fill a form field and submit it using Mechanize. ```python br["field_name"] = "value" response = br.submit() ``` -------------------------------- ### Get All Forms Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/01-browser.md Retrieves all HTML forms present on the current page. Useful for iterating through forms to inspect their properties like name and action. ```python for form in br.forms(): print(form.name, form.action) ``` -------------------------------- ### Clone Browser Instance Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/01-browser.md Create a shallow copy of the browser instance. The new instance shares the CookieJar and handlers but has independent navigation history. ```python br2 = copy.copy(br) # Share cookies and config, independent history ``` -------------------------------- ### Build Opener with Handlers Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/05-utilities.md Constructs an opener director with specified handlers, such as authentication. Useful for authenticated requests. ```python from mechanize import build_opener, HTTPBasicAuthHandler, HTTPPasswordMgr pm = HTTPPasswordMgr() pm.add_password("Realm", "http://example.com", "user", "pass") auth_handler = HTTPBasicAuthHandler(pm) opener = build_opener(auth_handler) response = opener.open("http://example.com/protected") ``` -------------------------------- ### Get Global Form Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/01-browser.md Retrieves the global form, which contains controls not associated with any specific FORM element. Returns None if no global form exists. ```python gf = br.global_form() if gf: print(gf.controls) ``` -------------------------------- ### Click Link to Get Request Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/01-browser.md Finds a link and returns a Request object for it without opening the link. This allows inspection or modification of the request before it's sent. ```python req = br.click_link(text="Download") # Can inspect or modify request before opening print(req.get_full_url()) ``` -------------------------------- ### Select Form by Action URL Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/00-index.md Illustrates selecting a form based on the URL specified in its 'action' attribute. ```python br.select_form(action="/submit") ``` -------------------------------- ### Handle ItemNotFoundError in Select Control Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/07-errors.md Catch ItemNotFoundError when attempting to set a value for a select control or checkbox/radio button that does not exist. The example shows how to retrieve valid options. ```python select = form.find_control(name="country") try: select.value = "InvalidCountry" except mechanize.ItemNotFoundError: print(f"Valid options: {select.possible_values()}") ``` -------------------------------- ### urlopen Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/05-utilities.md Opens a URL using the default opener. Can be used to fetch content from a given URL. ```APIDOC ## urlopen ### Description Open a URL using the default opener. ### Method Signature ```python def urlopen(url, data=None, timeout=_GLOBAL_DEFAULT_TIMEOUT): ``` ### Parameters * **url**: The URL or Request object to open. * **data**: Optional POST data. Can be a dictionary or bytes. * **timeout**: Socket timeout in seconds. ### Returns Response object ### Example ```python from mechanize import urlopen response = urlopen("http://example.com") html = response.read() ``` ``` -------------------------------- ### Select Form by Name Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/00-index.md Shows how to select a specific form using its 'name' attribute. ```python br.select_form(name="login") ``` -------------------------------- ### Get Current Response Object Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/01-browser.md Retrieve a copy of the current response object. This is useful for inspecting response headers or content after a page has been loaded. Returns None if no page has been loaded yet. ```python resp = br.response() if resp: content_type = resp.info().get("Content-Type") ``` -------------------------------- ### Basic Web Browsing and Form Interaction Source: https://github.com/python-mechanize/mechanize/blob/master/docs/index.md Demonstrates opening a URL, following links based on regular expressions, interacting with HTML forms by setting field values, submitting forms, and navigating browser history. Use this for general web scraping and form automation tasks. ```python import re import mechanize br = mechanize.Browser() br.open("http://www.example.com/") # follow second link with element text matching regular expression response1 = br.follow_link(text_regex=r"cheese\s*shop", nr=1) print(br.title()) print(response1.geturl()) print(response1.info()) # headers print(response1.read()) # body br.select_form(name="order") # Browser passes through unknown attributes (including methods) # to the selected HTMLForm. br["cheeses"] = ["mozzarella", "caerphilly"] # (the method here is __setitem__) # Submit current form. Browser calls .close() on the current response on # navigation, so this closes response1 response2 = br.submit() # print currently selected form (don't call .submit() on this, use br.submit()) print(br.form) response3 = br.back() # back to cheese shop (same data as response1) # the history mechanism returns cached response objects # we can still use the response, even though it was .close()d response3.get_data() # like .seek(0) followed by .read() response4 = br.reload() # fetches from server for form in br.forms(): print(form) # .links() optionally accepts the keyword args of .follow_/.find_link() for link in br.links(url_regex="python.org"): print(link) br.follow_link(link) # takes EITHER Link instance OR keyword args br.back() ``` -------------------------------- ### make_response Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/05-utilities.md Creates a dummy response object, useful for testing purposes. Allows manual construction of response data and headers. ```APIDOC ## make_response ### Description Create a dummy response for testing. ### Method Signature ```python def make_response(data, headers, url="http://example.com/"): ``` ### Parameters * **data**: The response body content, as a string or bytes. * **headers**: A list of (name, value) tuples representing the response headers. * **url**: The URL associated with the response. Defaults to "http://example.com/". ### Returns Response object ### Example ```python from mechanize import make_response html = "Test" resp = make_response(html, [("Content-type", "text/html")]) ``` ``` -------------------------------- ### Building Openers Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/06-types.md Defines the signature for building an opener director with specified handlers. ```APIDOC ## build_opener ### Description Builds an OpenerDirector with the given handlers. ### Method `build_opener` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Parameters - **handlers** (*BaseHandler) - Variable number of BaseHandler instances to include in the opener. ### Response #### Success Response - **OpenerDirector** - An instance of OpenerDirector. ### Response Example ```json { "example": "OpenerDirector object" } ``` ``` -------------------------------- ### Open Local File Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/01-browser.md Opens a local HTML file for browsing. Provide the full path to the file. ```python br.open_local_file("/path/to/file.html") ``` -------------------------------- ### Enabling Gzip Compression and SSL Configuration Source: https://github.com/python-mechanize/mechanize/blob/master/docs/index.md Configures the browser to support gzip compression for requests and to handle SSL certificates, including disabling verification. Use for optimizing bandwidth and managing secure connections. ```python # Tell the browser to send the Accept-Encoding: gzip header to the server # to indicate it supports gzip Content-Encoding br.set_request_gzip(True) # Do not verify SSL certificates import ssl br.set_ca_data(context=ssl._create_unverified_context(cert_reqs=ssl.CERT_NONE)) ``` -------------------------------- ### Configure Proxy Handler Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/08-configuration.md Instantiate a ProxyHandler with a dictionary mapping URL schemes to their respective proxy URLs. This handler is used to set up proxy configurations. ```python handler = mechanize.ProxyHandler({ "http": "http://proxy:8080", "https": "https://proxy:8080" }) ``` -------------------------------- ### Build Opener with Proxy Handler Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/08-configuration.md Create a Mechanize opener instance that includes a configured ProxyHandler. This allows you to specify custom proxy settings for the opener. ```python opener = mechanize.build_opener( mechanize.ProxyHandler({ "http": "http://proxy:8080", "https": "https://proxy:8080" }) ) ``` -------------------------------- ### Submit a Form Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/01-browser.md Select a form, populate its fields, and submit it. The response object is returned. ```python br.select_form(name="login") br["username"] = "user" br["password"] = "pass" response = br.submit() ``` -------------------------------- ### Build Opener with Handlers using OpenerFactory Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/05-utilities.md Creates a custom opener chain using OpenerFactory, allowing the inclusion of various handlers like authentication and cookie processors. ```python from mechanize import OpenerFactory, HTTPBasicAuthHandler, HTTPPasswordMgr factory = OpenerFactory() pm = HTTPPasswordMgr() pm.add_password("Realm", "http://example.com", "user", "pass") opener = factory.build_opener( HTTPBasicAuthHandler(pm), mechanize.HTTPCookieProcessor() ) response = opener.open("http://example.com") ``` -------------------------------- ### Set Proxy Authentication Credentials Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/08-configuration.md Configure proxy authentication using a proxy password manager. Requires specifying proxy realm, host, username, and password. ```python browser.set_proxy_password_manager(password_mgr) ``` ```python from mechanize import HTTPProxyPasswordMgr, Browser ppm = HTTPProxyPasswordMgr() ppm.add_password("Proxy Realm", "proxy.example.com", "proxyuser", "proxypass") br = Browser() br.set_proxy_password_manager(ppm) ``` -------------------------------- ### Browser Constructor Options Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/08-configuration.md The Browser class constructor accepts several parameters to customize its behavior, such as history tracking, request class, content parsing, and factory class. ```APIDOC ## Browser Constructor Options The `Browser` class constructor accepts these parameters: ```python Browser( history=None, request_class=None, content_parser=None, factory_class=Factory, allow_xhtml=False ) ``` ### Parameters #### Constructor Parameters - **history** (History) - Optional - Custom history object for tracking navigation. Defaults to `History()`. - **request_class** (type) - Optional - Request class for form submissions and URL opening. Defaults to `mechanize.Request`. - **content_parser** (callable) - Optional - Custom HTML parser function with signature: `func(data, url, response_info, transport_encoding, default_encoding, is_html)`. Defaults to `None`. - **factory_class** (type) - Optional - HTML parsing factory for forms/links extraction. Defaults to `Factory`. - **allow_xhtml** (bool) - Optional - Whether to parse XHTML as HTML. Defaults to `False`. ``` -------------------------------- ### Set HTML Content and Select Form Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/01-browser.md Load a string of HTML content as the current response and immediately select the first form within it for interaction. Useful for testing form handling. ```python br.set_html("
") br.select_form(nr=0) ``` -------------------------------- ### CacheFTPHandler Initialization Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/04-handlers.md Initializes the CacheFTPHandler for FTP URLs, providing connection caching and timeout options. ```APIDOC ## CacheFTPHandler FTP handler with connection caching. **Module:** `mechanize._urllib2` **Signature:** ```python class CacheFTPHandler(FTPHandler): def __init__(self, cache=None, timeout=0): ``` | Parameter | Type | Description | |-----------|------|-------------| | cache | dict | Connection cache (default: automatic) | | timeout | int | Cache timeout in seconds (0 = unlimited) | ``` -------------------------------- ### Create Opener with Specific Handlers Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/08-configuration.md Build a custom opener by specifying individual handlers like HTTPSHandler, ProxyHandler, and HTTPBasicAuthHandler. This allows fine-grained control over how requests are processed. ```python opener = mechanize.build_opener( mechanize.HTTPSHandler(check_ssl_cert=False), mechanize.ProxyHandler({"http": "http://proxy:8080"}), mechanize.HTTPBasicAuthHandler(pm), mechanize.HTTPCookieProcessor(jar), mechanize.HTTPRefreshProcessor(max_time=3), mechanize.HTTPEquivProcessor() ) ``` -------------------------------- ### Set System Proxies Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/08-configuration.md Configure Mechanize to use the system's default proxy settings. This is the default behavior if no proxies are explicitly set. ```python br.set_proxies() ``` -------------------------------- ### OpenerDirector Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/06-types.md Chains together various handlers to process requests and responses, providing methods to open URLs and add custom handlers. ```APIDOC ## OpenerDirector **Module:** `mechanize._opener` **Type:** class Chains handlers for request/response processing. **Key methods:** ```python def open(fullurl: str | Request, data=None, timeout=_GLOBAL_DEFAULT_TIMEOUT) -> Response def add_handler(handler: BaseHandler) -> None ``` **Used by:** UserAgentBase (extends this) ``` -------------------------------- ### Create Mock Response for Testing Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/05-utilities.md Generates a dummy response object from provided data and headers. Useful for unit testing components that interact with responses. ```python from mechanize import make_response html = "Test" resp = make_response(html, [("Content-type", "text/html")]) ``` -------------------------------- ### Configure Supported URL Schemes Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/08-configuration.md Customize the list of URL schemes that the UserAgentBase instance will handle. By default, it supports 'http', 'https', 'ftp', and 'file'. ```python ua.set_handled_schemes(["http", "https"]) ``` -------------------------------- ### UserAgentBase.open Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/05-utilities.md Opens a URL using a UserAgentBase instance, compatible with the urllib2 interface. It can handle POST data and timeouts. ```APIDOC ## UserAgentBase.open ### Description Opens a URL (urllib2-compatible interface). Supports POST data and socket timeouts. ### Method open ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body - **fullurl** (str or Request) - Required - URL or Request object - **data** (dict or bytes) - Optional - POST data - **timeout** (float) - Optional - Socket timeout in seconds ### Request Example ```python ua = mechanize.UserAgentBase() response = ua.open("http://example.com") content = response.read() ``` ### Response #### Success Response - **Response object** - The object returned by opening the URL. #### Response Example ```python # Example response object usage content = response.read() ``` ``` -------------------------------- ### Set HTTP Authentication Credentials Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/08-configuration.md Configure HTTP authentication using a password manager. Supports default realm or per-realm password management. ```python browser.set_password_manager(password_mgr) ``` ```python from mechanize import HTTPPasswordMgr, HTTPPasswordMgrWithDefaultRealm, Browser # Basic setup pm = HTTPPasswordMgrWithDefaultRealm() pm.add_password(None, "http://example.com", "user", "password") br = Browser() br.set_password_manager(pm) # Per-realm passwords pm = HTTPPasswordMgr() pm.add_password("Realm Name", "http://example.com", "user", "pass") br.set_password_manager(pm) ``` -------------------------------- ### Browser Class Initialization Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/01-browser.md Initializes the Browser class, which serves as the main interface for programmatic web browsing. It supports stateful browsing, history, and HTML form/link parsing. ```APIDOC ## Browser Class Main browser class implementing the urllib2-like interface with additional HTML form and link support. **Module:** `mechanize._mechanize` ### Signature ```python class Browser(UserAgentBase): def __init__(self, history=None, request_class=None, content_parser=None, factory_class=Factory, allow_xhtml=False) ``` ### Parameters | Parameter | Type | Required | Default | Description | |-----------|------|----------|---------|-------------| | history | History | No | History() | Browser history object implementing the History interface. Must not be shared between browsers. | | request_class | type | No | mechanize.Request | Request class to instantiate for URL requests | | content_parser | callable | No | None | Function that parses received HTML/XHTML content. See _html.content_parser() for interface details | | factory_class | type | No | Factory | HTML Factory class for parsing forms, links, etc. | | allow_xhtml | bool | No | False | Whether to treat XHTML as HTML | ``` -------------------------------- ### Set Environment Variables for Proxy Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/08-configuration.md Configure proxy settings using environment variables. This is useful for system-wide proxy configurations that Mechanize can inherit. ```bash export http_proxy=http://proxy.example.com:8080 export no_proxy=localhost,internal.company.com python script.py ``` -------------------------------- ### Browser.open Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/01-browser.md Opens a URL and loads its content, making it available for subsequent form and link extraction. This method updates the browser's state, including history. ```APIDOC ## open Open a URL and load it for subsequent form/link extraction. ### Signature ```python def open(self, url_or_request, data=None, timeout=_GLOBAL_DEFAULT_TIMEOUT): ``` ### Parameters | Parameter | Type | Description | |-----------|------|-------------| | url_or_request | str or Request | URL string or Request object | | data | dict or bytes | Data for POST request. Dict is encoded as application/x-www-form-urlencoded | | timeout | float | Timeout in seconds | ### Returns Response object (file-like interface) ### Example ```python br = Browser() response = br.open("http://example.com") html_content = response.read() ``` ``` -------------------------------- ### HTTP Password Manager with Default Realm Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/04-handlers.md HTTPPasswordMgrWithDefaultRealm allows specifying default credentials when no specific realm matches. This is useful for providing fallback authentication. ```python pm = mechanize.HTTPPasswordMgrWithDefaultRealm() pm.add_password(None, "http://example.com", "user", "pass") # Default pm.add_password("Special Realm", "http://example.com", "admin", "admin_pass") ``` -------------------------------- ### Download URL to Local File with Progress Hook Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/05-utilities.md Downloads content from a URL to a local file, with an optional progress reporting hook. If no filename is provided, it's auto-generated. ```python from mechanize import urlretrieve def progress(blocknum, blocksize, totalsize): percent = min(blocknum * blocksize, totalsize) / totalsize * 100 print(f"Download: {percent:.1f}%") filename, headers = urlretrieve( "http://example.com/file.zip", "file.zip", reporthook=progress ) ``` -------------------------------- ### HTTPHandler Initialization Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/04-handlers.md Initializes the HTTPHandler for handling HTTP URLs. It allows customization of SSL certificate checking and the HTTP request class. ```APIDOC ## HTTPHandler Handles HTTP URLs. **Module:** `mechanize._urllib2` **Signature:** ```python class HTTPHandler(BaseHandler): def __init__(self, check_ssl_cert=True, http_request_class=None): ``` | Parameter | Type | Description | |-----------|------|-------------| | check_ssl_cert | bool | Whether to verify SSL certificates | | http_request_class | type | Custom Request class for HTTP | ``` -------------------------------- ### HTTP Authentication Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/00-index.md Configures HTTP Basic Authentication for a browser instance. ```python pm = mechanize.HTTPPasswordMgrWithDefaultRealm() pm.add_password(None, "http://example.com", "user", "password") br = mechanize.Browser() br.set_password_manager(pm) br.open("http://example.com/protected") ``` -------------------------------- ### build_opener Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/05-utilities.md Builds an opener with specified handlers. This is used to create custom request handlers for mechanize. ```APIDOC ## build_opener ### Description Build an opener with specified handlers. ### Method Signature ```python def build_opener(*handlers): ``` ### Parameters * **handlers**: A variable number of BaseHandler objects to include in the opener. ### Returns OpenerDirector ### Example ```python from mechanize import build_opener, HTTPBasicAuthHandler, HTTPPasswordMgr pm = HTTPPasswordMgr() pm.add_password("Realm", "http://example.com", "user", "pass") auth_handler = HTTPBasicAuthHandler(pm) opener = build_opener(auth_handler) response = opener.open("http://example.com/protected") ``` ``` -------------------------------- ### Click Image Submit Button Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/02-forms.md Simulate clicking an image submit button at specific coordinates. ```python # Click at image map coordinates req = form.click(name="map", coord=(100, 50)) ``` -------------------------------- ### Select Form by Index Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/00-index.md Demonstrates selecting a form on a page using its numerical index. ```python br.select_form(nr=0) ``` -------------------------------- ### Open URL with Mechanize Source: https://github.com/python-mechanize/mechanize/blob/master/docs/index.md This snippet shows the basic usage of mechanize for opening a URL and reading its content. It highlights that urllib2 imports should be replaced with mechanize imports when using the library. ```python import mechanize response = mechanize.urlopen("http://www.example.com/") print(response.read()) ``` -------------------------------- ### follow_link Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/01-browser.md Finds a link based on the provided criteria and then opens that link, navigating the browser to the linked resource. ```APIDOC ## follow_link ### Description Find a link and open it. ### Method Not specified (assumed to be part of a class) ### Endpoint Not applicable ### Parameters #### Path Parameters - **link** (Link) - Optional - Pre-found Link object #### Query Parameters - **kwds** (varies) - Optional - find_link() keyword arguments if link not provided ### Request Example ```python # Follow link by text br.follow_link(text="Next Page") # Follow pre-found link link = br.find_link(text_regex=r"python") br.follow_link(link=link) ``` ### Response **Returns:** Response object **Raises:** LinkNotFoundError - if no matching link found ``` -------------------------------- ### Follow Link Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/01-browser.md Finds a link based on provided criteria and opens it, navigating the browser to the linked URL. Can also follow a pre-found Link object. ```python # Follow link by text br.follow_link(text="Next Page") ``` ```python # Follow pre-found link link = br.find_link(text_regex=r"python") br.follow_link(link=link) ``` -------------------------------- ### Add a Cookie to CookieJar Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/03-cookies.md Demonstrates how to create a Cookie object and add it to a CookieJar. This is useful for manually managing cookies before making requests. ```python jar = mechanize.CookieJar() cookie = mechanize.Cookie( version=0, name="session", value="abc123", port=None, port_specified=False, domain=".example.com", domain_specified=True, domain_initial_dot=True, path="/", path_specified=True, secure=False, expires=None, discard=True, comment=None, comment_url=None, rfc2109=False ) jar.add_cookie(cookie) ``` -------------------------------- ### HTTPSHandler Initialization Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/04-handlers.md Initializes the HTTPSHandler for handling HTTPS URLs. It supports SSL certificate verification and custom HTTPS request classes. ```APIDOC ## HTTPSHandler Handles HTTPS URLs. **Module:** `mechanize._urllib2` **Signature:** ```python class HTTPSHandler(BaseHandler): def __init__(self, check_ssl_cert=True, https_request_class=None): ``` | Parameter | Type | Description | |-----------|------|-------------| | check_ssl_cert | bool | Whether to verify SSL certificates | | https_request_class | type | Custom Request class for HTTPS | ``` -------------------------------- ### Follow Link Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/00-index.md Shows how to follow a specific link on a web page by its text content. ```python br.follow_link(text="Next Page") ``` -------------------------------- ### Configure Proxy Servers Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/08-configuration.md Set proxy servers for HTTP and HTTPS traffic. Can also specify a function to bypass proxies for certain hosts. ```python browser.set_proxies(proxies, proxy_bypass=None) ``` ```python # Manual proxy br.set_proxies({ "http": "http://proxy.example.com:8080", "https": "https://proxy.example.com:8080" }) ``` -------------------------------- ### Handle HTTP Errors Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/00-index.md Shows how to catch and handle HTTP errors, such as 404 Not Found. ```python except mechanize.HTTPError as e: if e.code == 404: print("Not found") else: print(f"HTTP {e.code}: {e.msg}") ``` -------------------------------- ### History Key Methods Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/06-types.md Manages browser navigation history. Methods include adding history entries, navigating back, and clearing history. ```python def add(request: Request, response: Response) -> None def back(n: int, _response: Response) -> tuple[Request, Response] def clear() -> None def close() -> None ``` -------------------------------- ### AbstractDigestAuthHandler Initialization Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/04-handlers.md Base class for Digest authentication handlers, requiring a password manager. ```APIDOC ## AbstractDigestAuthHandler Base class for Digest authentication handlers. **Module:** `mechanize._urllib2` **Signature:** ```python class AbstractDigestAuthHandler(BaseHandler): def __init__(self, password_mgr): ``` ``` -------------------------------- ### Basic HTTP Authentication Handler Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/04-handlers.md Configure HTTP Basic Authentication using HTTPBasicAuthHandler. This handler requires a password manager instance containing the credentials. ```python pm = mechanize.HTTPPasswordMgr() pm.add_password("Realm", "http://example.com", "user", "password") handler = mechanize.HTTPBasicAuthHandler(pm) opener = mechanize.build_opener(handler) response = opener.open("http://example.com/protected") ``` -------------------------------- ### Factory Constructor and Type Annotations Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/06-types.md Creates forms and links from HTML responses. The constructor allows specifying default encoding and XHTML compatibility. ```python class Factory: def __init__(self, default_encoding="utf-8", allow_xhtml=False) default_encoding: str allow_xhtml: bool ``` -------------------------------- ### Manually Creating and Opening a Request with Mechanize Source: https://github.com/python-mechanize/mechanize/blob/master/docs/faq.md Emulate browser requests by manually creating a mechanize.Request object and opening it. This is useful when JavaScript modifies form submissions or content dynamically. ```python request = mechanize.Request("http://example.com/page") request.add_header("User-Agent", "My User Agent") response = browser.open(request) ``` -------------------------------- ### Create mechanize Browser Instance Source: https://github.com/python-mechanize/mechanize/blob/master/docs/browser_api.md Instantiate a mechanize Browser object to begin web scraping or automation tasks. ```python from mechanize import Browser br = Browser() ``` -------------------------------- ### Click Submit Button Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/02-forms.md Find a submit button by its name and simulate a click action to submit the form. ```python # Find and click specific submit button req = form.click(name="save_button") ``` -------------------------------- ### Enabling Debugging Output Source: https://github.com/python-mechanize/mechanize/blob/master/docs/index.md Configures the browser to log detailed information about redirects, response bodies, and HTTP headers. Essential for troubleshooting and understanding the browser's actions. ```python # Log information about HTTP redirects and Refreshes. br.set_debug_redirects(True) # Log HTTP response bodies (i.e. the HTML, most of the time). br.set_debug_responses(True) # Print HTTP headers. br.set_debug_http(True) # To make sure you're seeing all debug output: logger = logging.getLogger("mechanize") logger.addHandler(logging.StreamHandler(sys.stdout)) logger.setLevel(logging.INFO) ``` -------------------------------- ### Proxy Configuration Source: https://github.com/python-mechanize/mechanize/blob/master/_autodocs/08-configuration.md Methods for configuring proxy servers for HTTP and HTTPS requests, including setting specific proxy URLs and defining bypass hosts. ```APIDOC ## Proxy Configuration ### `set_proxies(proxies, proxy_bypass=None)` Configure proxy servers. | Parameter | Type | Description | |-----------|------|-------------| | proxies | dict | Mapping from scheme to proxy URL. `None` uses system settings. | | proxy_bypass | callable | Function(hostname) returning bool - `True` to bypass proxy. | **Examples:** ```python # Manual proxy br.set_proxies({ "http": "http://proxy.example.com:8080", "https": "https://proxy.example.com:8080" }) ``` ```