### wget Download Example Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/download_reference.md Use wget to download resources by reading URLs from a file. ```bash wget -B basefile.txt ``` -------------------------------- ### Practical Examples from README Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/resource_formats.md Shows typical records found in the 'papers.txt' file, derived from references in the project's README. ```text Outlier Analysis | https://link.springer.com/book/10.1007/978-3-319-47578-3 ``` ```text Outlier Ensembles: An Introduction | https://www.springer.com/gp/book/9783319547640 ``` ```text Data Mining: Concepts and Techniques (3rd) | https://www.elsevier.com/books/data-mining-concepts-and-techniques/han/978-0-12-381479-1 ``` ```text LOF: identifying density-based local outliers | http://www.dbs.ifi.lmu.de/Publikationen/Papers/LOF.pdf ``` ```text Isolation forest | https://cs.nju.edu.cn/zhouzh/zhouzh.files/publication/icdm08b.pdf ``` ```text COPOD: Copula-Based Outlier Detection | https://arxiv.org/abs/2009.09463 ``` ```text ECOD: Unsupervised Outlier Detection Using Empirical Cumulative Distribution Functions | https://arxiv.org/abs/2201.00382 ``` -------------------------------- ### Install and Run URL Checker Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/TECHNICAL_SUMMARY.md Install necessary Python packages and run the url_checker.py script. Suitable for CI/CD pipelines, scheduled checks, and pre-publication validation. ```bash pip install requests urllib3 python url_checker.py ``` -------------------------------- ### Example Execution of download.py Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/download_reference.md Demonstrates how to run the download.py script from the project's root directory. The output shows the progress messages printed during the download process. ```bash # Run from project root directory python download.py # Output: # Downloading Paper Title from https://arxiv.org/abs/1234.5678 # Downloading Another Paper from https://example.com/paper.pdf # ... ``` -------------------------------- ### Example Records from papers.txt Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/resource_formats.md Provides sample records as they would appear in the 'papers.txt' file, adhering to the pipe-delimited format. ```text Isolation Forest | https://cs.nju.edu.cn/zhouzh/zhouzh.files/publication/icdm08b.pdf ``` ```text XGBOD: improving supervised outlier detection with unsupervised representation learning | https://arxiv.org/abs/1912.00290 ``` ```text PyOD: A Python Toolbox for Scalable Outlier Detection | https://www.jmlr.org/papers/volume20/19-011/19-011.pdf ``` -------------------------------- ### Example Usage of build_session Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/url_checker_reference.md Demonstrates how to obtain a configured session object and use it to perform a HEAD request with a specified timeout. ```python session = build_session() response = session.head("https://example.com", timeout=8) ``` -------------------------------- ### Example Usage of should_fallback_to_get Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/url_checker_reference.md Shows how to use the `should_fallback_to_get` function to conditionally perform a GET request after a HEAD request returns a specific status code. ```python if should_fallback_to_get(405): # Try GET after HEAD returned 405 Method Not Allowed response = session.get(url) ``` -------------------------------- ### Script Execution Example Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/api_summary.md Demonstrates how to run the url_checker script from the command line. Optional arguments allow specifying input files, timeouts, and worker counts. ```bash python url_checker.py [--file FILE] [--timeout SEC] [--workers N] [--show-ok] ``` -------------------------------- ### Example CheckResult Instance: Successful Check (GET Fallback) Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/data_types.md An example of a CheckResult instance representing a successful GET request after a HEAD request failed or was not suitable. ```python CheckResult( url='https://arxiv.org/pdf/1912.00290.pdf', ok=True, status_code=200, method='GET', detail='OK (fallback)' ) ``` -------------------------------- ### Example Usage of URL Checker Utility Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/url_checker_reference.md Demonstrates how to run the url_checker.py script from the command line with different options for checking files, setting timeouts, and controlling output. ```bash # Check default README.rst python url_checker.py # Check with custom timeout and workers python url_checker.py --file README_CN.rst --timeout 8 --workers 20 # Show all URLs including successful ones python url_checker.py --show-ok ``` -------------------------------- ### Direct File Import Example (Python) Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/api_summary.md Demonstrates how to import modules directly from files within the project root when the project is not structured as a package. Ensure the project path is correctly added to `sys.path`. ```python # These will NOT work: from anomaly-detection-resources import download # ❌ Not a package from anomaly-detection-resources import url_checker # ❌ Not a package # This works (files are in project root): import sys sys.path.insert(0, '/path/to/project') from url_checker import extract_urls # ✓ Direct file import from download import * # ✓ Would work but file not designed for import ``` -------------------------------- ### Console Output Example Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/download_reference.md This snippet shows the expected console output during the script's execution, indicating which papers are being downloaded and from where. It continues for each line in the input file. ```text Downloading PyOD A Python Toolbox for Scalable Outlier Detection from https://www.jmlr.org/papers/volume20/19-011/19-011.pdf Downloading XGBOD improving supervised outlier detection with unsupervised representation learning from https://arxiv.org/abs/1912.00290 Downloading Isolation Forest from https://cs.nju.edu.cn/zhouzh/zhouzh.files/publication/icdm08b.pdf [... continues for each line in papers.txt ...] ``` -------------------------------- ### Example CheckResult Instance: Successful Check (HEAD) Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/data_types.md An example of a CheckResult instance representing a successful HEAD request to a URL. ```python CheckResult( url='https://github.com/yzhao062/pyod', ok=True, status_code=200, method='HEAD', detail='OK' ) ``` -------------------------------- ### Import Statements for download.py Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/INDEX.md These are the import statements required for the download.py script. Verify that these modules are installed. ```python import re, pathlib, urllib.request ``` -------------------------------- ### Example CheckResult Instance: Server Error with GET Fallback Failure Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/data_types.md An example of a CheckResult instance showing a server error (403 Forbidden) where a GET fallback also failed. ```python CheckResult( url='https://example.com/forbidden', ok=False, status_code=403, method='GET', detail='Fallback failed after HEAD 403' ) ``` -------------------------------- ### Example CheckResult Instance: HTTP Error (404 Not Found) Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/data_types.md An example of a CheckResult instance indicating an HTTP 404 Not Found error for a URL. ```python CheckResult( url='https://example.com/missing-paper.pdf', ok=False, status_code=404, method='HEAD', detail='HTTP error' ) ``` -------------------------------- ### Example CheckResult Instance: Request Timeout Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/data_types.md An example of a CheckResult instance indicating a request timeout when trying to access a URL. ```python CheckResult( url='https://very-slow-server.example/paper.pdf', ok=False, status_code=None, method='HEAD', detail='RequestException: HTTPSConnectionPool(host=\'very-slow-server.example\', port=443): Read timed out.' ) ``` -------------------------------- ### Programmatic URL Checking Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/api_summary.md Shows how to import and use key functions from the url_checker module in Python scripts. This example extracts URLs from content, checks them individually, and prints the results. ```python from url_checker import extract_urls, check_one_url, build_session urls = extract_urls(content) for url in urls: result = check_one_url(url, timeout=8) print(f"{result.url}: {result.detail}") ``` -------------------------------- ### reStructuredText Grid Table Example Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/resource_formats.md Illustrates the reStructuredText grid table format for organizing data, including headers and rows with different column types. ```rst ============ ====================== ===== ========== ===================== Column 1 Column 2 Year Reference Materials ============ ====================== ===== ========== ===================== Data Type Paper Title 2024 [#Ref]_ `[PDF] `_ ============ ====================== ===== ========== ===================== ``` -------------------------------- ### Example Usage of check_one_url Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/url_checker_reference.md Illustrates how to call `check_one_url` and interpret the returned `CheckResult` object to display whether the URL is valid and the details of the check. ```python result = check_one_url("https://example.com/paper.pdf", timeout=8) if result.ok: print(f"✓ {result.url} [{result.method} {result.status_code}]") else: print(f"✗ {result.url} [{result.method}] {result.detail}") ``` -------------------------------- ### Configure requests.Session with Retry Strategy Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/data_types.md Configure a reusable HTTP session with automatic retry logic for specified HTTP status codes and methods. This setup enhances robustness for network requests. ```python session = requests.Session() # Retry strategy retry = Retry( total=2, # Max 2 total retries connect=2, # Max 2 connection retries read=2, # Max 2 read retries backoff_factor=0.5, # Exponential backoff status_forcelist=(429, 500, 502, 503, 504), # Retry on these HTTP codes allowed_methods=frozenset({"HEAD", "GET"}) # Only retry these methods ) adapter = HTTPAdapter(max_retries=retry) session.mount("http://", adapter) session.mount("https://", adapter) # Browser-like User-Agent session.headers.update({ "User-Agent": "Mozilla/5.0 (compatible; URLChecker/2.0; +https://github.com/yzhao062/anomaly-detection-resources)" }) ``` -------------------------------- ### URL Checker Common Failure: OK (fallback) Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/url_checker_reference.md Explanation of the 'OK (fallback)' message, indicating that while the HEAD request failed, the GET request to the URL succeeded, suggesting the server may not support HEAD requests. ```text "OK (fallback)" | HEAD failed but GET succeeded | Server doesn't support HEAD ``` -------------------------------- ### Example CheckResult Instance: Network Connection Error Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/data_types.md An example of a CheckResult instance representing a network connection error, such as a DNS resolution failure. ```python CheckResult( url='https://unreachable-domain.invalid/paper', ok=False, status_code=None, method='HEAD/GET', detail='RequestException: [Errno -3] Name or service not known' ) ``` -------------------------------- ### Recommended Download Tools (Unix) Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/TECHNICAL_SUMMARY.md On Unix systems, it is recommended to use `wget` or `aria2c` for downloading files from a list of URLs. These tools offer features like parallel downloads and resume capabilities. ```bash # wget -B basefile.txt # reads URLs from file with -O naming # aria2c -i url_list.txt # parallel downloads with resume ``` -------------------------------- ### Determine Fallback to GET Request Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/url_checker_reference.md Checks if a HEAD request's status code indicates that a fallback to a GET request might be appropriate. This is useful for servers that do not properly support HEAD requests. ```python def should_fallback_to_get(status_code: int) -> bool: pass ``` -------------------------------- ### should_fallback_to_get(status_code: int) Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/url_checker_reference.md Determines if a GET request should be attempted after a HEAD request returns a specific status code. This is useful for handling servers that might not support HEAD requests properly but do support GET. ```APIDOC ## should_fallback_to_get(status_code: int) ### Description Determine whether to fall back from HEAD request to GET request based on HTTP response status code. ### Parameters #### Path Parameters - **status_code** (int) - Required - HTTP status code from HEAD request ### Returns `True` if fallback to GET should be attempted, `False` otherwise ### Fallback Status Codes 403, 405, 406, 429, 500, 501, 502, 503 ### Status Code Meanings - `403` Forbidden (may allow GET) - `405` Method Not Allowed (likely allows GET) - `406` Not Acceptable - `429` Too Many Requests (rate limiting) - `500` Internal Server Error (temporary) - `501` Not Implemented (HEAD not supported) - `502` Bad Gateway (temporary) - `503` Service Unavailable (temporary) ### Example ```python if should_fallback_to_get(405): # Try GET after HEAD returned 405 Method Not Allowed response = session.get(url) ``` ``` -------------------------------- ### Basic Download Script Usage Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/download_reference.md Run the download script and check the results. Ensure the input file 'resource_urls/papers.txt' exists and is correctly formatted before execution. ```bash # Ensure resource_urls/papers.txt exists with data in format: # Paper Title | https://example.com/paper.pdf # Run download python download.py # Check results ls -lh resources/ | head ``` -------------------------------- ### Run Download Script (Windows) Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/TECHNICAL_SUMMARY.md Execute the download.py script on Windows systems. Note that this script has known issues and is not recommended for production without fixes. Verify downloads using 'ls resources/ | wc -l'. ```bash # Windows only python download.py ls resources/ | wc -l # Verify downloads ``` -------------------------------- ### Example Transformation with Hyphens and Apostrophes Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/resource_formats.md Illustrates that apostrophes and hyphens are retained in the paper title after special character removal. ```text "Self-Supervised Anomaly Detection: The Solution" → "Self-Supervised Anomaly Detection The Complete Solution" ``` -------------------------------- ### Python Path Handling (Windows vs. Unix) Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/download_reference.md Demonstrates the incorrect hardcoded Windows path separators that fail on Unix-like systems and the correct approach using `pathlib` for cross-platform compatibility. ```python # Line 17 - FAILS on Unix f = open('resource_urls\papers.txt', 'r') ``` ```python # Line 30 - FAILS on Unix urllib.request.urlretrieve(url, "resources\" + file_name + '.pdf') ``` ```python from pathlib import Path input_file = Path('resource_urls') / 'papers.txt' output_file = Path('resources') / f'{file_name}.pdf' ``` -------------------------------- ### Run download.py Script Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/api_summary.md Executes the download.py script to download papers from a specified text file to a designated directory. Configuration is hardcoded within the script. ```bash python download.py ``` -------------------------------- ### reStructuredText Footnote Reference Example Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/resource_formats.md Shows the reStructuredText syntax for in-text footnote references and their corresponding definitions at the end of the document. ```rst [#Zhao2019PYOD]_ (in-text reference) .. [#Zhao2019PYOD] Zhao, Y... (at end of document in References section) ``` -------------------------------- ### Check Disk Space Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/download_reference.md Before downloading, verify that there is sufficient disk space available. Consider that typical academic papers range from 500KB to 5MB each. ```bash # Ensure sufficient space for PDFs df -h . # Typical academic papers: 500KB - 5MB each ``` -------------------------------- ### Verify Downloaded Files are PDFs Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/download_reference.md After downloading, use the 'file' command to verify that the downloaded files are indeed PDFs. ```bash file resources/*.pdf ``` -------------------------------- ### Verify Input File Format Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/download_reference.md Check the first few lines of the input file to ensure it adheres to the expected 'Paper Title | URL' format. ```bash head resource_urls/papers.txt # Output should be: # Paper Title 1 | https://example.com/paper1.pdf # Paper Title 2 | https://example.com/paper2.pdf ``` -------------------------------- ### Download All Papers using download.py Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/INDEX.md Execute this command to download all available papers. Note that this script may require modifications for Unix-based systems due to path issues. ```bash python download.py # Creates resources/ with downloaded PDFs ``` -------------------------------- ### Show help with url_checker.py Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/api_summary.md Displays the help message for the url_checker.py script. ```bash python url_checker.py --help ``` -------------------------------- ### Example Transformation of Special Characters Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/resource_formats.md Demonstrates how special characters are removed from a paper title string. Quotes are preserved, while colons and angle brackets are removed. ```text Input: "A 'Great' Paper: The | More Data" Output: "A 'Great' Paper The Story More Data" (quotes are preserved, colons removed, angle brackets removed) ``` -------------------------------- ### Full Workflow with url_checker.py Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/api_summary.md This pattern demonstrates the complete workflow: parsing arguments, reading a file, extracting URLs, checking them concurrently, and reporting results. It utilizes ThreadPoolExecutor for parallel processing. ```python from url_checker import extract_urls, check_all, parse_args # 1. Get arguments args = parse_args() # 2. Read file with open(args.file) as f: content = f.read() # 3. Extract URLs urls = extract_urls(content) # 4. Check all results = check_all(urls, timeout=args.timeout, workers=args.workers) # 5. Report for result in results: if not result.ok: print(f"Failed: {result.url} - {result.detail}") ``` -------------------------------- ### Main Entry Point for URL Checker Utility Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/url_checker_reference.md The main function serves as the entry point for the command-line utility. It handles argument parsing, URL extraction, concurrent checking, and result reporting, returning an appropriate exit code. ```python def main() -> int: pass ``` -------------------------------- ### URL Checker Common Failure: Fallback Failed Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/url_checker_reference.md Explanation of the 'Fallback failed after HEAD NNN' message, meaning both HEAD and GET requests to the URL resulted in an error. ```text "Fallback failed after HEAD NNN" | GET after HEAD failed | Both HEAD and GET returned error ``` -------------------------------- ### URL Checker CLI Options Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/url_checker_reference.md Available command-line options for configuring the URL checker, such as input file, timeout, and concurrency. ```bash --file FILE Path to RST/Markdown file to check (default: README.rst) --timeout SECONDS HTTP request timeout in seconds (default: 8) --workers NUM Number of concurrent worker threads (default: 16) --show-ok Print successful URLs in output (default: off) --help, -h Show help message and exit ``` -------------------------------- ### Check Single URL with Fallback Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/url_checker_reference.md Validates a single URL by first attempting an HTTP HEAD request and falling back to a GET request if necessary. It returns a detailed `CheckResult` object. ```python def check_one_url(url: str, timeout: int) -> CheckResult: pass ``` -------------------------------- ### Python Exception Handling for Downloads Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/download_reference.md Illustrates how to wrap file download operations in a try-except block to gracefully handle potential errors such as network issues or invalid URLs, preventing script crashes. ```python try: urllib.request.urlretrieve(url, str(output_file)) print(f"✓ Downloaded {file_name}") except Exception as e: print(f"✗ Failed to download {file_name}: {e}") continue ``` -------------------------------- ### Check default file with url_checker.py Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/api_summary.md Runs the url_checker.py script using the default input file. ```bash python url_checker.py ``` -------------------------------- ### reStructuredText Hyperlink Syntax Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/resource_formats.md Demonstrates the syntax for creating hyperlinks in reStructuredText, both as explicit URLs and as inline references. ```rst `Link Text `_ ``` ```rst See `PyOD library `_ ``` -------------------------------- ### should_fallback_to_get Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/api_summary.md Determines if a fallback to the HTTP GET method should be attempted after an initial HEAD request fails or returns a specific status code. This is useful for servers that might not handle HEAD requests correctly. ```APIDOC ## should_fallback_to_get ### Description Determine if GET should be tried after HEAD. ### Signature `should_fallback_to_get(status_code: int) -> bool` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```python fallback = should_fallback_to_get(405) # Example status code print(fallback) ``` ### Response #### Success Response - **fallback** (bool) - True if GET should be tried, False otherwise. ### Response Example ```json { "fallback_to_get": true } ``` ``` -------------------------------- ### Run URL Checker with Custom Parameters Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/url_checker_reference.md Executes the URL checker script with custom file, timeout, and worker count. ```bash python url_checker.py --file README_CN.rst --timeout 10 --workers 20 ``` -------------------------------- ### Custom file and timeout with url_checker.py Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/api_summary.md Runs the url_checker.py script with a specified input file and request timeout. ```bash python url_checker.py --file README_CN.rst --timeout 10 ``` -------------------------------- ### URL Checker Module Summary Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/TECHNICAL_SUMMARY.md This Python module extracts and validates HTTP/HTTPS URLs from content using concurrent requests. It supports fallback from HEAD to GET requests for servers that reject HEAD requests. ```python class CheckResult: url: str ok: bool status_code: int | None method: str detail: str ``` -------------------------------- ### Check README URLs with Verbose Reporting Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/TECHNICAL_SUMMARY.md Use this command to check URLs found in a README file with detailed reporting and multiple workers for faster processing. ```bash python url_checker.py --file README.rst --show-ok --workers 20 ``` -------------------------------- ### Extract URLs from Content Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/resource_formats.md Uses a regular expression to find all raw URLs starting with http or https within a given string. The pattern matches any non-whitespace characters following the protocol until a whitespace character is encountered. ```python raw_urls = re.findall(r"https?://\S+", content) ``` -------------------------------- ### check_one_url(url: str, timeout: int) Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/url_checker_reference.md Checks the validity of a single URL using an HTTP HEAD request, with an automatic fallback to a GET request if necessary. It returns a detailed result object indicating the URL's status, the method used, and any relevant details. ```APIDOC ## check_one_url(url: str, timeout: int) ### Description Check the validity of a single URL using HTTP HEAD request (with automatic GET fallback). Returns detailed result including status code and method used. ### Parameters #### Path Parameters - **url** (str) - Required - URL to check - **timeout** (int) - Required - Request timeout in seconds ### Returns `CheckResult` dataclass with fields: - `url` (str): The URL that was checked - `ok` (bool): Whether the URL is valid (HTTP status < 400) - `status_code` (int | None): HTTP status code (None if connection failed) - `method` (str): HTTP method used ("HEAD", "GET", or "HEAD/GET") - `detail` (str): Human-readable status message ### Behavior 1. Attempts HEAD request to the URL 2. If status code is < 400 (success), returns `CheckResult(ok=True)` 3. If status code matches fallback list (403, 405, etc.), attempts GET request: - If GET succeeds (< 400), returns `CheckResult(ok=True, method="GET", detail="OK (fallback)")` - If GET fails, returns `CheckResult(ok=False)` with GET failure details 4. If HEAD fails with non-fallback error, returns `CheckResult(ok=False)` 5. On any `RequestException`, returns `CheckResult(ok=False, detail="RequestException: ...")` ### Example ```python result = check_one_url("https://example.com/paper.pdf", timeout=8) if result.ok: print(f"✓ {result.url} [{result.method} {result.status_code}]") else: print(f"✗ {result.url} [{result.method}] {result.detail}") ``` ``` -------------------------------- ### Understand input file format for URLs Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/README.md The input file format for URLs consists of a title and a URL, separated by a pipe symbol ('|'). Each line represents a distinct entry. Ensure URLs are correctly formatted and accessible. ```text PyOD: A Python Toolbox for Scalable Outlier Detection | https://www.jmlr.org/papers/volume20/19-011/19-011.pdf Isolation Forest | https://cs.nju.edu.cn/zhouzh/zhouzh.files/publication/icdm08b.pdf ``` -------------------------------- ### Run URL Checker CLI Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/TECHNICAL_SUMMARY.md Execute the url_checker.py script from the command line to check URLs in a specified file. Configure timeout, worker count, and whether to display successful checks. ```bash python url_checker.py --file README.rst --timeout 8 --workers 16 --show-ok ``` -------------------------------- ### build_session Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/api_summary.md Creates and configures a `requests.Session` object. This session is pre-configured with retry mechanisms suitable for making HTTP requests. ```APIDOC ## build_session ### Description Create a configured requests session with retries. ### Signature `build_session() -> requests.Session` ### Parameters None ### Request Example ```python session = build_session() response = session.get("https://example.com") print(response.status_code) ``` ### Response #### Success Response - **session** (requests.Session) - A configured HTTP session object. ### Response Example ```json { "session_type": "requests.Session" } ``` ```