Anomaly Detection Resources

https://github.com/yzhao062/anomaly-detection-resources
Admin
A curated collection of resources for anomaly detection, including papers, datasets, and code.
Tokens:33,050
Snippets:592
Trust Score:9.3
Update:3 months ago
Show doc for...
Tokens:
Raw
### wget Download Example

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/download_reference.md

Use wget to download resources by reading URLs from a file.

```bash
wget -B basefile.txt
```

--------------------------------

### Practical Examples from README

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/resource_formats.md

Shows typical records found in the 'papers.txt' file, derived from references in the project's README.

```text
Outlier Analysis | https://link.springer.com/book/10.1007/978-3-319-47578-3
```

```text
Outlier Ensembles: An Introduction | https://www.springer.com/gp/book/9783319547640
```

```text
Data Mining: Concepts and Techniques (3rd) | https://www.elsevier.com/books/data-mining-concepts-and-techniques/han/978-0-12-381479-1
```

```text
LOF: identifying density-based local outliers | http://www.dbs.ifi.lmu.de/Publikationen/Papers/LOF.pdf
```

```text
Isolation forest | https://cs.nju.edu.cn/zhouzh/zhouzh.files/publication/icdm08b.pdf
```

```text
COPOD: Copula-Based Outlier Detection | https://arxiv.org/abs/2009.09463
```

```text
ECOD: Unsupervised Outlier Detection Using Empirical Cumulative Distribution Functions | https://arxiv.org/abs/2201.00382
```

--------------------------------

### Install and Run URL Checker

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/TECHNICAL_SUMMARY.md

Install necessary Python packages and run the url_checker.py script. Suitable for CI/CD pipelines, scheduled checks, and pre-publication validation.

```bash
pip install requests urllib3
python url_checker.py
```

--------------------------------

### Example Execution of download.py

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/download_reference.md

Demonstrates how to run the download.py script from the project's root directory. The output shows the progress messages printed during the download process.

```bash
# Run from project root directory
python download.py

# Output:
# Downloading Paper Title from https://arxiv.org/abs/1234.5678
# Downloading Another Paper from https://example.com/paper.pdf
# ...
```

--------------------------------

### Example Records from papers.txt

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/resource_formats.md

Provides sample records as they would appear in the 'papers.txt' file, adhering to the pipe-delimited format.

```text
Isolation Forest | https://cs.nju.edu.cn/zhouzh/zhouzh.files/publication/icdm08b.pdf
```

```text
XGBOD: improving supervised outlier detection with unsupervised representation learning | https://arxiv.org/abs/1912.00290
```

```text
PyOD: A Python Toolbox for Scalable Outlier Detection | https://www.jmlr.org/papers/volume20/19-011/19-011.pdf
```

--------------------------------

### Example Usage of build_session

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/url_checker_reference.md

Demonstrates how to obtain a configured session object and use it to perform a HEAD request with a specified timeout.

```python
session = build_session()
response = session.head("https://example.com", timeout=8)
```

--------------------------------

### Example Usage of should_fallback_to_get

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/url_checker_reference.md

Shows how to use the `should_fallback_to_get` function to conditionally perform a GET request after a HEAD request returns a specific status code.

```python
if should_fallback_to_get(405):
    # Try GET after HEAD returned 405 Method Not Allowed
    response = session.get(url)
```

--------------------------------

### Script Execution Example

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/api_summary.md

Demonstrates how to run the url_checker script from the command line. Optional arguments allow specifying input files, timeouts, and worker counts.

```bash
python url_checker.py [--file FILE] [--timeout SEC] [--workers N] [--show-ok]
```

--------------------------------

### Example CheckResult Instance: Successful Check (GET Fallback)

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/data_types.md

An example of a CheckResult instance representing a successful GET request after a HEAD request failed or was not suitable.

```python
CheckResult(
    url='https://arxiv.org/pdf/1912.00290.pdf',
    ok=True,
    status_code=200,
    method='GET',
    detail='OK (fallback)'
)
```

--------------------------------

### Example Usage of URL Checker Utility

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/url_checker_reference.md

Demonstrates how to run the url_checker.py script from the command line with different options for checking files, setting timeouts, and controlling output.

```bash
# Check default README.rst
python url_checker.py

# Check with custom timeout and workers
python url_checker.py --file README_CN.rst --timeout 8 --workers 20

# Show all URLs including successful ones
python url_checker.py --show-ok
```

--------------------------------

### Direct File Import Example (Python)

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/api_summary.md

Demonstrates how to import modules directly from files within the project root when the project is not structured as a package. Ensure the project path is correctly added to `sys.path`.

```python
# These will NOT work:
from anomaly-detection-resources import download  # ❌ Not a package
from anomaly-detection-resources import url_checker  # ❌ Not a package

# This works (files are in project root):
import sys
sys.path.insert(0, '/path/to/project')
from url_checker import extract_urls  # ✓ Direct file import
from download import *  # ✓ Would work but file not designed for import
```

--------------------------------

### Console Output Example

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/download_reference.md

This snippet shows the expected console output during the script's execution, indicating which papers are being downloaded and from where. It continues for each line in the input file.

```text
Downloading PyOD A Python Toolbox for Scalable Outlier Detection from https://www.jmlr.org/papers/volume20/19-011/19-011.pdf
Downloading XGBOD improving supervised outlier detection with unsupervised representation learning from https://arxiv.org/abs/1912.00290
Downloading Isolation Forest from https://cs.nju.edu.cn/zhouzh/zhouzh.files/publication/icdm08b.pdf
[... continues for each line in papers.txt ...]
```

--------------------------------

### Example CheckResult Instance: Successful Check (HEAD)

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/data_types.md

An example of a CheckResult instance representing a successful HEAD request to a URL.

```python
CheckResult(
    url='https://github.com/yzhao062/pyod',
    ok=True,
    status_code=200,
    method='HEAD',
    detail='OK'
)
```

--------------------------------

### Import Statements for download.py

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/INDEX.md

These are the import statements required for the download.py script. Verify that these modules are installed.

```python
import re, pathlib, urllib.request
```

--------------------------------

### Example CheckResult Instance: Server Error with GET Fallback Failure

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/data_types.md

An example of a CheckResult instance showing a server error (403 Forbidden) where a GET fallback also failed.

```python
CheckResult(
    url='https://example.com/forbidden',
    ok=False,
    status_code=403,
    method='GET',
    detail='Fallback failed after HEAD 403'
)
```

--------------------------------

### Example CheckResult Instance: HTTP Error (404 Not Found)

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/data_types.md

An example of a CheckResult instance indicating an HTTP 404 Not Found error for a URL.

```python
CheckResult(
    url='https://example.com/missing-paper.pdf',
    ok=False,
    status_code=404,
    method='HEAD',
    detail='HTTP error'
)
```

--------------------------------

### Example CheckResult Instance: Request Timeout

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/data_types.md

An example of a CheckResult instance indicating a request timeout when trying to access a URL.

```python
CheckResult(
    url='https://very-slow-server.example/paper.pdf',
    ok=False,
    status_code=None,
    method='HEAD',
    detail='RequestException: HTTPSConnectionPool(host=\'very-slow-server.example\', port=443): Read timed out.'
)
```

--------------------------------

### Programmatic URL Checking

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/api_summary.md

Shows how to import and use key functions from the url_checker module in Python scripts. This example extracts URLs from content, checks them individually, and prints the results.

```python
from url_checker import extract_urls, check_one_url, build_session
urls = extract_urls(content)
for url in urls:
    result = check_one_url(url, timeout=8)
    print(f"{result.url}: {result.detail}")
```

--------------------------------

### reStructuredText Grid Table Example

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/resource_formats.md

Illustrates the reStructuredText grid table format for organizing data, including headers and rows with different column types.

```rst
============  ======================  =====  ==========  =====================
Column 1      Column 2                Year   Reference   Materials
============  ======================  =====  ==========  =====================
Data Type     Paper Title             2024   [#Ref]_     `[PDF] <url>`_
============  ======================  =====  ==========  =====================
```

--------------------------------

### Example Usage of check_one_url

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/url_checker_reference.md

Illustrates how to call `check_one_url` and interpret the returned `CheckResult` object to display whether the URL is valid and the details of the check.

```python
result = check_one_url("https://example.com/paper.pdf", timeout=8)
if result.ok:
    print(f"✓ {result.url} [{result.method} {result.status_code}]")
else:
    print(f"✗ {result.url} [{result.method}] {result.detail}")
```

--------------------------------

### Configure requests.Session with Retry Strategy

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/data_types.md

Configure a reusable HTTP session with automatic retry logic for specified HTTP status codes and methods. This setup enhances robustness for network requests.

```python
session = requests.Session()

# Retry strategy
retry = Retry(
    total=2,                                    # Max 2 total retries
    connect=2,                                  # Max 2 connection retries
    read=2,                                     # Max 2 read retries
    backoff_factor=0.5,                        # Exponential backoff
    status_forcelist=(429, 500, 502, 503, 504), # Retry on these HTTP codes
    allowed_methods=frozenset({"HEAD", "GET"})  # Only retry these methods
)

adapter = HTTPAdapter(max_retries=retry)
session.mount("http://", adapter)
session.mount("https://", adapter)

# Browser-like User-Agent
session.headers.update({
    "User-Agent": "Mozilla/5.0 (compatible; URLChecker/2.0; +https://github.com/yzhao062/anomaly-detection-resources)"
})
```

--------------------------------

### URL Checker Common Failure: OK (fallback)

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/url_checker_reference.md

Explanation of the 'OK (fallback)' message, indicating that while the HEAD request failed, the GET request to the URL succeeded, suggesting the server may not support HEAD requests.

```text
"OK (fallback)" | HEAD failed but GET succeeded | Server doesn't support HEAD
```

--------------------------------

### Example CheckResult Instance: Network Connection Error

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/data_types.md

An example of a CheckResult instance representing a network connection error, such as a DNS resolution failure.

```python
CheckResult(
    url='https://unreachable-domain.invalid/paper',
    ok=False,
    status_code=None,
    method='HEAD/GET',
    detail='RequestException: [Errno -3] Name or service not known'
)
```

--------------------------------

### Recommended Download Tools (Unix)

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/TECHNICAL_SUMMARY.md

On Unix systems, it is recommended to use `wget` or `aria2c` for downloading files from a list of URLs. These tools offer features like parallel downloads and resume capabilities.

```bash
# wget -B basefile.txt  # reads URLs from file with -O naming
# aria2c -i url_list.txt  # parallel downloads with resume
```

--------------------------------

### Determine Fallback to GET Request

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/url_checker_reference.md

Checks if a HEAD request's status code indicates that a fallback to a GET request might be appropriate. This is useful for servers that do not properly support HEAD requests.

```python
def should_fallback_to_get(status_code: int) -> bool:
    pass
```

--------------------------------

### should_fallback_to_get(status_code: int)

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/url_checker_reference.md

Determines if a GET request should be attempted after a HEAD request returns a specific status code. This is useful for handling servers that might not support HEAD requests properly but do support GET.

```APIDOC
## should_fallback_to_get(status_code: int)

### Description
Determine whether to fall back from HEAD request to GET request based on HTTP response status code.

### Parameters
#### Path Parameters
- **status_code** (int) - Required - HTTP status code from HEAD request

### Returns
`True` if fallback to GET should be attempted, `False` otherwise

### Fallback Status Codes
403, 405, 406, 429, 500, 501, 502, 503

### Status Code Meanings
- `403` Forbidden (may allow GET)
- `405` Method Not Allowed (likely allows GET)
- `406` Not Acceptable
- `429` Too Many Requests (rate limiting)
- `500` Internal Server Error (temporary)
- `501` Not Implemented (HEAD not supported)
- `502` Bad Gateway (temporary)
- `503` Service Unavailable (temporary)

### Example
```python
if should_fallback_to_get(405):
    # Try GET after HEAD returned 405 Method Not Allowed
    response = session.get(url)
```
```

--------------------------------

### Basic Download Script Usage

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/download_reference.md

Run the download script and check the results. Ensure the input file 'resource_urls/papers.txt' exists and is correctly formatted before execution.

```bash
# Ensure resource_urls/papers.txt exists with data in format:
# Paper Title | https://example.com/paper.pdf

# Run download
python download.py

# Check results
ls -lh resources/ | head
```

--------------------------------

### Run Download Script (Windows)

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/TECHNICAL_SUMMARY.md

Execute the download.py script on Windows systems. Note that this script has known issues and is not recommended for production without fixes. Verify downloads using 'ls resources/ | wc -l'.

```bash
# Windows only
python download.py
ls resources/ | wc -l  # Verify downloads
```

--------------------------------

### Example Transformation with Hyphens and Apostrophes

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/resource_formats.md

Illustrates that apostrophes and hyphens are retained in the paper title after special character removal.

```text
"Self-Supervised Anomaly Detection: The <Complete> Solution"
→ "Self-Supervised Anomaly Detection The Complete Solution"
```

--------------------------------

### Python Path Handling (Windows vs. Unix)

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/download_reference.md

Demonstrates the incorrect hardcoded Windows path separators that fail on Unix-like systems and the correct approach using `pathlib` for cross-platform compatibility.

```python
# Line 17 - FAILS on Unix
f = open('resource_urls\papers.txt', 'r')
```

```python
# Line 30 - FAILS on Unix  
urllib.request.urlretrieve(url, "resources\" + file_name + '.pdf')
```

```python
from pathlib import Path
input_file = Path('resource_urls') / 'papers.txt'
output_file = Path('resources') / f'{file_name}.pdf'
```

--------------------------------

### Run download.py Script

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/api_summary.md

Executes the download.py script to download papers from a specified text file to a designated directory. Configuration is hardcoded within the script.

```bash
python download.py
```

--------------------------------

### reStructuredText Footnote Reference Example

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/resource_formats.md

Shows the reStructuredText syntax for in-text footnote references and their corresponding definitions at the end of the document.

```rst
[#Zhao2019PYOD]_        (in-text reference)
.. [#Zhao2019PYOD] Zhao, Y... (at end of document in References section)
```

--------------------------------

### Check Disk Space

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/download_reference.md

Before downloading, verify that there is sufficient disk space available. Consider that typical academic papers range from 500KB to 5MB each.

```bash
# Ensure sufficient space for PDFs
df -h .
# Typical academic papers: 500KB - 5MB each
```

--------------------------------

### Verify Downloaded Files are PDFs

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/download_reference.md

After downloading, use the 'file' command to verify that the downloaded files are indeed PDFs.

```bash
file resources/*.pdf
```

--------------------------------

### Verify Input File Format

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/download_reference.md

Check the first few lines of the input file to ensure it adheres to the expected 'Paper Title | URL' format.

```bash
head resource_urls/papers.txt
# Output should be:
# Paper Title 1 | https://example.com/paper1.pdf
# Paper Title 2 | https://example.com/paper2.pdf
```

--------------------------------

### Download All Papers using download.py

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/INDEX.md

Execute this command to download all available papers. Note that this script may require modifications for Unix-based systems due to path issues.

```bash
python download.py
# Creates resources/ with downloaded PDFs
```

--------------------------------

### Show help with url_checker.py

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/api_summary.md

Displays the help message for the url_checker.py script.

```bash
python url_checker.py --help
```

--------------------------------

### Example Transformation of Special Characters

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/resource_formats.md

Demonstrates how special characters are removed from a paper title string. Quotes are preserved, while colons and angle brackets are removed.

```text
Input:  "A 'Great' Paper: The <Story> | More Data"
Output: "A 'Great' Paper The Story More Data"
         (quotes are preserved, colons removed, angle brackets removed)
```

--------------------------------

### Full Workflow with url_checker.py

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/api_summary.md

This pattern demonstrates the complete workflow: parsing arguments, reading a file, extracting URLs, checking them concurrently, and reporting results. It utilizes ThreadPoolExecutor for parallel processing.

```python
from url_checker import extract_urls, check_all, parse_args

# 1. Get arguments
args = parse_args()

# 2. Read file
with open(args.file) as f:
    content = f.read()

# 3. Extract URLs
urls = extract_urls(content)

# 4. Check all
results = check_all(urls, timeout=args.timeout, workers=args.workers)

# 5. Report
for result in results:
    if not result.ok:
        print(f"Failed: {result.url} - {result.detail}")
```

--------------------------------

### Main Entry Point for URL Checker Utility

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/url_checker_reference.md

The main function serves as the entry point for the command-line utility. It handles argument parsing, URL extraction, concurrent checking, and result reporting, returning an appropriate exit code.

```python
def main() -> int:
    pass
```

--------------------------------

### URL Checker Common Failure: Fallback Failed

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/url_checker_reference.md

Explanation of the 'Fallback failed after HEAD NNN' message, meaning both HEAD and GET requests to the URL resulted in an error.

```text
"Fallback failed after HEAD NNN" | GET after HEAD failed | Both HEAD and GET returned error
```

--------------------------------

### URL Checker CLI Options

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/url_checker_reference.md

Available command-line options for configuring the URL checker, such as input file, timeout, and concurrency.

```bash
--file FILE              Path to RST/Markdown file to check (default: README.rst)
--timeout SECONDS        HTTP request timeout in seconds (default: 8)
--workers NUM            Number of concurrent worker threads (default: 16)
--show-ok                Print successful URLs in output (default: off)
--help, -h               Show help message and exit
```

--------------------------------

### Check Single URL with Fallback

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/url_checker_reference.md

Validates a single URL by first attempting an HTTP HEAD request and falling back to a GET request if necessary. It returns a detailed `CheckResult` object.

```python
def check_one_url(url: str, timeout: int) -> CheckResult:
    pass
```

--------------------------------

### Python Exception Handling for Downloads

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/download_reference.md

Illustrates how to wrap file download operations in a try-except block to gracefully handle potential errors such as network issues or invalid URLs, preventing script crashes.

```python
try:
    urllib.request.urlretrieve(url, str(output_file))
    print(f"✓ Downloaded {file_name}")
except Exception as e:
    print(f"✗ Failed to download {file_name}: {e}")
    continue
```

--------------------------------

### Check default file with url_checker.py

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/api_summary.md

Runs the url_checker.py script using the default input file.

```bash
python url_checker.py
```

--------------------------------

### reStructuredText Hyperlink Syntax

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/resource_formats.md

Demonstrates the syntax for creating hyperlinks in reStructuredText, both as explicit URLs and as inline references.

```rst
`Link Text <https://example.com>`_
```

```rst
See `PyOD library <https://github.com/yzhao062/pyod>`_
```

--------------------------------

### should_fallback_to_get

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/api_summary.md

Determines if a fallback to the HTTP GET method should be attempted after an initial HEAD request fails or returns a specific status code. This is useful for servers that might not handle HEAD requests correctly.

```APIDOC
## should_fallback_to_get

### Description
Determine if GET should be tried after HEAD.

### Signature
`should_fallback_to_get(status_code: int) -> bool`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Request Example
```python
fallback = should_fallback_to_get(405) # Example status code
print(fallback)
```

### Response
#### Success Response
- **fallback** (bool) - True if GET should be tried, False otherwise.

### Response Example
```json
{
  "fallback_to_get": true
}
```
```

--------------------------------

### Run URL Checker with Custom Parameters

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/url_checker_reference.md

Executes the URL checker script with custom file, timeout, and worker count.

```bash
python url_checker.py --file README_CN.rst --timeout 10 --workers 20
```

--------------------------------

### Custom file and timeout with url_checker.py

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/api_summary.md

Runs the url_checker.py script with a specified input file and request timeout.

```bash
python url_checker.py --file README_CN.rst --timeout 10
```

--------------------------------

### URL Checker Module Summary

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/TECHNICAL_SUMMARY.md

This Python module extracts and validates HTTP/HTTPS URLs from content using concurrent requests. It supports fallback from HEAD to GET requests for servers that reject HEAD requests.

```python
class CheckResult:
    url: str
    ok: bool
    status_code: int | None
    method: str
    detail: str
```

--------------------------------

### Check README URLs with Verbose Reporting

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/TECHNICAL_SUMMARY.md

Use this command to check URLs found in a README file with detailed reporting and multiple workers for faster processing.

```bash
python url_checker.py --file README.rst --show-ok --workers 20
```

--------------------------------

### Extract URLs from Content

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/resource_formats.md

Uses a regular expression to find all raw URLs starting with http or https within a given string. The pattern matches any non-whitespace characters following the protocol until a whitespace character is encountered.

```python
raw_urls = re.findall(r"https?://\S+", content)
```

--------------------------------

### check_one_url(url: str, timeout: int)

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/url_checker_reference.md

Checks the validity of a single URL using an HTTP HEAD request, with an automatic fallback to a GET request if necessary. It returns a detailed result object indicating the URL's status, the method used, and any relevant details.

```APIDOC
## check_one_url(url: str, timeout: int)

### Description
Check the validity of a single URL using HTTP HEAD request (with automatic GET fallback). Returns detailed result including status code and method used.

### Parameters
#### Path Parameters
- **url** (str) - Required - URL to check
- **timeout** (int) - Required - Request timeout in seconds

### Returns
`CheckResult` dataclass with fields:
- `url` (str): The URL that was checked
- `ok` (bool): Whether the URL is valid (HTTP status < 400)
- `status_code` (int | None): HTTP status code (None if connection failed)
- `method` (str): HTTP method used ("HEAD", "GET", or "HEAD/GET")
- `detail` (str): Human-readable status message

### Behavior
1. Attempts HEAD request to the URL
2. If status code is < 400 (success), returns `CheckResult(ok=True)`
3. If status code matches fallback list (403, 405, etc.), attempts GET request:
   - If GET succeeds (< 400), returns `CheckResult(ok=True, method="GET", detail="OK (fallback)")`
   - If GET fails, returns `CheckResult(ok=False)` with GET failure details
4. If HEAD fails with non-fallback error, returns `CheckResult(ok=False)`
5. On any `RequestException`, returns `CheckResult(ok=False, detail="RequestException: ...")`

### Example
```python
result = check_one_url("https://example.com/paper.pdf", timeout=8)
if result.ok:
    print(f"✓ {result.url} [{result.method} {result.status_code}]")
else:
    print(f"✗ {result.url} [{result.method}] {result.detail}")
```
```

--------------------------------

### Understand input file format for URLs

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/README.md

The input file format for URLs consists of a title and a URL, separated by a pipe symbol ('|'). Each line represents a distinct entry. Ensure URLs are correctly formatted and accessible.

```text
PyOD: A Python Toolbox for Scalable Outlier Detection | https://www.jmlr.org/papers/volume20/19-011/19-011.pdf
Isolation Forest | https://cs.nju.edu.cn/zhouzh/zhouzh.files/publication/icdm08b.pdf
```

--------------------------------

### Run URL Checker CLI

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/TECHNICAL_SUMMARY.md

Execute the url_checker.py script from the command line to check URLs in a specified file. Configure timeout, worker count, and whether to display successful checks.

```bash
python url_checker.py --file README.rst --timeout 8 --workers 16 --show-ok
```

--------------------------------

### build_session

Source: https://github.com/yzhao062/anomaly-detection-resources/blob/master/_autodocs/api_summary.md

Creates and configures a `requests.Session` object. This session is pre-configured with retry mechanisms suitable for making HTTP requests.

```APIDOC
## build_session

### Description
Create a configured requests session with retries.

### Signature
`build_session() -> requests.Session`

### Parameters
None

### Request Example
```python
session = build_session()
response = session.get("https://example.com")
print(response.status_code)
```

### Response
#### Success Response
- **session** (requests.Session) - A configured HTTP session object.

### Response Example
```json
{
  "session_type": "requests.Session"
}
```
```
Tokens:
Raw