JobSpy

https://github.com/speedyapply/jobspy
Admin
JobSpy is a job scraping library that aggregates job postings from multiple popular job boards...
Tokens:81,735
Snippets:432
Trust Score:-
License:MIT
Update:2 weeks ago
Show doc for...
Tokens:
Raw
### create_session usage example

Source: https://github.com/speedyapply/jobspy/blob/main/_autodocs/api-reference/util.md

Example usage of create_session with a list of proxies, a CA certificate, TLS enabled, and a custom user agent. Demonstrates how to create a session and make a GET request.

```python
from jobspy.util import create_session

session = create_session(
    proxies=["user:pass@host:8080", "localhost"],
    ca_cert="/etc/ssl/certs/proxy.pem",
    is_tls=True,
    user_agent="Mozilla/5.0",
)
r = session.get("https://example.com")
```

--------------------------------

### Using Indeed scraper with ScraperInput

Source: https://github.com/speedyapply/jobspy/blob/main/_autodocs/api-reference/model.md

Example of using a concrete scraper (Indeed) with proxies and user_agent. Creates a ScraperInput with search criteria and calls scrape, then prints the number of jobs returned.

```python
from jobspy import Indeed, ScraperInput

scraper = Indeed(proxies=["localhost"], user_agent="my-agent/1.0")
response = scraper.scrape(ScraperInput(search_term="engineer", country="USA"))
print(len(response.jobs))  # each JobPost has .title, .company_name, ...
```

--------------------------------

### Indeed constructor with session setup

Source: https://github.com/speedyapply/jobspy/blob/main/_autodocs/api-reference/indeed.md

Initializes the Indeed scraper with optional proxies, CA certificate, and user agent. Creates a RequestsRotating session with is_tls=False and sets base_url to None, which is later set in scrape() from the country.

```python
def __init__(
    self,
    proxies: list[str] | str | None = None,
    ca_cert: str | None = None,
    user_agent: str | None = None,
):
    super().__init__(proxies=proxies, ca_cert=ca_cert, user_agent=user_agent)
    self.session = create_session(
        proxies=self.proxies, ca_cert=ca_cert, is_tls=False
    )
    self.base_url = None
```

--------------------------------

### ScraperInput usage example

Source: https://github.com/speedyapply/jobspy/blob/main/_autodocs/api-reference/model.md

Shows how to instantiate ScraperInput with common search criteria. The commented line notes the deprecated linkedin_fetch_description flag that sets fetch_description=True.

```python
from jobspy import ScraperInput
from jobspy.model import JobType, DescriptionFormat, Country

inp = ScraperInput(
    search_term="backend engineer",
    location="Berlin",
    country=Country.GERMANY,
    distance=30,
    is_remote=True,
    job_type=JobType.FULL_TIME,
    results_wanted=10,
    hours_old=48,
    description_format=DescriptionFormat.PLAIN,
    offset=10,
)
# linkedin_fetch_description=True  # deprecated; sets fetch_description=True
```

--------------------------------

### Install python-jobspy

Source: https://github.com/speedyapply/jobspy/blob/main/README.md

Install the Python package python-jobspy. Requires Python version 3.10 or higher.

```bash
pip install -U python-jobspy
```

--------------------------------

### Custom board scraper subclass example

Source: https://github.com/speedyapply/jobspy/blob/main/_autodocs/api-reference/model.md

Shows how to build a custom board scraper by subclassing Scraper. Call super().__init__ with proxies, ca_cert, and user_agent, create a session with create_session() from jobspy.util, and implement scrape() to return a JobResponse.

```python
from jobspy.model import Scraper, ScraperInput, JobResponse, JobPost
from jobspy.util import create_session

class MyBoard(Scraper):
    def scrape(self, scraper_input: ScraperInput) -> JobResponse:
        self.scraper_input = scraper_input
        self.session = create_session(proxies=self.proxies, ca_cert=self.ca_cert)
        jobs = []  # ... fetch and parse ...
        return JobResponse(jobs=jobs)
```

--------------------------------

### Usage example for jobspy.util helpers

Source: https://github.com/speedyapply/jobspy/blob/main/_autodocs/api-reference/util.md

Shows how to use the utility functions `extract_salary` and `format_description` from `jobspy.util`. The first call parses a salary string into interval, low, high, and currency values; the second converts HTML description to Markdown and extracts email addresses.

```python
from jobspy.util import extract_salary, format_description, get_enum_from_job_type
from jobspy.model import DescriptionFormat

interval, low, high, currency = extract_salary(
    "Salary: $65,000 - $85,000/year", enforce_annual_salary=False
)
# ('yearly', 65000.0, 85000.0, 'USD')

desc, emails = format_description(
    "<p>Join us! Email <b>hi@example.com</b></p>",
    DescriptionFormat.MARKDOWN,
)
print(desc)    # 'Join us! Email **hi@example.com**'
print(emails)  # ['hi@example.com']
```

--------------------------------

### GET https://gateway.bdjobs.com/jobapply/api/JobSubsystem/Job-Details

Source: https://github.com/speedyapply/jobspy/blob/main/_autodocs/endpoints.md

Fetch detailed information for a specific job posting by job ID.

```APIDOC

## GET https://gateway.bdjobs.com/jobapply/api/JobSubsystem/Job-Details

### Description
Retrieve detailed information for a specific job by its ID. The response data may be empty, in which case a message is logged.

### Method
GET

### Endpoint
https://gateway.bdjobs.com/jobapply/api/JobSubsystem/Job-Details?jobId={id}&ln=1&IsCorporate=false

### Parameters
#### Query Parameters
- **jobId** (string or integer) - Required - The job ID to fetch details for.
- **ln** (integer) - Required - Language code (e.g., 1 for English).
- **IsCorporate** (boolean) - Required - Whether the job is corporate (false in example).

### Response
#### Success Response (200)
- **data** (array) - Array of job detail objects, possibly empty. Each object may contain: **SkillsRequired**, **ApplyURL**, **CompanyID**, **HaveAliasName**, **CompanyWeb**, **CompanyAddress**, **CompanyBusiness**.
- **message** (string) - A message (e.g., error or success).

#### Response Example
{
  "data": [
    {
      "SkillsRequired": "",
      "ApplyURL": "",
      "CompanyID": 0,
      "HaveAliasName": false,
      "CompanyWeb": "",
      "CompanyAddress": "",
      "CompanyBusiness": ""
    }
  ],
  "message": ""
}
```

--------------------------------

### Single board scrape with Site enum and offset

Source: https://github.com/speedyapply/jobspy/blob/main/_autodocs/api-reference/scrape-jobs.md

Scrapes a single board using the `Site.INDEED` enum and an `offset` to start at the 26th result. Checks if the returned DataFrame is empty, which may indicate blocking or no matches, and notes that rate limits are frequent on Glassdoor/LinkedIn.

```python
from jobspy import Site, scrape_jobs

df = scrape_jobs(
    site_name=Site.INDEED,
    search_term="data scientist",
    location="New York, NY",
    results_wanted=30,
    offset=25,          # start at the 26th result
    hours_old=168,
    country_indeed="USA",
)
if df.empty:
    print("No results (blocked or no matches); rate limits are frequent on Glassdoor/LinkedIn")
else:
    print(df["job_url"].tolist())
```

--------------------------------

### Scrape Indeed Jobs with Indeed and ScraperInput

Source: https://github.com/speedyapply/jobspy/blob/main/_autodocs/api-reference/indeed.md

Basic example of using the Indeed scraper with a ScraperInput to search for jobs. The search term includes quotes and a minus sign for exclusion, and the country is passed as a string accepted by the pydantic model. The response jobs are iterated to print job details including compensation and remote status.

```python
from jobspy import Indeed, ScraperInput

scraper = Indeed()
response = scraper.scrape(
    ScraperInput(
        search_term='"engineering intern" software 2025 -tax',
        location="Arlington, VA",
        country="USA",          # str accepted by the pydantic model
        distance=25,
        results_wanted=100,
        hours_old=168,
        is_remote=True,
        job_type="fulltime",
    )
)
for job in response.jobs:
    print(
        job.id, job.title, job.company_name,
        job.compensation,          # Compensation(interval=YEARLY, min=…, max=…, currency='USD')
        job.is_remote, job.date_posted,
    )
```

--------------------------------

### Google.__init__

Source: https://github.com/speedyapply/jobspy/blob/main/_autodocs/api-reference/google.md

Initializes the Google scraper with optional proxies, CA certificate, and user agent. The user agent is not forwarded to the session.

```APIDOC
## Google.__init__

### Description
Initializes the Google scraper with the Google Jobs search URL. Sets up instance attributes for session, input, and pagination.

### Method
Constructor

### Signature
```python
def __init__(
    self, proxies: list[str] | str | None = None, ca_cert: str | None = None, user_agent: str | None = None
):
```

### Parameters
- **proxies** (list[str] | str | None) - Optional - Proxy list or string.
- **ca_cert** (str | None) - Optional - CA certificate path.
- **user_agent** (str | None) - Optional - Not forwarded to the session; no code path applies it.

### Notes
- Sets `self.url` to `https://www.google.com/search` and `self.jobs_url` to `https://www.google.com/async/callback:550`.
- Initializes `jobs_per_page = 10`, `seen_urls` as an empty set, and other attributes to `None`.
```

--------------------------------

### GET https://api.bdjobs.com/Jobs/api/JobSearch/GetJobSearch

Source: https://github.com/speedyapply/jobspy/blob/main/_autodocs/endpoints.md

Search for jobs on BDJobs using the REST API.

```APIDOC

## GET https://api.bdjobs.com/Jobs/api/JobSearch/GetJobSearch

### Description
Search for jobs on BDJobs by keyword, with optional filters for location, posted within, job nature, and workplace. Results are paginated.

### Method
GET

### Endpoint
https://api.bdjobs.com/Jobs/api/JobSearch/GetJobSearch?rpp=50&isPro=0&ToggleJobs=true&isFresher=false&keyword={term}[&location=<id>][&postedWithin=<d>][&jobNature=<code>][&workplace=1]&pg={page}

### Parameters
#### Query Parameters
- **rpp** (integer) - Required - Results per page (fixed to 50 in example).
- **isPro** (integer) - Required - Flag for pro jobs (fixed to 0 in example).
- **ToggleJobs** (boolean) - Required - Toggle to show jobs (fixed to true in example).
- **isFresher** (boolean) - Required - Filter for fresher jobs (fixed to false in example).
- **keyword** (string) - Required - Search term.
- **location** (integer) - Optional - Location ID from the locations table.
- **postedWithin** (integer) - Optional - Number of days for posted-within filter.
- **jobNature** (string) - Optional - Job nature code.
- **workplace** (integer) - Optional - Workplace filter (e.g., 1).
- **pg** (integer) - Required - Page number.

### Response
#### Success Response (200)
- **data** (array) - List of job objects.
- **premiumData** (array) - Premium job listings, prepended on page 1.
- **common** (object) - Contains **totalpages** (integer) indicating total pages.

#### Response Example
{
  "data": [],
  "premiumData": [],
  "common": {
    "totalpages": 10
  }
}
```

--------------------------------

### Indeed.__init__

Source: https://github.com/speedyapply/jobspy/blob/main/_autodocs/api-reference/indeed.md

Initializes the Indeed scraper with optional proxies, CA certificate, and user agent. Sets up a rotating session with TLS disabled and initializes base_url to None.

```APIDOC
## Indeed.__init__

### Description
Initializes the Indeed scraper with optional proxies, CA certificate, and user agent. Sets up a rotating session with TLS disabled and initializes base_url to None.

### Method
Constructor

### Signature
```python
def __init__(
    self,
    proxies: list[str] | str | None = None,
    ca_cert: str | None = None,
    user_agent: str | None = None,
):
```

### Parameters
- **proxies** (`list[str] | str | None`) - Optional - Proxy list/string.
- **ca_cert** (`str | None`) - Optional - CA cert path.
- **user_agent** (`str | None`) - Optional - Applied to the request headers per call (`request_headers["user-agent"]`), overriding the constant.

### Notes
- `self.base_url` is set in `scrape()` from the country.

```

--------------------------------

### BDJobs.scrape(scraper_input: ScraperInput) -> JobResponse

Source: https://github.com/speedyapply/jobspy/blob/main/_autodocs/api-reference/bdjobs.md

Searches BDJobs for jobs based on the provided ScraperInput. It sends a GET request to the search API, handles pagination, and returns a JobResponse containing the scraped job posts.

```APIDOC
## BDJobs.scrape(scraper_input: ScraperInput) -> JobResponse

### Description
Searches BDJobs for jobs based on the provided ScraperInput. It sends a GET request to the search API, handles pagination, and returns a JobResponse containing the scraped job posts.

### Method
GET

### Endpoint
https://api.bdjobs.com/Jobs/api/JobSearch/GetJobSearch

### Parameters
#### Query Parameters
- **keyword** (string) - Optional - Search term for job search.
- **location** (integer) - Optional - Location ID for a division or district (e.g., Dhaka).
- **postedWithin** (integer) - Optional - Number of days within which the job was posted.
- **jobNature** (string) - Optional - Job type code (e.g., FullTime, PartTime, Contract, Intern).
- **workplace** (integer) - Optional - Set to 1 for remote jobs.
- **rpp** (integer) - Optional - Results per page (default 50).
- **isPro** (integer) - Optional - Professional jobs flag (default 0).
- **ToggleJobs** (string) - Optional - Toggle jobs flag (default "true").
- **isFresher** (string) - Optional - Fresher jobs flag (default "false").
- **pg** (integer) - Optional - Page number for pagination.

### Response
#### Success Response (200)
- **data** (array) - List of job postings.
- **premiumData** (array) - Premium job postings (prepended to page 1 results).
- **common.totalpages** (integer) - Total number of pages available.

#### Response Example
{
  "data": [
    {
      "Jobid": "12345",
      "JobTitle": "Software Engineer",
      "publishDate": "2023-10-01T00:00:00Z",
      "WorkPlace": "Home",
      "JobType": "FullTime",
      "location": "Dhaka, Bangladesh",
      "SkillsRequired": "Python, Django",
      "ApplyURL": "https://example.com/apply",
      "CompanyID": "67890",
      "CompanyWeb": "https://company.com",
      "CompanyName": "Example Corp"
    }
  ],
  "common": {
    "totalpages": 5
  }
}

### Error Handling
- Non-200 responses are logged and ignored.
- Exceptions are logged and ignored.
- Pages past the total page count are ignored.
- Per-job failures are logged as "skipping job: ...".
```

--------------------------------

### Google._get_initial_cursor_and_jobs

Source: https://github.com/speedyapply/jobspy/blob/main/_autodocs/api-reference/google.md

Builds the search query, fetches the initial Google search page, and extracts the forward cursor and initial job listings.

```APIDOC
## Google._get_initial_cursor_and_jobs

### Description
Builds the search query based on the scraper input, performs a GET request to the initial search URL, extracts the forward cursor from the page, and parses embedded job listings.

### Method
_get_initial_cursor_and_jobs

### Signature
```python
def _get_initial_cursor_and_jobs() -> Tuple[str, list[JobPost]]:
```

### Returns
- **Tuple[str, list[JobPost]]** - A tuple containing the forward cursor string and a list of `JobPost` objects.

### Behavior
- Constructs query as `f"{search_term} jobs"` plus job-type label, `near {location}`, time filter, and `remote` unless `google_search_term` overrides.
- Time filters: `<=24h` → `"since yesterday"`, `<=72h` → `"in the last 3 days"`, `<=168h` → `"in the last week"`, else `"in the last month"`.
- Job-type words: `Full time`, `Part time`, `Internship`, `Contract`.
- GETs `self.url` with parameters `{"q": query, "udm": "8"}`.
- Extracts `data-async-fc` from `<div jsname="Yust4d">` element.
- Parses jobs via `find_job_info_initial_page()`.
```

--------------------------------

### LinkedIn Job Detail Endpoint URL

Source: https://github.com/speedyapply/jobspy/blob/main/_autodocs/endpoints.md

The URL for fetching a specific job's details by its ID. Used only when fetch_description is enabled. The response must be 200 and the URL path must start with /jobs/view/.

```http
GET https://www.linkedin.com/jobs/view/{job_id}
```

--------------------------------

### LinkedIn Search Endpoint URL

Source: https://github.com/speedyapply/jobspy/blob/main/_autodocs/endpoints.md

The URL for LinkedIn's job search API. It uses query parameters like keywords, location, distance, and start for pagination. The response is an HTML page with job cards.

```http
GET https://www.linkedin.com/jobs-guest/jobs/api/seeMoreJobPostings/search
```

--------------------------------

### Fetch full job descriptions from ZipRecruiter

Source: https://github.com/speedyapply/jobspy/blob/main/_autodocs/api-reference/ziprecruiter.md

Fetch full job descriptions by setting fetch_description=True. Each job page is fetched with allow_redirects=True, which populates fields like company_url_direct and company_num_employees.

```python
from jobspy import ZipRecruiter, ScraperInput

resp = ZipRecruiter().scrape(
    ScraperInput(search_term="analyst", results_wanted=10, fetch_description=True)
)
# each job page is fetched with allow_redirects=True; fields like
# company_url_direct, company_num_employees become populated
print(resp.jobs[0].description if resp.jobs else "no jobs")
```

--------------------------------

### Naukri.scrape()

Source: https://github.com/speedyapply/jobspy/blob/main/_autodocs/api-reference/naukri.md

The Naukri scraper is invoked via the `Naukri().scrape()` method with a `ScraperInput` object. It returns a response containing job listings. The example shows how to fetch jobs with various filters and how to handle potential 406 errors.

```APIDOC
## [POST] /scrape

### Description
Scrapes Naukri job listings based on the provided search criteria. Returns a response object containing a list of jobs with details such as title, company, compensation, and more.

### Method
POST

### Endpoint
/scrape

### Parameters
#### Request Body
- **search_term** (string) - Required - The job search term, e.g., "python developer".
- **location** (string) - Optional - The location to search in, e.g., "Pune".
- **results_wanted** (integer) - Optional - The number of job results to return.
- **hours_old** (integer) - Optional - Only return jobs posted within this many hours.
- **job_type** (string) - Optional - The type of job, e.g., "fulltime" or "internship".
- **fetch_description** (boolean) - Optional - Whether to fetch full job descriptions.
- **is_remote** (boolean) - Optional - Filter for remote jobs.
- **user_agent** (string) - Optional - Custom user agent string for the request.

### Request Example
```python
from jobspy import Naukri, ScraperInput

resp = Naukri().scrape(
    ScraperInput(
        search_term="python developer",
        location="Pune",
        results_wanted=20,
        hours_old=168,
        job_type="fulltime",
    )
)
```

### Response
#### Success Response (200)
- **jobs** (array) - List of job objects.
  - **id** (string) - Job identifier.
  - **title** (string) - Job title.
  - **company_name** (string) - Name of the company.
  - **work_from_home_type** (string) - Work-from-home type.
  - **skills** (array) - List of required skills.
  - **experience_range** (string) - Experience range.
  - **compensation** (string) - Compensation details.
  - **company_industry** (string) - Industry of the company.

#### Response Example
```python
for job in resp.jobs:
    print(job.id, job.title, job.company_name, job.work_from_home_type,
          job.skills, job.experience_range, job.compensation)
```

### Error Handling
- **406** - Rate limiting or outdated `nkparam_public_key`. Check JobSpy:Naukri logs for details.
```

--------------------------------

### LinkedIn.__init__

Source: https://github.com/speedyapply/jobspy/blob/main/_autodocs/api-reference/linkedin.md

Initializes the LinkedIn scraper with optional proxies, CA certificate, and user agent. Sets up a rotating session with browser headers.

```APIDOC
## LinkedIn.__init__

### Description
Initializes the LinkedIn scraper instance. Sets up a rotating session with browser headers and optional proxy, CA certificate, and user agent overrides.

### Method
Constructor

### Signature
```python
def __init__(
    self, proxies: list[str] | str | None = None, ca_cert: str | None = None, user_agent: str | None = None
):
```

### Parameters
- **proxies** (list[str] | str | None) - Optional - Proxy list/string for rotation. Recommended due to LinkedIn rate limiting around the 10th page per IP.
- **ca_cert** (str | None) - Optional - CA cert path for proxy verification.
- **user_agent** (str | None) - Optional - Replaces the default Chrome UA from constant.py when truthy.

### Example
```python
from jobspy import LinkedIn
scraper = LinkedIn(proxies=["http://proxy1:8080"], user_agent="Mozilla/5.0")
```
```

--------------------------------

### scrape_jobs - Single Board with Site Enum and Offset

Source: https://github.com/speedyapply/jobspy/blob/main/_autodocs/api-reference/scrape-jobs.md

Scrapes a single job board using the explicit Site enum and an offset to start from a specific result, with handling for empty results due to blocking or rate limits.

```APIDOC
## scrape_jobs - Single Board with Site Enum and Offset

### Description
Scrapes a single job board using the explicit Site enum and an offset to start from a specific result, with handling for empty results due to blocking or rate limits.

### Method
Function Call

### Endpoint
`scrape_jobs(site_name, search_term, location, results_wanted, offset, hours_old, country_indeed)`

### Parameters
#### Function Parameters
- **site_name** (Site enum) - Required - The job board to scrape, e.g., `Site.INDEED`.
- **search_term** (string) - Required - The job search query, e.g., "data scientist".
- **location** (string) - Required - The location to search in, e.g., "New York, NY".
- **results_wanted** (integer) - Required - Number of job results to retrieve.
- **offset** (integer) - Optional - Start at the Nth result (e.g., 25 starts at the 26th result).
- **hours_old** (integer) - Required - Maximum age of job postings in hours.
- **country_indeed** (string) - Required - Country code for Indeed searches, e.g., "USA".

### Request Example
```python
from jobspy import Site, scrape_jobs

df = scrape_jobs(
    site_name=Site.INDEED,
    search_term="data scientist",
    location="New York, NY",
    results_wanted=30,
    offset=25,          # start at the 26th result
    hours_old=168,
    country_indeed="USA",
)
if df.empty:
    print("No results (blocked or no matches); rate limits are frequent on Glassdoor/LinkedIn")
else:
    print(df["job_url"].tolist())
```

### Response
#### Success Response
- Returns a pandas DataFrame containing job listings.

#### Response Example
```python
# DataFrame with columns such as job_url, etc.
```
```

--------------------------------

### Google scraper constructor

Source: https://github.com/speedyapply/jobspy/blob/main/_autodocs/api-reference/google.md

Initializes the Google scraper, setting the search and async callback URLs. The user_agent parameter is not forwarded to the session; only proxies and ca_cert are passed to the parent constructor.

```python
def __init__(
    self, proxies: list[str] | str | None = None, ca_cert: str | None = None, user_agent: str | None = None
):
    """
    Initializes Google Scraper with the Goodle jobs search url
    """
    super().__init__(proxies=proxies, ca_cert=ca_cert)

    self.country = None
    self.session = None
    self.scraper_input = None
    self.jobs_per_page = 10
    self.seen_urls = set()
    self.url = "https://www.google.com/search"
    self.jobs_url = "https://www.google.com/async/callback:550"
```

--------------------------------

### Fetch Glassdoor job descriptions in batches

Source: https://github.com/speedyapply/jobspy/blob/main/_autodocs/api-reference/glassdoor.md

Uses Glassdoor with a proxy to fetch job descriptions for 'product manager' in the USA. Note that Glassdoor is rate-limit heavy, so jobs may be incomplete; watch JobSpy:Glassdoor logs.

```python
from jobspy import Glassdoor, ScraperInput

resp = Glassdoor(proxies=["localhost"]).scrape(
    ScraperInput(search_term="product manager", country="USA",
                 results_wanted=50, fetch_description=True)
)
print(resp.jobs[0].description if resp.jobs else "no results")
# glassdoor is rate-limit heavy: jobs may be incomplete; watch JobSpy:Glassdoor logs
```

--------------------------------

### Catching deprecated IndeedException

Source: https://github.com/speedyapply/jobspy/blob/main/_autodocs/errors.md

Example of catching a deprecated exception (IndeedException) in a try/except block. With the current version, these exceptions are never raised, so the except branch is only reachable if the exception is instantiated manually. Prefer checking the returned DataFrame or logs instead.

```python
from jobspy.exception import IndeedException

try:
    # ... a scrape that used to raise IndeedException in old versions
    pass
except IndeedException as exc:
    print(exc)   # only reachable if instantiating manually
```

--------------------------------

### Direct Google class usage

Source: https://github.com/speedyapply/jobspy/blob/main/_autodocs/api-reference/google.md

Instantiate the `Google` class with proxies and call `scrape` with a `ScraperInput`. Board failures are logged, not raised, so inspect `resp.jobs` for partial results.

```python
from jobspy import Google, ScraperInput

g = Google(proxies=["localhost"])
resp = g.scrape(ScraperInput(search_term="engineer", location="Berlin", results_wanted=20))
for job in resp.jobs:
    print(job.title, job.company_name, job.location.display_location(), job.job_url)
# board failures are logged, not raised; inspect resp.jobs for partial results
```

--------------------------------

### Fetch Bayt job descriptions with location widening

Source: https://github.com/speedyapply/jobspy/blob/main/_autodocs/api-reference/bayt.md

Scrapes Bayt for engineer jobs in Saudi Arabia, requesting 10 results and fetching descriptions. Prints the first 100 characters of each description, job type, and vacancy count. If the location is unknown, Bayt logs a warning and widens the search automatically instead of failing.

```python
from jobspy import Bayt, ScraperInput

resp = Bayt().scrape(
    ScraperInput(search_term="engineer", location="Saudi Arabia",
                 results_wanted=10, fetch_description=True)
)
for job in resp.jobs:
    print(job.description[:100] if job.description else "(no description)")
    print(job.job_type, job.vacancy_count)
# An unknown location logs "Bayt: location '...' not found, searching ..."
# and widens automatically instead of failing.
```

--------------------------------

### Scrape Naukri jobs with basic ScraperInput

Source: https://github.com/speedyapply/jobspy/blob/main/_autodocs/api-reference/naukri.md

Basic Naukri scraping example: searches for 'python developer' in Pune with 20 results, 168 hours old, full-time jobs. Prints job fields including id, title, company, work-from-home type, skills, experience range, and compensation.

```python
from jobspy import Naukri, ScraperInput

resp = Naukri().scrape(
    ScraperInput(
        search_term="python developer",
        location="Pune",
        results_wanted=20,
        hours_old=168,
        job_type="fulltime",
    )
)
for job in resp.jobs:
    print(job.id, job.title, job.company_name, job.work_from_home_type,
          job.skills, job.experience_range, job.compensation)
```

--------------------------------

### Scraper Constructor

Source: https://github.com/speedyapply/jobspy/blob/main/_autodocs/configuration.md

Documentation for the Scraper constructor options, which are common to all scraper subclasses.

```APIDOC
## Scraper Constructor

### Description
Every scraper subclass (`LinkedIn`, `Indeed`, `Glassdoor`, `ZipRecruiter`, `Google`, `Bayt`, `Naukri`, `BDJobs`) accepts the same three options.

### Parameters

| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
| `proxies` | `list[str] \| str \| None` | no | `None` | Proxy list. One string is promoted to a list. Used by `create_session()` to build a rotating session; `"localhost"` disables proxying for that request. |
| `ca_cert` | `str \| None` | no | `None` | CA certificate file applied as `session.verify`. |
| `user_agent` | `str \| None` | no | `None` | UA override. **Google ignores it** (see [google.md](api-reference/google.md)); LinkedIn/Indeed apply it explicitly to headers; the rest via `create_session`. |
```

--------------------------------

### scrape_jobs() options

Source: https://github.com/speedyapply/jobspy/blob/main/_autodocs/configuration.md

Complete table of options for the main scrape_jobs() function, including types, defaults, and descriptions.

```APIDOC
## scrape_jobs()

### Description
Main entry point for JobSpy. Accepts all configuration options as keyword arguments and returns job listings from the selected job boards.

### Parameters

#### Keyword Arguments
- **site_name** (`str | list[str] | Site | list[Site] | None`) - Optional - Boards to query. Default: `None` (all sites except `google`).
- **search_term** (`str | None`) - Optional - Search keywords. Default: `None`.
- **google_search_term** (`str | None`) - Optional - Google-only query override. Default: `None`.
- **location** (`str | None`) - Optional - Location text (per-board interpretation). Default: `None`.
- **distance** (`int | None`) - Optional - Radius in miles. Default: `50`.
- **is_remote** (`bool | None`) - Optional - Remote-only. `None` → `False`. Default: `False`.
- **job_type** (`str | None`) - Optional - `fulltime`, `parttime`, `internship`, `contract`, (also `temporary`, `perdiem`, `nights`, `other`, `summer`, `volunteer`); raises `Exception` if unknown. Default: `None`.
- **easy_apply** (`bool | None`) - Optional - Board-hosted apply only. Default: `None`.
- **results_wanted** (`int | None`) - Optional - Max results **per site**. `None` → `15`. Default: `15`.
- **country_indeed** (`str | None`) - Optional - Country name parsed by `Country.from_string`. `None` → `"usa"`. Default: `"usa"`.
- **proxies** (`list[str] | str | None`) - Optional - Proxy strings, round-robined per request. Default: `None`.
- **ca_cert** (`str | None`) - Optional - CA cert path for proxies. Default: `None`.
- **description_format** (`str | None`) - Optional - `markdown` | `html` | `plain`. Default: `"markdown"`.
- **linkedin_fetch_description** (`bool | None`) - Optional - Deprecated; forces `fetch_description=True`. Default: `False`.
- **linkedin_company_ids** (`list[int] | None`) - Optional - LinkedIn company filter. Default: `None`.
- **offset** (`int | None`) - Optional - Skip first N results per site. `None` → `0`. Default: `0`.
- **hours_old** (`int | None`) - Optional - Posting-age window in hours. Default: `None`.
- **enforce_annual_salary** (`bool | None`) - Optional - Convert non-yearly pay to annual. Default: `False`.
- **verbose** (`int | None`) - Optional - `0` errors, `1` +warnings, `2` all logs. Default: `1`.
- **user_agent** (`str | None`) - Optional - UA override. Default: `None`.
- **fetch_description** (`bool | None`) - Optional - Fetch per-job detail pages. Default: `False`.
- **\*\*kwargs** (any) - Optional - Ignored with `FutureWarning`.

### Normalizations
- `is_remote=None → False`
- `results_wanted=None → 15`
- `offset=None → 0`
- `country_indeed=None → "usa"`
- `distance` stays as passed (Indeed substitutes 50 internally).
```

--------------------------------

### Handle Indeed Scrape Errors with Custom User Agent

Source: https://github.com/speedyapply/jobspy/blob/main/_autodocs/api-reference/indeed.md

Error handling pattern for Indeed scraping. Indeed never raises for board failures, so check the results and log output. The example sets a custom user_agent and checks if resp.jobs is empty, then raises a RuntimeError. Log messages indicate blocked status (429) or GraphQL error payload.

```python
from jobspy import Indeed, ScraperInput

scraper = Indeed(user_agent="MyApp/1.0")
resp = scraper.scrape(ScraperInput(search_term="engineer", results_wanted=10))
if not resp.jobs:
    # check JobSpy:Indeed logs: "Indeed response status code 429" => blocked;
    # "Indeed: [...]" => GraphQL error payload
    raise RuntimeError("Indeed returned no jobs")
```

--------------------------------

### Scrape Glassdoor jobs with search parameters

Source: https://github.com/speedyapply/jobspy/blob/main/_autodocs/api-reference/glassdoor.md

Creates a Glassdoor scraper and performs a search for 'data analyst' in San Francisco with filters like distance, hours_old, job_type, is_remote, and results_wanted. Prints job details for each result.

```python
from jobspy import Glassdoor, ScraperInput, Country

scraper = Glassdoor()
resp = scraper.scrape(
    ScraperInput(
        search_term="data analyst",
        location="San Francisco, CA",
        country=Country.USA,
        distance=50,
        hours_old=24,          # rounded up: fromAge=1 (days)
        job_type="contract",
        is_remote=True,
        results_wanted=30,
    )
)
for job in resp.jobs:
    print(job.id, job.title, job.company_name, job.listing_type, job.date_posted)
```

--------------------------------

### Scrape Naukri internships with fetch_description and is_remote

Source: https://github.com/speedyapply/jobspy/blob/main/_autodocs/api-reference/naukri.md

Naukri scraping with custom user agent, fetching descriptions and remote internships. Note that 406 responses indicate rate limiting or an outdated nkparam_public_key; check JobSpy:Naukri logs for details.

```python
from jobspy import Naukri, ScraperInput

resp = Naukri(user_agent="Mozilla/5.0").scrape(
    ScraperInput(search_term="intern", job_type="internship",
                 results_wanted=10, fetch_description=True, is_remote=True)
)
for job in resp.jobs:
    print(job.title, job.compensation, job.company_industry)
# 406 responses mean rate limiting or an outdated nkparam_public_key;
# watch JobSpy:Naukri logs for the explanation.
```

--------------------------------

### Utility functions and sessions

Source: https://github.com/speedyapply/jobspy/blob/main/_autodocs/api-reference/README.md

Utility functions and sessions for common operations like session creation, logging, description formatting, salary extraction, and more.

```APIDOC
## create_session, create_logger, set_logger_level, format_description, extract_salary, extract_emails_from_text, map_str_to_site, get_enum_from_job_type, convert_to_annual, utc_date, markdown_converter, plain_converter, desired_order

### Description
Utility functions and sessions for common operations like session creation, logging, description formatting, salary extraction, and more.

### Method
Function calls

### Endpoint
`create_session`, `create_logger`, `set_logger_level`, `format_description`, `extract_salary`, `extract_emails_from_text`, `map_str_to_site`, `get_enum_from_job_type`, `convert_to_annual`, `utc_date`, `markdown_converter`, `plain_converter`, `desired_order`

### Parameters
- `create_session()` - Creates a requests session.
- `create_logger()` - Creates a logger.
- `set_logger_level()` - Sets the logger level.
- `format_description()` - Formats a description.
- `extract_salary()` - Extracts salary information.
- `extract_emails_from_text()` - Extracts emails from text.
- `map_str_to_site()` - Maps a string to a Site enum.
- `get_enum_from_job_type()` - Gets an enum from a job type.
- `convert_to_annual()` - Converts salary to annual.
- `utc_date()` - Returns the current UTC date.
- `markdown_converter()` - Converts to markdown.
- `plain_converter()` - Converts to plain text.
- `desired_order()` - Returns the desired order.

### Request Example
```python
from jobspy.util import create_session, create_logger, set_logger_level, format_description, extract_salary, extract_emails_from_text, map_str_to_site, get_enum_from_job_type, convert_to_annual, utc_date, markdown_converter, plain_converter, desired_order

session = create_session()
logger = create_logger()
set_logger_level(logger, "INFO")
```

### Response
#### Success Response
Returns the respective utility outputs.

#### Response Example
```python
print(session)
```
```

--------------------------------

### create_session

Source: https://github.com/speedyapply/jobspy/blob/main/_autodocs/api-reference/util.md

Factory function for creating proxy-rotating sessions with configurable TLS impersonation and cookie handling.

```APIDOC
## create_session

### Description
Factory function that creates a proxy-rotating HTTP session, either using curl_cffi with Chrome TLS impersonation or plain requests with optional cookie clearing.

### Signature
```python
def create_session(
    *,
    proxies: list[str] | str | None = None,
    ca_cert: str | None = None,
    is_tls: bool = True,
    clear_cookies: bool = False,
    user_agent: str | None = None,
) -> RequestsRotating | TLSRotating:
```

### Parameters
- **proxies** (`list[str] | str | None`) - Optional - `'user:pass@host:port'` or `'localhost'` strings. A single string is treated as a one-element list.
- **ca_cert** (`str | None`) - Optional - CA cert path; assigned to `session.verify` (used for proxy TLS verification).
- **is_tls** (`bool`) - Optional - `True` → `TLSRotating` (curl_cffi with Chrome impersonation); `False` → `RequestsRotating` (plain `requests`, supports `clear_cookies`).
- **clear_cookies** (`bool`) - Optional - Only meaningful with `is_tls=False`: clears the cookie jar before every request.
- **user_agent** (`str | None`) - Optional - Overrides `session.headers["user-agent"]` when truthy.

### Returns
`RequestsRotating | TLSRotating` - A session object with proxy rotation and default timeouts.

### Example
```python
from jobspy.util import create_session

session = create_session(
    proxies=["user:pass@host:8080", "localhost"],
    ca_cert="/etc/ssl/certs/proxy.pem",
    is_tls=True,
    user_agent="Mozilla/5.0",
)
r = session.get("https://example.com")
```
```

--------------------------------

### Bayt().scrape()

Source: https://github.com/speedyapply/jobspy/blob/main/_autodocs/api-reference/bayt.md

Scrapes job listings from Bayt.com based on the provided ScraperInput parameters. Returns a response object containing a list of jobs with details like id, title, company name, location, compensation, job type, vacancy count, and description.

```APIDOC
## Bayt().scrape()

### Description
Scrapes job listings from Bayt.com based on the provided ScraperInput parameters. Returns a response object containing a list of jobs with details like id, title, company name, location, compensation, job type, vacancy count, and description.

### Method
scrape

### Endpoint
Bayt().scrape(ScraperInput)

### Parameters
#### ScraperInput
- **search_term** (string) - Required - The job search query (e.g., "software engineer").
- **location** (string) - Optional - The location to search in (e.g., "Dubai, UAE"). If not found, the scraper logs a warning and widens the search automatically.
- **results_wanted** (integer) - Optional - The maximum number of job results to return.
- **hours_old** (integer) - Optional - Only return jobs posted within this many hours.
- **is_remote** (boolean) - Optional - If true, only return remote jobs.
- **fetch_description** (boolean) - Optional - If true, fetch and include job descriptions in the results.

### Request Example
```python
from jobspy import Bayt, ScraperInput

resp = Bayt().scrape(
    ScraperInput(
        search_term="software engineer",
        location="Dubai, UAE",
        results_wanted=30,
        hours_old=72,
        is_remote=True,
    )
)
```

### Response
#### Success Response
- **jobs** (list) - A list of job objects, each containing:
  - **id** (string) - Job identifier.
  - **title** (string) - Job title.
  - **company_name** (string) - Name of the company.
  - **location** (object) - Location details; use `display_location()` to get a string.
  - **compensation** (string) - Compensation information.
  - **description** (string) - Job description (if fetched).
  - **job_type** (string) - Type of job (e.g., full-time, part-time).
  - **vacancy_count** (integer) - Number of vacancies.

#### Response Example
```python
for job in resp.jobs:
    print(job.id, job.title, job.company_name,
          job.location.display_location(), job.compensation)
```

### Notes
- If the specified location is not found, the scraper logs a message like "Bayt: location '...' not found, searching ..." and automatically widens the search instead of failing.
```

--------------------------------

### Sample output from scrape_jobs

Source: https://github.com/speedyapply/jobspy/blob/main/README.md

Sample output from the scrape_jobs call, showing job details including site, title, company, location, salary, and URL.

```plaintext
SITE           TITLE                             COMPANY           CITY          STATE  JOB_TYPE  INTERVAL  MIN_AMOUNT  MAX_AMOUNT  JOB_URL                                            DESCRIPTION
indeed         Software Engineer                 AMERICAN SYSTEMS  Arlington     VA     None      yearly    200000      150000      https://www.indeed.com/viewjob?jk=5e409e577046...  THIS POSITION COMES WITH A 10K SIGNING BONUS!...
indeed         Senior Software Engineer          TherapyNotes.com  Philadelphia  PA     fulltime  yearly    135000      110000      https://www.indeed.com/viewjob?jk=da39574a40cb...  About Us TherapyNotes is the national leader i...
linkedin       Software Engineer - Early Career  Lockheed Martin   Sunnyvale     CA     fulltime  yearly    None        None        https://www.linkedin.com/jobs/view/3693012711      Description:By bringing together people that u...
linkedin       Full-Stack Software Engineer      Rain              New York      NY     fulltime  yearly    None        None        https://www.linkedin.com/jobs/view/3696158877      Rain’s mission is to create the fastest and ea...
zip_recruiter Software Engineer - New Grad       ZipRecruiter      Santa Monica  CA     fulltime  yearly    130000      150000      https://www.ziprecruiter.com/jobs/ziprecruiter...  We offer a hybrid work environment. Most US-ba...
zip_recruiter Software Developer                 TEKsystems        Phoenix       AZ     fulltime  hourly    65          75          https://www.ziprecruiter.com/jobs/teksystems-0...  Top Skills' Details• 6 years of Java developme...


```

--------------------------------

### LinkedIn, Indeed, Glassdoor, ZipRecruiter, Google, Bayt, Naukri, BDJobs

Source: https://github.com/speedyapply/jobspy/blob/main/_autodocs/api-reference/README.md

Site-specific scraper classes that implement the `Scraper` ABC for each job board.

```APIDOC
## LinkedIn, Indeed, Glassdoor, ZipRecruiter, Google, Bayt, Naukri, BDJobs

### Description
Site-specific scraper classes that implement the `Scraper` ABC for each job board.

### Method
Class instantiation and method calls

### Endpoint
`LinkedIn`, `Indeed`, `Glassdoor`, `ZipRecruiter`, `Google`, `Bayt`, `Naukri`, `BDJobs`

### Parameters
- `site` (Site) - Required - The job board site.
- `proxy` (str) - Optional - Proxy URL.
- `ca_cert` (str) - Optional - Path to CA certificate.
- `verify` (bool) - Optional - Verify SSL certificates.
- `session` (requests.Session) - Optional - Custom session object.

### Methods
- `scrape()` - Scrapes job postings based on the provided `ScraperInput`.

### Request Example
```python
from jobspy import LinkedIn, Indeed, Glassdoor, ZipRecruiter, Google, Bayt, Naukri, BDJobs

scraper = LinkedIn()
response = scraper.scrape(scraper_input)
```

### Response
#### Success Response
Returns a `JobResponse` object containing job postings.

#### Response Example
```python
print(response.jobs)
```
```
Tokens:
Raw