### GET / Source: https://r.jina.ai/docs This endpoint allows for crawling web pages with various customization options via headers. Preset values are applied to options not explicitly set by the caller. ```APIDOC ## GET / ### Description This endpoint allows for crawling web pages with various customization options via headers. Preset values are applied to options not explicitly set by the caller. ### Method GET ### Endpoint https://r.jina.ai/ ### Headers - **X-With-Iframe** (string) - Enable filling iframe contents into main. (violates standards) - **X-With-Shadow-Dom** (string) - Enable filling shadow dom contents into main. (violates standards) - **X-User-Agent** (string) - Override User-Agent. - **X-Timeout** (string) - Specify timeout in seconds. Max 180. - **X-Locale** (string) - Specify browser locale for the page. - **X-Referer** (string) - Specify referer for the page. - **X-Token-Budget** (string) - Specify a budget in tokens. If the resulting token cost exceeds the budget, the request is rejected. Note this parameter is ignored for the search endpoint. - **X-Max-Tokens** (string) - Trim the response content at a maximum number of tokens. - **X-Assert-StatusCode** (string) - Assert the HTTP status code of the crawled page. If the actual status code does not match the asserted one, the request is rejected with 422 Unprocessable Entity. - **X-Respond-Timing** (string) - Explicitly specify the respond timing. One of the following: * html: directly return unrendered HTML * visible-content: return immediately when any content becomes available * mutation-idle: wait for DOM mutations to settle and remain unchanged for at least 0.2s * resource-idle: wait for no additional resources that would affect page logic and content has SUCCEEDED loading in 0.5s * media-idle: wait for no additional resources, including media resources, has SUCCEEDED loading in 0.5s * network-idle: wait for full load of webpage, also known as networkidle0. - **X-Engine** (string) - Specify the engine to use for crawling. Supported: browser, direct, cf-browser-rendering - **X-Base** (string) - Select base modes of relative URLs. Supported: initial, final - **X-Remove-Overlay** (string) - Specify whether to remove overlay elements from the page. - **X-Detach-Invisibles** (string) - Temporarily detach elements with computed `display:none` before snapshotting, then restore them. Requires the browser engine and disables caching. - **X-Markdown-Chunking** (string) - Opt-in markdown chunking. Supported values: * true * h1 * h2 * h3 * h4 * h5 * structured * s1 * s2 * s3 * s4 * s5 Note if you are expecting text return, the chunking is done by injecting character \u001D - **X-Md-Heading-Style** (string) - Heading style of the generated markdown. This is an option passed through to Turndown. Supported: setext, atx - **X-Md-Hr** (string) - Hr text of the generated markdown. This is an option passed through to Turndown. - **X-Md-Bullet-List-Marker** (string) - Bullet list marker of the generated markdown. This is an option passed through to Turndown. Supported: -, +, * - **X-Md-Em-Delimiter** (string) - Em delimiter of the generated markdown. This is an option passed through to Turndown. Supported: _, * - **X-Md-Strong-Delimiter** (string) - Strong delimiter of the generated markdown. This is an option passed through to Turndown. Supported: **, __ - **X-Md-Link-Style** (string) - Link style of the generated markdown. This is an option passed through to Turndown. Supported: inlined, referenced, discarded - **X-Md-Link-Reference-Style** (string) - Link reference style of the generated markdown. This is an option passed through to Turndown. Supported: full, collapsed, shortcut, discarded - **X-Page** (string) - Steer which page to return for uploaded files (PDF, DOC, XLS, PPT, etc.). 1-indexed; ignored when no file/PDF is uploaded. ### Responses #### Success Response (200) OK #### Response Example ```json { "code": 200, "status": 20000, "data": "string", "meta": null } ``` ``` -------------------------------- ### GET / Source: https://r.jina.ai/docs This endpoint allows for web scraping and content retrieval with extensive customization options via headers. ```APIDOC ## GET / ### Description This endpoint serves as the primary interface for web scraping and content retrieval. It supports a wide array of customizable headers to control the crawling process, response format, and content processing. ### Method GET ### Endpoint https://r.jina.ai/ ### Parameters #### Query Parameters None explicitly documented. #### Headers - **X-With-Iframe** (string) - Enable filling iframe contents into main. (violates standards) - **X-With-Shadow-Dom** (string) - Enable filling shadow dom contents into main. (violates standards) - **X-User-Agent** (string) - Override User-Agent. - **X-Timeout** (string) - Specify timeout in seconds. Max 180. - **X-Locale** (string) - Specify browser locale for the page. - **X-Referer** (string) - Specify referer for the page. - **X-Token-Budget** (string) - Specify a budget in tokens. If the resulting token cost exceeds the budget, the request is rejected. Note this parameter is ignored for the search endpoint. - **X-Max-Tokens** (string) - Trim the response content at a maximum number of tokens. - **X-Assert-Status-Code** (string) - Assert the HTTP status code of the crawled page. If the actual status code does not match the asserted one, the request is rejected with 422 Unprocessable Entity. - **X-Respond-Timing** (string) - Explicitly specify the respond timing. One of the following: * html: directly return unrendered HTML * visible-content: return immediately when any content becomes available * mutation-idle: wait for DOM mutations to settle and remain unchanged for at least 0.2s * resource-idle: wait for no additional resources that would affect page logic and content has SUCCEEDED loading in 0.5s * media-idle: wait for no additional resources, including media resources, has SUCCEEDED loading in 0.5s * network-idle: wait for full load of webpage, also known as networkidle0. - **X-Engine** (string) - Specify the engine to use for crawling. Supported: browser, direct, cf-browser-rendering - **X-Base** (string) - Select base modes of relative URLs. Supported: initial, final - **X-Remove-Overlay** (string) - Specify whether to remove overlay elements from the page. - **X-Detach-Invisibles** (string) - Temporarily detach elements with computed `display:none` before snapshotting, then restore them. Requires the browser engine and disables caching. - **X-Markdown-Chunking** (string) - Opt-in markdown chunking. Supported values: * true * h1 * h2 * h3 * h4 * h5 * structured * s1 * s2 * s3 * s4 * s5 Note if you are expecting text return, the chunking is done by injecting character \u001D - **X-Md-Heading-Style** (string) - Heading style of the generated markdown. This is an option passed through to Turndown. Supported: setext, atx - **X-Md-Hr** (string) - Hr text of the generated markdown. This is an option passed through to Turndown. - **X-Md-Bullet-List-Marker** (string) - Bullet list marker of the generated markdown. This is an option passed through to Turndown. Supported: -, +, * - **X-Md-Em-Delimiter** (string) - Em delimiter of the generated markdown. This is an option passed through to Turndown. Supported: _, * - **X-Md-Strong-Delimiter** (string) - Strong delimiter of the generated markdown. This is an option passed through to Turndown. Supported: **, __ - **X-Md-Link-Style** (string) - Link style of the generated markdown. This is an option passed through to Turndown. Supported: inlined, referenced, discarded - **X-Md-Link-Reference-Style** (string) - Link reference style of the generated markdown. This is an option passed through to Turndown. Supported: full, collapsed, shortcut, discarded - **X-Page** (string) - Steer which page to return for uploaded files (PDF, DOC, XLS, PPT, etc.). 1-indexed; ignored when no file/PDF is uploaded. ### Request Example ```http GET https://r.jina.ai/https://example.com Host: r.jina.ai X-Respond-Timing: mutation-idle X-Engine: browser ``` ### Response #### Success Response (200) - **code** (integer) - HTTP status code. - **status** (integer) - Internal status code. - **data** (string) - The retrieved content or data. - **meta** (null) - Metadata, currently null. #### Response Example ```json { "code": 200, "status": 20000, "data": "string", "meta": null } ``` ``` -------------------------------- ### Jina AI API - GET Request Source: https://r.jina.ai/docs This section details the GET endpoint for the Jina AI API and the various headers that can be used to modify its behavior. ```APIDOC ## GET /url ### Description This endpoint allows you to retrieve content from a given URL with various customization options controlled by request headers. ### Method GET ### Endpoint `get/{url}` https://r.jina.ai/{url} ### Headers - **X-With-Iframe** (string) - Enable filling iframe contents into main. (violates standards) - **X-With-Shadow-Dom** (string) - Enable filling shadow dom contents into main. (violates standards) - **X-User-Agent** (string) - Override User-Agent. - **X-Timeout** (string) - Specify timeout in seconds. Max 180. - **X-Locale** (string) - Specify browser locale for the page. - **X-Referer** (string) - Specify referer for the page. - **X-Token-Budget** (string) - Specify a budget in tokens. If the resulting token cost exceeds the budget, the request is rejected. Note this parameter is ignored for the search endpoint. - **X-Max-Tokens** (string) - Trim the response content at a maximum number of tokens. - **X-Assert-Status-Code** (string) - Assert the HTTP status code of the crawled page. If the actual status code does not match the asserted one, the request is rejected with 422 Unprocessable Entity. - **X-Respond-Timing** (string) - Explicitly specify the respond timing. One of the following: * html: directly return unrendered HTML * visible-content: return immediately when any content becomes available * mutation-idle: wait for DOM mutations to settle and remain unchanged for at least 0.2s * resource-idle: wait for no additional resources that would affect page logic and content has SUCCEEDED loading in 0.5s * media-idle: wait for no additional resources, including media resources, has SUCCEEDED loading in 0.5s * network-idle: wait for full load of webpage, also known as networkidle0. - **X-Engine** (string) - Specify the engine to use for crawling. Supported: browser, direct, cf-browser-rendering - **X-Base** (string) - Select base modes of relative URLs. Supported: initial, final - **X-Remove-Overlay** (string) - Specify whether to remove overlay elements from the page. - **X-Detach-Invisibles** (string) - Temporarily detach elements with computed `display:none` before snapshotting, then restore them. Requires the browser engine and disables caching. - **X-Markdown-Chunking** (string) - Opt-in markdown chunking. Supported values: * true * h1 * h2 * h3 * h4 * h5 * structured * s1 * s2 * s3 * s4 * s5 Note if you are expecting text return, the chunking is done by injecting character \u001D - **X-Md-Heading-Style** (string) - Heading style of the generated markdown. This is an option passed through to Turndown. Supported: setext, atx - **X-Md-Hr** (string) - Hr text of the generated markdown. This is an option passed through to Turndown. - **X-Md-Bullet-List-Marker** (string) - Bullet list marker of the generated markdown. This is an option passed through to Turndown. Supported: -, +, * - **X-Md-Em-Delimiter** (string) - Em delimiter of the generated markdown. This is an option passed through to Turndown. Supported: _, * - **X-Md-Strong-Delimiter** (string) - Strong delimiter of the generated markdown. This is an option passed through to Turndown. Supported: **, __ - **X-Md-Link-Style** (string) - Link style of the generated markdown. This is an option passed through to Turndown. Supported: inlined, referenced, discarded - **X-Md-Link-Reference-Style** (string) - Link reference style of the generated markdown. This is an option passed through to Turndown. Supported: full, collapsed, shortcut, discarded - **X-Page** (string) - Steer which page to return for uploaded files (PDF, DOC, XLS, PPT, etc.). 1-indexed; ignored when no file/PDF is uploaded. ### Responses #### Success Response (200) OK Content type: text/plain; charset=utf-8, application/json, text/event-stream #### Response Example No sample ``` -------------------------------- ### Response Samples Source: https://r.jina.ai/docs This snippet shows the possible response types and content types for the Jina AI API. ```APIDOC ## Response Samples ### Success Response (200) #### Content Type - text/plain; charset=utf-8 - application/json - text/event-stream - text/plain; charset=utf-8 ### Response Example No sample provided in the source. ``` -------------------------------- ### POST /post/{url} Source: https://r.jina.ai/docs This endpoint allows you to post content to a specified URL. It accepts various options to control the crawling and processing behavior. ```APIDOC ## POST /post/{url} ### Description This endpoint allows you to post content to a specified URL. It accepts various options to control the crawling and processing behavior. ### Method POST ### Endpoint https://r.jina.ai/{url} ### Parameters #### Query Parameters - **preset** (string Enum: "reader" "index" "research" "agent" "spider") - Optional - Specifies a preset configuration for the crawler. - **retainLinks** (string Enum: "none" "all" "text" "gpt-oss" Default: "all") - Optional - Determines which links to retain. - **withLinksSummary** (PseudoBooleanLoose) - Optional - Whether to include a summary of links. - **withImagesSummary** (PseudoBooleanLoose) - Optional - Whether to include a summary of images. - **noCache** (PseudoBooleanLoose) - Optional - Disables caching. - **removeOverlay** (PseudoBooleanLoose) - Optional - Removes overlay elements. - **detachInvisibles** - Optional - Detaches invisible elements. - **noGfm** - Optional - Disables GitHub Flavored Markdown. - **cacheTolerance** (number) - Optional - Tolerance for cache. - **targetSelector** (Array of strings) - Optional - CSS selectors for target elements. - **waitForSelector** (Array of strings) - Optional - CSS selectors to wait for. - **removeSelector** (Array of strings) - Optional - CSS selectors to remove. - **keepImgDataUrl** (PseudoBooleanLoose) - Optional - Keeps image data URLs. - **withIframe** - Optional - Includes iframe content. - **withShadowDom** (PseudoBooleanLoose) - Optional - Includes shadow DOM content. - **setCookies** (Array of object|string) - Optional - Sets cookies. - **storageState** (StorageState) - Optional - Provides storage state for the session. - **exportStorageState** (PseudoBooleanLoose) - Optional - Exports storage state. - **proxyUrl** (string) - Optional - URL for the proxy server. - **proxy** (string) - Optional - Proxy configuration. - **userAgent** (string) - Optional - Custom user agent string. - **engine** - Optional - Specifies the rendering engine. - **injectPageScript** (Array of strings) - Optional - Scripts to inject into the page. - **injectFrameScript** (Array of strings) - Optional - Scripts to inject into frames. - **timeout** (number) - Optional - Timeout in seconds for the request. Must be between 1 and 180. - **locale** (string) - Optional - Locale for the request. - **referer** (string) - Optional - Referer header value. - **tokenBudget** (number) - Optional - Token budget for the request. - **assertStatusCode** (number) - Optional - Asserts the status code of the response. - **viewport** (Viewport) - Optional - Viewport configuration. - **instruction** (string) - Optional - Custom instruction for the API. - **jsonSchema** (object) - Optional - JSON schema for validation. - **robotsTxt** (string) - Optional - Content of robots.txt. - **doNotTrack** (object) - Optional - Do not track settings. - **markdown** (TurnDownTweakableOptions) - Optional - Markdown conversion options. - **respondTiming** (string Enum: "html" "visible-content" "mutation-idle" "resource-idle" "media-idle" "network-idle") - Optional - Specifies when to respond. - **markdownChunking** - Optional - Markdown chunking options. - **maxTokens** (number) - Optional - Maximum tokens for the response. Must be at least 500. - **customHeader** (Record ) - Optional - Custom headers to include in the request. ### Responses #### Success Response (200) OK ``` -------------------------------- ### getIndex Source: https://r.jina.ai/docs Retrieves and processes web content based on the provided query parameters. This endpoint is designed to fetch various forms of content from URLs, with extensive options for customization. ```APIDOC ## getIndex ### Description Retrieves and processes web content based on the provided query parameters. This endpoint is designed to fetch various forms of content from URLs, with extensive options for customization. ### Method GET ### Endpoint /getIndex ### Parameters #### Query Parameters - **url** (string) - Required - The URL of the web page to process. - **html** (string) - Optional - HTML content to process instead of fetching from a URL. - **base** (string) - Optional - Specifies the base for processing, either 'initial' or 'final'. Defaults to 'initial'. - **pdf** (FancyFile|string) - Optional - A PDF file or Base64 encoded PDF to process. - **file** (FancyFile|string) - Optional - A file or Base64 encoded file to process. - **page** (number) - Optional - The 1-indexed page number to return for uploaded files. Ignored when no file/PDF is uploaded. Must be an integer >= 1. - **respondWith** (string) - Optional - Specifies the format of the response. Options include 'content', 'markdown', 'html', 'text', 'pageshot', 'screenshot', 'vlm', 'readerlm-v2', 'frontmatter'. Defaults to 'content'. - **withGeneratedAlt** (PseudoBoolean) - Optional - Whether to generate alt text for images. - **retainImages** (string) - Optional - Specifies which images to retain. Options: 'none', 'all', 'alt', 'all_p', 'alt_p'. Defaults to 'all'. - **retainMedia** (string) - Optional - Specifies which media to retain. Options: 'none', 'text', 'link', 'image', 'html'. Defaults to 'link'. - **preset** (string) - Optional - A preset configuration for the reader. Options: 'reader', 'index', 'research', 'agent', 'spider'. - **retainLinks** (string) - Optional - Specifies which links to retain. Options: 'none', 'all', 'text', 'gpt-oss'. Defaults to 'all'. - **withLinksSummary** (PseudoBoolean|string) - Optional - Whether to include a summary of links. - **withImagesSummary** (PseudoBooleanLoose) - Optional - Whether to include a summary of images. - **noCache** (PseudoBooleanLoose) - Optional - Whether to disable caching. - **removeOverlay** (PseudoBooleanLoose) - Optional - Whether to remove overlays. - **detachInvisibles** (PseudoBooleanLoose) - Optional - Whether to detach invisible elements. - **noGfm** (PseudoBoolean|string) - Optional - Whether to disable GitHub Flavored Markdown. - **cacheTolerance** (number) - Optional - Tolerance for cache in seconds. - **targetSelector** (Array) - Optional - CSS selectors for elements to target. - **waitForSelector** (Array) - Optional - CSS selectors to wait for before processing. - **removeSelector** (Array) - Optional - CSS selectors for elements to remove. - **keepImgDataUrl** (PseudoBooleanLoose) - Optional - Whether to keep image data URLs. - **withIframe** (PseudoBoolean|string) - Optional - Whether to process content within iframes. - **withShadowDom** (PseudoBooleanLoose) - Optional - Whether to process content within shadow DOM. - **setCookies** (Array) - Optional - Cookies to set for the request. - **exportStorageState** (PseudoBooleanLoose) - Optional - Whether to export storage state. - **proxyUrl** (string) - Optional - URL of a proxy server to use. - **proxy** (string) - Optional - Proxy configuration. - **userAgent** (string) - Optional - User agent string to use for the request. - **engine** (string) - Optional - The rendering engine to use. Options: 'auto', 'browser', 'curl', 'cf-browser-rendering'. - **injectPageScript** (Array) - Optional - Scripts to inject into the page. - **injectFrameScript** (Array) - Optional - Scripts to inject into frames. - **timeout** (number) - Optional - Timeout in seconds for the request. Must be between 1 and 180. - **locale** (string) - Optional - Locale for the request. - **referer** (string) - Optional - Referer header for the request. - **tokenBudget** (number) - Optional - Token budget for the request. - **assertStatusCode** (number) - Optional - Assert the status code of the response. - **instruction** (string) - Optional - An instruction for the reader. ### Request Example ```json { "url": "https://example.com", "respondWith": "markdown", "retainImages": "alt" } ``` ### Response #### Success Response (200) - **content** (string) - The processed content in the requested format. ``` -------------------------------- ### Header Parameters Source: https://r.jina.ai/docs These headers can be used to customize the request and control the crawler's output. ```APIDOC ## Header Parameters ### Authorization * Type: string * Description: Jina Token for authentication. * Format: `Authorization: Bearer {YOUR_JINA_TOKEN}` ### Accept * Type: string * Description: Specifies your preference for the response format. Supported formats: `text/event-stream`, `application/json` or `text/json`. ### X-Cache-Tolerance * Type: string * Description: Sets internal cache tolerance in seconds. ### X-No-Cache * Type: string * Description: Ignores internal cache if specified with a value. Equivalent to `X-Cache-Tolerance: 0`. ### X-Respond-With * Type: string * Description: Specifies the preferred form of the crawled data. Supported formats: `markdown`, `html`, `text`, `pageshot`, `screenshot`, `content`, `frontmatter`, `readerlm-v2`, `vlm`, or any combination. Default: `content`. ### X-Wait-For-Selector * Type: string * Description: Specifies a CSS selector to wait for before returning. Example: `X-Wait-For-Selector: .content-block`. ### X-Target-Selector * Type: string * Description: Specifies a CSS selector for the target return instead of the full HTML. Implies `X-Wait-For-Selector`. ### X-Remove-Selector * Type: string * Description: Specifies a CSS selector to remove elements from the full HTML. Example: `X-Remove-Selector: nav`. ### X-Keep-Img-Data-Url * Type: string * Description: Keep data-url as is instead of transforming them to object-url. Only applicable when targeting markdown format. Example: `X-Keep-Img-Data-Url: true`. ### X-Proxy-Url * Type: string * Description: Specifies a custom proxy. Supported protocols: `http`, `https`, `socks4`, `socks5`. For authentication: `https://user:pass@host:port`. ### X-Proxy * Type: string * Description: Use a proxy server provided by Jina. Optionally specify a two-letter country code. ### X-Robots-Txt * Type: string * Description: Load and conform to the respective robot.txt on the target origin. Optionally specify a bot UA to check against. ### DNT * Type: string * Description: When set to 1, prevent the result of this request to be cached. ### X-Set-Cookie * Type: string * Description: Sets cookie(s) to the headless browser for your request. Syntax is the same with standard Set-Cookie. ### X-Export-Storage-State * Type: string * Description: Export the browser storage state (cookies and per-origin localStorage). Requires the browser engine and disables caching. ### X-With-Generated-Alt * Type: string * Description: Enable automatic alt-text generating for images without meaningful alt-text. Note: Does not work when `X-Respond-With` is specified. ### X-With-Images-Summary * Type: string * Description: Enable dedicated summary section for images on the page. ### X-With-links-Summary * Type: string * Description: Enable dedicated summary section for hyper links on the page. ### X-Retain-Links * Type: string * Description: Link retention modes. Supported modes: `all`, `none`, `text`, `gpt-oss`. ### X-Retain-Images * Type: string * Description: Image retention modes. Supported modes: `all`, `none`, `alt`, `all_p`, `alt_p`. ### X-Retain-Media * Type: string * Description: Media retention modes for video, audio, and embedded video iframes. Supported modes: `none`, `text`, `link` (default), `image`, `html`. ### X-Preset * Type: string * Description: Apply a preset configuration for common scenarios. Supported presets: `reader`, `index`, `research`, `agent`, `spider`. ``` -------------------------------- ### Jina AI API Response Sample Source: https://r.jina.ai/docs This is a sample JSON response structure for a successful API call. It includes a status code, a data field which contains the processed content, and metadata. ```json { "code": 200, "status": 20000, "data": "string", "meta": null } ``` -------------------------------- ### getIndex Source: https://r.jina.ai/docs Retrieves indexed information from a given URL with various customization options. ```APIDOC ## getIndex ### Description Retrieves indexed information from a given URL with various customization options. ### Query Parameters - **url** (string) - Required - The URL to crawl. - **html** (string) - Optional - HTML content to process. - **base** (string) - Optional - Base for relative URLs. Defaults to 'initial'. Enum: "initial" "final". - **pdf** (FancyFile|string) - Optional - PDF file to process. - **file** (FancyFile|string) - Optional - File to process. - **page** (number) - Optional - Page number to return for uploaded files (1-indexed). Ignored when no file/PDF is uploaded. - **respondWith** (string) - Optional - Format of the response. Defaults to 'content'. Enum: "content" "markdown" "html" "text" "pageshot" "screenshot" "vlm" "readerlm-v2" "frontmatter". - **withGeneratedAlt** (PseudoBoolean) - Optional - Whether to generate alt text for images. - **retainImages** (string) - Optional - How to retain images. Defaults to 'all'. Enum: "none" "all" "alt" "all_p" "alt_p". - **retainMedia** (string) - Optional - How to retain media. Defaults to 'link'. Enum: "none" "text" "link" "image" "html". - **preset** (string) - Optional - Preset configuration for crawling. Enum: "reader" "index" "research" "agent" "spider". - **retainLinks** (string) - Optional - How to retain links. Defaults to 'all'. Enum: "none" "all" "text" "gpt-oss". - **withLinksSummary** (PseudoBoolean|string) - Optional - Whether to include a summary of links. - **withImagesSummary** (PseudoBooleanLoose) - Optional - Whether to include a summary of images. - **noCache** (PseudoBooleanLoose) - Optional - Whether to disable caching. - **removeOverlay** (PseudoBooleanLoose) - Optional - Whether to remove overlays. - **detachInvisibles** (PseudoBooleanLoose) - Optional - Whether to detach invisible elements. - **noGfm** (PseudoBoolean|string) - Optional - Whether to disable GitHub Flavored Markdown. - **cacheTolerance** (number) - Optional - Cache tolerance in seconds. - **targetSelector** (Array) - Optional - CSS selectors for target elements. - **waitForSelector** (Array) - Optional - CSS selectors to wait for. - **removeSelector** (Array) - Optional - CSS selectors to remove. - **keepImgDataUrl** (PseudoBooleanLoose) - Optional - Whether to keep image data URLs. - **withIframe** (PseudoBoolean|string) - Optional - Whether to process iframes. - **withShadowDom** (PseudoBooleanLoose) - Optional - Whether to process shadow DOM. - **setCookies** (Array) - Optional - Cookies to set. - **exportStorageState** (PseudoBooleanLoose) - Optional - Whether to export storage state. - **proxyUrl** (string) - Optional - URL for the proxy server. - **proxy** (string) - Optional - Proxy configuration. - **userAgent** (string) - Optional - User agent string. - **engine** (string) - Optional - Rendering engine to use. Enum: "auto" "browser" "curl" "cf-browser-rendering". - **injectPageScript** (Array) - Optional - JavaScript scripts to inject into the page. - **injectFrameScript** (Array) - Optional - JavaScript scripts to inject into iframes. - **timeout** (number) - Optional - Timeout in seconds. Must be between 1 and 180. - **locale** (string) - Optional - Locale for the request. - **referer** (string) - Optional - Referer header. - **tokenBudget** (number) - Optional - Token budget for the request. - **assertStatusCode** (number) - Optional - Assert the status code of the response. - **instruction** (string) - Optional - Instruction for the crawler. - **jsonSchema** (object) - Optional - JSON schema for validation. ``` -------------------------------- ### crawlByPostingToIndex Source: https://r.jina.ai/docs This function allows you to crawl a URL and post the content to an index. It accepts a wide range of options to control the crawling process. ```APIDOC ## crawlByPostingToIndex ### Description Allows crawling a URL and posting the content to an index with customizable options. ### Method POST (Assumed, as it's posting to an index) ### Endpoint /index (Assumed, based on 'postingToIndex') ### Parameters #### Query Parameters - **url** (string) - The URL to crawl. - **html** (string) - The HTML content to process. - **base** (string) - Defaults to 'initial'. Can be 'initial' or 'final'. - **pdf** (FancyFile|string) - A PDF file or Base64 encoded PDF string. - **file** (FancyFile|string) - A file or Base64 encoded file string. - **page** (number) - Steer which page to return for uploaded files (1-indexed). Ignored when no file/PDF is uploaded. Must be an integer >= 1. - **respondWith** (string) - Defaults to 'content'. Can be 'content', 'markdown', 'html', 'text', 'pageshot', 'screenshot', 'vlm', 'readerlm-v2', 'frontmatter', or a custom string. - **withGeneratedAlt** (PseudoBoolean) - Enables generation of alt text. - **retainImages** (string) - Defaults to 'all'. Can be 'none', 'all', 'alt', 'all_p', 'alt_p'. - **retainMedia** (string) - Defaults to 'link'. Can be 'none', 'text', 'link', 'image', 'html'. - **preset** (string) - Can be 'reader', 'index', 'research', 'agent', 'spider'. - **retainLinks** (string) - Defaults to 'all'. Can be 'none', 'all', 'text', 'gpt-oss'. - **withLinksSummary** (PseudoBoolean|string) - Enables summary of links. - **withImagesSummary** (PseudoBooleanLoose) - Enables summary of images. - **noCache** (PseudoBooleanLoose) - Disables caching. - **removeOverlay** (PseudoBooleanLoose) - Removes overlays. - **detachInvisibles** (PseudoBooleanLoose) - Detaches invisible elements. - **noGfm** (PseudoBoolean|string) - Disables GitHub Flavored Markdown parsing. - **cacheTolerance** (number) - Tolerance for cache in seconds. - **targetSelector** (Array) - CSS selectors for elements to target. - **waitForSelector** (Array) - CSS selectors to wait for before processing. - **removeSelector** (Array) - CSS selectors for elements to remove. - **keepImgDataUrl** (PseudoBooleanLoose) - Keeps image data URLs. - **withIframe** (PseudoBoolean|string) - Includes iframe content. - **withShadowDom** (PseudoBooleanLoose) - Includes content from shadow DOM. - **setCookies** (Array) - Cookies to set for the request. - **exportStorageState** (PseudoBooleanLoose) - Exports storage state. - **proxyUrl** (string) - URL for the proxy server. - **proxy** (string) - Proxy configuration. - **userAgent** (string) - User agent string to use for the request. - **engine** (string) - Defaults to 'auto'. Can be 'auto', 'browser', 'curl', 'cf-browser-rendering', or a custom string. - **injectPageScript** (Array) - Scripts to inject into the page. - **injectFrameScript** (Array) - Scripts to inject into frames. - **timeout** (number) - Timeout in seconds for the operation. Must be between 1 and 180. - **locale** (string) - Locale for the request. - **referer** (string) - Referer header for the request. - **tokenBudget** (number) - Token budget for the operation. - **assertStatusCode** (number) - Assert the HTTP status code of the response. - **instruction** (string) - Instruction for the crawler. - **jsonSchema** (object) - A JSON schema to validate the output against. ``` -------------------------------- ### Crawler Options Source: https://r.jina.ai/docs These options can be configured within the CrawlerOptions object to modify crawling behavior. ```APIDOC ## Crawler Options ### robotsTxt * Type: string * Description: Controls robots.txt behavior. ### doNotTrack * Type: any or null * Description: Can be any value to indicate do not track. ### respondTiming * Type: string * Enum: "html", "visible-content", "mutation-idle", "resource-idle", "media-idle", "network-idle" * Description: Specifies when the response should be considered ready. ### markdownChunking * Type: string or string * Enum: "true", "h1", "h2", "h3", "h4", "h5", "structured", "s1", "s2", "s3", "s4", "s5", string * Description: Controls markdown chunking behavior. ### maxTokens * Type: number * Description: Maximum number of tokens to generate. Must be >= 500. ``` -------------------------------- ### Jina AI API Endpoint Source: https://r.jina.ai/docs The Jina AI API allows users to scrape web content and process it using various options. The primary endpoint is POST /. ```APIDOC ## POST / ### Description This endpoint allows for web scraping and content processing with a wide range of configurable options. ### Method POST ### Endpoint https://r.jina.ai/ ### Parameters #### Query Parameters - **preset** (string Enum: "reader" "index" "research" "agent" "spider") - Specifies a preset configuration for the crawler. - **retainLinks** (string Enum: "none" "all" "text" "gpt-oss", Default: "all") - Controls which links are retained from the scraped content. - **withLinksSummary** (PseudoBoolean) - Whether to include a summary of links. - **withImagesSummary** (PseudoBooleanLoose) - Whether to include a summary of images. - **noCache** (PseudoBooleanLoose) - Disables caching for the request. - **removeOverlay** (PseudoBooleanLoose) - Removes overlay elements from the page. - **detachInvisibles** (PseudoBooleanLoose) - Detaches invisible elements. - **noGfm** (PseudoBoolean) - Disables GitHub Flavored Markdown processing. - **cacheTolerance** (number) - Tolerance level for cache matching. - **targetSelector** (Array of strings) - CSS selectors to target specific elements. - **waitForSelector** (Array of strings) - CSS selectors to wait for before processing. - **removeSelector** (Array of strings) - CSS selectors for elements to remove. - **keepImgDataUrl** (PseudoBooleanLoose) - Keeps image data URLs. - **withIframe** (PseudoBoolean) - Whether to process iframe content. - **withShadowDom** (PseudoBooleanLoose) - Processes content within shadow DOM. - **setCookies** (Array of object|string) - Cookies to set for the request. - **storageState** (object StorageState) - State object for storage. - **exportStorageState** (PseudoBooleanLoose) - Exports the storage state. - **proxyUrl** (string) - URL of the proxy to use. - **proxy** (string) - Proxy configuration. - **userAgent** (string) - User agent string to use for the request. - **engine** (string) - The rendering engine to use. - **injectPageScript** (Array of strings) - Scripts to inject into the page. - **injectFrameScript** (Array of strings) - Scripts to inject into iframes. - **timeout** (number) - Timeout in seconds for the request (1-180). - **locale** (string) - Locale for the request. - **referer** (string) - Referer header for the request. - **tokenBudget** (number) - Token budget for processing. - **assertStatusCode** (number) - Assert the status code of the response. - **viewport** (object Viewport) - Viewport configuration. - **instruction** (string) - Instructions for processing. - **jsonSchema** (object) - JSON schema for validation. - **robotsTxt** (string) - Robots.txt content. - **doNotTrack** (object) - Do not track settings. - **markdown** (object TurnDownTweakableOptions) - Markdown processing options. - **respondTiming** (string Enum: "html" "visible-content" "mutation-idle" "resource-idle" "media-idle" "network-idle") - Timing strategy for response. - **markdownChunking** (string) - Markdown chunking strategy. - **maxTokens** (number) - Maximum tokens for processing (>= 500). - **customHeader** (object) - Custom headers to include in the request. ### Responses #### Success Response (200) OK ```