EAIWS Crawler Plugin

https://docs.pcon-solutions.com/eaiws/plugins/crawler/1.8/
Admin
EAIWS Crawler Plugin is a data collection and processing plugin that mass collects OFML data via the...
Tokens:14,488
Snippets:96
Trust Score:9.4
Update:7 months ago
Show doc for...
Tokens:
Raw
### EAIWS Crawler Plugin Global Configuration Example (JSON)

Source: https://docs.pcon-solutions.com/eaiws/plugins/crawler/1.8/index

An example of the global configuration settings for the EAIWS crawler plugin, represented in JSON format. This includes parameters like retry counts, timeouts, and database cleanup intervals.

```json
{
  "callbackConnectionRetries": 2,
  "callbackConnectionTimeout": "PT5S",
  "callbackDelayBeforeRetry": "PT1M",
  "callbackReadTimeout": "PT1M",
  "databaseCleanupInterval": "PT1D",
  "databaseMaxAge": "PT7D",
  "databaseMaxSize": 0,
  "databaseNameForJournal": "journal.db",
  "databaseNameFormat": "yyyy-MM-dd-HH-mm-ss'.db'",
  "maxBasketWorkersPerJob": 8,
  "maxGlobalConcurrentRunningBasketWorkers": 32,
  "minFreeSpace": 10485760,
  "spacePatrolInterval": "P1M"
}
```

--------------------------------

### Start Crawling Process API Request

Source: https://docs.pcon-solutions.com/eaiws/plugins/crawler/1.8/index

This is an example of a REST API request to initiate a crawling process using the EAIWS Crawler plugin. The POST request is sent to the '/EAIWS/plugins/Crawler/' endpoint. A successful request returns a 'jobId' which can be used to monitor the task's progress.

```http
POST /EAIWS/plugins/Crawler/ HTTP/1.1
Host: your_server_address
Content-Type: application/json

{
  "some_parameter": "some_value"
}
```

--------------------------------

### POST /EAIWS/plugins/Crawler/

Source: https://docs.pcon-solutions.com/eaiws/plugins/crawler/1.8/index

Starts the crawling process. Returns a `jobId` upon success. The actual processing time can vary significantly.

```APIDOC
## POST /EAIWS/plugins/Crawler/

### Description

This request starts the crawling process. In case of success, a `jobId` is delivered as a response. However, the actual processing of the job can take from a few seconds to several hours. The status and the final result can be requested using other API endpoints.

### Method

POST

### Endpoint

/EAIWS/plugins/Crawler/

### Request Body

*(No specific request body fields are detailed in the provided text, but it's implied that configuration parameters might be sent here to initiate the crawl.)*

### Response

#### Success Response (200)

- **jobId** (string) - The unique identifier for the initiated crawling job.

#### Response Example

```json
{
  "jobId": "some-unique-job-id"
}
```
```

--------------------------------

### Start Crawling Process

Source: https://docs.pcon-solutions.com/eaiws/plugins/crawler/1.8/index

Initiates a web crawling process. This endpoint allows for detailed configuration of the crawling job, including data sources, callback URLs, and debug settings.

```APIDOC
## POST /websites/pcon-solutions_eaiws_plugins_crawler_1_8

### Description
Starts a web crawling process with specified configurations.

### Method
POST

### Endpoint
/websites/pcon-solutions_eaiws_plugins_crawler_1_8

### Parameters
#### Query Parameters
- **debugMaxDataSourceEntries** (integer) - Optional - This is a debug function that can be used to limit the number of articles that are indexed. The indexing is stopped as soon as the number of articles specified by this value has been processed.

#### Request Body
- **workerConfig** (string) - Required - Configuration for the worker.
- **dataSource** (object) - Required - Data source configuration.
  - **type** (string) - Required - Type of the data source (e.g., "catalog").
  - **data** (object) - Required - Data specific to the source type.
    - **displayMode** (string) - Required - Display mode for the data (e.g., "All").
- **openSessionArguments** (array) - Optional - Arguments for opening a session.
  - **first** (string) - Required - First element of the argument pair.
  - **second** (string) - Required - Second element of the argument pair.
- **callbackUrl** (string) - Optional - URL to receive progress notifications.
- **clientId** (string) - Optional - Client identifier.
- **accessToken** (string) - Optional - Access token for authentication.
- **refreshToken** (string) - Optional - Refresh token for authentication.
- **priceDate** (string) - Optional - Date for price information (YYYY-MM-DD).
- **languageList** (array) - Optional - List of languages to consider.
  - (string) - Required - Language code.
- **disable** (array) - Optional - List of features to disable.
  - (string) - Required - Feature name.
- **geometryExportSettings** (object) - Optional - Settings for geometry export.
  - **export key1** (array) - Optional - List of geometry export keys.
    - (string) - Required - Geometry export key.
  - **export key2** (array) - Optional - List of geometry export keys.
    - (string) - Required - Geometry export key.
- **imageExportSettings** (object) - Optional - Settings for image export.
  - **rendering key1** (array) - Optional - List of image rendering keys.
    - (string) - Required - Image rendering key.
  - **rendering key2** (array) - Optional - List of image rendering keys.
    - (string) - Required - Image rendering key.
- **basketWorkers** (integer) - Optional - Number of basket workers.
- **hooks** (array) - Optional - Hooks for processing.
  - **type** (string) - Required - Type of the hook (e.g., "ItemAppData").
  - **data** (object) - Required - Data for the hook.
    - **mode** (string) - Required - Mode of the hook data (e.g., "xml").
    - **appKey** (string) - Required - Application key for the hook.
    - **value** (string) - Required - Value for the hook.
- **debugFilterCatalogIds** (array) - Optional - List of catalog IDs for debugging filters.
  - (string) - Required - Catalog ID.
- **debugIndexOnlyEveryNthArticle** (integer) - Optional - Debug setting to index only every Nth article.
- **debugMaxDataSourceEntries** (integer) - Optional - Debug setting to limit the maximum number of data source entries to index.

### Request Example
```json
{
  "workerConfig": "string",
  "dataSource": {
    "type": "catalog",
    "data": {
      "displayMode": "All"
    }
  },
  "openSessionArguments": [
    {
      "first": "string",
      "second": "string"
    }
  ],
  "callbackUrl": "http://example.com",
  "clientId": "string",
  "accessToken": "string",
  "refreshToken": "string",
  "priceDate": "2025-01-31",
  "languageList": [
    "string"
  ],
  "disable": [
    "string"
  ],
  "geometryExportSettings": {
    "export key1": [
      "string"
    ],
    "export key2": [
      "string"
    ]
  },
  "imageExportSettings": {
    "rendering key1": [
      "string"
    ],
    "rendering key2": [
      "string"
    ]
  },
  "basketWorkers": 0,
  "hooks": [
    {
      "type": "ItemAppData",
      "data": {
        "mode": "xml",
        "appKey": "string",
        "value": "string"
      }
    }
  ],
  "debugFilterCatalogIds": [
    "string"
  ],
  "debugIndexOnlyEveryNthArticle": 0,
  "debugMaxDataSourceEntries": 0
}
```

### Response
#### Success Response (200)
- **jobId** (string) - Required - The unique ID assigned to a crawling job.

#### Response Example
```json
{
  "jobId": "string"
}
```

### Callbacks
- **POST Progress notification**: Receives progress updates during the crawling process.
```

--------------------------------

### Start Crawling Job Response (JSON)

Source: https://docs.pcon-solutions.com/eaiws/plugins/crawler/1.8/index

Upon successfully starting a crawling job, the API returns a JSON object containing a unique `jobId` that can be used to track the job's progress or cancel it.

```json
{
  "jobId": "string"
}
```

--------------------------------

### GET /websites/pcon-solutions_eaiws_plugins_crawler_1_8/article_price_information

Source: https://docs.pcon-solutions.com/eaiws/plugins/crawler/1.8/index

Retrieves information about article prices stored within the OFML data. This includes purchase prices, sales prices, currency codes, weights, and discountability status.

```APIDOC
## GET /websites/pcon-solutions_eaiws_plugins_crawler_1_8/article_price_information

### Description
Retrieves information about article prices stored within the OFML data. This includes purchase prices, sales prices, currency codes, weights, and discountability status. All values are directly from the article data and are not adjusted by a calculation schema.

### Method
GET

### Endpoint
/websites/pcon-solutions_eaiws_plugins_crawler_1_8/article_price_information

### Parameters
#### Query Parameters
- **parentId** (string) - Required - The parent ID used in the `article` table.
- **distributionRegion** (string) - Required - The distribution region for which the prices are valid.

### Response
#### Success Response (200)
- **parentId** (string) - The same `parentId` that is used in the `article` table.
- **distributionRegion** (string) - The distribution region for which the prices are valid.
- **currencyCode** (string) - <ISO 4217> If the article position has a price, then this field contains the ISO 4217 currency code of the price.
- **pdPurchasePrice** (number) - <float> This field contains the purchase price for the current configuration of the article as determined by the product data. If no purchase price could be determined, the field is not available. The field will not be provided if the currencies of purchase price and sales price are different and the value of the purchase price is non-zero.
- **pdSalesPrice** (number) - <float> This field contains the sales price for the current configuration of the article as determined by the product data. If no sales price could be determined, the field is not available.
- **purchasePrice** (number) - <float> This field contains the purchase price as read from the product database, possibly adjusted by an OFML price profile. If no purchase price could be determined, the field is not available. The field will not be provided if the currencies of purchase price and sales price are different and the value of the purchase price is non-zero.
- **salesPrice** (number) - <float> This field contains the sales price as read from the product database, possibly adjusted by an OFML price profile. If no sales price could be determined, the field is not available.
- **salesUnitGrossWeight** (number) - <float> The gross weight of the sales unit.
- **salesUnitNetWeight** (number) - <float> The net weight of the sales unit.
- **salesUnitSize** (number) - <float> The size of the sales unit.
- **salesUnitVolume** (number) - <float> The volume of the sales unit.
- **isDiscountable** (integer) - [ 0 .. 1 ] If the value is set to `1`, the item is discountable.

#### Response Example
```json
{
  "parentId": "ARTICLE123",
  "distributionRegion": "US",
  "currencyCode": "USD",
  "pdPurchasePrice": 100.50,
  "pdSalesPrice": 150.75,
  "purchasePrice": 95.00,
  "salesPrice": 145.00,
  "salesUnitGrossWeight": 2.5,
  "salesUnitNetWeight": 2.0,
  "salesUnitSize": 10.0,
  "salesUnitVolume": 5.0,
  "isDiscountable": 1
}
```
```

--------------------------------

### Get Crawling Progress

Source: https://docs.pcon-solutions.com/eaiws/plugins/crawler/1.8/index

Retrieves the current progress of a specified crawling job.

```APIDOC
## GET /EAIWS/plugins/Crawler/progress/{jobId}

### Description
Retrieves the current progress of a specified crawling job.

### Method
GET

### Endpoint
/EAIWS/plugins/Crawler/progress/{jobId}

### Parameters
#### Path Parameters
- **jobId** (any) - Required - The unique ID assigned to the crawling job.

### Response
#### Success Response (200)
- Returns the current crawling progress. The exact schema for the progress details is not provided in the input but is expected to be a JSON object.

#### Response Example
(Example response structure not provided in the input text)
```

--------------------------------

### Start Crawling Job Request Payload (JSON)

Source: https://docs.pcon-solutions.com/eaiws/plugins/crawler/1.8/index

This JSON payload is used to initiate a crawling job. It includes configuration for the worker, data source, callback URL, authentication details, and various debugging and export settings. The `debugMaxDataSourceEntries` field can limit the number of articles indexed.

```json
{
  "workerConfig": "string",
  "dataSource": {
    "type": "catalog",
    "data": {
      "displayMode": "All"
    }
  },
  "openSessionArguments": [
    {
      "first": "string",
      "second": "string"
    }
  ],
  "callbackUrl": "http://example.com",
  "clientId": "string",
  "accessToken": "string",
  "refreshToken": "string",
  "priceDate": "2025-01-31",
  "languageList": [
    "string"
  ],
  "disable": [
    "string"
  ],
  "geometryExportSettings": {
    "export key1": [
      "string"
    ],
    "export key2": [
      "string"
    ]
  },
  "imageExportSettings": {
    "rendering key1": [
      "string"
    ],
    "rendering key2": [
      "string"
    ]
  },
  "basketWorkers": 0,
  "hooks": [
    {
      "type": "ItemAppData",
      "data": {
        "mode": "xml",
        "appKey": "string",
        "value": "string"
      }
    }
  ],
  "debugFilterCatalogIds": [
    "string"
  ],
  "debugIndexOnlyEveryNthArticle": 0,
  "debugMaxDataSourceEntries": 0
}
```

--------------------------------

### Get Crawling Results

Source: https://docs.pcon-solutions.com/eaiws/plugins/crawler/1.8/index

Retrieves the actual crawling result for a given job ID. The crawling must be finished before the database can be retrieved.

```APIDOC
## GET /EAIWS/plugins/Crawler/download/{jobId}

### Description
This request can be used to retrieve the actual crawling result. The crawling must be finished before the database can be retrieved.

### Method
GET

### Endpoint
/EAIWS/plugins/Crawler/download/{jobId}

### Parameters
#### Path Parameters
- **jobId** (any) - Required - The unique id assigned to a crawling job.

### Response
#### Success Response (200)
- **Binary data** (application/octet-stream) - Binary data of the SQLite database file.

#### Response Example
(Binary data of SQLite database file)
```

--------------------------------

### EAIWS Crawler Plugin Worker Configuration (XML)

Source: https://docs.pcon-solutions.com/eaiws/plugins/crawler/1.8/index

This XML configuration file defines the enabled features and behavior for a specific crawling task. The filename is used to reference the configuration when starting a crawl. It follows the Java properties XML format.

```xml
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE properties SYSTEM "http://java.sun.com/dtd/properties.dtd">
<properties>
    <entry key="article">true</entry>
</properties>
```

--------------------------------

### Crawling Task Table Schema

Source: https://docs.pcon-solutions.com/eaiws/plugins/crawler/1.8/index

This JSON object defines the schema for the 'crawling_task' table within the journal database. It includes fields for tracking crawling tasks, such as unique job ID, filename, start and end times, final progress state, and a flag indicating if the associated database file has been deleted.

```json
{
  "id": "string",
  "filename": "string",
  "startTime": 0,
  "endTime": 0,
  "finaleProgressState": "string",
  "deleted": 1
}
```

--------------------------------

### Article Image API

Source: https://docs.pcon-solutions.com/eaiws/plugins/crawler/1.8/index

Access article renderings. This endpoint provides binary image data and metadata such as keys and rendering times.

```APIDOC
## GET /websites/pcon-solutions_eaiws_plugins_crawler_1_8/article_image

### Description
Retrieves article renderings, including binary image data and associated metadata.

### Method
GET

### Endpoint
/websites/pcon-solutions_eaiws_plugins_crawler_1_8/article_image

### Parameters
#### Query Parameters
- **parentId** (string) - Optional - The `parentId` of the article.
- **key** (string) - Optional - The key defined during the crawling request.
- **image** (string) - Optional - The binary data of the generated rendering.
- **renderTime** (integer) - Optional - Time in milliseconds required for the rendering.

### Response
#### Success Response (200)
- **parentId** (string) - The `parentId` of the article.
- **key** (string) - The key corresponding to the crawling request.
- **image** (string) - The binary data of the generated rendering.
- **renderTime** (integer) - Time in milliseconds for the rendering.

#### Response Example
{
  "parentId": "string",
  "key": "string",
  "image": "<binary>",
  "renderTime": 12345
}
```

--------------------------------

### pCon.ui Compatible Article List Format

Source: https://docs.pcon-solutions.com/eaiws/plugins/crawler/1.8/index

Defines the structure for an article list compatible with pCon.ui integration. It specifies required fields like manufacturer ID, base article number, series ID, and variant code for each article.

```json
[
  {
    "moc": "EG",
    "ban": 4520,
    "sid": "OFFICE2",
    "ovc": "string"
  }
]
```

--------------------------------

### Crawler Plugin Configuration

Source: https://docs.pcon-solutions.com/eaiws/plugins/crawler/1.8/index

This section details the configuration parameters for the Crawler plugin, including options for debugging, data processing, and logging.

```APIDOC
## Crawler Plugin Configuration

This section details the configuration parameters for the Crawler plugin.

### Parameters

- **debug_fapish_stat_dump** (boolean) - Optional - Default: `false` - Specifies whether all information accessible via `/proc/[pid]/stat` of the `fapish` process should be saved as a JSON dump. Requires `debug_fapish_stat` to be enabled.
- **post_processing_drop_tables** (Array of strings) - Optional - Allows deletion of specified tables from the database after crawling.

### Request Body Example

```json
{
  "basket_task_scheduler": "simple",
  "insert_ofml_option_no_complex_type": false,
  "use_new_fapish_for_each_article": false,
  "log_level": "Fatal",
  "log_stack_trace": false,
  "catalog_node": true,
  "catalog_node_resources": false,
  "catalog_node_text": false,
  "catalog_node_translation": false,
  "catalog_node_images": false,
  "catalog_node_images_parse_for_correct_dimensions": false,
  "catalog_package": false,
  "article": false,
  "article_obx": false,
  "article_export": false,
  "article_images": false,
  "article_price_information": false,
  "article_text": false,
  "article_copy_paste_test": false,
  "article_property": false,
  "article_property_value": false,
  "debug_fapish_stat": false,
  "debug_fapish_stat_dump": false,
  "post_processing_drop_tables": [
    "string"
  ]
}
```
```

--------------------------------

### EAIWS Crawler Plugin Global Configuration (XML)

Source: https://docs.pcon-solutions.com/eaiws/plugins/crawler/1.8/index

This XML configuration file sets global parameters that apply to all crawling tasks. It defines settings such as maximum workers, timeouts, and database management intervals. The file is located at etc/plugins/crawler/config.xml.

```xml
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE properties SYSTEM "http://java.sun.com/dtd/properties.dtd">
<properties>
    <entry key="maxBasketWorkersPerJob">8</entry>
    <entry key="maxGlobalConcurrentRunningBasketWorkers">32</entry>
</properties>
```

--------------------------------

### SQL Query for Uncreatable Articles

Source: https://docs.pcon-solutions.com/eaiws/plugins/crawler/1.8/index

This SQL query identifies entries in the 'catalog_node' table where the 'itemType' is 'Article' but no corresponding 'parentId' exists in the 'article' table, indicating failed article creation.

```sql
SELECT
    *
FROM
    catalog_node
WHERE
    itemType = 'Article' AND
    parentId NOT IN (
        SELECT parentId FROM article
    )

```

--------------------------------

### POST /websites/pcon-solutions_eaiws_plugins_crawler_1_8

Source: https://docs.pcon-solutions.com/eaiws/plugins/crawler/1.8/index

This endpoint allows for the configuration and initiation of a data crawling process. It accepts a detailed schema for defining the data source, session arguments, export settings, and hooks to customize the crawling behavior.

```APIDOC
## POST /websites/pcon-solutions_eaiws_plugins_crawler_1_8

### Description
Initiates a data crawling process with detailed configuration options for data sources, session arguments, export settings, and hooks.

### Method
POST

### Endpoint
/websites/pcon-solutions_eaiws_plugins_crawler_1_8

### Parameters
#### Request Body
- **workerConfig** (string) - Required - Name of the worker config to be used.
- **dataSource** (object) - Optional - Influences the data to be indexed. Can be one of:
  - Catalog based data source
  - OBK based data source
  - pCon.ui compatible articles list
  - Catalog node key based data source
  - Search based data source
  - Property permutation
  > Note: If no value is provided, the `Catalog based data source` is used.
  - **type** (string) - Required - Value: "catalog" - Name of the data source to be used.
  - **data** (object) - Optional
  - **displayMode** (string) - Optional - Default: "All" - Enum: "All", "Planning2D", "Planning3D", "Configuration", "CAD", "AllVisible", "AllVisibleBasket" - The DisplayMode used for the catalog.
- **openSessionArguments** (Array of objects) - Optional - Arguments for creating the EAIWS session.
  - **first** (string) - Optional
  - **second** (string) - Optional
- **callbackUrl** (string) - Optional - URI - URL to be called by the plugin after crawling finishes.
- **clientId** (string) - Optional - The `clientId` for obtaining package groups via pCon.update.
- **accessToken** (string) - Optional - A short-lived pCon.login access token.
- **refreshToken** (string) - Optional - A long-lived pCon.login refresh token.
- **priceDate** (string) - Optional - Date - The value to be set as the price date. Defaults to the current date.
- **languageList** (Array of strings) - Optional - ISO 639 alpha-2 language codes for exporting properties.
- **disable** (Array of strings) - Optional - List of features to disable that are enabled in the worker config.
- **geometryExportSettings** (object) - Optional - Mapping for generating CAD exports.
  - **[export key*additional property]** (Array of strings) - Optional - List of CAD export settings.
- **imageExportSettings** (object) - Optional - Mapping for rendering article images.
  - **[rendering key*additional property]** (Array of strings) - Optional - List of rendering settings.
- **basketWorkers** (integer) - Optional - Number of workers for processing article data. Cannot exceed the maximum value from the worker config.
- **hooks** (Array of objects) - Optional - Hooks to influence crawling behavior or extend stored data.
  - **type** (string) - Required - Value: "ItemAppData" or "DeterministicOBXIds"
  - **data** (object) - Required (if type is "ItemAppData")
    - **mode** (string) - Required - Enum: "xml", "text" - Determines how data is written.
    - **appKey** (string) - Required - The appKey for storing the value.
    - **value** (string) - Required - The value to be stored.
- **debugFilterCatalogIds** (Array of strings) - Optional - Debug function to filter catalog entries by catalog IDs.
- **debugIndexOnlyEveryNthArticle** (integer) - Optional - Debug function to limit the number of indexed articles.

### Request Example
```json
{
  "workerConfig": "default_worker_config",
  "dataSource": {
    "type": "catalog",
    "data": {},
    "displayMode": "All"
  },
  "openSessionArguments": [
    {
      "first": "arg1",
      "second": "arg2"
    }
  ],
  "callbackUrl": "https://example.com/callback",
  "clientId": "your_client_id",
  "accessToken": "your_access_token",
  "priceDate": "2023-10-27",
  "languageList": ["en", "de"],
  "disable": ["feature1", "feature2"],
  "geometryExportSettings": {
    "dwg_export": ["layer=0", "format=R2018"]
  },
  "imageExportSettings": {
    "png_render": ["resolution=300", "quality=90"]
  },
  "basketWorkers": 4,
  "hooks": [
    {
      "type": "ItemAppData",
      "data": {
        "mode": "xml",
        "appKey": "custom_data",
        "value": "<data>some_value</data>"
      }
    }
  ],
  "debugFilterCatalogIds": ["catalog123"],
  "debugIndexOnlyEveryNthArticle": 10
}
```

### Response
#### Success Response (200)
- **status** (string) - Indicates the status of the crawling process.
- **message** (string) - A message providing details about the crawling process.

#### Response Example
```json
{
  "status": "success",
  "message": "Crawling process initiated successfully."
}
```
```

--------------------------------

### Crawler Plugin Configuration Options

Source: https://docs.pcon-solutions.com/eaiws/plugins/crawler/1.8/index

This JSON object represents the configuration settings for the Crawler plugin. It includes options for task scheduling, logging, catalog and article processing, debugging flags like 'debug_fapish_stat_dump', and post-processing actions such as dropping tables.

```json
{
  "basket_task_scheduler": "simple",
  "insert_ofml_option_no_complex_type": false,
  "use_new_fapish_for_each_article": false,
  "log_level": "Fatal",
  "log_stack_trace": false,
  "catalog_node": true,
  "catalog_node_resources": false,
  "catalog_node_text": false,
  "catalog_node_translation": false,
  "catalog_node_images": false,
  "catalog_node_images_parse_for_correct_dimensions": false,
  "catalog_package": false,
  "article": false,
  "article_obx": false,
  "article_export": false,
  "article_images": false,
  "article_price_information": false,
  "article_text": false,
  "article_copy_paste_test": false,
  "article_property": false,
  "article_property_value": false,
  "debug_fapish_stat": false,
  "debug_fapish_stat_dump": false,
  "post_processing_drop_tables": [
    "string"
  ]
}
```

--------------------------------

### Article Data API

Source: https://docs.pcon-solutions.com/eaiws/plugins/crawler/1.8/index

Retrieve detailed information about articles generated from OFML data. This includes article identifiers, configuration details, and potential inconsistencies.

```APIDOC
## GET /websites/pcon-solutions_eaiws_plugins_crawler_1_8/article

### Description
Retrieves basic information about articles generated from OFML data.

### Method
GET

### Endpoint
/websites/pcon-solutions_eaiws_plugins_crawler_1_8/article

### Parameters
#### Query Parameters
- **parentId** (string) - Optional - A unique id generated by the data source.
- **packageName** (string) - Optional - The OFML name of the package containing the OFML article data.
- **progId** (string) - Optional - The OFML program id of the package.
- **distributionRegion** (string) - Optional - The OFML distribution region of the package.
- **baseArticleNumber** (string) - Optional - The base article number.
- **finalArticleNumber** (string) - Optional - The final article number.
- **ofmlVariantCode** (string) - Optional - The OFML variant code of the OFML article.
- **manufacturerId** (string) - Optional - The commercial identifier of the article’s manufacturer.
- **seriesId** (string) - Optional - The identifier for the article’s series.
- **hasInconsistency** (integer) - Optional - Indicates if EAIWS detected an inconsistency (0 or 1).
- **geometryChecksum** (string) - Optional - Checksum based on information relevant for CAD exports.
- **obx** (string) - Optional - The content of the exported OBX file (requires `article_obx` feature).
- **insertTime** (integer) - Optional - Time in milliseconds for article creation.
- **sessionId** (string) - Optional - The sessionId used for indexing this article (requires debug feature activation).

### Response
#### Success Response (200)
- **parentId** (string) - A unique id generated by the data source.
- **packageName** (string) - The OFML name of the package.
- **progId** (string) - The OFML program id of the package.
- **distributionRegion** (string) - The OFML distribution region of the package.
- **baseArticleNumber** (string) - The base article number.
- **finalArticleNumber** (string) - The final article number.
- **ofmlVariantCode** (string) - The OFML variant code.
- **manufacturerId** (string) - The commercial identifier of the article’s manufacturer.
- **seriesId** (string) - The identifier for the article’s series.
- **hasInconsistency** (integer) - Indicates inconsistency (0 or 1).
- **geometryChecksum** (string) - Checksum for CAD exports.
- **obx** (string) - The content of the exported OBX file.
- **insertTime** (integer) - Time in milliseconds for article creation.
- **sessionId** (string) - The sessionId used for indexing this article.

#### Response Example
{
  "parentId": "string",
  "packageName": "string",
  "progId": "string",
  "distributionRegion": "string",
  "baseArticleNumber": "string",
  "finalArticleNumber": "string",
  "ofmlVariantCode": "string",
  "manufacturerId": "string",
  "seriesId": "string",
  "hasInconsistency": 0,
  "geometryChecksum": "string",
  "obx": "<binary>",
  "insertTime": 12345,
  "sessionId": "string"
}
```

--------------------------------

### Article Property Schema

Source: https://docs.pcon-solutions.com/eaiws/plugins/crawler/1.8/index

Defines the schema for article properties, including their IDs, parent references, language, classification, type, display attributes, and current values.

```APIDOC
## GET /websites/pcon-solutions_eaiws_plugins_crawler_1_8/article_property

### Description
Retrieves a list of article properties.

### Method
GET

### Endpoint
/websites/pcon-solutions_eaiws_plugins_crawler_1_8/article_property

### Parameters
#### Query Parameters
- **id** (integer) - Optional - A unique id for this entry.
- **parentId** (string) - Optional - The same `parentId` that is used in the `article` table.
- **languageTag** (string) - Optional - The language used while processing the data.
- **class** (string) - Optional - The property class is a symbolic name used to classify properties.
- **name** (string) - Optional - The property name is the symbolic name of the property.

### Response
#### Success Response (200)
- **id** (integer) - A unique id for this entry. This value is used for mapping between this and the `article_property_value` table.
- **parentId** (string) - The same `parentId` that is used in the `article` table.
- **languageTag** (string) - The language used while processing the data.
- **class** (string) - The property class is a symbolic name used to classify properties. Property class and property name uniquely identify the property with respect to the article. The property class names stored in this field correspond to the property class name stored in the name field of the PropertyClass structure.
- **classText** (string) - The property classText contains the textual description of the property class.
- **name** (string) - The property name is the symbolic name of the property. The property name is unique relative to the property class, but there is no guarantee that no two properties of an article have the same property name.
- **type** (string) - This is the fundamental type of the property. More detailed information can be found in the EAIWS spec.
- **text** (string) - The property text is a short human readable description of the property, suitable for use in a tabular property editor. The language of the text depends on the configuration of the current session and the languages available in the product data.
- **width** (integer) - This is the maximum number of characters for properties of type Character. For other property types the value of this field is undefined and should not be used. The value of this field will never change for a particular property.
- **digits** (integer) - This is the maximum number of significant decimal digits for properties of type Numeric and Length. For other property types the value of this field is undefined and should not be used. The value of this field will never change for a particular property.
- **decDigits** (integer) - For properties of type Number and Length, this is the number of decimal digits right of the decimal point (the precision). For other property types the value of this field is undefined and should not be used. The value of this field will never change for a particular property.
- **visible** (integer) - [ 0 .. 1 ] This field indicates whether the property should be displayed by a property editor. The value of this field may change depending on the current configuration.
- **editable** (integer) - [ 0 .. 1 ] This field indicates whether the property may be changed by the client. It should be ignored for invisible properties. The value of this field may change depending on the current configuration. The field editable is always true if field visible is false. The OFML API does not allow to differentiate between editable and read-only invisible properties, and EAIWS does not prevent the client from setting the value of an invisible property, so the value true is more appropriate.
- **addValues** (integer) - [ 0 .. 1 ] This field indicates whether the property may assume values (and may be set by the client to a value) other then the values found in the choice list of the property. For numeric and length properties, the restrictions imposed by possible intervals are not affected by the value of this field.
- **choiceList** (integer) - [ 0 .. 1 ] This field indicates whether a property has a choice list. Unless the field addValues is `1` a choice list restricts the allowed property values to the elements of the choice list. The value of this field may change depending on the current configuration.
- **valueValue** (string) - This field contains information about the current internal value of the property.
- **valueText** (string) - This field contains information about the current human readable value of the property.
- **valueSmallIcon** (string) - <binary> If the property has a choice list, and a small icon is available for the current property value, then this field contains the image for the small icon.
- **valueLargeIcon** (string) - <binary> If the property has a choice list, and a large icon is available for the current property value, then this field contains the image for the large icon.

#### Response Example
```json
[
  {
    "id": 123,
    "parentId": "article-456",
    "languageTag": "en-US",
    "class": "color",
    "classText": "Color",
    "name": "primaryColor",
    "type": "String",
    "text": "Primary Color",
    "width": 50,
    "digits": 0,
    "decDigits": 0,
    "visible": 1,
    "editable": 1,
    "addValues": 0,
    "choiceList": 1,
    "valueValue": "#FF0000",
    "valueText": "Red",
    "valueSmallIcon": "/icons/red_small.png",
    "valueLargeIcon": "/icons/red_large.png"
  }
]
```
```

--------------------------------

### Retrieve Crawling Results API Endpoint

Source: https://docs.pcon-solutions.com/eaiws/plugins/crawler/1.8/index

This API endpoint allows users to download the actual crawling results. The crawling process must be completed before the database can be retrieved. It requires a jobId as a path parameter.

```http
GET /EAIWS/plugins/Crawler/download/{jobId}
```

--------------------------------

### Article Export API

Source: https://docs.pcon-solutions.com/eaiws/plugins/crawler/1.8/index

Retrieve CAD exports for articles. This endpoint provides access to binary data of generated exports, along with metadata like keys and export times.

```APIDOC
## GET /websites/pcon-solutions_eaiws_plugins_crawler_1_8/article_export

### Description
Retrieves CAD exports for articles, including binary data and associated metadata.

### Method
GET

### Endpoint
/websites/pcon-solutions_eaiws_plugins_crawler_1_8/article_export

### Parameters
#### Query Parameters
- **parentId** (string) - Optional - The `parentId` of the article.
- **key** (string) - Optional - The key defined during the crawling request.
- **data** (string) - Optional - The binary data of the generated export.
- **exportTime** (integer) - Optional - Time in milliseconds required for the export.

### Response
#### Success Response (200)
- **parentId** (string) - The `parentId` of the article.
- **key** (string) - The key corresponding to the crawling request.
- **data** (string) - The binary data of the generated export.
- **exportTime** (integer) - Time in milliseconds for the export.

#### Response Example
{
  "parentId": "string",
  "key": "string",
  "data": "<binary>",
  "exportTime": 12345
}
```

--------------------------------

### Article Copy-Paste Test API

Source: https://docs.pcon-solutions.com/eaiws/plugins/crawler/1.8/index

Access the results of the copy-paste test for articles. This helps identify errors in OFML data by exporting an article as OBX and then creating a new article from it.

```APIDOC
## GET /websites/pcon-solutions_eaiws_plugins_crawler_1_8/article_copy_paste_test

### Description
Retrieves the results of the copy-paste test for articles, indicating if an error occurred during article creation from OBX data.

### Method
GET

### Endpoint
/websites/pcon-solutions_eaiws_plugins_crawler_1_8/article_copy_paste_test

### Parameters
#### Query Parameters
- **parentId** (string) - Optional - The same `parentId` that is used in the `article` table.
- **failed** (integer) - Optional - If 1, the OFML article data contains an error preventing creation from OBX.

### Response
#### Success Response (200)
- **parentId** (string) - The `parentId` of the article.
- **failed** (integer) - Indicates if the copy-paste test failed (1) or succeeded (0).

#### Response Example
{
  "parentId": "string",
  "failed": 0
}
```

--------------------------------

### Crawler Status Response Schema

Source: https://docs.pcon-solutions.com/eaiws/plugins/crawler/1.8/index

Schema for the response indicating the status of a crawling task.

```APIDOC
## Response Schema: application/json

### Description
Schema for the response indicating the status of a crawling task.

### Fields
- **finished** (boolean) - Required - If set to `true` all crawling tasks including post processing are finished. Note: `true` only indicates that the crawling task was finished, not necessarily successful. Evaluate the `cancelled` property as well.
- **cancelled** (boolean) - Required - If set to `true`, the crawling was cancelled and the database is most likely incomplete.
- **cancelReason** (string) - A message describing why the crawling was cancelled.
- **counting** (boolean) - Required - Only when this is set to `true`, the final values for the `total` property of the `items` entries are determined. Before this is the case, these can change at any time or new items can be added.
- **items** (Array of objects) - Required - This list contains a detailed overview of the found entries and their types as well as status information about their processing.
  - **type** (string) - Required - The type of the entry. For example, article or folder.
  - **processed** (string) - Required - The number of the already processed entries.
  - **total** (string) - Required - The total number of entries found.

### Response Example (200)
```json
{
  "finished": true,
  "cancelled": true,
  "cancelReason": "string",
  "counting": true,
  "items": [
    {
      "type": "string",
      "processed": "string",
      "total": "string"
    }
  ]
}
```
```

--------------------------------

### Cancel Crawling Process

Source: https://docs.pcon-solutions.com/eaiws/plugins/crawler/1.8/index

Cancels a currently running crawling job. This endpoint can optionally delete the database and control whether the response is returned before or after the cancellation process completes.

```APIDOC
## POST /EAIWS/plugins/Crawler/cancel/{jobId}

### Description
Cancels a running crawling job. The database can optionally be deleted, and the response can be returned non-blockingly.

### Method
POST

### Endpoint
/EAIWS/plugins/Crawler/cancel/{jobId}

### Parameters
#### Path Parameters
- **jobId** (any) - Required - The unique ID assigned to the crawling job to be cancelled.

#### Request Body
- **deleteDatabase** (boolean) - Optional - If set to `true`, the database file will be deleted immediately.
- **nonBlocking** (boolean) - Optional - If set to `true`, the request is responded to before the crawling process and all associated threads or workers have completed their execution.

### Request Example
```json
{
  "deleteDatabase": true,
  "nonBlocking": true
}
```

### Response
#### Success Response (200)
- **finished** (boolean) - Required - If set to `true`, all crawling tasks including post-processing are finished. Note: This indicates completion, not necessarily success.
- **cancelled** (boolean) - Required - If set to `true`, the crawling was cancelled and the database is likely incomplete.
- **cancelReason** (string) - Optional - A message describing why the crawling was cancelled.
- **counting** (boolean) - Required - If set to `true`, the final values for the `total` property of the `items` entries are determined. Before this is the case, these can change or new items can be added.
- **items** (array) - Required - A list containing a detailed overview of found entries and their types, as well as status information about their processing.
  - **type** (string) - Required - The type of the entry (e.g., "article" or "folder").
  - **processed** (string) - Required - The number of already processed entries.
  - **total** (string) - Required - The total number of entries found.

#### Response Example
```json
{
  "finished": true,
  "cancelled": true,
  "cancelReason": "string",
  "counting": true,
  "items": [
    {
      "type": "string",
      "processed": "string",
      "total": "string"
    }
  ]
}
```
```

--------------------------------

### Cancel Crawling Job Response (JSON)

Source: https://docs.pcon-solutions.com/eaiws/plugins/crawler/1.8/index

After a crawling job is cancelled, this JSON response indicates the status of the cancellation. It includes whether the process has finished (`finished`), if it was indeed cancelled (`cancelled`), the reason for cancellation (`cancelReason`), and details about the processed items.

```json
{
  "finished": true,
  "cancelled": true,
  "cancelReason": "string",
  "counting": true,
  "items": [
    {
      "type": "string",
      "processed": "string",
      "total": "string"
    }
  ]
}
```

--------------------------------

### Progress Notification Callback Payload (JSON)

Source: https://docs.pcon-solutions.com/eaiws/plugins/crawler/1.8/index

This JSON payload is sent as a callback to notify about the progress of a crawling job. It includes the `jobId` and a boolean indicating whether the job was `successful`.

```json
{
  "jobId": "string",
  "successful": true
}
```

--------------------------------

### Cancel Crawling Job Request Payload (JSON)

Source: https://docs.pcon-solutions.com/eaiws/plugins/crawler/1.8/index

This JSON payload is used to cancel an ongoing crawling job. It allows specifying whether to delete the database (`deleteDatabase`) and if the response should be non-blocking (`nonBlocking`).

```json
{
  "deleteDatabase": true,
  "nonBlocking": true
}
```
Tokens:
Raw