### Print Installed Version
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/configuration.md
Displays the currently installed version of the Percollate tool.
```bash
percollate --version
```
--------------------------------
### CLI Invocation Example
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/architecture.md
Demonstrates how to invoke the Percollate CLI with multiple operands and an output file option.
```bash
percollate pdf url1 url2 url3 --output bundle.pdf
```
--------------------------------
### Install Project Dependencies
Source: https://github.com/danburzo/percollate/blob/main/CONTRIBUTING.md
Install the necessary Node.js dependencies for the Percollate project.
```bash
npm install
```
--------------------------------
### Nunjucks Template Example
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/architecture.md
An example of Nunjucks template syntax for rendering content. It demonstrates how to access context variables like title, author, and date, and how to loop through items.
```nunjucks
{{ title }}
{{ author }}
{{ date | humandate }}
{% for item in items %}
{{ item.content }}
{% endfor %}
```
--------------------------------
### Run Percollate CLI Example
Source: https://github.com/danburzo/percollate/blob/main/CONTRIBUTING.md
Execute the Percollate CLI to generate a PDF from a URL. Ensure the script has execution permissions.
```bash
./cli.js pdf --output some.pdf http://example.com
```
--------------------------------
### Generate PDF with Output Path
Source: https://github.com/danburzo/percollate/blob/main/README.md
Example of generating a PDF file from a URL and specifying the output file name.
```bash
percollate pdf https://example.com -o my-example.pdf
```
--------------------------------
### API: EPUB with Inline Images
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/README.md
API usage example for generating a self-contained EPUB with all images inlined.
```javascript
import { configure, epub } from 'percollate';
configure();
await epub(['https://example.com'], {
output: 'article.epub',
inline: true // Self-contained
});
```
--------------------------------
### Install Percollate from AUR
Source: https://github.com/danburzo/percollate/blob/main/README.md
Install the community-maintained package for Percollate from the Arch User Repository using an AUR helper.
```bash
yay -S nodejs-percollate
```
--------------------------------
### Install Percollate Globally
Source: https://github.com/danburzo/percollate/blob/main/README.md
Install the Percollate command-line tool globally using npm. Ensure you have Node.js 14.17.0 or later.
```bash
npm install -g percollate
```
--------------------------------
### Feed Processing Integration Example
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/api-feeds.md
Demonstrates how to use `isFeed` and `processFeed` to parse RSS feed XML. It then processes each entry, creating a JSDOM for each item and cleaning it up.
```javascript
import { isFeed, processFeed } from './src/feeds.js';
const dom = new JSDOM(feedXmlBuffer, {
contentType: 'text/xml',
url: feedUrl
});
if (isFeed(dom.window.document)) {
const entries = processFeed(dom.window.document);
// Process each entry as a separate article
const items = await Promise.all(
entries.map(entry => {
const itemDOM = new JSDOM(wrapHTMLFragment(entry), {
url: entry.url
});
return cleanupItem(itemDOM, options);
})
);
}
```
--------------------------------
### API: Generate PDF
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/README.md
API usage example for generating a PDF from a list of URLs with custom output, title, and author options.
```javascript
import { configure, pdf } from 'percollate';
configure();
const result = await pdf(['https://example.com'], {
output: 'page.pdf',
title: 'My Page',
author: 'Jane Doe'
});
```
--------------------------------
### Basic EPUB Conversion
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/api-main.md
Converts a single URL to an EPUB file with a specified title. Ensure 'percollate' is installed and configured.
```javascript
import { configure, epub } from 'percolate';
configure();
// Basic EPUB
await epub(['https://example.com'], {
output: 'book.epub',
title: 'My Book'
});
```
--------------------------------
### HTML Conversion to File
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/api-main.md
Converts a web page URL to an HTML file. Ensure 'percollate' is installed and configured.
```javascript
import { configure, html } from 'percollate';
configure();
// Save to file
await html(['https://example.com'], {
output: 'page.html'
});
```
--------------------------------
### API: Custom Styling for PDF
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/README.md
API usage example for generating a PDF with custom page size, margins, text alignment, and external CSS/template files.
```javascript
await pdf(['https://example.com'], {
output: 'custom.pdf',
css: '
@page { size: A3 landscape; margin: 1cm; }
.article__content p { text-align: left; }
',
template: './my-template.html',
style: './my-styles.css'
});
```
--------------------------------
### Set Puppeteer Product Before Installation
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/configuration.md
Configures the browser product (Chrome or Firefox) that Puppeteer will use before installing Percollate. This determines which browser binaries are downloaded.
```bash
PUPPETEER_PRODUCT=firefox npm install percollate
```
--------------------------------
### HTML Conversion to Stdout
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/api-main.md
Converts a web page URL to HTML and outputs it to standard output. Ensure 'percollate' is installed and configured.
```javascript
// Output to stdout
await html(['https://example.com'], {
output: '-'
});
```
--------------------------------
### Style Header/Footer Templates
Source: https://github.com/danburzo/percollate/blob/main/README.md
Example of applying CSS styles to a footer template. Styles for header/footer templates do not inherit from the main page.
```css
.footer-template {
font-size: 10pt;
font-weight: bold;
}
```
--------------------------------
### Basic Markdown Conversion
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/api-main.md
Converts a web page URL to a Markdown file. Ensure 'percollate' is installed and configured.
```javascript
import { configure, md } from 'percollate';
configure();
// Basic Markdown output
await md(['https://example.com'], {
output: 'article.md'
});
```
--------------------------------
### Custom Markdown Formatting
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/api-main.md
Converts a web page to Markdown with custom formatting options for emphasis and strong text. Ensure 'percollate' is installed and configured.
```javascript
// Custom Markdown formatting
await md(['https://example.com'], {
output: 'custom.md',
'md.emphasis': 'asterisk',
'md.strong': 'asterisk'
});
```
--------------------------------
### EPUB Bundling Process
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/architecture.md
Describes the steps involved in creating an EPUB file using the epubgen library and an archiver for zip structure.
```javascript
items[] → bundleEpub()
↓
epubgen(data, output_path, options)
↓
archiver('zip') → EPUB structure:
mimetype
META-INF/
container.xml
com.apple.ibooks.display-options.xml
OEBPS/
content.opf (metadata)
nav.xhtml (navigation)
toc.ncx (table of contents)
style.css
{item.id}.xhtml (articles)
cover.png (if enabled)
rr-{uuid}.* (remote resources if bundled)
↓
fs.createWriteStream(output_path)
```
--------------------------------
### configure()
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/api-main.md
Initializes the percollate environment by setting up the Nunjucks templating engine with custom filters. This function must be called once before any bundle operations.
```APIDOC
## Function: `configure()`
### Description
Initializes the percollate environment by setting up the Nunjucks templating engine with custom filters. This function must be called once before any bundle operations.
### Returns
`void`
### Example
```javascript
import { configure, pdf } from 'percollate';
configure();
await pdf(['https://example.com'], { output: 'example.pdf' });
```
```
--------------------------------
### Main Library API Functions
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/README.md
Functions for initializing percollate and generating outputs in various formats. Use `configure()` to initialize before calling output generation functions like `pdf()`, `epub()`, `html()`, or `md()`.
```javascript
// Initialize percollate
configure(): void
// Generate outputs
async pdf(urls, options): Promise<{items, options} | null>
async epub(urls, options): Promise<{items, options} | null>
async html(urls, options): Promise<{items, options} | null>
async md(urls, options): Promise<{items, options} | null>
```
--------------------------------
### Set Bundle Title
Source: https://github.com/danburzo/percollate/blob/main/README.md
Generate an EPUB file from multiple URLs and provide a custom title for the bundle.
```bash
percollate epub http://example.com/page-1 http://example.com/page-2 --title="Best Of Example"
```
--------------------------------
### Show Percollate Help Text
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/configuration.md
Displays all available commands and options for the Percollate CLI. Use this to understand the full range of functionalities.
```bash
percollate --help
```
--------------------------------
### Importing All Main Percollate Functions
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/exports.md
Imports and demonstrates the usage of core Percollate functions for configuration and content conversion.
```javascript
import {
configure,
pdf,
epub,
html,
md
} from 'percolate';
configure();
```
--------------------------------
### Markdown Conversion to Stdout
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/api-main.md
Converts a web page URL to Markdown and outputs it to standard output. Ensure 'percollate' is installed and configured.
```javascript
// To stdout
await md(['https://example.com'], {
output: '-'
});
```
--------------------------------
### Convert Web Pages to PDF (Basic)
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/api-main.md
Converts a single web page URL to a PDF file. Ensure configure() has been called prior to this operation.
```javascript
import { configure, pdf } from 'percollate';
configure();
// Basic usage
await pdf(['https://example.com'], {
output: 'example.pdf'
});
```
--------------------------------
### Convert Single Web Page to PDF
Source: https://github.com/danburzo/percollate/blob/main/README.md
Basic command to convert a single web page to a PDF file.
```bash
percollate pdf --output=some.pdf https://example.com
```
--------------------------------
### outputPath(items, options, ext, cache?)
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/api-utilities.md
Generates an output file path for a bundle based on provided items, options, extension, and an optional cache. It handles different output scenarios like individual files or a single bundle.
```APIDOC
## Function: `outputPath(items, options, ext, cache?)`
Generates an output file path for the bundle.
### Parameters
| Parameter | Type | Description |
|-----------|------|-------------|
| items | `ParsedItem[]` | Array of parsed items (at least one) |
| options | object | Must have `output` and `individual` properties |
| ext | `string` | File extension (e.g., `.pdf`, `.epub`) |
| cache | object | Slug cache to prevent filename collisions |
### Returns
File path string based on:
1. If `individual` and `output`: `{output}-{slugified-title}{ext}`
2. If `output`: `{output}` (as-is)
3. If not individual and multiple items: `percollate-{timestamp}{ext}`
4. If single item: `{slugified-title}{ext}`
### Example
```javascript
import outputPath from './src/util/output-path.js';
const items = [{ title: 'My Article' }];
// With explicit output
outputPath(items, {output: 'custom.pdf'}, '.pdf');
// 'custom.pdf'
// Individual files
outputPath(items, {output: 'output.pdf', individual: true}, '.pdf', {});
// 'output-my-article.pdf'
// Auto-generated
outputPath(items, {}, '.pdf');
// 'my-article.pdf'
```
```
--------------------------------
### Language Utilities Exports
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/exports.md
Language utility functions from `src/util/language.js` for converting text to ISO 639-1 codes and getting language attributes.
```APIDOC
## Language Utilities Exports
#### `textToIso6391(text: string)`
- **Type**: function
- **Parameters**: `text` (string)
- **Returns**: `string | null`
#### `getLanguageAttribute(doc: Document)`
- **Type**: function
- **Parameters**: `doc` (Document)
- **Returns**: `string | null`
```
--------------------------------
### Run ESLint for Code Style
Source: https://github.com/danburzo/percollate/blob/main/CONTRIBUTING.md
Execute ESLint to enforce coding style rules.
```bash
npm run lint
```
--------------------------------
### Use Percollate PDF API with Options
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/configuration.md
Demonstrates how to use the Percollate PDF function as a library, passing configuration options as the second argument. This allows programmatic control over output, metadata, and features like table of contents.
```javascript
import { pdf } from 'percollate';
const result = await pdf(['https://example.com'], {
output: 'my.pdf',
title: 'My Document',
author: 'Jane Doe',
toc: true,
css: '@page { size: A4; }',
debug: true
});
```
--------------------------------
### Self-Contained HTML with Embedded Images
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/api-main.md
Creates a self-contained HTML file with images embedded as base64 data URLs. Ensure 'percollate' is installed and configured.
```javascript
// Self-contained HTML with embedded images
await html(['https://example.com'], {
output: 'standalone.html',
inline: true
});
```
--------------------------------
### epub(urls, options)
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/api-main.md
Converts one or more web pages into an EPUB file. It accepts an array of URLs or file paths and an optional options object for customization. The function returns a promise that resolves to an object containing the parsed items and the options used, or null if no valid URLs were provided.
```APIDOC
## Function: `epub(urls, options)`
Converts one or more web pages into an EPUB file.
### Parameters
| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| urls | `string[]` or `stream.Readable[]` | Yes | — | Array of URLs, file paths, or stdin streams |
| options | `EPUBOptions` | No | `{}` | Configuration object for EPUB generation |
### Options Object (EPUBOptions)
Inherits all options from PDFOptions, except:
- `browser`: Ignored for EPUB (Puppeteer used only for cover image)
- `hyphenate`: defaults to `false` (not `true`)
Additional EPUB-specific options:
| Key | Type | Default | Description |
|-----|------|---------|-------------|
| inline | `boolean` | false | Embed images as base64 data URLs instead of fetching them during reading |
| xhtml | `boolean` | true | Serialize content as XHTML (required for EPUB spec compliance) |
| mapRemoteResources | `boolean` | !inline | Bundle external images with EPUB; disabled if inline is true |
### Returns
Promise resolving to an object containing:
- `items`: array of `ParsedItem` objects
- `options`: the final options object used
- Returns `null` if no valid URLs were provided
### Example
```javascript
import { configure, epub } from 'percollate';
configure();
// Basic EPUB
await epub(['https://example.com'], {
output: 'book.epub',
title: 'My Book'
});
// Multiple articles
await epub(
[
'https://example.com/article1',
'https://example.com/article2'
],
{
output: 'articles.epub',
title: 'Article Collection',
author: 'Editor'
}
);
// Self-contained EPUB with embedded images
await epub(['https://example.com'], {
output: 'offline.epub',
inline: true
});
```
```
--------------------------------
### Get Language Attribute from Document
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/api-utilities.md
Retrieves the language code specified in the HTML document's `lang` attribute. Returns null if the attribute is not present.
```javascript
import { getLanguageAttribute } from './src/util/language.js';
import { JSDOM } from 'jsdom';
const dom = new JSDOM('
');
getLanguageAttribute(dom.window.document); // 'en'
const dom2 = new JSDOM('');
getLanguageAttribute(dom2.window.document); // null
```
--------------------------------
### Image Fetching in EPUB Archiving
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/architecture.md
Demonstrates the sequential fetching and appending of remote resources (images) to an EPUB archive. Each image is fetched and then added to the archiver with its mapped name.
```javascript
for (let resource of remoteResources) {
fetch(resource.original)
.then(body => archiver.append(body, {name: resource.mapped}))
}
```
--------------------------------
### EPUB Conversion of Multiple Articles
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/api-main.md
Converts multiple article URLs into a single EPUB file, specifying title and author. Ensure 'percollate' is installed and configured.
```javascript
// Multiple articles
await epub(
[
'https://example.com/article1',
'https://example.com/article2'
],
{
output: 'articles.epub',
title: 'Article Collection',
author: 'Editor'
}
);
```
--------------------------------
### Self-Contained EPUB with Embedded Images
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/api-main.md
Creates a self-contained EPUB where images are embedded as base64 data URLs. This is useful for offline reading. Ensure 'percollate' is installed and configured.
```javascript
// Self-contained EPUB with embedded images
await epub(['https://example.com'], {
output: 'offline.epub',
inline: true
});
```
--------------------------------
### EPUB Generation Options
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/types.md
Configuration object for EPUB generation. Extends PDFOptions and includes specific fields for XHTML serialization and remote resource mapping.
```typescript
{
xhtml?: boolean;
mapRemoteResources?: boolean;
// ... all PDFOptions fields
}
```
--------------------------------
### Get File Extension from MIME Type
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/api-utilities.md
Converts a MIME type string into its corresponding file extension. Useful for determining the file format based on its MIME type.
```javascript
import { extForMimetype } from './src/util/file-mimetype.js';
extForMimetype('image/png'); // '.png'
extForMimetype('image/jpeg'); // '.jpg'
extForMimetype('image/webp'); // '.webp'
```
--------------------------------
### html(urls, options)
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/api-main.md
Converts one or more web pages into a single HTML file. It takes an array of URLs or file paths and an optional configuration object. The function returns a promise that resolves to an object containing parsed items and options, or null if no valid URLs are supplied.
```APIDOC
## Function: `html(urls, options)`
Converts one or more web pages into a single HTML file.
```javascript
async function html(
urls: string[] | Readable[],
options?: HTMLOptions
): Promise<{items: ParsedItem[], options: HTMLOptions} | null>
```
### Parameters
| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| urls | `string[]` or `stream.Readable[]` | Yes | — | Array of URLs, file paths, or stdin streams |
| options | `HTMLOptions` | No | `{}` | Configuration object for HTML generation |
### Options Object (HTMLOptions)
Same as PDFOptions, except:
- `hyphenate`: defaults to `false`
- `browser`: Ignored (not used for HTML)
- `output`: Can be `-` to write to stdout
- `sandbox`: Ignored (not used for HTML)
### Returns
Promise resolving to an object containing:
- `items`: array of `ParsedItem` objects
- `options`: the final options object used
- Returns `null` if no valid URLs were provided
### Example
```javascript
import { configure, html } from 'percollate';
configure();
// Save to file
await html(['https://example.com'], {
output: 'page.html'
});
// Output to stdout
await html(['https://example.com'], {
output: '-'
});
// Self-contained HTML with embedded images
await html(['https://example.com'], {
output: 'standalone.html',
inline: true
});
```
```
--------------------------------
### Override Default Fonts with --css
Source: https://github.com/danburzo/percollate/blob/main/README.md
Overrides the default font stacks for main and alternative fonts using the --css option. Ensure fonts are installed locally.
```bash
percollate pdf --css ":root { --main-font: 'PT Serif'; --alt-font: Roboto; }" http://example.com
```
--------------------------------
### Bundle Multiple Web Pages into a Single PDF
Source: https://github.com/danburzo/percollate/blob/main/README.md
Command to bundle several web pages into one PDF document.
```bash
percollate pdf --output=some.pdf https://example.com/page1 https://example.com/page2
```
--------------------------------
### addExif
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/api-other-modules.md
Embeds metadata (Title, Author) into a PDF file. It loads the PDF, sets the specified metadata, and returns the modified PDF data as a Uint8Array. Existing metadata is not updated; it starts fresh.
```APIDOC
## Function: addExif
### Description
Embeds metadata (Title, Author) into a PDF file. It loads the PDF, sets the specified metadata, and returns the modified PDF data as a Uint8Array. Existing metadata is not updated; it starts fresh.
### Parameters
- **existingPdfBytes** (`Buffer`) - Required - PDF file data as Buffer
- **metaData** (`object`) - Optional - Metadata to embed; keys: `Title`, `Author`
### Returns
- `Promise` - Resolves to a `Uint8Array` containing the modified PDF data (with metadata embedded).
### Example
```javascript
import { readFileSync, writeFileSync } from 'fs';
import addExif from './src/exif.js';
const pdfBuffer = readFileSync('input.pdf');
const newPdf = await addExif(pdfBuffer, {
Title: 'My Document',
Author: 'Jane Doe'
});
writeFileSync('output.pdf', newPdf);
```
### Source
`src/exif.js:8–19`
```
--------------------------------
### Bundle URLs from a Text File into a PDF
Source: https://github.com/danburzo/percollate/blob/main/README.md
Uses a newline-delimited text file of URLs and pipes them to percollate for PDF bundling.
```bash
cat urls.txt | xargs percollate pdf --output=some.pdf
```
--------------------------------
### Select Browser for Rendering
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/configuration.md
Specifies the rendering engine (e.g., Firefox) for PDF and cover page generation. Note that Firefox rendering is experimental and has limitations.
```bash
percollate pdf --browser=firefox https://example.com
```
--------------------------------
### Parse CLI Arguments
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/api-other-modules.md
Demonstrates parsing command-line arguments into command, options, and operands using the `cliopts` function. This is useful for building command-line interfaces.
```javascript
import { cliopts } from './src/cli-opts.js';
const args = ['pdf', '--output=my.pdf', '--title=My Page', 'https://example.com'];
const result = cliopts(args);
// {
// command: 'pdf',
// opts: {output: 'my.pdf', title: 'My Page'},
// operands: ['https://example.com']
// }
```
--------------------------------
### Basic Command Structure
Source: https://github.com/danburzo/percollate/blob/main/README.md
General syntax for using percollate commands, specifying a command, options, and one or more URLs.
```bash
percollate [options] url [url]...
```
--------------------------------
### Convert Web Feed Entries to Individual EPUB Files
Source: https://github.com/danburzo/percollate/blob/main/README.md
Creates separate EPUB files for each entry in an Atom or RSS feed using the --individual flag.
```bash
percollate epub --individual https://example.com/posts.xml
```
--------------------------------
### Use Custom CSS Stylesheet
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/configuration.md
Employ the --style flag to specify a custom CSS file for styling the output. This replaces the default stylesheet entirely. The path is relative to the current directory.
```bash
percollate pdf --style=./my-styles.css https://example.com
```
--------------------------------
### CLI: Multiple URLs with TOC
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/README.md
Command-line interface command to generate a PDF from multiple URLs, including a table of contents with a specified level.
```bash
percollate pdf --title="Collection" --toc --toc-level=2 \
https://example.com/article1 \
https://example.com/article2
```
--------------------------------
### Configure Percollate Environment
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/api-main.md
Initializes the percollate environment by setting up the Nunjucks templating engine. This function must be called once before any bundle operations.
```javascript
import { configure, pdf } from 'percollate';
configure();
await pdf(['https://example.com'], { output: 'example.pdf' });
```
--------------------------------
### CLI: Basic PDF Generation
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/README.md
Command-line interface command to generate a PDF from a single URL.
```bash
percollate pdf --output=article.pdf https://example.com
```
--------------------------------
### Strategy Pattern for Output Formatting
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/architecture.md
Abstracts the output format by using a strategy pattern. This allows the same input to be processed into different output formats (PDF, EPUB, HTML, Markdown) by selecting the appropriate bundling function.
```javascript
const strategies = {
pdf: bundlePdf,
epub: bundleEpub,
html: bundleHtml,
md: bundleMd
};
await strategies[command](items, options);
```
--------------------------------
### md(urls, options)
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/api-main.md
Converts one or more web pages into Markdown format. This function accepts an array of URLs or file paths and an optional configuration object. It returns a promise that resolves to an object containing Markdown-formatted items and options, or null if no valid URLs were provided.
```APIDOC
## Function: `md(urls, options)`
Converts one or more web pages into Markdown format.
```javascript
async function md(
urls: string[] | Readable[],
options?: MarkdownOptions
): Promise<{items: ParsedItem[], options: MarkdownOptions} | null>
```
### Parameters
| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| urls | `string[]` or `stream.Readable[]` | Yes | — | Array of URLs, file paths, or stdin streams |
| options | `MarkdownOptions` | No | `{}` | Configuration object for Markdown generation |
### Options Object (MarkdownOptions)
Same as HTMLOptions, plus:
| Key | Type | Default | Description |
|-----|------|---------|-------------|
| md.* | `string` or `boolean` | See defaults | Options passed to `mdast-util-to-markdown`. Use `--md.option=value` syntax in CLI. Keys: `bullet`, `bulletOther`, `bulletOrdered`, `bulletOrderedOther`, `closeAtx`, `emphasis`, `fence`, `fences`, `incrementListMarker`, `listItemIndent`, `quote`, `resourceLink`, `rule`, `ruleRepetition`, `ruleSpaces`, `setext`, `strong`, `tightDefinitions` |
Default Markdown options:
```javascript
{
fences: true,
emphasis: '_',
strong: '_',
resourceLink: true,
rule: '-'
}
```
### Returns
Promise resolving to an object containing:
- `items`: array of `ParsedItem` objects (Markdown variant)
- `options`: the final options object used
- Returns `null` if no valid URLs were provided
### Example
```javascript
import { configure, md } from 'percollate';
configure();
// Basic Markdown output
await md(['https://example.com'], {
output: 'article.md'
});
// To stdout
await md(['https://example.com'], {
output: '-'
});
// Custom Markdown formatting
await md(['https://example.com'], {
output: 'custom.md',
'md.emphasis': 'asterisk',
'md.strong': 'asterisk'
});
```
```
--------------------------------
### Configuration Options
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/INDEX.txt
Documents over 40 command-line and environment variable options for configuring Percollate's behavior, including output, bundling, content processing, and styling.
```APIDOC
## Configuration Options
### Description
Provides a comprehensive reference for all command-line options and environment variables available for configuring Percollate's behavior across various aspects of content processing and output generation.
### Option Categories
- **Command-line Options**: Over 40 options documented.
- **Output Options**: `--output`, `--individual`, `--wait`, etc.
- **Bundling Options**: `--title`, `--author`, `--toc`, etc.
- **Content Processing**: `--url`, `--no-amp`, `--unsafe`, etc.
- **Styling Options**: `--style`, `--css`, `--template`.
- **Table of Contents Options**
- **Cover Page Options**
- **Text Processing Options**
- **Image Options**
- **Browser/Rendering Options**
- **Markdown-Specific Options**
- **Debug/Help Options**
- **Environment Variables**
- **API Usage Mapping**
### Documentation Location
Detailed documentation for all options can be found in `configuration.md`.
```
--------------------------------
### Convert Web Pages to PDF from Stdin
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/api-main.md
Reads HTML content from stdin and converts it to a PDF file, specifying the base URL for resolving relative links. Requires prior configuration.
```javascript
import { createReadStream } from 'fs';
import { configure, pdf } from 'percollate';
configure();
const stream = createReadStream('article.html');
await pdf([stream], {
output: 'stdin.pdf',
url: 'https://example.com'
});
```
--------------------------------
### Clone Percollate Repository
Source: https://github.com/danburzo/percollate/blob/main/CONTRIBUTING.md
Clone the Percollate repository to your local machine to begin development.
```bash
git clone git@github.com:danburzo/percollate.git
cd percollate
```
--------------------------------
### Convert Web Feed to EPUB
Source: https://github.com/danburzo/percollate/blob/main/README.md
Processes an Atom or RSS feed and creates an EPUB book from its entries.
```bash
percollate epub https://example.com/posts.xml
```
--------------------------------
### cliopts
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/exports.md
Parses command-line arguments into command, options, and operands.
```APIDOC
## cliopts
### Description
Parses command-line arguments into command, options, and operands.
### Parameters
- **args** (string[]) - An array of command-line arguments.
### Returns
- **{command, opts, operands}** - An object containing the parsed command, options, and operands.
```
--------------------------------
### Browser Rendering Workflow
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/architecture.md
Illustrates the workflow for browser rendering using Puppeteer. A single browser instance is launched, new pages are created for specific tasks like PDF generation or screenshots, and the browser is closed upon completion.
```javascript
const browser = await launch(options);
const page = await browser.newPage();
await page.pdf(pdfOptions); // or page.screenshot() for cover
await browser.close();
```
--------------------------------
### CLI: EPUB from Feed
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/README.md
Command-line interface command to generate an EPUB from an RSS or Atom feed URL, specifying a custom title.
```bash
percollate epub --title="My Feed" https://example.com/feed.xml
```
--------------------------------
### epub(urls, options)
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/api-main.md
Converts one or more web pages into an EPUB file. It accepts an array of URLs or readable streams and an optional configuration object for EPUB generation.
```APIDOC
## Function: `epub(urls, options)`
### Description
Converts one or more web pages into an EPUB file. It accepts an array of URLs or readable streams and an optional configuration object for EPUB generation.
### Parameters
| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| urls | `string[]` or `stream.Readable[]` | Yes | — | Array of URLs, file paths, or stdin streams |
| options | `EPUBOptions` | No | `{}` | Configuration object for EPUB generation |
```
--------------------------------
### pdf(urls, options)
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/api-main.md
Converts one or more web pages into a PDF file. It accepts an array of URLs or readable streams and an optional configuration object for PDF generation.
```APIDOC
## Function: `pdf(urls, options)`
### Description
Converts one or more web pages into a PDF file. It accepts an array of URLs or readable streams and an optional configuration object for PDF generation.
### Parameters
| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| urls | `string[]` or `stream.Readable[]` | Yes | — | Array of URLs (http/https/file), local file paths, or stdin streams (pass `-` for stdin) |
| options | `PDFOptions` | No | `{}` | Configuration object for PDF generation |
### Options Object (PDFOptions)
| Key | Type | Default | Description |
|-----|------|---------|-------------|
| output | `string` | Auto-generated filename | Output file path for the PDF; use `-` for stdout (not allowed for PDF) |
| style | `string` | Default stylesheet | Path to custom CSS stylesheet |
| css | `string` | `''` | Inline CSS to append to stylesheet |
| template | `string` | Default template | Path to custom HTML template (Nunjucks format) |
| title | `string` | First page title or "Untitled" | Bundle title, used in metadata and cover |
| author | `string` | First page byline | Bundle author, used in metadata and cover |
| url | `string[]` | — | Base URLs for resolving relative links when using stdin |
| wait | `number` | 0 | Delay in seconds between processing multiple URLs (0 = parallel) |
| individual | `boolean` | false | Export each URL as a separate PDF file |
| cover | `boolean` | auto | Generate a cover page (auto: enabled if multiple items or title provided) |
| toc | `boolean` | auto | Generate a table of contents (auto: enabled if multiple items) |
| toc-level | `number` | 1 | Heading depth for TOC (1–6); implies `--toc` if > 1 |
| hyphenate | `boolean` | true | Enable hyphenation and text justification |
| inline | `boolean` | false | Embed images as base64 data URLs |
| amp | `boolean` | true | Use AMP version of pages if available |
| browser | `string` | 'chrome' | Browser engine: 'chrome' or 'firefox' |
| sandbox | `boolean` | true | Enable Puppeteer sandbox (set false to disable for restricted environments) |
| debug | `boolean` | false | Print verbose debug information |
| unsafe | `boolean` | false | Disable JSDOM HTML validation for invalid markup |
| fetch | `object` | `{}` | Custom fetch options (headers, timeout, etc.) |
### Returns
Promise resolving to an object containing:
- `items`: array of `ParsedItem` objects (see types.md)
- `options`: the final options object used
- Returns `null` if no valid URLs were provided
### Throws
- When PDF rendering fails (Puppeteer error)
- When file I/O fails
- When URL fetch fails (unless caught and ignored during cleanup)
### Example
```javascript
import { configure, pdf } from 'percollate';
configure();
// Basic usage
await pdf(['https://example.com'], {
output: 'example.pdf'
});
// Multiple pages with title
await pdf(
['https://example.com/1', 'https://example.com/2'],
{
output: 'bundle.pdf',
title: 'My Collection',
author: 'Jane Doe'
}
);
// With custom CSS
await pdf(['https://example.com'], {
output: 'custom.pdf',
css: '@page { size: A3 landscape; margin: 0 }'
});
// Read from stdin with explicit URL
import { createReadStream } from 'fs';
const stream = createReadStream('article.html');
await pdf([stream], {
output: 'stdin.pdf',
url: 'https://example.com'
});
```
```
--------------------------------
### Use Custom HTML Template
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/configuration.md
Specify a custom Nunjucks HTML template using the --template flag. This allows for complete control over the output structure. Available context variables include filetype, title, author, date, items, style, and options.
```bash
percollate pdf --template=./my-template.html https://example.com
```
--------------------------------
### Set Bundle Author
Source: https://github.com/danburzo/percollate/blob/main/README.md
Generate a PDF file from a URL and specify the author for the document.
```bash
percollate pdf --author="Ella Example" http://example.com
```
--------------------------------
### Process HTML from Standard Input for PDF Conversion
Source: https://github.com/danburzo/percollate/blob/main/README.md
Reads HTML from stdin and converts it to PDF, specifying the source URL with --url for correct relative link resolution.
```bash
curl https://example.com/page1 | percollate pdf --url=https://example.com/page1 -
```
--------------------------------
### cliopts(args)
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/api-other-modules.md
Parses command-line arguments into command, options, and operands. It supports various flag formats and value casting.
```APIDOC
## Function: cliopts(args)
### Description
Parses command-line arguments into command, options, and operands. It supports various flag formats and value casting.
### Parameters
#### Path Parameters
None
#### Query Parameters
None
#### Request Body
None
### Parameters
- **args** (`string[]`) - Required - Array of CLI arguments (usually from `process.argv.slice(2)`)
### Returns
Object with:
- `command`: First recognized command ('pdf', 'epub', 'html', 'md') or `undefined`
- `opts`: Parsed options object with camelCase keys
- `operands`: Non-option arguments (URLs, file paths, etc.)
### Option Formats
- Short flags: `-o path` (single letter)
- Long flags: `--output=path` or `--output path`
- Negation: `--no-flag` sets flag to `false`
- Multi-value: `--url=url1 --url=url2` creates array
### Value Casting
- `'true'` → `true`
- `'false'` → `false`
- Numeric strings → numbers
- Other strings → strings as-is
### Example
```javascript
import { cliopts } from './src/cli-opts.js';
const args = ['pdf', '--output=my.pdf', '--title=My Page', 'https://example.com'];
const result = cliopts(args);
// {
// command: 'pdf',
// opts: {output: 'my.pdf', title: 'My Page'},
// operands: ['https://example.com']
// }
```
```
--------------------------------
### Content Fetching Strategies
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/architecture.md
Illustrates the different methods used by Percollate to fetch content based on the input source type.
```javascript
URL (http/https)
↓
fetch(url) → response.arrayBuffer()
File path
↓
fs.readFile(path) → Buffer
File URL (file://)
↓
fs.readFile(decoded_path) → Buffer
Stream (stdin)
↓
slurp(stream) → Buffer
```
--------------------------------
### Export Each URL as a Separate File
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/configuration.md
Enable the --individual flag to export each processed URL into its own output file. This applies to all commands.
```bash
percollate pdf --individual url1 url2 url3 # Produces 3 PDFs
```
--------------------------------
### Transform Multiple Web Pages into Individual PDFs
Source: https://github.com/danburzo/percollate/blob/main/README.md
Converts several web pages into separate PDF files using the --individual flag.
```bash
percollate pdf --individual https://example.com/page1 https://example.com/page2
```
--------------------------------
### Cleanup Item with Options
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/api-cleanup-item.md
Demonstrates how to use the `cleanupItem` function with various options for processing a JSDOM instance. It shows the expected output for title, content, and table of contents.
```javascript
import { JSDOM } from 'jsdom';
import cleanupItem from './src/cleanup-item.js';
const html = '
Title
Content
';
const dom = new JSDOM(html, { url: 'https://example.com' });
const item = await cleanupItem(dom, {
mapRemoteResources: false,
inline: false,
xhtml: false,
hyphenate: true,
'toc-level': 2,
debug: false
}, null, {
err: process.stderr,
out: process.stdout,
UA: 'percollate/4.3.0'
});
console.log(item.title);
console.log(item.content);
console.log(item.toc);
```
--------------------------------
### Generate Cover Page
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/configuration.md
Enables the generation of a cover page for PDF and EPUB outputs. This is often auto-enabled when bundling multiple items or using the `--title` option.
```bash
percollate pdf --cover --title="My Book" https://example.com
```
--------------------------------
### Enable Debugging for HTML Output
Source: https://github.com/danburzo/percollate/blob/main/CONTRIBUTING.md
Use the --debug flag to save the temporary HTML file for inspection.
```bash
percollate --debug ...
```
--------------------------------
### Main Module Public Exports
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/exports.md
Exports the core functions for configuring and processing content in various formats.
```javascript
export { configure, pdf, epub, html, md };
```
--------------------------------
### Exporting Available and Default Markdown Options
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/exports.md
Exports the set of available markdown options and the default configuration for markdown processing.
```javascript
export {
AVAILABLE_MARKDOWN_OPTIONS,
DEFAULT_MARKDOWN_OPTIONS
};
```
--------------------------------
### Validate EPUB with epubcheck
Source: https://github.com/danburzo/percollate/blob/main/CONTRIBUTING.md
Use the epubcheck tool to validate the generated EPUB file.
```bash
npx epubcheck ./book.epub
```
--------------------------------
### Run Automated Tests
Source: https://github.com/danburzo/percollate/blob/main/CONTRIBUTING.md
Execute the automated test suite to ensure functionality.
```bash
npm run test
```
--------------------------------
### Percollate Options Object
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/architecture.md
Illustrates the structure of the options object used for configuration during a run. It includes various settings for output, styling, and internal caching mechanisms like a slug cache to prevent filename collisions.
```javascript
{
output?: string,
style?: string,
css?: string,
// ... 20+ more options
slugCache: {} // Tracks generated filenames to prevent collisions
}
```
--------------------------------
### Importing File Mimetype Utilities
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/exports.md
Imports utility functions for looking up file MIME types and checking if a URL points to an image.
```javascript
import { lookupMimetype, isImageURL } from './src/util/file-mimetype.js';
```
--------------------------------
### fetchBase64
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/exports.md
Fetches a resource from a URL and returns its content as a Base64 encoded string.
```APIDOC
## fetchBase64
### Description
Fetches a resource from a URL and returns its content as a Base64 encoded string.
### Parameters
- **url** (string) - The URL of the resource to fetch.
- **fetchOptions** (object) - Optional. Options to pass to the fetch request.
### Returns
- **Promise** - A promise that resolves with the Base64 encoded content.
```
--------------------------------
### remoteResources
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/api-other-modules.md
Scans a document for external image sources (img src, picture source, srcset) and maps them to local filenames for bundling with EPUB files. It generates unique filenames, replaces URLs with relative paths, and returns a list of resources to be fetched.
```APIDOC
## Function: remoteResources
### Description
Scans a document for external image sources (img src, picture source, srcset) and maps them to local filenames for bundling with EPUB files. It generates unique filenames, replaces URLs with relative paths, and returns a list of resources to be fetched.
### Parameters
- **doc** (`Document`) - Required - JSDOM document with `` and `` elements
### Returns
- `RemoteResource[]` - Array of `RemoteResource` objects:
```javascript
{
original: string, // Original URL
mapped: string, // Generated local filename (e.g., 'rr-12345678.jpg')
origin: string, // Document origin (for referrer)
mimetype: string // Detected MIME type (e.g., 'image/jpeg')
}
```
### Example
```javascript
import remoteResources from './src/remote-resources.js';
import { JSDOM } from 'jsdom';
const dom = new JSDOM(`
`, { url: 'https://example.com' });
const resources = remoteResources(dom.window.document);
// Returns: [
// {original: 'https://example.com/photo.jpg', mapped: 'rr-uuid1.jpg', origin: 'https://example.com', mimetype: 'image/jpeg'},
// {original: 'https://example.com/large.webp', mapped: 'rr-uuid2.webp', origin: 'https://example.com', mimetype: 'image/webp'},
// ...
// ]
```
### Source
`src/remote-resources.js:1–60`
```
--------------------------------
### Convert Web Pages to PDF with Custom CSS
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/api-main.md
Generates a PDF from web pages using a custom CSS stylesheet for formatting. The configure() function must be called first.
```javascript
import { configure, pdf } from 'percollate';
configure();
// With custom CSS
await pdf(['https://example.com'], {
output: 'custom.pdf',
css: '@page { size: A3 landscape; margin: 0 }'
});
```
--------------------------------
### imagesAtFullSize(doc)
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/api-enhancements.md
Extracts full-size image URLs from thumbnail links, applies URL mapping corrections, and cleans up image attributes and query parameters.
```APIDOC
## imagesAtFullSize(doc)
### Description
Extracts full-size image URLs from thumbnail links.
### Method
`imagesAtFullSize(doc: Document): void`
### Parameters
#### Path Parameters
- **doc** (`Document`) - JSDOM window.document object
### Behavior
Transforms patterns like:
```html
```
To:
```html
```
### Exclusions
The following URL patterns are excluded from image extraction:
- Wikipedia image file pages (`wikipedia.org/wiki/[lang]:`)
- GitHub-hosted images (may be embedded Markdown)
### URL Mapping
Special URL patterns are corrected:
- **Blogger images**: `/s1600-h/` → `/s1600/` (removes redirect wrapper)
### Additional Processing
- Removes `width` and `height` attributes from all images
- Strips query parameters from image URLs (e.g., `?w=1024`)
### Example
```javascript
doc.body.innerHTML = '
';
imagesAtFullSize(doc);
// Result:
```
```
--------------------------------
### Main Module Exports
Source: https://github.com/danburzo/percollate/blob/main/_autodocs/exports.md
Exports from the main module (`index.js`) include functions for configuring the library and processing content into various formats (PDF, EPUB, HTML, Markdown).
```APIDOC
## Main Module Exports
### Public Exports
#### `configure()`
- **Type**: function
- **Parameters**: `()`
- **Returns**: `void`
#### `pdf(urls: string[], options?: object)`
- **Type**: async function
- **Parameters**: `urls` (string[]), `options` (object, optional)
- **Returns**: `Promise<{items, options} | null>`
#### `epub(urls: string[], options?: object)`
- **Type**: async function
- **Parameters**: `urls` (string[]), `options` (object, optional)
- **Returns**: `Promise<{items, options} | null>`
#### `html(urls: string[], options?: object)`
- **Type**: async function
- **Parameters**: `urls` (string[]), `options` (object, optional)
- **Returns**: `Promise<{items, options} | null>`
#### `md(urls: string[], options?: object)`
- **Type**: async function
- **Parameters**: `urls` (string[]), `options` (object, optional)
- **Returns**: `Promise<{items, options} | null>`
### Test Exports
#### `__test__.fetchContent(ref: string | Readable, fetchOptions?: object)`
- **Type**: async function
- **Parameters**: `ref` (string or Readable), `fetchOptions` (object, optional)
- **Returns**: `Promise<{buffer, contentType}>`
```