### Install strtok3 using npm

Source: https://github.com/borewit/strtok3/blob/master/README.md

Use this command to install the strtok3 package. This is the primary method for adding the library to your project.

```sh
npm install strtok3
```

--------------------------------

### Example Usage of peekBuffer with mayBeLess

Source: https://github.com/borewit/strtok3/blob/master/README.md

Demonstrates how to use the peekBuffer method with the mayBeLess option set to true. This prevents an EOF error if fewer bytes than requested can be read.

```javascript
tokenizer.peekBuffer(buffer, {mayBeLess: true});
```

--------------------------------

### Parse data from Blob

Source: https://github.com/borewit/strtok3/blob/master/README.md

Example of creating a tokenizer from a Blob using `fromBlob` and reading a UINT8 token. Requires Node.js `fs` module for `openAsBlob`.

```javascript
import { fromBlob } from 'strtok3';
import { openAsBlob } from 'node:fs';
import * as Token from 'token-types';

async function parse() {
  const blob = await openAsBlob('somefile.bin');

  const tokenizer = fromBlob(blob);

  const myUint8Number = await tokenizer.readToken(Token.UINT8);
  console.log(`My number: ${myUint8Number}`);   
}

parse();

```

--------------------------------

### Implement IGetToken for Custom Binary Structures

Source: https://context7.com/borewit/strtok3/llms.txt

Define custom tokens by implementing the IGetToken interface for objects with `len` and `get` properties. This allows for fully custom binary decoders for structures like MAC addresses or IPv4 addresses.

```typescript
import { fromBuffer } from 'strtok3';
import type { IGetToken } from 'strtok3';

// Custom token: 6-byte MAC address decoded to "XX:XX:XX:XX:XX:XX" string
const MacAddressToken: IGetToken<string> = {
  len: 6,
  get(buf: Uint8Array, off: number): string {
    return Array.from(buf.subarray(off, off + 6))
      .map(b => b.toString(16).padStart(2, '0'))
      .join(':');
  }
};

// Custom token: 4-byte IPv4 address decoded to "A.B.C.D" string
const IPv4Token: IGetToken<string> = {
  len: 4,
  get(buf: Uint8Array, off: number): string {
    return Array.from(buf.subarray(off, off + 4)).join('.');
  }
};

async function parseNetworkHeader() {
  const data = new Uint8Array([
    0x00, 0x1a, 0x2b, 0x3c, 0x4d, 0x5e, // MAC: 00:1a:2b:3c:4d:5e
    192, 168, 1, 42                       // IPv4: 192.168.1.42
  ]);

  const tokenizer = fromBuffer(data);

  const mac = await tokenizer.readToken(MacAddressToken);
  console.log('MAC address:', mac);   // "00:1a:2b:3c:4d:5e"

  const ip = await tokenizer.readToken(IPv4Token);
  console.log('IPv4 address:', ip);   // "192.168.1.42"

  await tokenizer.close();
}

parseNetworkHeader();

```

--------------------------------

### Read raw bytes with tokenizer.readBuffer()

Source: https://context7.com/borewit/strtok3/llms.txt

Fills a provided Uint8Array with raw bytes from the stream. Supports specifying a starting position and a `mayBeLess` option to prevent errors when fewer bytes are available than requested.

```typescript
import { fromBuffer } from 'strtok3';

const source = new Uint8Array([0x01, 0x02, 0x03, 0x04, 0x05, 0x06]);
const tokenizer = fromBuffer(source);

async function readRaw() {
  // Read 3 bytes into a buffer
  const buf = new Uint8Array(3);
  const n = await tokenizer.readBuffer(buf);
  console.log('Bytes read:', n);    // 3
  console.log('Buffer:', buf);      // Uint8Array [1, 2, 3]
  console.log('Position:', tokenizer.position); // 3

  // Read from a specific position (random-access tokenizers only)
  const buf2 = new Uint8Array(2);
  await tokenizer.readBuffer(buf2, { position: 4 });
  console.log('At offset 4:', buf2); // Uint8Array [5, 6]

  // Read with mayBeLess: won't throw if fewer bytes remain
  const buf3 = new Uint8Array(10);
  const read = await tokenizer.readBuffer(buf3, { mayBeLess: true });
  console.log('Bytes available:', read); // 0 (already at EOF)

  await tokenizer.close();
}

readRaw();

```

--------------------------------

### readToken

Source: https://github.com/borewit/strtok3/blob/master/README.md

Reads a specific token from the tokenizer stream based on a provided token definition. Optionally, reading can start from a specified position. Returns a promise resolving with the value of the read token.

```APIDOC
## readToken

### Description
Reads a specific token from the tokenizer stream based on a provided token definition. Optionally, reading can start from a specified position. Returns a promise resolving with the value of the read token.

### Method
`readToken<Value>(token: IGetToken<Value>, position: number = this.position): Promise<Value>`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Parameters
- **token** (IGetToken) - The definition of the token to read from the tokenizer stream.
- **position** (number, optional) - The offset within the file to begin reading. If not provided, reading starts from the current file position.

### Request Example
```javascript
// Example usage (assuming tokenizer and a token definition exist)
const myTokenDefinition = { /* ... definition ... */ };
tokenizer.readToken(myTokenDefinition).then(tokenValue => {
  console.log('Read token:', tokenValue);
});
```

### Response
#### Success Response (200)
- **Promise<Value>** - A promise that resolves with the value of the token read from the stream.
```

--------------------------------

### fromFile()

Source: https://context7.com/borewit/strtok3/llms.txt

Opens a file by path and returns a FileTokenizer that supports full random access (seeking to arbitrary offsets). Automatically populates fileInfo.size and fileInfo.path. Must be closed after use to release the file descriptor.

```APIDOC
## fromFile()

### Description
Opens a file by path and returns a `FileTokenizer` that supports full **random access** (seeking to arbitrary offsets). Automatically populates `fileInfo.size` and `fileInfo.path`. Must be closed after use to release the file descriptor.

### Method
`fromFile(filePath: string): Promise<FileTokenizer>`

### Parameters
- **filePath** (string) - The path to the file to open.

### Request Example
```ts
import { fromFile } from 'strtok3';
import * as Token from 'token-types';

async function parseId3v1Tag(filePath: string) {
  const tokenizer = await fromFile(filePath);
  try {
    console.log('File size:', tokenizer.fileInfo.size);       // e.g. 8192
    console.log('File path:', tokenizer.fileInfo.path);       // e.g. "/audio/track.mp3"
    console.log('Random access:', tokenizer.supportsRandomAccess()); // true

    const id3HeaderSize = 128;
    const id3Header = new Uint8Array(id3HeaderSize);

    // Seek directly to the last 128 bytes (ID3v1 tag location)
    await tokenizer.readBuffer(id3Header, {
      position: tokenizer.fileInfo.size - id3HeaderSize
    });

    // First 3 bytes should be "TAG"
    const tag = new TextDecoder('utf-8').decode(id3Header.subarray(0, 3));
    console.log('ID3v1 tag marker:', tag); // "TAG"

    // Reset position to beginning for further parsing
    tokenizer.setPosition(0);
    const firstByte = await tokenizer.readToken(Token.UINT8);
    console.log('First byte:', firstByte);
  } finally {
    await tokenizer.close(); // Always close to release the file handle
  }
}

parseId3v1Tag('./test/resources/id3v1.mp3');
```

### Response
- **FileTokenizer**: An object with methods for reading tokens, managing position, and file information.
```

--------------------------------

### fromWebStream()

Source: https://github.com/borewit/strtok3/blob/master/README.md

Creates a tokenizer from a WHATWG ReadableStream, suitable for web environments.

```APIDOC
## fromWebStream()

### Description
Create a tokenizer from a [WHATWG ReadableStream](https://developer.mozilla.org/en-US/docs/Web/API/ReadableStream).

### Method Signature
```ts
function fromWebStream(webStream: AnyWebByteStream, options?: ITokenizerOptions): ReadStreamTokenizer
```

### Parameters
#### Path Parameters
- **webStream** (ReadableStream) - Required - WHATWG ReadableStream to read from.
- **options** (ITokenizerOptions) - Optional - Tokenizer options.

### Returns
A [*tokenizer*](#tokenizer-object).

### Example
```js
import { fromWebStream } from 'strtok3';
import * as Token from 'token-types';

async function parse() {
  const tokenizer = fromWebStream(readableStream);
  try {
    const myUint8Number = await tokenizer.readToken(Token.UINT8);
    console.log(`My number: ${myUint8Number}`);
  } finally {
    await tokenizer.close();
  }
}

parse();
```
```

--------------------------------

### fromFile()

Source: https://github.com/borewit/strtok3/blob/master/README.md

Creates a tokenizer directly from a local file path. This function is Node.js specific.

```APIDOC
## fromFile()

### Description
Creates a [*tokenizer*](#tokenizer-object) from a local file.

### Method Signature
```ts
function fromFile(sourceFilePath: string): Promise<FileTokenizer>
```

### Parameters
#### Path Parameters
- **sourceFilePath** (string) - Required - Path to file to read from.

### Notes
- Only available for Node.js engines.
- `fromFile` automatically embeds [file-information](#file-information).

### Returns
A Promise resolving to a [*tokenizer*](#tokenizer-object) which can be used to parse a file.

### Example
```js
import { fromFile } from 'strtok3';
import * as Token from 'token-types';

async function parse() {
  const tokenizer = await fromFile('somefile.bin');
  try {
    const myNumber = await tokenizer.readToken(Token.UINT8);
    console.log(`My number: ${myNumber}`);
  } finally {
    tokenizer.close(); // Close the file
  }
}

parse();
```
```

--------------------------------

### fromBuffer()

Source: https://github.com/borewit/strtok3/blob/master/README.md

Creates a tokenizer from in-memory data, such as a Uint8Array or Node.js Buffer.

```APIDOC
## fromBuffer()

### Description
Create a tokenizer from memory ([Uint8Array](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Uint8Array) or Node.js [Buffer](https://nodejs.org/api/buffer.html)).

### Method Signature
```ts
function fromBuffer(uint8Array: Uint8Array, options?: ITokenizerOptions): BufferTokenizer
```

### Parameters
#### Path Parameters
- **uint8Array** (Uint8Array) - Required - Buffer or Uint8Array to read from.
- **options** (ITokenizerOptions) - Optional - Tokenizer options.

### Returns
A [*tokenizer*](#tokenizer-object).

### Example
```js
import { fromBuffer } from 'strtok3';
import * as Token from 'token-types';

const tokenizer = fromBuffer(buffer);

async function parse() {
  const myUint8Number = await tokenizer.readToken(Token.UINT8);
  console.log(`My number: ${myUint8Number}`);
}

parse();
```
```

--------------------------------

### IFileInfo Interface

Source: https://github.com/borewit/strtok3/blob/master/README.md

Provides optional metadata about the file being tokenized.

```APIDOC
## IFileInfo Interface

Provides optional metadata about the file being tokenized.

| Attribute | Type    | Description                                                                                       |
|-----------|---------|---------------------------------------------------------------------------------------------------||
| size      | number  | File size in bytes                                                                                ||
| mimeType  | string  | [MIME-type](https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types) of file. ||
| path      | string  | File path                                                                                         ||
| url       | string  | File URL                                                                                          |
```

--------------------------------

### Create Tokenizer from Web Stream

Source: https://github.com/borewit/strtok3/blob/master/README.md

Use `fromWebStream` to create a tokenizer from a WHATWG ReadableStream. The stream must be available when calling.

```javascript
import { fromWebStream } from 'strtok3';
import * as Token from 'token-types';

async function parse() {
  const tokenizer = fromWebStream(readableStream);
  try {
    const myUint8Number = await tokenizer.readToken(Token.UINT8);
    console.log(`My number: ${myUint8Number}`);
  } finally {
    await tokenizer.close();
  }
}

parse();

```

--------------------------------

### fromBlob()

Source: https://context7.com/borewit/strtok3/llms.txt

Creates a tokenizer from a browser Blob or File, or a Node.js file opened as a Blob (Node.js >= 20). This method supports random access and automatically populates file information.

```APIDOC
## fromBlob() — Create a tokenizer from a Blob or File

Creates a `BlobTokenizer` from a browser [`Blob`](https://developer.mozilla.org/en-US/docs/Web/API/Blob) or [`File`](https://developer.mozilla.org/en-US/docs/Web/API/File). Supports random access. Automatically populates `fileInfo.size` and `fileInfo.mimeType` from the blob.

### Method
```ts
function fromBlob(blob: Blob | File): BlobTokenizer
```

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Request Example
```ts
import { fromBlob } from 'strtok3';
import * as Token from 'token-types';

// Browser: handle a file input
async function handleFileInput(file: File) {
  const tokenizer = fromBlob(file);

  console.log('File size:', tokenizer.fileInfo.size);
  console.log('MIME type:', tokenizer.fileInfo.mimeType);
  console.log('Random access:', tokenizer.supportsRandomAccess());

  const header = await tokenizer.readToken(Token.UINT32_BE);
  console.log('Header magic:', header.toString(16));

  tokenizer.setPosition(100);
  const chunk = new Uint8Array(16);
  await tokenizer.readBuffer(chunk);
  console.log('Bytes at offset 100:', chunk);

  await tokenizer.close();
}

// Node.js (≥20): open a local file as a Blob
import { openAsBlob } from 'node:fs';
async function handleNodeFile(filePath: string) {
  const blob = await openAsBlob(filePath);
  const tokenizer = fromBlob(blob);
  const firstByte = await tokenizer.readToken(Token.UINT8);
  console.log('First byte:', firstByte);
  await tokenizer.close();
}

handleFileInput(someFile);
handleNodeFile('./somefile.bin');
```

### Response
#### Success Response (200)
- **tokenizer**: `BlobTokenizer` - An object that provides methods for reading tokens from the blob, supporting random access.

#### Response Example
```json
{
  "tokenizer": { ... }
}
```
```

--------------------------------

### Tokenizer Instantiation Methods

Source: https://github.com/borewit/strtok3/blob/master/README.md

The strtok3 module provides several methods to create a tokenizer object. These methods can be used to process data from different sources like Blobs, Buffers, Files, Streams, or Web Streams.

```APIDOC
## Tokenizer Instantiation

Use one of the following methods to instantiate an abstract tokenizer:

### `fromBlob(blob: Blob): Promise<Tokenizer>`
Creates a tokenizer from a Blob object.

### `fromBuffer(buffer: Uint8Array | ArrayBuffer): Promise<Tokenizer>`
Creates a tokenizer from a Buffer or ArrayBuffer.

### `fromFile(filePath: string): Promise<Tokenizer>`
Creates a tokenizer from a file path. This method is only available in Node.js environments.

### `fromStream(stream: ReadableStream): Promise<Tokenizer>`
Creates a tokenizer from a Node.js Readable stream. This method is only available in Node.js environments.

### `fromWebStream(stream: ReadableStream): Promise<Tokenizer>`
Creates a tokenizer from a Web Stream.

All methods return a `Tokenizer` object, either directly or via a promise.
```

--------------------------------

### Create Tokenizer from File Path (Node.js)

Source: https://context7.com/borewit/strtok3/llms.txt

Creates a FileTokenizer from a local file path, enabling full random access. Automatically populates file information and must be closed after use to release the file descriptor.

```typescript
import { fromFile } from 'strtok3';
import * as Token from 'token-types';

async function parseId3v1Tag(filePath: string) {
  const tokenizer = await fromFile(filePath);
  try {
    console.log('File size:', tokenizer.fileInfo.size);       // e.g. 8192
    console.log('File path:', tokenizer.fileInfo.path);       // e.g. "/audio/track.mp3"
    console.log('Random access:', tokenizer.supportsRandomAccess()); // true

    const id3HeaderSize = 128;
    const id3Header = new Uint8Array(id3HeaderSize);

    // Seek directly to the last 128 bytes (ID3v1 tag location)
    await tokenizer.readBuffer(id3Header, {
      position: tokenizer.fileInfo.size - id3HeaderSize
    });

    // First 3 bytes should be "TAG"
    const tag = new TextDecoder('utf-8').decode(id3Header.subarray(0, 3));
    console.log('ID3v1 tag marker:', tag); // "TAG"

    // Reset position to beginning for further parsing
    tokenizer.setPosition(0);
    const firstByte = await tokenizer.readToken(Token.UINT8);
    console.log('First byte:', firstByte);
  } finally {
    await tokenizer.close(); // Always close to release the file handle
  }
}

parseId3v1Tag('./test/resources/id3v1.mp3');
```

--------------------------------

### Convert Web-API Readable Stream to Node.js Stream

Source: https://github.com/borewit/strtok3/blob/master/README.md

Demonstrates how to convert a Web-API readable stream into a Node.js readable stream using `readable-web-to-node-stream`, and then create a strtok3 tokenizer from it. This is useful for using strtok3 in a web environment.

```javascript
import { fromWebStream } from 'strtok3';
import { ReadableWebToNodeStream } from 'readable-web-to-node-stream';

(async () => {

  const response = await fetch(url);
  const readableWebStream = response.body; // Web-API readable stream
  const webStream = new ReadableWebToNodeStream(readableWebStream); // convert to Node.js readable stream

  const tokenizer = fromWebStream(webStream); // And we now have tokenizer in a web environment
})();
```

--------------------------------

### Create Tokenizer from File

Source: https://github.com/borewit/strtok3/blob/master/README.md

Use `fromFile` to create a tokenizer from a local file path. This function is only available in Node.js environments and automatically embeds file information.

```javascript
import { fromFile } from 'strtok3';
import * as Token from 'token-types';

async function parse() {
  const tokenizer = await fromFile('somefile.bin');
  try {
    const myNumber = await tokenizer.readToken(Token.UINT8);
    console.log(`My number: ${myNumber}`);
  } finally {
    tokenizer.close(); // Close the file
  }
}

parse();


```

--------------------------------

### fromBlob()

Source: https://github.com/borewit/strtok3/blob/master/README.md

Creates a tokenizer from a Blob or File object. This is useful for browser environments.

```APIDOC
## fromBlob()

### Description
Create a tokenizer from a [Blob](https://developer.mozilla.org/en-US/docs/Web/API/Blob).

### Method Signature
```ts
function fromBlob(blob: Blob, options?: ITokenizerOptions): BlobTokenizer
```

### Parameters
#### Path Parameters
- **blob** (Blob | File) - Required - Blob or File to read from.
- **options** (ITokenizerOptions) - Optional - Tokenizer options.

### Returns
A [*tokenizer*](#tokenizer-object).

### Example
```js
import { fromBlob } from 'strtok3';
import { openAsBlob } from 'node:fs';
import * as Token from 'token-types';

async function parse() {
  const blob = await openAsBlob('somefile.bin');
  const tokenizer = fromBlob(blob);
  const myUint8Number = await tokenizer.readToken(Token.UINT8);
  console.log(`My number: ${myUint8Number}`);   
}

parse();
```
```

--------------------------------

### Create Tokenizer from Blob

Source: https://github.com/borewit/strtok3/blob/master/README.md

Use `fromBlob` to create a tokenizer from a Blob or File. Ensure the Blob is available before calling.

```ts
function fromBlob(blob: Blob, options?: ITokenizerOptions): BlobTokenizer
```

--------------------------------

### Create Tokenizer from Buffer

Source: https://github.com/borewit/strtok3/blob/master/README.md

Use `fromBuffer` to create a tokenizer from a Uint8Array or Node.js Buffer. The buffer must be available when calling.

```javascript
import { fromBuffer } from 'strtok3';
import * as Token from 'token-types';

const tokenizer = fromBuffer(buffer);

async function parse() {
  const myUint8Number = await tokenizer.readToken(Token.UINT8);
  console.log(`My number: ${myUint8Number}`);
}

parse();

```

--------------------------------

### Create Tokenizer from WHATWG ReadableStream

Source: https://context7.com/borewit/strtok3/llms.txt

Use `fromWebStream` to wrap a WHATWG `ReadableStream`. Works in browsers and Node.js (≥18). Does not support random access.

```typescript
import { fromWebStream } from 'strtok3';
import * as Token from 'token-types';

async function parseRemoteBinary(url: string) {
  const response = await fetch(url);
  if (!response.body) throw new Error('No response body');

  // response.body is a WHATWG ReadableStream<Uint8Array>
  const tokenizer = fromWebStream(response.body);

  try {
    // Read a 2-byte magic number (e.g. PNG: 0x8950, MP3: 0xfffb)
    const magic = await tokenizer.readToken(Token.UINT16_BE);
    console.log('Magic bytes:', magic.toString(16));

    // Read next 4 bytes as a signed 32-bit little-endian integer
    const value = await tokenizer.readToken(Token.INT32_LE);
    console.log('INT32_LE value:', value);

    // Peek at next byte without consuming it
    const nextByte = await tokenizer.peekToken(Token.UINT8);
    console.log('Next byte (peeked):', nextByte);
    console.log('Position still at:', tokenizer.position); // not advanced
  } finally {
    await tokenizer.close(); // releases the ReadableStream reader lock
  }
}

parseRemoteBinary('https://example.com/data.bin');
```

--------------------------------

### Create Tokenizer from Blob or File

Source: https://context7.com/borewit/strtok3/llms.txt

Use `fromBlob` to create a tokenizer from a browser `Blob` or `File`. Supports random access and automatically populates `fileInfo.size` and `fileInfo.mimeType`.

```typescript
import { fromBlob } from 'strtok3';
import * as Token from 'token-types';

// Browser: handle a file input
async function handleFileInput(file: File) {
  const tokenizer = fromBlob(file);

  console.log('File size:', tokenizer.fileInfo.size);     // from blob.size
  console.log('MIME type:', tokenizer.fileInfo.mimeType); // from blob.type
  console.log('Random access:', tokenizer.supportsRandomAccess()); // true

  // Read the first 4 bytes as a big-endian UINT32 (e.g. to detect file type)
  const header = await tokenizer.readToken(Token.UINT32_BE);
  console.log('Header magic:', header.toString(16));

  // Jump to byte 100 for a random-access read
  tokenizer.setPosition(100);
  const chunk = new Uint8Array(16);
  await tokenizer.readBuffer(chunk);
  console.log('Bytes at offset 100:', chunk);

  await tokenizer.close();
}

// Node.js (≥20): open a local file as a Blob
import { openAsBlob } from 'node:fs';
const blob = await openAsBlob('./somefile.bin');
const tokenizer = fromBlob(blob);
const firstByte = await tokenizer.readToken(Token.UINT8);
console.log('First byte:', firstByte);
await tokenizer.close();
```

--------------------------------

### fromStream()

Source: https://context7.com/borewit/strtok3/llms.txt

Creates a tokenizer from a Node.js Readable stream. This method does not support random access and is optimized for forward-only reads.

```APIDOC
## fromStream() — Create a tokenizer from a Node.js Readable stream (Node.js only)

Wraps a Node.js `stream.Readable` in a `ReadStreamTokenizer`. If the stream was created with `fs.createReadStream`, file size and path are automatically detected from the stream's `.path` property. Does **not** support random access (forward-only reads).

### Method
```ts
async fromStream(stream: NodeJS.ReadableStream): Promise<ReadStreamTokenizer>
```

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Request Example
```ts
import { fromStream } from 'strtok3';
import { createReadStream } from 'node:fs';
import * as Token from 'token-types';

async function parseStream(filePath: string) {
  const stream = createReadStream(filePath);
  const tokenizer = await fromStream(stream);

  try {
    console.log('File size:', tokenizer.fileInfo.size);
    console.log('Random access:', tokenizer.supportsRandomAccess());

    const magic = await tokenizer.readToken(Token.UINT32_BE);
    console.log('Magic number:', magic.toString(16));

    await tokenizer.ignore(10);
    console.log('Position after ignore:', tokenizer.position);

    const peeked = await tokenizer.peekToken(Token.UINT16_BE);
    console.log('Peeked UINT16_BE:', peeked);
    console.log('Position unchanged:', tokenizer.position);

    const buf = new Uint8Array(1024);
    const bytesRead = await tokenizer.readBuffer(buf, { mayBeLess: true });
    console.log('Remaining bytes read:', bytesRead);
  } catch (err) {
    if (err instanceof EndOfStreamError) {
      console.log('Reached end of stream');
    } else {
      throw err;
    }
  } finally {
    await tokenizer.close();
  }
}

parseStream('./audio.mp3');
```

### Response
#### Success Response (200)
- **tokenizer**: `ReadStreamTokenizer` - An object that provides methods for reading tokens from the stream.

#### Response Example
```json
{
  "tokenizer": { ... }
}
```
```

--------------------------------

### Create Tokenizer from Node.js Readable Stream

Source: https://context7.com/borewit/strtok3/llms.txt

Use `fromStream` to wrap a Node.js `stream.Readable`. File size and path are auto-detected if created with `fs.createReadStream`. Does not support random access.

```typescript
import { fromStream } from 'strtok3';
import { createReadStream } from 'node:fs';
import * as Token from 'token-types';
import { EndOfStreamError } from 'strtok3';

async function parseStream(filePath: string) {
  const stream = createReadStream(filePath);
  const tokenizer = await fromStream(stream);

  try {
    console.log('File size:', tokenizer.fileInfo.size);  // auto-detected from fs stream
    console.log('Random access:', tokenizer.supportsRandomAccess()); // false

    // Read a 4-byte big-endian unsigned integer
    const magic = await tokenizer.readToken(Token.UINT32_BE);
    console.log('Magic number:', magic.toString(16)); // e.g. "fffb9064"

    // Skip 10 bytes
    await tokenizer.ignore(10);
    console.log('Position after ignore:', tokenizer.position); // 14

    // Peek ahead without advancing the position
    const peeked = await tokenizer.peekToken(Token.UINT16_BE);
    console.log('Peeked UINT16_BE:', peeked);
    console.log('Position unchanged:', tokenizer.position); // still 14

    // Drain remaining bytes with mayBeLess to avoid EndOfStreamError
    const buf = new Uint8Array(1024);
    const bytesRead = await tokenizer.readBuffer(buf, { mayBeLess: true });
    console.log('Remaining bytes read:', bytesRead);
  } catch (err) {
    if (err instanceof EndOfStreamError) {
      console.log('Reached end of stream');
    } else {
      throw err;
    }
  } finally {
    await tokenizer.close();
  }
}

parseStream('./audio.mp3');
```

--------------------------------

### IReadChunkOptions Interface

Source: https://github.com/borewit/strtok3/blob/master/README.md

Options for reading chunks of data from the tokenizer.

```APIDOC
## IReadChunkOptions Interface

Each attribute is optional:

| Attribute | Type    | Description                                                                                                                                                                                                                   |
|-----------|---------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| length    | number  | Requested number of bytes to read.                                                                                                                                                                                            |
| position  | number  | Position where to peek from the file. If position is null, data will be read from the [current file position](#attribute-tokenizerposition). Position may not be less then [tokenizer.position](#attribute-tokenizerposition) |
| mayBeLess | boolean | If and only if set, will not throw an EOF error if less than the requested `mayBeLess` could be read.                                                                                                                        |

Example usage:
```js
  tokenizer.peekBuffer(buffer, {mayBeLess: true});
```
```

--------------------------------

### Create Tokenizer from Buffer

Source: https://context7.com/borewit/strtok3/llms.txt

Creates a BufferTokenizer from a Uint8Array or Buffer for in-memory data. Supports random access and reading various token types.

```typescript
import { fromBuffer } from 'strtok3';
import * as Token from 'token-types';

// Binary data: a 1-byte length prefix followed by a UTF-8 string
const data = new Uint8Array([0x05, 0x68, 0x65, 0x6c, 0x6c, 0x6f]); // \x05hello

const tokenizer = fromBuffer(data);

async function parse() {
  // Read the 1-byte unsigned integer length prefix
  const length = await tokenizer.readToken(Token.UINT8);
  console.log('String length:', length); // 5

  // Read 'length' bytes as a UTF-8 string
  const str = await tokenizer.readToken(new Token.StringType(length, 'utf-8'));
  console.log('String value:', str); // "hello"

  // tokenizer.position is now 6
  console.log('Position:', tokenizer.position); // 6

  // Random access: jump back to offset 0 and re-read
  (tokenizer as any).setPosition(0);
  const lengthAgain = await tokenizer.readToken(Token.UINT8);
  console.log('Re-read length:', lengthAgain); // 5

  await tokenizer.close();
}

parse();
```

--------------------------------

### Define IGetToken Interface

Source: https://github.com/borewit/strtok3/blob/master/README.md

Defines the interface for a token, specifying its length and how to decode a value from a buffer.

```typescript
export interface IGetToken<T> {

  /**
   * Length in bytes of encoded value
   */
  len: number;

  /**
   * Decode value from buffer at offset
   * @param buf Buffer to read the decoded value from
   * @param off Decode offset
   */
  get(buf: Uint8Array, off: number): T;
}
```

--------------------------------

### Peek raw bytes with tokenizer.peekBuffer()

Source: https://context7.com/borewit/strtok3/llms.txt

Reads raw bytes into a Uint8Array without advancing the tokenizer's position. Supports the same options as `readBuffer()`.

```typescript
import { fromBuffer } from 'strtok3';

const data = new Uint8Array([0x01, 0x02, 0x03, 0x04, 0x05]);
const tokenizer = fromBuffer(data);

async function peekAhead() {
  const peek1 = new Uint8Array(3);
  await tokenizer.peekBuffer(peek1, { length: 3 });
  console.log('Peeked:', peek1);              // [1, 2, 3]
  console.log('Position:', tokenizer.position); // 0

  // Read 1 byte — advances position
  const read1 = new Uint8Array(1);
  await tokenizer.readBuffer(read1);
  console.log('Read:', read1[0]);             // 1
  console.log('Position:', tokenizer.position); // 1

  // Peek again from current position
  const peek2 = new Uint8Array(3);
  await tokenizer.peekBuffer(peek2, { length: 3 });
  console.log('Peeked after read:', peek2);   // [2, 3, 4]
  console.log('Position:', tokenizer.position); // still 1

  await tokenizer.close();
}

peekAhead();

```

--------------------------------

### tokenizer.peekBuffer()

Source: https://context7.com/borewit/strtok3/llms.txt

Reads raw bytes into a provided `Uint8Array` without advancing the tokenizer's position. It supports the same options as `readBuffer()`.

```APIDOC
## tokenizer.peekBuffer()

### Description
Reads raw bytes into a provided `Uint8Array` from the current position without advancing the tokenizer's position pointer. Supports the same `IReadChunkOptions` as `readBuffer()`.

### Method
`tokenizer.peekBuffer(buffer: Uint8Array, options?: IReadChunkOptions)`

### Parameters
*   **buffer** (Uint8Array) - The buffer to fill with bytes peeked from the stream.
*   **options** (IReadChunkOptions, Optional) - Configuration options:
    *   **length** (number) - The number of bytes to peek. Defaults to `buffer.length`.
    *   **position** (number) - The position in the stream to start peeking from (supported by random-access tokenizers).
    *   **mayBeLess** (boolean) - If true, the method will not throw `EndOfStreamError` if fewer bytes than requested are available. Defaults to `false`.

### Request Example
```ts
import { fromBuffer } from 'strtok3';

const data = new Uint8Array([0x01, 0x02, 0x03, 0x04, 0x05]);
const tokenizer = fromBuffer(data);

const peek1 = new Uint8Array(3);
await tokenizer.peekBuffer(peek1, { length: 3 });
console.log('Peeked:', peek1); // [1, 2, 3]
console.log('Position:', tokenizer.position); // 0

await tokenizer.readBuffer(new Uint8Array(1)); // Advances position to 1

const peek2 = new Uint8Array(3);
await tokenizer.peekBuffer(peek2, { length: 3 });
console.log('Peeked after read:', peek2); // [2, 3, 4]
console.log('Position:', tokenizer.position); // still 1

await tokenizer.close();
```

### Response
#### Success Response
Fills the provided buffer with bytes from the stream without advancing the position. The return value is the number of bytes actually read.

#### Response Example
```json
3
```
```

--------------------------------

### Tokenizer Attributes

Source: https://github.com/borewit/strtok3/blob/master/README.md

Details on the attributes available on the Tokenizer object.

```APIDOC
## Tokenizer Attributes

- **fileInfo**

  Optional attribute describing the file information, see [IFileInfo](#IFileInfo)

- **position**

  Pointer to the current position in the [*tokenizer*](#tokenizer-object) stream.
  If a *position* is provided to a _read_ or _peek_ method, is should be, at least, equal or greater than this value.
```

--------------------------------

### tokenizer.readNumber() / tokenizer.peekNumber()

Source: https://context7.com/borewit/strtok3/llms.txt

Convenience methods for reading numeric tokens using a pre-allocated internal buffer, avoiding a heap allocation per call. Accepts any IGetToken<number> token from token-types.

```APIDOC
## tokenizer.readNumber() / tokenizer.peekNumber()

### Description
Convenience methods for reading numeric tokens using a pre-allocated internal buffer, avoiding a heap allocation per call. Accepts any `IGetToken<number>` token from `token-types`.

### Method
- `readNumber(token: IGetToken<number>): Promise<number>`
- `peekNumber(token: IGetToken<number>): Promise<number>`

### Parameters
- **token** (`IGetToken<number>`): The token definition for the number to be read.

### Request Example
```ts
import { fromBuffer } from 'strtok3';
import * as Token from 'token-types';

const data = new Uint8Array([
  0x00, 0x02,       // UINT16_BE = 2
  0x00, 0x00, 0x00, 0x2a // UINT32_BE = 42
]);

async function readNumbers() {
  const tokenizer = fromBuffer(data);

  // peekNumber does not advance position
  const peeked = await tokenizer.peekNumber(Token.UINT16_BE);
  console.log('Peeked UINT16_BE:', peeked);   // 2
  console.log('Position:', tokenizer.position); // 0

  // readNumber advances position
  const n1 = await tokenizer.readNumber(Token.UINT16_BE);
  console.log('readNumber UINT16_BE:', n1);   // 2
  console.log('Position:', tokenizer.position); // 2

  const n2 = await tokenizer.readNumber(Token.UINT32_BE);
  console.log('readNumber UINT32_BE:', n2);   // 42

  await tokenizer.close();
}

readNumbers();
```

### Response
- **Return Value**: The numeric value read from the tokenizer.
```

--------------------------------

### peekBuffer

Source: https://github.com/borewit/strtok3/blob/master/README.md

Peeks (reads ahead) data from the tokenizer into a provided buffer without advancing the stream pointer. It returns a promise that resolves with the number of bytes read, which might be less than requested if the 'mayBeLess' flag is set.

```APIDOC
## peekBuffer

### Description
Peeks (reads ahead) data from the tokenizer into a provided buffer without advancing the stream pointer. It returns a promise that resolves with the number of bytes read, which might be less than requested if the 'mayBeLess' flag is set.

### Method
`peekBuffer(uint8Array: Uint8Array, options?: IReadChunkOptions): Promise<number>`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Parameters
- **uint8Array** (Uint8Array) - Target buffer to write the data read (peeked) to.
- **options** (IReadChunkOptions) - An object that may contain a `mayBeLess` flag to indicate if the read can return fewer bytes than requested.

### Request Example
```javascript
// Example usage (assuming tokenizer is initialized)
tokenizer.peekBuffer(new Uint8Array(512)).then(bytesPeeked => {
  console.log(`Peeked ${bytesPeeked} bytes.`);
});
```

### Response
#### Success Response (200)
- **Promise<number>** - A promise that resolves with the number of bytes successfully peeked.
```

--------------------------------

### ignore

Source: https://github.com/borewit/strtok3/blob/master/README.md

Advances the tokenizer's offset pointer by a specified number of bytes. Returns a promise resolving to the number of bytes ignored.

```APIDOC
## ignore

### Description
Advance the offset pointer with the token number of bytes provided.

### Method Signature
```ts
ignore(length: number): Promise<number>
```

### Parameters
#### Path Parameters
- **length** (number) - Required - Number of bytes to ignore. Will advance the `tokenizer.position`

### Returns
A promise resolving to the number of bytes ignored from the *tokenizer-stream*.
```

--------------------------------

### Read and Peek Numeric Tokens with strtok3

Source: https://context7.com/borewit/strtok3/llms.txt

Use `readNumber` and `peekNumber` for efficient numeric token reads. `peekNumber` does not advance the tokenizer's position, while `readNumber` does. Both accept `IGetToken<number>` types.

```typescript
import { fromBuffer } from 'strtok3';
import * as Token from 'token-types';

const data = new Uint8Array([
  0x00, 0x02,       // UINT16_BE = 2
  0x00, 0x00, 0x00, 0x2a // UINT32_BE = 42
]);

async function readNumbers() {
  const tokenizer = fromBuffer(data);

  // peekNumber does not advance position
  const peeked = await tokenizer.peekNumber(Token.UINT16_BE);
  console.log('Peeked UINT16_BE:', peeked);   // 2
  console.log('Position:', tokenizer.position); // 0

  // readNumber advances position
  const n1 = await tokenizer.readNumber(Token.UINT16_BE);
  console.log('readNumber UINT16_BE:', n1);   // 2
  console.log('Position:', tokenizer.position); // 2

  const n2 = await tokenizer.readNumber(Token.UINT32_BE);
  console.log('readNumber UINT32_BE:', n2);   // 42

  await tokenizer.close();
}

readNumbers();
```

--------------------------------

### readBuffer

Source: https://github.com/borewit/strtok3/blob/master/README.md

Reads data from the tokenizer into a provided buffer (Uint8Array). The stream pointer advances by the number of bytes read. It returns a promise that resolves with the number of bytes read, which may be less than requested if the 'mayBeLess' flag is set in options.

```APIDOC
## readBuffer

### Description
Reads data from the tokenizer into a provided buffer (Uint8Array). The stream pointer advances by the number of bytes read. It returns a promise that resolves with the number of bytes read, which may be less than requested if the 'mayBeLess' flag is set in options.

### Method
`readBuffer(buffer: Uint8Array, options?: IReadChunkOptions): Promise<number>`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Parameters
- **buffer** (Uint8Array) - Target buffer to write the data read to.
- **options** (IReadChunkOptions) - An object that may contain a `mayBeLess` flag to indicate if the read can return fewer bytes than requested.

### Request Example
```javascript
// Example usage (assuming tokenizer is initialized)
tokenizer.readBuffer(new Uint8Array(1024)).then(bytesRead => {
  console.log(`Read ${bytesRead} bytes.`);
});
```

### Response
#### Success Response (200)
- **Promise<number>** - A promise that resolves with the number of bytes successfully read.
```

--------------------------------

### tokenizer.abort()

Source: https://context7.com/borewit/strtok3/llms.txt

Cancels in-flight async read operations. Useful for aborting long-running stream reads. Can also be triggered automatically via an `AbortController` passed in `ITokenizerOptions.abortSignal`.

```APIDOC
## tokenizer.abort()

### Description
Cancels in-flight async read operations. Useful for aborting long-running stream reads. Can also be triggered automatically via an `AbortController` passed in `ITokenizerOptions.abortSignal`.

### Method
- `abort(): void`

### Request Example
```ts
import { fromStream } from 'strtok3';
import { PassThrough } from 'node:stream';
import * as Token from 'token-types';

async function abortExample() {
  const stream = new PassThrough();
  const tokenizer = await fromStream(stream);

  // Start a read that will never complete (stream has no data yet)
  const readPromise = tokenizer.readToken(new Token.StringType(100, 'utf-8'));

  // Abort after 500ms
  setTimeout(() => tokenizer.abort(), 500);

  try {
    await readPromise;
  } catch (err) {
    console.log('Read aborted:', (err as Error).message);
  } finally {
    await tokenizer.close();
  }
}

// Alternatively, use AbortController:
async function abortWithController(url: string) {
  const controller = new AbortController();
  const response = await fetch(url);
  const tokenizer = fromWebStream(response.body!, {
    abortSignal: controller.signal
  });

  const readPromise = tokenizer.readToken(Token.UINT32_BE);

  controller.abort(); // triggers abort automatically via signal listener

  try {
    await readPromise;
  } catch (err) {
    console.log('Aborted via AbortController:', (err as Error).message);
  } finally {
    await tokenizer.close();
  }
}
```

### Response
- **Return Value**: None. This method aborts pending operations.
```

--------------------------------

### close

Source: https://github.com/borewit/strtok3/blob/master/README.md

Cleans up any resources associated with the tokenizer, such as closing file pointers.

```APIDOC
## close

### Description
Clean up resources, such as closing a file pointer if applicable.
```

--------------------------------

### tokenizer.peekToken()

Source: https://context7.com/borewit/strtok3/llms.txt

Reads a typed token from the stream at the current position without advancing the position pointer. This is useful for inspecting upcoming data to make parsing decisions.

```APIDOC
## tokenizer.peekToken()

### Description
Reads a typed token from the current position without advancing the tokenizer's position pointer. It's identical to `readToken()` in functionality except for the position advancement.

### Method
`tokenizer.peekToken(token: ITokenType)`

### Parameters
*   **token** (ITokenType) - The type of token to peek at (e.g., `Token.UINT32_BE`).

### Request Example
```ts
import { fromBuffer } from 'strtok3';
import * as Token from 'token-types';

const data = new Uint8Array([0x89, 0x50, 0x4e, 0x47]); // PNG magic bytes
const tokenizer = fromBuffer(data);

const magic = await tokenizer.peekToken(Token.UINT32_BE);
console.log('Position after peek:', tokenizer.position); // 0

if (magic === 0x89504e47) {
  console.log('Detected: PNG file');
}
await tokenizer.close();
```

### Response
#### Success Response
Returns the decoded token value based on the provided `ITokenType` without changing the tokenizer's position.

#### Response Example
```json
"89504e47"
```
```