### SQL Example: Get sqlite-lines Version
Source: https://github.com/asg017/sqlite-lines/blob/main/docs.md
This SQL query demonstrates how to retrieve the version string of the currently loaded sqlite-lines extension. This is useful for verifying the installed version and ensuring compatibility.
```sql
select lines_version();
```
--------------------------------
### SQL Example: Get sqlite-lines Debug Information
Source: https://github.com/asg017/sqlite-lines/blob/main/docs.md
This SQL query retrieves detailed debugging information about the sqlite-lines build. It includes the version, commit hash, and build date, which can be helpful for troubleshooting.
```sql
select lines_debug();
```
--------------------------------
### Install sqlite-lines for Deno
Source: https://github.com/asg017/sqlite-lines/blob/main/README.md
Provides the import path for using sqlite-lines in Deno. Deno manages dependencies via URLs.
```bash
deno.land/x/sqlite_lines
```
--------------------------------
### Build WASM sql.js with sqlite-lines
Source: https://github.com/asg017/sqlite-lines/blob/main/docs.md
This command builds the WebAssembly version of sql.js, incorporating sqlite-lines functionalities. It requires the Emscripten SDK to be installed. The `make test-sqljs` command starts a local server for manual testing.
```bash
make sqljs
# will start a local server and open tests/test-sqljs.html for manual testing
make test-sqljs
```
--------------------------------
### SQL Example: Use `lines_read` Table Function
Source: https://github.com/asg017/sqlite-lines/blob/main/docs.md
This SQL query shows how to use the `lines_read` table function to read lines from a file specified by its path. It also includes an example of how to parse JSON data from each line using the `->` operator.
```sql
select * from lines_read("my-file.txt");
select
line -> '$.id' as id,
line -> '$.name' as name
from lines_read("my-file.ndjson");
```
--------------------------------
### Install sqlite-lines NPM Package
Source: https://github.com/asg017/sqlite-lines/blob/main/npm/sqlite-lines/README.md
Installs the sqlite-lines package using npm. This command is used by Node.js developers to add the package to their project.
```bash
npm install sqlite-lines
```
--------------------------------
### Install Datasette plugin for sqlite-lines
Source: https://github.com/asg017/sqlite-lines/blob/main/README.md
Installs the datasette-sqlite-lines plugin for Datasette. This allows Datasette to utilize the sqlite-lines functionality.
```bash
datasette install datasette-sqlite-lines
```
--------------------------------
### Install sqlite-lines for Ruby
Source: https://github.com/asg017/sqlite-lines/blob/main/README.md
Installs the sqlite-lines gem for Ruby. This command uses the RubyGems package manager.
```bash
gem install sqlite-lines
```
--------------------------------
### Test sqlite-lines Loadable Extension
Source: https://github.com/asg017/sqlite-lines/blob/main/docs.md
This command runs tests for the loadable extension of sqlite-lines. It requires Python 3 to be installed on the system.
```bash
make test-loadable
```
--------------------------------
### sqlite-lines CLI: Basic Transformations
Source: https://context7.com/asg017/sqlite-lines/llms.txt
Illustrates basic usage of the sqlite-lines CLI tool for transforming text files. Examples include uppercasing text and adding line numbers.
```bash
# Transform each line: uppercase text with line numbers
cat names.txt | sqlite-lines 'rowid || " " || upper(d)'
# 1 ALICE
# 2 BOB
# 3 CAROL
```
--------------------------------
### Load sqlite-lines as a loadable extension in Datasette
Source: https://github.com/asg017/sqlite-lines/blob/main/README.md
Demonstrates how to load the sqlite-lines extension with Datasette using the `--load-extension` flag. This example uses a version that limits filesystem access for security.
```bash
datasette data.db --load-extension ./lines_nofs0
```
--------------------------------
### Install sqlite-lines Python Package
Source: https://github.com/asg017/sqlite-lines/blob/main/python/sqlite_lines/README.md
Installs the sqlite-lines Python package using pip. This command is used to add the package to your Python environment.
```bash
pip install sqlite-lines
```
--------------------------------
### Load sqlite-lines Extension in Python
Source: https://github.com/asg017/sqlite-lines/blob/main/python/sqlite_lines/README.md
Demonstrates how to use the sqlite-lines Python package to load the sqlite-lines extension into a sqlite3 connection. It shows how to get the extension path and then load it.
```python
import sqlite_lines
# Get the path to the loadable extension
print(sqlite_lines.loadable_lines())
# Load the extension into a connection
import sqlite3
conn = sqlite3.connect(':memory:')
sqlite_lines.load(conn)
# Verify the extension is loaded by calling a function from it
print(conn.execute('select lines_version()').fetchone())
```
--------------------------------
### Get sqlite-lines Extension Path in Python
Source: https://github.com/asg017/sqlite-lines/blob/main/python/sqlite_lines/README.md
Retrieves the full path to the locally installed sqlite-lines extension using the `loadable_lines()` function. This path can be used with `sqlite3.Connection.load_extension()`.
```python
import sqlite_lines
print(sqlite_lines.loadable_lines())
# '/.../venv/lib/python3.9/site-packages/sqlite_lines/lines0'
```
--------------------------------
### sqlite-lines CLI: Creating New JSON Objects
Source: https://context7.com/asg017/sqlite-lines/llms.txt
Demonstrates creating new JSON objects from existing data using the sqlite-lines CLI. This example constructs JSON objects with 'id' and 'score' fields.
```bash
# Create new JSON objects
cat data.ndjson | sqlite-lines 'json_object("id", d ->> "$.id", "score", d ->> "$.stats.score")'
# {"id":1,"score":95}
# {"id":2,"score":87}
```
--------------------------------
### SQL: Get sqlite-lines extension version
Source: https://context7.com/asg017/sqlite-lines/llms.txt
The `lines_version()` scalar function returns the semantic version string of the currently loaded `sqlite-lines` extension. This is useful for verifying the installed version and ensuring compatibility.
```sql
SELECT lines_version();
-- 'v0.2.2'
```
--------------------------------
### SQL Example: Use `lines` Table Function
Source: https://github.com/asg017/sqlite-lines/blob/main/docs.md
This SQL query demonstrates the usage of the `lines` table function, which splits a given text document into individual lines. It shows how to select the line content and its corresponding line number (rowid).
```sql
select rowid, line from lines('a
b
c');
```
--------------------------------
### Load sqlite-lines Extension in Deno
Source: https://context7.com/asg017/sqlite-lines/llms.txt
Shows how to load the sqlite-lines extension into a Deno x/sqlite3 Database instance. It includes examples for querying the version and processing log files using `lines_read()`.
```typescript
import { Database } from "https://deno.land/x/sqlite3@0.8.0/mod.ts";
import * as sqlite_lines from "https://deno.land/x/sqlite_lines@v0.2.2/mod.ts";
const db = new Database(":memory:");
db.enableLoadExtension = true;
sqlite_lines.load(db);
// Query version
const [version] = db
.prepare("SELECT lines_version()")
.value<[string]>()!;
console.log(version); // "v0.2.2"
// Process log file
const stmt = db.prepare(`
SELECT rowid, line
FROM lines_read(?)
WHERE line LIKE '%ERROR%'
`);
for (const [lineNum, content] of stmt.values<[number, string]>("app.log")) {
console.log(`Line ${lineNum}: ${content}`);
}
```
--------------------------------
### Get sqlite-lines Loadable Path for better-sqlite3
Source: https://github.com/asg017/sqlite-lines/blob/main/npm/sqlite-lines/README.md
Retrieves the file system path to the compiled sqlite-lines extension. This path is intended for use with the .loadExtension() method in better-sqlite3.
```javascript
import Database from "better-sqlite3";
import * as sqlite_lines from "sqlite-lines";
const db = new Database(":memory:");
db.loadExtension(sqlite_lines.getLoadablePath());
```
--------------------------------
### Get sqlite-lines Loadable Path for node-sqlite3
Source: https://github.com/asg017/sqlite-lines/blob/main/npm/sqlite-lines/README.md
Obtains the file system path for the sqlite-lines extension, suitable for use with the .loadExtension() method in node-sqlite3. This ensures the correct extension is loaded for the database.
```javascript
import sqlite3 from "sqlite3";
import * as sqlite_lines from "sqlite-lines";
const db = new sqlite3.Database(":memory:");
db.loadExtension(sqlite_lines.getLoadablePath());
```
--------------------------------
### Load sqlite-lines Extension in Node.js
Source: https://context7.com/asg017/sqlite-lines/llms.txt
Demonstrates how to load the sqlite-lines extension for both better-sqlite3 and node-sqlite3 in Node.js. It shows how to get the loadable path and use the `lines_read()` function to query data from files.
```javascript
// Using better-sqlite3
import Database from "better-sqlite3";
import * as sqlite_lines from "sqlite-lines";
const db = new Database(":memory:");
db.loadExtension(sqlite_lines.getLoadablePath());
// Verify installation
const version = db.prepare("SELECT lines_version()").pluck().get();
console.log(version); // "v0.2.2"
// Process NDJSON file
const stmt = db.prepare(`
SELECT
line ->> '$.name' as name,
line ->> '$.email' as email
FROM lines_read(?)
WHERE line ->> '$.active' = true
`);
for (const row of stmt.iterate('users.ndjson')) {
console.log(`${row.name}: ${row.email}`);
}
// Using node-sqlite3
import sqlite3 from "sqlite3";
import * as sqlite_lines from "sqlite-lines";
const db = new sqlite3.Database(":memory:");
db.loadExtension(sqlite_lines.getLoadablePath());
db.each(
"SELECT line ->> '$.id' as id FROM lines_read('data.ndjson')",
(err, row) => {
if (err) throw err;
console.log(row.id);
}
);
```
--------------------------------
### sqlite-lines CLI: Filtering Data
Source: https://context7.com/asg017/sqlite-lines/llms.txt
Shows how to filter data from NDJSON files using the sqlite-lines CLI. This example extracts names of users older than 30.
```bash
# Filter: names of users older than 30
cat users.ndjson | sqlite-lines 'd ->> "$.name"' 'd ->> "$.age" > 30'
# Carol
# David
```
--------------------------------
### Load sqlite-lines Extension in Ruby
Source: https://context7.com/asg017/sqlite-lines/llms.txt
Demonstrates loading the sqlite-lines extension into a Ruby SQLite3 database connection. It includes examples for checking the version and processing NDJSON files using `lines_read()`.
```ruby
require 'sqlite3'
require 'sqlite_lines'
db = SQLite3::Database.new(':memory:')
db.enable_load_extension(true)
SqliteLines.load(db)
db.enable_load_extension(false)
# Check version
version = db.get_first_value('SELECT lines_version()')
puts version # "v0.2.2"
# Process NDJSON
db.execute(<<-SQL, ['events.ndjson']) do |row|
SELECT
line ->> '$.event_type' as event_type,
COUNT(*) as count
FROM lines_read(?)
GROUP BY 1
ORDER BY 2 DESC
SQL
puts "#{row[0]}: #{row[1]}"
end
# Get loadable path for manual loading
puts SqliteLines.lines_loadable_path
# '/path/to/gems/sqlite_lines/lines0'
```
--------------------------------
### Filter Words using LIKE Operator
Source: https://github.com/asg017/sqlite-lines/blob/main/benchmarks/README.md
Compares the speed of sqlite-lines with SQLite's LIKE operator against the traditional `cat | grep` command for filtering words starting with 'sh' from a dictionary file.
```bash
# cat | grep approach
# cat /usr/share/dict/words | grep '^sh'
# sqlite-lines + SQLite LIKE approach
# cat /usr/share/dict/words | sqlite-lines -c "SELECT line FROM lines WHERE line LIKE 'sh%'"
```
--------------------------------
### Filter lines with SQL using sqlite-lines CLI
Source: https://github.com/asg017/sqlite-lines/blob/main/README.md
Shows how to filter lines from standard input using the sqlite-lines CLI with a SQL WHERE clause. This example retrieves names of people older than 40 from a JSON Lines file.
```bash
# get the names of all people older than 40
cat data.ndjson | sqlite-lines 'd ->> "$.name"' 'd ->> "$.age" > 40'
```
--------------------------------
### Process NDJSON Data with sqlite-lines and SQLite JSON Functions
Source: https://github.com/asg017/sqlite-lines/blob/main/README.md
Shows how to use `sqlite-lines` with ndjson files and SQLite's built-in JSON functions to extract and aggregate data. This example calculates the top 5 country codes from Google's Quick, Draw! dataset.
```sql
select
line ->> '$.countrycode' as countrycode,
count(*)
from lines_read('./calendar.ndjson')
group by 1
order by 2 desc
limit 5;
/*
┌─────────────┬──────────┐
│ countrycode │ count(*) │
├─────────────┼──────────┤
│ US │ 141001 │
│ GB │ 22560 │
│ CA │ 11759 │
│ RU │ 9250 │
│ DE │ 8748 │
└─────────────┴──────────┘
*/
```
--------------------------------
### SQL: Get sqlite-lines extension debug information
Source: https://context7.com/asg017/sqlite-lines/llms.txt
The `lines_debug()` scalar function provides detailed debugging information about the `sqlite-lines` extension, including its version, build date, commit hash, and build configuration (e.g., filesystem access enabled/disabled).
```sql
SELECT lines_debug();
/*
Version: v0.2.2
Date: 2022-05-15T16:57:23Z-0700
Commit: c87a67c6e76
*/
-- No-filesystem build will show:
/*
Version: v0.2.2
Date: 2022-05-15T16:57:23Z-0700
Commit: c87a67c6e76
NO FILESYSTEM
*/
```
--------------------------------
### SQL: Parse inline JSON array into rows using lines()
Source: https://context7.com/asg017/sqlite-lines/llms.txt
This example demonstrates using the `lines()` function in conjunction with SQLite's JSON operators (`->>`) to parse newline-delimited JSON (NDJSON) strings directly within SQL queries. Each line is treated as a separate JSON object.
```sql
SELECT
line ->> '$.id' as id,
line ->> '$.name' as name
FROM lines('{"id":1,"name":"Alice"}
{"id":2,"name":"Bob"}
{"id":3,"name":"Carol"}');
/*
┌────┬───────┐
│ id │ name │
├────┼───────┤
│ 1 │ Alice │
│ 2 │ Bob │
│ 3 │ Carol │
└────┴───────┘
*/
```
--------------------------------
### Build sqlite-lines Loadable Extension
Source: https://github.com/asg017/sqlite-lines/blob/main/docs.md
This command builds the runtime-loadable extension for sqlite-lines. The output is a shared library (`.dylib` for MacOS, `.so` for Linux) that can be dynamically loaded into SQLite.
```bash
make loadable
```
--------------------------------
### Clone and Navigate sqlite-lines Repository
Source: https://github.com/asg017/sqlite-lines/blob/main/docs.md
This snippet shows the basic commands to clone the sqlite-lines repository from GitHub and navigate into the project directory. It's the initial step for building or using the library.
```bash
git clone git@github.com:asg017/sqlite-lines.git
cd sqlite-lines
```
--------------------------------
### Load sqlite-lines with better-sqlite3
Source: https://github.com/asg017/sqlite-lines/blob/main/npm/sqlite-lines/README.md
Demonstrates how to load the sqlite-lines extension into a better-sqlite3 database instance. It imports necessary modules, creates an in-memory database, loads the extension using getLoadablePath(), and then queries the version of the extension.
```javascript
import Database from "better-sqlite3";
import * as sqlite_lines from "sqlite-lines";
const db = new Database(":memory:");
db.loadExtension(sqlite_lines.getLoadablePath());
const version = db.prepare("select lines_version()").pluck().get();
console.log(version); // "v0.2.0"
```
--------------------------------
### Show Tool Versions
Source: https://github.com/asg017/sqlite-lines/blob/main/benchmarks/README.md
Displays the versions of various tools used in the benchmarks, including ndjson-map, Python, DuckDB, dsq, sqlite-lines, pandas, and zq. This is achieved by running the --version flag or equivalent commands for each tool.
```bash
+ ndjson-map --version
0.3.1
+ python3 --version
Python 3.8.10
+ duckdb --version
v0.4.0 da9ee490d
+ dsq --version
dsq 0.17.0
+ sqlite3 :memory: '.load ../dist/lines0' 'select lines_version()'
v0.0.1
+ python3 -c 'import pandas as pd; print(pd.__version__)'
1.4.2
+ zq --version
Version: v1.0.0
```
--------------------------------
### Load sqlite-lines with node-sqlite3
Source: https://github.com/asg017/sqlite-lines/blob/main/npm/sqlite-lines/README.md
Shows how to load the sqlite-lines extension into a node-sqlite3 database instance. It imports the required libraries, initializes a database, loads the extension, and then retrieves the extension's version.
```javascript
import sqlite3 from "sqlite3";
import * as sqlite_lines from "sqlite-lines";
const db = new sqlite3.Database(":memory:");
db.loadExtension(sqlite_lines.getLoadablePath());
db.get("select lines_version()", (err, row) => {
console.log(row); // {json_schema_version(): "v0.2.0"}
});
```
--------------------------------
### Initialize and Query SQL.js Database (JavaScript)
Source: https://github.com/asg017/sqlite-lines/blob/main/tests/test-sqljs.html
Initializes an SQL.js database with a custom file locator and executes several predefined queries. The results are then formatted and displayed in an HTML table. This snippet requires the SQL.js library and htl for DOM manipulation.
```javascript
import { html } from "https://cdn.skypack.dev/htl";
const config = {
locateFile: (filename, prefix) => {
return `../dist/${filename}`;
},
};
initSqlJs(config).then(function (SQL) {
var db = new SQL.Database();
function printResults(query) {
let stmt = db.prepare(query);
const columns = stmt.getColumnNames();
const rows = [];
while (stmt.step()) {
rows.push(stmt.get());
}
document.body.appendChild(html`
${query}
${columns.map((column) => html`| ${column} | `)}
${rows.map(
(values) => html` ${values.map((d) => html`${d} | `)}
`
)}
`);
}
try {
printResults(`select name from pragma_function_list where name like 'lines%'`);
printResults(`select name from pragma_module_list where name like 'lines%'`);
printResults("select lines_version(), lines_debug()");
printResults(`select rowid, line from lines('a|b|c|d|e', '|')`);
document.querySelector("#results").innerText = "✅ tests passed";
} catch {
document.querySelector("#results").innerText = "❌ test(s) failed, check console";
}
});
```
--------------------------------
### Build sqlite-lines CLI
Source: https://github.com/asg017/sqlite-lines/blob/main/README.md
Instructions to build the command-line interface for sqlite-lines. This involves using the 'make' command to compile the CLI, which can then be executed from the terminal.
```bash
make cli
./dist/sqlite-lines
```
--------------------------------
### Build sqlite-lines CLI
Source: https://github.com/asg017/sqlite-lines/blob/main/docs.md
This command builds a standalone command-line interface for sqlite-lines. This CLI can be used to interact with sqlite-lines functionalities directly from the terminal.
```bash
make cli
make test-cli
```
--------------------------------
### Load sqlite-lines as a loadable extension in SQLite CLI
Source: https://github.com/asg017/sqlite-lines/blob/main/README.md
Demonstrates how to load the sqlite-lines extension using the SQLite command-line interface. This allows direct use of its functions within SQL queries.
```sql
.load ./lines0
select lines_version();
-- v0.0.1
```
--------------------------------
### Check Supported Platform
Source: https://github.com/asg017/sqlite-lines/blob/main/npm/sqlite-lines/README.md
Determines the operating system and CPU architecture of the current machine using Node.js process properties. This is useful for verifying compatibility with the sqlite-lines NPM package.
```bash
$ node -e 'console.log([process.platform, process.arch])'
[ 'darwin', 'x64' ]
```
--------------------------------
### Build SQLite CLI with sqlite-lines Included
Source: https://github.com/asg017/sqlite-lines/blob/main/docs.md
These commands build a custom SQLite command-line interface (CLI) that includes the sqlite-lines functionalities statically linked. This allows using sqlite-lines functions directly within the SQLite CLI.
```bash
make sqlite3
make test-sqlite3
```
--------------------------------
### Transform lines with SQL using sqlite-lines CLI
Source: https://github.com/asg017/sqlite-lines/blob/main/README.md
Demonstrates how to use the sqlite-lines CLI to transform lines of text from standard input using SQL expressions. It shows how to process each line, including uppercasing text and extracting JSON values.
```bash
$ cat names.txt | sqlite-lines 'rowid || upper(d)'
1ALEX
2BRIAN
3CRAIG
```
```bash
$ cat data.ndjson | sqlite-lines 'd ->> "$.id"'
```
```bash
$ cat data.ndjson | sqlite-lines 'json_object("name", d ->> "$.name", "age": d ->> "$.stats.age")'
```
--------------------------------
### sqlite-lines CLI: Aggregating Data
Source: https://context7.com/asg017/sqlite-lines/llms.txt
Illustrates how to aggregate data using the sqlite-lines CLI, specifically counting events by type from an NDJSON file.
```bash
# Aggregate: count events by type
cat events.ndjson | sqlite-lines 'd ->> "$.type" || ": " || count(*)' '1=1' 'd ->> "$.type"'
# click: 1523
# view: 8934
# purchase: 342
```
--------------------------------
### Load sqlite-lines Extension in Deno
Source: https://github.com/asg017/sqlite-lines/blob/main/deno/README.md
This snippet demonstrates how to load the sqlite-lines extension into a Deno SQLite database using the x/sqlite3 and x/sqlite_lines modules. It initializes an in-memory database, enables extension loading, loads the sqlite_lines extension, and then queries the extension's version.
```javascript
import { Database } from "https://deno.land/x/sqlite3@0.8.0/mod.ts";
import * as sqlite_lines from "https://deno.land/x/sqlite_lines@v0.2.2/mod.ts";
const db = new Database(":memory:");
db.enableLoadExtension = true;
sqlite_lines.load(db);
const [version] = db
.prepare("select lines_version()")
.value<[string]>()!;
console.log(version);
```
--------------------------------
### Deno Permissions for x/sqlite_lines
Source: https://github.com/asg017/sqlite-lines/blob/main/deno/README.md
This command shows the necessary Deno permissions to run applications using the x/sqlite_lines module. It requires network and filesystem access to download and cache the SQLite extension, along with unstable features and Foreign Function Interface (FFI) support.
```bash
deno run -A --unstable
```
--------------------------------
### Load and Read Lines from a File in SQL
Source: https://github.com/asg017/sqlite-lines/blob/main/README.md
Demonstrates how to load the sqlite-lines extension and read lines from a specified file using the `lines_read()` function in SQL. This is the basic usage for accessing file content line by line within SQLite.
```sql
.load ./lines0
select line from lines_read('logs.txt');
```
--------------------------------
### Aggregate lines with SQL using sqlite-lines CLI
Source: https://github.com/asg017/sqlite-lines/blob/main/README.md
Illustrates how to aggregate lines from standard input using the sqlite-lines CLI with a SQL GROUP BY clause. This functionality allows for summarizing data based on specified criteria.
```bash
# Example for aggregation (specific SQL expression would be needed)
# cat data.ndjson | sqlite-lines 'column_to_group_by' 'aggregation_expression' 'd ->> "$.column_to_group_by"'
```
--------------------------------
### Parse and Calculate Sum from NDJSON File
Source: https://github.com/asg017/sqlite-lines/blob/main/benchmarks/README.md
Compares the performance of different tools in parsing a 180MB NDJSON file and calculating the sum of elements in a specific property. Includes Python's standard file iteration, ndjson-cli, dsq, sqlite-utils, zq, Pandas, sqlite-lines, and DuckDB.
```bash
# Example of a Python script for parsing NDJSON
# (Actual script content not provided in the text, this is illustrative)
# py.sh
# import json
# total = 0
# with open('large.ndjson', 'r') as f:
# for line in f:
# data = json.loads(line)
# total += data['some_property']
# print(total)
# Example of ndjson-cli usage
# ndjson-cli '.[your_property]' < large.ndjson | node -e "let sum=0; process.stdin.on('data', chunk => sum += parseInt(chunk)); process.stdin.on('end', () => console.log(sum))"
# Example of dsq usage
# dsq --input-format ndjson --output-format json 'SELECT SUM(your_property) FROM file'
# Example of sqlite-lines usage
# cat large.ndjson | sqlite-lines -c "SELECT SUM(json_extract(line, '$.your_property')) FROM lines"
```
--------------------------------
### Compile sqlite-lines Object File
Source: https://github.com/asg017/sqlite-lines/blob/main/docs.md
This command compiles the sqlite-lines library into an object file (`.o`). This object file can then be linked into your own C/C++ applications, allowing you to statically include sqlite-lines functionality.
```bash
make dist/lines0.o
```
--------------------------------
### Load sqlite-lines Extension Manually in Python
Source: https://github.com/asg017/sqlite-lines/blob/main/python/sqlite_lines/README.md
Demonstrates manually loading the sqlite-lines extension into a sqlite3 connection object. This involves enabling extension loading, calling `sqlite_lines.load()`, and then disabling extension loading.
```python
import sqlite_lines
import sqlite3
conn = sqlite3.connect(':memory:')
# Enable extension loading
conn.enable_load_extension(True)
# Load the sqlite-lines extension
sqlite_lines.load(conn)
# Disable extension loading for security
conn.enable_load_extension(False)
# Verify the extension is loaded
print(conn.execute('select lines_version()').fetchone())
# ('v0.1.0')
```
--------------------------------
### Load sqlite-lines as a loadable extension in Node.js
Source: https://github.com/asg017/sqlite-lines/blob/main/README.md
Illustrates loading the sqlite-lines extension in Node.js using the `better-sqlite3` library. This enables the use of SQLite extensions within Node.js applications.
```javascript
const Database = require("better-sqlite3");
const db = new Database(":memory:");
db.loadExtension("./lines0");
console.log(db.prepare("select lines_version()").get());
// { 'lines_version()': 'v0.0.1' }
```
--------------------------------
### Read Lines from All Files in a Directory using fsdir and lines_read
Source: https://github.com/asg017/sqlite-lines/blob/main/README.md
Illustrates how to combine SQLite's `fsdir()` function with `lines_read()` to iterate through all files in a directory and read lines from those matching a specific pattern (e.g., '.txt'). This is useful for batch processing log files or other text-based data.
```sql
select
name as file,
lines.rowid as line_number,
line
from fsdir('logs')
join lines_read(name) as lines
where name like '%.txt';
/*
┌─────────────────────┬──────┐
│ file │ line_number | line │
├───────┼─────────────┤──────┤
| a.txt | 1 | x |
| a.txt | 2 | y |
| a.txt | 3 | z |
| b.txt | 1 | xx |
| b.txt | 2 | yy |
| c.txt | 1 | xxx |
└───────┴─────────────┴──────┘
*/
```
--------------------------------
### Python: Load sqlite-lines extension into connection
Source: https://context7.com/asg017/sqlite-lines/llms.txt
This Python code snippet shows how to load the `sqlite-lines` extension into a `sqlite3` connection using the `sqlite_lines.load()` function. It also demonstrates how to verify the extension is loaded by querying its version and processing an NDJSON file.
```python
import sqlite3
import sqlite_lines
# Connect and load extension
conn = sqlite3.connect(':memory:')
conn.enable_load_extension(True)
sqlite_lines.load(conn)
conn.enable_load_extension(False)
# Verify installation
version = conn.execute('SELECT lines_version()').fetchone()[0]
print(version) # 'v0.2.2'
# Process NDJSON file
results = conn.execute('''
SELECT
line ->> '$.user_id' as user_id,
line ->> '$.event' as event,
line ->> '$.timestamp' as ts
FROM lines_read('events.ndjson')
WHERE line ->> '$.event' = 'purchase'
LIMIT 10
''').fetchall()
for row in results:
print(f"User {row[0]}: {row[1]} at {row[2]}")
# Get loadable path for manual loading
print(sqlite_lines.loadable_path())
# '/path/to/site-packages/sqlite_lines/lines0'
```
--------------------------------
### Python Integration: sqlite_lines.load()
Source: https://context7.com/asg017/sqlite-lines/llms.txt
Loads the sqlite-lines extension into a Python sqlite3 connection. Provides `loadable_path()` for manual loading.
```APIDOC
## sqlite_lines.load(connection)
### Description
Loads the sqlite-lines extension into a Python sqlite3 connection. Also available: `loadable_path()` returns the path to the extension for manual loading.
### Method
Python Function
### Parameters
#### Path Parameters
None
#### Query Parameters
None
#### Request Body
- **connection** (sqlite3.Connection) - The active SQLite connection object.
### Request Example
```python
import sqlite3
import sqlite_lines
# Connect and load extension
conn = sqlite3.connect(':memory:')
conn.enable_load_extension(True)
sqlite_lines.load(conn)
conn.enable_load_extension(False)
# Verify installation
version = conn.execute('SELECT lines_version()').fetchone()[0]
print(version) # 'v0.2.2'
# Process NDJSON file
results = conn.execute('''
SELECT
line ->> '$.user_id' as user_id,
line ->> '$.event' as event,
line ->> '$.timestamp' as ts
FROM lines_read('events.ndjson')
WHERE line ->> '$.event' = 'purchase'
LIMIT 10
''').fetchall()
for row in results:
print(f"User {row[0]}: {row[1]} at {row[2]}")
# Get loadable path for manual loading
print(sqlite_lines.loadable_path())
# '/path/to/site-packages/sqlite_lines/lines0'
```
### Response
#### Success Response (200)
This function does not return a value directly but modifies the connection to load the extension.
#### Response Example
None
```
--------------------------------
### Load sqlite-lines as a loadable extension in Python
Source: https://github.com/asg017/sqlite-lines/blob/main/README.md
Shows how to load the sqlite-lines extension within a Python script using the `sqlite3` module. It enables loading external extensions into an in-memory SQLite database.
```python
import sqlite3
con = sqlite3.connect(":memory:")
con.enable_load_extension(True)
con.load_extension("./lines0")
print(con.execute("select lines_version()").fetchone())
# ('v0.0.1',)
```
--------------------------------
### sqlite-lines CLI: Extracting JSON Fields
Source: https://context7.com/asg017/sqlite-lines/llms.txt
Demonstrates using the sqlite-lines CLI to extract specific JSON fields from NDJSON files. It shows how to target fields using the `->>` operator.
```bash
# Extract JSON fields from NDJSON
cat users.ndjson | sqlite-lines 'd ->> "$.name"'
# Alice
# Bob
# Carol
```
--------------------------------
### SQL Scalar Function: lines_debug()
Source: https://context7.com/asg017/sqlite-lines/llms.txt
Returns debugging information including version, build date, commit hash, and build configuration.
```APIDOC
## lines_debug()
### Description
Returns debugging information including version, build date, commit hash, and build configuration (e.g., whether filesystem access is disabled).
### Method
SQL Function
### Parameters
None
### Request Example
```sql
SELECT lines_debug();
```
### Response
#### Success Response (200)
- **debug_info** (text) - A multi-line string containing debugging information.
#### Response Example
```text
Version: v0.2.2
Date: 2022-05-15T16:57:23Z-0700
Commit: c87a67c6e76
NO FILESYSTEM
```
```
--------------------------------
### Insert NDJSON Objects into SQLite Table
Source: https://github.com/asg017/sqlite-lines/blob/main/benchmarks/README.md
Demonstrates the ability of sqlite-lines to transform NDJSON data and insert it into a flat SQLite table, contrasting its performance with sqlite-utils and DuckDB's approach.
```bash
# sqlite-lines for insertion
# cat data.ndjson | sqlite-lines --table my_table --create --pk id
# sqlite-utils for insertion (illustrative, as it's slower)
# sqlite-utils insert my_database.db my_table data.ndjson --detect-types
# DuckDB for insertion (different format, not directly comparable for SQLite)
# duckdb -c "CREATE TABLE my_table AS SELECT * FROM read_json_objects('data.ndjson');"
```
--------------------------------
### Table Functions
Source: https://github.com/asg017/sqlite-lines/blob/main/docs.md
Provides information about table functions available in sqlite-lines.
```APIDOC
## Table Functions
### `lines(document, [delimeter])`
**Description**: A table function that reads a given document (TEXT or BLOB) into memory and generates a row for each line.
**Method**: SQL
**Endpoint**: N/A
### Parameters
#### Path Parameters
None
#### Query Parameters
None
#### Request Body
- **document** (TEXT or BLOB) - Required - The document to read.
- **delimeter** (char) - Optional - The single character delimiter to split lines on. Defaults to newline (`\n`).
### Request Example
```sql
SELECT rowid, line FROM lines('a\nb\nc');
```
### Response
#### Success Response (200)
- **line** (text) - The content of the line.
- **rowid** (integer) - The line number, starting at 1.
#### Response Example
```json
{
"rowid": 1,
"line": "a"
}
{
"rowid": 2,
"line": "b"
}
{
"rowid": 3,
"line": "c"
}
```
### `lines_read(path, [delimeter])`
**Description**: A table function that reads a file from the given path and generates a row for each line.
**Method**: SQL
**Endpoint**: N/A
### Parameters
#### Path Parameters
None
#### Query Parameters
None
#### Request Body
- **path** (text) - Required - The path to the file to read.
- **delimeter** (char) - Optional - The single character delimiter to split lines on. Defaults to newline (`\n`).
### Request Example
```sql
SELECT * FROM lines_read("my-file.txt");
SELECT
line -> '$.id' as id,
line -> '$.name' as name
FROM lines_read("my-file.ndjson");
```
### Response
#### Success Response (200)
- **line** (text) - The content of the line.
- **path** (text) - The path of the file being read.
#### Response Example
```json
{
"line": "This is the first line.",
"path": "my-file.txt"
}
{
"line": "This is the second line.",
"path": "my-file.txt"
}
```
```
--------------------------------
### SQL Scalar Function: lines_version()
Source: https://context7.com/asg017/sqlite-lines/llms.txt
Returns the semantic version string of the loaded sqlite-lines extension.
```APIDOC
## lines_version()
### Description
Returns the semantic version string of the loaded sqlite-lines extension.
### Method
SQL Function
### Parameters
None
### Request Example
```sql
SELECT lines_version();
```
### Response
#### Success Response (200)
- **version** (text) - The semantic version string of the extension.
#### Response Example
```json
"v0.2.2"
```
```
--------------------------------
### Handle Large NDJSON File Parsing
Source: https://github.com/asg017/sqlite-lines/blob/main/benchmarks/README.md
Evaluates the capability of sqlite-lines to handle extremely large NDJSON datasets (5GB+) without running into memory limitations, specifically addressing the SQLite maximum blob size. Other tools that are too slow or memory-intensive are omitted.
```bash
# sqlite-lines is designed to handle large files by not loading the entire file into memory.
# The benchmark likely involves piping a large NDJSON file to sqlite-lines.
# Example (conceptual):
# cat large_dataset.ndjson | sqlite-lines -c "SELECT COUNT(*) FROM lines"
```
--------------------------------
### SQL Table Function: lines_read()
Source: https://context7.com/asg017/sqlite-lines/llms.txt
Reads a file from the filesystem and generates a row for each line. Streams the file without loading it entirely into memory.
```APIDOC
## lines_read(path, [delimiter])
### Description
Reads a file from the filesystem and generates a row for each line. Unlike `lines()`, this streams the file without loading it entirely into memory, allowing processing of files larger than SQLite's 1GB limit. Same return columns and delimiter options as `lines()`.
### Method
SQL Function
### Parameters
#### Path Parameters
None
#### Query Parameters
None
#### Request Body
None
### Request Example
```sql
-- Read all lines from a text file
SELECT rowid as line_number, line
FROM lines_read('data.txt');
-- Parse NDJSON file and extract fields
SELECT
line ->> '$.id' as id,
line ->> '$.name' as name,
line ->> '$.stats.score' as score
FROM lines_read('users.ndjson');
-- Aggregate NDJSON data: top 5 countries in a dataset
SELECT
line ->> '$.countrycode' as countrycode,
count(*) as cnt
FROM lines_read('./calendar.ndjson')
GROUP BY 1
ORDER BY 2 DESC
LIMIT 5;
-- Read lines from multiple files using fsdir()
SELECT
name as file,
lines.rowid as line_number,
line
FROM fsdir('logs')
JOIN lines_read(name) as lines
WHERE name LIKE '%.txt';
-- Filter specific line by rowid (optimized single-row lookup)
SELECT line FROM lines_read('config.txt') WHERE rowid = 5;
-- Use custom delimiter for pipe-separated files
SELECT line FROM lines_read('data.psv', '|');
```
### Response
#### Success Response (200)
- **rowid** (integer) - Line number starting at 1.
- **line** (text) - The content of the line.
#### Response Example
```json
{
"rowid": 1,
"line": "This is the first line."
}
```
```
--------------------------------
### Scalar Functions
Source: https://github.com/asg017/sqlite-lines/blob/main/docs.md
Provides information about scalar functions available in sqlite-lines.
```APIDOC
## Scalar Functions
### `lines_version()`
**Description**: Returns the version string of the `sqlite-lines` library.
**Method**: SQL
**Endpoint**: N/A
### Parameters
None
### Request Example
```sql
SELECT lines_version();
```
### Response
#### Success Response (200)
- **version** (string) - The version string of `sqlite-lines`.
#### Response Example
```json
{
"version": "v0.0.0"
}
```
### `lines_debug()`
**Description**: Returns a string with debugging information for `sqlite-lines`, including version, commit hash, and build date.
**Method**: SQL
**Endpoint**: N/A
### Parameters
None
### Request Example
```sql
SELECT lines_debug();
```
### Response
#### Success Response (200)
- **debug_info** (string) - A string containing debugging information.
#### Response Example
```json
{
"debug_info": "Version: v0.0.0\nDate: 2022-05-15T16:57:23Z-0700\nCommit: c87a67c6e76\nNO FILESYSTEM"
}
```
```
--------------------------------
### SQL Table Function: lines()
Source: https://context7.com/asg017/sqlite-lines/llms.txt
Reads a TEXT or BLOB value into memory and generates a row for each line. Supports custom delimiters.
```APIDOC
## lines(document, [delimiter])
### Description
Reads a TEXT or BLOB value into memory and generates a row for each line in the document. Returns `line` (text content) and `rowid` (line number starting at 1). The default delimiter is `\n` with automatic `\r` stripping for CRLF files. Custom single-character delimiters are supported.
### Method
SQL Function
### Parameters
#### Path Parameters
None
#### Query Parameters
None
#### Request Body
None
### Request Example
```sql
-- Basic usage: split a string into lines
SELECT rowid, line FROM lines('apple\nbanana\ncherry');
-- Custom delimiter: split by comma
SELECT rowid, line FROM lines('red,green,blue', ',');
-- Parse inline JSON array into rows
SELECT
line ->> '$.id' as id,
line ->> '$.name' as name
FROM lines('{"id":1,"name":"Alice"}\n{"id":2,"name":"Bob"}\n{"id":3,"name":"Carol"}');
```
### Response
#### Success Response (200)
- **rowid** (integer) - Line number starting at 1.
- **line** (text) - The content of the line.
#### Response Example
```json
{
"rowid": 1,
"line": "apple"
}
```
```
--------------------------------
### SQL: Read lines from a file using lines_read()
Source: https://context7.com/asg017/sqlite-lines/llms.txt
The `lines_read()` SQL function reads a file from the filesystem line by line. It streams the file content, avoiding memory limitations for large files. It returns the line content and its row number, similar to the `lines()` function.
```sql
-- Read all lines from a text file
SELECT rowid as line_number, line
FROM lines_read('data.txt');
-- Parse NDJSON file and extract fields
SELECT
line ->> '$.id' as id,
line ->> '$.name' as name,
line ->> '$.stats.score' as score
FROM lines_read('users.ndjson');
-- Aggregate NDJSON data: top 5 countries in a dataset
SELECT
line ->> '$.countrycode' as countrycode,
count(*) as cnt
FROM lines_read('./calendar.ndjson')
GROUP BY 1
ORDER BY 2 DESC
LIMIT 5;
/*
┌─────────────┬────────┐
│ countrycode │ cnt │
├─────────────┼────────┤
│ US │ 141001 │
│ GB │ 22560 │
│ CA │ 11759 │
│ RU │ 9250 │
│ DE │ 8748 │
└─────────────┴────────┘
*/
-- Read lines from multiple files using fsdir()
SELECT
name as file,
lines.rowid as line_number,
line
FROM fsdir('logs')
JOIN lines_read(name) as lines
WHERE name LIKE '%.txt';
-- Filter specific line by rowid (optimized single-row lookup)
SELECT line FROM lines_read('config.txt') WHERE rowid = 5;
-- Use custom delimiter for pipe-separated files
SELECT line FROM lines_read('data.psv', '|');
```
--------------------------------
### SQL: Read lines from text or blob with custom delimiter
Source: https://context7.com/asg017/sqlite-lines/llms.txt
The `lines()` SQL function supports custom single-character delimiters for splitting text or BLOB data. This allows processing data formats other than standard line-based files, such as CSV or pipe-separated values.
```sql
SELECT rowid, line FROM lines('red,green,blue', ',');
/*
┌───────┬───────┐
│ rowid │ line │
├───────┼───────┤
│ 1 │ red │
│ 2 │ green │
│ 3 │ blue │
└───────┴───────┘
*/
```
--------------------------------
### SQL: Read lines from text or blob with default delimiter
Source: https://context7.com/asg017/sqlite-lines/llms.txt
The `lines()` SQL function reads a TEXT or BLOB value and splits it into rows based on newline characters. It returns the line content and its row number. This is useful for processing in-memory strings or small files.
```sql
SELECT rowid, line FROM lines('apple
banana
cherry');
/*
┌───────┬────────┐
│ rowid │ line │
├───────┼────────┤
│ 1 │ apple │
│ 2 │ banana │
│ 3 │ cherry │
└───────┴────────┘
*/
```
=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.