### Install ison-go Library Source: https://github.com/ison-format/ison/blob/main/ison-go/README.md Use 'go get' to install the ison-go library. This command fetches and installs the specified package and its dependencies. ```bash go get github.com/maheshvaikri-code/ison/ison-go ``` -------------------------------- ### Install isonantic-go Source: https://github.com/ison-format/ison/blob/main/isonantic-go/README.md Use 'go get' to add the isonantic-go library to your project. ```bash go get github.com/maheshvaikri-code/ison/isonantic-go ``` -------------------------------- ### Install Qdrant Plugin Source: https://github.com/ison-format/ison/blob/main/ison-py/README.md Install the Qdrant client library. ```bash pip install qdrant-client ``` -------------------------------- ### Quick Start: Parse ISON with ES Modules Source: https://github.com/ison-format/ison/blob/main/ison-js/README.md Shows how to import and use the ISON parser in projects utilizing ES Modules. Assumes 'ison-parser' is installed. ```javascript import ISON from 'ison-parser'; const doc = ISON.loads(isonText); ``` -------------------------------- ### Install ISON-py Source: https://github.com/ison-format/ison/blob/main/ison-py/README.md Install the library using pip. ```bash pip install ison-py ``` -------------------------------- ### Install Pinecone Plugin Source: https://github.com/ison-format/ison/blob/main/ison-py/README.md Install the Pinecone client library. ```bash pip install pinecone-client ``` -------------------------------- ### Install ChromaDB Plugin Source: https://github.com/ison-format/ison/blob/main/ison-py/README.md Install the ChromaDB client library. ```bash pip install chromadb ``` -------------------------------- ### Configure ISON C++ Examples Source: https://github.com/ison-format/ison/blob/main/ison-cpp/CMakeLists.txt Adds the example executable if ISON_BUILD_EXAMPLES is enabled. ```cmake # Examples if(ISON_BUILD_EXAMPLES) add_executable(ison_example examples/example.cpp) target_link_libraries(ison_example PRIVATE ison_cpp) endif() ``` -------------------------------- ### Install ISONantic Source: https://github.com/ison-format/ison/blob/main/isonantic/README.md Install the library using pip. ```bash pip install isonantic ``` -------------------------------- ### Install Dependencies and Run Benchmark Source: https://github.com/ison-format/ison/blob/main/benchmark/README.md Installs necessary Python packages, sets up API keys if needed, and runs the benchmark. View logs using 'cat benchmark_300_latest.log'. ```bash # 1. Install dependencies pip install tiktoken pyyaml requests ison-py toon-llm # 2. Set your API key (for accuracy tests) # Edit benchmark_300.py and set DEEPSEEK_API_KEY # Or use your own LLM API # 3. Run the benchmark python benchmark_300.py # 4. View results cat benchmark_300_latest.log ``` -------------------------------- ### Compile and Run ISON Example Source: https://github.com/ison-format/ison/blob/main/ison-cpp/README.md Compile an example C++ file that uses the ISON format and then run the resulting executable. Ensure the include path is set correctly. ```bash g++ -std=c++17 -I include examples/example.cpp -o example ./example ``` -------------------------------- ### Install ISON Packages Source: https://github.com/ison-format/ison/blob/main/README.md Installation commands for various programming languages. ```bash npm install ison-parser # JavaScript npm install ison-ts # TypeScript with full types npm install isonantic-ts # Validation & schemas ``` ```bash pip install ison-py # Parser pip install isonantic # Validation & schemas ``` ```toml [dependencies] ison-rs = "1.0" isonantic-rs = "1.0" # Validation & schemas ``` ```bash # Just copy the header cp ison-cpp/include/ison_parser.hpp /your/project/ ``` ```bash go get github.com/maheshvaikri-code/ison/ison-go go get github.com/maheshvaikri-code/ison/isonantic-go ``` -------------------------------- ### Install ison-cpp via System Build Source: https://github.com/ison-format/ison/blob/main/ison-cpp/README.md Build and install the library system-wide using CMake. ```bash mkdir build && cd build cmake .. sudo make install ``` -------------------------------- ### Install Dependencies Source: https://github.com/ison-format/ison/blob/main/benchmark/README.md Install the required Python packages for running the benchmark. ```bash pip install tiktoken pyyaml requests ison-py toon-llm ``` -------------------------------- ### Quick Start: Define, Parse, and Validate ISON Data Source: https://github.com/ison-format/ison/blob/main/isonantic-rust/README.md This example demonstrates how to define a schema for user data, parse ISON text, validate the parsed document against the schema, and access the validated data. ```rust use isonantic_rs::prelude::*; use ison_rs::parse; fn main() -> Result<()> { // Define a schema let user_schema = table("users") .field("id", int().required()) .field("name", string().min(1).max(100)) .field("email", string().email()) .field("active", boolean().default_value(true)); // Parse ISON let ison_text = r#"# table.users id name email active 1 Alice alice@example.com true 2 Bob bob@example.com false "#; let doc = parse(ison_text).expect("Parse failed"); // Validate let users = user_schema.validate(&doc)?; // Access validated data for user in users.iter() { println!("{}: {}", user.get_int("id").unwrap(), user.get_string("name").unwrap() ); } Ok(()) } ``` -------------------------------- ### Install ison-ts Source: https://github.com/ison-format/ison/blob/main/ison-ts/README.md Installation commands for common package managers. ```bash npm install ison-ts # or yarn add ison-ts # or pnpm add ison-ts ``` -------------------------------- ### Install ison-cpp via Bash Source: https://github.com/ison-format/ison/blob/main/ison-cpp/README.md Copy the header file to your project directory. ```bash cp include/ison_parser.hpp /your/project/include/ ``` -------------------------------- ### Install SQLAlchemy Plugin Source: https://github.com/ison-format/ison/blob/main/ison-py/README.md Install the necessary packages for MySQL support. ```bash pip install sqlalchemy pymysql # For MySQL ``` -------------------------------- ### Install isonantic-ts Source: https://github.com/ison-format/ison/blob/main/isonantic-ts/README.md Installation commands for npm, yarn, and pnpm. ```bash npm install isonantic-ts ison-ts # or yarn add isonantic-ts ison-ts # or pnpm add isonantic-ts ison-ts ``` -------------------------------- ### Install ISON Extension from VSIX Source: https://github.com/ison-format/ison/blob/main/ison-vscode/README.md Commands to compile and install the ISON extension locally from the source repository. ```bash cd ison-vscode npm install npm run compile npm run package code --install-extension ison-lang-1.0.0.vsix ``` -------------------------------- ### Quick Start: Parse ISON in Browser Source: https://github.com/ison-format/ison/blob/main/ison-js/README.md Example of parsing ISON data directly in a web browser after including the parser via a script tag. Assumes the CDN script is loaded. ```javascript ``` -------------------------------- ### Install n8n-nodes-ison via npm Source: https://github.com/ison-format/ison/blob/main/n8n-nodes-ison/README.md Use this command to manually install the n8n-nodes-ison package. Ensure you have Node.js and npm installed. ```bash npm install n8n-nodes-ison ``` -------------------------------- ### ISON Reference Syntax Example Source: https://github.com/ison-format/ison/blob/main/benchmark/README.md Shows the usage of reference syntax within ISON. ```text # ISON (with references) :user:1 :product:101 1 1299.99 ``` -------------------------------- ### ISON Format Example Source: https://github.com/ison-format/ison/blob/main/benchmark/README.md Demonstrates the space-separated structure and reference syntax of ISON. ```text table.users id name email active 1 Alice alice@example.com true 2 Bob bob@example.com false table.orders id user_id product total 1001 :user:1 Widget 29.99 ``` -------------------------------- ### Configure ISON C++ Installation Source: https://github.com/ison-format/ison/blob/main/ison-cpp/CMakeLists.txt Defines installation paths, export targets, and package configuration files for the library. ```cmake # Installation if(ISON_INSTALL) include(GNUInstallDirs) install(TARGETS ison_cpp EXPORT ison_cppTargets INCLUDES DESTINATION ${CMAKE_INSTALL_INCLUDEDIR} ) install(FILES include/ison_parser.hpp DESTINATION ${CMAKE_INSTALL_INCLUDEDIR} ) install(EXPORT ison_cppTargets FILE ison_cppTargets.cmake NAMESPACE ison:: DESTINATION ${CMAKE_INSTALL_LIBDIR}/cmake/ison_cpp ) # Package config include(CMakePackageConfigHelpers) configure_package_config_file( ${CMAKE_CURRENT_SOURCE_DIR}/cmake/ison_cppConfig.cmake.in ${CMAKE_CURRENT_BINARY_DIR}/ison_cppConfig.cmake INSTALL_DESTINATION ${CMAKE_INSTALL_LIBDIR}/cmake/ison_cpp ) write_basic_package_version_file( ${CMAKE_CURRENT_BINARY_DIR}/ison_cppConfigVersion.cmake VERSION ${PROJECT_VERSION} COMPATIBILITY SameMajorVersion ) install(FILES ${CMAKE_CURRENT_BINARY_DIR}/ison_cppConfig.cmake ${CMAKE_CURRENT_BINARY_DIR}/ison_cppConfigVersion.cmake DESTINATION ${CMAKE_INSTALL_LIBDIR}/cmake/ison_cpp ) endif() ``` -------------------------------- ### Install Python Packages for Benchmark Source: https://github.com/ison-format/ison/blob/main/benchmark/README.md Installs essential Python libraries for token counting, YAML parsing, API requests, and ISON/TOON format handling. ```bash pip install tiktoken # Token counting (o200k_base tokenizer) pip install pyyaml # YAML format support pip install requests # API calls for LLM accuracy tests pip install ison-py # Official ISON parser pip install toon-llm # Official TOON parser ``` -------------------------------- ### Quick Start with ISON Source: https://github.com/ison-format/ison/blob/main/ison-ts/README.md Basic workflow for parsing ISON text, accessing data, and converting to JSON. ```typescript import { parse, dumps, fromDict, Reference } from 'ison-ts'; // Parse ISON text const isonText = ` table.users id:int name:string email active:bool 1 Alice alice@example.com true 2 Bob bob@example.com false `; const doc = parse(isonText); // Access blocks const users = doc.getBlock('users'); console.log(`Users: ${users.size()}`); // Access values with type checking for (const row of users.rows) { console.log(`${row.id}: ${row.name} (${row.active ? 'active' : 'inactive'})`); } // Convert to JSON const json = doc.toJson(); // Serialize back to ISON const isonOutput = dumps(doc); ``` -------------------------------- ### View Console Benchmark Results Source: https://github.com/ison-format/ison/blob/main/benchmark/README.md Example output format from the benchmark console. ```text ================================================================================ OVERALL RESULTS - ALL DATASETS COMBINED ================================================================================ Format Tokens vs JSON Accuracy Acc/1K TokWins AccWins ------------------------------------------------------------------------------------- >>> ISON 2,314 +73.0% 100.0% 43.22 9 9 TOON 2,996 +65.1% 100.0% 33.38 0 9 JSON Compact 4,952 +42.3% 95.6% 19.30 0 8 JSON 8,578 +0.0% 93.3% 10.88 0 7 ``` -------------------------------- ### Benchmark Execution Commands Source: https://github.com/ison-format/ison/blob/main/benchmark/BENCHMARK_300.md Commands for installing dependencies and running the ISON benchmark suite in various modes. ```bash # Install dependencies pip install tiktoken pyyaml requests ison-py toon-llm # Run full benchmark (takes ~45 minutes) python benchmark_v2.py # Run token-only mode (no API calls) python benchmark_v2.py --no-accuracy # Run dry run (10 questions only) python benchmark_v2.py --dry-run ``` -------------------------------- ### Install isonantic.hpp via Copy Source: https://github.com/ison-format/ison/blob/main/isonantic-cpp/README.md Copy the header file to your project's include directory for header-only integration. ```bash cp include/isonantic.hpp /your/project/include/ ``` -------------------------------- ### Install ISON Parser via npm Source: https://github.com/ison-format/ison/blob/main/ison-js/README.md Install the ISON parser package using npm. This is the standard method for Node.js projects. ```bash npm install ison-parser ``` -------------------------------- ### Example User Data in ISON Format Source: https://github.com/ison-format/ison/blob/main/benchmark/README.md Illustrates a small user dataset with ID, name, email, and active status, formatted using ISON. ```text table.users id name email active 1 Alice alice@example.com true 2 Bob bob@example.com false 3 Charlie charlie@example.com true ``` -------------------------------- ### Install ison-cpp via CMake Source: https://github.com/ison-format/ison/blob/main/ison-cpp/README.md Add the library as a subdirectory and link it to your target. ```cmake add_subdirectory(ison-parser-cpp) target_link_libraries(your_target PRIVATE ison::parser) ``` -------------------------------- ### ISONL Format: Streaming Example Source: https://github.com/ison-format/ison/blob/main/ison-js/README.md Provides a concrete example of ISONL data, illustrating how multiple records can be streamed, each on a new line, representing individual data entries. ```text table.users|id name email|1 Alice alice@example.com table.users|id name email|2 Bob bob@example.com ``` -------------------------------- ### Run Development Commands Source: https://github.com/ison-format/ison/blob/main/README.md Commands for cloning, installing dependencies, and running tests across different ecosystems. ```bash # Clone the repository git clone https://github.com/maheshvaikri-code/ison.git cd ison # JavaScript/TypeScript cd ison-js && npm install && npm test cd ison-ts && npm install && npm test cd isonantic-ts && npm install && npm test # Python cd ison-py && pip install -e . && pytest cd isonantic && pip install -e . && pytest # Rust cd ison-rust && cargo test cd isonantic-rust && cargo test # C++ cd ison-cpp && mkdir build && cd build && cmake .. && cmake --build . && ctest # Go cd ison-go && go test -v ./... cd isonantic-go && go test -v ./... ``` -------------------------------- ### CLI Usage Source: https://github.com/ison-format/ison/blob/main/ison-py/README.md Examples of using the ISON command-line interface for format conversion and validation. ```APIDOC ## CLI Usage ### Description This section provides examples of how to use the `ison` command-line tool for common tasks like converting between JSON and ISON formats, and validating ISON files. ### Commands - `ison input.json -o output.ison`: Convert a JSON file to ISON format. - `ison input.ison --to-json -o output.json`: Convert an ISON file to JSON format. - `ison input.ison --validate`: Validate the syntax and structure of an ISON file. ``` -------------------------------- ### ISON Format Syntax Example Source: https://context7.com/ison-format/ison/llms.txt Demonstrates the tabular structure of ISON, including block headers, field definitions, null values, and reference syntax. ```ison # Comments start with # table.users id:int name:string email active:bool 1 Alice alice@example.com true 2 "Bob Smith" bob@example.com false 3 ~ ~ true # ~ or null for null values table.orders id user_id product 1 :1 Widget # :1 = reference to id 1 2 :user:42 Gadget # :user:42 = namespaced reference 3 :MEMBER_OF:10 Sprocket # :MEMBER_OF:10 = relationship reference object.config key value debug true ``` -------------------------------- ### Quick Start: Define Schema, Parse, and Validate ISON Source: https://github.com/ison-format/ison/blob/main/isonantic-cpp/README.md Demonstrates defining a user schema, parsing ISON text, validating the document against the schema, and accessing validated data. Includes basic error handling for validation failures. ```cpp #include "isonantic.hpp" #include "ison_parser.hpp" // ison-cpp using namespace isonantic; int main() { // Define schema auto user_schema = table("users") .field("id", integer().required()) .field("name", string().min(1).max(100)) .field("email", string().email()) .field("active", boolean().default_value(true)); // Parse ISON auto doc = ison::parse(ison_text); // Validate try { auto users = user_schema.validate(doc); // Access validated data for (const auto& user : users) { std::cout << user.get_int("id").value() << ": " << user.get_string("name").value() << std::endl; } } catch (const ValidationError& e) { std::cerr << "Validation failed:" << std::endl; for (const auto& err : e.errors) { std::cerr << " " << err.field << ": " << err.message << std::endl; } } return 0; } ``` -------------------------------- ### Quick Start with ISON Parsing Source: https://github.com/ison-format/ison/blob/main/isonantic-ts/README.md Define schemas, parse ISON text, and perform type-safe access. ```typescript import { i, document, ValidationError } from 'isonantic-ts'; import { parse } from 'ison-ts'; // Define schemas const UserSchema = i.table('users', { id: i.int(), name: i.string().min(1), email: i.string().email(), active: i.boolean().default(true), }); const OrderSchema = i.table('orders', { id: i.string(), user_id: i.ref(), total: i.number().positive(), }); // Create document schema const DocSchema = document({ users: UserSchema, orders: OrderSchema, }); // Parse and validate ISON const isonText = ` table.users id name email active 1 Alice alice@example.com true 2 Bob bob@example.com false table.orders id user_id total O1 :1 99.99 O2 :2 149.50 `; const doc = parse(isonText); const validated = DocSchema.parse(doc.toDict()); // Type-safe access validated.users[0].name; // string validated.orders[0].total; // number ``` -------------------------------- ### TOON Format Example Source: https://github.com/ison-format/ison/blob/main/benchmark/README.md Demonstrates the comma-separated structure of TOON. ```text users[2]{id,name,email,active}: 1,Alice,alice@example.com,true 2,Bob,bob@example.com,false ``` -------------------------------- ### Test Results Output Source: https://github.com/ison-format/ison/blob/main/ison-rust/README.md Example output from running the test suite. ```text running 9 tests test tests::test_dumps_with_delimiter ... ok test tests::test_isonl ... ok test tests::test_ison_to_json ... ok test tests::test_json_to_ison ... ok test tests::test_parse_references ... ok test tests::test_parse_simple_table ... ok test tests::test_roundtrip ... ok test tests::test_type_inference ... ok test tests::test_version ... ok test result: ok. 9 passed; 0 failed; 0 ignored Doc-tests ison_rs test result: ok. 1 passed; 0 failed; 1 ignored ``` -------------------------------- ### ISON Format: Objects Example Source: https://github.com/ison-format/ison/blob/main/ison-js/README.md Shows the ISON format for key-value pairs, similar to JSON objects. Useful for configuration settings or simple data structures. ```text object.config timeout 30 debug true api_key "sk-xxx" ``` -------------------------------- ### Example ISON Format Source: https://github.com/ison-format/ison/blob/main/benchmark/README.md A basic representation of data in the ISON format. ```json {"user_id": 1, "product_id": 101, "quantity": 1, "total": 1299.99} ``` -------------------------------- ### Example E-commerce Data with ISON References Source: https://github.com/ison-format/ison/blob/main/benchmark/README.md Demonstrates e-commerce data including users, products, and orders, utilizing ISON references for relationships between tables. ```text table.users id name email premium 1 "Alice Johnson" alice@shop.com true table.products id name price stock category 101 "Laptop Pro" 1299.99 50 Electronics table.orders id user_id product_id quantity total 1001 :user:1 :product:101 1 1299.99 ``` -------------------------------- ### Creating Documents Programmatically Source: https://github.com/ison-format/ison/blob/main/ison-rust/README.md Guide on how to construct ISON documents in memory using the library's types. ```APIDOC ## Creating Documents Programmatically ### Description Build ISON `Document` objects from scratch in your Rust code. ### Core Types - `Document`: The top-level container for all ISON data. - `Block`: Represents a table, object, or meta section within a document. - `FieldInfo`: Stores metadata about a field, including its name and type. - `Value`: Represents the data within a field. ### Steps 1. Create a new `Document`. 2. Create `Block` instances, specifying their kind and name. 3. Define `fields` and `field_info` for each block. 4. Populate `rows` for table blocks (each row is a `HashMap`). 5. Add blocks to the `Document`. ### Request Example ```rust use ison_parser::{Document, Block, FieldInfo, Value, dumps}; use std::collections::HashMap; // Create a new document let mut doc = Document::new(); // Create a 'users' table block let mut users_block = Block::new("table", "users"); users_block.fields = vec!["id".to_string(), "name".to_string(), "active".to_string()]; users_block.field_info = vec![ FieldInfo::with_type("id", "int"), FieldInfo::with_type("name", "string"), FieldInfo::with_type("active", "bool"), ]; // Add a row to the users block let mut row1 = HashMap::new(); row1.insert("id".to_string(), Value::Int(1)); row1.insert("name".to_string(), Value::String("Alice".to_string())); row1.insert("active".to_string(), Value::Bool(true)); users_block.rows.push(row1); // Add another row let mut row2 = HashMap::new(); row2.insert("id".to_string(), Value::Int(2)); row2.insert("name".to_string(), Value::String("Bob".to_string())); row2.insert("active".to_string(), Value::Bool(false)); users_block.rows.push(row2); // Add the block to the document doc.blocks.push(users_block); // Serialize the document to ISON let ison_output = dumps(&doc, true); println!("{}", ison_output); ``` ``` -------------------------------- ### Quick Start: Parse ISON and Access Data Source: https://github.com/ison-format/ison/blob/main/ison-go/README.md Demonstrates parsing ISON data from a string, accessing specific blocks and rows, and converting the parsed document to JSON. Includes basic error handling. ```go package main import ( "fmt" "github.com/maheshvaikri-code/ison/ison-go" ) func main() { // Parse ISON doc, err := ison.Parse(` table.users id:int name:string email active:bool 1 Alice alice@example.com true 2 Bob bob@example.com false `) if err != nil { panic(err) } // Access data users, _ := doc.Get("users") for _, row := range users.Rows { name, _ := row["name"].AsString() active, _ := row["active"].AsBool() fmt.Printf("%s (active: %v)\n", name, active) } // Convert to JSON jsonStr, _ := doc.ToJSON() fmt.Println(jsonStr) } ``` -------------------------------- ### ISON Format Quick Reference Source: https://github.com/ison-format/ison/blob/main/n8n-nodes-ison/README.md A quick reference guide to the ISON format, illustrating syntax for comments, block headers, fields with types, data rows, and references. ```ison # Comments start with # table.users # Block header id:int name:string email active:bool # Fields with optional types 1 Alice alice@example.com true # Data rows 2 "Bob Smith" bob@example.com false # Quoted strings for spaces table.orders id user_id product 1 :1 Widget # :1 = reference to id 1 2 :user:42 Gadget # :user:42 = namespaced ref ``` -------------------------------- ### Quick Start: Define and Validate ISON Document Source: https://github.com/ison-format/ison/blob/main/isonantic-go/README.md Define schemas for users and orders, create a document schema, parse ISON data, and validate it against the defined schema. Handles basic types, string constraints, and ISON references. ```go package main import ( "fmt" ison "github.com/maheshvaikri-code/ison/ison-go" "github.com/maheshvaikri-code/ison/isonantic-go" ) func main() { // Define schemas using the I namespace (like Zod's z) userSchema := isonantic.I.Table("users", map[string]isonantic.Schema{ "id": isonantic.I.Int(), "name": isonantic.I.String().Min(1).Max(100), "email": isonantic.I.String().Email(), "active": isonantic.I.Bool().Default(true), }) orderSchema := isonantic.I.Table("orders", map[string]isonantic.Schema{ "id": isonantic.I.String(), "user_id": isonantic.I.Ref(), "total": isonantic.I.Number().Positive(), }) // Create document schema docSchema := isonantic.Document(map[string]isonantic.Schema{ "users": userSchema, "orders": orderSchema, }) // Parse ISON doc, _ := ison.Parse(` table.users id name email active 1 Alice alice@example.com true 2 Bob bob@example.com false table.orders id user_id total O1 :1 99.99 O2 :2 149.50 `) // Validate result := docSchema.SafeParse(doc.ToDict()) if result.Success { fmt.Println("Document is valid!") } else { fmt.Println("Validation errors:", result.Error) } } ``` -------------------------------- ### Convert Between ISON and ISONL Formats Source: https://github.com/ison-format/ison/blob/main/ison-ts/README.md Demonstrates converting between ISONL and ISON formats using the 'ison-ts' library. Ensure 'ison-ts' is installed. ```typescript import { isonToIsonl, isonlToIson } from 'ison-ts'; // Convert between formats const isonl = isonToIsonl(isonText); const ison = isonlToIson(isonlText); ``` -------------------------------- ### Example Graph Data with ISON References Source: https://github.com/ison-format/ison/blob/main/benchmark/README.md Shows graph data with nodes and edges, using ISON references to define relationships like 'works_at' and 'knows' between nodes. ```text table.nodes id type name 1 person Alice 2 person Bob 3 company "Acme Corp" table.edges source target relation :node:1 :node:3 works_at :node:1 :node:2 knows ``` -------------------------------- ### ISONL Streaming Format Source: https://github.com/ison-format/ison/blob/main/ison-py/README.md Examples of parsing ISONL strings and streaming large ISONL files efficiently. ```APIDOC ## ISONL Streaming Format ### Description Shows how to parse ISONL (Interchange Simple Object Notation Lines) text, which is suitable for streaming and fine-tuning datasets. It also demonstrates how to process large ISONL files line by line using a generator, which is memory-efficient. ### Method ```python from ison_parser import loads_isonl, dumps_isonl, isonl_stream # Parse ISONL isonl_text = """ table.examples|instruction response|"Summarize this" "Brief summary..." table.examples|instruction response|"Translate to Spanish" "Hola mundo" """ doc = loads_isonl(isonl_text) # Stream large files (constant memory) with open("large_dataset.isonl", "r") as f: for record in isonl_stream(f): process(record) ``` ### Parameters - `isonl_text` (str): A string containing data in ISONL format. - `f` (file object): An open file object pointing to an ISONL file. ### Response - `doc` (Document): A Document object representing the parsed ISONL data. - `record` (ISONLRecord): An object representing a single record from the ISONL stream. ``` -------------------------------- ### Convert Between ISON and JSON Source: https://github.com/ison-format/ison/blob/main/ison-go/README.md Provides examples for converting ISON data to a JSON string and for parsing a JSON string into an ISON document structure. ```go // ISON to JSON jsonStr, err := ison.ToJSON(` table.users id:int name:string 1 Alice 2 Bob `) // JSON to ISON Document doc, err := ison.FromJSON(`{ "users": [ {"id": 1, "name": "Alice"}, {"id": 2, "name": "Bob"} ] }`) output := ison.Dumps(doc) ``` -------------------------------- ### Install ISON Parser via CDN Source: https://github.com/ison-format/ison/blob/main/ison-js/README.md Include the ISON parser in your HTML using a CDN link for browser-based projects. ```html ``` -------------------------------- ### ISON Format: References Example Source: https://github.com/ison-format/ison/blob/main/ison-js/README.md Demonstrates the use of native references in ISON for relational data. The ':ID' syntax allows linking records, similar to foreign keys in databases. ```text table.orders id customer_id total O1 :C1 99.99 O2 :C2 149.50 ``` -------------------------------- ### Ecommerce Dataset Reference Syntax Source: https://github.com/ison-format/ison/blob/main/benchmark/BENCHMARK_300.md Example of using ISON reference syntax to represent relational data in an orders table. ```text table.orders id customer_id product_id quantity total status 1001 :customer:1 :product:101 1 1299.99 shipped 1002 :customer:2 :product:102 2 59.98 delivered ``` -------------------------------- ### Quick Start: Parse ISON in Node.js Source: https://github.com/ison-format/ison/blob/main/ison-js/README.md Demonstrates parsing an ISON string and accessing data within the parsed document in a Node.js environment. Requires the 'ison-parser' package. ```javascript const ISON = require('ison-parser'); // Parse ISON const isonText = ` table.users id name email 1 Alice alice@example.com 2 Bob bob@example.com `; const doc = ISON.loads(isonText); console.log(doc.blocks[0].rows[0].name); // "Alice" // Convert to JSON const jsonData = doc.toDict(); ``` -------------------------------- ### FromDict with Options Source: https://github.com/ison-format/ison/blob/main/ison-go/README.md Shows how to create an ISON document from a Go map, with options for automatic reference creation and smart ordering. ```APIDOC ## FromDict with Options ```go data := map[string]interface{}{ "users": []interface{}{ map[string]interface{}{"id": 1, "name": "Alice"}, }, } // Basic conversion doc := ison.FromDict(data) // With auto-refs and smart ordering opts := ison.FromDictOptions{ AutoRefs: true, // Convert foreign keys to references SmartOrder: true, // Reorder columns for optimal LLM comprehension } doc := ison.FromDictWithOptions(data, opts) ``` ``` -------------------------------- ### Use ISON MCP Server and Client Source: https://github.com/ison-format/ison/blob/main/ison-py/README.md Initialize the MCP server and client for AI assistant communication. ```python from ison_parser.integrations import ISONMCPServer, ISONMCPClient # Server exposes: parse_ison, format_ison, validate_ison, query_ison server = ISONMCPServer() # Client with local fallback async with ISONMCPClient() as client: result = await client.parse_ison(ison_text) ``` -------------------------------- ### JSON Format Example Source: https://github.com/ison-format/ison/blob/main/benchmark/README.md Standard JSON representation for comparison. ```json { "users": [ {"id": 1, "name": "Alice", "email": "alice@example.com", "active": true} ] } ``` -------------------------------- ### ISONL Format Specification Source: https://github.com/ison-format/ison/blob/main/ison-py/README.md Example of the ISONL line-based record format. ```text kind.name|field1 field2 field3|value1 value2 value3 ``` ```text table.users|id name email|1 Alice alice@example.com table.users|id name email|2 Bob bob@example.com table.orders|id user total|O1 :1 99.99 ``` -------------------------------- ### ISONL Conversion Source: https://github.com/ison-format/ison/blob/main/ison-go/README.md Provides examples for converting between ISON and ISONL formats. ```APIDOC ## ISONL Conversion ### Convert ISON to ISONL ```go isonlText, err := ison.ISONToISONL(isonText) ``` ### Convert ISONL to ISON ```go isonText, err := ison.ISONLToISON(isonlText) ``` ``` -------------------------------- ### Build Project with Visual Studio Source: https://github.com/ison-format/ison/blob/main/ison-cpp/README.md Commands to create a build directory, configure the project using CMake for Visual Studio, and build the project in Release configuration. ```cmd mkdir build cd build cmake .. -G "Visual Studio 17 2022" cmake --build . --config Release ``` -------------------------------- ### Stream Data with ISONL Source: https://github.com/ison-format/ison/blob/main/ison-vscode/README.md Example of the ISONL streaming format for representing data records. ```isonl table.users|id:int name:string|1 Alice table.users|id:int name:string|2 Bob table.orders|id:int user_id:ref product|1 :1 Widget ``` -------------------------------- ### File I/O Operations Source: https://github.com/ison-format/ison/blob/main/ison-go/README.md Demonstrates how to load and save ISON and ISONL files using the library. ```APIDOC ## File I/O ### Load from file ```go doc, err := ison.Load("data.ison") ``` ### Save to file ```go err := ison.Dump(doc, "output.ison") ``` ### Load/Save ISONL files ```go doc, err := ison.LoadISONL("stream.isonl") err := ison.DumpISONL(doc, "stream.isonl") ``` ``` -------------------------------- ### Install isonantic-cpp via CMake Source: https://github.com/ison-format/ison/blob/main/isonantic-cpp/README.md Integrate isonantic-cpp into your CMake project by adding the subdirectory and linking the library. ```cmake add_subdirectory(isonantic-cpp) target_link_libraries(your_target PRIVATE isonantic::isonantic) ``` -------------------------------- ### Programmatically Create and Serialize ISON Document Source: https://github.com/ison-format/ison/blob/main/ison-go/README.md Shows how to build an ISON document structure in Go by creating blocks, adding fields with types, and adding rows with values, then serializing it to an ISON string. ```go // Create document programmatically doc := ison.NewDocument() block := ison.NewBlock("table", "users") block.AddField("id", "int") block.AddField("name", "string") block.AddRow(ison.Row{ "id": ison.Int(1), "name": ison.String("Alice"), }) doc.AddBlock(block) // Serialize to ISON output := ison.Dumps(doc) fmt.Println(output) // Output: // table.users // id:int name:string // 1 Alice ``` -------------------------------- ### Clone Repository Source: https://github.com/ison-format/ison/blob/main/benchmark/README.md Commands to clone the ISON repository and navigate to the benchmark directory. ```bash git clone https://github.com/maheshvaikri-code/ison.git cd ison/benchmark ``` -------------------------------- ### Define ISON Data Structure Source: https://github.com/ison-format/ison/blob/main/README.md Example of defining tables, objects, and data rows with type annotations and references. ```text # Comments start with # table.users # Block: kind.name id:int name:string email active:bool # Fields with optional types 1 Alice alice@example.com true # Data rows (space-separated) 2 "Bob Smith" bob@example.com false # Quoted strings for spaces 3 ~ ~ true # ~ or null for null values table.orders id user_id product 1 :1 Widget # :1 = reference to id 1 2 :user:42 Gadget # :user:42 = namespaced reference object.config # Single-row object block key value debug true --- # Summary separator count 100 # Summary row ``` -------------------------------- ### Parse and Use ISON References Source: https://github.com/ison-format/ison/blob/main/ison-go/README.md Demonstrates parsing ISON data containing references, extracting reference details like ID and Namespace, and checking if it's a relationship reference. ```go // Parse references doc, _ := ison.Parse(` table.orders id user_id product 1 :1 Widget 2 :user:42 Gadget 3 :OWNS:5 Gizmo `) orders, _ := doc.Get("orders") for _, row := range orders.Rows { if ref, ok := row["user_id"].AsRef(); ok { fmt.Printf("ID: %s\n", ref.ID) fmt.Printf("Namespace: %s\n", ref.Namespace) fmt.Printf("Is Relationship: %v\n", ref.IsRelationship()) fmt.Printf("ISON format: %s\n", ref.ToISON()) } } ``` -------------------------------- ### Perform File I/O with ISON Source: https://github.com/ison-format/ison/blob/main/ison-go/README.md Use these functions to load and save ISON or ISONL files from the filesystem. ```go // Load from file doc, err := ison.Load("data.ison") // Save to file err := ison.Dump(doc, "output.ison") // Load/Save ISONL files doc, err := ison.LoadISONL("stream.isonl") err := ison.DumpISONL(doc, "stream.isonl") ``` -------------------------------- ### Social Graph Dataset Reference Syntax Source: https://github.com/ison-format/ison/blob/main/benchmark/BENCHMARK_300.md Example of using ISON reference syntax to represent edges in a social graph. ```text table.edges source target relation :node:1 :node:2 follows :node:1 :node:3 follows ``` -------------------------------- ### Build ison-cpp with CMake Source: https://github.com/ison-format/ison/blob/main/ison-cpp/README.md Standard build process for the project using CMake. ```bash mkdir build && cd build cmake .. make # Run tests ./test_ison_parser # Run example ./ison_example ``` -------------------------------- ### Handle ISON References Source: https://github.com/ison-format/ison/blob/main/ison-cpp/README.md Create and inspect simple, namespaced, or relationship references. ```cpp // Simple reference :42 Reference ref("42"); ref.id; // "42" ref.to_ison(); // ":42" // Namespaced reference :user:101 Reference ref("101", "user"); ref.id; // "101" ref.type; // "user" ref.get_namespace(); // "user" ref.is_relationship(); // false // Relationship reference :MEMBER_OF:10 Reference ref("10", "MEMBER_OF"); ref.is_relationship(); // true ref.relationship_type(); // "MEMBER_OF" ``` -------------------------------- ### Define ISON Data Structures Source: https://github.com/ison-format/ison/blob/main/ison-vscode/README.md Example of defining tables and objects in the ISON format, including type annotations and references. ```ison # Users table with typed fields table.users id:int name:string email:string active:bool 1 Alice alice@example.com true 2 Bob bob@example.com false # Orders with references table.orders id:int user_id:ref product amount:float 1 :1 Widget 29.99 2 :2 Gadget 49.99 # Configuration object object.config key value debug true version 1.0.0 ``` -------------------------------- ### Parse and Access ISON Data Source: https://github.com/ison-format/ison/blob/main/README.md Examples of parsing ISON strings and accessing row data across different programming languages. ```javascript import { parse, dumps, toJSON } from 'ison-parser'; const doc = parse(` table.users id:int name:string active:bool 1 Alice true 2 Bob false `); console.log(doc.users.rows); // [{ id: 1, name: 'Alice', active: true }, ...] console.log(toJSON(doc)); // Standard JSON output ``` ```python from ison_py import parse, dumps, to_json doc = parse(""" table.users id:int name:string active:bool 1 Alice true 2 Bob false """) for row in doc['users']['rows']: print(f"{row['id']}: {row['name']}") # Convert to JSON print(to_json(doc)) ``` ```rust use ison_rs::{parse, dumps}; let doc = parse(r#" table.users id:int name:string active:bool 1 Alice true 2 Bob false "#)?; let users = doc.get("users").unwrap(); for row in &users.rows { let name = row.get("name").and_then(|v| v.as_str()).unwrap(); println!("{}", name); } ``` ```cpp #include "ison_parser.hpp" auto doc = ison::parse(R"( table.users id:int name:string active:bool 1 Alice true 2 Bob false )"); for (const auto& row : doc["users"].rows) { std::cout << ison::as_string(row.at("name")) << std::endl; } ``` ```go import "github.com/maheshvaikri-code/ison/ison-go" doc, _ := ison.Parse(` table.users id:int name:string active:bool 1 Alice true 2 Bob false `) users, _ := doc.Get("users") for _, row := range users.Rows { name, _ := row["name"].AsString() fmt.Println(name) } ``` -------------------------------- ### Streaming ISONL with Channels Source: https://github.com/ison-format/ison/blob/main/ison-go/README.md Illustrates how to stream ISONL records using Go channels for efficient processing of large files. ```APIDOC ## Streaming ISONL with Channels ```go file, _ := os.Open("large_data.isonl") defer file.Close() // Stream records via channel for record := range ison.ISONLStream(file) { name, _ := record.Values["name"].AsString() fmt.Printf("Processing: %s\n", name) } ``` ``` -------------------------------- ### Python: Load ISON from File and Dump Source: https://context7.com/ison-format/ison/llms.txt Load ISON data from a file and dump it to another file with column alignment. ```python # Load from file doc = load('data.ison') dump(doc, 'output.ison', align_columns=True) ``` -------------------------------- ### ISON Format: Tables Example Source: https://github.com/ison-format/ison/blob/main/ison-js/README.md Illustrates the structure of ISON tables, which are used for representing structured data with rows and columns. Includes headers and data rows. ```text table.users id name email active 1 Alice alice@example.com true 2 Bob bob@example.com false ``` -------------------------------- ### Create and Check ISON Value Types Source: https://github.com/ison-format/ison/blob/main/ison-go/README.md Illustrates creating various ISON value types (null, bool, int, float, string, ref) and performing type checks and safe conversions using methods like IsNull() and AsInt(). ```go // Create values nullVal := ison.Null() boolVal := ison.Bool(true) intVal := ison.Int(42) floatVal := ison.Float(3.14) strVal := ison.String("hello") refVal := ison.Ref(ison.Reference{ID: "1", Namespace: "user"}) // Check types if nullVal.IsNull() { fmt.Println("Value is null") } // Safe type conversion if i, ok := intVal.AsInt(); ok { fmt.Printf("Integer: %d\n", i) } // Get raw interface{} raw := intVal.Interface() // returns int64(42) ``` -------------------------------- ### Compile and Run ISON Parser Test Source: https://github.com/ison-format/ison/blob/main/ison-cpp/README.md Compile the ISON parser test file using g++ and then run the executable. Ensure you have the necessary include paths. ```bash g++ -std=c++17 -I include tests/test_ison_parser.cpp -o test_ison ./test_ison ``` -------------------------------- ### Define Table Schema for Orders Source: https://github.com/ison-format/ison/blob/main/isonantic-rust/README.md Example of defining a table schema named 'orders' with various field types including string, reference, float, and string with a default value. ```rust let schema = table("orders") .field("id", string().required()) .field("user_id", reference().required()) .field("total", float().positive()) .field("status", string().default_value("pending")); let orders = schema.validate(&doc)?; ``` -------------------------------- ### Run ISON MCP Server Source: https://github.com/ison-format/ison/blob/main/ison-py/README.md Execute the MCP server module. ```bash python -m ison_parser.integrations.mcp_server ``` -------------------------------- ### Accessing Validated Data with Type-Safe Getters Source: https://github.com/ison-format/ison/blob/main/isonantic-rust/README.md Shows how to retrieve data from validated ISON rows using type-specific getter methods like `get_int`, `get_string`, and `get_bool`. A generic `get` method is also available. ```rust let users = schema.validate(&doc)?; // Iterate rows for row in users.iter() { // Type-safe getters let id: Option = row.get_int("id"); let name: Option<&str> = row.get_string("name"); let active: Option = row.get_bool("active"); // Generic getter let value: Option<&ValidatedValue> = row.get("field"); } // Index access let first_user = &users[0]; ``` -------------------------------- ### ISON Format Syntax Reference Source: https://github.com/ison-format/ison/blob/main/ison-cpp/README.md Illustrates the basic syntax of the ISON format, including comments, block headers, field definitions, data rows, null values, and references. ```ison # Comment table.users # Block header: kind.name id:int name:string active:bool # Field definitions with optional types 1 Alice true # Data rows 2 "Bob Smith" false # Quoted strings for spaces 3 ~ null # null values (~ or null) table.orders id user_id product 1 :1 Widget # :1 = reference to id 1 2 :user:42 Gadget # :user:42 = namespaced reference 3 :MEMBER_OF:10 Thing # :MEMBER_OF:10 = relationship reference object.config # Single-row object block key value debug true --- Total 100 # Summary row ``` -------------------------------- ### View Results Source: https://github.com/ison-format/ison/blob/main/benchmark/README.md Display the contents of the latest benchmark log file. ```bash cat benchmark_300_latest.log ``` -------------------------------- ### References Source: https://github.com/ison-format/ison/blob/main/ison-rust/README.md How to work with ISON references, including simple, namespaced, and relationship references. ```APIDOC ## References ### Description Handle ISON references, which are used to link to other data points. ### Reference Types - **Simple Reference**: e.g., `:42` - **Namespaced Reference**: e.g., `:user:101` - **Relationship Reference**: e.g., `:MEMBER_OF:10` ### `Reference` Struct Methods - `Reference::new(id: &str) -> Reference`: Creates a simple reference. - `Reference::with_type(id: &str, type_name: &str) -> Reference`: Creates a namespaced or relationship reference. - `ref.to_ison() -> String`: Converts the reference back to its ISON string representation. - `ref.get_namespace() -> Option<&str>`: Gets the namespace (type) of the reference. - `ref.is_relationship() -> bool`: Checks if the reference represents a relationship. - `ref.relationship_type() -> Option<&str>`: Gets the type of relationship if it's a relationship reference. ### Request Example ```rust use ison_parser::Reference; // Simple reference let ref1 = Reference::new("42"); println!("Simple ref ISON: {}", ref1.to_ison()); // Output: :42 // Namespaced reference let ref2 = Reference::with_type("101", "user"); println!("Namespaced ref ISON: {}", ref2.to_ison()); // Output: :user:101 println!("Namespace: {:?}", ref2.get_namespace()); // Output: Some("user") // Relationship reference let ref3 = Reference::with_type("10", "MEMBER_OF"); println!("Relationship ref ISON: {}", ref3.to_ison()); // Output: :MEMBER_OF:10 println!("Is relationship: {}", ref3.is_relationship()); // Output: true println!("Relationship type: {:?}", ref3.relationship_type()); // Output: Some("MEMBER_OF") ``` ```