### Installing Tables.jl Package in Julia Source: https://github.com/juliadata/tables.jl/blob/main/README.md This snippet demonstrates how to install the Tables.jl package in a Julia environment. It uses Julia's built-in package manager, Pkg, to add the 'Tables' package to the current environment, making its functionalities available for use. ```Julia import Pkg; Pkg.add("Tables") ``` -------------------------------- ### Defining Row and Column Tables for Round Trip Testing Source: https://github.com/juliadata/tables.jl/blob/main/docs/src/implementing-the-interface.md Creates example standard "row" and "column" tables using a Vector of NamedTuples and a NamedTuple of Vectors, respectively. These serve as common Tables.jl sources for performing round trip tests with sink functions. ```julia rt = [(a=1, b=4.0, c="7"), (a=2, b=5.0, c="8"), (a=3, b=6.0, c="9")] ct = (a=[1,2,3], b=[4.0, 5.0, 6.0]) ``` -------------------------------- ### Helper Function to Get Vector - Julia Source: https://github.com/juliadata/tables.jl/blob/main/docs/src/using-the-interface.md This is a helper function used in the DataFrames.jl Tables.jl implementation. It returns the input directly if it's already an `AbstractVector`, otherwise it collects the input into a new vector. ```julia getvector(x::AbstractVector) = x getvector(x) = collect(x) ``` -------------------------------- ### Accessing Tables.jl Data Row-by-Row and Column-by-Column (Julia) Source: https://github.com/juliadata/tables.jl/blob/main/docs/src/using-the-interface.md This snippet demonstrates the two primary ways to access data from a Tables.jl-compatible source: row-by-row using `Tables.rows` and column-by-column using `Tables.columns`. It shows basic iteration over rows and columns, and how to retrieve individual values or entire columns. ```julia # access data of input table `x` row-by-row # Tables.rows must return a row iterator rows = Tables.rows(x) # we can iterate through each row for row in rows # example of getting all values in the row # don't worry, there are other ways to more efficiently process rows rowvalues = [Tables.getcolumn(row, col) for col in Tables.columnnames(row)] end # access data of input table `x` column-by-column # Tables.columns returns an object where individual, entire columns can be accessed columns = Tables.columns(x) # iterate through each column name in table for col in Tables.columnnames(columns) # retrieve entire column by column name # a column is an indexable collection # with known length (i.e. supports # `length(column)` and `column[i]`) column = Tables.getcolumn(columns, col) end ``` -------------------------------- ### Performing Round Trip Tests with Tables.jl Sink Functions Source: https://github.com/juliadata/tables.jl/blob/main/docs/src/implementing-the-interface.md Demonstrates converting different table sources (row table, column table, matrix) into other table types or standard Julia structures using sink functions like `Tables.matrix`, `columntable`, and `rowtable`, and verifies the results using `@test` assertions. ```julia # let's turn our row table into a plain Julia Matrix object mat = Tables.matrix(rt) # test that our matrix came out like we expected @test mat[:, 1] == [1, 2, 3] @test size(mat) == (3, 3) @test eltype(mat) == Any # so we successfully consumed a row-oriented table, # now let's try with a column-oriented table mat2 = Tables.matrix(ct) @test eltype(mat2) == Float64 @test mat2[:, 1] == ct.a # now let's take our matrix input, and make a column table out of it tbl = Tables.table(mat) |> columntable @test keys(tbl) == (:Column1, :Column2, :Column3) @test tbl.Column1 == [1, 2, 3] # and same for a row table tbl2 = Tables.table(mat2) |> rowtable @test length(tbl2) == 3 @test map(x->x.Column1, tbl2) == [1.0, 2.0, 3.0] ``` -------------------------------- ### Testing Tables.jl Interface for a MatrixTable Source Source: https://github.com/juliadata/tables.jl/blob/main/docs/src/implementing-the-interface.md Demonstrates how to test a custom table source type (`MatrixTable`) using the Tables.jl API methods like `istable`, `rowaccess`, `columnaccess`, `getcolumn`, `columnnames`, and property access (`mattbl.Column1`) to ensure correct interface implementation. ```julia # first, create a MatrixTable from our matrix input mattbl = Tables.table(mat) # test that the MatrixTable `istable` @test Tables.istable(typeof(mattbl)) # test that it defines row access @test Tables.rowaccess(typeof(mattbl)) @test Tables.rows(mattbl) === mattbl # test that it defines column access @test Tables.columnaccess(typeof(mattbl)) @test Tables.columns(mattbl) === mattbl # test that we can access the first "column" of our matrix table by column name @test mattbl.Column1 == [1,2,3] # test our `Tables.AbstractColumns` interface methods @test Tables.getcolumn(mattbl, :Column1) == [1,2,3] @test Tables.getcolumn(mattbl, 1) == [1,2,3] @test Tables.columnnames(mattbl) == [:Column1, :Column2, :Column3] # now let's iterate our MatrixTable to get our first MatrixRow matrow = first(mattbl) @test eltype(mattbl) == typeof(matrow) # now we can test our `Tables.AbstractRow` interface methods on our MatrixRow @test matrow.Column1 == 1 @test Tables.getcolumn(matrow, :Column1) == 1 @test Tables.getcolumn(matrow, 1) == 1 @test propertynames(mattbl) == propertynames(matrow) == [:Column1, :Column2, :Column3] ``` -------------------------------- ### Loading Table Data into SQLite (Unknown Schema) - Julia Source: https://github.com/juliadata/tables.jl/blob/main/docs/src/using-the-interface.md This function handles loading data from a Tables.jl compatible source into SQLite when the schema is initially unknown (`Tables.schema` returns `nothing`). It iterates the first row to determine column names, constructs a partial `Tables.Schema`, and then proceeds to iterate all rows using `Tables.eachcolumn` to bind values to an INSERT statement within a transaction. ```julia function load!(sch::Nothing, rows, db::SQLite.DB, tablename) # sch is nothing === unknown schema # start iteration on input table rows state = iterate(rows) state === nothing && return row, st = state # query column names of first row names = Tables.columnnames(row) # partially construct Tables.Schema by at least passing # the column names to it sch = Tables.Schema(names, nothing) # create table if needed createtable!(db, tablename, sch) # build insert statement params = chop(repeat("?,", length(names))) stmt = Stmt(db, "INSERT INTO $nm VALUES ($params)") # start a transaction for inserting rows transaction(db) do while true # just like before, we can still use `Tables.eachcolumn` # even with our partially constructed Tables.Schema # to apply a function to each value in the row Tables.eachcolumn(sch, row) do val, i, nm bind!(stmt, i, val) end sqlite3_step(stmt.handle) sqlite3_reset(stmt.handle) # keep iterating rows until we finish state = iterate(rows, st) state === nothing && break row, st = state end end return name end ``` -------------------------------- ### Loading Table Data into SQLite (Known Schema) - Julia Source: https://github.com/juliadata/tables.jl/blob/main/docs/src/using-the-interface.md This function loads data from a Tables.jl compatible source into a SQLite database table. It assumes the schema is known and provided by `Tables.schema`. It iterates through rows and uses `Tables.eachcolumn` to bind values to an INSERT statement within a transaction. ```julia function load!(table, db::SQLite.DB, tablename) # get input table rows rows = Tables.rows(table) # query for schema of data sch = Tables.schema(rows) # create table using tablename and schema from input table createtable!(db, tablename, sch) # build insert statement params = chop(repeat("?,", length(sch.names))) stmt = Stmt(db, "INSERT INTO $tablename VALUES ($params)") # start a transaction for inserting rows transaction(db) do # iterate over rows in the input table for row in rows # Tables.jl provides a utility function # Tables.eachcolumn, which allows efficiently # applying a function to each column value in a row # it's called with a schema and row, and applies # a user-provided function to the column value `val`, index `i` # and column name `nm`. Here, we bind the row values # to our parameterized SQL INSERT statement and then # call `sqlite3_step` to execute the INSERT statement. Tables.eachcolumn(sch, row) do val, i, nm bind!(stmt, i, val) end sqlite3_step(stmt.handle) sqlite3_reset(stmt.handle) end end return end ``` -------------------------------- ### DataFrame Construction from Tables.jl Source (Julia) Source: https://github.com/juliadata/tables.jl/blob/main/docs/src/using-the-interface.md Defines functions to construct a DataFrame from a Tables.jl compatible source. It extracts columns and names using Tables.jl functions and dispatches to an internal `fromcolumns` function, handling potential column copying based on the source type. ```Julia # note that copycols is ignored in this definition (Tables.CopiedColumns implies copies have already been made) fromcolumns(x::Tables.CopiedColumns, names; copycols::Bool=true) = DataFrame(AbstractVector[getvector(Tables.getcolumn(x, nm) for nm in names], Index(names), copycols=false) fromcolumns(x, names; copycols::Bool=true) = DataFrame(AbstractVector[getvector(Tables.getcolumn(x, nm) for nm in names], Index(names), copycols=copycols) function DataFrame(x; copycols::Bool=true) # get columns from input table source cols = Tables.columns(x) # get column names as Vector{Symbol}, which is required # by core DataFrame constructor names = collect(Symbol, Tables.columnnames(cols)) return fromcolumns(cols, names; copycols=copycols) end ``` -------------------------------- ### Defining a Julia Matrix for Tables.jl Testing Source: https://github.com/juliadata/tables.jl/blob/main/docs/src/implementing-the-interface.md Defines a 3x3 Julia matrix literal with mixed data types (Integer, Float, String) to be used as input for testing custom Tables.jl source implementations. ```julia mat = [1 4.0 "7"; 2 5.0 "8"; 3 6.0 "9"] ``` -------------------------------- ### Implement Tables.schema for MatrixTable (Julia) Source: https://github.com/juliadata/tables.jl/blob/main/docs/src/implementing-the-interface.md Defines basic getter methods (`names`, `matrix`, `lookup`) to access internal fields of a `MatrixTable` instance without conflicting with `getproperty`. It then implements the `Tables.schema` method from the Tables.jl interface, returning a `Tables.Schema` object containing column names and their common element type. ```Julia names(m::MatrixTable) = getfield(m, :names) matrix(m::MatrixTable) = getfield(m, :matrix) lookup(m::MatrixTable) = getfield(m, :lookup) # schema is column names and types Tables.schema(m::MatrixTable{T}) where {T} = Tables.Schema(names(m), fill(eltype(T), size(matrix(m), 2))) ``` -------------------------------- ### Implement Tables.AbstractColumns Interface for MatrixTable (Julia) Source: https://github.com/juliadata/tables.jl/blob/main/docs/src/implementing-the-interface.md Implements the `Tables.columnaccess` and `Tables.columns` methods, indicating that `MatrixTable` provides column access and returns itself as the columns object. It then defines the required `Tables.getcolumn` methods for accessing columns by type, name, or index, and the `Tables.columnnames` method to return the column names. ```Julia # column interface Tables.columnaccess(::Type{<:MatrixTable}) = true Tables.columns(m::MatrixTable) = m # required Tables.AbstractColumns object methods Tables.getcolumn(m::MatrixTable, ::Type{T}, col::Int, nm::Symbol) where {T} = matrix(m)[:, col] Tables.getcolumn(m::MatrixTable, nm::Symbol) = matrix(m)[:, lookup(m)[nm]] Tables.getcolumn(m::MatrixTable, i::Int) = matrix(m)[:, i] Tables.columnnames(m::MatrixTable) = names(m) ``` -------------------------------- ### Defining MatrixTable Type and Declaring it as a Table (Julia) Source: https://github.com/juliadata/tables.jl/blob/main/docs/src/implementing-the-interface.md This snippet defines a Julia struct `MatrixTable` that wraps an `AbstractVecOrMat` to serve as a custom table type. It also implements the `Tables.istable` method for `MatrixTable`, declaring that this type conforms to the Tables.jl interface. ```Julia struct MatrixTable{T <: AbstractVecOrMat} <: Tables.AbstractColumns names::Vector{Symbol} lookup::Dict{Symbol, Int} matrix::T end # declare that MatrixTable is a table Tables.istable(::Type{<:MatrixTable}) = true ``` -------------------------------- ### Implement Tables.rows Interface and MatrixRow Type for MatrixTable (Julia) Source: https://github.com/juliadata/tables.jl/blob/main/docs/src/implementing-the-interface.md Implements the `Tables.rowaccess` and `Tables.rows` methods, indicating that `MatrixTable` provides row access and returns itself as the rows iterator. It defines the iteration interface (`eltype`, `length`, `iterate`) for `MatrixTable`, using a custom `MatrixRow` type as the element type. The `MatrixRow` struct is defined to represent a single row view, and the required `Tables.AbstractRow` methods (`getcolumn`, `columnnames`) are implemented for this type. ```Julia # declare that any MatrixTable defines its own \`Tables.rows\` method rowaccess(::Type{<:MatrixTable}) = true # just return itself, which means MatrixTable must iterate \`Tables.AbstractRow\`-compatible objects rows(m::MatrixTable) = m # the iteration interface, at a minimum, requires \`eltype\`, \`length\`, and \`iterate\` # for \`MatrixTable\` \`eltype\`, we're going to provide a custom row type Base.eltype(m::MatrixTable{T}) where {T} = MatrixRow{T} Base.length(m::MatrixTable) = size(matrix(m), 1) Base.iterate(m::MatrixTable, st=1) = st > length(m) ? nothing : (MatrixRow(st, m), st + 1) # a custom row type; acts as a "view" into a row of an AbstractVecOrMat struct MatrixRow{T} <: Tables.AbstractRow row::Int source::MatrixTable{T} end # required \`Tables.AbstractRow\` interface methods (same as for \`Tables.AbstractColumns\` object before) # but this time, on our custom row type getcolumn(m::MatrixRow, ::Type, col::Int, nm::Symbol) = getfield(getfield(m, :source), :matrix)[getfield(m, :row), col] getcolumn(m::MatrixRow, i::Int) = getfield(getfield(m, :source), :matrix)[getfield(m, :row), i] getcolumn(m::MatrixRow, nm::Symbol) = getfield(getfield(m, :source), :matrix)[getfield(m, :row), getfield(getfield(m, :source), :lookup)[nm]] columnnames(m::MatrixRow) = names(getfield(m, :source)) ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.