### Example: Create GDS file and get folders of nodes Source: https://github.com/zhengxwen/gdsfmt/blob/master/man/getfolder.gdsn.Rd This example creates a GDS file, adds several nodes (label, double, list, data.frame), and then uses getfolder.gdsn on various nodes to retrieve their folders. It also demonstrates getting the root folder and cleaning up by closing the file and deleting the temporary file. ```R # cteate a GDS file f <- createfn.gds("test.gds") add.gdsn(f, "label", NULL) add.gdsn(f, "double", seq(1, 1000, 0.4)) add.gdsn(f, "list", list(X=1:10, Y=seq(1, 10, 0.25))) add.gdsn(f, "data.frame", data.frame(X=1:19, Y=seq(1, 10, 0.5))) f getfolder.gdsn(index.gdsn(f, "label")) getfolder.gdsn(index.gdsn(f, "double")) getfolder.gdsn(index.gdsn(f, "list/X")) getfolder.gdsn(index.gdsn(f, "data.frame/Y")) getfolder.gdsn(f$root) # close the GDS file closefn.gds(f) # delete the temporary file unlink("test.gds", force=TRUE) ``` -------------------------------- ### Create a GDS file and add a list Source: https://github.com/zhengxwen/gdsfmt/blob/master/man/name.gdsn.Rd Creates a GDS file 'test.gds' and adds a list with elements x, y, and z. This is the setup for the subsequent name.gdsn examples. ```R f <- createfn.gds("test.gds") # add a list to "test.gds" add.gdsn(f, name="list", val=list(x=c(1,2), y=c("T","B","C"), z=TRUE)) node <- index.gdsn(f, "list/x") ``` -------------------------------- ### gdsRegisterCloudHandler example Source: https://github.com/zhengxwen/gdsfmt/blob/master/_autodocs/api-reference/cloud-handlers.md Example of registering a cloud handler for a custom scheme 'myfs' and then unregistering it. ```r library(gdsfmt) my_open <- function(filename, ...) { # assume a companion package implements this stop("Not implemented, this is only an example.") } old <- gdsRegisterCloudHandler("myfs", my_open, pkgname="myPkg") gdsCloudHandlers() try(openfn.gds("myfs://bucket/file.gds")) # forwards to my_open() gdsUnregisterCloudHandler("myfs") ``` -------------------------------- ### index.gdsn usage example Source: https://github.com/zhengxwen/gdsfmt/blob/master/_autodocs/api-reference/structure-operations.md Example showing how to create a GDS file, add a folder and a node, then retrieve the node using index.gdsn. Also demonstrates the silent parameter returning NULL for a missing node. ```r f <- createfn.gds("test.gds") folder <- addfolder.gdsn(f, "dir") add.gdsn(folder, "a", 1:10) n <- index.gdsn(f, "dir/a") # the "a" node n2 <- index.gdsn(f, "dir/nope", silent=TRUE) # NULL closefn.gds(f) ``` -------------------------------- ### Example usage of readex.gdsn Source: https://github.com/zhengxwen/gdsfmt/blob/master/_autodocs/api-reference/read-and-apply.md Example demonstrating how to create a GDS file, add a vector and a matrix, and use readex.gdsn with various selection and substitution options. ```r f <- createfn.gds("test.gds") add.gdsn(f, "vector", 1:128) add.gdsn(f, "matrix", matrix(as.character(1:(10*6)), nrow=10)) readex.gdsn(index.gdsn(f, "vector"), sel=rep(c(TRUE, FALSE), 64)) readex.gdsn(index.gdsn(f, "vector"), sel=c(4, 1, 10, NA, 12, NA), .value=c(NA, 1, 12), .substitute=c(6, 7, NA)) readex.gdsn(index.gdsn(f, "matrix"), sel=list(rep(c(TRUE, FALSE), 5), c(1, 3, 6))) closefn.gds(f) ``` -------------------------------- ### Example: create GDS file, add attributes, copy attributes, and clean up Source: https://github.com/zhengxwen/gdsfmt/blob/master/man/put.attr.gdsn.Rd Complete example from the documentation. It creates a GDS file, adds an integer node, sets attributes of different types, retrieves and deletes an attribute, copies attributes from one node to another, then closes the file and deletes the temporary file. ```R # cteate a GDS file f <- createfn.gds("test.gds") node <- add.gdsn(f, "int", val=1:10000) put.attr.gdsn(node, "missing.value", 10000) put.attr.gdsn(node, "one.value", 1L) put.attr.gdsn(node, "string", c("ABCDEF", "THIS", paste(letters, collapse=""))) put.attr.gdsn(node, "bool", c(TRUE, TRUE, FALSE)) f get.attr.gdsn(node) delete.attr.gdsn(node, "one.value") get.attr.gdsn(node) node2 <- add.gdsn(f, "char", val=letters) get.attr.gdsn(node2) put.attr.gdsn(node2, val=node) get.attr.gdsn(node2) # close the GDS file closefn.gds(f) # delete the temporary file unlink("test.gds", force=TRUE) ``` -------------------------------- ### Create GDS file, add node, set attribute, and get attributes Source: https://github.com/zhengxwen/gdsfmt/blob/master/man/get.attr.gdsn.Rd This example creates a GDS file 'test.gds', adds an integer node 'int' with values 1:10000, sets an attribute 'missing.value' to 10000, and then retrieves the attributes with get.attr.gdsn. It also closes the file and deletes the temporary file. ```R # cteate a GDS file f <- createfn.gds("test.gds") node <- add.gdsn(f, "int", val=1:10000) put.attr.gdsn(node, "missing.value", 10000) f get.attr.gdsn(node) # close the GDS file closefn.gds(f) # delete the temporary file unlink("test.gds", force=TRUE) ``` -------------------------------- ### getfolder.gdsn usage example Source: https://github.com/zhengxwen/gdsfmt/blob/master/_autodocs/api-reference/structure-operations.md Example showing how to create a GDS file, add a node, and retrieve its parent folder using getfolder.gdsn. ```r f <- createfn.gds("test.gds") n <- add.gdsn(f, "x", 1:10) parent <- getfolder.gdsn(n) # equals f$root closefn.gds(f) ``` -------------------------------- ### moveto.gdsn example: relocate nodes in a GDS file Source: https://github.com/zhengxwen/gdsfmt/blob/master/man/moveto.gdsn.Rd This example creates a GDS file, adds nodes of various types, and demonstrates the moveto.gdsn function with different relpos options: after, before, replace, replace+rename, and into. It also shows how to move a node into a different folder and closes the file. ```R # cteate a GDS file f <- createfn.gds("test.gds") L <- -2500:2499 # commom types add.gdsn(f, "label", NULL) add.gdsn(f, "int", 1:10000, compress="ZIP", closezip=TRUE) add.gdsn(f, "int.matrix", matrix(L, nrow=100, ncol=50)) add.gdsn(f, "double", seq(1, 1000, 0.4)) add.gdsn(f, "character", c("int", "double", "logical", "factor")) f # + [ ] # |--+ label # |--+ int { Int32 10000 ZIP(34.74%) } # |--+ int.matrix { Int32 100x50 } # |--+ double { Float64 2498 } # |--+ character { VStr8 4 } n1 <- index.gdsn(f, "label") n2 <- index.gdsn(f, "double") moveto.gdsn(n1, n2, relpos="after") f moveto.gdsn(n1, n2, relpos="before") f moveto.gdsn(n1, n2, relpos="replace") f n2 <- index.gdsn(f, "int") moveto.gdsn(n1, n2, relpos="replace+rename") f # move a node into a different folder addfolder.gdsn(f, "sub") moveto.gdsn(index.gdsn(f, "int.matrix"), index.gdsn(f, "sub"), relpos="into") f # close the GDS file closefn.gds(f) # delete the temporary file unlink("test.gds", force=TRUE) ``` -------------------------------- ### apply.gdsn usage examples Source: https://github.com/zhengxwen/gdsfmt/blob/master/_autodocs/api-reference/read-and-apply.md A comprehensive example demonstrating column means, indexed iteration with selection, writing results to a target node, and iterating over multiple variables in lockstep. ```r f <- createfn.gds("test.gds") n <- add.gdsn(f, "matrix", val=matrix(1:(10*6), nrow=10)) # column means v <- apply.gdsn(n, margin=2, FUN=function(x) mean(x), as.is="double") # iterate with an index over a selection apply.gdsn(n, margin=1, var.index="relative", selection=list(rep(c(TRUE, FALSE), 5), rep(c(TRUE, FALSE), 3)), FUN=function(i, x) { cat("index:", i, "\n"); x }, as.is="none") # write results into a target node (matrix transpose) dst <- add.gdsn(f, "transpose", storage="int", valdim=c(6, 0)) apply.gdsn(n, margin=1, FUN=`c`, as.is="gdsnode", target.node=dst) readmode.gdsn(dst) read.gdsn(dst) # multiple variables in lockstep X <- add.gdsn(f, "X", matrix(1:50, nrow=10)) Y <- add.gdsn(f, "Y", matrix((1:50)/100, nrow=10)) v <- apply.gdsn(list(X=X, Y=Y), margin=c(1, 1), FUN=function(x) sum(x), as.is="double") closefn.gds(f) ``` -------------------------------- ### setdim.gdsn usage example Source: https://github.com/zhengxwen/gdsfmt/blob/master/_autodocs/api-reference/data-operations.md Example showing how to create a GDS file, add a node with 12 elements, resize it to a 4x3 matrix, and read the result. ```r f <- createfn.gds("test.gds") n <- add.gdsn(f, "n", 1:12) setdim.gdsn(n, c(4, 3)) read.gdsn(n) closefn.gds(f) ``` -------------------------------- ### readmode.gdsn example Source: https://github.com/zhengxwen/gdsfmt/blob/master/_autodocs/api-reference/data-operations.md Example showing how to finalize a compressed stream by calling readmode.gdsn after adding a node with compression. ```r n <- add.gdsn(f, "x", 1:1000, compress="ZIP") readmode.gdsn(n) # finalize the compressed stream ``` -------------------------------- ### Create GDS file and add node Source: https://github.com/zhengxwen/gdsfmt/blob/master/man/unload.gdsn.Rd Creates a GDS file named 'test.gds' and adds a node 'val' with values 1:1000. This is the setup for the unload.gdsn example. ```R f <- createfn.gds("test.gds") node <- add.gdsn(f, "val", 1:1000) node ``` -------------------------------- ### Create a GDS file and add a list node Source: https://github.com/zhengxwen/gdsfmt/blob/master/man/ls.gdsn.Rd Creates a GDS file named 'test.gds' and adds a list node with elements x, y, and z. This is the setup for the subsequent ls.gdsn examples. ```R # cteate a GDS file f <- createfn.gds("test.gds") # add a list to "test.gds" node <- add.gdsn(f, name="list", val=list(x=c(1,2), y=c("T","B","C"), z=TRUE)) ``` -------------------------------- ### Create GDS file and add data Source: https://github.com/zhengxwen/gdsfmt/blob/master/man/read.gdsn.Rd Creates a GDS file and adds a vector, list, data.frame, and matrix to it. This is the setup for the subsequent read.gdsn examples. ```R # cteate a GDS file f <- createfn.gds("test.gds") add.gdsn(f, "vector", 1:128) add.gdsn(f, "list", list(X=1:10, Y=seq(1, 10, 0.25))) add.gdsn(f, "data.frame", data.frame(X=1:19, Y=seq(1, 10, 0.5))) add.gdsn(f, "matrix", matrix(1:12, ncol=4)) f ``` -------------------------------- ### permdim.gdsn usage example Source: https://github.com/zhengxwen/gdsfmt/blob/master/_autodocs/api-reference/data-operations.md Example showing how to create a GDS file, add a 10x10 matrix node, transpose it in-place using permdim.gdsn, and read the result. ```r f <- createfn.gds("test.gds") n <- add.gdsn(f, "mat", matrix(1:100, 10, 10)) permdim.gdsn(n, c(2, 1)) # in-place transpose read.gdsn(n) closefn.gds(f) ``` -------------------------------- ### Example: Using digest.gdsn to create and verify hash digests Source: https://github.com/zhengxwen/gdsfmt/blob/master/man/digest.gdsn.Rd This example creates a GDS file, adds raw and integer nodes, computes hash digests with different algorithms and actions, verifies data integrity, and cleans up. It requires the 'digest' and 'tools' packages. Note that sha384 may not work with digest_0.6.11. ```R library(digest) library(tools) # cteate a GDS file f <- createfn.gds("test.gds") val <- as.raw(rep(1:128, 1024)) n1 <- add.gdsn(f, "raw1", val) n2 <- add.gdsn(f, "int1", as.integer(val)) n3 <- add.gdsn(f, "int2", as.integer(val), compress="ZIP", closezip=TRUE) digest.gdsn(n1) digest.gdsn(n1, action="Robject") digest.gdsn(n1, action="add") digest.gdsn(n1, action="add.Robj") writeBin(read.gdsn(n1, .useraw=TRUE), con="test1.bin") write.gdsn(n1, 0, start=1027, count=1) digest.gdsn(n1, action="add") digest.gdsn(n1, action="add.Robj") digest.gdsn(n1, "sha1", action="add") digest.gdsn(n1, "sha256", action="add") # digest.gdsn(n1, "sha384", action="add") ## digest_0.6.11 does not work digest.gdsn(n1, "sha512", action="add") writeBin(read.gdsn(n1, .useraw=TRUE), con="test2.bin") print(n1, attribute=TRUE) digest.gdsn(n1, action="verify") digest.gdsn(n1, action="clear") print(n1, attribute=TRUE) digest.gdsn(n2) digest.gdsn(n2, action="Robject") # using R object digest.gdsn(n2) == digest.gdsn(n3) # FALSE digest.gdsn(n2, action="Robject") == digest.gdsn(n3, action="Robject") # TRUE # close the GDS file closefn.gds(f) # check with other program md5sum(c("test1.bin", "test2.bin")) # delete the temporary file unlink(c("test.gds", "test1.bin", "test2.bin"), force=TRUE) ``` -------------------------------- ### gdsCloudHandlers example Source: https://github.com/zhengxwen/gdsfmt/blob/master/_autodocs/api-reference/cloud-handlers.md Example of listing registered cloud handlers. ```r h <- gdsCloudHandlers() if (nrow(h) > 0L) print(h) ``` -------------------------------- ### Install development version of gdsfmt from Github Source: https://github.com/zhengxwen/gdsfmt/blob/master/vignettes/gdsfmt.Rmd Install the development version of gdsfmt from Github using devtools. For developers/testers only. Requires building from source, so make and compilers must be installed on your system. ```r library("devtools") install_github("zhengxwen/gdsfmt") ``` -------------------------------- ### Example: show a matrix node Source: https://github.com/zhengxwen/gdsfmt/blob/master/_autodocs/api-reference/classes-and-methods.md Creates a GDS file, adds a matrix, and shows the node. Demonstrates the S4 show method for gdsn.class. ```r f <- createfn.gds("test.gds") n <- add.gdsn(f, "mat", matrix(1:100, 10, 10)) show(n) # or simply `n` closefn.gds(f) ``` -------------------------------- ### Example: create and print a GDS file Source: https://github.com/zhengxwen/gdsfmt/blob/master/_autodocs/api-reference/classes-and-methods.md Creates a GDS file, adds an integer vector, and prints the file object. Demonstrates the print method for gds.class. ```r f <- createfn.gds("test.gds") add.gdsn(f, "int", 1:100) print(f) ``` -------------------------------- ### addfolder.gdsn usage example Source: https://github.com/zhengxwen/gdsfmt/blob/master/_autodocs/api-reference/structure-operations.md Example showing how to create a directory folder and a virtual folder in a GDS file. The virtual folder links to another GDS file via gds.fn. ```r f <- createfn.gds("test.gds") dir <- addfolder.gdsn(f, "dir") add.gdsn(dir, "x", 1:10) vf <- addfolder.gdsn(f, "vdir", type="virtual", gds.fn="other.gds") closefn.gds(f) ``` -------------------------------- ### write.gdsn usage example Source: https://github.com/zhengxwen/gdsfmt/blob/master/_autodocs/api-reference/data-operations.md Example showing how to create a GDS file, add a matrix node, overwrite a 2x1 block at rows 2-3, column 2 with NA values, then replace the entire data field with a new matrix. The example closes the file at the end. ```r f <- createfn.gds("test.gds") n <- add.gdsn(f, "mat", matrix(1:20, ncol=5)) # overwrite a 2x1 block at rows 2-3, column 2 write.gdsn(n, val=c(NA, NA), start=c(2, 2), count=c(2, 1)) # replace everything write.gdsn(n, matrix(1:24, ncol=6)) closefn.gds(f) ``` -------------------------------- ### addfile.gdsn usage example Source: https://github.com/zhengxwen/gdsfmt/blob/master/_autodocs/api-reference/structure-operations.md Example showing how to embed a file into a GDS node and extract it back using getfile.gdsn. ```r f <- createfn.gds("test.gds") writeLines("hello", "tmp.txt") n <- addfile.gdsn(f, "tmp.txt", filename="tmp.txt", compress="ZIP") getfile.gdsn(n, "out.txt") # extract it back closefn.gds(f) ``` -------------------------------- ### append.gdsn usage example Source: https://github.com/zhengxwen/gdsfmt/blob/master/_autodocs/api-reference/data-operations.md Example showing how to create a GDS file, add an empty integer vector node, append data to it, and append another node's data as a fast raw block copy. The example closes the file at the end. ```r f <- createfn.gds("test.gds") n <- add.gdsn(f, "n", storage="int", valdim=c(0)) # empty vector append.gdsn(n, 1:100) n2 <- add.gdsn(f, "n2", storage="int", valdim=c(0)) append.gdsn(n2, n, allow.block=TRUE) # fast raw block copy closefn.gds(f) ``` -------------------------------- ### compression.gdsn example Source: https://github.com/zhengxwen/gdsfmt/blob/master/_autodocs/api-reference/data-operations.md Example showing how to compress a node's data field with ZIP compression. After calling compression.gdsn, the node is in write mode; readmode.gdsn closes the stream and finishes compressing. ```r f <- createfn.gds("test.gds") n <- add.gdsn(f, "int.matrix", matrix(1:50*100, nrow=100, ncol=50)) compression.gdsn(n, "ZIP") readmode.gdsn(n) closefn.gds(f) ``` -------------------------------- ### objdesp.gdsn usage example Source: https://github.com/zhengxwen/gdsfmt/blob/master/_autodocs/api-reference/structure-operations.md Creates a GDS file, adds a compressed integer node, retrieves its description, and prints storage and compression fields. Closes the file at the end. ```r f <- createfn.gds("test.gds") n <- add.gdsn(f, "x", 1:100, compress="ZIP.max") dp <- objdesp.gdsn(n) dp$storage # "int32" dp$compress # "ZIP.max" closefn.gds(f) ``` -------------------------------- ### showfile.gds example Source: https://github.com/zhengxwen/gdsfmt/blob/master/_autodocs/api-reference/file-operations.md Example usage of showfile.gds. Creates two GDS files, then calls showfile.gds() to print and return a data.frame of open files, and showfile.gds(closeall=TRUE) to close all open files. ```r f1 <- createfn.gds("a.gds") f2 <- createfn.gds("b.gds") df <- showfile.gds() # prints and returns the table showfile.gds(closeall=TRUE) # closes everything ``` -------------------------------- ### createfn.gds example: basic creation and duplicate-check relaxed Source: https://github.com/zhengxwen/gdsfmt/blob/master/_autodocs/api-reference/file-operations.md Example usage of createfn.gds. The first block creates a file with default settings, adds a node, and closes it. The second block creates a file with allow.duplicate=TRUE and use.abspath=FALSE, then prints the filename as given (not the absolute path) and closes it. ```r library(gdsfmt) # basic creation f <- createfn.gds("test.gds") add.gdsn(f, "x", val=1:100) closefn.gds(f) # create with duplicate-check relaxed f2 <- createfn.gds("test2.gds", allow.duplicate=TRUE, use.abspath=FALSE) print(f2$filename) # as given, not the absolute path closefn.gds(f2) ``` -------------------------------- ### Example: using exist.gdsn to test paths Source: https://github.com/zhengxwen/gdsfmt/blob/master/_autodocs/api-reference/structure-operations.md Creates a GDS file, adds a variable 'x', and tests for the existence of 'x' and 'y'. The first call returns TRUE, the second FALSE. ```r f <- createfn.gds("test.gds") add.gdsn(f, "x", 1:10) exist.gdsn(f, "x") # TRUE exist.gdsn(f, "y") # FALSE closefn.gds(f) ``` -------------------------------- ### Install gdsfmt and dependencies in R Source: https://github.com/zhengxwen/gdsfmt/blob/master/README.md Install the R packages getopt, optparse, crayon, and gdsfmt (via BiocManager) needed for the GDS command-line tools. Run inside an R environment. ```R install.packages("getopt", repos="http://cran.r-project.org") install.packages("optparse", repos="http://cran.r-project.org") install.packages("crayon", repos="http://cran.r-project.org") if (!requireNamespace("BiocManager", quietly=TRUE)) install.packages("BiocManager") BiocManager::install("gdsfmt") ``` -------------------------------- ### setdim.gdsn example: integer node Source: https://github.com/zhengxwen/gdsfmt/blob/master/man/setdim.gdsn.Rd Creates a GDS file, adds an integer node with values 1:24, and demonstrates changing its dimensions with setdim.gdsn, including using permute=TRUE. The example shows reading the node after each change and closes the file at the end. ```R # cteate a GDS file f <- createfn.gds("test.gds") n <- add.gdsn(f, "int", val=1:24) read.gdsn(n) setdim.gdsn(n, c(6, 4)) read.gdsn(n) setdim.gdsn(n, c(8, 5), permute=TRUE) read.gdsn(n) setdim.gdsn(n, c(3, 4), permute=TRUE) read.gdsn(n) n <- add.gdsn(f, "bit3", val=1:24, storage="bit3") read.gdsn(n) setdim.gdsn(n, c(6, 4)) read.gdsn(n) setdim.gdsn(n, c(8, 5), permute=TRUE) read.gdsn(n) setdim.gdsn(n, c(3, 4), permute=TRUE) read.gdsn(n) # close the GDS file closefn.gds(f) # delete the temporary file unlink("test.gds", force=TRUE) ``` -------------------------------- ### name.gdsn usage example Source: https://github.com/zhengxwen/gdsfmt/blob/master/_autodocs/api-reference/structure-operations.md Creates a GDS file, adds a node, and retrieves its name. The output is `"x"`. ```r f <- createfn.gds("test.gds") n <- add.gdsn(f, "x", 1:10) name.gdsn(n) # "x" closefn.gds(f) ``` -------------------------------- ### Read matrix with simplify='none' Source: https://github.com/zhengxwen/gdsfmt/blob/master/man/read.gdsn.Rd Reads the same subset as the previous example but with simplify='none', forcing the result to remain a matrix. The output is shown as a comment. ```R read.gdsn(index.gdsn(f, "matrix"), start=c(2,2), count=c(-1,1), simplify="none") # [,1] <- a matrix # [1,] 5 # [2,] 6 ``` -------------------------------- ### Example: create GDS file, add data, and showfile.gds Source: https://github.com/zhengxwen/gdsfmt/blob/master/man/showfile.gds.Rd Creates a GDS file 'test.gds', adds an integer vector, then calls showfile.gds() to list open files and showfile.gds(closeall=TRUE) to close them. Finally deletes the temporary file. ```R # cteate a GDS file f <- createfn.gds("test.gds") add.gdsn(f, "int", val=1:10000) showfile.gds() showfile.gds(closeall=TRUE) # delete the temporary file unlink("test.gds", force=TRUE) ``` -------------------------------- ### Write to a GDS node with start and count Source: https://github.com/zhengxwen/gdsfmt/blob/master/man/write.gdsn.Rd Creates a GDS file, adds a matrix node, and writes a subset of values using start and count. The example shows writing NA values to a 2x1 block starting at position (2,2). ```R # cteate a GDS file f <- createfn.gds("test.gds") ################################################### n <- add.gdsn(f, "matrix", matrix(1:20, ncol=5)) read.gdsn(n) write.gdsn(n, val=c(NA, NA), start=c(2, 2), count=c(2, 1)) read.gdsn(n) ################################################### n <- add.gdsn(f, "n", val=1:12) read.gdsn(n) write.gdsn(n, matrix(1:24, ncol=6)) read.gdsn(n) write.gdsn(n, array(1:24, c(4,3,2))) read.gdsn(n) # close the GDS file closefn.gds(f) # delete the temporary file unlink("test.gds", force=TRUE) ``` -------------------------------- ### Example: Create, sync, and close a GDS file Source: https://github.com/zhengxwen/gdsfmt/blob/master/man/sync.gds.Rd Sets verbose mode, creates a GDS file, adds an integer node with 10000 values, sets an attribute, syncs to disk, retrieves the attribute, closes the file, and deletes the temporary file. ```R options(gds.verbose=TRUE) # cteate a GDS file f <- createfn.gds("test.gds") node <- add.gdsn(f, "int", val=1:10000) put.attr.gdsn(node, "missing.value", 10000) f sync.gds(f) get.attr.gdsn(node) # close the GDS file closefn.gds(f) # delete the temporary file unlink("test.gds", force=TRUE) ``` -------------------------------- ### Install gdsfmt from Bioconductor using BiocManager Source: https://github.com/zhengxwen/gdsfmt/blob/master/README.md Install the gdsfmt package from the Bioconductor repository. Requires the BiocManager package; if it is not installed, the snippet installs it first. ```R if (!requireNamespace("BiocManager", quietly=TRUE)) install.packages("BiocManager") BiocManager::install("gdsfmt") ``` -------------------------------- ### gdsUnregisterCloudHandler example Source: https://github.com/zhengxwen/gdsfmt/blob/master/_autodocs/api-reference/cloud-handlers.md Example of unregistering multiple cloud handlers. ```r gdsUnregisterCloudHandler(c("myfs", "s3")) ``` -------------------------------- ### summarize.gdsn example Source: https://github.com/zhengxwen/gdsfmt/blob/master/_autodocs/api-reference/diagnostics.md Example usage of summarize.gdsn to compute summary statistics. ```r f <- createfn.gds("test.gds") n <- add.gdsn(f, "x", sample(0:3, 1000, replace=TRUE), storage="bit2") summarize.gdsn(n) closefn.gds(f) ``` -------------------------------- ### Create GDS file and apply.gdsn with margin and substitution Source: https://github.com/zhengxwen/gdsfmt/blob/master/man/apply.gdsn.Rd Create a GDS file with sample data and run apply.gdsn with different margin and argument settings. The first call uses as.is="none"; the second uses default arguments; the third and fourth use .value and .substitute to replace values. ```R f <- createfn.gds("test.gds") X <- matrix(1:50, nrow=10) Y <- matrix((1:50)/100, nrow=10) Z1 <- factor(c(rep(c("ABC", "DEF", "ETD"), 3), "TTT")) Z2 <- c(TRUE, FALSE, TRUE, FALSE, TRUE) node.X <- add.gdsn(f, "X", X) node.Y <- add.gdsn(f, "Y", Y) node.Z1 <- add.gdsn(f, "Z1", Z1) node.Z2 <- add.gdsn(f, "Z2", Z2) ``` ```R v <- apply.gdsn(list(X=node.X, Y=node.Y, Z=node.Z1), margin=c(1, 1, 1), FUN=print, as.is="none") ``` ```R v <- apply.gdsn(list(X=node.X, Y=node.Y, Z=node.Z2), margin=c(2, 2, 1), FUN=print) ``` ```R v <- apply.gdsn(list(X=node.X, Y=node.Y, Z=node.Z2), margin=c(2, 2, 1), FUN=print, .value=35:45, .substitute=NA) ``` ```R v <- apply.gdsn(list(X=node.X, Y=node.Y, Z=node.Z2), margin=c(2, 2, 1), FUN=print, .value=35:45, .substitute=NA) ``` -------------------------------- ### is.sparse.gdsn example Source: https://github.com/zhengxwen/gdsfmt/blob/master/_autodocs/api-reference/diagnostics.md Example usage of is.sparse.gdsn to check if a node is sparse. ```r f <- createfn.gds("test.gds") n <- add.gdsn(f, "sp", val=1:100, storage="sp.int32") is.sparse.gdsn(n) # TRUE closefn.gds(f) ``` -------------------------------- ### is.element.gdsn example Source: https://github.com/zhengxwen/gdsfmt/blob/master/_autodocs/api-reference/diagnostics.md Example usage of is.element.gdsn to test membership of node elements against a set. ```r f <- createfn.gds("test.gds") n <- add.gdsn(f, "x", c(1, 2, 3, 4, 5)) is.element.gdsn(n, c(2, 4)) # FALSE TRUE FALSE TRUE FALSE closefn.gds(f) ``` -------------------------------- ### digest.gdsn example Source: https://github.com/zhengxwen/gdsfmt/blob/master/_autodocs/api-reference/diagnostics.md Example usage of digest.gdsn covering various actions: compute, add, verify, and clear. ```r library(digest) f <- createfn.gds("test.gds") n <- add.gdsn(f, "int1", as.integer(1:1024)) digest.gdsn(n) # md5 of the raw stream digest.gdsn(n, action="Robject") # md5 of the R object digest.gdsn(n, action="add") # store as attribute digest.gdsn(n, "sha1", action="add") print(n, attribute=TRUE) digest.gdsn(n, action="verify") # ok write.gdsn(n, 0, start=10, count=1) # corrupt one element try(digest.gdsn(n, action="verify")) # error digest.gdsn(n, action="clear") # remove barcodes closefn.gds(f) ``` -------------------------------- ### Set up parallel cluster with cl.cores Source: https://github.com/zhengxwen/gdsfmt/blob/master/man/clusterApply.gdsn.Rd Loads the parallel package and creates a cluster using the 'cl.cores' option, defaulting to 2 cores if not set. ```R library(parallel) # Use option cl.core to choose an appropriate cluster size. cl <- makeCluster(getOption("cl.cores", 2L)) ``` -------------------------------- ### Register and test a cloud URL scheme handler Source: https://github.com/zhengxwen/gdsfmt/blob/master/man/gdsRegisterCloudHandler.Rd Shows the full workflow: list current handlers, define a toy handler, register it for the 'myfs' scheme, observe dispatch via openfn.gds, then unregister. The handler must accept the URL as first argument and have a '...' argument. Registering a scheme replaces any previous handler. ```r # the schemes currently registered by companion packages, if any gdsCloudHandlers() # a toy handler, for illustration only my_open <- function(filename, ...) stop("Not implemented, this is only an example.") gdsRegisterCloudHandler("myfs", my_open, pkgname="myPkg") gdsCloudHandlers() # openfn.gds() now dispatches "myfs://" URLs to my_open() try(openfn.gds("myfs://bucket/file.gds")) gdsUnregisterCloudHandler("myfs") gdsCloudHandlers() ``` -------------------------------- ### assign.gdsn usage example Source: https://github.com/zhengxwen/gdsfmt/blob/master/_autodocs/api-reference/data-operations.md Example demonstrating appending columns from one node to another and subsetting with reordering and NA substitution. ```r f <- createfn.gds("test.gds") n1 <- add.gdsn(f, "n1", 1:100) n2 <- add.gdsn(f, "n2", storage="int", valdim=c(20, 0)) assign.gdsn(n2, n1, resize=FALSE, append=TRUE) # append columns read.gdsn(n2) # subset + reorder + substitute NAs n4 <- add.gdsn(f, "n4", matrix(1:48, 6)) assign.gdsn(n4, seldim=list(c(4,2,6,NA), c(5,6,NA,2,8,NA,4))) closefn.gds(f) ``` -------------------------------- ### add.gdsn usage examples Source: https://github.com/zhengxwen/gdsfmt/blob/master/_autodocs/api-reference/structure-operations.md Shows adding various node types (integer, matrix, logical, factor, list) and demonstrates error handling when replacing an existing node without replace=TRUE. ```r f <- createfn.gds("test.gds") add.gdsn(f, "int", val=1:10000) add.gdsn(f, "mat", val=matrix(1:100, 10), storage="int32", compress="ZIP") add.gdsn(f, "logical", val=c(TRUE, FALSE, NA)) add.gdsn(f, "factor", val=as.factor(c("AA", "BB"))) add.gdsn(f, "list", val=list(X=1:10, Y="a")) # child nodes created # error handling: replacing requires replace=TRUE try(add.gdsn(f, "int", val=2:5)) # error add.gdsn(f, "int", val=2:5, replace=TRUE) # ok closefn.gds(f) ``` -------------------------------- ### Apply function over columns with apply.gdsn Source: https://github.com/zhengxwen/gdsfmt/blob/master/man/apply.gdsn.Rd Applies a function over columns (margin=2) of the GDS nodes, including a selection example and an example on the string node n2. ```R apply.gdsn(n1, margin=2, FUN=function(x) print(x), as.is="none") apply.gdsn(n1, margin=2, selection = list(rep(c(TRUE, FALSE), 5), rep(c(TRUE, FALSE), 3)), FUN=function(x) print(x), as.is="none") apply.gdsn(n2, margin=2, selection = list(rep(c(TRUE, FALSE), 5), rep(c(TRUE, FALSE), 3)), FUN=function(x) print(x), as.is="none") ``` -------------------------------- ### read.gdsn usage example Source: https://github.com/zhengxwen/gdsfmt/blob/master/_autodocs/api-reference/read-and-apply.md Shows how to create a GDS file, add a matrix node, and perform full reads, block reads, simplified reads, and value substitution. Includes cleanup with closefn.gds. ```r f <- createfn.gds("test.gds") n <- add.gdsn(f, "mat", matrix(1:100, 10, 10)) # full read, block read, simplified read read.gdsn(n) read.gdsn(n, start=c(2, 2), count=c(3, 3)) read.gdsn(n, start=c(2, 2), count=c(-1, -1)) # all the rest # value substitution read.gdsn(n, .value=16:25, .substitute=NA) closefn.gds(f) ``` -------------------------------- ### Create a folder and add variables with addfolder.gdsn() Source: https://github.com/zhengxwen/gdsfmt/blob/master/vignettes/gdsfmt.Rmd Creates a folder named 'folder' using addfolder.gdsn() and adds an integer and a double variable to it. The double variable is hidden with visible=FALSE. ```r folder <- addfolder.gdsn(gfile, "folder") add.gdsn(folder, "int", val=1:1000) add.gdsn(folder, "double", val=seq(1, 100, 0.4), visible=FALSE) ``` -------------------------------- ### clusterApply.gdsn usage example Source: https://github.com/zhengxwen/gdsfmt/blob/master/_autodocs/api-reference/read-and-apply.md Example of using clusterApply.gdsn to apply a function over the rows of a matrix stored in a GDS file across a parallel cluster of 2 workers. ```r library(parallel) f <- createfn.gds("test1.gds") add.gdsn(f, "matrix", val=matrix(1:(10*6), nrow=10)) closefn.gds(f) cl <- makeCluster(2L) res <- clusterApply.gdsn(cl, "test1.gds", "matrix", margin=1, FUN=function(x) x) stopCluster(cl) ``` -------------------------------- ### Example: print a node with attributes Source: https://github.com/zhengxwen/gdsfmt/blob/master/_autodocs/api-reference/classes-and-methods.md Creates a GDS file, adds a compressed integer vector, and prints the root node with attributes and depth limit. Demonstrates the print method for gdsn.class. ```r f <- createfn.gds("test.gds") add.gdsn(f, "int", 1:10000, compress="ZIP", closezip=TRUE) print(f$root, attribute=TRUE, depth=1) ``` -------------------------------- ### Open and read a GDS file Source: https://github.com/zhengxwen/gdsfmt/blob/master/man/openfn.gds.Rd Creates a GDS file, adds a list, closes it, reopens it, reads the list, and deletes the temporary file. Demonstrates the full workflow for using openfn.gds. ```R # cteate a GDS file f <- createfn.gds("test.gds") # add a list to "test.gds" node <- add.gdsn(f, name="list", val=list(x=c(1,2), y=c("T","B","C"), z=TRUE)) # close closefn.gds(f) # open the same file f <- openfn.gds("test.gds") # read (node <- index.gdsn(f, "list")) read.gdsn(node) # close the GDS file closefn.gds(f) # delete the temporary file unlink("test.gds", force=TRUE) ``` -------------------------------- ### Create a simple GDS file and display it Source: https://github.com/zhengxwen/gdsfmt/blob/master/vignettes/gdsfmt.Rmd Creates a GDS file with two variables and prints the file object to show its structure. ```R gfile <- createfn.gds("test.gds") n1 <- add.gdsn(gfile, "I1", val=1:100) n2 <- add.gdsn(gfile, "I2", val=matrix(1:20, nrow=4)) gfile ``` -------------------------------- ### Write a subset of data with write.gdsn Source: https://github.com/zhengxwen/gdsfmt/blob/master/vignettes/gdsfmt.Rmd Writes a subset of data to an existing GDS variable using start and count arguments. -1 in count means the size of that dimension, and the corresponding start element should be 1. ```R write.gdsn(n, rep(0,5), start=c(2,1), count=c(1,-1)) show(n) ``` -------------------------------- ### Switching a GDS node to read mode Source: https://github.com/zhengxwen/gdsfmt/blob/master/man/readmode.gdsn.Rd This example demonstrates creating a compressed GDS node, attempting to read it while in write mode, switching to read mode using readmode.gdsn, and finally reading the data. ```R # cteate a GDS file f <- createfn.gds("test.gds") # commom types n <- add.gdsn(f, "int", val=1:100, compress="ZIP") # you can not read the variable "int" because of writing mode # read.gdsn(n) readmode.gdsn(n) # now you can read "int" read.gdsn(n) # close the GDS file closefn.gds(f) # delete the temporary file unlink("test.gds", force=TRUE) ``` -------------------------------- ### Create GDS file and generate sample data Source: https://github.com/zhengxwen/gdsfmt/wiki/Compresion---Decompression-algorithm Initializes a GDS file and creates a random binary integer vector of length 1,000,000 to be used for compression testing. ```R library(gdsfmt) # cteate a GDS file f <- createfn.gds("test.gds") set.seed(100) v <- sample.int(2, 1000000, replace=TRUE) - 1L table(v) ``` -------------------------------- ### diagnosis.gds example Source: https://github.com/zhengxwen/gdsfmt/blob/master/_autodocs/api-reference/file-operations.md Example usage of diagnosis.gds. Creates a GDS file with a compressed node, then performs file-level and node-level diagnosis. The file-level call returns a list with stream and log; the node-level call returns a list describing the byte streams owned by the node. ```r f <- createfn.gds("test.gds") add.gdsn(f, "x", 1:100, compress="ZIP_RA") closefn.gds(f) f <- openfn.gds("test.gds") d <- diagnosis.gds(f) # file-level diagnosis d2 <- diagnosis.gds(index.gdsn(f, "x")) # node-level diagnosis closefn.gds(f) ``` -------------------------------- ### Create a GDS file and print its contents Source: https://github.com/zhengxwen/gdsfmt/blob/master/man/print.gds.class.Rd Creates a GDS file, adds integer and matrix nodes, sets an attribute, and prints the file with various options. Then shows individual nodes and closes the file. ```R # cteate a GDS file f <- createfn.gds("test.gds") add.gdsn(f, "int", 1:100) add.gdsn(f, "int.matrix", matrix(1:(50*100), nrow=100, ncol=50)) put.attr.gdsn(index.gdsn(f, "int.matrix"), "int", 1:10) print(f, all=TRUE) print(f, all=TRUE, attribute=TRUE) print(f, all=TRUE, attribute=TRUE, attribute.trim=FALSE) show(index.gdsn(f, "int")) show(index.gdsn(f, "int.matrix")) # close the GDS file closefn.gds(f) # delete the temporary file unlink("test.gds", force=TRUE) ``` -------------------------------- ### Create second GDS file and link to first Source: https://github.com/zhengxwen/gdsfmt/blob/master/man/addfolder.gdsn.Rd Creates a second GDS file, adds an integer vector, and links to the first file using a virtual folder. Then closes the file. ```R # create the second GDS file f2 <- createfn.gds("test2.gds") add.gdsn(f2, "int", 101:200) # link to the first file addfolder.gdsn(f2, "virtual_folder", type="virtual", gds.fn="test1.gds") f2 # open the GDS file closefn.gds(f2) ``` -------------------------------- ### Clean up fragments with cleanup.gds Source: https://github.com/zhengxwen/gdsfmt/blob/master/man/cleanup.gds.Rd Calls cleanup.gds on the file to defragment it. This is the main purpose of the example. ```R # clean up fragments cleanup.gds("test.gds") ``` -------------------------------- ### delete.gdsn: Complete Example Workflow Source: https://github.com/zhengxwen/gdsfmt/blob/master/man/delete.gdsn.Rd Creates a GDS file, adds a list node, attempts deletion (with an error in a dontrun block), deletes the node with force=TRUE, closes the file, and removes the temporary file. ```r # cteate a GDS file f <- createfn.gds("test.gds") # add a list to "test.gds" node <- add.gdsn(f, name="list", val=list(x=c(1,2), y=c("T", "B", "C"), z=TRUE)) f \dontrun{ # delete "node", but an error occurs delete.gdsn(node) } # delete "node" delete.gdsn(node, TRUE) # close the GDS file closefn.gds(f) # delete the temporary file unlink("test.gds", force=TRUE) ```