### Slice Column with Named Keywords Source: https://scicloj.github.io/tablecloth Demonstrates using `:start` and `:end` keywords for clarity when slicing a column. This example slices with a step of 2. ```clojure (-> (tcc/column (range 10)) (tcc/slice :start :end 2)) ``` ```clojure #tech.v3.dataset.column[5] null [0, 2, 4, 6, 8] ``` -------------------------------- ### List All Datatypes Source: https://scicloj.github.io/tablecloth Retrieves a list of all supported datatypes in the underlying 'tech.ml' system. No setup is required. ```clojure (tech.v3.datatype.casting/all-datatypes) ``` ```clojure [:array-buffer :big-integer :bitmap :bool :boolean :byte :char :dataset :days :decimal :double :duration :epoch-days :epoch-hours :epoch-microseconds :epoch-milliseconds :epoch-nanoseconds :epoch-seconds :float :float32 :float64 :hours :instant :int :int16 :int32 :int64 :int8 :keyword :list :local-date :local-date-time :local-time :long :microseconds :milliseconds :nanoseconds :native-buffer :object :packed-duration :packed-instant :packed-local-date :packed-local-time :packed-milli-instant :persistent-map :persistent-set :persistent-vector :seconds :short :string :tensor :text :time-microseconds :time-milliseconds :time-nanoseconds :time-seconds :uint16 :uint32 :uint64 :uint8 :uuid :weeks :years :zoned-date-time] ``` -------------------------------- ### Define Dataset for Tidyr Examples Source: https://scicloj.github.io/tablecloth Defines a sample dataset with string and nil values for demonstrating tidyr-like operations. ```clojure (def df (tc/dataset {:x ["a" "a" nil nil] :y ["b" nil "b" nil]})) ``` -------------------------------- ### Return First Two Rows Per Month Source: https://scicloj.github.io/tablecloth Use `head(.SD, 2)` in R or `tc/head 2` in Clojure after grouping by month to get the first two rows for each month. The R example uses `kable` for display, while the Clojure example uses `tc/head 6` to display the first 6 rows of the result. ```R ans <- flights[, head(.SD, 2), by = month] kable(head(ans)) ``` -------------------------------- ### Create Dataset for Multi-choice Example Source: https://scicloj.github.io/tablecloth Initializes a dataset named `multi` with columns for ID, choice1, choice2, and choice3, including some nil values to test missing data handling. ```clojure (def multi (tc/dataset {:id [1 2 3 4] :choice1 ["A" "C" "D" "B"] :choice2 ["B" "B" nil "D"] :choice3 ["C" nil nil nil]})) ``` -------------------------------- ### Select Column Names by Datatype Regex Source: https://scicloj.github.io/tablecloth Apply a regular expression to column datatypes to select columns. This example selects columns with datatypes starting with ':int'. ```clojure (tc/column-names DS #"^:int.*" :datatype) ``` ```clojure (:V1 :V2) ``` -------------------------------- ### Get Column Names of Grouped Dataset as Regular Dataset Source: https://scicloj.github.io/tablecloth Shows how to group a dataset by ':V1', convert the grouped result into a regular dataset, and then get its column names. The resulting columns are ':name', ':group-id', and ':data'. ```clojure (-> DS (tc/group-by :V1) (tc/as-regular-dataset) (tc/column-names)) ``` ```text (:name :group-id :data) ``` -------------------------------- ### Slice Column from Start Source: https://scicloj.github.io/tablecloth Extracts a slice of a column starting from a specified index to the end. The example shows slicing from index 5 onwards. ```clojure (-> (tcc/column (range 10)) (tcc/slice 5)) ``` ```clojure #tech.v3.dataset.column[5] null [5, 6, 7, 8, 9] ``` -------------------------------- ### Get Tablecloth Version Source: https://scicloj.github.io/tablecloth Retrieves the version of the tablecloth library from the project.clj file. ```clojure (def tablecloth-version (nth (read-string (slurp "project.clj")) 2)) ``` -------------------------------- ### Define Column A Source: https://scicloj.github.io/tablecloth Defines a column named 'a' with integer values. This is a setup step for demonstrating column operations. ```clojure (def a (tcc/column [20 30 40 50])) ``` -------------------------------- ### Select Columns by Name Prefix Source: https://scicloj.github.io/tablecloth Selects columns whose names start with a specific prefix, using a string predicate. ```clojure (tc/select-columns DS #(clojure.string/starts-with? (name %) "V")) ``` -------------------------------- ### Add Multiple Columns Source: https://scicloj.github.io/tablecloth Adds several new columns to the dataset simultaneously. This example adds `:v6` using the square root of `:V1` and `:v7` with a constant value 'X'. ```clojure (def DS (tc/add-columns DS {:v6 (dfn/sqrt (DS :V1)) :v7 "X"})) ``` -------------------------------- ### Select Columns using Complement Source: https://scicloj.github.io/tablecloth Use `select-columns` with `complement` to create a dataset excluding specific columns. This example selects all columns except `:V1`. ```clojure (tc/select-columns DS (complement #{:V1})) ``` ```clojure _unnamed [9 3]: 1 | 0.5 | A ---|---|--- 2 | 1.0 | B 3 | 1.5 | C 4 | 0.5 | A 5 | 1.0 | B 6 | 1.5 | C 7 | 0.5 | A 8 | 1.0 | B 9 | 1.5 | C ``` -------------------------------- ### Get Basic Dataset Info Source: https://scicloj.github.io/tablecloth Retrieves a concise summary of the dataset, including column names, row count, column count, and whether it's a result of a group-by operation. ```clojure (tc/info ds :basic) ``` -------------------------------- ### Get General Dataset Info Source: https://scicloj.github.io/tablecloth Provides general information about the dataset. The default option includes basic statistics for columns. ```clojure (tc/info ds) ``` -------------------------------- ### Return First Two Rows Per Month (Clojure) Source: https://scicloj.github.io/tablecloth Use `tc/group-by` and `tc/head` in Clojure to achieve the same result as the R example. `tc/head 2` is applied to each group, and `tc/head 6` displays the first 6 rows of the final result. ```Clojure (-> flights (tc/group-by ["month"]) (tc/head 2) ;; head applied on each group (tc/ungroup) (tc/head 6)) ``` -------------------------------- ### Get Columns as Vectors Source: https://scicloj.github.io/tablecloth Retrieve all columns from a dataset as a sequence of vectors. Each inner vector represents a column. ```clojure (tc/columns DS :as-vecs) ``` -------------------------------- ### Get tech.ml.dataset Version Source: https://scicloj.github.io/tablecloth Retrieves the version of the tech.ml.dataset library from the deps.edn file. ```clojure (def tech-ml-version (get-in (read-string (slurp "deps.edn")) [:deps 'techascent/tech.ml.dataset :mvn/version])) ``` -------------------------------- ### Create a New Dataset with One Column Source: https://scicloj.github.io/tablecloth Creates a new dataset containing only a single column derived from existing data. This example creates `:v8` by adding 1 to the values in `:V3`. ```clojure (tc/dataset {:v8 (dfn/+ (DS :V3) 1)}) ``` -------------------------------- ### Get Columns as Sequence Source: https://scicloj.github.io/tablecloth Retrieve all columns from a dataset as a sequence. Useful for iterating or processing all columns. ```clojure (take 2 (tc/columns ds)) ``` -------------------------------- ### Aggregate by Multiple Groups Source: https://scicloj.github.io/tablecloth Performs aggregation on groups defined by multiple columns. This example groups by both `:V4` and `:V1`. ```clojure (-> DS (tc/group-by [:V4 :V1]) ``` -------------------------------- ### Get Column Names After Grouping by :V1 Source: https://scicloj.github.io/tablecloth Demonstrates how to group a dataset by the ':V1' column and then retrieve the column names of the resulting grouped dataset. ```clojure (-> DS (tc/group-by :V1) (tc/column-names)) ``` ```text (:V1 :V2 :V3 :V4) ``` -------------------------------- ### Create Dataset with Empty Single Value Column Name Source: https://scicloj.github.io/tablecloth This example demonstrates setting an empty string for the single-value column name and assigning a dataset name, while also disabling the error column. ```clojure (tc/dataset 999 {:single-value-column-name "" :dataset-name "Single value" :error-column? false}) ``` -------------------------------- ### Remove Multiple Columns Source: https://scicloj.github.io/tablecloth Removes multiple specified columns from the dataset. This example removes `:v6` and `:v7`. ```clojure (def DS (tc/drop-columns DS [:v6 :v7])) ``` -------------------------------- ### Create a Dataset for Splitting Source: https://scicloj.github.io/tablecloth This code snippet demonstrates how to create a sample dataset with 'id', 'partition', and 'group' columns, which can then be used for splitting operations. ```clojure (def for-splitting (tc/dataset (map-indexed (fn [id v] {:id id :partition v :group (rand-nth [:g1 :g2 :g3])}) (concat (repeat 20 :a) (repeat 5 :b))))) ``` -------------------------------- ### Get Groups as Sequence from Grouped Dataset Source: https://scicloj.github.io/tablecloth Demonstrates how to group a dataset and then extract the groups as a sequence of datasets. This is achieved by accessing the ':data' column of the grouped dataset. ```clojure (let [ds (-> {"a" [1 1 2 2] "b" ["a" "b" "c" "d"]} (tc/dataset) (tc/group-by "a"))] (seq (ds :data))) ;; seq is not necessary but Markdown treats `:data` as command here ``` ```text ( Group: 1 [2 2]: 1 | a ---|--- 1 | b Group: 2 [2 2]: 2 | c ---|--- 2 | d ) ``` -------------------------------- ### Slice Column with Start, End, and Step Source: https://scicloj.github.io/tablecloth Extracts a slice of a column with specified start, end, and step values. This example slices from index 0 to 9 with a step of 2. ```clojure (-> (tcc/column (range 10)) (tcc/slice 0 9 2)) ``` ```clojure #tech.v3.dataset.column[5] null [0, 2, 4, 6, 8] ``` -------------------------------- ### Get First N Rows Source: https://scicloj.github.io/tablecloth Retrieves the first N rows from the dataset. Defaults to the first 5 rows if N is not specified. ```clojure (tc/head DS) ``` -------------------------------- ### Load Anscombe Dataset Source: https://scicloj.github.io/tablecloth Loads the Anscombe dataset from a CSV file. This dataset is often used for statistical examples. ```clojure (def anscombe (tc/dataset "data/anscombe.csv")) ``` -------------------------------- ### Slice Column with Start and End Source: https://scicloj.github.io/tablecloth Extracts a slice of a column between specified start and end indices (exclusive of the end index). This example slices from index 1 up to (but not including) index 4. ```clojure (-> (tcc/column (range 10)) (tcc/slice 1 4)) ``` ```clojure #tech.v3.dataset.column[4] null [1, 2, 3, 4] ``` -------------------------------- ### Import Nippy File and Take Head Source: https://scicloj.github.io/tablecloth Load data from a `.nippy` file and display the first 10 rows using `tc/head`. ```clojure (-> (tc/dataset "data/example-genres.nippy") (tc/head 10)) ``` -------------------------------- ### Select Column Names by Regex Source: https://scicloj.github.io/tablecloth Use a regular expression to select column names that match the pattern. This example selects columns ending with '1' or '4'. ```clojure (tc/column-names DS #".*[14]") ``` ```clojure (:V1 :V4) ``` -------------------------------- ### Display Sample Dataset Source: https://scicloj.github.io/tablecloth Displays the content of the 'for-splitting' dataset, showing its rows and columns. ```clojure for-splitting ``` -------------------------------- ### Reorder columns by datatype Source: https://scicloj.github.io/tablecloth Reorder columns by selecting specific datatypes to appear at the beginning. This example places non-integer columns first, followed by integer columns. ```clojure (tc/reorder-columns DS (tc/column-names DS (complement #{:int64}) :datatype)) ``` -------------------------------- ### Select Columns from Grouped Dataset Source: https://scicloj.github.io/tablecloth When applied to a grouped dataset, `select-columns` operates on each group independently. This example selects columns `:V2` and `:V3` from each group. ```clojure (-> DS (tc/group-by :V1) (tc/select-columns [:V2 :V3]) (tc/groups->map)) ``` ```clojure { ``` 1 ``` __| Group: 1 [5 2]: | 1 | 0.5 ---|--- 3 | 1.5 5 | 1.0 7 | 0.5 9 | 1.5 ``` 2 ``` __| Group: 2 [4 2]: | 2 | 1.0 ---|--- 4 | 0.5 6 | 1.5 8 | 1.0 } ``` -------------------------------- ### Create Dataset from Arrow File Source: https://scicloj.github.io/tablecloth Loads data from an Arrow file and selects specific columns. Ensure the 'data/alldtypes.arrow-feather' file exists and is in Arrow format. ```clojure (-> (tc/dataset "data/alldtypes.arrow-feather" {:file-type :arrow}) (tc/select-columns ["uints" "longs" "ubytes" "strings" "doubles"])) ``` -------------------------------- ### Utility Column Creation Source: https://scicloj.github.io/tablecloth Shows how to quickly create columns filled with ones or zeros. ```APIDOC ## Utility Column Creation ### Description Quickly create columns of ones or zeros. ### Code Examples **Column of Ones:** ```clojure (tcc/ones 10) ``` **Column of Zeros:** ```clojure (tcc/zeros 10) ``` ``` -------------------------------- ### Create a Dataset Source: https://scicloj.github.io/tablecloth Creates a new dataset from a map of column names to sequences. Ensure the sequences are of equal length. ```clojure (def DS (tc/dataset {:V1 (take 9 (cycle [1 2])) :V2 (range 1 10) :V3 (take 9 (cycle [0.5 1.0 1.5])) :V4 (take 9 (cycle ["A" "B" "C"]}))) ``` -------------------------------- ### Get Row Numbers of Specific Rows by Group Source: https://scicloj.github.io/tablecloth Add a row ID, select columns, group by a key, select specific row indices within each group, and then `tc/ungroup` to get the row numbers of those specific rows per group. ```clojure (-> DS (tc/add-column :row-id (range)) (tc/select-columns [:V4 :row-id]) (tc/group-by :V4) (tc/select-rows [0 2]) (tc/ungroup)) ``` -------------------------------- ### Select Column by Name (Vector) Source: https://scicloj.github.io/tablecloth To select a column named `:all`, or a sequence/map, enclose it in a vector. This example returns an empty sequence as no such column exists. ```clojure (tc/column-names DS [:all]) ``` ```clojure () ``` -------------------------------- ### Display tech-ml-version Source: https://scicloj.github.io/tablecloth Prints the retrieved tech.ml.dataset version. ```clojure tech-ml-version ``` -------------------------------- ### Get Row Number of First Observation by Group Source: https://scicloj.github.io/tablecloth Add a row ID using `range`, select relevant columns, group by a key, and then use `tc/first` to get the row number of the first observation per group. `tc/ungroup` flattens the result. ```clojure (-> DS (tc/add-column :row-id (range)) (tc/select-columns [:V4 :row-id]) (tc/group-by :V4) (tc/first) (tc/ungroup)) ``` -------------------------------- ### Apply dec/inc to numerical columns Source: https://scicloj.github.io/tablecloth Update numerical columns by applying a sequence of functions. This example applies `dec` then `inc` to columns identified by `:type/numerical`. ```clojure (tc/update-columns DS :type/numerical [(partial map dec) (partial map inc)]) ``` -------------------------------- ### Get the class of a Dataset Source: https://scicloj.github.io/tablecloth Retrieves the Java class of the Tablecloth Dataset object. ```clojure (class DS) ``` -------------------------------- ### Aggregate by Group Source: https://scicloj.github.io/tablecloth Performs aggregation on groups defined by one or more columns. This example calculates the sum of `:V2` for each unique value in `:V4`. ```clojure (-> DS (tc/group-by [:V4]) (tc/aggregate {:sumV2 #(dfn/sum (% :V2))})) ``` -------------------------------- ### Column Creation Source: https://scicloj.github.io/tablecloth Demonstrates how to create an empty column and columns from vectors or sequences. ```APIDOC ## Column Creation ### Description Creating an empty column is straightforward. You can also create a Column from a vector or a sequence. ### Code Examples **Empty Column:** ```clojure (tcc/column) ``` **Column from Vector:** ```clojure (tcc/column [1 2 3 4 5]) ``` **Column from Sequence:** ```clojure (tcc/column `(1 2 3 4 5)) ``` ``` -------------------------------- ### Add a New Column Source: https://scicloj.github.io/tablecloth Adds a new column to the dataset, calculating its values based on an existing column. This example adds `:v5` using the natural logarithm of `:V1`. ```clojure (tc/map-columns DS :v5 [:V1] dfn/log) ``` ```clojure (def DS (tc/add-column DS :v5 (dfn/log (DS :V1)))) ``` -------------------------------- ### Update specific columns with functions Source: https://scicloj.github.io/tablecloth Assign specific functions to columns using a map. This example reverses column `:V1` and shuffles column `:V2`. ```clojure ^:note-to-test/skip (tc/update-columns DS {:V1 reverse :V2 (comp shuffle seq)}) ``` -------------------------------- ### Import Excel File Source: https://scicloj.github.io/tablecloth Load data from an Excel file (`.xlsx`). Ensure `dataset-io` is added to dependencies for automatic support. ```clojure (tc/dataset "data/singleSheet.xlsx") ``` -------------------------------- ### Remove a Single Column Source: https://scicloj.github.io/tablecloth Removes a specified column from the dataset. This example removes the `:v5` column. ```clojure (def DS (tc/drop-columns DS :v5)) ``` -------------------------------- ### Select First 8 Columns of Dataset Source: https://scicloj.github.io/tablecloth Selects the first 8 columns from the loaded world bank population dataset for initial inspection. ```clojure (->> world-bank-pop (tc/column-names) (take 8) (tc/select-columns world-bank-pop)) ``` -------------------------------- ### Display Stocks Dataset Source: https://scicloj.github.io/tablecloth Prints the loaded stocks-tidyr dataset to the console. Useful for inspecting the initial state of the data. ```clojure stocks-tidyr ``` -------------------------------- ### Load Production Dataset Source: https://scicloj.github.io/tablecloth Loads the production.csv dataset, used to demonstrate pivoting with multiple column selectors. ```clojure (def production (tc/dataset "data/production.csv")) ``` -------------------------------- ### Create Dataset using let-dataset macro Source: https://scicloj.github.io/tablecloth The `tc/let-dataset` macro allows defining columns using bindings, similar to R's `tibble` function, enabling column references within bindings. ```clojure (tc/let-dataset [x (range 1 6) y 1 z (dfn/+ x y)]) ``` -------------------------------- ### Get Dataset Name Source: https://scicloj.github.io/tablecloth Retrieves the current name associated with the dataset. This is often a file path or an identifier. ```clojure (tc/dataset-name ds) ``` -------------------------------- ### Convert to Wide Form on Time Column with Row Selection Source: https://scicloj.github.io/tablecloth Converts the stocks-long dataset to a wider format, pivoting on the 'time' column. It also selects a subset of rows before pivoting, demonstrating combined operations. Note: This example is marked to be skipped in tests. ```clojure ^:note-to-test/skip (-> stocks-long (tc/select-rows (range 0 30 4)) (tc/pivot->wider "time" :price {:drop-missing? false})) ``` -------------------------------- ### Load Stocks Dataset Source: https://scicloj.github.io/tablecloth Loads the stocks dataset from a CSV file. The :key-fn keyword option ensures that column names are keywords. ```clojure (defonce stocks (tc/dataset "https://raw.githubusercontent.com/techascent/tech.ml.dataset/master/test/data/stocks.csv" {:key-fn keyword})) ``` -------------------------------- ### Display Cross Join Dataset Source: https://scicloj.github.io/tablecloth Displays the content of the 'cjds' dataset, which is used for cross join examples. ```clojure cjds ``` -------------------------------- ### Select Numerical Columns and Get Columns as Double Arrays Source: https://scicloj.github.io/tablecloth Select only numerical columns, take the first few rows, and then represent them as a double-precision floating-point array. This is useful for numerical computations. ```clojure (-> ds (tc/select-columns :type/numerical) (tc/head) (tc/columns :as-double-arrays)) ``` -------------------------------- ### Summarize All Columns with Custom Function Source: https://scicloj.github.io/tablecloth Applies a custom aggregation function to all columns in a dataset. The example uses a custom max function that sorts the column in descending order and takes the first element. ```clojure (tc/aggregate-columns DS :all (fn [col] (first (sort #(compare %2 %1) col)))) ``` -------------------------------- ### Select Column Elements by Index Array Source: https://scicloj.github.io/tablecloth Selects specific elements from a column based on an array of index positions. This example selects elements at indices 1 and 9. ```clojure (-> (tcc/column (range 10)) (tcc/select [1 9])) ``` ```clojure #tech.v3.dataset.column[2] null [1, 9] ``` -------------------------------- ### Get Data Dimensions using data.table Source: https://scicloj.github.io/tablecloth Retrieves the dimensions (number of rows and columns) of the flights data.table in R. ```R dim(flights) ``` -------------------------------- ### Display a Dataset Source: https://scicloj.github.io/tablecloth Prints the contents of the dataset to the console. ```clojure DS ``` -------------------------------- ### Get Last Row of Dataset Source: https://scicloj.github.io/tablecloth Retrieves the last row from a dataset. This function is applicable to grouped datasets as well. ```clojure (tc/last DS) ``` -------------------------------- ### Get First Row of Dataset Source: https://scicloj.github.io/tablecloth Retrieves the first row from a dataset. This function is applicable to grouped datasets as well. ```clojure (tc/first DS) ``` -------------------------------- ### Generate Holdout Splits with Specific Steps Source: https://scicloj.github.io/tablecloth Creates data splits with specified percentages and a step size. `shuffle? false` ensures deterministic splits. Use this when you need precise control over the size of training and testing sets. ```clojure ^:note-to-test/skip (-> (tc/split for-splitting :holdouts {:steps [0.05 0.95 1.5] :shuffle? false}) (tc/group-by [:$split-id :$split-name])) ``` -------------------------------- ### Get Column Count of Dataset Source: https://scicloj.github.io/tablecloth Retrieves the total number of columns in a dataset. Useful for understanding the dimensionality of the data. ```clojure (tc/column-count ds) ``` -------------------------------- ### Create Empty Dataset with Column Names Source: https://scicloj.github.io/tablecloth Provide a map with `:column-names` to an empty dataset to define column headers. ```clojure (tc/dataset nil {:column-names [:a :b]}) ``` -------------------------------- ### Modify Column Values Source: https://scicloj.github.io/tablecloth Updates the values in an existing column based on a function. This example squares the values in `:V1`. ```clojure (tc/map-columns DS :V1 [:V1] #(dfn/pow % 2)) ``` ```clojure (def DS (tc/add-column DS :V1 (dfn/pow (DS :V1) 2))) ``` -------------------------------- ### Display tablecloth-version Source: https://scicloj.github.io/tablecloth Prints the retrieved tablecloth version. ```clojure tablecloth-version ``` -------------------------------- ### Display Dataset X Source: https://scicloj.github.io/tablecloth Displays the contents of the dataset 'x'. This is a preliminary step to visualize the data before performing join operations. ```clojure x ``` -------------------------------- ### Pivot to Wider Format with Custom Value Concatenation and Vector Source: https://scicloj.github.io/tablecloth Reshapes the income dataset to a wider format, spreading the 'variable' column. This example customizes value concatenation using `vector` and `vec` for column concatenation, while also ensuring missing values are not dropped. ```clojure (tc/pivot->wider income "variable" ["estimate" "moe"] {:concat-columns-with vec :concat-value-with vector :drop-missing? false}) ``` -------------------------------- ### Grouping by Multiple Columns (:V1 and :V3) Source: https://scicloj.github.io/tablecloth Illustrates grouping a dataset by two columns, ':V1' and ':V3', and retrieving the result as a sequence of datasets. The group names in this case are maps representing the combination of values from the grouping columns. ```clojure (tc/group-by DS [:V1 :V3] {:result-type :as-seq}) ``` -------------------------------- ### View Content of Grouped Dataset as Map Source: https://scicloj.github.io/tablecloth Illustrates how to group a dataset by ':V1' and retrieve the content as a map. The output shows the structure of the map, including group names, group IDs, and the data within each group. ```clojure (tc/columns (tc/group-by DS :V1) :as-map) ``` -------------------------------- ### Get Columns as Map Source: https://scicloj.github.io/tablecloth Access all columns from a dataset as a map, where keys are column names. Useful for direct lookup by name. ```clojure (keys (tc/columns ds :as-map)) ``` -------------------------------- ### Select a Column from Dataset Source: https://scicloj.github.io/tablecloth Access a specific column from a dataset by its name. This is a direct way to get a column's data. ```clojure (ds "wind") ``` ```clojure (tc/column ds "date") ``` -------------------------------- ### Display Stocks Dataset Source: https://scicloj.github.io/tablecloth Displays the loaded stocks dataset. This is a simple way to inspect the data after loading. ```clojure stocks ``` -------------------------------- ### Load US Rent and Income Dataset Source: https://scicloj.github.io/tablecloth Loads the US rent and income dataset from a CSV file. This dataset contains state-level rent and income data. ```clojure (def income (tc/dataset "data/us_rent_income.csv")) ``` -------------------------------- ### Get Row Count of Dataset Source: https://scicloj.github.io/tablecloth Retrieves the total number of rows in a dataset. This is a fundamental operation for understanding dataset size. ```clojure (tc/row-count ds) ``` -------------------------------- ### Create Dataset from Parquet File Source: https://scicloj.github.io/tablecloth Loads data from a Parquet file and selects specific columns. Ensure the 'data/titanic.parquet' file exists. ```clojure (-> (tc/dataset "data/titanic.parquet") (tc/select-columns ["PassengerId" "Pclass" "Age" "Embarked" "Survived"])) ``` -------------------------------- ### Select rows matching a regex pattern Source: https://scicloj.github.io/tablecloth Filters rows where the :V4 column starts with 'B'. Uses string matching with regex. ```clojure (tc/select-rows DS (comp (partial re-matches #"^B") str :V4)) ``` -------------------------------- ### Create Dataset from URL Source: https://scicloj.github.io/tablecloth Loads data from a CSV file hosted at a URL. The dataset is stored in a defonce variable 'ds' for later use. ```clojure (defonce ds (tc/dataset "https://vega.github.io/vega-lite/examples/data/seattle-weather.csv")) ``` ```clojure ds ``` -------------------------------- ### Get Data Dimensions using Clojure Dataset API Source: https://scicloj.github.io/tablecloth Retrieves the shape (number of rows and columns) of the flights dataset in Clojure. ```Clojure (tc/shape flights) ``` -------------------------------- ### Access Column Element by Index Source: https://scicloj.github.io/tablecloth Accesses an element in a column at a specific index using the `get` method, similar to Clojure vectors. ```clojure (-> (tcc/column [1 2 3 4 5]) (get 3)) ``` ```clojure 4 ``` -------------------------------- ### Import CSV File Source: https://scicloj.github.io/tablecloth Load data directly from a CSV file using `tc/dataset`. ```clojure (tc/dataset "data/family.csv") ``` -------------------------------- ### Get Last N Rows Source: https://scicloj.github.io/tablecloth Retrieves the last N rows from the dataset. Defaults to the last 5 rows if N is not specified. ```clojure (tc/tail DS) ``` -------------------------------- ### Create Dataset from Array (as rows with names) Source: https://scicloj.github.io/tablecloth Specify column names and layout when creating a dataset from an array of arrays. ```clojure (-> (map int-array [[1 2] [3 4] [5 6]]) (into-array) (tc/dataset {:layout :as-rows :column-names [:a :b]})) ``` -------------------------------- ### Convert Grouped Dataset to Sequence of Groups Source: https://scicloj.github.io/tablecloth Shows how to group a dataset by column 'a' and then explicitly convert the grouped result into a sequence of datasets using `groups->seq`. ```clojure (-> {"a" [1 1 2 2] "b" ["a" "b" "c" "d"]} (tc/dataset) (tc/group-by "a") (tc/groups->seq)) ``` ```text ( Group: 1 [2 2]: 1 | a ---|--- 1 | b Group: 2 [2 2]: 2 | c ---|--- 2 | d ) ``` -------------------------------- ### Create Dataset from Array of Arrays (as columns) Source: https://scicloj.github.io/tablecloth Use the `:layout :as-columns` option to interpret inner arrays as columns when creating a dataset from an array of arrays. ```clojure (-> (map int-array [[1 2] [3 4] [5 6]]) (into-array) (tc/dataset {:layout :as-columns})) ``` -------------------------------- ### Get Random Rows with Seed Source: https://scicloj.github.io/tablecloth Selects a specified number of random rows, using a seed for reproducibility. Repetition is allowed by default. ```clojure (tc/random DS 5 {:seed 42}) ``` -------------------------------- ### Get Random Row from Dataset Source: https://scicloj.github.io/tablecloth Selects a single random row from the dataset. Use the `:seed` option to ensure reproducible results. ```clojure (tc/rand-nth DS) ``` ```clojure (tc/rand-nth DS {:seed 42}) ``` -------------------------------- ### Load Construction Dataset Source: https://scicloj.github.io/tablecloth Loads the construction dataset from the 'data/construction.csv' file into a Tablecloth dataset named `construction`. ```clojure (def construction (tc/dataset "data/construction.csv")) ``` -------------------------------- ### Order by Column and Custom Function Source: https://scicloj.github.io/tablecloth Orders the dataset first by a specified column in descending order, and then by a custom function (product of three columns) in ascending order. ```clojure (tc/order-by DS [:V4 (fn [row] (* (:V1 row) (:V2 row) (:V3 row)))] [:desc :asc]) ``` -------------------------------- ### Get Rows as Sequence of Maps Source: https://scicloj.github.io/tablecloth Retrieve rows from a dataset as a sequence of maps. Each map represents a row with column names as keys. ```clojure (clojure.pprint/pprint (take 2 (tc/rows ds :as-maps))) ``` -------------------------------- ### Get Rows as Sequence of Sequences Source: https://scicloj.github.io/tablecloth Extract rows from a dataset as a sequence of sequences. Each inner sequence represents a row's values. ```clojure (take 2 (tc/rows ds)) ``` -------------------------------- ### Load and Prepare Billboard Dataset Source: https://scicloj.github.io/tablecloth Loads the billboard dataset and drops boolean columns. This prepares the data for further transformation by removing irrelevant columns. ```clojure (def bilboard (-> (tc/dataset "data/billboard.csv.gz") (tc/drop-columns :type/boolean))) ;; drop some boolean columns, tidyr just skips them ``` -------------------------------- ### Get Dataset Shape Source: https://scicloj.github.io/tablecloth Returns the shape of the dataset as a vector containing the row count and column count, e.g., [rows columns]. ```clojure (tc/shape ds) ``` -------------------------------- ### Define Dataset X for Joins Source: https://scicloj.github.io/tablecloth Defines a sample dataset named 'x' with columns 'Id', 'X1', and 'XY'. This dataset is used in subsequent join operations. ```clojure (def x (tc/dataset {"Id" ["A" "B" "C" "C"] "X1" [1 3 5 7] "XY" ["x2" "x4" "x6" "x8"]})) ``` -------------------------------- ### Join Columns with Default Settings Source: https://scicloj.github.io/tablecloth Joins specified columns into a new column named :joined using default settings. Source columns are dropped by default. ```clojure (tc/join-columns DSm :joined [:V1 :V2 :V4]) ``` -------------------------------- ### Load Dataset Source: https://scicloj.github.io/tablecloth Loads a dataset from a CSV file. This is a common first step before performing data transformations. ```clojure (def family (tc/dataset "data/family.csv")) ``` -------------------------------- ### Replace Values Conditionally Source: https://scicloj.github.io/tablecloth Replaces values in a column based on a condition. This example sets values in `:V2` to 0.0 if they are less than 4.0. ```clojure (def DS (tc/map-columns DS :V2 [:V2] #(if (< % 4.0) 0.0 %))) ``` -------------------------------- ### Define Column B Source: https://scicloj.github.io/tablecloth Defines a column named 'b' using a range of integers. Used in conjunction with column 'a' for operation examples. ```clojure (def b (tcc/column (range 4))) ``` -------------------------------- ### Create Dataset from Sequence of Maps with Varying Data Source: https://scicloj.github.io/tablecloth Handles sequences of maps where values might be collections, creating columns accordingly. ```clojure (tc/dataset [{:a 1 :b [1 2 3]} {:a 2 :b [3 4]}]) ``` -------------------------------- ### Select Random Row Strategy Source: https://scicloj.github.io/tablecloth Uses the `:random` strategy to select a random row when applying uniqueness. This example is skipped in tests. ```clojure ^:note-to-test/skip (tc/unique-by DS :V1 {:strategy :random}) ``` -------------------------------- ### Create Dataset from Map with Multiple Columns Source: https://scicloj.github.io/tablecloth A map with multiple key-value pairs creates a dataset with multiple columns. ```clojure (tc/dataset {:A [3 4 5] :B ["X" "Y" "Z"]}) ``` -------------------------------- ### Select Columns by Excluding Name Prefix Source: https://scicloj.github.io/tablecloth Selects columns whose names do not start with a specific prefix, using a negated string predicate. ```clojure (tc/select-columns DS #(not (clojure.string/starts-with? (name %) "V2"))) ``` -------------------------------- ### Access Column Element by Index (nth) Source: https://scicloj.github.io/tablecloth Accesses an element in a column at a specific index using the `nth` method, providing an alternative to `get`. ```clojure (-> (tcc/column [1 2 3 4 5]) (nth 3)) ``` ```clojure 4 ``` -------------------------------- ### Load World Bank Population Dataset Source: https://scicloj.github.io/tablecloth Loads the world bank population dataset from a gzipped CSV file into a Tablecloth dataset. ```clojure (def world-bank-pop (tc/dataset "data/world_bank_pop.csv.gz")) ``` -------------------------------- ### Load Dataset Source: https://scicloj.github.io/tablecloth Loads a dataset from a specified CSV file path. This is a foundational step before performing any data manipulations. ```clojure (def who (tc/dataset "data/who.csv.gz")) ``` -------------------------------- ### Create Dataset for Extraction Source: https://scicloj.github.io/tablecloth Defines a dataset with a single column 'x' containing hyphen-separated strings, used for demonstrating column extraction. ```clojure (def df-extract (tc/dataset {:x [nil "a-b" "a-d" "b-c" "d-e"]})) ``` -------------------------------- ### Get Values (Groups) of Grouped Dataset as Map Source: https://scicloj.github.io/tablecloth Shows how to group a dataset by ':V1' and extract only the values (the grouped datasets) when the result type is ':as-map'. ```clojure (vals (tc/group-by DS :V1 {:result-type :as-map})) ``` -------------------------------- ### Get Keys of Grouped Dataset as Map Source: https://scicloj.github.io/tablecloth Demonstrates grouping a dataset by ':V1' and extracting only the keys (group names) when the result type is set to ':as-map'. ```clojure (keys (tc/group-by DS :V1 {:result-type :as-map})) ``` ```text (1 2) ``` -------------------------------- ### Create Dataset with Spaces for Separation Source: https://scicloj.github.io/tablecloth Defines a dataset with a single column 'x' containing strings with spaces, for demonstrating separation by space. ```clojure (def df-separate2 (tc/dataset {:x ["a" "a b" nil "a b c"]})) ``` -------------------------------- ### Define Dataset X for Bind Source: https://scicloj.github.io/tablecloth Defines a dataset 'x' with a single column 'V1' containing integers 1, 2, and 3. ```clojure (def x (tc/dataset {:V1 [1 2 3]})) ``` -------------------------------- ### Get Single Entry using get-entry Source: https://scicloj.github.io/tablecloth Retrieve a single value from the dataset using the `get-entry` function, specifying the column name and row index. ```clojure (tc/get-entry ds "wind" 2) ``` -------------------------------- ### Get Single Entry using get-in Source: https://scicloj.github.io/tablecloth Retrieve a single value from the dataset using the `get-in` function, specifying the column name and row index. ```clojure (get-in ds ["wind" 2]) ``` -------------------------------- ### Get Dataset Column Metadata Source: https://scicloj.github.io/tablecloth Fetches metadata for each column in the dataset, including name, type, row count, and whether it contains null values. ```clojure (tc/info ds :columns) ``` -------------------------------- ### Sort Column with Comparator Source: https://scicloj.github.io/tablecloth Sorts a column using a custom comparator function. This example sorts based on the `:position` key within map elements. ```clojure (-> (tcc/column [{:position 2 :text "and then stopped"} {:position 1 :text "I ran fast"}]) (tcc/sort-column (fn [a b] (< (:position a) (:position b))))) ``` -------------------------------- ### Display Dataset with Spaces Source: https://scicloj.github.io/tablecloth Displays the content of a dataset containing strings with spaces, showing how initial data is represented. ```clojure df-separate2 ``` -------------------------------- ### Create Empty Dataset Source: https://scicloj.github.io/tablecloth Use `tc/dataset` with no arguments to create an empty dataset. ```clojure (tc/dataset) ``` -------------------------------- ### Create Dataset with Mixed Delimiters Source: https://scicloj.github.io/tablecloth Defines a dataset with a single column 'x' containing strings with various delimiters like '?', '.', and ':', used for demonstrating flexible separation. ```clojure (def df-separate3 (tc/dataset {:x ["a?b" nil "a.b" "b:c"]})) ``` -------------------------------- ### Drop columns from grouped dataset Source: https://scicloj.github.io/tablecloth When applied to a grouped dataset, `drop-columns` operates on each group independently. The example shows dropping specific columns after grouping. ```clojure (-> DS (tc/group-by :V1) (tc/drop-columns [:V2 :V3]) (tc/groups->map)) ``` -------------------------------- ### Grouping by Provided Row Indexes Source: https://scicloj.github.io/tablecloth Shows how to group a dataset using custom maps of group names to row indexes. This allows a single row to be assigned to multiple groups. ```clojure (tc/group-by DS {"group-a" [1 2 1 2] "group-b" [5 5 5 1]} {:result-type :as-seq}) ``` -------------------------------- ### Split Dataset with Bootstrap Sampling Source: https://scicloj.github.io/tablecloth This snippet demonstrates splitting a dataset using bootstrap sampling with `tc/split->seq`. It groups the data by `:group`, then applies bootstrap sampling with specified parameters like `:seed`, `:ratio`, and `:repeats`. ```clojure ^:note-to-test/skip (-> for-splitting (tc/group-by :group) (tc/split->seq :bootstrap {:partition-selector :partition :seed 11 :ratio 0.8 :repeats 2}) (first)) ``` -------------------------------- ### Create Dataset from Sequence of Maps Source: https://scicloj.github.io/tablecloth A sequence of maps creates a dataset where keys become column names and values populate rows. ```clojure (tc/dataset [{:a 1 :b 3} {:b 2 :a 99}]) ``` -------------------------------- ### Create Dataset from Array (as rows) Source: https://scicloj.github.io/tablecloth Reading from arrays defaults to interpreting elements as rows. ```clojure (tc/dataset [[:a 1] [:b 2] [:c 3]]) ``` -------------------------------- ### Get Random Non-Repeating Rows Source: https://scicloj.github.io/tablecloth Selects a specified number of unique random rows from the dataset. Ensure the requested number does not exceed the dataset's row count. ```clojure (tc/random DS 5 {:repeat? false}) ``` -------------------------------- ### Get Random Rows with Repetition Source: https://scicloj.github.io/tablecloth Selects random rows from the dataset, allowing for repetitions. By default, it selects a number of rows equal to the dataset's row count. ```clojure (tc/random DS) ``` ```clojure (tc/random DS 5) ``` -------------------------------- ### Define Dataset Z for Bind Source: https://scicloj.github.io/tablecloth Defines a dataset 'z' with two columns, 'V1' and 'V2'. 'V1' contains 7, 8, 9 and 'V2' contains three zeros. ```clojure (def z (tc/dataset {:V1 [7 8 9] :V2 [0 0 0]})) ``` -------------------------------- ### Remove Columns Using a Complement Set Source: https://scicloj.github.io/tablecloth Removes columns from the dataset by specifying a set of columns to keep using `complement`. This example keeps all columns except `:V3`. ```clojure (def DS (tc/select-columns DS (complement #{:V3}))) ``` -------------------------------- ### Load Dataset for Pivoting Source: https://scicloj.github.io/tablecloth Loads the fish_encounters.csv dataset. This is a prerequisite for demonstrating the pivot->wider function. ```clojure (def fish (tc/dataset "data/fish_encounters.csv")) ``` -------------------------------- ### Create Dataset for Separation Source: https://scicloj.github.io/tablecloth Defines a dataset with a single column 'x' containing string data, used for demonstrating column separation. ```clojure (def df-separate (tc/dataset {:x [nil "a.b" "a.d" "b.c"]})) ``` -------------------------------- ### Create Dataset from Mixed Array Types (as columns) Source: https://scicloj.github.io/tablecloth Handles arrays containing different data types, mapping them to columns with specified names. ```clojure (-> (map to-array [[:a :z] ["ee" "ww"] [9 10]]) (into-array) (tc/dataset {:column-names [:a :b :c] :layout :as-columns})) ``` -------------------------------- ### Select Rows and Columns Source: https://scicloj.github.io/tablecloth Selects a specific row from a column. Use `tc/select-rows` for row selection and `tc/select-columns` for column selection. ```clojure (-> DS (tc/select-rows 4) (tc/select-columns :V3)) ;; select forth row from `:V3` column ``` ```clojure (-> DS (tc/select :V3 4)) ;; select forth row from `:V3` column ``` -------------------------------- ### Select Numerical Columns and Get Rows as Double Arrays Source: https://scicloj.github.io/tablecloth Select only numerical columns, take the first few rows, and then represent them as a double-precision floating-point array. This is useful for numerical computations. ```clojure (-> ds (tc/select-columns :type/numerical) (tc/head) (tc/rows :as-double-arrays)) ``` -------------------------------- ### Join and Separate Columns Together Source: https://scicloj.github.io/tablecloth Demonstrates joining multiple columns into a single map column, and then separating that map column back into individual columns using a custom function. ```clojure (-> DSm (tc/join-columns :joined [:V1 :V2 :V4] {:result-type :map}) (tc/separate-column :joined [:v1 :v2 :v4] (juxt :V1 :V2 :V4))) ``` -------------------------------- ### Unroll One by One for Cartesian Product Source: https://scicloj.github.io/tablecloth Demonstrates how unrolling columns sequentially can lead to a cartesian product of the unfolded sequences. This can result in a significantly larger dataset. ```clojure (-> DS (tc/fold-by [:V4 :V1]) (tc/unroll [:V2]) (tc/unroll [:V3])) ``` -------------------------------- ### Define Dataset for Crosstab Source: https://scicloj.github.io/tablecloth Defines a sample dataset named 'ctds' using a map with vector values for columns 'a', 'b', and 'c'. This dataset is used in subsequent crosstab examples. ```clojure (def ctds (tc/dataset {:a [:foo :foo :bar :bar :foo :foo] :b [:one :one :two :one :two :one] :c [:dull :dull :shiny :dull :dull :shiny]})) ``` -------------------------------- ### Map columns on a grouped dataset Source: https://scicloj.github.io/tablecloth Apply `map-columns` to a grouped dataset to perform operations within each group. This example sums numerical columns per group and then regroups the data. ```clojure (-> DS (tc/group-by :V4) (tc/map-columns :sum-of-numbers (tc/column-names DS #{:int64 :float64} :datatype) (fn [& rows] (reduce + rows))) (tc/ungroup)) ``` -------------------------------- ### Create Dataset from Excel File Source: https://scicloj.github.io/tablecloth Loads data from an Excel file. Tablecloth automatically supports Excel, Arrow, and Parquet if 'dataset-io' is added to your project dependencies. ```clojure (tc/dataset "data/singleSheet.xlsx") ``` -------------------------------- ### Select Single Column by Name as Dataset (List) Source: https://scicloj.github.io/tablecloth Selects a single column from the dataset by its name, provided as a list, and returns it as a new dataset. ```clojure (tc/select-columns DS [:V2]) ;; as dataset ``` -------------------------------- ### Require Tablecloth Column API Source: https://scicloj.github.io/tablecloth This code requires the Tablecloth Column API and aliases it as `tcc` for easier use throughout the project. This is a common setup step before interacting with column functionalities. ```clojure (require '[tablecloth.column.api :as tcc]) ``` -------------------------------- ### Cast Data (Long to Wide) - String Coercion and Count Source: https://scicloj.github.io/tablecloth This example demonstrates casting data to wide format after coercing values to strings and then counting occurrences. It handles boolean-like string categories. ```clojure (-> mDS (tc/map-columns :value #(str (> % 5))) ;; coerce to strings (tc/pivot->wider :value :variable {:fold-fn vec}) (tc/update-columns ["true" "false"] (partial map #(if (sequential? %) (count %) 1)))) ``` -------------------------------- ### Display Dataset Source: https://scicloj.github.io/tablecloth Prints the defined dataset 'DS' to the console. ```clojure DS ``` -------------------------------- ### Pivot Production with Multiple Columns Source: https://scicloj.github.io/tablecloth Pivots the production dataset using both 'product' and 'country' columns to create new columns, with 'production' as the value column. Column names are joined using the default separator. ```clojure (tc/pivot->wider production ["product" "country"] "production") ```