Tablecloth

https://scicloj.github.io/tablecloth/
Admin
Tablecloth is a dataset (data frame) manipulation API for Clojure that provides a simple and...
Tokens:59,460
Snippets:898
Trust Score:9.7
Update:5 months ago
Show doc for...
Tokens:
Raw
### Slice Column with Named Keywords

Source: https://scicloj.github.io/tablecloth

Demonstrates using `:start` and `:end` keywords for clarity when slicing a column. This example slices with a step of 2.

```clojure
(-> (tcc/column (range 10))
    (tcc/slice :start :end 2))
```

```clojure
#tech.v3.dataset.column<int64>[5]
null
[0, 2, 4, 6, 8]
```

--------------------------------

### List All Datatypes

Source: https://scicloj.github.io/tablecloth

Retrieves a list of all supported datatypes in the underlying 'tech.ml' system. No setup is required.

```clojure
(tech.v3.datatype.casting/all-datatypes)
```

```clojure
[:array-buffer
 :big-integer
 :bitmap
 :bool
 :boolean
 :byte
 :char
 :dataset
 :days
 :decimal
 :double
 :duration
 :epoch-days
 :epoch-hours
 :epoch-microseconds
 :epoch-milliseconds
 :epoch-nanoseconds
 :epoch-seconds
 :float
 :float32
 :float64
 :hours
 :instant
 :int
 :int16
 :int32
 :int64
 :int8
 :keyword
 :list
 :local-date
 :local-date-time
 :local-time
 :long
 :microseconds
 :milliseconds
 :nanoseconds
 :native-buffer
 :object
 :packed-duration
 :packed-instant
 :packed-local-date
 :packed-local-time
 :packed-milli-instant
 :persistent-map
 :persistent-set
 :persistent-vector
 :seconds
 :short
 :string
 :tensor
 :text
 :time-microseconds
 :time-milliseconds
 :time-nanoseconds
 :time-seconds
 :uint16
 :uint32
 :uint64
 :uint8
 :uuid
 :weeks
 :years
 :zoned-date-time]
```

--------------------------------

### Define Dataset for Tidyr Examples

Source: https://scicloj.github.io/tablecloth

Defines a sample dataset with string and nil values for demonstrating tidyr-like operations.

```clojure
(def df (tc/dataset {:x ["a" "a" nil nil]
                      :y ["b" nil "b" nil]}))
```

--------------------------------

### Return First Two Rows Per Month

Source: https://scicloj.github.io/tablecloth

Use `head(.SD, 2)` in R or `tc/head 2` in Clojure after grouping by month to get the first two rows for each month. The R example uses `kable` for display, while the Clojure example uses `tc/head 6` to display the first 6 rows of the result.

```R
ans <- flights[, head(.SD, 2), by = month]
kable(head(ans))
```

--------------------------------

### Create Dataset for Multi-choice Example

Source: https://scicloj.github.io/tablecloth

Initializes a dataset named `multi` with columns for ID, choice1, choice2, and choice3, including some nil values to test missing data handling.

```clojure
(def multi (tc/dataset {:id [1 2 3 4]
                         :choice1 ["A" "C" "D" "B"]
                         :choice2 ["B" "B" nil "D"]
                         :choice3 ["C" nil nil nil]}))
```

--------------------------------

### Select Column Names by Datatype Regex

Source: https://scicloj.github.io/tablecloth

Apply a regular expression to column datatypes to select columns. This example selects columns with datatypes starting with ':int'.

```clojure
(tc/column-names DS #"^:int.*" :datatype)
```

```clojure
(:V1 :V2)
```

--------------------------------

### Get Column Names of Grouped Dataset as Regular Dataset

Source: https://scicloj.github.io/tablecloth

Shows how to group a dataset by ':V1', convert the grouped result into a regular dataset, and then get its column names. The resulting columns are ':name', ':group-id', and ':data'.

```clojure
(-> DS
    (tc/group-by :V1)
    (tc/as-regular-dataset)
    (tc/column-names))
```

```text
(:name :group-id :data)
```

--------------------------------

### Slice Column from Start

Source: https://scicloj.github.io/tablecloth

Extracts a slice of a column starting from a specified index to the end. The example shows slicing from index 5 onwards.

```clojure
(-> (tcc/column (range 10))
    (tcc/slice 5))
```

```clojure
#tech.v3.dataset.column<int64>[5]
null
[5, 6, 7, 8, 9]
```

--------------------------------

### Get Tablecloth Version

Source: https://scicloj.github.io/tablecloth

Retrieves the version of the tablecloth library from the project.clj file.

```clojure
(def tablecloth-version (nth (read-string (slurp "project.clj")) 2))
```

--------------------------------

### Define Column A

Source: https://scicloj.github.io/tablecloth

Defines a column named 'a' with integer values. This is a setup step for demonstrating column operations.

```clojure
(def a (tcc/column [20 30 40 50]))
```

--------------------------------

### Select Columns by Name Prefix

Source: https://scicloj.github.io/tablecloth

Selects columns whose names start with a specific prefix, using a string predicate.

```clojure
(tc/select-columns DS #(clojure.string/starts-with? (name %) "V"))
```

--------------------------------

### Add Multiple Columns

Source: https://scicloj.github.io/tablecloth

Adds several new columns to the dataset simultaneously. This example adds `:v6` using the square root of `:V1` and `:v7` with a constant value 'X'.

```clojure
(def DS (tc/add-columns DS {:v6 (dfn/sqrt (DS :V1))
                                       :v7 "X"}))
```

--------------------------------

### Select Columns using Complement

Source: https://scicloj.github.io/tablecloth

Use `select-columns` with `complement` to create a dataset excluding specific columns. This example selects all columns except `:V1`.

```clojure
(tc/select-columns DS (complement #{:V1}))
```

```clojure
_unnamed [9 3]:
1 | 0.5 | A  
---|---|---
2 | 1.0 | B  
3 | 1.5 | C  
4 | 0.5 | A  
5 | 1.0 | B  
6 | 1.5 | C  
7 | 0.5 | A  
8 | 1.0 | B  
9 | 1.5 | C  
```

--------------------------------

### Get Basic Dataset Info

Source: https://scicloj.github.io/tablecloth

Retrieves a concise summary of the dataset, including column names, row count, column count, and whether it's a result of a group-by operation.

```clojure
(tc/info ds :basic)
```

--------------------------------

### Get General Dataset Info

Source: https://scicloj.github.io/tablecloth

Provides general information about the dataset. The default option includes basic statistics for columns.

```clojure
(tc/info ds)
```

--------------------------------

### Return First Two Rows Per Month (Clojure)

Source: https://scicloj.github.io/tablecloth

Use `tc/group-by` and `tc/head` in Clojure to achieve the same result as the R example. `tc/head 2` is applied to each group, and `tc/head 6` displays the first 6 rows of the final result.

```Clojure
(-> flights
    (tc/group-by ["month"])
    (tc/head 2) ;; head applied on each group
    (tc/ungroup)
    (tc/head 6))
```

--------------------------------

### Get Columns as Vectors

Source: https://scicloj.github.io/tablecloth

Retrieve all columns from a dataset as a sequence of vectors. Each inner vector represents a column.

```clojure
(tc/columns DS :as-vecs)
```

--------------------------------

### Get tech.ml.dataset Version

Source: https://scicloj.github.io/tablecloth

Retrieves the version of the tech.ml.dataset library from the deps.edn file.

```clojure
(def tech-ml-version (get-in (read-string (slurp "deps.edn")) [:deps 'techascent/tech.ml.dataset :mvn/version]))
```

--------------------------------

### Create a New Dataset with One Column

Source: https://scicloj.github.io/tablecloth

Creates a new dataset containing only a single column derived from existing data. This example creates `:v8` by adding 1 to the values in `:V3`.

```clojure
(tc/dataset {:v8 (dfn/+ (DS :V3) 1)})
```

--------------------------------

### Get Columns as Sequence

Source: https://scicloj.github.io/tablecloth

Retrieve all columns from a dataset as a sequence. Useful for iterating or processing all columns.

```clojure
(take 2 (tc/columns ds))
```

--------------------------------

### Aggregate by Multiple Groups

Source: https://scicloj.github.io/tablecloth

Performs aggregation on groups defined by multiple columns. This example groups by both `:V4` and `:V1`.

```clojure
(-> DS
    (tc/group-by [:V4 :V1])
```

--------------------------------

### Get Column Names After Grouping by :V1

Source: https://scicloj.github.io/tablecloth

Demonstrates how to group a dataset by the ':V1' column and then retrieve the column names of the resulting grouped dataset.

```clojure
(-> DS
    (tc/group-by :V1)
    (tc/column-names))
```

```text
(:V1 :V2 :V3 :V4)
```

--------------------------------

### Create Dataset with Empty Single Value Column Name

Source: https://scicloj.github.io/tablecloth

This example demonstrates setting an empty string for the single-value column name and assigning a dataset name, while also disabling the error column.

```clojure
(tc/dataset 999 {:single-value-column-name ""
                 :dataset-name "Single value"
                 :error-column? false})
```

--------------------------------

### Remove Multiple Columns

Source: https://scicloj.github.io/tablecloth

Removes multiple specified columns from the dataset. This example removes `:v6` and `:v7`.

```clojure
(def DS (tc/drop-columns DS [:v6 :v7]))
```

--------------------------------

### Create a Dataset for Splitting

Source: https://scicloj.github.io/tablecloth

This code snippet demonstrates how to create a sample dataset with 'id', 'partition', and 'group' columns, which can then be used for splitting operations.

```clojure
(def for-splitting (tc/dataset (map-indexed (fn [id v] {:id id
                                                        :partition v
                                                        :group (rand-nth [:g1 :g2 :g3])}) (concat (repeat 20 :a) (repeat 5 :b)))))

```

--------------------------------

### Get Groups as Sequence from Grouped Dataset

Source: https://scicloj.github.io/tablecloth

Demonstrates how to group a dataset and then extract the groups as a sequence of datasets. This is achieved by accessing the ':data' column of the grouped dataset.

```clojure
(let [ds (-> {"a" [1 1 2 2]
              "b" ["a" "b" "c" "d"]} 
             (tc/dataset)
             (tc/group-by "a"))]
  (seq (ds :data))) ;; seq is not necessary but Markdown treats `:data` as command here
```

```text
(
Group: 1 [2 2]:
1 | a  
---|---
1 | b  
Group: 2 [2 2]:
2 | c  
---|---
2 | d  
)
```

--------------------------------

### Slice Column with Start, End, and Step

Source: https://scicloj.github.io/tablecloth

Extracts a slice of a column with specified start, end, and step values. This example slices from index 0 to 9 with a step of 2.

```clojure
(-> (tcc/column (range 10))
    (tcc/slice 0 9 2))
```

```clojure
#tech.v3.dataset.column<int64>[5]
null
[0, 2, 4, 6, 8]
```

--------------------------------

### Get First N Rows

Source: https://scicloj.github.io/tablecloth

Retrieves the first N rows from the dataset. Defaults to the first 5 rows if N is not specified.

```clojure
(tc/head DS)
```

--------------------------------

### Load Anscombe Dataset

Source: https://scicloj.github.io/tablecloth

Loads the Anscombe dataset from a CSV file. This dataset is often used for statistical examples.

```clojure
(def anscombe (tc/dataset "data/anscombe.csv"))
```

--------------------------------

### Slice Column with Start and End

Source: https://scicloj.github.io/tablecloth

Extracts a slice of a column between specified start and end indices (exclusive of the end index). This example slices from index 1 up to (but not including) index 4.

```clojure
(-> (tcc/column (range 10))
    (tcc/slice 1 4))
```

```clojure
#tech.v3.dataset.column<int64>[4]
null
[1, 2, 3, 4]
```

--------------------------------

### Import Nippy File and Take Head

Source: https://scicloj.github.io/tablecloth

Load data from a `.nippy` file and display the first 10 rows using `tc/head`.

```clojure
(-> 
 (tc/dataset "data/example-genres.nippy")
 (tc/head 10))
```

--------------------------------

### Select Column Names by Regex

Source: https://scicloj.github.io/tablecloth

Use a regular expression to select column names that match the pattern. This example selects columns ending with '1' or '4'.

```clojure
(tc/column-names DS #".*[14]")
```

```clojure
(:V1 :V4)
```

--------------------------------

### Display Sample Dataset

Source: https://scicloj.github.io/tablecloth

Displays the content of the 'for-splitting' dataset, showing its rows and columns.

```clojure
for-splitting

```

--------------------------------

### Reorder columns by datatype

Source: https://scicloj.github.io/tablecloth

Reorder columns by selecting specific datatypes to appear at the beginning. This example places non-integer columns first, followed by integer columns.

```clojure
(tc/reorder-columns DS (tc/column-names DS (complement #{:int64}) :datatype))
```

--------------------------------

### Select Columns from Grouped Dataset

Source: https://scicloj.github.io/tablecloth

When applied to a grouped dataset, `select-columns` operates on each group independently. This example selects columns `:V2` and `:V3` from each group.

```clojure
(-> DS
    (tc/group-by :V1)
    (tc/select-columns [:V2 :V3])
    (tc/groups->map))
```

```clojure
{
```
1
```
__|
Group: 1 [5 2]: | 1 | 0.5  
---|---
3 | 1.5  
5 | 1.0  
7 | 0.5  
9 | 1.5  
```
2
```
__|
Group: 2 [4 2]: | 2 | 1.0  
---|---
4 | 0.5  
6 | 1.5  
8 | 1.0  
}
```

--------------------------------

### Create Dataset from Arrow File

Source: https://scicloj.github.io/tablecloth

Loads data from an Arrow file and selects specific columns. Ensure the 'data/alldtypes.arrow-feather' file exists and is in Arrow format.

```clojure
(->
 (tc/dataset "data/alldtypes.arrow-feather"  {:file-type :arrow})
 (tc/select-columns ["uints"
                     "longs"
                     "ubytes"
                     "strings"
                     "doubles"]))

```

--------------------------------

### Utility Column Creation

Source: https://scicloj.github.io/tablecloth

Shows how to quickly create columns filled with ones or zeros.

```APIDOC
## Utility Column Creation

### Description
Quickly create columns of ones or zeros.

### Code Examples

**Column of Ones:**
```clojure
(tcc/ones 10)
```

**Column of Zeros:**
```clojure
(tcc/zeros 10)
```
```

--------------------------------

### Create a Dataset

Source: https://scicloj.github.io/tablecloth

Creates a new dataset from a map of column names to sequences. Ensure the sequences are of equal length.

```clojure
(def DS (tc/dataset {:V1 (take 9 (cycle [1 2]))
                      :V2 (range 1 10)
                      :V3 (take 9 (cycle [0.5 1.0 1.5]))
                      :V4 (take 9 (cycle ["A" "B" "C"]})))

```

--------------------------------

### Get Row Numbers of Specific Rows by Group

Source: https://scicloj.github.io/tablecloth

Add a row ID, select columns, group by a key, select specific row indices within each group, and then `tc/ungroup` to get the row numbers of those specific rows per group.

```clojure
(-> DS
    (tc/add-column :row-id (range))
    (tc/select-columns [:V4 :row-id])
    (tc/group-by :V4)
    (tc/select-rows [0 2])
    (tc/ungroup))
```

--------------------------------

### Select Column by Name (Vector)

Source: https://scicloj.github.io/tablecloth

To select a column named `:all`, or a sequence/map, enclose it in a vector. This example returns an empty sequence as no such column exists.

```clojure
(tc/column-names DS [:all])
```

```clojure
()
```

--------------------------------

### Display tech-ml-version

Source: https://scicloj.github.io/tablecloth

Prints the retrieved tech.ml.dataset version.

```clojure
tech-ml-version
```

--------------------------------

### Get Row Number of First Observation by Group

Source: https://scicloj.github.io/tablecloth

Add a row ID using `range`, select relevant columns, group by a key, and then use `tc/first` to get the row number of the first observation per group. `tc/ungroup` flattens the result.

```clojure
(-> DS
    (tc/add-column :row-id (range))
    (tc/select-columns [:V4 :row-id])
    (tc/group-by :V4)
    (tc/first)
    (tc/ungroup))
```

--------------------------------

### Apply dec/inc to numerical columns

Source: https://scicloj.github.io/tablecloth

Update numerical columns by applying a sequence of functions. This example applies `dec` then `inc` to columns identified by `:type/numerical`.

```clojure
(tc/update-columns DS :type/numerical [(partial map dec)
                                        (partial map inc)])
```

--------------------------------

### Get the class of a Dataset

Source: https://scicloj.github.io/tablecloth

Retrieves the Java class of the Tablecloth Dataset object.

```clojure
(class DS)

```

--------------------------------

### Aggregate by Group

Source: https://scicloj.github.io/tablecloth

Performs aggregation on groups defined by one or more columns. This example calculates the sum of `:V2` for each unique value in `:V4`.

```clojure
(-> DS
    (tc/group-by [:V4])
    (tc/aggregate {:sumV2 #(dfn/sum (% :V2))}))
```

--------------------------------

### Column Creation

Source: https://scicloj.github.io/tablecloth

Demonstrates how to create an empty column and columns from vectors or sequences.

```APIDOC
## Column Creation

### Description
Creating an empty column is straightforward. You can also create a Column from a vector or a sequence.

### Code Examples

**Empty Column:**
```clojure
(tcc/column)
```

**Column from Vector:**
```clojure
(tcc/column [1 2 3 4 5])
```

**Column from Sequence:**
```clojure
(tcc/column `(1 2 3 4 5))
```
```

--------------------------------

### Update specific columns with functions

Source: https://scicloj.github.io/tablecloth

Assign specific functions to columns using a map. This example reverses column `:V1` and shuffles column `:V2`.

```clojure
^:note-to-test/skip
(tc/update-columns DS {:V1 reverse
                        :V2 (comp shuffle seq)})
```

--------------------------------

### Add a New Column

Source: https://scicloj.github.io/tablecloth

Adds a new column to the dataset, calculating its values based on an existing column. This example adds `:v5` using the natural logarithm of `:V1`.

```clojure
(tc/map-columns DS :v5 [:V1] dfn/log)
```

```clojure
(def DS (tc/add-column DS :v5 (dfn/log (DS :V1))))
```

--------------------------------

### Import Excel File

Source: https://scicloj.github.io/tablecloth

Load data from an Excel file (`.xlsx`). Ensure `dataset-io` is added to dependencies for automatic support.

```clojure
(tc/dataset "data/singleSheet.xlsx")
```

--------------------------------

### Remove a Single Column

Source: https://scicloj.github.io/tablecloth

Removes a specified column from the dataset. This example removes the `:v5` column.

```clojure
(def DS (tc/drop-columns DS :v5))
```

--------------------------------

### Select First 8 Columns of Dataset

Source: https://scicloj.github.io/tablecloth

Selects the first 8 columns from the loaded world bank population dataset for initial inspection.

```clojure
(->> world-bank-pop
     (tc/column-names)
     (take 8)
     (tc/select-columns world-bank-pop))
```

--------------------------------

### Display Stocks Dataset

Source: https://scicloj.github.io/tablecloth

Prints the loaded stocks-tidyr dataset to the console. Useful for inspecting the initial state of the data.

```clojure
stocks-tidyr
```

--------------------------------

### Load Production Dataset

Source: https://scicloj.github.io/tablecloth

Loads the production.csv dataset, used to demonstrate pivoting with multiple column selectors.

```clojure
(def production (tc/dataset "data/production.csv"))
```

--------------------------------

### Create Dataset using let-dataset macro

Source: https://scicloj.github.io/tablecloth

The `tc/let-dataset` macro allows defining columns using bindings, similar to R's `tibble` function, enabling column references within bindings.

```clojure
(tc/let-dataset [x (range 1 6)
                  y 1
                  z (dfn/+ x y)])
```

--------------------------------

### Get Dataset Name

Source: https://scicloj.github.io/tablecloth

Retrieves the current name associated with the dataset. This is often a file path or an identifier.

```clojure
(tc/dataset-name ds)
```

--------------------------------

### Convert to Wide Form on Time Column with Row Selection

Source: https://scicloj.github.io/tablecloth

Converts the stocks-long dataset to a wider format, pivoting on the 'time' column. It also selects a subset of rows before pivoting, demonstrating combined operations. Note: This example is marked to be skipped in tests.

```clojure
^:note-to-test/skip
(-> stocks-long
    (tc/select-rows (range 0 30 4))
    (tc/pivot->wider "time" :price {:drop-missing? false}))
```

--------------------------------

### Load Stocks Dataset

Source: https://scicloj.github.io/tablecloth

Loads the stocks dataset from a CSV file. The :key-fn keyword option ensures that column names are keywords.

```clojure
(defonce stocks (tc/dataset "https://raw.githubusercontent.com/techascent/tech.ml.dataset/master/test/data/stocks.csv" {:key-fn keyword}))
```

--------------------------------

### Display Cross Join Dataset

Source: https://scicloj.github.io/tablecloth

Displays the content of the 'cjds' dataset, which is used for cross join examples.

```clojure
cjds
```

--------------------------------

### Select Numerical Columns and Get Columns as Double Arrays

Source: https://scicloj.github.io/tablecloth

Select only numerical columns, take the first few rows, and then represent them as a double-precision floating-point array. This is useful for numerical computations.

```clojure
(-> ds
    (tc/select-columns :type/numerical)
    (tc/head)
    (tc/columns :as-double-arrays))
```

--------------------------------

### Summarize All Columns with Custom Function

Source: https://scicloj.github.io/tablecloth

Applies a custom aggregation function to all columns in a dataset. The example uses a custom max function that sorts the column in descending order and takes the first element.

```clojure
(tc/aggregate-columns DS :all (fn [col] (first (sort #(compare %2 %1) col))))
```

--------------------------------

### Select Column Elements by Index Array

Source: https://scicloj.github.io/tablecloth

Selects specific elements from a column based on an array of index positions. This example selects elements at indices 1 and 9.

```clojure
(-> (tcc/column (range 10))
    (tcc/select [1 9]))
```

```clojure
#tech.v3.dataset.column<int64>[2]
null
[1, 9]
```

--------------------------------

### Get Data Dimensions using data.table

Source: https://scicloj.github.io/tablecloth

Retrieves the dimensions (number of rows and columns) of the flights data.table in R.

```R
dim(flights)
```

--------------------------------

### Display a Dataset

Source: https://scicloj.github.io/tablecloth

Prints the contents of the dataset to the console.

```clojure
DS

```

--------------------------------

### Get Last Row of Dataset

Source: https://scicloj.github.io/tablecloth

Retrieves the last row from a dataset. This function is applicable to grouped datasets as well.

```clojure
(tc/last DS)
```

--------------------------------

### Get First Row of Dataset

Source: https://scicloj.github.io/tablecloth

Retrieves the first row from a dataset. This function is applicable to grouped datasets as well.

```clojure
(tc/first DS)
```

--------------------------------

### Generate Holdout Splits with Specific Steps

Source: https://scicloj.github.io/tablecloth

Creates data splits with specified percentages and a step size. `shuffle? false` ensures deterministic splits. Use this when you need precise control over the size of training and testing sets.

```clojure
^:note-to-test/skip
(-> (tc/split for-splitting :holdouts {:steps [0.05 0.95 1.5]
                                       :shuffle? false})
    (tc/group-by [:$split-id :$split-name]))
```

--------------------------------

### Get Column Count of Dataset

Source: https://scicloj.github.io/tablecloth

Retrieves the total number of columns in a dataset. Useful for understanding the dimensionality of the data.

```clojure
(tc/column-count ds)
```

--------------------------------

### Create Empty Dataset with Column Names

Source: https://scicloj.github.io/tablecloth

Provide a map with `:column-names` to an empty dataset to define column headers.

```clojure
(tc/dataset nil {:column-names [:a :b]})
```

--------------------------------

### Modify Column Values

Source: https://scicloj.github.io/tablecloth

Updates the values in an existing column based on a function. This example squares the values in `:V1`.

```clojure
(tc/map-columns DS :V1 [:V1] #(dfn/pow % 2))
```

```clojure
(def DS (tc/add-column DS :V1 (dfn/pow (DS :V1) 2)))
```

--------------------------------

### Display tablecloth-version

Source: https://scicloj.github.io/tablecloth

Prints the retrieved tablecloth version.

```clojure
tablecloth-version
```

--------------------------------

### Pivot to Wider Format with Custom Value Concatenation and Vector

Source: https://scicloj.github.io/tablecloth

Reshapes the income dataset to a wider format, spreading the 'variable' column. This example customizes value concatenation using `vector` and `vec` for column concatenation, while also ensuring missing values are not dropped.

```clojure
(tc/pivot->wider income "variable" ["estimate" "moe"] {:concat-columns-with vec
                                                        :concat-value-with vector
                                                        :drop-missing? false})
```

--------------------------------

### Display Dataset X

Source: https://scicloj.github.io/tablecloth

Displays the contents of the dataset 'x'. This is a preliminary step to visualize the data before performing join operations.

```clojure
x
```

--------------------------------

### Grouping by Multiple Columns (:V1 and :V3)

Source: https://scicloj.github.io/tablecloth

Illustrates grouping a dataset by two columns, ':V1' and ':V3', and retrieving the result as a sequence of datasets. The group names in this case are maps representing the combination of values from the grouping columns.

```clojure
(tc/group-by DS [:V1 :V3] {:result-type :as-seq})
```

--------------------------------

### View Content of Grouped Dataset as Map

Source: https://scicloj.github.io/tablecloth

Illustrates how to group a dataset by ':V1' and retrieve the content as a map. The output shows the structure of the map, including group names, group IDs, and the data within each group.

```clojure
(tc/columns (tc/group-by DS :V1) :as-map)
```

--------------------------------

### Get Columns as Map

Source: https://scicloj.github.io/tablecloth

Access all columns from a dataset as a map, where keys are column names. Useful for direct lookup by name.

```clojure
(keys (tc/columns ds :as-map))
```

--------------------------------

### Select a Column from Dataset

Source: https://scicloj.github.io/tablecloth

Access a specific column from a dataset by its name. This is a direct way to get a column's data.

```clojure
(ds "wind")
```

```clojure
(tc/column ds "date")
```

--------------------------------

### Display Stocks Dataset

Source: https://scicloj.github.io/tablecloth

Displays the loaded stocks dataset. This is a simple way to inspect the data after loading.

```clojure
stocks
```

--------------------------------

### Load US Rent and Income Dataset

Source: https://scicloj.github.io/tablecloth

Loads the US rent and income dataset from a CSV file. This dataset contains state-level rent and income data.

```clojure
(def income (tc/dataset "data/us_rent_income.csv"))
```

--------------------------------

### Get Row Count of Dataset

Source: https://scicloj.github.io/tablecloth

Retrieves the total number of rows in a dataset. This is a fundamental operation for understanding dataset size.

```clojure
(tc/row-count ds)
```

--------------------------------

### Create Dataset from Parquet File

Source: https://scicloj.github.io/tablecloth

Loads data from a Parquet file and selects specific columns. Ensure the 'data/titanic.parquet' file exists.

```clojure
(->
 (tc/dataset "data/titanic.parquet")
 (tc/select-columns ["PassengerId"
                     "Pclass"
                     "Age"
                     "Embarked" 
                     "Survived"]))

```

--------------------------------

### Select rows matching a regex pattern

Source: https://scicloj.github.io/tablecloth

Filters rows where the :V4 column starts with 'B'. Uses string matching with regex.

```clojure
(tc/select-rows DS (comp (partial re-matches #"^B") str :V4))

```

--------------------------------

### Create Dataset from URL

Source: https://scicloj.github.io/tablecloth

Loads data from a CSV file hosted at a URL. The dataset is stored in a defonce variable 'ds' for later use.

```clojure
(defonce ds (tc/dataset "https://vega.github.io/vega-lite/examples/data/seattle-weather.csv"))

```

```clojure
ds

```

--------------------------------

### Get Data Dimensions using Clojure Dataset API

Source: https://scicloj.github.io/tablecloth

Retrieves the shape (number of rows and columns) of the flights dataset in Clojure.

```Clojure
(tc/shape flights)
```

--------------------------------

### Access Column Element by Index

Source: https://scicloj.github.io/tablecloth

Accesses an element in a column at a specific index using the `get` method, similar to Clojure vectors.

```clojure
(-> (tcc/column [1 2 3 4 5])
    (get 3))
```

```clojure
4
```

--------------------------------

### Create Dataset from Array (as rows with names)

Source: https://scicloj.github.io/tablecloth

Specify column names and layout when creating a dataset from an array of arrays.

```clojure
(-> (map int-array [[1 2] [3 4] [5 6]])
    (into-array)
    (tc/dataset {:layout :as-rows
                 :column-names [:a :b]}))
```

--------------------------------

### Import CSV File

Source: https://scicloj.github.io/tablecloth

Load data directly from a CSV file using `tc/dataset`.

```clojure
(tc/dataset "data/family.csv")
```

--------------------------------

### Get Last N Rows

Source: https://scicloj.github.io/tablecloth

Retrieves the last N rows from the dataset. Defaults to the last 5 rows if N is not specified.

```clojure
(tc/tail DS)
```

--------------------------------

### Convert Grouped Dataset to Sequence of Groups

Source: https://scicloj.github.io/tablecloth

Shows how to group a dataset by column 'a' and then explicitly convert the grouped result into a sequence of datasets using `groups->seq`.

```clojure
(-> {"a" [1 1 2 2]
     "b" ["a" "b" "c" "d"]} 
    (tc/dataset)
    (tc/group-by "a")
    (tc/groups->seq))
```

```text
(
Group: 1 [2 2]:
1 | a  
---|---
1 | b  
Group: 2 [2 2]:
2 | c  
---|---
2 | d  
)
```

--------------------------------

### Create Dataset from Array of Arrays (as columns)

Source: https://scicloj.github.io/tablecloth

Use the `:layout :as-columns` option to interpret inner arrays as columns when creating a dataset from an array of arrays.

```clojure
(-> (map int-array [[1 2] [3 4] [5 6]])
    (into-array)
    (tc/dataset {:layout :as-columns}))
```

--------------------------------

### Get Random Rows with Seed

Source: https://scicloj.github.io/tablecloth

Selects a specified number of random rows, using a seed for reproducibility. Repetition is allowed by default.

```clojure
(tc/random DS 5 {:seed 42})
```

--------------------------------

### Get Random Row from Dataset

Source: https://scicloj.github.io/tablecloth

Selects a single random row from the dataset. Use the `:seed` option to ensure reproducible results.

```clojure
(tc/rand-nth DS)
```

```clojure
(tc/rand-nth DS {:seed 42})
```

--------------------------------

### Load Construction Dataset

Source: https://scicloj.github.io/tablecloth

Loads the construction dataset from the 'data/construction.csv' file into a Tablecloth dataset named `construction`.

```clojure
(def construction (tc/dataset "data/construction.csv"))
```

--------------------------------

### Order by Column and Custom Function

Source: https://scicloj.github.io/tablecloth

Orders the dataset first by a specified column in descending order, and then by a custom function (product of three columns) in ascending order.

```clojure
(tc/order-by DS [:V4 (fn [row] (* (:V1 row) (:V2 row) (:V3 row)))] [:desc :asc])
```

--------------------------------

### Get Rows as Sequence of Maps

Source: https://scicloj.github.io/tablecloth

Retrieve rows from a dataset as a sequence of maps. Each map represents a row with column names as keys.

```clojure
(clojure.pprint/pprint (take 2 (tc/rows ds :as-maps)))
```

--------------------------------

### Get Rows as Sequence of Sequences

Source: https://scicloj.github.io/tablecloth

Extract rows from a dataset as a sequence of sequences. Each inner sequence represents a row's values.

```clojure
(take 2 (tc/rows ds))
```

--------------------------------

### Load and Prepare Billboard Dataset

Source: https://scicloj.github.io/tablecloth

Loads the billboard dataset and drops boolean columns. This prepares the data for further transformation by removing irrelevant columns.

```clojure
(def bilboard (-> (tc/dataset "data/billboard.csv.gz")
                  (tc/drop-columns :type/boolean))) ;; drop some boolean columns, tidyr just skips them
```

--------------------------------

### Get Dataset Shape

Source: https://scicloj.github.io/tablecloth

Returns the shape of the dataset as a vector containing the row count and column count, e.g., [rows columns].

```clojure
(tc/shape ds)
```

--------------------------------

### Define Dataset X for Joins

Source: https://scicloj.github.io/tablecloth

Defines a sample dataset named 'x' with columns 'Id', 'X1', and 'XY'. This dataset is used in subsequent join operations.

```clojure
(def x (tc/dataset {"Id" ["A" "B" "C" "C"]
                     "X1" [1 3 5 7]
                     "XY" ["x2" "x4" "x6" "x8"]}))
```

--------------------------------

### Join Columns with Default Settings

Source: https://scicloj.github.io/tablecloth

Joins specified columns into a new column named :joined using default settings. Source columns are dropped by default.

```clojure
(tc/join-columns DSm :joined [:V1 :V2 :V4])
```

--------------------------------

### Create Dataset from Sequence of Maps with Varying Data

Source: https://scicloj.github.io/tablecloth

Handles sequences of maps where values might be collections, creating columns accordingly.

```clojure
(tc/dataset [{:a 1 :b [1 2 3]} {:a 2 :b [3 4]}])
```

--------------------------------

### Replace Values Conditionally

Source: https://scicloj.github.io/tablecloth

Replaces values in a column based on a condition. This example sets values in `:V2` to 0.0 if they are less than 4.0.

```clojure
(def DS (tc/map-columns DS :V2 [:V2] #(if (< % 4.0) 0.0 %)))
```

--------------------------------

### Load Dataset

Source: https://scicloj.github.io/tablecloth

Loads a dataset from a CSV file. This is a common first step before performing data transformations.

```clojure
(def family (tc/dataset "data/family.csv"))
```

--------------------------------

### Define Column B

Source: https://scicloj.github.io/tablecloth

Defines a column named 'b' using a range of integers. Used in conjunction with column 'a' for operation examples.

```clojure
(def b (tcc/column (range 4)))
```

--------------------------------

### Select Random Row Strategy

Source: https://scicloj.github.io/tablecloth

Uses the `:random` strategy to select a random row when applying uniqueness. This example is skipped in tests.

```clojure
^:note-to-test/skip
(tc/unique-by DS :V1 {:strategy :random})
```

--------------------------------

### Create Dataset from Map with Multiple Columns

Source: https://scicloj.github.io/tablecloth

A map with multiple key-value pairs creates a dataset with multiple columns.

```clojure
(tc/dataset {:A [3 4 5] :B ["X" "Y" "Z"]})
```

--------------------------------

### Select Columns by Excluding Name Prefix

Source: https://scicloj.github.io/tablecloth

Selects columns whose names do not start with a specific prefix, using a negated string predicate.

```clojure
(tc/select-columns DS #(not (clojure.string/starts-with? (name %) "V2")))
```

--------------------------------

### Access Column Element by Index (nth)

Source: https://scicloj.github.io/tablecloth

Accesses an element in a column at a specific index using the `nth` method, providing an alternative to `get`.

```clojure
(-> (tcc/column [1 2 3 4 5])
    (nth 3))
```

```clojure
4
```

--------------------------------

### Load World Bank Population Dataset

Source: https://scicloj.github.io/tablecloth

Loads the world bank population dataset from a gzipped CSV file into a Tablecloth dataset.

```clojure
(def world-bank-pop (tc/dataset "data/world_bank_pop.csv.gz"))
```

--------------------------------

### Load Dataset

Source: https://scicloj.github.io/tablecloth

Loads a dataset from a specified CSV file path. This is a foundational step before performing any data manipulations.

```clojure
(def who (tc/dataset "data/who.csv.gz"))
```

--------------------------------

### Create Dataset for Extraction

Source: https://scicloj.github.io/tablecloth

Defines a dataset with a single column 'x' containing hyphen-separated strings, used for demonstrating column extraction.

```clojure
(def df-extract (tc/dataset {:x [nil "a-b" "a-d" "b-c" "d-e"]}))
```

--------------------------------

### Get Values (Groups) of Grouped Dataset as Map

Source: https://scicloj.github.io/tablecloth

Shows how to group a dataset by ':V1' and extract only the values (the grouped datasets) when the result type is ':as-map'.

```clojure
(vals (tc/group-by DS :V1 {:result-type :as-map}))
```

--------------------------------

### Get Keys of Grouped Dataset as Map

Source: https://scicloj.github.io/tablecloth

Demonstrates grouping a dataset by ':V1' and extracting only the keys (group names) when the result type is set to ':as-map'.

```clojure
(keys (tc/group-by DS :V1 {:result-type :as-map}))
```

```text
(1 2)
```

--------------------------------

### Create Dataset with Spaces for Separation

Source: https://scicloj.github.io/tablecloth

Defines a dataset with a single column 'x' containing strings with spaces, for demonstrating separation by space.

```clojure
(def df-separate2 (tc/dataset {:x ["a" "a b" nil "a b c"]}))
```

--------------------------------

### Define Dataset X for Bind

Source: https://scicloj.github.io/tablecloth

Defines a dataset 'x' with a single column 'V1' containing integers 1, 2, and 3.

```clojure
(def x (tc/dataset {:V1 [1 2 3]}))
```

--------------------------------

### Get Single Entry using get-entry

Source: https://scicloj.github.io/tablecloth

Retrieve a single value from the dataset using the `get-entry` function, specifying the column name and row index.

```clojure
(tc/get-entry ds "wind" 2)
```

--------------------------------

### Get Single Entry using get-in

Source: https://scicloj.github.io/tablecloth

Retrieve a single value from the dataset using the `get-in` function, specifying the column name and row index.

```clojure
(get-in ds ["wind" 2])
```

--------------------------------

### Get Dataset Column Metadata

Source: https://scicloj.github.io/tablecloth

Fetches metadata for each column in the dataset, including name, type, row count, and whether it contains null values.

```clojure
(tc/info ds :columns)
```

--------------------------------

### Sort Column with Comparator

Source: https://scicloj.github.io/tablecloth

Sorts a column using a custom comparator function. This example sorts based on the `:position` key within map elements.

```clojure
(-> (tcc/column [{:position 2
                  :text "and then stopped"}
                 {:position 1
                  :text "I ran fast"}])
    (tcc/sort-column (fn [a b] (< (:position a) (:position b)))))
```

--------------------------------

### Display Dataset with Spaces

Source: https://scicloj.github.io/tablecloth

Displays the content of a dataset containing strings with spaces, showing how initial data is represented.

```clojure
df-separate2
```

--------------------------------

### Create Empty Dataset

Source: https://scicloj.github.io/tablecloth

Use `tc/dataset` with no arguments to create an empty dataset.

```clojure
(tc/dataset)
```

--------------------------------

### Create Dataset with Mixed Delimiters

Source: https://scicloj.github.io/tablecloth

Defines a dataset with a single column 'x' containing strings with various delimiters like '?', '.', and ':', used for demonstrating flexible separation.

```clojure
(def df-separate3 (tc/dataset {:x ["a?b" nil "a.b" "b:c"]}))
```

--------------------------------

### Drop columns from grouped dataset

Source: https://scicloj.github.io/tablecloth

When applied to a grouped dataset, `drop-columns` operates on each group independently. The example shows dropping specific columns after grouping.

```clojure
(-> DS
    (tc/group-by :V1)
    (tc/drop-columns [:V2 :V3])
    (tc/groups->map))
```

--------------------------------

### Grouping by Provided Row Indexes

Source: https://scicloj.github.io/tablecloth

Shows how to group a dataset using custom maps of group names to row indexes. This allows a single row to be assigned to multiple groups.

```clojure
(tc/group-by DS {"group-a" [1 2 1 2]
                  "group-b" [5 5 5 1]} {:result-type :as-seq})
```

--------------------------------

### Split Dataset with Bootstrap Sampling

Source: https://scicloj.github.io/tablecloth

This snippet demonstrates splitting a dataset using bootstrap sampling with `tc/split->seq`. It groups the data by `:group`, then applies bootstrap sampling with specified parameters like `:seed`, `:ratio`, and `:repeats`.

```clojure
^:note-to-test/skip
(-> for-splitting
    (tc/group-by :group)
    (tc/split->seq :bootstrap {:partition-selector :partition :seed 11 :ratio 0.8 :repeats 2})
    (first))
```

--------------------------------

### Create Dataset from Sequence of Maps

Source: https://scicloj.github.io/tablecloth

A sequence of maps creates a dataset where keys become column names and values populate rows.

```clojure
(tc/dataset [{:a 1 :b 3} {:b 2 :a 99}])
```

--------------------------------

### Create Dataset from Array (as rows)

Source: https://scicloj.github.io/tablecloth

Reading from arrays defaults to interpreting elements as rows.

```clojure
(tc/dataset [[:a 1] [:b 2] [:c 3]])
```

--------------------------------

### Get Random Non-Repeating Rows

Source: https://scicloj.github.io/tablecloth

Selects a specified number of unique random rows from the dataset. Ensure the requested number does not exceed the dataset's row count.

```clojure
(tc/random DS 5 {:repeat? false})
```

--------------------------------

### Get Random Rows with Repetition

Source: https://scicloj.github.io/tablecloth

Selects random rows from the dataset, allowing for repetitions. By default, it selects a number of rows equal to the dataset's row count.

```clojure
(tc/random DS)
```

```clojure
(tc/random DS 5)
```

--------------------------------

### Define Dataset Z for Bind

Source: https://scicloj.github.io/tablecloth

Defines a dataset 'z' with two columns, 'V1' and 'V2'. 'V1' contains 7, 8, 9 and 'V2' contains three zeros.

```clojure
(def z (tc/dataset {:V1 [7 8 9]
                     :V2 [0 0 0]}))
```

--------------------------------

### Remove Columns Using a Complement Set

Source: https://scicloj.github.io/tablecloth

Removes columns from the dataset by specifying a set of columns to keep using `complement`. This example keeps all columns except `:V3`.

```clojure
(def DS (tc/select-columns DS (complement #{:V3})))
```

--------------------------------

### Load Dataset for Pivoting

Source: https://scicloj.github.io/tablecloth

Loads the fish_encounters.csv dataset. This is a prerequisite for demonstrating the pivot->wider function.

```clojure
(def fish (tc/dataset "data/fish_encounters.csv"))
```

--------------------------------

### Create Dataset for Separation

Source: https://scicloj.github.io/tablecloth

Defines a dataset with a single column 'x' containing string data, used for demonstrating column separation.

```clojure
(def df-separate (tc/dataset {:x [nil "a.b" "a.d" "b.c"]}))
```

--------------------------------

### Create Dataset from Mixed Array Types (as columns)

Source: https://scicloj.github.io/tablecloth

Handles arrays containing different data types, mapping them to columns with specified names.

```clojure
(-> (map to-array [[:a :z] ["ee" "ww"] [9 10]])
    (into-array)
    (tc/dataset {:column-names [:a :b :c]
                 :layout :as-columns}))
```

--------------------------------

### Select Rows and Columns

Source: https://scicloj.github.io/tablecloth

Selects a specific row from a column. Use `tc/select-rows` for row selection and `tc/select-columns` for column selection.

```clojure
(-> DS
    (tc/select-rows 4)
    (tc/select-columns :V3)) ;; select forth row from `:V3` column
```

```clojure
(-> DS
    (tc/select :V3 4)) ;; select forth row from `:V3` column
```

--------------------------------

### Select Numerical Columns and Get Rows as Double Arrays

Source: https://scicloj.github.io/tablecloth

Select only numerical columns, take the first few rows, and then represent them as a double-precision floating-point array. This is useful for numerical computations.

```clojure
(-> ds
    (tc/select-columns :type/numerical)
    (tc/head)
    (tc/rows :as-double-arrays))
```

--------------------------------

### Join and Separate Columns Together

Source: https://scicloj.github.io/tablecloth

Demonstrates joining multiple columns into a single map column, and then separating that map column back into individual columns using a custom function.

```clojure
(-> DSm
    (tc/join-columns :joined [:V1 :V2 :V4] {:result-type :map})
    (tc/separate-column :joined [:v1 :v2 :v4] (juxt :V1 :V2 :V4)))
```

--------------------------------

### Unroll One by One for Cartesian Product

Source: https://scicloj.github.io/tablecloth

Demonstrates how unrolling columns sequentially can lead to a cartesian product of the unfolded sequences. This can result in a significantly larger dataset.

```clojure
(-> DS
    (tc/fold-by [:V4 :V1])
    (tc/unroll [:V2])
    (tc/unroll [:V3]))
```

--------------------------------

### Define Dataset for Crosstab

Source: https://scicloj.github.io/tablecloth

Defines a sample dataset named 'ctds' using a map with vector values for columns 'a', 'b', and 'c'. This dataset is used in subsequent crosstab examples.

```clojure
(def ctds (tc/dataset {:a [:foo :foo :bar :bar :foo :foo]
                       :b [:one :one :two :one :two :one]
                       :c [:dull :dull :shiny :dull :dull :shiny]}))
```

--------------------------------

### Map columns on a grouped dataset

Source: https://scicloj.github.io/tablecloth

Apply `map-columns` to a grouped dataset to perform operations within each group. This example sums numerical columns per group and then regroups the data.

```clojure
(-> DS
    (tc/group-by :V4)
    (tc/map-columns :sum-of-numbers
                     (tc/column-names DS  #{:int64 :float64} :datatype)
                     (fn [& rows]
                       (reduce + rows)))
    (tc/ungroup))
```

--------------------------------

### Create Dataset from Excel File

Source: https://scicloj.github.io/tablecloth

Loads data from an Excel file. Tablecloth automatically supports Excel, Arrow, and Parquet if 'dataset-io' is added to your project dependencies.

```clojure
(tc/dataset "data/singleSheet.xlsx")

```

--------------------------------

### Select Single Column by Name as Dataset (List)

Source: https://scicloj.github.io/tablecloth

Selects a single column from the dataset by its name, provided as a list, and returns it as a new dataset.

```clojure
(tc/select-columns DS [:V2]) ;; as dataset
```

--------------------------------

### Require Tablecloth Column API

Source: https://scicloj.github.io/tablecloth

This code requires the Tablecloth Column API and aliases it as `tcc` for easier use throughout the project. This is a common setup step before interacting with column functionalities.

```clojure
(require '[tablecloth.column.api :as tcc])
```

--------------------------------

### Cast Data (Long to Wide) - String Coercion and Count

Source: https://scicloj.github.io/tablecloth

This example demonstrates casting data to wide format after coercing values to strings and then counting occurrences. It handles boolean-like string categories.

```clojure
(-> mDS
    (tc/map-columns :value #(str (> % 5))) ;; coerce to strings
    (tc/pivot->wider :value :variable {:fold-fn vec})
    (tc/update-columns ["true" "false"] (partial map #(if (sequential? %) (count %) 1))))
```

--------------------------------

### Display Dataset

Source: https://scicloj.github.io/tablecloth

Prints the defined dataset 'DS' to the console.

```clojure
DS
```

--------------------------------

### Pivot Production with Multiple Columns

Source: https://scicloj.github.io/tablecloth

Pivots the production dataset using both 'product' and 'country' columns to create new columns, with 'production' as the value column. Column names are joined using the default separator.

```clojure
(tc/pivot->wider production ["product" "country"] "production")
```

--------------------------------

### Remove Duplicates by Custom Function

Source: https://scicloj.github.io/tablecloth

Removes duplicate rows based on the result of a custom function applied to each row. This example uses the modulo 3 of the value in column :V2 to define uniqueness.

```clojure
(tc/unique-by DS (fn [m] (mod (:V2 m) 3)))
```
Tokens:
Raw