### Install SmarterCSV Gem

Source: https://github.com/tilo/smarter_csv/blob/main/README.md

This snippet shows how to add the SmarterCSV gem to your application's Gemfile and install it using Bundler, or how to install it directly using the gem command.

```Ruby
gem 'smarter_csv'
```

```Shell
$ bundle

```

```Shell
$ gem install smarter_csv
```

--------------------------------

### Install Smarter CSV Gem

Source: https://github.com/tilo/smarter_csv/wiki/Getting-Started

This snippet shows the command to install the Smarter CSV gem using RubyGems. Ensure you have Ruby installed and configured.

```ruby
gem install smarter_csv
```

--------------------------------

### Populate Database with SmarterCSV (Ruby)

Source: https://github.com/tilo/smarter_csv/blob/main/docs/examples.md

Shows how to use SmarterCSV to process a CSV file and populate a database, such as MySQL or MongoDB. It demonstrates using a block to process each row (hash) and includes an example of key mapping for transformations.

```ruby
# without using chunks:
filename = '/tmp/some.csv'
options = {:key_mapping => {:unwanted_row => nil, :old_row_name => :new_name}}
n = SmarterCSV.process(filename, options) do |array|
      # we're passing a block in, to process each resulting hash / =row (the block takes array of hashes)
      # when chunking is not enabled, there is only one hash in each array
      MyModel.create( array.first )
end

 => returns number of chunks / rows we processed
```

--------------------------------

### Per Key Value Converters - Example 2

Source: https://github.com/tilo/smarter_csv/blob/main/docs/basic_write_api.md

Provides a more advanced example of `value_converters`. It formats the 'balance' key as currency with specific rules for floats and integers, and maps the 'active' key to emojis.

```Ruby
options = {
 value_converters: {
 active: ->(v) { !!v ? '✅' : '❌' },
 balance: ->(v) do
 case v
 when Float
 '$%.2f' % v.round(2)
 when Integer
 "$#{v}"
 else
 v.to_s
 end
 end,
 }
}
```

--------------------------------

### Per Key Value Converters - Example 1

Source: https://github.com/tilo/smarter_csv/blob/main/docs/basic_write_api.md

Shows a basic example of using `value_converters` to transform the boolean value of the 'active' key into 'YES' or 'NO' strings.

```Ruby
options = {
 value_converters: {
 active: ->(v) { !!v ? 'YES' : 'NO' },
 }
}
```

--------------------------------

### Process CSV for Sidekiq Batch Jobs (Ruby)

Source: https://github.com/tilo/smarter_csv/blob/main/docs/examples.md

Illustrates how to process a CSV file with SmarterCSV and insert batch jobs into Sidekiq. It shows how to configure chunk size and pass processed data to Sidekiq workers for parallel processing.

```ruby
filename = '/tmp/input.csv' # CSV file containing ids or data to process
options = { :chunk_size => 100 }
n = SmarterCSV.process(filename, options) do |chunk|
  Sidekiq::Client.push_bulk(
    'class' => SidekiqIndividualWorkerClass,
    'args' => chunk,
  )
  # OR:
  # SidekiqBatchWorkerClass.process_async(chunk ) # pass an array of hashes to Sidekiq workers for parallel processing
end
=> returns number of chunks
```

--------------------------------

### Ruby: Header Transformations Configuration

Source: https://github.com/tilo/smarter_csv/wiki/Transmogrify-my-Data

Demonstrates how to configure header transformations in SmarterCSV using an array of Procs. The `:none` symbol can be used to start with an empty configuration, and custom Procs can be defined for specific transformations.

```ruby
header_transformations: [ :none, :keys_as_strings ]
```

--------------------------------

### Process CSV to Array of Hashes (Ruby)

Source: https://github.com/tilo/smarter_csv/blob/main/docs/examples.md

Demonstrates how SmarterCSV processes a CSV file into an array of hashes. Each hash represents a row, with keys corresponding to column headers. Only columns with non-null values are included in the resulting hashes.

```ruby
$ cat pets.csv
first name,last name,dogs,cats,birds,fish
Dan,McAllister,2,,,
Lucy,Laweless,,5,,
Miles,O'Brian,,,,21
Nancy,Homes,2,,1,
$ irb
> require 'smarter_csv'
 => true
> pets_by_owner = SmarterCSV.process('/tmp/pets.csv')
 => [ {:first_name=>"Dan", :last_name=>"McAllister", :dogs=>"2"},
      {:first_name=>"Lucy", :last_name=>"Laweless", :cats=>"5"},
      {:first_name=>"Miles", :last_name=>"O'Brian", :fish=>"21"},
      {:first_name=>"Nancy", :last_name=>"Homes", :dogs=>"2", :birds=>"1"}
    ]
```

--------------------------------

### Process iTunes DB Dump with Comments

Source: https://github.com/tilo/smarter_csv/blob/main/docs/row_col_sep.md

Reads a file with CRTL-A as the column separator and CTRL-B\n as the record separator. It uses `comment_regexp` to ignore lines starting with '#'. The example also shows chunking and header mapping similar to the first example.

```ruby
# Consider a file with CRTL-A as col_separator, and with CTRL-B\n as record_separator (hello iTunes!)
filename = '/tmp/strange_db_dump'
options = {
  :col_sep => "\cA", :row_sep => "\cB\n", :comment_regexp => /^#/, 
  :chunk_size => 100 , :key_mapping => {:export_date => nil, :name => :genre},
}
n = SmarterCSV.process(filename, options) do |chunk|
  SidekiqWorkerClass.process_async(chunk) # pass an array of hashes to Sidekiq workers for parallel processing
end
=> returns number of chunks
```

--------------------------------

### Global Value Converter with :_all - Ruby

Source: https://github.com/tilo/smarter_csv/blob/main/docs/basic_write_api.md

Demonstrates how to use the :_all global value converter in Smarter CSV to apply transformations to all fields. This example specifically adds double quotes around all string values, requiring `disable_auto_quoting` to be set to true.

```Ruby
options = {
  value_converters: {
    disable_auto_quoting: true, # ⚠️ Important: turn off auto-quoting because we're messing with it below
    active: ->(v) { !!v ? 'YES' : 'NO' },
    _all: ->(_k, v) { v.is_a?(String) ? "\"#{v}\"" : v } # only double-quote string fields
  }
}
```

--------------------------------

### Ruby Number Formatting Examples

Source: https://github.com/tilo/smarter_csv/wiki/Parsing-Floats,-Integers,-Money,-and-Dates

Illustrates common number formatting styles used in different countries, including decimal separators and grouping. This is provided as context for why SmarterCSV does not automatically parse these formats.

```Text
|Style| Countries |
---------------------------------------------------------------------------
| 1,234,567.89 | Australia, Canada (English-speaking, unofficial), China, Hong Kong, Ireland, Israel, Japan, Korea, Malaysia, Mexico, New Zealand, Pakistan, Philippines, Singapore, Taiwan, Thailand, United Kingdom, United States |
| 1234567.89 | SI style (English version), Canada (English-speaking), China, Sri Lanka, Switzerland (officially encouraged for currency numbers) |
| 1234567,89 | SI style (French version), Albania, Austria, Belgium (French), Belgium (Dutch: alternative), Brazil, Bulgaria, Canada (French-speaking), Costa Rica, Croatia, Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Italy, Kosovo, Latin Europe, Netherlands (alternative), Norway, Peru, Poland, Portugal, Romania, Russia, Serbia, Slovakia, Slovenia, South Africa, Spain (official), Sweden, Switzerland (officially encouraged for non-currency numbers), Ukraine |
| 1,234,567·89 | Ireland, Malaysia, Malta, Philippines, Singapore, Taiwan, Thailand, United Kingdom (older, typically hand written)[32] |
| 1.234.567,89 | Argentina, Austria, Belgium (Dutch: most common), Bosnia and Herzegovina, Brazil, Chile, Croatia,[33][34] Denmark, Greece, Indonesia, Italy, Netherlands (most common), Portugal, Romania, Russia, Slovenia, Spain,[35] Sweden (not recommended), Turkey |
| 12,34,567.89 | Bangladesh, India (see Indian Numbering System) |
| 1'234'567.89 | Switzerland (printed, computing, currency, everyday use), Liechtenstein |
| 1'234'567,89 | Switzerland (handwriting) |
| 1.234.567'89 | Spain (handwriting) |
| 123,4567.89 | China (based on powers of 10 000 — see Chinese numerals) |
---------------------------------------------------------------------------
```

--------------------------------

### Process CSV with Exotic Separators and Comments

Source: https://github.com/tilo/smarter_csv/wiki/The-Basics

Shows how to handle CSV files with non-standard column and row separators, such as control characters. It also demonstrates ignoring lines that start with a specific comment pattern using `comment_regexp` and processing in chunks.

```ruby
filename = '/tmp/strange_db_dump'   # a file with CRTL-A as col_separator, and with CTRL-B\n as record_separator (hello iTunes!)
options = {
  :col_sep => "\cA", :row_sep => "\cB\n", :comment_regexp => /^#/, :chunk_size => 100 
}
n = SmarterCSV.process(filename, options) do |chunk|
    Resque.enque( ResqueWorkerClass, chunk ) # pass chunks of CSV-data to Resque workers for parallel processing
end
=> returns number of chunks
```

--------------------------------

### Ruby SmarterCSV Custom Proc Example

Source: https://github.com/tilo/smarter_csv/wiki/Parsing-Floats,-Integers,-Money,-and-Dates

Demonstrates how to define a custom Proc in SmarterCSV v2.0 to handle country-specific number formatting. The suggested approach involves stripping decimal separators before attempting numerical conversion.

```Ruby
# Example of a custom Proc for SmarterCSV v2.0
# This is a conceptual example, actual implementation would depend on specific needs.

def custom_number_parser(value)
  # Attempt to strip common decimal separators and grouping characters
  # This is a simplified example; a real-world scenario might need more robust logic
  normalized_value = value.to_s.gsub(/[.,' ]/, '')

  # Try converting to float or integer
  begin
    Float(normalized_value)
  rescue ArgumentError
    begin
      Integer(normalized_value)
    rescue ArgumentError
      value # Return original value if conversion fails
    end
  end
end

# Usage within SmarterCSV (conceptual):
# SmarterCSV.process(file, { hash_transformations: { 'column_name' => method(:custom_number_parser) } })

```

--------------------------------

### Process iTunes DB Dump with Custom Separators and Comments

Source: https://github.com/tilo/smarter_csv/blob/main/docs/row_col_sep.md

Reads an iTunes DB dump file using custom column ('\cA') and row ('\cB\n') separators. It also configures comment lines starting with '#' and maps specific header fields. The data is processed in chunks and passed to Sidekiq workers for asynchronous processing.

```ruby
filename = '/tmp/itunes_db_dump'
options = {
  :col_sep => "\cA", :row_sep => "\cB\n", :comment_regexp => /^#/, 
  :chunk_size => 100 , :key_mapping => {export_date: nil, name: :genre},
}
n = SmarterCSV.process(filename, options) do |chunk|
  SidekiqWorkerClass.process_async(chunk) # pass an array of hashes to Sidekiq workers for parallel processing
end
=> returns number of chunks
```

--------------------------------

### Contributing to SmarterCSV

Source: https://github.com/tilo/smarter_csv/blob/main/README.md

Standard Git workflow for contributing to the SmarterCSV project, including forking, creating branches, committing changes, and submitting pull requests.

```Shell
git checkout -b my-new-feature
git commit -am 'Added some feature'
git push origin my-new-feature

```

--------------------------------

### Process CSV with Custom Column and Row Separators

Source: https://github.com/tilo/smarter_csv/blob/main/docs/row_col_sep.md

Reads a file with non-standard column ('#') and row ('|') separators. This example demonstrates explicitly setting the `col_sep` and `row_sep` options to handle such formats.

```ruby
filename = '/tmp/input_file.txt'
recordsA = SmarterCSV.process(filename, {col_sep: "#", row_sep: "|"}) 

=> returns an array of hashes
```

--------------------------------

### SmarterCSV 2.x Defaults

Source: https://github.com/tilo/smarter_csv/wiki/Upgrading-from-SmarterCSV-1.x-to-2.x

This snippet shows the default configurations for SmarterCSV version 2.x, including header transformations, header validations, data transformations, data validations, hash transformations, and hash validations.

```ruby
{
    header_transformations: [:keys_as_symbols],
    header_validations:   [ :unique_headers ],
    data_transformations: [ :replace_blank_with_nil ],
    data_validations: [],
    hash_transformations: [ :strip_spaces, :remove_blank_values ],
    hash_validations: []
  }
```

--------------------------------

### Smarter CSV: Handling Missing Keys

Source: https://github.com/tilo/smarter_csv/blob/main/docs/options.md

Configures how Smarter CSV handles missing keys during the mapping process. It can ignore all missing keys, make specific keys optional, or raise an exception.

```ruby
smarter_csv.silence_missing_keys = true
# or
smarter_csv.silence_missing_keys = [:key1, :key2]

```

--------------------------------

### Generate CSV with Simplified Interface

Source: https://github.com/tilo/smarter_csv/blob/main/docs/basic_write_api.md

Demonstrates the simplified interface for generating CSV files using SmarterCSV. It takes a filename and options, and processes data in batches, writing each record to the CSV.

```Ruby
SmarterCSV.generate(filename, options) do |csv_writer|

 MyModel.find_in_batches(batch_size: 100) do |batch|
 batch.pluck(:name, :description, :instructor).each do |record|
 csv_writer << record
 end
 end

end
```

--------------------------------

### SmarterCSV Process Duplicate Headers

Source: https://github.com/tilo/smarter_csv/blob/main/docs/header_transformations.md

Demonstrates how SmarterCSV processes duplicate headers by default and with a custom suffix. It shows how to map these transformed headers to meaningful keys using `key_mapping`.

```Ruby
data = SmarterCSV.process('/tmp/dupe.csv')
   => [{:name=>"Carl", :name2=>"Edward", :name3=>"Sagan"}]
```

```Ruby
data = SmarterCSV.process('/tmp/dupe.csv', {duplicate_header_suffix: '_'})
```

```Ruby
options = {
  duplicate_header_suffix: '_', 
  key_mapping: {
    name: :first_name, 
    name_2: :middle_name, 
    name_3: :last_name,
  }
}
data = SmarterCSV.process('/tmp/dupe.csv', options)
```

--------------------------------

### Generate CSV with Full Interface

Source: https://github.com/tilo/smarter_csv/blob/main/docs/basic_write_api.md

Illustrates the full interface for writing CSV files with SmarterCSV. It involves creating a Writer instance, writing data in batches, and finalizing the process.

```Ruby
csv_writer = SmarterCSV::Writer.new(file_path, options)

MyModel.find_in_batches(batch_size: 100) do |batch|
 batch.pluck(:name, :description, :instructor).each do |record|
 csv_writer << record
 end

csv_writer.finalize
```

--------------------------------

### Smarter CSV: Key Type Handling

Source: https://github.com/tilo/smarter_csv/blob/main/docs/options.md

Determines whether to use strings or symbols as keys in the resulting hashes. `strings_as_keys` set to true uses strings.

```ruby
smarter_csv.strings_as_keys = true

```

--------------------------------

### Populate MongoDB with CSV Chunks (Ruby)

Source: https://github.com/tilo/smarter_csv/blob/main/docs/batch_processing.md

Illustrates how to efficiently populate a MongoDB database by processing a CSV file in chunks. It uses SmarterCSV with specified chunk size and key mapping, passing each chunk to a block that inserts the data into a MongoDB collection using `insert_all`.

```ruby
filename = '/tmp/some.csv'
options = {:chunk_size => 100, :key_mapping => {:unwanted_row => nil, :old_row_name => :new_name}}
n = SmarterCSV.process(filename, options) do |chunk|
      MyModel.insert_all( chunk )
end
```

--------------------------------

### Troubleshoot CSV Parsing with hexdump

Source: https://github.com/tilo/smarter_csv/blob/main/docs/basic_read_api.md

Demonstrates using the `hexdump` command-line tool to inspect CSV files for hidden control characters or byte sequences, such as BOMs, which can affect parsing.

```bash
$ hexdump -C spec/fixtures/bom_test_feff.csv
```

--------------------------------

### Smarter CSV: Required Keys and Headers

Source: https://github.com/tilo/smarter_csv/blob/main/docs/options.md

Specifies required keys after header transformation. The `required_headers` option is deprecated and should be replaced with `required_keys`. If no keys are specified (nil), no validation is performed.

```ruby
smarter_csv.required_keys = [:key1, :key2]
# or
smarter_csv.required_headers = [:key1, :key2] # Deprecated

```

--------------------------------

### SmarterCSV 1.x Backwards Compatible Mode

Source: https://github.com/tilo/smarter_csv/wiki/Upgrading-from-SmarterCSV-1.x-to-2.x

This snippet illustrates how to configure SmarterCSV to maintain backward compatibility with version 1.x settings, specifically including the conversion of values to numeric types.

```ruby
{
     header_transformations: [:keys_as_symbols],
     header_validations:[:unique_headers],
     data_transformations: [ :replace_blank_with_nil ],
     data_validations: [],
     hash_transformations: [:strip_spaces, :remove_blank_values, :convert_values_to_numeric],
     hash_validations: []
  }
```

--------------------------------

### Process CSV with Unicode and BOM Encoding

Source: https://github.com/tilo/smarter_csv/blob/main/docs/basic_read_api.md

Shows how to open and process CSV files containing Unicode characters, specifically handling files with a Byte Order Mark (BOM) using Ruby's File.open with encoding.

```ruby
File.open(filename, "r:bom|utf-8") do |f|
  data = SmarterCSV.process(f);
end
```

--------------------------------

### Smarter CSV: Key Mapping and Header Manipulation

Source: https://github.com/tilo/smarter_csv/blob/main/docs/options.md

Options for managing keys and headers during CSV processing. `remove_unmapped_keys` removes columns not present in `key_mapping`, `downcase_header` converts headers to lowercase, and `keep_original_headers` preserves original headers.

```ruby
smarter_csv.remove_unmapped_keys = true
smarter_csv.downcase_header = true
smarter_csv.keep_original_headers = true

```

--------------------------------

### Smarter CSV: CSV Writing Configuration Options

Source: https://github.com/tilo/smarter_csv/blob/main/docs/options.md

This snippet outlines the configuration options for CSV writing in Smarter CSV. It details parameters such as row separator (:row_sep), column separator (:col_sep), quote character (:quote_char), and options for forcing quotes (:force_quotes). It also covers header management with :headers and :map_headers, value converters (:value_converters), header converters (:header_converter), automatic header discovery (:discover_headers), and quote disabling (:disable_auto_quoting).

```Python
csv_writer = SmarterCSVWriter(
    row_sep='\n',          # Separates rows
    col_sep=',',           # Separates each value in a row
    quote_char='"',        # To quote CSV fields
    force_quotes=False,    # Forces each individual value to be quoted
    headers=[],            # Provide specific keys for headers, disables automatic detection
    map_headers={},        # Maps keys to user-specified header values, disables automatic detection
    value_converters=None, # Define lambdas to programmatically modify values for specific keys or all fields
    header_converter=None, # Define a lambda to programmatically modify headers
    discover_headers=True, # Automatically detects all keys before writing the header
    disable_auto_quoting=False, # Manually disable auto-quoting of special characters
    quote_headers=False    # Force quoting all headers
)
```

--------------------------------

### Configure CSV Reading Options

Source: https://github.com/tilo/smarter_csv/blob/main/docs/options.md

This snippet outlines the various options available for configuring how Smarter CSV reads CSV files. These options control aspects like chunk size, file encoding, column and row separators, header presence, and duplicate header handling.

```ruby
SmarterCSV.process('file.csv', {
  chunk_size: 1000,
  file_encoding: 'UTF-8',
  invalid_byte_sequence: '',
  force_utf8: false,
  skip_lines: 0,
  comment_regexp: /\A#/, 
  col_sep: ',',
  row_sep: :auto,
  auto_row_sep_chars: 500,
  quote_char: '"',
  headers_in_file: true,
  duplicate_header_suffix: '_',
  user_provided_headers: nil,
  remove_empty_hashes: true,
  verbose: false,
  with_line_numbers: false,
  missing_header_prefix: 'column_',
  strict: false,
  key_mapping: nil
})
```

--------------------------------

### Custom Hash Transformations with Procs

Source: https://github.com/tilo/smarter_csv/wiki/Hash-Transformations

Allows users to define their own custom hash transformations using Ruby Procs for more specific data manipulation needs.

```Ruby
SmarterCSV.process(file, {
  :hash_transformations => [
    Proc.new { |header, row| row[:column_name] = row[:column_name].upcase },
    :strip_spaces
  ]
})
```

--------------------------------

### Smarter CSV: Whitespace and Character Stripping

Source: https://github.com/tilo/smarter_csv/blob/main/docs/options.md

Configures whitespace handling and character removal from headers. `strip_whitespace` removes leading/trailing whitespace from values and headers. `strip_chars_from_headers` uses a RegExp to clean header characters.

```ruby
smarter_csv.strip_whitespace = true
smarter_csv.strip_chars_from_headers = /[^a-zA-Z0-9_]/ # Example: remove non-alphanumeric characters except underscore

```

--------------------------------

### Process CSV with SmarterCSV (Full Interface)

Source: https://github.com/tilo/smarter_csv/blob/main/docs/basic_read_api.md

Utilizes the full interface of SmarterCSV for detailed control and access to internal states. This allows for more in-depth processing and inspection of CSV data.

```ruby
reader = SmarterCSV::Reader.new(file_or_input, options)
data = reader.process

puts reader.raw_headers
```

```ruby
reader = SmarterCSV::Reader.new(file_or_input, options)
data = reader.process do 
   # do something here
end

puts reader.raw_headers
```

--------------------------------

### Read Vanilla CSV File

Source: https://github.com/tilo/smarter_csv/wiki/The-Basics

Demonstrates reading a standard comma-separated CSV file. The gem automatically cleans up extra spaces around fields and converts header names to Ruby symbols, handling potential data inconsistencies.

```ruby
$ cat  /tmp/test.csv
"CATEGORY " ," FIRST  NAME" , " AGE "
 "Red","John" , " 34 "
```

```ruby
data = SmarterCSV.process( filename )
 => [{:category=>"Red", :first_name=>"John", :age=>"34"}]
```

--------------------------------

### Ruby: Define Custom Hash Validations with Procs

Source: https://github.com/tilo/smarter_csv/wiki/Hash-Validations

Demonstrates how to create custom hash validation logic using Ruby Procs for flexible data validation within Smarter CSV.

```Ruby
# Example of defining a custom validation using a Proc
custom_validation = Proc.new do |csv_row, options|
  # Custom validation logic here
  # Return true if valid, false otherwise
end

# Usage within Smarter CSV (conceptual)
# SmarterCSV.process(file, { :hash_validation => custom_validation })

```

--------------------------------

### Ruby: Define and Use Custom Value Converters with Smarter CSV

Source: https://github.com/tilo/smarter_csv/blob/main/docs/value_converters.md

This snippet demonstrates how to define custom Ruby classes for value conversion (e.g., `DateConverter`, `DollarConverter`, `MoneyConverter`, `BooleanConverter`) and apply them to specific CSV columns using Smarter CSV's `value_converters` option. It shows the process of reading a CSV file with custom data types and accessing the converted values.

```ruby
$ cat spec/fixtures/with_dates.csv
first,last,date,price,member
Ben,Miller,10/30/1998,$44.50,TRUE
Tom,Turner,2/1/2011,$15.99,False
Ken,Smith,01/09/2013,$199.99,true

$ irb
> require 'smarter_csv'
> require 'date'

# define a custom converter class, which implements self.convert(value)
class DateConverter
  def self.convert(value)
    Date.strptime( value, '%m/%d/%Y') # parses custom date format into Date instance
  end
end

class DollarConverter
  def self.convert(value)
    value.sub('$','').to_f # strips the dollar sign and creates a Float value
  end
end

require 'money'
class MoneyConverter
  def self.convert(value)
    # depending on locale you might want to also remove the indicator for thousands, e.g. comma 
    Money.from_amount(value.gsub(/[s\$]/,'').to_f) # creates a Money instance (based on cents)
  end
end

class BooleanConverter
  def self.convert(value)
    case value
    when /true/i
      true
    when /false/i
      false
    else
      nil
    end
  end
end

options = {value_converters: {date: DateConverter, price: DollarConverter, member: BooleanConverter}}
data = SmarterCSV.process("spec/fixtures/with_dates.csv", options)
first_record = data.first
first_record[:date]
  => #<Date: 1998-10-30 ((2451117j,0s,0n),+0s,2299161j)>
first_record[:date].class
  => Date
first_record[:price]
  => 44.50
first_record[:price].class
  => Float
first_record[:member]
  => true

```

--------------------------------

### Smarter CSV: Value Conversion and Filtering

Source: https://github.com/tilo/smarter_csv/blob/main/docs/options.md

Options for converting and filtering CSV values. `value_converters` allows custom class-based conversions. `remove_empty_values` and `remove_zero_values` filter out empty or zero numeric values respectively. `remove_values_matching` removes values matching a given regular expression.

```ruby
smarter_csv.value_converters = {:header => MyConverterClass}
smarter_csv.remove_empty_values = true
smarter_csv.remove_zero_values = true
smarter_csv.remove_values_matching = /^#ERROR$/

```

--------------------------------

### Process CSV in Chunks with Key Mapping (Ruby)

Source: https://github.com/tilo/smarter_csv/blob/main/docs/batch_processing.md

Demonstrates processing a CSV file in chunks of a specified size, applying key mapping to transform column names. The output is an array of arrays, where each inner array represents a chunk of processed rows (hashes).

```ruby
SmarterCSV.process('/tmp/pets.csv', {:chunk_size => 2, :key_mapping => {:first_name => :first, :last_name => :last}})
```

--------------------------------

### Process CSV Chunks and Post-Process with a Block (Ruby)

Source: https://github.com/tilo/smarter_csv/blob/main/docs/batch_processing.md

Shows how to process a CSV file in chunks and pass each chunk to a block for custom operations, such as creating virtual attributes or modifying existing ones. The method returns the total number of chunks processed.

```ruby
SmarterCSV.process('/tmp/pets.csv', {:chunk_size => 2, :key_mapping => {:first_name => :first, :last_name => :last}}) do |chunk|
  chunk.each do |h|
    h[:full_name] = [h[:first],h[:last]].join(' ')
    h.delete(:first) ; h.delete(:last)
  end
  puts chunk.inspect
end
```

--------------------------------

### Process CSV with Raw Headers

Source: https://github.com/tilo/smarter_csv/wiki/The-Basics

Demonstrates how to retrieve the original, unprocessed header names from a CSV file, including any leading/trailing whitespace. This is achieved by disabling header transformations using `header_transformations: [:none]`.

```ruby
data = SmarterCSV.process('/tmp/test.csv', {header_transformations: [:none]})
 => [{"CATEGORY "=>"Red", " FIRST  NAME"=>"John", " AGE "=>"35"}]
```

--------------------------------

### Validate Required Keys in CSV Import

Source: https://github.com/tilo/smarter_csv/blob/main/docs/header_validations.md

Demonstrates how to use the `required_keys` option with SmarterCSV to ensure that specific keys are present in every row of a CSV file. If any row is missing these keys, a `SmarterCSV::MissingKeys` exception will be raised.

```ruby
options = {
  required_keys: [:source_account, :destination_account, :amount]
}
data = SmarterCSV.process("/tmp/transactions.csv", options)

=> this will raise SmarterCSV::MissingKeys if any row does not contain these three keys
```

--------------------------------

### Process CSV with SmarterCSV (Simplified)

Source: https://github.com/tilo/smarter_csv/blob/main/docs/basic_read_api.md

Reads CSV files using the simplified interface of SmarterCSV. It automatically detects separators and can process rows individually or in chunks.

```ruby
array_of_hashes = SmarterCSV.process(file_or_input, options, &block)
```

```ruby
SmarterCSV.process(file_or_input, options, &block) do |hash|
   # process one row of CSV
end
```

```ruby
SmarterCSV.process(file_or_input, {chunk_size: 100}, &block) do |array_of_hashes|
   # process one chunk of up to 100 rows of CSV data
end
```

--------------------------------

### Process Remote CSV with Unicode Encoding

Source: https://github.com/tilo/smarter_csv/blob/main/docs/basic_read_api.md

Illustrates processing a remote CSV file with Unicode characters by specifying the UTF-8 encoding in the `open-uri` call.

```ruby
require 'open-uri'
file_location = 'http://your.remote.org/sample.csv'
open(file_location, 'r:utf-8') do |f|   # don't forget to specify the UTF-8 encoding!!
  data = SmarterCSV.process(f)
end
```