TestProf

https://test-prof.evilmartians.io
Admin
TestProf is a profiling and optimization toolbox for Ruby and Rails test suites that helps identify...
Tokens:21,107
Snippets:310
Trust Score:7.8
Update:1 month ago
Show doc for...
Tokens:
Raw
### Standard RSpec before(:each) setup

Source: https://test-prof.evilmartians.io/guide/recipes/before_all

Example of standard RSpec setup that runs before every individual test case.

```ruby
describe BeatleWeightedSearchQuery do
  before(:each) do
    @paul = create(:beatle, name: "Paul")
    @ringo = create(:beatle, name: "Ringo")
    @george = create(:beatle, name: "George")
    @john = create(:beatle, name: "John")
  end

  # and about 15 examples here
end
```

--------------------------------

### RSpec before(:all) setup

Source: https://test-prof.evilmartians.io/guide/recipes/before_all

Example of using RSpec's native before(:all) hook.

```ruby
describe BeatleWeightedSearchQuery do
  before(:all) do
    @paul = create(:beatle, name: "Paul")
    # ...
  end

  # ...
end
```

--------------------------------

### TestProf before_all setup

Source: https://test-prof.evilmartians.io/guide/recipes/before_all

Example of using the TestProf before_all hook to wrap an example group in a transaction.

```ruby
describe BeatleWeightedSearchQuery do
  before_all do
    @paul = create(:beatle, name: "Paul")
    # ...
  end

  # ...
end
```

--------------------------------

### Measure setup time with RSpecDissect

Source: https://test-prof.evilmartians.io/guide/playbook

Use RSpecDissect to measure time spent in let/before blocks compared to actual example execution time.

```sh
RD_PROF=1 bin/rspec
```

--------------------------------

### View MemoryProf output example

Source: https://test-prof.evilmartians.io/guide/profilers/memory_prof

Sample output showing memory usage metrics for test groups and individual examples.

```sh
[TEST PROF INFO] MemoryProf results

Final RSS: 673KB

Top 5 groups (by RSS):

AnswersController (./spec/controllers/answers_controller_spec.rb:3) – +80KB (13.50%)
QuestionsController (./spec/controllers/questions_controller_spec.rb:3) – +32KB  (9.08%)
CommentsController (./spec/controllers/comments_controller_spec.rb:3) – +16KB (3.27%)

Top 5 examples (by RSS):

destroys question (./spec/controllers/questions_controller_spec.rb:38) – +144KB (24.38%)
change comments count (./spec/controllers/comments_controller_spec.rb:7) – +120KB (20.00%)
change Votes count (./spec/shared_examples/controllers/voted_examples.rb:23) – +90KB (16.36%)
change Votes count (./spec/shared_examples/controllers/voted_examples.rb:23) – +64KB (12.86%)
fails (./spec/shared_examples/controllers/invalid_examples.rb:3) – +32KB (5.00%)
```

--------------------------------

### Run random test examples

Source: https://test-prof.evilmartians.io/guide/recipes/tests_sampling

Use the SAMPLE environment variable to specify the number of examples to execute.

```sh
SAMPLE=10 rspec
```

--------------------------------

### View RSpecDissect report output

Source: https://test-prof.evilmartians.io/guide/profilers/rspec_dissect

Example of the console output generated by RSpecDissect showing total setup time and top slowest suites.

```sh
[TEST PROF INFO] RSpecDissect report

Total time: 25:14.870
Total setup time: 14:36.482

Top 5 slowest suites setup time:

Webhooks::DispatchTransition (./spec/services/webhooks/dispatch_transition_spec.rb:3) – 00:29.895 of 00:33.706 / 327 (before: 00:25.346, before let: 00:19.389, lazy let: 00:04.543)
  ↳ user – 00:09.323 (327)
  ↳ funnel – 00:06.231 (122)
  ↳ account – 00:04.360 (91)

FunnelsController (./spec/controllers/funnels_controller_spec.rb:3) – 00:22.117 of 00:43.649 / 133 (before: 00:22.117, before let: 00:18.695, lazy let: 00:00.000)
  ↳ user – 00:09.323 (327)
  ↳ funnel – 00:06.231 (122)
...
```

--------------------------------

### View EventProf output example

Source: https://test-prof.evilmartians.io/guide/profilers/event_prof

Example of the console output generated by EventProf when tracking SQL queries.

```sh
[TEST PROF INFO] EventProf results for sql.active_record

Total time: 00:00.256 of 00:00.512 (50.00%)
Total events: 1031

Top 5 slowest suites (by time):

AnswersController (./spec/controllers/answers_controller_spec.rb:3) – 00:00.119 (549 / 20) of 00:00.200 (59.50%)
QuestionsController (./spec/controllers/questions_controller_spec.rb:3) – 00:00.105 (360 / 18) of 00:00.125 (84.00%)
CommentsController (./spec/controllers/comments_controller_spec.rb:3) – 00:00.032 (122 / 4) of 00:00.064 (50.00%)

Top 5 slowest tests (by time):

destroys question (./spec/controllers/questions_controller_spec.rb:38) – 00:00.022 (29) of 00:00.064 (34.38%)
change comments count (./spec/controllers/comments_controller_spec.rb:7) – 00:00.011 (34) of 00:00.022 (50.00%)
change Votes count (./spec/shared_examples/controllers/voted_examples.rb:23) – 00:00.008 (25) of 00:00.022 (36.36%)
change Votes count (./spec/shared_examples/controllers/voted_examples.rb:23) – 00:00.008 (32) of 00:00.035 (22.86%)
fails (./spec/shared_examples/controllers/invalid_examples.rb:3) – 00:00.007 (34) of 00:00.014 (50.00%)
```

--------------------------------

### Configure EventProf for per-example profiling

Source: https://test-prof.evilmartians.io/guide/profilers/event_prof

Enable profiling for individual examples by setting the per_example configuration option.

```ruby
TestProf::EventProf.configure do |config|
  config.per_example = true
end
```

--------------------------------

### View FactoryProf output example

Source: https://test-prof.evilmartians.io/guide/profilers/factory_prof

Displays the standard output format for FactoryProf, showing usage counts and timing metrics.

```sh
[TEST PROF INFO] Factories usage

Total: 15285
Total top-level: 10286
Total time: 04:31.222 (out of 07.16.124)
Total uniq factories: 119

   name           total   top-level   total time    time per call      top-level time

   user            6091        2715    115.7671s          0.0426s            50.2517s
   post            2142        2098     93.3152s          0.0444s            92.1915s
   ...
```

--------------------------------

### Registering a dump with before hooks

Source: https://test-prof.evilmartians.io/guide/recipes/any_fixture

Use the before hook to perform setup tasks like tenant creation before fixture execution.

```ruby
TestProf::AnyFixture.register_dump(
  "account",
  before: proc do
    begin
      Apartment::Tenant.create("test")
    rescue
      nil
    end
    Apartment::Tenant.create("test")
  end
) do
  # ...
end
```

--------------------------------

### Minitest before_all usage

Source: https://test-prof.evilmartians.io/guide/recipes/before_all

Example of using before_all within a Minitest class.

```ruby
require "test_prof/recipes/minitest/before_all"

class MyBeatlesTest < Minitest::Test
  include TestProf::BeforeAll::Minitest

  before_all do
    @paul = create(:beatle, name: "Paul")
    @ringo = create(:beatle, name: "Ringo")
    @george = create(:beatle, name: "George")
    @john = create(:beatle, name: "John")
  end

  # define tests which could access the object defined within `before_all`
end
```

--------------------------------

### Install Profiling Gems

Source: https://test-prof.evilmartians.io/guide/playbook

Add StackProf or Vernier to your project using Bundler.

```sh
bundle add stackprof
# or
bundle add vernier
```

--------------------------------

### Configure RSpec for FactoryAllStub

Source: https://test-prof.evilmartians.io/guide/recipes/factory_all_stub

Setup instructions for RSpec to automatically initialize and provide the factory:stub context.

```ruby
require "test_prof/recipes/rspec/factory_all_stub"
```

```ruby
describe "User" do
  let(:user) { create(:user) }

  it "is valid", factory: :stub do
    # use `build_stubbed` instead of `create`
    expect(user).to be_valid
  end
end
```

--------------------------------

### Standard let! usage

Source: https://test-prof.evilmartians.io/guide/recipes/let_it_be

Example of creating test data using standard RSpec let! blocks, which re-creates data for every example.

```ruby
describe BeatleWeightedSearchQuery do
  let!(:paul) { create(:beatle, name: "Paul") }
  let!(:ringo) { create(:beatle, name: "Ringo") }
  let!(:george) { create(:beatle, name: "George") }
  let!(:john) { create(:beatle, name: "John") }

  specify { expect(subject.call("john")).to contain_exactly(john) }

  # and more examples here
end
```

--------------------------------

### Install TestProf gem

Source: https://test-prof.evilmartians.io/guide/getting_started

Add the gem to the test group in your Gemfile.

```ruby
group :test do
  gem "test-prof", "~> 1.0"
end
```

--------------------------------

### StackProf Report Output

Source: https://test-prof.evilmartians.io/guide/profilers/ruby_profilers

Example output showing the paths to generated StackProf reports after a test run.

```sh
...

[TEST PROF INFO] StackProf report generated: tmp/test_prof/stack-prof-report-wall-raw-total.dump
[TEST PROF INFO] StackProf JSON report generated: tmp/test_prof/stack-prof-report-wall-raw-total.json
```

--------------------------------

### before_all usage

Source: https://test-prof.evilmartians.io/guide/recipes/let_it_be

Example of using the before_all helper to share data across tests, requiring the use of instance variables.

```ruby
describe BeatleWeightedSearchQuery do
  before_all do
    @paul = create(:beatle, name: "Paul")
    # ...
  end

  specify { expect(subject.call("joh")).to contain_exactly(@john) }

  # ...
end
```

--------------------------------

### Mark slow examples with RSpecStamp

Source: https://test-prof.evilmartians.io/guide/profilers/event_prof

Automatically tag slow examples by providing the EVENT_PROF_STAMP environment variable.

```sh
EVENT_PROF="sql.active_record" EVENT_PROF_STAMP="slow:sql" rspec ...
```

--------------------------------

### AnyFixture usage report output

Source: https://test-prof.evilmartians.io/guide/recipes/any_fixture

Example output format for AnyFixture usage statistics.

```sh
[TEST PROF INFO] AnyFixture usage stats:

       key    build time  hit count    saved time

      user     00:00.004          4     00:00.017
      post     00:00.002          1     00:00.002

Total time spent: 00:00.006
Total time saved: 00:00.019
Total time wasted: 00:00.000
```

--------------------------------

### Profile individual RSpec examples

Source: https://test-prof.evilmartians.io/guide/profilers/ruby_profilers

Use the :rprof tag to profile specific RSpec examples.

```ruby
it "is doing heavy stuff", :rprof do
  # ...
end
```

--------------------------------

### Configure MemoryProf output count

Source: https://test-prof.evilmartians.io/guide/profilers/memory_prof

Adjust the number of displayed examples or groups using environment variables or CLI flags.

```sh
TEST_MEM_PROF='rss' TEST_MEM_PROF_COUNT=10 rspec ...
```

```sh
# Run a specific file using CLI option
ruby test/my_super_test.rb --mem-prof=rs --mem-prof-top-count=10
```

--------------------------------

### Define class for profiling

Source: https://test-prof.evilmartians.io/guide/profilers/event_prof

Example class containing a method to be profiled.

```ruby
class Work
  def do_smth(*args)
    # do something
  end
end
```

--------------------------------

### View report generation output

Source: https://test-prof.evilmartians.io/guide/profilers/ruby_profilers

Example of the console output indicating where the generated report is saved.

```sh
[TEST PROF INFO] RubyProf report generated: tmp/test_prof/ruby-prof-report-flat-wall-total.txt
```

--------------------------------

### Profile Individual RSpec Examples

Source: https://test-prof.evilmartians.io/guide/profilers/ruby_profilers

Use the :sprof metadata tag to profile specific RSpec examples.

```ruby
it "is doing heavy stuff", :sprof do
  # ...
end
```

--------------------------------

### Define FactoryBot factories

Source: https://test-prof.evilmartians.io/guide/recipes/factory_default

Example factory definitions for hierarchical models.

```ruby
factory :account do
end

factory :user do
  account
end

factory :project do
  account
  user
end

factory :task do
  account
  project
  user
end
```

--------------------------------

### Stamp slow examples automatically

Source: https://test-prof.evilmartians.io/guide/playbook

Marks the slowest test examples with a custom RSpec tag based on factory creation time.

```sh
EVENT_PROF=factory.create EVENT_PROF_STAMP=slow:factory bin/rspec spec/models
```

--------------------------------

### Example factory cascade structure

Source: https://test-prof.evilmartians.io/guide/profilers/factory_prof

A demonstration of how nested factory definitions result in recursive create calls and specific stack traces.

```ruby
factory :comment do
  answer
  author
end

factory :answer do
  question
  author
end

factory :question do
  author
end

create(:comment) #=> creates 5 records

# And the corresponding stack is:
# [:comment, :answer, :question, :author, :author, :author]
```

--------------------------------

### Define Associations with Traits

Source: https://test-prof.evilmartians.io/guide/recipes/factory_default

Example factory definitions using associations with specific traits.

```ruby
factory :comment do
  user
end

factory :post do
  association :user, factory: %i[user able_to_post]
end

factory :view do
  association :user, factory: %i[user unable_to_post_only_view]
end
```

--------------------------------

### Tag slow examples with RSpecStamp

Source: https://test-prof.evilmartians.io/guide/profilers/rspec_dissect

Automatically mark the slowest example groups with a custom tag using the RD_PROF_STAMP environment variable.

```sh
RD_PROF=1 RD_PROF_STAMP="slow" rspec ...
```

--------------------------------

### Run random test groups

Source: https://test-prof.evilmartians.io/guide/recipes/tests_sampling

Use the SAMPLE_GROUPS environment variable to execute a random set of example groups or suites.

```sh
SAMPLE_GROUPS=10 rspec
```

--------------------------------

### Enable Per-Example Logging in RSpec

Source: https://test-prof.evilmartians.io/guide/recipes/logging

Use the :log tag to enable verbose logging for specific examples or groups.

```ruby
# Add the tag and you will see a lot of interesting stuff in your console
it "does smthng weird", :log do
  # ...
end

# or for the group
describe "GET #index", :log do
  # ...
end
```

--------------------------------

### Define fixture with AnyFixture disabled

Source: https://test-prof.evilmartians.io/guide/recipes/any_fixture

Example of defining a fixture that executes when AnyFixture is disabled.

```ruby
# Here we create a new account if AnyFixture is disabled
let(:account) { fixture(:account) { create(:account) } }
```

--------------------------------

### Install ruby-prof gem

Source: https://test-prof.evilmartians.io/guide/profilers/ruby_profilers

Add the ruby-prof gem to your Gemfile within the development and test groups.

```ruby
# Gemfile
group :development, :test do
  gem "ruby-prof", ">= 1.4.0", require: false
end
```

--------------------------------

### Run FactoryDoctor with Auto-Stubbing

Source: https://test-prof.evilmartians.io/guide/recipes/factory_all_stub

Command to execute FactoryDoctor and automatically mark offensive examples with the factory:stub tag.

```sh
FDOC=1 FDOC_STAMP=factory:stub rspec ./spec/models
```

--------------------------------

### Register global callbacks with tags

Source: https://test-prof.evilmartians.io/guide/recipes/before_all

Define callbacks for RSpec example groups using specific tags to execute logic before transactions begin.

```ruby
TestProf::BeforeAll.configure do |config|
  config.before(:begin, reset_sequences: true, foo: :bar) do
    warn <<~MESSAGE
      Do NOT create objects outside of transaction
      because all db sequences will be reset to 1
      in every single example, so that IDs of new objects
      can get into conflict with the long-living ones.
    MESSAGE
  end
end
```

--------------------------------

### let_it_be usage

Source: https://test-prof.evilmartians.io/guide/recipes/let_it_be

Example of using let_it_be to share data across tests without needing instance variables or manual before_all blocks.

```ruby
describe BeatleWeightedSearchQuery do
  let_it_be(:paul) { create(:beatle, name: "Paul") }
  let_it_be(:ringo) { create(:beatle, name: "Ringo") }
  let_it_be(:george) { create(:beatle, name: "George") }
  let_it_be(:john) { create(:beatle, name: "John") }

  specify { expect(subject.call("john")).to contain_exactly(john) }

  # and more examples here
end
```

--------------------------------

### TagProf Report Output

Source: https://test-prof.evilmartians.io/guide/profilers/tag_prof

Example of the console output generated by TagProf when grouping by test type.

```sh
[TEST PROF INFO] TagProf report for type

       type          time   total  %total   %time           avg

    request     00:04.808      42   33.87   54.70     00:00.114
 controller     00:02.855      42   33.87   32.48     00:00.067
      model     00:01.127      40   32.26   12.82     00:00.028
```

--------------------------------

### Define and use let_it_be aliases

Source: https://test-prof.evilmartians.io/guide/recipes/let_it_be

Configure custom aliases for let_it_be in rails_helper.rb to simplify test setup with default options.

```ruby
# rails_helper.rb
TestProf::LetItBe.configure do |config|
  # define an alias with `refind: true` by default
  config.alias_to :let_it_be_with_refind, refind: true
end

# then use it in your tests
describe "smth" do
  let_it_be_with_refind(:foo) { Foo.create }

  # refind can still be overridden
  let_it_be_with_refind(:bar, refind: false) { Bar.create }
end
```

--------------------------------

### View Vernier report output

Source: https://test-prof.evilmartians.io/guide/profilers/ruby_profilers

Example of the console output indicating the location of the generated JSON report.

```sh
...

[TEST PROF INFO] Vernier report generated: tmp/test_prof/vernier-report-wall-raw-total.json
```

--------------------------------

### Minitest Unknown Grouping Output

Source: https://test-prof.evilmartians.io/guide/profilers/tag_prof

Example output when Minitest cannot determine a root test directory for grouping.

```sh
[TEST PROF INFO] TagProf report for type

       type          time   sql.active_record  total  %total   %time           avg

__unknown__     00:04.808           00:01.402     42   33.87   54.70     00:00.114
 controller     00:02.855           00:00.921     42   33.87   32.48     00:00.067
      model     00:01.127           00:00.446     40   32.26   12.82     00:00.028
```

--------------------------------

### Identify inefficient database usage in tests

Source: https://test-prof.evilmartians.io/guide/profilers/factory_doctor

Examples of tests that perform unnecessary database writes compared to optimized versions.

```ruby
# with FactoryBot
it "validates name presence" do
  user = create(:user)
  user.name = ""
  expect(user).not_to be_valid
end

# with Fabrication
it "validates name presence" do
  user = Fabricate(:user)
  user.name = ""
  expect(user).not_to be_valid
end
```

```ruby
# with FactoryBot
it "validates name presence" do
  user = build_stubbed(:user)
  user.name = ""
  expect(user).not_to be_valid
end

# with Fabrication
it "validates name presence" do
  user = Fabricate.build(:user)
  user.name = ""
  expect(user).not_to be_valid
end
```

--------------------------------

### Aggregate RSpec examples

Source: https://test-prof.evilmartians.io/guide/misc/rubocop

Refactor multiple assertions into a single example using the :aggregate_failures metadata.

```ruby
# bad
it { is_expected.to be_success }
it { is_expected.to have_header("X-TOTAL-PAGES", 10) }
it { is_expected.to have_header("X-NEXT-PAGE", 2) }
its(:status) { is_expected.to eq(200) }

# good
it "returns the second page", :aggregate_failures do
  is_expected.to be_success
  is_expected.to have_header("X-TOTAL-PAGES", 10)
  is_expected.to have_header("X-NEXT-PAGE", 2)
  expect(subject.status).to eq(200)
end
```

--------------------------------

### Profile individual RSpec examples

Source: https://test-prof.evilmartians.io/guide/profilers/ruby_profilers

Use the :vernier metadata tag to profile specific RSpec examples.

```ruby
it "is doing heavy stuff", :vernier do
  # ...
end
```

--------------------------------

### Implement a custom database adapter for before_all

Source: https://test-prof.evilmartians.io/guide/recipes/before_all

Define a class with begin_transaction and rollback_transaction methods, then assign an instance to TestProf::BeforeAll.adapter.

```ruby
class MyDBAdapter
  # before_all adapters must implement two methods:
  # - begin_transaction
  # - rollback_transaction
  def begin_transaction
    # ...
  end

  def rollback_transaction
    # ...
  end
end

# And then set adapter for `BeforeAll` module
TestProf::BeforeAll.adapter = MyDBAdapter.new
```

--------------------------------

### Initialize and Control FactoryAllStub

Source: https://test-prof.evilmartians.io/guide/recipes/factory_all_stub

Methods to initialize, enable, and disable the all-stub mode manually.

```ruby
TestProf::FactoryAllStub.init
```

```ruby
TestProf::FactoryAllStub.enable!
```

```ruby
TestProf::FactoryAllStub.disable!
```

--------------------------------

### Monitor with options

Source: https://test-prof.evilmartians.io/guide/profilers/event_prof

Configure a monitor with top-level tracking and a guard proc.

```ruby
TestProf::EventProf.monitor(
  Sidekiq::Client,
  "sidekiq.inline",
  :raw_push,
  top_level: true,
  guard: ->(*) { Sidekiq::Testing.inline? }
)
```

--------------------------------

### Profile Application Boot with StackProf

Source: https://test-prof.evilmartians.io/guide/profilers/ruby_profilers

Profile the application boot process by setting TEST_STACK_PROF to 'boot'.

```sh
# pick some random spec (1 is enough)
$ TEST_STACK_PROF=boot bundle exec rspec ./spec/some_spec.rb

...
[TEST PROF INFO] StackProf report generated: tmp/test_prof/stack-prof-report-wall-raw-boot.dump
[TEST PROF INFO] StackProf JSON report generated: tmp/test_prof/stack-prof-report-wall-raw-boot.json
```

--------------------------------

### Profile Application Boot

Source: https://test-prof.evilmartians.io/guide/playbook

Run a specific test file with the boot profiling flag enabled.

```sh
TEST_STACK_PROF=boot rspec ./spec/some_spec.rb
```

--------------------------------

### Define Fabrication fabricators

Source: https://test-prof.evilmartians.io/guide/recipes/factory_default

Example fabricator definitions for hierarchical models.

```ruby
Fabricator(:account) do
end

Fabricator(:user) do
  account
end

# etc.
```

--------------------------------

### Register lifecycle hooks for before_all

Source: https://test-prof.evilmartians.io/guide/recipes/before_all

Use the configure block to define callbacks that execute around transaction opening and rollback events.

```ruby
TestProf::BeforeAll.configure do |config|
  config.before(:begin) do
    # do something before transaction opens
  end
  # after(:begin) is also available

  config.after(:rollback) do
    # do something after transaction closes
  end
  # before(:rollback) is also available
end
```

--------------------------------

### Profile application boot

Source: https://test-prof.evilmartians.io/guide/profilers/ruby_profilers

Profile the application boot process by setting the TEST_VERNIER variable to 'boot'.

```sh
# pick some random spec (1 is enough)
TEST_VERNIER=boot bundle exec rspec ./spec/some_spec.rb
```

--------------------------------

### Configuring AnyFixture settings

Source: https://test-prof.evilmartians.io/guide/recipes/any_fixture

Adjust global settings such as dump storage directory and CLI import behavior.

```ruby
TestProf::AnyFixture.configure do |config|
  # Where to store dumps (by default, TestProf.artifact_path + '/any_dumps')
  config.dumps_dir = "any_dumps"
  # Include mathing queries into a dump (in addition to INSERT/UPDATE/DELETE queries)
  config.dump_matching_queries = /^$/
  # Whether to try using CLI tools such as psql or sqlite3 to restore dumps or not (and use ActiveRecord instead)
  config.import_dump_via_cli = false
end
```

--------------------------------

### Run custom event profile

Source: https://test-prof.evilmartians.io/guide/profilers/event_prof

Execute the test suite with the custom event enabled.

```sh
EVENT_PROF=my.work bundle exec rake test
```

--------------------------------

### Stamp bad examples with RSpecStamp

Source: https://test-prof.evilmartians.io/guide/profilers/factory_doctor

Automatically tag inefficient tests using FDOC_STAMP.

```sh
FDOC=1 FDOC_STAMP="fdoc:consider" rspec ...
```

--------------------------------

### Configure Usage Statistics

Source: https://test-prof.evilmartians.io/guide/recipes/factory_default

Enable reporting for FactoryDefault usage statistics.

```ruby
TestProf::FactoryDefault.configure do |config|
  config.report_summary = true
  # Report stats prints the detailed usage information (including summary)
  config.report_stats = true
end
```

```sh
$ FACTORY_DEFAULT_SUMMARY=1 bundle exec rspec

FactoryDefault summary: hit=11 miss=3
```

--------------------------------

### Run FactoryBot instrumentation

Source: https://test-prof.evilmartians.io/guide/profilers/event_prof

Use this command to profile top-level FactoryBot.create calls.

```sh
EVENT_PROF=factory.create bundle exec rspec
```

--------------------------------

### Run RuboCop with TestProf plugin dynamically

Source: https://test-prof.evilmartians.io/guide/misc/rubocop

Execute RuboCop with the plugin enabled via the command line.

```sh
bundle exec rubocop --plugin 'test-prof' --only RSpec/AggregateExamples
```

--------------------------------

### Define let_it_be in tests

Source: https://test-prof.evilmartians.io/guide/recipes/let_it_be

Use let_it_be to define records that persist across examples within a group.

```ruby
describe MySuperDryService do
  let_it_be(:user) { create(:user) }

  # ...
end
```

--------------------------------

### Analyze database and factory performance

Source: https://test-prof.evilmartians.io/guide/playbook

Commands to isolate time spent in database queries versus factory creation.

```sh
# Database interactions
EVENT_PROF=sql.active_record bin/rspec spec/models

# Factories
EVENT_PROF=factory.create bin/rspec spec/models
```

--------------------------------

### Standard test case

Source: https://test-prof.evilmartians.io/guide/recipes/factory_default

A test case demonstrating the factory cascade issue where multiple accounts and users are created.

```ruby
describe "PATCH #update" do
  let(:task) { create(:task) }

  it "works" do
    patch :update, id: task.id, task: {completed: "t"}
    expect(response).to be_success
  end

  # ...
end
```

--------------------------------

### Profile dependency usage with EventProf

Source: https://test-prof.evilmartians.io/guide/playbook

Commands to identify overhead from specific gems like Sidekiq and Wisper within a test group.

```sh
EVENT_PROF=sidekiq.inline bin/rspec spec/models
```

```sh
EVENT_PROF=wisper.publisher.broadcast bin/rspec spec/models
```

--------------------------------

### Configure TestProf settings

Source: https://test-prof.evilmartians.io/guide/getting_started

Customize global settings such as output directory, timestamps, and logging.

```ruby
TestProf.configure do |config|
  # the directory to put artifacts (reports) in ('tmp/test_prof' by default)
  config.output_dir = "tmp/test_prof"

  # use unique filenames for reports (by simply appending current timestamp)
  config.timestamps = true

  # color output
  config.color = true

  # where to write logs (defaults)
  config.output = $stdout

  # alternatively, you can specify a custom logger instance
  config.logger = MyLogger.new
end
```

--------------------------------

### Ignore Groups in TPSProf

Source: https://test-prof.evilmartians.io/guide/profilers/tps_prof

Exclude specific RSpec example groups from tracking by applying the tps_prof: :ignore metadata.

```ruby
RSpec.describe "SlowButOk", tps_prof: :ignore do
  # ...
end
```

--------------------------------

### Activate FactoryProf via Environment Variables

Source: https://test-prof.evilmartians.io/guide/profilers/factory_prof

Enable the profiler by setting the FPROF environment variable before running test commands.

```bash
FPROF=1 rspec

# or
FPROF=1 bundle exec rake test
```

--------------------------------

### Run Test Suite with StackProf

Source: https://test-prof.evilmartians.io/guide/profilers/ruby_profilers

Activate StackProf profiling by setting the TEST_STACK_PROF environment variable before running tests.

```sh
TEST_STACK_PROF=1 bundle exec rake test

# or for RSpec
TEST_STACK_PROF=1 bundle exec rspec ...
```

--------------------------------

### Loading connection classes

Source: https://test-prof.evilmartians.io/guide/recipes/before_all

Ensuring connection classes are loaded before running tests.

```ruby

# Ensure connection classes are loaded
ApplicationRecord
AccountsRecord
```

--------------------------------

### Activate MemoryProf for Minitest

Source: https://test-prof.evilmartians.io/guide/profilers/memory_prof

Enable memory profiling in Minitest using environment variables or CLI options.

```sh
TEST_MEM_PROF='rss' rake test
TEST_MEM_PROF='alloc' rake test
TEST_MEM_PROF='gc' rake test
```

```sh
# Run a specific file using CLI option
ruby test/my_super_test.rb --mem-prof=rss

# Show the list of possible options:
ruby test/my_super_test.rb --help
```

--------------------------------

### Visualize factory cascades

Source: https://test-prof.evilmartians.io/guide/playbook

Generate a flamegraph to visualize factory cascades and identify specific factories requiring optimization.

```sh
FPROF=flamegraph bin/rspec --tag slow:factory
```

--------------------------------

### Verify Trait-based Default Usage

Source: https://test-prof.evilmartians.io/guide/recipes/factory_default

Demonstrates how defaults behave when traits are specified in associations.

```ruby
# Create a default with trait
user = create_default(:user_poster, :able_to_post)

# When an association has no traits specified, the default with trait is used
create(:comment).user == user #=> true
# When an association has the matching trait specified, the default is used, too
create(:post).user == user #=> true
# When the association's trait differs, default is skipped
create(:view).user == user #=> false
```

--------------------------------

### Use Logging Helpers

Source: https://test-prof.evilmartians.io/guide/recipes/logging

Use with_logging or with_ar_logging to enable logging for specific code blocks.

```ruby
it "does somthing" do
  do_smth
  # show logs only for the code within the block
  with_logging do
    # ...
  end
end
```

--------------------------------

### AnyFixture execution flow

Source: https://test-prof.evilmartians.io/guide/recipes/any_fixture

The lifecycle of SQL dump generation and restoration during test execution.

```sh
# first run
$ bundle exec rspec

# AnyFixture.register_dump is called:
# - is SQL dump present? No
# - run block and write all modifying queries to a new SQL dump
# AnyFixture.clean is called:
# - clean all the affected tables

# second run
$ bundle exec rspec

# AnyFixture.register_dump is called:
# - is SQL dump present? Yes
# - restore dump (do not run block)
# AnyFixture.clean is called:
# - clean all the affected tables
```

--------------------------------

### Use built-in reload and refind modifiers

Source: https://test-prof.evilmartians.io/guide/recipes/let_it_be

Apply reload or refind modifiers to let_it_be to ensure objects are in a pristine state for each example.

```ruby
# Use reload: true option to reload user object (assuming it's an instance of ActiveRecord)
# for every example
let_it_be(:user, reload: true) { create(:user) }

# it is almost equal to
before_all { @user = create(:user) }
let(:user) { @user.reload }

# You can also specify refind: true option to hard-reload the record
let_it_be(:user, refind: true) { create(:user) }

# it is almost equal to
before_all { @user = create(:user) }
let(:user) { User.find(@user.id) }
```

--------------------------------

### Configure FactoryDefault in spec_helper.rb

Source: https://test-prof.evilmartians.io/guide/recipes/factory_default

Load the FactoryDefault recipe in your RSpec configuration.

```ruby
require "test_prof/recipes/rspec/factory_default"
```

--------------------------------

### Multiple database connection classes

Source: https://test-prof.evilmartians.io/guide/recipes/before_all

Defining connection classes for multiple databases.

```ruby
class Users < AccountsRecord
  # ...
end

class Articles < ApplicationRecord
  # ...
end
```

--------------------------------

### Demonstrate let_it_be override behavior

Source: https://test-prof.evilmartians.io/guide/recipes/let_it_be

Shows how let_it_be records persist even when overridden in nested contexts, unlike let!.

```ruby
context "A" do
  let!(:user) { create(:user, name: "a") }
  let_it_be(:post) { create(:post, title: "A") }

  specify { expect(User.all.pluck(:name)).to eq ["a"] }
  specify { expect(Post.all.pluck(:title)).to eq ["A"] }

  context "B" do
    let!(:user) { create(:user, name: "b") }
    let_it_be(:post) { create(:post, title: "B") }

    specify { expect(User.all.pluck(:name)).to eq ["b"] }
    specify { expect(Post.all.pluck(:title)).to eq ["B"] } # fails, because there are two posts
  end
end
```

--------------------------------

### RSpec configuration

Source: https://test-prof.evilmartians.io/guide/recipes/before_all

Requiring the before_all recipe in your RSpec helper.

```ruby
require "test_prof/recipes/rspec/before_all"
```

--------------------------------

### Configuring global before and after hooks

Source: https://test-prof.evilmartians.io/guide/recipes/any_fixture

Define global hooks to execute logic during dump creation or restoration processes.

```ruby
TestProf::AnyFixture.configure do |config|
  config.before_dump do |dump:, import:|
    # dump is an object containing information about the dump (e.g., dump.digest)
    # import is true if we're restoring a dump and false otherwise
    # do something
  end

  config.after_dump do |dump:, import:|
    # ...
  end
end
```

--------------------------------

### Enable Rails fixtures in before_all

Source: https://test-prof.evilmartians.io/guide/recipes/before_all

Configure Rails fixtures for specific hooks or globally to allow fixture usage within before_all blocks.

```ruby
before_all(setup_fixtures: true) do
  @user = users(:john)
  @post = create(:post, user: user)
end
```

```ruby
TestProf::BeforeAll.configure do |config|
  config.setup_fixtures = true
end
```

--------------------------------

### Load Isolator patch for before_all

Source: https://test-prof.evilmartians.io/guide/recipes/before_all

Explicitly load the Isolator patch after loading before_all or let_it_be to ensure compatibility.

```ruby
# after loading before_all or/and let_it_be
require "test_prof/before_all/isolator"
```

--------------------------------

### Implement a custom instrumentation wrapper

Source: https://test-prof.evilmartians.io/guide/profilers/event_prof

Define a wrapper module with a subscribe method to integrate custom instrumentation engines.

```ruby
# Wrapper over your instrumentation
module MyEventsWrapper
  # Should contain the only one method
  def self.subscribe(event)
    raise ArgumentError, "Block is required!" unless block_given?

    ::MyEvents.subscribe(event) do |start, finish, *|
      yield (finish - start)
    end
  end
end
```

--------------------------------

### Configuring default watch paths

Source: https://test-prof.evilmartians.io/guide/recipes/any_fixture

Add additional files or directories to the watch list to trigger dump invalidation when they change.

```ruby
TestProf::AnyFixture.configure do |config|
  # you can use exact file paths or globs
  config.default_dump_watch_paths << Rails.root.join("spec/factories/**/*")
end
```

--------------------------------

### Measure factory default summary

Source: https://test-prof.evilmartians.io/guide/playbook

Measure the impact of adding create_default to your factories.

```sh
FACTORY_DEFAULT_SUMMARY=1 bin/rspec --tag slow:factory

# More hits — better
FactoryDefault summary: hit=11 miss=3
```

--------------------------------

### Using FactoryDefault

Source: https://test-prof.evilmartians.io/guide/recipes/factory_default

Using create_default to automatically reuse parent records in tests.

```ruby
describe "PATCH #update" do
  let(:account) { create_default(:account) }
  let(:project) { create_default(:project) }
  let(:task) { create(:task) }

  # and if we need more projects, users, tasks with the same parent record,
  # we just write
  let(:another_project) { create(:project) } # uses the same account
  let(:another_task) { create(:task) } # uses the same account

  it "works" do
    patch :update, id: task.id, task: {completed: "t"}
    expect(response).to be_success
  end
end
```

--------------------------------

### Require Logging Recipe

Source: https://test-prof.evilmartians.io/guide/recipes/logging

Include this line in your test helper file to enable the logging functionality.

```ruby
require "test_prof/recipes/logging"
```

--------------------------------

### Activate EventProf in Minitest

Source: https://test-prof.evilmartians.io/guide/profilers/event_prof

Configure EventProf for Minitest using environment variables or CLI flags.

```sh
# Collect SQL queries stats for every suite and example
EVENT_PROF='sql.active_record' rake test
```

```sh
# Run a specific file using CLI option
ruby test/my_super_test.rb --event-prof=sql.active_record

# Show the list of possible options:
ruby test/my_super_test.rb --help
```

--------------------------------

### Configure FactoryDoctor settings

Source: https://test-prof.evilmartians.io/guide/profilers/factory_doctor

Customize event tracking and time thresholds for database operations.

```ruby
TestProf::FactoryDoctor.configure do |config|
  # Which event to track within test example to consider them "DB-dirty"
  config.event = "sql.active_record"
  # Consider result "good" if the time in DB is less then the threshold
  config.threshold = 0.01
end
```

--------------------------------

### Configure AnyFixture for Minitest

Source: https://test-prof.evilmartians.io/guide/recipes/any_fixture

Ensure the database is cleaned after test runs by registering a cleanup hook in your test helper.

```ruby
# test_helper.rb

require "test_prof/any_fixture"

at_exit { TestProf::AnyFixture.clean }
```

--------------------------------

### Enable Factory Default Profiling

Source: https://test-prof.evilmartians.io/guide/recipes/factory_default

Run tests with the FACTORY_DEFAULT_PROF environment variable set to 1 to output association usage statistics.

```sh
$ FACTORY_DEFAULT_PROF=1 bundle exec rspec spec/some/file_spec.rb

.....

[TEST PROF INFO] Factory associations usage:

               factory      count    total time

                  user         17     00:42.010
         user[traited]         15     00:31.560
  user{tag:"some tag"}          1     00:00.205

Total associations created: 33
Total uniq associations created: 3
Total time spent: 01:13.775
```

--------------------------------

### Run Sampled Tests

Source: https://test-prof.evilmartians.io/guide/playbook

Execute a subset of tests multiple times to identify application-wide performance issues.

```sh
SAMPLE=100 bin/rails test
# or
SAMPLE=100 bin/rspec
```

--------------------------------

### Registering an AnyFixture SQL dump

Source: https://test-prof.evilmartians.io/guide/recipes/any_fixture

Use register_dump within a before(:all) block to track and persist database state across test runs.

```ruby
RSpec.shared_context "account", account: true do
  # You should call AnyFixture outside of transaction to re-use the same
  # data between examples
  before(:all) do
    # The block is called once per test run (similary to #register)
    TestProf::AnyFixture.register_dump("account") do
      # Do anything here, AnyFixture keeps track of affected DB tables
      # For example, you can use factories here
      account = FactoryBot.create(:account, name: "test")

      # or with Fabrication
      account = Fabricate(:account, name: "test")

      # or with plain old AR
      account = Account.create!(name: "test")

      # updates are also tracked
      account.update!(tag: "sql-dump")
    end
  end

 # Here, we MUST use a custom way to retrieve a record: since we restore the data
 # from a plain SQL dump, we have no knowledge of Ruby objects
  let(:account) { Account.find_by!(name: "test") }
end
```

--------------------------------

### Track Events with TagProf

Source: https://test-prof.evilmartians.io/guide/profilers/tag_prof

Combine TagProf with EventProf to track specific activity durations.

```sh
TAG_PROF=type TAG_PROF_EVENT=sql.active_record rspec
```

```sh
[TEST PROF INFO] TagProf report for type

       type          time   sql.active_record  total  %total   %time           avg

    request     00:04.808           00:01.402     42   33.87   54.70     00:00.114
 controller     00:02.855           00:00.921     42   33.87   32.48     00:00.067
      model     00:01.127           00:00.446     40   32.26   12.82     00:00.028
```

--------------------------------

### Require let_it_be in RSpec helper

Source: https://test-prof.evilmartians.io/guide/recipes/let_it_be

Include the recipe in your test configuration file.

```ruby
require "test_prof/recipes/rspec/let_it_be"
```

--------------------------------

### Register monitors on demand

Source: https://test-prof.evilmartians.io/guide/profilers/event_prof

Register and activate custom events only when explicitly requested.

```ruby
TestProf::EventProf::CustomEvents.register("my.work") do
  TestProf::EventProf.monitor(Work, "my.work", :do_smth)
end

# Then call `activate_all` with the provided event
TestProf::EventProf::CustomEvents.activate_all(TestProf::EventProf.config.event)
```

--------------------------------

### Enable TestProf RuboCop plugin

Source: https://test-prof.evilmartians.io/guide/misc/rubocop

Add the test-prof plugin to your .rubocop.yml configuration file.

```yml
# .rubocop.yml
plugins:
 - 'test-prof'
```

--------------------------------

### Set and Retrieve Factory Defaults

Source: https://test-prof.evilmartians.io/guide/recipes/factory_default

Use set_factory_default to define objects for associations and get_factory_default to retrieve them.

```ruby
let(:user) { create(:user) }

before { FactoryBot.set_factory_default(:user, user) }

# You can also set the default factory with traits
FactoryBot.set_factory_default([:user, :admin], admin)

# Or (since v1.4)
FactoryBot.set_factory_default(:user, :admin, admin)

# You can also register a default record for specific attribute overrides
Fabricate.set_fabricate_default(:post, post, state: "draft")
```

```ruby
# This method also supports traits
admin = FactoryBot.get_factory_default(:user, :admin)
```

--------------------------------

### Configure let_it_be duplicate reporting

Source: https://test-prof.evilmartians.io/guide/recipes/let_it_be

Configures how the system handles duplicate let_it_be definitions by either warning or raising an exception.

```ruby
TestProf::LetItBe.configure do |config|
  config.report_duplicates = :warn # Rspec.warn_with
  config.report_duplicates = :raise # Kernel.raise
end
```

--------------------------------

### Enable and Use AnyFixture DSL

Source: https://test-prof.evilmartians.io/guide/recipes/any_fixture

The DSL provides a 'fixture' method and lifecycle callbacks. It can be included in RSpec configuration or Minitest base classes.

```ruby
require "test_prof/any_fixture/dsl"

# Enable DSL in RSpec
RSpec.configure do |config|
  config.include TestProf::AnyFixture::DSL
end

# Minitest
class MyBaseTestCase < Minitest::Test
  include TestProf::AnyFixture::DSL
end

# and then you can use `fixture` method (which is just an alias for `TestProf::AnyFixture.register`)
before(:all) { fixture(:account) { create(:account) } }

# You can also use it to fetch the record (instead of storing it in instance variable)
let(:account) { fixture(:account) }

# You can just use `before_fixtures_reset` or `after_fixtures_reset` callbacks
before_fixtures_reset { Post.delete_all }
after_fixtures_reset { Post.delete_all }
```

--------------------------------

### Configure RubyProf printer

Source: https://test-prof.evilmartians.io/guide/profilers/ruby_profilers

Specify the output printer via environment variables or within the TestProf configuration block.

```sh
TEST_RUBY_PROF=call_stack bundle exec rake test
```

```ruby
TestProf::RubyProf.configure do |config|
  config.printer = :call_stack
end
```

--------------------------------

### Configure TestProf StackProf

Source: https://test-prof.evilmartians.io/guide/playbook

Set the default output format for StackProf profiles to JSON.

```ruby
TestProf::StackProf.configure do |config|
  config.format = "json"
end
```

--------------------------------

### Enable Global Logging via Environment Variables

Source: https://test-prof.evilmartians.io/guide/recipes/logging

Use the LOG environment variable to control logging output during test execution.

```sh
# log everything to stdout
LOG=all rspec ...

# or
LOG=all rake test

# log only Active Record statements
LOG=ar rspec ...
```

--------------------------------

### Run suite-wide Vernier profiling

Source: https://test-prof.evilmartians.io/guide/profilers/ruby_profilers

Activate profiling for the entire test suite using the TEST_VERNIER environment variable.

```sh
TEST_VERNIER=1 bundle exec rake test

# or for RSpec
TEST_VERNIER=1 bundle exec rspec ...
```

--------------------------------

### Run Sidekiq jobs instrumentation

Source: https://test-prof.evilmartians.io/guide/profilers/event_prof

Collect statistics for Sidekiq jobs run inline.

```sh
EVENT_PROF=sidekiq.jobs bundle exec rspec
```

--------------------------------

### Configure Variations Limit

Source: https://test-prof.evilmartians.io/guide/profilers/factory_prof

Limit the number of traits/overrides displayed to prevent long output lines.

```bash
FPROF=1 FPROF_VARIATIONS_LIMIT=5 rspec

# or
FPROF=1 FPROF_VARIATIONS_LIMIT=5 bundle exec rake test
```

```ruby
TestProf::FactoryProf.configure do |config|
  config.variations_limit = 5
end
```

--------------------------------

### Activate RSpecDissect

Source: https://test-prof.evilmartians.io/guide/profilers/rspec_dissect

Enable the profiler by setting the RD_PROF environment variable when running RSpec.

```sh
RD_PROF=1 rspec ...
```

--------------------------------

### Activate FactoryDoctor in Minitest

Source: https://test-prof.evilmartians.io/guide/profilers/factory_doctor

Enable FactoryDoctor via environment variable or CLI flag.

```sh
FDOC=1 ruby ...
```

```sh
ruby ... --factory-doctor
```

--------------------------------

### Configure AnyFixture for RSpec

Source: https://test-prof.evilmartians.io/guide/recipes/any_fixture

Include the AnyFixture recipe in your RSpec helper file to enable its functionality.

```ruby
require "test_prof/recipes/rspec/any_fixture"
```

--------------------------------

### Include Logging Helpers in Minitest

Source: https://test-prof.evilmartians.io/guide/recipes/logging

Manually include the LoggingHelpers module to use logging helpers in Minitest classes.

```ruby
class MyLoggingTest < Minitest::Test
  include TestProf::Rails::LoggingHelpers
end
```

--------------------------------

### Identify slow factories

Source: https://test-prof.evilmartians.io/guide/playbook

Run this command to identify the most frequently used factories in tests tagged as slow.

```sh
FPROF=1 bin/rspec --tag slow:factory
```

--------------------------------

### Run Sidekiq inline instrumentation

Source: https://test-prof.evilmartians.io/guide/profilers/event_prof

Profile time spent running Sidekiq jobs, excluding nested jobs.

```sh
EVENT_PROF=sidekiq.inline bundle exec rspec
```

--------------------------------

### Generate HTML Report

Source: https://test-prof.evilmartians.io/guide/profilers/tag_prof

Command to generate an interactive HTML report for TagProf.

```sh
TAG_PROF=type TAG_PROF_FORMAT=html bundle exec rspec
```

--------------------------------

### Export Profile Results to JSON

Source: https://test-prof.evilmartians.io/guide/profilers/factory_prof

Save profiling data to a JSON file by setting the FPROF variable to json.

```bash
FPROF=json rspec

# or
FPROF=json bundle exec rake test
```

--------------------------------

### Generate FactoryFlame report via environment variable

Source: https://test-prof.evilmartians.io/guide/profilers/factory_prof

Set the FPROF environment variable to 'flamegraph' before running your test suite to enable report generation.

```sh
FPROF=flamegraph rspec

# or
FPROF=flamegraph bundle exec rake test
```

--------------------------------

### Define a shared context for Sidekiq inline mode

Source: https://test-prof.evilmartians.io/guide/recipes/rspec_stamp

Create a shared context that conditionally applies Sidekiq inline testing mode based on a specific tag.

```ruby
shared_context "sidekiq:inline", sidekiq: :inline do
  around(:each) { |ex| Sidekiq::Testing.inline!(&ex) }
end
```

--------------------------------

### Configure global let_it_be freezing

Source: https://test-prof.evilmartians.io/guide/recipes/let_it_be

Use RSpec configuration to enable freezing for specific metadata tags, allowing for gradual adoption of leakage detection.

```ruby
# spec/spec_helper.rb
RSpec.configure do |config|
  # ...
  config.define_derived_metadata(let_it_be_frost: true) do |metadata|
    metadata[:let_it_be_modifiers] ||= {freeze: true}
  end
end
```

--------------------------------

### Configure TPSProf Programmatically

Source: https://test-prof.evilmartians.io/guide/profilers/tps_prof

Set profiling and strict mode thresholds within your RSpec test helper configuration.

```ruby
TestProf::TPSProf.configure do |config|
  config.top_count = 15
  # Profiling thresholds
  config.min_examples_count = 5
  config.min_group_time = 3
  config.min_target_tps = 20
  # Strict mode settings
  config.max_examples_count = 100
  config.max_group_time = 60
  config.min_tps = 5
end
```

--------------------------------

### Configure Factory Variations

Source: https://test-prof.evilmartians.io/guide/profilers/factory_prof

Enable reporting for factory traits and overrides via configuration or environment variables.

```ruby
TestProf::FactoryProf.configure do |config|
  config.include_variations = true
end
```

```bash
$ FPROF=1 FPROF_VARS=1 bin/rails test
```

--------------------------------

### Register custom instrumenter in EventProf

Source: https://test-prof.evilmartians.io/guide/profilers/event_prof

Assign the custom wrapper module to the instrumenter configuration option.

```ruby
TestProf::EventProf.configure do |config|
  config.instrumenter = MyEventsWrapper
end
```
Tokens:
Raw