### Execute Command with Test Files as Arguments Example Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Example of using '--exec-args' to echo test files. ```bash $ parallel_tests --exec-args echo > echo spec/a_spec.rb spec/b_spec.rb ``` -------------------------------- ### Setup Environment from Scratch Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md A comprehensive rake task to create databases and load the schema, ideal for CI environments. ```bash rake parallel:setup ``` -------------------------------- ### Install parallel_tests Gem Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Install the parallel_tests gem using Bundler. ```bash gem install parallel_tests ``` -------------------------------- ### GitLab CI Parallel Test Setup Source: https://github.com/grosser/parallel_tests/wiki/Distributed-Parallel-Tests-on-CI-systems Integrate `parallel_tests` with GitLab CI using the `parallel` keyword. This example demonstrates how to dynamically determine test groups based on GitLab CI variables like `CI_NODE_INDEX` and `CI_NODE_TOTAL`. ```yaml parallel_tests: parallel: 8 variables: # if you use https://docs.gitlab.com/ee/ci/yaml/#parallel # instead of running parallel per-core, this setting allows you # how many jobs should run per-container in parallel # fallback/default: PARALLEL_STEPS=2 # which means that two parallel jobs are running per container PARALLEL_STEPS: "" image: ruby stage: test services: - name: postgres:13 alias: postgresql script: - | set -x # CI_NODE_INDEX is a GitLab variable # telling you at what CI_NODE you are currently running # when you are using https://docs.gitlab.com/ee/ci/yaml/#parallel INDEX="${CI_NODE_INDEX:-}" # CI_NODE_TOTAL is always set by GitLab-CI-runner # if you have not configured 'parallel' CI_NODE_TOTAL=1 # otherwise total will be either CPU cores or 1 if not parallelized TOTAL="${CI_NODE_TOTAL:-1}" # how many parallel_steps should run per CI_NODE-host steps="${PARALLEL_STEPS:-2}" # the index, corrected to count at 0 instead of 1 index="$((${INDEX}-1))" # with steps we allow groups to run START="$(($steps * $index + 1))" STOP="$(($steps * $index + $steps))" # generate a list of which range of groups should be run # generates something like "1,2" for the first two groups, "3,4" groups="$(seq -s, "${START}" 1 "${STOP}")" # the total amount of job we are running TOTAL_JOBS="$(( ${TOTAL} * ${steps} ))" rake parallel:create["${steps}"] rake parallel:rake[db:structure:load,"${steps}"] rake parallel:seed["${steps}"] export DB_SEED_ALREADY_DONE=1 rake parallel:rake[sphinx:parallel_setup,"${steps}"] parallel_rspec -n "${TOTAL_JOBS}" --only-group "${groups}" ./spec ``` -------------------------------- ### Minitest Runtime Logger Setup Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Add this to your test_helper.rb to enable runtime logging when RECORD_RUNTIME is set. The logfile can be configured. ```ruby if ENV['RECORD_RUNTIME'] require 'minitest' require 'parallel_tests/test/runtime_logger' # ParallelTests::Test::RuntimeLogger.logfile = "tmp/parallel_runtime_test.log" # where to write it end ``` -------------------------------- ### Specify Groups Example Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Example of using '--specify-groups' to define which specs run in which processes. ```bash $ parallel_tests -n 3 . --specify-groups '1_spec.rb,2_spec.rb|3_spec.rb' ``` -------------------------------- ### Parallel Setup/Teardown Logic Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Implement setup and cleanup logic that runs once across parallel processes, handling potential race conditions and ensuring proper execution order. ```ruby require "parallel_tests" # preparation: # affected by race-condition: first process may boot slower than the second # the Process.ppid will be the pod of the process that started the parallel tests # when not using TEST_ENV_NUMBER we use a unique file per process because ppid would be the users shell done = "/tmp/parallel-setup-done-#{ENV['TEST_ENV_NUMBER'] ? Process.ppid : Process.pid}" if ParallelTests.first_process? do_something File.write done, "true" else sleep 0.1 until File.exist?(done) end # cleanup: # could also use last_process? but that is just the last process to start, not the last to finish at_exit do if ParallelTests.first_process? File.unlink done ParallelTests.wait_for_other_processes_to_finish undo_something end end ``` -------------------------------- ### Install parallel_tests as a Plugin Source: https://github.com/grosser/parallel_tests/wiki/Rails-2 Install the parallel_tests plugin from GitHub and add its tasks to your Rakefile. ```bash gem install parallel ``` ```ruby config.gem "parallel" ``` ```bash ./script/plugin install git://github.com/grosser/parallel_tests.git ``` ```ruby begin; require 'vendor/plugins/parallel_tests/lib/parallel_tests/tasks'; rescue LoadError; end ``` -------------------------------- ### Start Headless with Unique Display for Parallel Tests Source: https://github.com/grosser/parallel_tests/wiki/Home Initialize Headless with a specific display number and reuse option to manage graphical environments in parallel tests. ```ruby Headless.new(display: 100, reuse: true, destroy_at_exit: false).start ``` -------------------------------- ### RSpec VerboseLogger Usage Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Use VerboseLogger to print a single line for starting and finishing each example, allowing you to see what is currently running in each process. ```text rspec /path/to/my_spec.rb:123 # should do something ``` -------------------------------- ### Create Additional Test Databases Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Use rake tasks to create the necessary test databases, especially for multi-database setups. ```bash rake parallel:create ``` ```bash rake parallel:create: # if using multi-db setup, would be `secondary` for example ``` -------------------------------- ### Install parallel_tests as a Gem Source: https://github.com/grosser/parallel_tests/wiki/Rails-2 Install the parallel_tests gem and add it to your Rails development environment and Rakefile. ```bash gem install parallel_tests ``` ```ruby config.gem "parallel_tests" ``` ```ruby begin; require 'parallel_tests/tasks'; rescue LoadError; end ``` -------------------------------- ### Docker Selenium shm_size Configuration Source: https://github.com/grosser/parallel_tests/wiki/Home Configure the `shm_size` for Selenium Docker containers to avoid browser crashes due to insufficient shared memory. Example using Docker Compose. ```yaml services: selenium: image: selenium/standalone-chrome:3.141.59@sha256:d0ed6e04a4b87850beb023e3693c453b825b938af48733c1c56fc671cd41fe51 shm_size: 1G ``` -------------------------------- ### Initialize Elasticsearch Test Cluster Options for Parallel Tests Source: https://github.com/grosser/parallel_tests/wiki/Home Define options for starting an Elasticsearch test cluster, ensuring each process uses a unique port, cluster name, and data path based on TEST_ENV_NUMBER. ```ruby es_options = { port: 9250 + ENV['TEST_ENV_NUMBER'].to_i, cluster_name: "cluster#{ENV['TEST_ENV_NUMBER']}", path_data: "/tmp/elasticsearch_test#{ENV['TEST_ENV_NUMBER']}" } ``` -------------------------------- ### Configure SimpleCov for Parallel Tests Source: https://github.com/grosser/parallel_tests/wiki/Home Setup SimpleCov to format reports correctly in parallel test environments, ensuring the report is only formatted once when the last process finishes. ```ruby if ENV['COVERAGE'] == 'true' require 'simplecov' require 'simplecov-console' SimpleCov.formatter = SimpleCov::Formatter::Console SimpleCov.start 'rails' if ENV['TEST_ENV_NUMBER'] # parallel specs SimpleCov.at_exit do result = SimpleCov.result result.format! if ParallelTests.number_of_running_processes <= 1 end end end RSpec.configure do |config| ... end ``` -------------------------------- ### Configure RSpec with In-Memory SQLite for Spork Source: https://github.com/grosser/parallel_tests/wiki/Home Add a setup lambda to `spec_helper.rb` to establish an in-memory SQLite database connection and load the schema. This resolves database lock issues when using Spork with parallel tests. ```ruby setup_sqlite_db = lambda do ActiveRecord::Base.establish_connection(adapter: 'sqlite3', database: ':memory:') load "#{Rails.root.to_s}/db/schema.rb" # use db agnostic schema by default end silence_stream(STDOUT, &setup_sqlite_db) ``` -------------------------------- ### Specify Custom Group Formation Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Use '--specify-groups' to define a custom formation of specs across multiple processes. Commas group specs in the same process, pipes start a new process. ```bash --specify-groups SPECS ``` -------------------------------- ### RSpec FailuresLogger Configuration Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Configure RSpec to use FailuresLogger to produce pasteable command-line snippets for each failed example. Add to .rspec_parallel or .rspec, or use via command line. ```bash --format progress --format ParallelTests::RSpec::FailuresLogger --out tmp/failing_specs.log ``` -------------------------------- ### Drop All Test Databases Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Remove all test databases created by parallel_tests. Supports multi-db setups. ```bash rake parallel:drop ``` ```bash rake parallel:drop: # for multi-db setup ``` -------------------------------- ### Update Rakefile for Rails 2 plugin Source: https://github.com/grosser/parallel_tests/wiki/Upgrading-0.6.x-to-0.7.x If you are using Rails 2 and installed parallel_tests as a plugin, deprecated task-autoloading has been removed. Update your Rakefile to reflect this change. ```ruby # add to your Rakefile begin; require 'vendor/plugins/parallel_tests/lib/parallel_tests/tasks'; rescue LoadError; end ``` -------------------------------- ### Travis CI Parallel Test Configuration Source: https://github.com/grosser/parallel_tests/wiki/Distributed-Parallel-Tests-on-CI-systems Configure Travis CI to run parallel tests by assigning test groups to different build workers using environment variables. Ensure `parallel_tests` is installed and configured in your project. ```yaml ... env: - "TEST_GROUP=1" - "TEST_GROUP=2" - "TEST_GROUP=3" - "TEST_GROUP=4" - "TEST_GROUP=5" - "TEST_GROUP=6" script: - bundle exec parallel_test spec/ -n 6 --only-group $TEST_GROUP --group-by filesize --type rspec ``` -------------------------------- ### Prepare Test Databases with Schema Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Copy the development schema into all test databases. This should be repeated after migrations. ```bash rake parallel:prepare ``` -------------------------------- ### Execute Command with Test Files as Arguments Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Use '--exec-args' to run a command with test files passed as arguments. ```bash --exec-args COMMAND ``` -------------------------------- ### Enable First Environment as 1 Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Use the --first-is-1 flag or set the PARALLEL_TEST_FIRST_IS_1 environment variable to true to make the first environment number 1. ```shell export PARALLEL_TEST_FIRST_IS_1=true ``` -------------------------------- ### Pass Test Options and Files via -- Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Use the '--' separator to pass arguments like test tags and formatters to the underlying test runner. ```bash parallel_rspec -- -t acceptance -f progress -- spec/foo_spec.rb spec/acceptance ``` -------------------------------- ### Configure Database URL with Environment Variable Source: https://github.com/grosser/parallel_tests/wiki/CircleCI-recipe Sets the database URL in `database.yml` using an environment variable, allowing for dynamic configuration in different test environments. ```yaml # database.yml url: <%= ... %><%= ENV['TEST_ENV_NUMBER'] %> ``` -------------------------------- ### Configure Cache Store with Namespaces Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md For Memcached, use different namespaces based on the test environment number to avoid cache collisions. ```ruby config.cache_store = ..., namespace: "test_#{ENV['TEST_ENV_NUMBER']}" ``` -------------------------------- ### Run a Subset of Test Files Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Specify directories or individual files to run tests only on the selected subset. ```bash parallel_test test/bar test/baz/foo_text.rb ``` -------------------------------- ### Run Migrations in Parallel Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Execute database migrations across all test databases in parallel. Useful after schema changes. ```bash rake parallel:migrate ``` ```bash rake parallel:migrate: # for multi-db setup ``` -------------------------------- ### Configure RSpec with ci_reporter and parallel_tests Source: https://github.com/grosser/parallel_tests/wiki/Home Set up environment variables and configuration files to use ci_reporter with RSpec and parallel_tests for generating reports. ```bash export CI_REPORTS=results ``` ```text --format progress --require ci/reporter/rake/rspec_loader --format CI::Reporter::RSpec:/dev/null ``` ```ruby if ENV["CI_REPORTS"] == "results" require "ci/reporter/rake/test_unit_loader" end ``` ```bash rake "parallel:features[,,--format progress --format junit --out ${CI_REPORTS} --no-profile -r features]" ``` ```bash bundle exec $(bundle show parallel_tests)/bin/parallel_test --type features -o '--format progress --format junit --out ${CI_REPORTS} --no-profile -r features' ``` ```text --format progress --require ci/reporter/rake/rspec_loader --format CI::Reporter::RSpec ``` ```bash rake parallel:spec ``` -------------------------------- ### Run Matching Files in Single Process Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Use '--single' to run all matching files within a single parallel process. ```bash -s, --single PATTERN ``` -------------------------------- ### Configure Redis Database for Parallel Tests Source: https://github.com/grosser/parallel_tests/wiki/Home Select a unique Redis database for each parallel test process by calculating the database index from TEST_ENV_NUMBER. Defaults to database 0 if TEST_ENV_NUMBER is not set. ```ruby db = ENV['TEST_ENV_NUMBER'].nil? ? 1 : (ENV['TEST_ENV_NUMBER'].presence || '1').to_i - 1 Redis.new(db: db) ``` -------------------------------- ### Run Spinach Tests in Parallel Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Execute Spinach tests in parallel using the parallel_spinach command. ```bash parallel_spinach ``` -------------------------------- ### Configure Parallel Test Databases in CircleCI Source: https://github.com/grosser/parallel_tests/wiki/CircleCI-recipe Sets up parallel test databases and loads the schema. It also restores and attaches workspaces for caching test artifacts. ```yaml environment: PARALLEL_TEST_PROCESSORS: "12" ... - run: name: setup parallel test databases command: | bundle exec rake parallel:create bundle exec rake parallel:load_schema - restore_cache: keys: - parallel-runtime-v1- - attach_workspace: at: workspace ... - persist_to_workspace: root: tmp paths: - parallel_runtime_rspec_*.log ... ``` -------------------------------- ### Configure ActiveStorage Disk Service Root for Parallel Tests Source: https://github.com/grosser/parallel_tests/wiki/Home Modify the 'root' path in config/storage.yml for the test environment to include TEST_ENV_NUMBER, ensuring each process uses a separate temporary storage directory. ```yaml test: service: Disk root: <%= Rails.root.join("tmp/storage#{ENV['TEST_ENV_NUMBER']}") %> ``` -------------------------------- ### Update command to run all tests Source: https://github.com/grosser/parallel_tests/wiki/Upgrading-0.6.x-to-0.7.x When running all tests from a specific folder, ensure you use the updated command structure. Passing a folder path is now required for targeted execution. ```bash parallel_spec -> parallel_rspec spec ``` -------------------------------- ### Execute Command in Parallel Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Use '--exec' to run a specified command in parallel, with ENV['TEST_ENV_NUMBER'] set. ```bash -e, --exec COMMAND ``` -------------------------------- ### Run Multiple Commands with Shell Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Execute multiple commands in parallel by using `sh -c` in conjunction with `--exec-args`. This allows for more complex command sequences to be run in parallel. ```bash parallel_test -n 3 --exec-args "sh -c \"echo 'hello world' && rspec $@\" --" ``` -------------------------------- ### Run Tests in Parallel Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Execute tests in parallel using the parallel_test command. ```bash parallel_test ``` -------------------------------- ### Group Tests by Type Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Configure how tests are grouped for parallel execution using the '--group-by' option. ```bash --group-by TYPE ``` -------------------------------- ### Run Tests with Specific Parallelism Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Force a specific number of parallel processes for testing, useful for performance analysis. ```bash rake "parallel:test[1]" --> force 1 CPU --> 86 seconds ``` ```bash rake parallel:test --> got 2 CPUs? --> 47 seconds ``` ```bash rake parallel:test --> got 4 CPUs? --> 26 seconds ``` -------------------------------- ### Sphinx Configuration for Parallel Tests Source: https://github.com/grosser/parallel_tests/wiki/Home Configure Sphinx settings, including ports, index locations, log files, and PID files, to be unique for each test process. This is added to the `test` section of `config/sphinx.yml`. ```yaml test: mysql41: <%= 9313 + ENV['TEST_ENV_NUMBER'].to_i %> indices_location: <%= File.join(Rails.root, "db", "sphinx", "test#{ENV['TEST_ENV_NUMBER']}") %> configuration_file: <%= File.join(Rails.root, "config", "test#{ENV['TEST_ENV_NUMBER']}.sphinx.conf")%> log: <%= File.join(Rails.root, "log", "test#{ENV['TEST_ENV_NUMBER']}.searchd.log") %> query_log: <%= File.join(Rails.root, "log", "test#{ENV['TEST_ENV_NUMBER']}.searchd.query.log") %> binlog_path: <%= File.join(Rails.root, "tmp", "binlog", "test#{ENV['TEST_ENV_NUMBER']}") %> pid_file: <%= File.join(Rails.root, "tmp", "pids", "test#{ENV['TEST_ENV_NUMBER']}.sphinx.pid") %> ``` ```yaml test: mysql41: <%= ENV['TEST_ENV_NUMBER'].to_i + 9307 %> pid_file: <%= File.join(Rails.root, "tmp", "searchd.#{ENV['TEST_ENV_NUMBER']}.pid") %> indices_location: <%= File.join(Rails.root, "db", "sphinx", "#{ENV['TEST_ENV_NUMBER']}") %> configuration_file: <%= File.join(Rails.root, "config", "test.#{ENV['TEST_ENV_NUMBER']}.sphinx.conf") %> binlog_path: <%= File.join(Rails.root, "db", "sphinx", "#{ENV['TEST_ENV_NUMBER']}", "binlog") %> ``` -------------------------------- ### Update formatter namespaces and log file location Source: https://github.com/grosser/parallel_tests/wiki/Upgrading-0.6.x-to-0.7.x Formatter namespaces have been unified, and the runtime log file for spec tests has moved to a new location. Update any references to these in your configuration. ```text ParallelSpec::SpecRuntimeLogger --> ParallelTests::RSpec::RuntimeLogger ``` ```text tmp/parallel_profile.log --> tmp/parallel_runtime_rspec.log ``` -------------------------------- ### Configure Database for Parallel Tests Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Modify your database.yml to use a unique database for each test process by appending ENV['TEST_ENV_NUMBER']. ```yaml test: database: yourproject_test<%= ENV['TEST_ENV_NUMBER'] %> ``` -------------------------------- ### Distribute Test Groups Across Machines Source: https://github.com/grosser/parallel_tests/wiki/Distributed-Parallel-Tests-on-CI-systems Use `--only-group` to specify which test groups a machine should run. This is useful when distributing tests across separate physical or virtual machines. ```bash parallel_test test -n 6 --only-group 1,2 ``` ```bash parallel_test test -n 6 --only-group 3,4 ``` ```bash parallel_test test -n 6 --only-group 5,6 ``` -------------------------------- ### Specify Number of Processes Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Control the number of parallel processes using the '-n' option. Defaults to available CPUs. ```bash -n PROCESSES ``` -------------------------------- ### Run Tests by Pattern with Regex Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Filter tests to run in parallel based on a regular expression pattern, targeting specific files or test suites. ```bash rake "parallel:test[^test/unit]" # every test file in test/unit folder ``` ```bash rake "parallel:test[user]" # run users_controller + user_helper + user tests ``` ```bash rake "parallel:test['user|product']" # run user and product related tests ``` ```bash rake "parallel:spec['spec\/(?!features)']" # run RSpec tests except the tests in spec/features ``` -------------------------------- ### Configure Action Mailer Cache Delivery for parallel_tests Source: https://github.com/grosser/parallel_tests/wiki/Home Ensure cache files differ for parallel processes by appending the test environment number to the cache file location in `config/environment/test.rb`. ```ruby config.action_mailer.cache_settings = { :location => "#{Rails.root}/tmp/cache/action_mailer_cache_delivery#{ENV['TEST_ENV_NUMBER']}.cache" } ``` -------------------------------- ### Pass Test Options to Runner Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Use the '-o' or '--test-options' flag to pass specific options to the underlying test runner. ```bash -o, --test-options 'OPTIONS' ``` -------------------------------- ### Configure Sunspot for Parallel Tests (RSpec >= 2.2.0) Source: https://github.com/grosser/parallel_tests/wiki/Home Update `config/sunspot.yml` to use `solr_home` instead of `data_path` for newer versions of Sunspot, dynamically setting the Solr home directory based on `TEST_ENV_NUMBER`. ```yaml test: solr: port: <%= 8981 + ENV['TEST_ENV_NUMBER'].to_i %> solr_home: <%= File.join(::Rails.root, 'solr', 'data', ::Rails.env, ENV['TEST_ENV_NUMBER'].to_i.to_s) %> ``` -------------------------------- ### Run Commands in Parallel Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Execute a given command in parallel across multiple processes. The `-n` option specifies the number of parallel processes, and `-e` or `--exec` specifies the command to run. ```bash parallel_test -n 3 -e 'ruby -e "puts %[hello from process #{ENV[:TEST_ENV_NUMBER.to_s].inspect}]"' hello from process "2" hello from process "" hello from process "3" ``` -------------------------------- ### Multiply Processes Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Use '--multiply-processes' to set the number of processes as a multiplier of the default. ```bash -m, --multiply-processes COUNT ``` -------------------------------- ### Run Arbitrary Rake Tasks in Parallel Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Execute any custom rake task in parallel across multiple processes. Limited parallelism can be specified. ```bash RAILS_ENV=test parallel_test -e "rake my:custom:task" ``` ```bash # or rake "parallel:rake[my:custom:task]" ``` ```bash # limited parallelism rake "parallel:rake[my:custom:task,2]" ``` -------------------------------- ### Configure Searchkick Index Name for Parallel Tests Source: https://github.com/grosser/parallel_tests/wiki/Home Set a unique index name for each test process by appending the TEST_ENV_NUMBER to the default index name. ```ruby class Product < ActiveRecord::Base searchkick index_name: "products#{ENV['TEST_ENV_NUMBER']}" end ``` -------------------------------- ### Run Only Specific Groups Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Use '--only-group' to execute only the specified group numbers, defaulting 'group-by' to 'filesize'. ```bash --only-group GROUP_INDEX[,GROUP_INDEX] ``` -------------------------------- ### Run Tests in Parallel Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Execute Minitest, RSpec, Cucumber, or Spinach tests in parallel using dedicated rake tasks. ```bash rake parallel:test # Minitest ``` ```bash rake parallel:spec # RSpec ``` ```bash rake parallel:features # Cucumber ``` ```bash rake parallel:features-spinach # Spinach ``` -------------------------------- ### Filter Tests with Pattern Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Use the '--pattern' option to run tests matching a specific regex pattern. ```bash -p, --pattern PATTERN ``` -------------------------------- ### Rails File Cache Store for Parallel Tests Source: https://github.com/grosser/parallel_tests/wiki/Home Configure Rails to use a unique file store for each test process to prevent concurrent access issues. Add this to `config/environments/test.rb`. ```ruby config.cache_store = :file_store, Rails.root.join("tmp", "cache", "paralleltests#{ENV['TEST_ENV_NUMBER']}") ``` -------------------------------- ### Add SimpleCov Gem for Test Environment Source: https://github.com/grosser/parallel_tests/wiki/Home Include the simplecov and simplecov-console gems in the test group of your Gemfile for code coverage reporting. ```ruby group :test do gem 'simplecov' gem 'simplecov-console' end ``` -------------------------------- ### RSpec JUnit Formatter for Parallel Tests Source: https://github.com/grosser/parallel_tests/wiki/Home Configure rspec_junit_formatter to output unique XML files per test process, which can then be combined. Add these lines to `.rspec_parallel`. ```text --format RspecJunitFormatter --out tmp/rspec<%= ENV['TEST_ENV_NUMBER'] %>.xml ``` -------------------------------- ### Register Poltergeist Driver with Unique Port for Parallel Tests Source: https://github.com/grosser/parallel_tests/wiki/Home Configure the Poltergeist driver to use a distinct port for each parallel test process by adding TEST_ENV_NUMBER to a base port. ```ruby Capybara.register_driver :poltergeist do |app| options = { port: 51674 + ENV['TEST_ENV_NUMBER'].to_i } Capybara::Poltergeist::Driver.new(app, options) end ``` -------------------------------- ### Configure Cucumber for Parallel Tests Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Use this formatter to log failed Cucumber scenarios to a file. The file can be used to rerun failures. ```ruby cucumber --format ParallelTests::Cucumber::FailuresLogger --out tmp/cucumber_failures.log ``` -------------------------------- ### DatabaseCleaner Configuration for RSpec Parallelization Source: https://github.com/grosser/parallel_tests/wiki/Home Avoid using the truncation strategy with DatabaseCleaner when parallelizing RSpec tests to prevent performance bottlenecks. Prefer the transaction strategy. ```text Do not use the truncation strategy in DatabaseCleaner, in your RSpec config. This strategy seems to cause a bottleneck which will negate any gain made through parallelization of your tests. If possible, use the transaction strategy over the truncation strategy. *Note: This issue does not seem to exist in relation to features, only to specs. ``` -------------------------------- ### Set Parallel Rails Environment Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Override the default 'test' Rails environment for parallel tests by exporting the PARALLEL_RAILS_ENV environment variable. ```shell export PARALLEL_RAILS_ENV=environment_name ``` -------------------------------- ### Specify Failure Exit Code Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Set a custom exit code for test failures using '--failure-exit-code'. ```bash --failure-exit-code INT ``` -------------------------------- ### RSpec SummaryLogger Configuration Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Configure RSpec to use SummaryLogger for logging test output without overwrites. Add to .rspec_parallel or .rspec, or use via command line. ```bash --format progress --format ParallelTests::RSpec::SummaryLogger --out tmp/spec_summary.log ``` -------------------------------- ### Custom Bash Script for Parallel Test Execution Source: https://github.com/grosser/parallel_tests/wiki/Home A bash script to execute files in parallel using `parallel_test --exec`. It finds all `.rb` files and distributes them across test groups based on the `PARALLEL_TEST_GROUPS` environment variable. ```bash #!/bin/bash # custom_parallel_script.sh # Usage: parallel_test --exec ./custom_parallel_script.sh FILES=$(find . -type f -name \*.rb) # workaround for the fact that TEST_ENV_NUMBER is '' for the 1st group - default to 1 if unset LOCAL_TEST_ENV_NUMBER=${TEST_ENV_NUMBER:-1} i=0 for f in $FILES; do if [[ -z "$PARALLEL_TEST_GROUPS" || $(($i % $PARALLEL_TEST_GROUPS)) -eq $LOCAL_TEST_ENV_NUMBER ]]; then echo $f # real action here fi ((i++)) done ``` -------------------------------- ### Configure Test::Unit with ci_reporter for parallel_tests Source: https://github.com/grosser/parallel_tests/wiki/Home Modify Test::Unit's TestRunner to use CI::Reporter's mediator when running on a build server hostname. ```ruby # add the ci_reporter to create reports for test-runs, since parallel_tests is not invoked through rake puts "running on #{Socket.gethostname}" if /buildserver/ =~ Socket.gethostname require 'ci/reporter/test_unit' module Test module Unit module UI module Console class TestRunner def create_mediator(suite) # swap in ci_reporter custom mediator return CI::Reporter::TestUnit.new(suite) end end end end end end end ``` -------------------------------- ### Require features directory for Cucumber step definitions Source: https://github.com/grosser/parallel_tests/wiki/Home Ensure Cucumber can find step definitions when features are in subdirectories by requiring the features folder. ```bash rake "parallel:features[4, '', '-rfeatures/']" ``` -------------------------------- ### Configure Mongoid Test Database for Parallel Tests (>= 5.x) Source: https://github.com/grosser/parallel_tests/wiki/Home Specify a unique database name for each test process in Mongoid 5.x and later by including TEST_ENV_NUMBER in the database name. ```yaml test: sessions: default: database: app_name_test<%= ENV['TEST_ENV_NUMBER'] %> hosts: - localhost:27017 ``` -------------------------------- ### Test Tools Module for Build File Cleanup Source: https://github.com/grosser/parallel_tests/wiki/Home A helper module `TestTools` with a `remove_build_files` method. This method is used to clean up temporary build artifacts, typically by removing a directory like `tmp/test_build`. ```ruby # spec/support/test_tools.rb module TestTools def self.remove_build_files # Doing the cleanup here... FileUtils.rm_rf(Rails.root.join('tmp','test_build')) end end ``` -------------------------------- ### Configure Cucumber Parallel Tests Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Add a 'parallel: foo' profile to your config/cucumber.yml to enable parallel test execution. ```yaml parallel: foo ``` -------------------------------- ### Execute Parallel RSpec Tests with CircleCI Source: https://github.com/grosser/parallel_tests/wiki/CircleCI-recipe This script configures RSpec for parallel execution, defines the test files, and uses CircleCI's test splitting capabilities. It logs runtime information and handles potential errors. ```bash if [[ -f workspace/parallel_runtime_rspec.log ]]; then cp workspace/parallel_runtime_rspec.log tmp/parallel_runtime_rspec.log else echo "No runtime log found, using filesize splitting" fi TEST_FILES=$(circleci tests glob "spec/controllers/**/*_spec.rb" ...... cat > .rspec_parallel <.xml --format ParallelTests::RSpec::RuntimeLogger --out tmp/parallel_runtime_rspec.log EOF RSPEC_COMMAND="bundle exec parallel_rspec \ -n ${PARALLEL_TEST_PROCESSORS:?}\ --runtime-log tmp/parallel_runtime_rspec.log" echo "${TEST_FILES}" | circleci tests run --verbose --split-by=timings --command="xargs -I{} bash -c '${RSPEC_COMMAND} {}'" rspec_status=$? cp tmp/parallel_runtime_rspec.log tmp/parallel_runtime_rspec_"${CIRCLE_NODE_INDEX}".log cp log/test*.log /tmp/test-results/ exit "${rspec_status}" ``` -------------------------------- ### Isolate Single Process Tests with Specific Number of Processes Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Use '--isolate-n' to specify the number of processes for isolated single tests. ```bash --isolate-n PROCESSES ``` -------------------------------- ### Run RSpec Tests in Parallel Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Execute RSpec tests in parallel using the parallel_rspec command. ```bash parallel_rspec ``` -------------------------------- ### Override Searchkick Environment for Parallel Tests Source: https://github.com/grosser/parallel_tests/wiki/Home Dynamically set the Searchkick environment based on the TEST_ENV_NUMBER for testing. ```ruby if Rails.env.test? Searchkick.env = "test#{ENV['TEST_ENV_NUMBER']}" end ``` -------------------------------- ### Override Test File Suffix Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Use '--suffix' to specify a custom regex pattern for identifying test files. ```bash --suffix PATTERN ``` -------------------------------- ### GitHub Actions CI Configuration for RSpec Source: https://github.com/grosser/parallel_tests/wiki/Distributed-Parallel-Tests-on-CI-systems Configure a GitHub Actions workflow to run RSpec tests in parallel using the matrix strategy and parallel_tests. Ensure CI_TOTAL_JOBS and CI_JOB_INDEX environment variables are set correctly for test distribution. ```yaml rspec: name: RSpec groups ${{ matrix.ci_job_index }} runs-on: ubuntu-latest services: postgres: ... env: BUNDLE_WITHOUT: "development" CI_TOTAL_JOBS: ${{ matrix.ci_total_jobs }} CI_JOB_INDEX: ${{ matrix.ci_job_index }} strategy: fail-fast: false matrix: # Set N number of parallel jobs you want to run. # Normally equal to the number of CPU cores but in this case relates to the total number of test groups to be run across all runners. ci_total_jobs: [24] # Remember to update ci_node_index below to 0..N-1 # When you run 2 parallel jobs then first job will have index 0, the second job will have index 1 etc. For larger runners runners with more CPU adjust accordingly. ci_job_index: [ "0, 1", "2, 3", ..... "22, 23" ] steps: - name: Checkout repository uses: actions/checkout@v4 - name: Setup Ruby uses: ruby/setup-ruby@v1 with: bundler-cache: true - name: Run RSpec test on group ${{ matrix.ci_job_index }} env: ... run: | echo "::group::parallel:setup" bundle exec rake parallel:setup[2] echo "::endgroup::" echo "::group::parallel:spec" bundle exec parallel_rspec -n "${CI_TOTAL_JOBS}" --only-group "${CI_JOB_INDEX}" ./spec echo "::endgroup::" ``` -------------------------------- ### Run Cucumber Tests in Parallel Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Execute Cucumber tests in parallel using the parallel_cucumber command. ```bash parallel_cucumber ``` -------------------------------- ### Set Parallel Test Processors Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Override the default number of processors for parallel tests by exporting the PARALLEL_TEST_PROCESSORS environment variable. ```shell export PARALLEL_TEST_PROCESSORS=13 ``` -------------------------------- ### Highest Exit Status Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Configure the exit status to be the highest exit status from any test run using '--highest-exit-status'. ```bash --highest-exit-status ``` -------------------------------- ### Configure Mongoid Test Database for Parallel Tests (<= 4.x) Source: https://github.com/grosser/parallel_tests/wiki/Home Specify a unique database name for each test process in Mongoid 4.x and earlier by including TEST_ENV_NUMBER in the database name. ```yaml test: client: default: database: app_name_test<%= ENV['TEST_ENV_NUMBER'] %> hosts: - localhost:27017 ``` -------------------------------- ### Configure Sunspot for Parallel Tests (RSpec) Source: https://github.com/grosser/parallel_tests/wiki/Home Update `config/sunspot.yml` to dynamically set the Solr port and data path based on the `TEST_ENV_NUMBER` environment variable for parallel RSpec tests. ```yaml test: solr: hostname: localhost port: <%= 8981 + ENV['TEST_ENV_NUMBER'].to_i %> log_level: WARNING data_path: <%= File.join(::Rails.root, 'solr', 'data', ::Rails.env, ENV['TEST_ENV_NUMBER'].to_i.to_s) %> ``` -------------------------------- ### Configure RSpec for Parallel Tests Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Add this formatter to your .rspec_parallel or .rspec file to enable parallel test logging. ```ruby --format ParallelTests::RSpec::VerboseLogger ``` -------------------------------- ### RSpec Runtime Logging Configuration Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Configure RSpec to log test runtimes for use with parallel_tests. This helps in balancing test groups based on actual execution times. ```bash --format progress --format ParallelTests::RSpec::RuntimeLogger --out tmp/parallel_runtime_rspec.log ``` -------------------------------- ### Combine RSpec JUnit XML Reports Source: https://github.com/grosser/parallel_tests/wiki/Home A shell command to rename RSpec JUnit XML files, ensuring they are treated as a single suite for CI reporting tools. This is useful when parallel tests generate multiple XML files. ```bash sed -i 's/name="rspec[0-9]"/name="rspec"/' tmp/junit*.xml # rename so they get combined ``` -------------------------------- ### Aggregate Parallel Test Logs in CircleCI Source: https://github.com/grosser/parallel_tests/wiki/CircleCI-recipe Combines runtime logs from parallel test runs into a single artifact and saves it. This step is useful for post-test analysis. ```yaml aggregate-parallel-test-logs: docker: - image: cimg/base:current steps: - attach_workspace: at: workspace - run: name: Combine runtime logs command: cat workspace/parallel_runtime_rspec_*.log > workspace/parallel_runtime_rspec.log - save_cache: key: parallel-runtime-v1-{{ epoch }} paths: - workspace/parallel_runtime_rspec.log - store_artifacts: path: workspace/parallel_runtime_rspec.log destination: parallel_runtime_rspec.log ``` -------------------------------- ### Set Searchkick Index Suffix for Parallel Tests Source: https://github.com/grosser/parallel_tests/wiki/Home Use Searchkick.index_suffix to append an environment-specific suffix for parallel testing, available in version 2.2.1+. ```ruby Searchkick.index_suffix = ENV["TEST_ENV_NUMBER"] ``` -------------------------------- ### Specify Test Type Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Use the '-t' or '--type' option to specify the type of tests being run (e.g., rspec, cucumber). ```bash -t TYPE ``` -------------------------------- ### Update parallel_spec command Source: https://github.com/grosser/parallel_tests/wiki/Upgrading-0.6.x-to-0.7.x The command-line interface for running RSpec tests has been updated. Use `parallel_rspec` instead of `parallel_spec`. ```bash parallel_spec -> parallel_rspec ``` -------------------------------- ### Capybara Server Port for Parallel Tests Source: https://github.com/grosser/parallel_tests/wiki/Home Assign a unique server port for Capybara to use in each parallel test process. This helps avoid port conflicts. ```ruby Capybara.server_port = 9887 + ENV['TEST_ENV_NUMBER'].to_i ``` -------------------------------- ### Pass Test Options with -o Flag Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Use the '-o' flag to pass complex test runner options, ensuring they are properly quoted. ```bash parallel_cucumber -n 2 -o '-p foo_profile --tags @only_this_tag or @only_that_tag --format summary' ``` -------------------------------- ### RSpec 1.x SpecSummaryLogger Configuration Source: https://github.com/grosser/parallel_tests/wiki/Rails-2 Configure RSpec 1.x to use the parallel_tests SpecSummaryLogger for logging test summaries. ```bash --format progress --require parallel_tests/rspec/summary_logger --format ParallelTests::RSpec::SummaryLogger:tmp/spec_summary.log ``` -------------------------------- ### Configure Cucumber.yml for Parallel Tests Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Add the FailuresLogger to the parallel profile in your cucumber.yml for integrated failure logging. ```ruby parallel: --format progress --format ParallelTests::Cucumber::FailuresLogger --out tmp/cucumber_failures.log ``` -------------------------------- ### Add parallel_tests Gem to Gemfile Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Include the parallel_tests gem in your Gemfile for development and test environments. ```ruby gem 'parallel_tests', group: [:development, :test] ``` -------------------------------- ### Patch Spring for Database Reconnection Source: https://github.com/grosser/parallel_tests/wiki/Spring This Ruby code patches the Spring::Application class to disconnect, reconfigure, and reconnect the database before the original connect_database method is called. It's useful for ensuring a fresh database connection when Spring is running. ```Ruby # config/spring.rb require 'spring/application' class Spring::Application alias connect_database_orig connect_database def connect_database disconnect_database reconfigure_database connect_database_orig end def reconfigure_database if active_record_configured? ActiveRecord::Base.configurations = Rails.application.config.database_configuration end end end ``` -------------------------------- ### RSpec After Suite Cleanup for Parallel Tests Source: https://github.com/grosser/parallel_tests/wiki/Home In `spec_helper.rb`, use `config.after(:suite)` to call `TestTools.remove_build_files` only when `TEST_ENV_NUMBER` is not set. This ensures cleanup runs after the suite finishes, unless running in a parallel environment where the rake task handles it. ```ruby # In spec_helper.rb config.after(:suite) do TestTools.remove_build_files unless ENV.key? 'TEST_ENV_NUMBER' end ``` -------------------------------- ### Define Capistrano Test Task with Streamed Output Source: https://github.com/grosser/parallel_tests/wiki/Remotely-with-capistrano Use this task to run tests on a remote machine via Capistrano. It streams the output and supports colorization for better readability. Requires a stable network connection. ```ruby task :test do [some preparatory stuff] test_stream "rake parallel:spec 2>&1", :colorize => true end def test_stream(command, options = {}) invoke_command(command) do |ch, stream, out| trap("INT") { puts 'Interrupted'; exit 0; } if options[:colorize] out.gsub!(/(?<= ^|[\.F*] ) \. (?= $|[\.F*] )/x, "\\033[32m.\\033[0m") # green out.gsub!(/(?<= ^|[\.F*] ) F (?= $|[\.F*] )/x, "\\033[31mF\\033[0m") # red out.gsub!(/(?<= ^|[\.F*] ) \* (?= $|[\.F*] )/x, "\\033[33m*\\033[0m") # orange end print out end end ``` -------------------------------- ### RSpec 1.x Runtime Logger Configuration Source: https://github.com/grosser/parallel_tests/wiki/Rails-2 Configure RSpec 1.x to use the parallel_tests runtime logger for tracking test execution times. ```bash --format progress --require parallel_tests/rspec/runtime_logger --format ParallelTests::RSpec::RuntimeLogger:tmp/parallel_runtime_rspec.log ``` -------------------------------- ### Fix 'unable to bind to locking port' error with Cucumber, Capybara, and Selenium Source: https://github.com/grosser/parallel_tests/wiki/Home Configure Capybara's server port and add a sleep delay based on the test environment number to resolve port binding issues when running parallel Selenium tests. ```ruby unless (env_no = ENV['TEST_ENV_NUMBER'].to_i).zero? # As described in the readme Capybara.server_port = 8888 + env_no # Enforces a sleep time, i need to multiply by 10 to achieve consistent results on # my 8 cores vm, may work for less though. sleep env_no * 10 end ``` -------------------------------- ### Use Rake Arguments with Quotes Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md When using rake arguments with parallel tests, especially in ZSH, use quotes to ensure correct parsing. ```shell rake "parallel:prepare[3]" ``` -------------------------------- ### Pass Arguments to Executed Command Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Pass additional arguments to the command executed by `parallel_test` using `--exec-args`. This is useful for commands that expect specific arguments, such as file paths. ```bash parallel_test -n 3 --exec-args echo spec/a_spec.rb spec/b_spec.rb spec/c_spec.rb spec/d_spec.rb spec/e_spec.rb ``` -------------------------------- ### Set Parallel Test Processor Multiplier Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Override the default processor multiplier for parallel tests by exporting the PARALLEL_TEST_MULTIPLY_PROCESSORS environment variable. ```shell export PARALLEL_TEST_MULTIPLY_PROCESSORS=.5 ``` -------------------------------- ### Alias for Parallel RSpec Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Create a shell alias for running parallel RSpec tests with specific options. ```shell alias prspec='parallel_rspec -m 2 --' ``` -------------------------------- ### Isolate Single Process Tests Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Use '--isolate' to prevent other tests from running in the same group as tests specified with '--single'. ```bash -i, --isolate ``` -------------------------------- ### Rails Sprockets File Cache for Parallel Tests Source: https://github.com/grosser/parallel_tests/wiki/Home Ensures Sprockets uses a separate cache directory for each test process to avoid template errors due to concurrent access. Add this to `config/environments/test.rb`. ```ruby if ENV['TEST_ENV_NUMBER'] assets_cache_path = Rails.root.join("tmp/cache/assets/paralleltests#{ENV['TEST_ENV_NUMBER']}") Rails.application.config.assets.cache = Sprockets::Cache::FileStore.new(assets_cache_path) end ``` -------------------------------- ### Rerun Failed Cucumber Scenarios Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Use the log file generated by FailuresLogger to rerun only the failed scenarios. ```bash cucumber @tmp/cucumber_failures.log ``` -------------------------------- ### Exclude Tests with Pattern Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Use the '--exclude-pattern' option to exclude tests matching a specific regex pattern. ```bash --exclude-pattern PATTERN ``` -------------------------------- ### Update Rake task for test patterns Source: https://github.com/grosser/parallel_tests/wiki/Upgrading-0.6.x-to-0.7.x The Rake task for specifying test patterns has changed. Paths now include the full path to the file, requiring an update to the Rakefile. ```ruby rake parallel:spec[^models] --> rake parallel:spec[^spec/models] ``` -------------------------------- ### RSpec 1.x SpecFailuresLogger Configuration Source: https://github.com/grosser/parallel_tests/wiki/Rails-2 Configure RSpec 1.x to use the parallel_tests SpecFailuresLogger for logging failing specs. ```bash --format progress --require parallel_tests/rspec/failures_logger --format ParallelTests::RSpec::FailuresLogger:tmp/failing_specs.log ``` -------------------------------- ### Rails Secret Key Base Argument Error Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md When encountering 'ArgumentError: secret_key_base' in Rails with parallel tests, configure a dynamic secret key base. ```ruby config.secret_key_base = Random.hex(64) ``` -------------------------------- ### Configure Cucumber retry with parallel_tests Source: https://github.com/grosser/parallel_tests/wiki/Home Specify the number of retries for Cucumber scenarios when using parallel_tests. ```rake parallel:features[,, --retry 1] ``` -------------------------------- ### Rake Task for Parallel Test Cleanup Source: https://github.com/grosser/parallel_tests/wiki/Home Define a rake task `parallel:clean_spec` that includes `:spec` and a custom `:clean` task. The `:clean` task calls `TestTools.remove_build_files` to remove artifacts after parallel tests. ```ruby require_relative '../../spec/support/test_tools.rb' namespace :parallel do desc "Run parallel spec Clear artifact after parallel testing" task :clean_spec => [:spec, :clean] desc "Remove artifacts after parallel tests" task :clean do TestTools.remove_build_files end end ``` -------------------------------- ### Use TEST_ENV_NUMBER in Tests Source: https://github.com/grosser/parallel_tests/blob/master/Readme.md Access the parallel test environment number within your tests to configure resources like databases or memcache. ```ruby # use `ENV['TEST_ENV_NUMBER']` inside your tests to select separate db/memcache/etc. (docker compose: expose it) ```