GitHub Actions CI config for MySQL, Redis, Elasticsearch in Ruby on Rails project with RSpec tests

Written by knapsackpro | Published 2021/01/16
Tech Story Tags: github | github-actions | continuous-integration | redis | mysql | elasticsearch | ruby | good-company

TLDR Ruby on Ruby on Rails project has MySQL, Redis, Elasticsearch databases on GitHub Actions. GitHub Actions yaml configuration will allow you to run tests on GitHub CI server. Tests are executed across parallel jobs thanks to matrix feature in GitHub Actions and the Knapsack Pro ruby gem that will auto-balance tests distribution across jobs. You can find results of executed CI builds in Actions tab on your GitHub repository page. You need to set up services with docker container to run each of the databases.via the TL;DR App

You will learn how to configure Ruby on Rails project on GitHub Actions. This specific Rails project has MySQL and Redis database. There is also Elasticsearch service running on CI. If your project is close to that setup below GitHub Actions yaml configuration will allow you to run tests on GitHub CI server.
If you happen to use PostgreSQL you can check our previous article about Rails app configuration with Postgres on GitHub Actions.

GitHub Actions configuration for Rails

In your repository, you need to create file 
.github/workflows/main.yaml
. Thanks to it GitHub Actions will run your CI builds. You can find results of executed CI builds in Actions tab on your GitHub repository page.
In this case, Rails application has MySQLRedis, and Elasticsearch databases. You need to set up services with docker container to run each. In the below config, there is also a step for health check the MySQL and Elasticsearch to ensure both are up and running before you can start running tests.
The tests are executed across parallel jobs thanks to matrix feature in GitHub Actions and the Knapsack Pro ruby gem that will auto-balance tests distribution across jobs. Auto balancing tests using Knapsack Pro Queue Mode will ensure each parallel job finish work at a similar time. Thanks to that there is no bottleneck (no slow job with too many tests to run) and you can enjoy fast CI build time because you get optimal tests split across parallel tasks.
# .github/workflows/main.yaml
name: Main

on: [push]

jobs:
  vm-job:
    name: CI
    runs-on: ubuntu-latest

    # If you need DB like MySQL then define service below.
    # Example for PostgreSQL and Redis can be found here:
    # https://github.com/actions/example-services/tree/master/.github/workflows
    services:
      # How to use MySQL
      mysql:
        image: mysql:5.7
        env:
          MYSQL_ROOT_PASSWORD: root
        ports:
        - 3306
        options: --health-cmd="mysqladmin ping" --health-interval=10s --health-timeout=5s --health-retries=3
      # How to use Redis
      redis:
        image: redis
        ports:
        - 6379/tcp
      # How to use Elasticsearch
      elasticsearch:
        image: elasticsearch:6.8.3
        ports:
        - 9200/tcp
        options: -e="discovery.type=single-node" --health-cmd="curl http://localhost:9200/_cluster/health" --health-interval=10s --health-timeout=5s --health-retries=10

    # https://help.github.com/en/articles/workflow-syntax-for-github-actions#jobsjob_idstrategymatrix
    strategy:
      fail-fast: false
      matrix:
        # Set N number of parallel jobs you want to run tests on.
        # Use higher number if you have slow tests to split them on more parallel jobs.
        # Remember to update ci_node_index below to 0..N-1
        ci_node_total: [5]
        # set N-1 indexes for parallel jobs
        # When you run 2 parallel jobs then first job will have index 0, the second job will have index 1 etc
        ci_node_index: [0, 1, 2, 3, 4]

    steps:
    - name: Checkout
      uses: actions/checkout@v1
      with:
        fetch-depth: 1
        
    - name: Set up Ruby 2.6
      uses: actions/setup-ruby@v1
      with:
        ruby-version: 2.6.5
        
    - uses: actions/cache@v1
      with:
        path: vendor/bundle
        key: ${{ runner.os }}-gems-${{ hashFiles('**/Gemfile.lock') }}
        restore-keys: |
          ${{ runner.os }}-gems-
    - name: Verify MySQL connection from host
      run: |
        sudo apt-get install -y mysql-client libmysqlclient-dev
        mysql --host 127.0.0.1 --port ${{ job.services.mysql.ports[3306] }} -uroot -proot -e "SHOW GRANTS FOR 'root'@'localhost'"
        mysql_tzinfo_to_sql /usr/share/zoneinfo | mysql --host 127.0.0.1 --port ${{ job.services.mysql.ports[3306] }} -uroot -proot mysql
    - name: Verify Elasticsearch connection from host
      env:
        ELASTIC_SEARCH_URL: http://localhost:${{ job.services.elasticsearch.ports[9200] }}
      run: |
        echo $ELASTIC_SEARCH_URL
        curl -fsSL "$ELASTIC_SEARCH_URL/_cat/health?h=status"
    - name: Bundle Install and Create DB
      env:
        RAILS_ENV: test
        DB_PASSWORD: root
        # tell Rails to use proper port for MySQL
        DB_PORT: ${{ job.services.mysql.ports[3306] }}
      run: |
        cp .env.sample .env
        cp config/database.yml.ci config/database.yml
        gem install bundler --version 1.17.3 --no-ri --no-rdoc
        bundle config path vendor/bundle
        bundle install --jobs 4 --retry 3 --path vendor/bundle
        # create DB for Rails 5.x
        bin/rails db:setup
        # create DB for Rails 6.x
        bin/rails db:prepare
    - name: Run tests
      env:
        DB_PASSWORD: root
        DB_PORT: ${{ job.services.mysql.ports[3306] }} # get randomly assigned published port
        REDIS_PORT: ${{ job.services.redis.ports[6379] }}
        ELASTIC_SEARCH_URL: http://localhost:${{ job.services.elasticsearch.ports[9200] }}
        RAILS_ENV: test
        # You need to set API tokens in GitHub repository Settings -> Secrets
        KNAPSACK_PRO_TEST_SUITE_TOKEN_RSPEC: ${{ secrets.KNAPSACK_PRO_TEST_SUITE_TOKEN_RSPEC }}
        KNAPSACK_PRO_TEST_SUITE_TOKEN_CUCUMBER: ${{ secrets.KNAPSACK_PRO_TEST_SUITE_TOKEN_CUCUMBER }}
        KNAPSACK_PRO_TEST_SUITE_TOKEN_MINITEST: ${{ secrets.KNAPSACK_PRO_TEST_SUITE_TOKEN_MINITEST }}
        KNAPSACK_PRO_TEST_SUITE_TEST_UNIT: ${{ secrets.KNAPSACK_PRO_TEST_SUITE_TOKEN_TEST_UNIT }}
        KNAPSACK_PRO_TEST_SUITE_TOKEN_SPINACH: ${{ secrets.KNAPSACK_PRO_TEST_SUITE_TOKEN_SPINACH }}
        KNAPSACK_PRO_CI_NODE_TOTAL: ${{ matrix.ci_node_total }}
        KNAPSACK_PRO_CI_NODE_INDEX: ${{ matrix.ci_node_index }}
        # if you use Knapsack Pro Queue Mode you must set below env variable
        # to be able to retry CI build and run previously recorded tests
        KNAPSACK_PRO_FIXED_QUEUE_SPLIT: true
      run: |
        # run tests in Knapsack Pro Regular Mode
        bundle exec rake knapsack_pro:rspec
        bundle exec rake knapsack_pro:cucumber
        bundle exec rake knapsack_pro:minitest
        bundle exec rake knapsack_pro:test_unit
        bundle exec rake knapsack_pro:spinach
        # you can use Knapsack Pro in Queue Mode once recorded first CI build with Regular Mode
        bundle exec rake knapsack_pro:queue:rspec
        bundle exec rake knapsack_pro:queue:cucumber
        bundle exec rake knapsack_pro:queue:minitest

How Knapsack Pro Queue Mode works

In this video, you will learn how dynamic test suite spilt across parallel jobs (parallel CI nodes) work.

How Knapsack Pro Regular Mode works

Knapsack Pro has also a deterministic way of splitting tests. Tests are split only once before running tests. This is the most simple way that you will try for the first time to record your CI build before you switch to Queue Mode.
You can learn more about Knapsack Pro in the installation guide.

Summary

If you would like to learn more about GitHub Actions check our previous article about testing Rails app with PostgreSQL on GitHub Actions. There is also a video showing it in action.
And if you are still considering using GitHub Actions, then our comparison of GitHub Actions vs other CIs may be helpful for you. The most commonly visited comparisons are: Github Actions vs Circle CIGithub Actions vs Jenkins, and Github Actions vs Travis CI.

Written by knapsackpro | Speed up your tests. Run 1 hour test suite in 2 minutes with optimal parallelisation on CI server.
Published by HackerNoon on 2021/01/16