nimblehq.co

ExUnit 🌡

Hero image for ExUnit 🌡

ExUnit which is a unit testing framework bundled with Elixir.

All general testing best practices and Web testing best practices apply to ExUnit but working with a framework adds another superset of conventions to follow.

Naming & Structure

  • Put all tests in the test directory with the same structure as the application.

    lib
├── crawler
│   └── keywords.ex
├── crawler_web
│   ├── controllers
│   │   └── keyword_controller.ex
test
├── crawler
│   └── keywords_test.exs
├── crawler_web
│   ├── controllers
│   │   └── keyword_controller_test.exs

  • Add the suffix *_test.exs to all test files.

    /test/crawler/keywords.ex
/test/crawler_web/controller/keyword_controller.ex

    /test/crawler/keywords_test.exs
/test/crawler_web/controller/keyword_controller_test.exs

  • Whenever possible, always run the tests concurrently by setting async: true to the ExUnit.Case in the test suite setup.

    defmodule Crawler.Keywords do
  use Crawler.DataCase, async: true

  # Test codes
end

  • In situations where it is impossible to run the test asynchronously, explicitly state async: false rather than leaving it blank. Prefer to add a why comment.

    defmodule Crawler.Keywords do
  use Crawler.DataCase

  # Test codes
end

    defmodule Crawler.Keywords do
  @moduledoc """
  This module needs to run as `async: false` because it contains the `Application.put_env`
  """
  use Crawler.DataCase, async: false

  setup_all do
    Application.put_env(:crawler, :balance_resource, Balances)

    on_exit(fn ->
      Application.put_env(:crawler, :balance_resource, BalancesMock)
    end)
  end

  # Test codes
end

Formatting

  • Use the function name with arity as a message of describe/2 block and order in the corresponding order in the module.

    # Keywords application module
defmodule Crawler.Keywords do
  def list(user) do
    # ...
  end

  def list(user, filter_params) do
    # ...
  end
end

# Keywords test module
defmodule Crawler.KeywordsTest do
  describe "list/1" do
    # ...
  end

  describe "list/2" do
    # ...
  end
end

  • Put the precondition(s) for a test before the expected value in the message for the test block. The precondition should always start with with, given or when, end with a comma, and be in the form of a sentence with proper grammar.

    test "updates user with valid params" do "" end
test "updates user given the actor has admin permissions" do "" end
test "renders error when user is not authenticated" do "" end

    test "with valid params, updates user" do "" end
test "given the actor has admin permissions, updates user" do "" end
test "when user is not authenticated, renders error" do "" end

  • Put the expression being tested to the left of the operator, and the expected value to the right.

    assert "elixir" == String.downcase("ELIXIR")

    assert String.downcase("ELIXIR") == "elixir"

  • Prefer to use == over pattern matching for assertions to ensure an exact match thus avoiding false positives with partial matches.

    test "returns keyword" do
  # ...

  assert %{type: "keyword", title: "rocket"} = created_keyword
end

    test "returns keyword" do
  # ...

  assert created_keyword == %{type: "keyword", title: "rocket"}
end

    However, use pattern matching for assertions when a partial match makes sense. For instance:

    • Making an assertion on a part of a string
      # Validating if the generated uid prefix with "keyword"
  assert "keyword" <> _ = created_keyword.uid
    • Validating the order of the items in a list
      # Validating if keywords are returned in alphabetical order
  assert [%{title: "apple"}, %{title: "rocket"}] = keywords

  • For asserting a nil or false value, prefer to use assert/1 with == over refute/1 to ensure exact value of the expression since refute/1 expect for nil and false.

    refute false_value
refute nil_value

    assert false_value == false
assert nil_value == nil

Fixtures

Factories

Use ExMachina to create a factory module for generating data. Prefer to define one factory module per one schema.

  • Defines a factory module. 
  defmodule Crawler.Keyword do
    # Keyword Schema
  end

  defmodule Crawler.KeywordFactory do
    defmacro __using__(_opts) do
      quote do
        def keyword_factory do
          %Crawler.Keyword{
            # Keyword attributes
          }
        end
      end
    end
  end
  • Add each factory to the main factory module. 
  defmodule Crawler.Factory do
    use ExMachina.Ecto, repo: Crawler.Repo

    use Crawler.KeywordFactory
    use Crawler.UserFactory
  end

Stubbing Network Requests

  • Stub all the request responses when the test includes the external HTTP requests. Prefer to use ExVCR.

    describe "fetch_user_detail/1" do
  test "returns fetched user detail" do
    use_cassette "get_user_detail_success" do
      # Test codes

      assert Users.fetch_user_detail("user_uid") == user_detail
    end
  end
end

  • Put all the recorded request responses in the support/fixtures directory (for the ExVCR called “vcr_cassette”)

    test
├── support
│   ├── fixtures
│   │   ├── vcr_cassettes
│   │   │   └── get_user_detail_success.json

Module Mocking

  • Prefer to use Mimic#expect over Mimic#stub to make sure the function is called.

    defmodule ProjectName.CalculatorTest do
  describe "age/1" do
    test "returns age given a specific naive datetime" do
      birth_date = ~N[1990-10-15 08:00:00.000000]

      expect(NaiveDateTime, :utc_now, fn -> ~N[2021-01-15 08:00:00.000000] end)

      assert Calculator.age(birth_date) == 30

      verify!()
    end
  end
end

The above code example is to test age calculation from the given birth date with the current date. It is a good idea to mock the current DateTime to avoid unexpected behavior. For example, the calculated age can be different based on the current date.

Phoenix Testing

When testing a Phoenix controller and isolate the controller functionality, split the testing into two parts, a controller test and a request test.

lib
├── crawler_web
│   ├── controllers
│   │   └── keyword_controller.ex
test
├── crawler_web
│   ├── controllers
│   │   └── keyword_controller_test.exs
│   ├── requests
│   │   └── keyword_request_test.exs

Controller test

  • Test only controller functionality without any plug included. The Phoenix controller provides the call/2 function to call the controller directly.

    defmodule CrawlerWeb.KeywordControllerTest do
  describe "index/2" do
    test "renders keywords for the given user", %{conn: conn} do
      # Test codes

      conn = CrawlerWeb.KeywordController.call(conn, :index)

      assert html_response(conn, 200) =~ "#{keyword.title}"
    end
  end
end

    When making an assertion, it should validate a controller response, an error handling, or result from a database operation. Below are examples to illustrate the main ideas:

    • Assertion when testing a keyword creation, the response contains the keyword title, and it persists the valid keyword.

      assert html_response(conn, 201) =~ "#{keyword_title}"

[keyword_in_db] = Repo.all(Keyword)
assert keyword_in_db == keyword_title

    • When testing a JSON response, assert on the complete controller response.

      assert json_response(conn, 404) == %{
              "code" => "not_found",
              "details" => %{},
              "message" => "Keyword not found",
              "object" => "error",
              "type" => "keyword_not_found"
            }

  • Use action name with arity as a message of a describe/2 block

    defmodule CrawlerWeb.KeywordControllerTest do
  describe "index/2" do
    # ...
  end

  describe "show/2" do
    # ...
  end
end

Request test

  • Request test covers the controller behavior with plugs included by making an HTTP request, so the request will go through all the plugs to the tested controller. The Phoenix.ConnTest module provides a helper function to make an HTTP request to a specific route (e.g. get/3, post/3). In the assertion, validating response status is sufficient as most of the controller functionalities are tested in the controller test.

    defmodule CrawlerWeb.KeywordRequestTest do
  describe "get /keyword" do
    test "returns 200 status if called with valid access token", %{conn: conn} do
      # Test codes

      conn =
        conn
        |> Conn.put_req_header("authorization", "bearer: " <> valid_token)
        |> get(Routes.keyword_path(conn, :index))

      assert conn.status == 200
    end

    test "returns 401 status if called with invalid access token", %{conn, conn} do
      # Test codes

      conn =
        conn
        |> Conn.put_req_header("authorization", "bearer: " <> invalid_token)
        |> get(Routes.keyword_path(conn, :index))

      assert conn.status == 401
    end
  end
end

  • Use an HTTP method with the route as a message of a describe/2 block

    describe "get /keywords" do
  # ...
end

describe "get /keywords/:id" do
  # ...
end

describe "post /keywords" do
  # ...
end