Simplifying Code Testing with Elixir Doctests

Elixir Doctests are a powerful tool for writing and testing your code. They allow you to write tests directly in your documentation, making it easy to keep your tests and documentation in sync. This not only ensures that your documentation is accurate but also provides practical examples of how your functions are used.

Benefits of Elixir Doctests

Using Doctests has several advantages:

Synchronizing Documentation and Tests: With Doctests, your documentation serves a dual purpose by providing usage examples and verifying correctness. This ensures that any changes in your code are immediately reflected in the documentation, keeping both up-to-date.
Ease of Use: Writing and running Doctests is straightforward. You simply include test cases within your module documentation, and Elixir takes care of the rest. This reduces the overhead associated with maintaining separate test files.
Quick Feedback Loop: Doctests allow developers to test their code as they write it. By running the examples embedded in the documentation, you can quickly verify that your functions perform as expected.

To illustrate, consider the setup and usage of Doctests:

First, enable the :doctest compiler in your mix.exs file by adding it to the list of compilers:

def project do
  [
    compilers: [:phoenix, :gettext, :doctest]
    # other configurations...
  ]
end

Once enabled, you can start embedding tests in your documentation. For instance, if you have a module named MyModule with a function double/1, a typical Doctest would look like this:

defmodule MyModule do
  @moduledoc """
  This is the documentation for MyModule.

  ## Examples

      iex> MyModule.double(2)
      4

      iex> MyModule.double(-2)
      -4

  """
  def double(n), do: n * 2
end

In this example, the documentation includes two tests for the MyModule.double/1 function. The tests mimic an iex session, making it clear how the function should behave with different inputs.

In the next chapters, we will delve deeper into writing, running, and best practices for using Elixir Doctests effectively.

Writing Doctests

Writing Doctests in Elixir involves adding test cases directly into your module documentation. These test cases are structured as code examples that demonstrate how your functions should be used and what results they should produce. The typical format of a Doctest includes an iex prompt, the input expression, and the expected output.

Consider the overall structure:

Module Documentation: Use the @moduledoc attribute to describe the module and its functions.
Usage Examples: Within the module documentation, include usage examples wrapped in triple quotes (""") and marked with the iex> prompt.
Expected Output: Each example should include the expected output directly below the input expression.

For instance:

defmodule MyModule do
  @moduledoc """
  This module provides basic arithmetic functions.

  ## Examples

      iex> MyModule.add(1, 2)
      3

      iex> MyModule.subtract(5, 3)
      2

  """

  def add(a, b), do: a + b
  def subtract(a, b), do: a - b
end

This clear, concise format makes it easy for anyone reading the documentation to understand how the function should be used and what results to expect. Moreover, it ensures that your documentation remains synchronized with the actual behavior of your code, as any discrepancies will be caught when running the Doctests.

In the following chapters, we will cover how to run these Doctests and interpret the results, as well as some best practices to maximize their effectiveness.

Running Doctests

Using mix doctest Command

Once you have written your Doctests, running them is straightforward. Elixir provides the mix test command, which will automatically detect and execute the Doctests embedded in your documentation. Follow these steps:

Open Your Terminal: Navigate to the root directory of your Elixir project.
Run mix test: Execute the command mix test. This command runs all tests, including unit tests and Doctests.
Monitor Output: Elixir will compile your project and execute all tests, providing feedback in the terminal.

For a more focused test run, you can use the mix test command with specific flags to only run certain tests.

Interpreting Test Results

After running mix test, Elixir provides a summary of the test results. Here’s how to interpret them:

Passing Tests: Tests that pass will be listed first, often with a dot (.) representing each successful test. If all tests pass, you should see a message indicating this, usually along with the total number of tests run.

Example:

Finished in 0.1 seconds
3 doctests, 0 failures

Failing Tests: If any tests fail, they will be listed with detailed error messages. These messages include:
- Location: The file and line number where the test is defined.
- Assertion: The input expression and expected output.
- Actual Output: What the function actually returned.

Example:


1) doctest MyModule (1) (MyModule)
test/my_module_test.exs:3
Doctest failed
code: MyModule.double(2)
expected: 4

     actual: 5

This detailed feedback allows you to quickly identify discrepancies between your documentation and your code, making it easier to resolve issues.

Handling Failures

When a Doctest fails, follow these steps to address the issue:

Review the Error Message: Understand what caused the test to fail by examining the expected and actual outputs.
Update the Code or Documentation: Make necessary adjustments to either your function implementation or the documented examples to ensure they align.
Re-run Tests: Execute mix test again to confirm that the changes have resolved the issue and that all tests now pass.

By keeping an eye on the test results and responding promptly to failures, you can maintain high-quality, reliable documentation that accurately reflects your code’s behavior.

In the next chapter, we will explore best practices for using Doctests effectively and ensuring they complement your overall testing strategy.

Best Practices

One of the primary benefits of Elixir Doctests is the synchronization of your documentation and tests. Here are some best practices to ensure they remain in sync:

Regularly Update Documentation: Whenever you modify a function, promptly update its documentation and corresponding Doctests. This keeps both your documentation and tests relevant.
Comprehensive Examples: Include various scenarios in your Doctests, such as typical cases, boundary conditions, and edge cases (though Unit Tests are also a great way of testing edge cases). This ensures that all possible uses of the function are documented and tested.
Clear Explanations: Write clear and concise explanations for your examples. This helps others understand the purpose and functionality of your code, making collaboration and maintenance easier.

Supplemental Nature of Doctests

While Doctests are a valuable tool, they should not replace traditional unit tests. They work best in conjunction with more detailed testing frameworks:

Complex Logic: Use unit tests to thoroughly test complex logic, ensuring all possible paths are covered. Doctests are excellent for simple, illustrative cases but may not cover all edge cases.
Edge Cases and Error Handling: Unit tests are better suited for testing edge cases and error handling. These scenarios can be difficult to illustrate in documentation but are crucial for robust software.
Performance and Integration Testing: Utilize other testing strategies, such as performance and integration tests, to complement Doctests. This ensures your application performs well and integrates seamlessly with other systems.

To summarize, Elixir Doctests offer several advantages for your development process:

Synchronization of Documentation and Tests: By writing tests directly in your documentation, you ensure both are always up-to-date and accurate.
Ease of Writing and Running Tests: Doctests simplify the testing process by reducing the need for separate test files and making it easy to run all tests with a single command.
Quick Feedback Loop: Developers can quickly test their code as they write it, ensuring functionality is verified immediately and reducing the likelihood of errors.

Integration with Unit Tests

For maximum effectiveness, integrate Doctests with your unit testing strategy:

Complementary Testing: Use Doctests for simple and illustrative cases, while employing unit tests for more complex scenarios, edge cases, and comprehensive testing.
Holistic Approach: A combination of Doctests, unit tests, performance tests, and integration tests provides a comprehensive testing strategy, ensuring your code is robust, efficient, and reliable.

By leveraging the strengths of both Doctests and unit tests, you can maintain high-quality, well-documented code that is easy to understand, use, and maintain. This integrated approach helps you build reliable software that meets users’ needs and withstands the challenges of real-world applications.

Alptuğ Dingil

Alptuğ joined Inspired in 2022 as a software engineer. Besides his customer projects he's always looking for a new challenge. So lately he got engaged with Kubernetes and the configuration of a DIY cluster and got certified as a Google professional cloud architect.