Kotlin Assertion Libraries - Kotest Assertions

— Introduction — AssertJ — Strikt — AssertK — KoTest Assertions — Atrium — Kluent — Conclusions —

Kotest

Intro

In this blog post we will be looking at version 4.6.0 of Kotest assertions. It is a part of the larger Kotest Framework that has been around since 2015. Its main draws are a large wealth of supported matchers and a novel syntax on account of it fully embracing Kotlin’s infix and extension function feature.

Installation

Kotest is its own testing framework, and also supports platforms outside the JVM like KotlinJS and Android. As such there is a downright overwhelming list of modules to pick from if you were to just search for „kotest“ on maven central. Luckily the documentation makes it clear which dependency is the right one for you. To use Kotest with your JVM project all you need is the core module:

Syntax

Kotest deviates to a great degree from the standards set by AssertJ (or even Hamcrest). First it discards the well known assertThat in favor of should. Then the various should matchers are not wrappers, but extension functions, so they too can be read as grammatically correct sentences in the form of x shouldBe y. For example if we wanted to run asserts on a String

the AssertJ version would look like this

while in Kotest things would look like that:

But that is not where the changes end. As already said, the matchers are also infix functions. As such we can pull things apart even more to the point that our code looks almost like prose:

There are two minor downsides to this approach. On the one hand vararg arguments don’t work with infix functions, so when you are comparing lists you will always need an additional listOf:

Another point is that Kotest’s interface is not fluent. However in my opinion this is not a problem since the infix version looks cleaner anyway. And thanks to Kotlin’s scope functions tests where you need to compare multiple fields are likely to look like this:

Apart from that Kotest’s syntax will mostly look familiar, with profiting off Kotlin’s lambdas being a minor selling point, e.g.

or

Error Messages

Kotest’s error messages are generally shorter than those AssertJ and usually not divided into multiple lines. For example a failed test for a string that was expected to be blank will look like this in AssertJ:

and like this in Kotest:

There are 2 issues with this approach. First is the lack of contextual information. Asking for a non-existent key in a map will print its full content in AssertJ:

Kotest on the other hand simply reports that the wanted key is missing:

So if you want to know what the map actually does contain you might have no choice but run grab the debugger and run the test another time.

The second problem is that error messages can quickly swell in size because they are dealing with large objects, or lists with many elements. For example a failure in comparing a list of OffsetDateTime objects would print each list element in its own line in AssertJ, so it’s easy to determine what went wrong:

The Kotest version puts everything onto a single line, so it’s not just difficult to read, it probably requires constant horizontal scrolling as well:

Fortunately this is not always the case, and Kotest does sometimes pull things apart for proper readability. For example asserting that all elements of a list are positive, and failing on half of them will list all successes and failures in their own lines:

In particular comparisons made with shouldBe will always put their inputs on individual lines:

There is also the option of using Kotest’s Clues feature, at least if you are willing to fully commit to Kotest and use it as your test framework in addition to its assertions library.

Setting up clues allows you to avoid potentially awkward and informative error messages by imbuing them with contextual information. For example if you have a failing assert that expect a value to not be null the error message will look like this:

The withClue scope function allows you to describe the assert you are making:

so that when the assert fails it will look like this:

Another option is to have an entire object serve as the clue:

will output

Supported Types and Diversity of Assertions

Kotest boasts a strong selection of matchers. Apart from the bare basics like Strings, Numbers, Lists, Maps and Exceptions, it also supports less commonly used types like URIs, KClasses and BigDecimals. Kotlin-exclusives like Channels and Pairs, and Java old-timers like Dates and Optionals are covered as well.

The list of individual asserts is solid as well, it’ll probably be a while until you actually find a gap you have to fill with a custom-made assert of your own.

There are also several additional modules offering support for e.g. Json, Arrow or Ktor.

Custom Asserts

To write a custom assertion for Kotest you need to create a function that returns an instance of the Matcher<T> interface. This interface has a single method you need to implement: fun test(value: T): MatcherResult. For example creating an assert that an Optional contains a specific value would look like this: (though technically this is unnecessary since Kotest natively includes a contains matcher for Optionals)

You can have both a positive and negative error message so that the new matcher can be used with both should and shouldNot assert:

giving you error messages like

and

In addition you can create a proper infix function for the matcher so that you don’t need the should and shouldNot go-betweens:

Quality of Documentation

Kotest’s Kdoc is somewhat unevenly distributed. Many of the more „obvious“ asserts (like shouldHaveSize) are not documented at all. However the Kdoc that does exist is quite thorough, containing both clear explanations, oftentimes good examples, and links to related asserts. So while it would have been nice for the docs to be this good consistently, in practice it is unlikely to be a problem.

Outside the IDE there is also Kotest’s online documentation serving as a thorough guide and reference to all of its features, like the list of all of its matchers, or its support for non-deterministic testing.

Active Development

At the time this blog is written Kotest shows a consistent and healthy number of pushed commits and accepted pull requests. In addition it is maintained by an organization with multiple members, so at least in theory it can boast a bus factor greater than one.

— Introduction — AssertJ — Strikt — AssertK — KoTest Assertions — Atrium — Kluent — Conclusions —