Kotlin Assertion Libraries - Introduction

Looking for a neat Kotlin assertion library? We got a blog series looking at several Kotlin Assertion Libraries for you! -> Introduction

— Introduction — AssertJ — Strikt — AssertK — KoTest Assertions — Atrium — Kluent — Conclusions —

Introduction

Kotlin places itself firmly as the most popular JVM Language after Java itself. With the recent major release of Version 1.4 an interest was sparked in our team how one can move “full Kotlin” from a testing perspective.
A major part of the quality of life in testing your code is the assertion tooling. With its type inference and infix and lambda notations Kotlin has the potential to provide us with clean, readable code, but can new assertion libraries live up to that promise?
This series of Blogposts shows you differences and quirks between five Kotlin-focused assertion libraries and my colleague Sebastian’s and my take on them. For a baseline we will start off with the fluid Java goodness of AssertJ!

What are we looking at?

Installation

How simple is it to install this library? Can we just skim through the readme and grab the first code block that looks like it’s a maven or gradle directive and be done with it? Or do we need a more complicated installation where we have to read paragraphs of documentation to understand how to mix and match the right collection of jars?

Syntax and error messages

The two major benefits for quality engineers in using assertion tools are the ability to write expressive, concise and (of course) correct test code and getting meaningful error messages in case the tests detect an error. Syntax should be as terse as possible and as verbose as necessary to allow us to clearly express our expectations and error messages should immediately make it clear what went wrong, where it happened and what the difference is between our expectation and the actual result, without us having to first parse an overly long explanation.

Therefore we will shine a light on the idiomatic style of the corresponding library.

Included assertion types

Who wants to extend the library themselves for every object type there is, especially for basic builtin types like Dates or BigDecimals? We don’t! That’s why the breadth and depth of provided assertion types matters. We not only want to write our test assertions using basic equality comparisons with the occasional negation or assertion on list sizes thrown in – we want to write assertions as close as possible to the actual state we want to verify, without having to tediously map or extract our result data. We want string1 equalsIgnoreCase string2 and not string1.toLowerCase() equals string2.toLowerCase(). We want list containsExactlyInAnyOrder x,y,z and not list contains x && list contains y && list contains z && list.size equals 3. And we want this depth of expression not just for the most common of types, but also for the breadth of all the others we may encounter in our everyday work, even if it’s just rarely – from Ints, Lists, Maps and Strings to BigDecimals, Futures, Classes, Exceptions and AtomicIntegers.

Writing your own assertion

We may hope to have an assertion included for every type. But the reality is that most of the time we create our own complex types for our complex domains. Our assertion tool needs to be easily extendable for those situations, be it for small asserts that only fill a small gap, or large additions that express the assertion of complex business rules.

Java interop and nullable types

Kotlin provides a new way to ensure null-safety. However we rarely have the luxury of working on a truly 100%-Kotlin code base. Kotlin will oftentimes be used as part of an existing java project. But even if it isn’t and you are lucky enough to work on a greenfield project, the bulk of the JVM’s library ecosystem is still written in pure Java. And using java libraries means dealing with the “?” nullability marker and the “!” platform type. Dread it, run from it, nullability arrives all the same. Our assertion library should provide means to handle such occurrences.

Active Development

A library should be under active development. We are not looking for daily commits or a minimum number of added lines of code per month, but we all can agree that we’d rather not rely on a library where the issues and PRs pile up without answer and the last activity was a typo fix back in 2015. On the other hand, too much churn is probably counter-productive and an indicator of immaturity, especially when it produces frequent breaking changes. So what we are looking for is a normal pace of development – features should be added, bugs should be fixed, issues should be answered and pull requests merged in a reasonable time frame. Additionally a library should preferably not be just a single person’s private side project, but instead be backed by a team or even a company, and have a community of users – a single developer losing interest should not mean a stop to all development and the death of the project.

Quality of documentation

Finding what we need should usually be as simple as typing a fitting word and letting auto-completion do the rest (I want to strictly compare lists, so it’s probably something like shouldContainExactly). For the few cases when that is not enough we should be able to turn towards solid documentation, both for the javadoc in our IDEs and comprehensive documentation hosted online, to look up feature summaries, tutorials or known edge cases and limitations.