Azure DocumentDB integration with Node.js using TypeScript and repository pattern

In my current project I work on a microservice built up on Node.js and running on Microsoft Azure. The service uses the Azure Cosmos DB DocumentDB service in order to store user data and much more. In contrast to the tutorial and samples of Azure we want to take advantage of TypeScript and its cool and helpful features like for example static typing, strict null-checks and generics/polymorphism. Therefore I developed a repository layer that uses the document client of the Azure documentdb in order to expose basic CRUD operations that can be reused every time a new collection is needed.

This is the first post of a blog series regarding ‘Microservices with Node.js, TypeScript and Microsoft Azure’. In the following months we will keep you up with different parts of this topic (Second Post).

Before we get started, I want to point out, that I will not go deeply into detail about ‘Why using TypeScript instead of JavaScript?’. There are plenty of articles and blog posts out there already, that point out, using TypeScript can make a big difference in so many facets compared to JavaScript. I personally think TypeScript is a good fit for developing enterprise applications, because of the high quality standards and moreover is really promising, given the fact that a great plenty of developers is driving it forward.

Concept / Motivation

Now, let’s get started. As I develop an enterprise application I need to build consistent and maintainable code. Therefore I try to use best practices and well-known programming patterns wherever it makes sense (not wherever I can ;-)). In this case I decided to use the repository abstraction layer. This pattern was introduced or mentioned first by Martin Fowler et al. in order to minimize redundant query logic and is by the way also well known by Spring Boot and Java developers as ‘Data Repositories’. This concept basically…

Mediates between the domain and data mapping layers using a collection-like interface for accessing domain objects.

The motivation behind it is to have some kind of encapsulation around the objects stored in the database and the operations that you use to create, read, update or delete (CRUD) them. This results in a more object-oriented layer of your persistent data.

This being said, let’s move on to the actual implementation and my approach of integrating Node.js with Azure CosmosDB DocumentDB using TypeScript and the repository pattern.

Implementation of a CRUD repository

In order to simplify the operations on our entities, we will build a basic CRUD repository that provides the necessary logic to initialize and setup a repository and provides the basic operations on it. As the foundation we use the documentdb package. In the following sections I will discuss important components and features in detail.

I use an abstract class as well as generics, which are two of the cool TypeScript features, in order to be able to extend it later. In our example with an users repository, the abstract class BaseCRUDRepository that is shown in the following code block implements the interface IRepository that defines the expected methods and properties such that extending classes can work with it. Moreover it introduces a generic, T. T defines the entity type for being returned or accepted by the different operation methods, and therefore extends AbstractMeta from the @types/documentdb definition file.

export abstract class BaseCRUDRepository<T extends AbstractMeta, U> implements IRepository<T, U> {
    protected database: DatabaseMeta | undefined;
    protected collection: CollectionMeta | undefined;
    protected updateProcedure: ProcedureMeta | undefined;
    protected bulkDeleteProcedure: ProcedureMeta | undefined;
    protected docDbClient: DocumentClient;
    protected classType: new (...args: any[]) => T;
    private collectionId: string;

    constructor(collectionId: string, classType: new (...args: any[]) => T) {
        this.docDbClient = DocumentDbClient;
        this.collectionId = collectionId;
        this.classType = classType;
    }

    public async create(obj: T): Promise<T> {
        try {
            await this.evaluateInit();
            await validateOrReject(obj);
            if (obj.id === undefined) {
                throw new RepositoryError('The id of the given object is undefined: ' + JSON.stringify(obj));
            }
            const retrieved = await new Promise<RetrievedDocument>((resolve, reject) => {
                this.docDbClient.createDocument(this.collection!._self, obj, (err: QueryError, result: RetrievedDocument) => {
                    if (err) {
                        reject(new DbQueryError(err));
                    } else {
                        resolve(result);
                    }
                });
            });
            return await transformAndValidate(this.classType, retrieved);
        } catch (e) {
            await this.handleRepositoryErrors(e);
        }
        throw new UnexpectedDbError();
    }

    /// ... other CRUD operations ...

    protected async evaluateInit(): Promise<void> {
        if (this.database === undefined || this.collection === undefined || this.updateProcedure === undefined || this.bulkDeleteProcedure === undefined) {
            await this.initDatabaseCollectionAndStoredProcedures();
        }
    }

    private async initDatabaseCollectionAndStoredProcedures(): Promise<void> {
        try {
            this.database = await DatabaseUtils.getOrCreateDatabase(this.docDbClient, DATABASE_ID);
            this.collection = await DatabaseUtils.getOrCreateCollection(this.docDbClient, this.database._self, this.collectionId);
            this.updateProcedure = await DatabaseUtils.getOrCreateUpdateStoredProcedure(this.docDbClient, this.collection._self);
            this.bulkDeleteProcedure = await DatabaseUtils.getOrCreateBulkDeleteStoredProcedure(this.docDbClient, this.collection._self);
        } catch (e) {
            // Error in getting or creating database, collection or stored procedures.
            throw e;
        }
    }
}

export abstract class BaseCRUDRepository<T extends AbstractMeta, U> implements IRepository<T, U> {

protected database: DatabaseMeta | undefined;

protected collection: CollectionMeta | undefined;

protected updateProcedure: ProcedureMeta | undefined;

protected bulkDeleteProcedure: ProcedureMeta | undefined;

protected docDbClient: DocumentClient;

protected classType: new (...args: any[]) => T;

private collectionId: string;

constructor(collectionId: string, classType: new (...args: any[]) => T) {

this.docDbClient = DocumentDbClient;

this.collectionId = collectionId;

this.classType = classType;

}

public async create(obj: T): Promise<T> {

try {

await this.evaluateInit();

await validateOrReject(obj);

if (obj.id === undefined) {

throw new RepositoryError('The id of the given object is undefined: ' + JSON.stringify(obj));

}

const retrieved = await new Promise<RetrievedDocument>((resolve, reject) => {

this.docDbClient.createDocument(this.collection!._self, obj, (err: QueryError, result: RetrievedDocument) => {

if (err) {

reject(new DbQueryError(err));

} else {

resolve(result);

}

});

return await transformAndValidate(this.classType, retrieved);

} catch (e) {

await this.handleRepositoryErrors(e);

}

throw new UnexpectedDbError();

}

/// ... other CRUD operations ...

protected async evaluateInit(): Promise<void> {

if (this.database === undefined || this.collection === undefined || this.updateProcedure === undefined || this.bulkDeleteProcedure === undefined) {

await this.initDatabaseCollectionAndStoredProcedures();

}

private async initDatabaseCollectionAndStoredProcedures(): Promise<void> {

try {

this.database = await DatabaseUtils.getOrCreateDatabase(this.docDbClient, DATABASE_ID);

this.collection = await DatabaseUtils.getOrCreateCollection(this.docDbClient, this.database._self, this.collectionId);

this.updateProcedure = await DatabaseUtils.getOrCreateUpdateStoredProcedure(this.docDbClient, this.collection._self);

this.bulkDeleteProcedure = await DatabaseUtils.getOrCreateBulkDeleteStoredProcedure(this.docDbClient, this.collection._self);

} catch (e) {

// Error in getting or creating database, collection or stored procedures.

throw e;

}

The constructor is used to set the collection id and the class type for the transformation of results retrieved from document db into the correct class-object with transformAndValidate(). But the most important part of the BaseCRUDRepository is the initialization method evaluateInit(), that has to be part of each operation. It ensures that all components, the database, collection and necessary stored procedures are correctly initialized and available. Of course everything is asynchronous such that we are able to use it easily with Promises and async/await.

Transformation and validation

In the example above you see the asynchronous create function that wraps a Promise around the callback based CosmosDB call and some additional utilities to validate and transform the given input and output. In our actual project we use the underlying NoSQL database together with some other service. To ensure that the plain objects we read will not affect our service in a negative way, we use the class-transformer-validator library along with class-transformer and class-validator. It provides some easy way to define what should be validated before saving to the database and which transformation and validation routines should be run before the create method will actually return. For this it is only necessary to attach the required validators to the target class, in our example the UserEntity. The validateOrReject() function will then use those validation decorators and throw an error in case a validation constraint is violated. Furthermore, the transformAndValidate() function uses the validators in order to validate after it instantiates an object of the target class into which it transforms the object obtained from the database.

Transformation and validation is a great advantage to guarantee the stability of our service. Thus, undesired effects caused by unexpected object structures are a thing of the past.

Stored procedures

Another important aspect of our encapsulation through the repository pattern is the easy use of stored procedures. In CosmosDB, the atomicity and isolation (ACID) of transactions such as update or removeAll (“all or nothing”) can only be ensured by means of stored procedures. For this reason, we create the necessary stored procedures directly in the initialization phase and then use them via the update or removeAll method.

Example usage

Lets put the parts together in an example. In the following code section, we instantiate our UsersRepository by simply extending the BaseCRUDRepository and specifying the name and entity class in the super() constructor call. This sets the basic information required to create the associated collection and stored procedures within the evaluateInit() call. It also provides the class to be returned by findOne, for example.

class UsersRepository extends BaseCRUDRepository<IUserEntity> {
    constructor() {
        super('users', UserEntity);
    }
}
export default new UsersRepository();

class UsersRepository extends BaseCRUDRepository<IUserEntity> {

constructor() {

super('users', UserEntity);

}

export default new UsersRepository();

Afterwards, we can easily use the UsersRepository by importing the instance and performing the required operations. Just as shown in the example below.

import { UserEntity } from './persistence/entities/UserEntity';
import UsersRepository from './persistence/UsersRepository';

doSomeDatabaseThings().catch((e) => console.error(e));

async function doSomeDatabaseThings(): Promise<void> {
    user = new UserEntity();
    user.key = 'key';
    user.text = 'text';
    user.date = new Date();
    user = await UsersRepository.create(user);
    console.log(`Created the user (with auto generated id): ${user}`);
}

import { UserEntity } from './persistence/entities/UserEntity';

import UsersRepository from './persistence/UsersRepository';

doSomeDatabaseThings().catch((e) => console.error(e));

async function doSomeDatabaseThings(): Promise<void> {

user = new UserEntity();

user.key = 'key';

user.text = 'text';

user.date = new Date();

user = await UsersRepository.create(user);

console.log(`Created the user (with auto generated id): ${user}`);

}

In the github repository provided with this blog post you’ll find even more complex examples.

Conclusion

The use of the repository pattern, supplemented by transformation and validation, facilitates all the tasks involving access to the CosmosDB in our project. Particularly the encapsulation of things like stored procedures required for the CosmosDB eases the construction of a number of repositories. The way I have introduced allows you now to quickly create new repositories by simply extending the BaseCRUDRepository. In addition, this approach gives you the opportunity to extend the repositories with individual methods for querying collections. This is also shown in the github repository linked below.

…of course things need to be tested in a professional enterprise development environment. But this should be it for now and I will come to this in a later blog post of this series.

Github repository

For the full example see here:

https://github.com/cg-frontend-development/node-typescript-azure-documentdb