25. July 2017

4 min

Smooth cloud-migration: Sharing the database

If only a soft migration instead of a big bang migration of an application to the cloud possible, the database must be shared between the old and the new application. Here is how we solved this in our project.

The situation before

We have a JEE based application on a portal server using JSF to deliver content to the user The relational database is accessed using JPA/Hibernate. The database schema is evolved executing flyway scripts during the deployment of the application. This application is released every 3 month.

The goal

The business logic of the application should extracted to a REST-Service running in a cloud, so that the business logic can be accessed both by the portal application (JSF) and some new applications developed by other teams. The deployment of the REST-Service should be as independent from the release cycle of the portal application is it could be, to support the needs of the new applications. The migration of the business logic can’t be done in one big bang step, because it would stop the development of new features for month. Therefore a smoth migration, where the old portal application and the new REST-Service uses the same database.

The problem

Both applications must use exactly the same database schema and the database schema can only be changed every three month when the portal application is released.

The new REST-Service is a Spring Boot application also using Hibernate to access the database. The easy part is to copy the JPA-Enties from the portal application to the REST-Service. But after the copy is done both applications can change the entities independent. When an entity is changed in one application, it must be changed in the other application. Because of the different release cycles of the applications, we have to prevent, that the two applications can be released based on different database schemas.

Because of its long release cycle the portal application is the master of the database schema. If the database migration scripts dont implement breaking changes (e.g. deleting a table or changing a tables name or adding not null columns), there should be no problem to deploy the REST-Service after the portal application is deployed.

Our migration solution

But when deploying the REST-Service it must be sure, that the database schema fits to JPA-Entities.

Hibernate Validation on startup

One solution is to validate the schema during startup. In Spring boot this can be activated in the application.yml

spring:
  ...
  jpa:
    ..
    hibernate:
      <strong>ddl-auto: validate
</strong>

spring:

...

jpa:

hibernate:

<strong>ddl-auto: validate

</strong>

If the validation fails the application will be shut down. The advantage is that we are sure that schema fits to the JPA-Entities. The only problem with this solution is, that it costs time during the startup, depending on the count of the JPA-Entities.

Hibernate Validation by Integration-Test

The schema validation can also be implemented by an integration-test, which is executed during the build. If the schema does not match the test will fail. Unfortunately hibernate’s SchemaValidator ignores SQLExceptions merely logging their occurrence. This would for example happen when there is no suitable database driver found on the class path – no validation takes place, but the test passes anyway. Therefore the test also opens and immediately closes a transaction to verify that a connection to the database can be successfully established.

public class SchemaValidationTest {

    /** Database connection url */
    private static final String DB_URL = "&lt;db-url&gt;";

    /** DB maintenance user */
    private static final String USERNAME = "&lt;db-user&gt;";

    /** DB password */
    private static final String PASSWORD = "&lt;db-pwd&gt;";

    /** Persistence unit name */
    private static final String PU_NAME = "&lt;name in persistence.xml&gt;";

    @Test
    public void validateHibernateObjectMappings() {
        Map&lt;String, Object&gt; configOverrides = new HashMap&lt;String, Object&gt;();
        configOverrides.put("hibernate.connection.url", DB_URL);
        configOverrides.put("hibernate.connection.username", USERNAME);
        configOverrides.put("hibernate.connection.password", PASSWORD);
        <strong>configOverrides.put("hibernate.hbm2ddl.auto", "validate");</strong>
        // default setting is a container managed transaction factory, however in this test there is no container
        configOverrides.put("hibernate.transaction.factory_class", "org.hibernate.transaction.JDBCTransactionFactory");

        Persistence.createEntityManagerFactory(PU_NAME, configOverrides)//
                .createEntityManager()//
                .unwrap(Session.class)//
                .beginTransaction()//
                .rollback();
    }

public class SchemaValidationTest {

/** Database connection url */

private static final String DB_URL = "<db-url>";

/** DB maintenance user */

private static final String USERNAME = "<db-user>";

/** DB password */

private static final String PASSWORD = "<db-pwd>";

/** Persistence unit name */

private static final String PU_NAME = "<name in persistence.xml>";

@Test

public void validateHibernateObjectMappings() {

Map<String, Object> configOverrides = new HashMap<String, Object>();

configOverrides.put("hibernate.connection.url", DB_URL);

configOverrides.put("hibernate.connection.username", USERNAME);

configOverrides.put("hibernate.connection.password", PASSWORD);

<strong>configOverrides.put("hibernate.hbm2ddl.auto", "validate");</strong>

// default setting is a container managed transaction factory, however in this test there is no container

configOverrides.put("hibernate.transaction.factory_class", "org.hibernate.transaction.JDBCTransactionFactory");

Persistence.createEntityManagerFactory(PU_NAME, configOverrides)//

.createEntityManager()//

.unwrap(Session.class)//

.beginTransaction()//

.rollback();

}

The advantage of this solution is, that server starts fast. The problem is, that the test is only executed building the application, not when deploying the application.

Check flyway version

Flyway has an extra table in the database containing the schema version. The application can check this version during the startup against a property in the application.yml. If the check fails the application will shutdown. The advantage of this solution is, that it is much more faster than the solution before. But if the error occurs it can be fixed quickly by the changing the application.yml, forgetting to change the JPA-Entities.

The application.yml:

database:
  schemaversion: "0080"

1 2	database: schemaversion: "0080"

The check:

@Slf4j
@Component
public class CheckDbSchemaListener implements ApplicationListener&lt;ApplicationReadyEvent&gt; {

    static final String VERSION_SQL = "select max(\"version\") from SCHEMA_HISTORY";

    @Autowired
    private EntityManager entityManager;

    @Autowired
    private ConfigurableApplicationContext appContext;

    @Value("${database.schemaversion}")
    private String schemaVersion;

    /** {@inheritDoc} */
    @Override
    public void onApplicationEvent(ApplicationReadyEvent event) {
        log.debug("Check DB schema expected version: " + schemaVersion);
        String actualVersion;
        try {
            Query query = entityManager.createNativeQuery(VERSION_SQL);
            actualVersion = (String) query.getSingleResult();
            log.info("The version of the DB schema is: " + actualVersion);
        } catch (Exception ex) {
            log.error("Cannot check DB Schema. Shutdown application: " + ex.getMessage(), ex);
            appContext.close();
            return;
        }
        if (actualVersion.equals(schemaVersion)) {
            log.debug("DB schema checked successfully");
        } else {
            log.error("Expected DB Schema [{}], Actual [{}]. Shutdown application ", schemaVersion, actualVersion);
            appContext.close();
        }
    }
}

@Slf4j

@Component

public class CheckDbSchemaListener implements ApplicationListener<ApplicationReadyEvent> {

static final String VERSION_SQL = "select max(\"version\") from SCHEMA_HISTORY";

@Autowired

private EntityManager entityManager;

@Autowired

private ConfigurableApplicationContext appContext;

@Value("${database.schemaversion}")

private String schemaVersion;

/** {@inheritDoc} */

@Override

public void onApplicationEvent(ApplicationReadyEvent event) {

log.debug("Check DB schema expected version: " + schemaVersion);

String actualVersion;

try {

Query query = entityManager.createNativeQuery(VERSION_SQL);

actualVersion = (String) query.getSingleResult();

log.info("The version of the DB schema is: " + actualVersion);

} catch (Exception ex) {

log.error("Cannot check DB Schema. Shutdown application: " + ex.getMessage(), ex);

appContext.close();

return;

}

if (actualVersion.equals(schemaVersion)) {

log.debug("DB schema checked successfully");

} else {

log.error("Expected DB Schema [{}], Actual [{}]. Shutdown application ", schemaVersion, actualVersion);

appContext.close();

}

What we did

For our REST-Service we implemented the validation on startup and check the flyway version, because the startup is fast enough and we want to be sure, that the schema fits to the JPA-Entities.

For the portal application we used the Validation by Integration-Test, because the startup time for the portal server is very long without validating the schema.