Observe Your Processes – How to Gain Insight in Process Data Using OpenAPM Tools

How high is the Offer-Acceptance-Rate of my job offers? How high is the rejection rate? How long does it take for an applicant to have an interview scheduled?

These are questions you may have asked yourself while trying to optimize your recruitment process. Unfortunately, the software you might use for your recruitment process may not provide these insights. Or maybe you have more branch-specific requirements?

Having these insights into your processes is the key to optimize the same processes – and not having the ability to gain these insights can be very frustrating. But what do you do when you face this lack of information provided by your tool of choice?

As it is with any other branch, the success of an IT company stands and falls with it’s workforce, so the application process naturally gains a central meaning. As it is in not quite any branch, we also have the tooling and the knowledge to change this lack of insights! Since our HR-Team did face this very problem I used some OpenAPM tools to create the process insights they need. In this blog post I will show you how it is done.

Status Quo

Jira is used as the central tool for managing the workflow of the recruitment process. Every job application is a ticket. Every status change of these tickets marks a new step in our recruitment process.

From here on we have one fairly simple goal: track these transitions and use the data to gain insights into how the workflow performs!

Collect Process Data with inspectIT Ocelot

In order to obtain this data I used the inspectIT Ocelot Agent. If you don’t know what this cute ocelot is about, you should definitely check it out! But to give a short summary: It is a java agent which uses byte-code manipulation to gain insights into the performance metrics of applications.

I have configured the inspectIT Agent to log so called events. These events are basically data-objects which you can enrich with data from a method’s context. This renders the question: which methods are we going to use?

I basically need to tell the agent to instrument methods connected to the lifecycle of a ticket in Jira. To do so I had to dig a bit through the code of Jira. I found the following three methods which I used as scopes for the configuration of the agent:

• setStatus
• create
• updateIssueWithComment

To gain information from the context of these methods, I declared one scope per

method:

Having these scopes defined I can now easily define actions to read data from a methods context. Here for example I will retrieve the value of a custom field in Jira:

The code may appear quite complex on first sight but it boils down to reading values and null checks.

Those of you who are familiar with the configuration of the inspectIT Agent may know that having scopes and actions defined, there is one thing left to do: rules.

In this rule for example, whenever create is triggered, we can read the title of the job offer with the help the a_get_customfield_value_from_issue action and pack it to an event:

Note that instead of MY_CUSTOM_FIELD_KEY the internal field of Jira needs to be added.

Events logged by the above code now always carry the id created. Additionally, the time when the event was logged is always added per default. Of course also our attribute location will be added to the data.

Having our rule defined, we only need to tell inspectIT where to send the data to. In this case we will use Logstash. You will find out why in the next paragraph!

Preprocessing Data with Logstash

To make Logstash accessible to the inspectIT agent, I need to tell it to provide a http-endpoint:

My event itself is quite boring as it lacks some data. In the case of Jira for example, I was only able to retrieve the location whenever a ticket was created or edited but not when its status changed. Thus, whenever a status-change is executed we need to find the creation-event of the corresponding ticket and copy some data to our new event:

As you can see, Logstash retrieves the timestamp of the preceding event, as well as it’s event-name, the weekday month and year it was logged.

With the timestamp of the preceding event, I can now calculate how long it took the process to transition from the preceding event to my current event:

The value will be saved in the field transition-duration-ms.

Knowing how much you can do with Logstash there are plenty of possibilities. However, to keep it short I will leave it at these examples and finally write the process data to an Elasticsearch instance:

Visualize Data with Grafana

Having only the data itself is quite boring so I set up a dashboard using Grafana. I used a drilldown approach with one main dashboard:

(no worries, all data you see here is auto-generated for testing purposes!)

As you hopefully notice, the main dashboard gives away some superficial data about how the process is currently performing. You can use links in each panel to gain more specific insights into each figure. For example, clicking on a link in the New Today panel, we can have a closer look on how the new applications are made up:

From here on you have endless possibility to display the data in a way that fits your needs.

Want to have a live process model?

Why don’t we use the Service Dependency Panel to display your process as a graph? We can simply create this since we always save the predecessor of an event! Each of the bubbles represents one transition.

(Of course, in the production environment there are no average response times in the milliseconds!)

Active Probing: How To Sleep Better With Your Tools in Production

Although I was able to test inspectIT with Jira in a test-environment, I still wanted to make sure that if an error would occur in production I would immediately learn of it. I think we can all agree that one sleeps better with the knowledge that you can learn about errors as soon as they occur. But how does one find new errors without reading the systems logs all day ?

The magic word is alerting.

In order to always be aware of possible errors in the system, I made use of something my colleagues already implemented for our production Jira: All logs are send to an Elasticsearch instance via Filebeat.

Since the data is already there only the alerting needs to be configured in the Elastic Stack Management. The decision fell to just throw an alert whenever a log containing “error” or “exception” occurs. Each of these possibilities is covered by one alert:

These alerts now check every 15 minutes if an error or an exception occurred. If this is the case, a log is written into another Elasticsearch index. This way only the index needs to be checked to make sure that Jira still runs as smoothly as it should.

Wrap Up

So let’s wrap up what I have done to achieve insights in our Jira-process:

I have configured inspectIT to send events to Logstash whenever a method in the lifecycle of a ticket is triggered. Logstash enriches these events with data of events that occurred beforehand and sends them to an Elasticsearch instance. This Elasticsearch instance serves as our data storage. From here on I was able to visualize the data via a Grafana dashboard. In order to spot errors linked to bytecode-manipulation I have also set up an alerting based on the logs of the Jira server.

Or since pictures say more than words – I did this:

A little word from our end user

To not only give you insights on how this tool looks from the technical side of things, my colleagues from our hr department were so nice to give me a little quote on how they use the tool:

We use the dashboard as a reporting tool for our application process. It allows us to track all processes from receiving an application to creating a contract. This allows us to always know the exact values of the duration of one application cycle. Furthermore, we can supply our departments with real-time reports about application channels, applications per job offering and/or the departments. We appreciate the tool very much since it captures and evaluates all information needed, thus we can access this information very easily. Additionally, the display of the data can be customized easily and the visualization is overall easy to use.

Conclusion

In this blog post I gave you a brief insight into how you can use the inspectIT Ocelot agent in combination with the ELK-Stack to monitor a process in Jira. This method can be applied to any other java application you have the possibility to add the agent to. I hope I was able to give you a better glimpse of how you can use the bytecode-manipulation of inspectIT to gain more insights into your application during runtime! What are your thoughts on this merge of APM-Topics and BPM-Topics? Feel free to express your views in the comments and of course ask questions if there are any! Or maybe you want to gain more insights in your processes (managed with or without Jira!) and need help to set up your own solution? Contact us!