How to add logging to your Dropwizard application with Sematext

We've been using Sematext as a logging provider since the start and we're really happy with it. They also provide log visualization using Kibana though their web interface. In this post we'll take a look at how to integrate it with your Dropwizard application.

What's also nice about Sematext is that they offer a pretty good deal on their free tier, so startups can use it to get started quickly and for free.

This is a summary of the technologies and products we will touch upon in this post:

• Dropwizard
• Sematext
• Maven
• Docker
• Kibana

What is Dropwizard?

Dropwizard is a really nice small framework for building Java-based web services. It is opinionated in the sense that it pulls in a number of libraries that works well for building Microservices, such as Jetty for HTTP, Jackson for JSON marshalling and Logback/SLF4J for logging. It does not however add too much of a framework to your application which means there's less magic than in a Spring application or a Java EE application.

We like how Dropwizard balances a good set of libraries without acting too much like a framework. It has been our default choice for building Java services for a number of years now.

Dropwizard's default logging system

Dropwizard provides a combination of human/machine readable format by default. It looks like this:

WARN  [2010-04-06 06:42:35,275] com.example.dw.Thing: Doing a thingERROR [2010-04-06 06:42:35,275] com.example.dw.Thing: This may get ugly.! java.lang.RuntimeException: oh noes!! at com.example.dw.Thing.run(Thing.java:16)!

To provide logs to external services such as Sematext we need to modify the output destination and format. Dropwizard supports extending its logging system by changing the underlying LoggingFactory in the Configuration class. We'll implement our own LogbackAutoConfigLoggingFactory that will add Logback's auto-config functionality to Dropwizard so that we can customize the Logback configuration so that it will produce a format that Sematext can interpret and forward the logs via Syslog.

Create a Sematext logging application

To get started you first need to register a new Sematext logging application. They offer a free tier so go to https://sematext.com, register and create a new application.

You will get an application ID that we will use in our logging configuration below.

Forwarding logs via Syslog

Create the following class in your project. When it is configured it will enable the auto-discovery of Logback configuration files in your resources folder of your project so you can override the Dropwizard defaults.

public class LogbackAutoConfigLoggingFactory implements LoggingFactory {  @JsonIgnore  private LoggerContext loggerContext;  @JsonIgnore  private final ContextInitializer contextInitializer;  public LogbackAutoConfigLoggingFactory() {    this.loggerContext = LoggingUtil.getLoggerContext();    this.contextInitializer = new ContextInitializer(loggerContext);  }  @Override  public void configure(MetricRegistry metricRegistry, String name) {    try {      contextInitializer.autoConfig();    } catch (JoranException e) {      throw new RuntimeException(e);    }  }  @Override  public void stop() {    loggerContext.stop();  }  @Override  public void reset() {    loggerContext.reset();  }}

Now, we need to override the getLoggingFactory() method in our Dropwizard configuration class to use our new class.

public class SematextDropwizardLoggingConfiguration extends Configuration {  @Override  public LoggingFactory getLoggingFactory() {    return new LogbackAutoConfigLoggingFactory();  }}

Next, add a logback.xml file to src/main/resources which contains the following. Note the {{TOKEN}} which is where you will add your Sematext application token:

<?xml version="1.0" encoding="UTF-8"?>
<configuration>

  <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
    <encoder>
      <pattern>%date %level %logger %thread - %message%n</pattern>
    </encoder>
  </appender>

  <appender name="SYSLOG" class="ch.qos.logback.classic.net.SyslogAppender">
    <syslogHost>logsene-syslog-receiver.eu.sematext.com</syslogHost>
    <port>514</port>
    <facility>LOCAL0</facility>
    <suffixPattern>{{TOKEN}}:@cee: {"thread":"%thread", "logger":"%logger",
      "message":"%message", "throwable":"%throwable"}
    </suffixPattern>
  </appender>

  <root level="info">
    <appender-ref ref="SYSLOG"></appender-ref>
    <appender-ref ref="STDOUT"></appender-ref>
  </root>

</configuration>

Note: We're using eu.sematext.com for the Europe region of Sematext above. You need to modify this in order to use a different region.

Testing the integration

You start your Dropwizard application by running:

java -Duser.timezone=UTC -jar target/sematext-dropwizard-logging-1.0-SNAPSHOT.jar server config.yml

Note: -Duser.timezone=UTC set the Java application to run with UTC as timezone so you will get your application logs in UTC in Sematext.

We now have fully functional logging for our Dropwizard application using Sematext. Log in to Sematext and look at your logs either through their log viewer or via Kibana

Troubleshooting

If you have problems finding your logs, double-check the following:

• Dependencies in your pom.xml
• Timestamps. Try not using time-based filtering in Kibana/Sematext and

Forwarding logs from Docker containers

If you are deploying your application as a Docker container in a Kubernetes cluster, Sematext provides an agent that is super simple to install in your as cluster. If you use the Docker agent for shipping your logs, it will forward logs that are appended to the console from your application. Instead of sending it to Syslog you will instead write the logs to the console directly and the agent will pick them up and forward them to Sematext.

• Sematext Docker Agent

Complete code example

A full project example for this setup can be found on our Github account:

• Dropwizard project with Sematext logging

Good luck with your logging!