Log4j2 and Clojure quick setup

February 25, 2020

As we all know, java logging is a mess and I believe most developers are wondering how output to terminal or file, can be a rocket science?

Those who want to save some sanity and keep things simple, I'm sure are already are using Timbre.

But, my use case was different - the application had to log as much as possible, but that had to be fast as possible and provide plenty of configurable options for the end user, without touching source code.

From these tests 3 years ago, log4j2 seems like ultimate solution, especially if properly configured (check Recommendation section in given link).

Setup

Without further ado, let's see how to get this up and running. I'll assume Leiningen is used.

In project.clj add this:

 :dependencies [...
                 [clojure.tools.logging "0.6.0"]
                 [org.apache.logging.log4j/log4j-api "2.13.0"]
                 [org.apache.logging.log4j/log4j-core "2.13.0"]
                 [org.apache.logging.log4j/log4j-jcl "2.13.0"]
                ...]

log4j-api and log4j-core are obligatory. log4j-jcl is router for Apache Commons Logging and is a good thing to have, especially because clojure.tools.logging order of detection will pick up Apache Commons library before Log4j, if found in classpath. Many libraries depends on Apache Commons and you can end up scratching your head, wondering why logger isn't working properly.

If you don't want log4j-jcl as dependency, this behaviour can be mitigated by setting clojure.tools.logging.factory property at JVM startup. In Leiningen project.clj this line will do the work:

:jvm-opts ["-Dclojure.tools.logging.factory=clojure.tools.logging.impl/log4j2-factory"]

log4j2.xml configuration

Log4j2 has tons of configuration options, but I wanted this behavior:

  • All logs should go to folder logs by default and the folder must be created if is not present.
  • User can override logs folder with logPath property.
  • Application logs (debug, info, error) goes to logs/application.log.
  • Errors and exceptions goes to logs/error.log.
  • Both files must be rotated after size reaches 100 MB. After they are rotated, older files must be gzipped.
  • After 5 rotations, delete old files. This will assure disk is not filled up with old log archives.
  • Application logs everything to console as well.

Here is configuration that will do above. Put it in resources/log4j2.xml file so Log4j can find it, but also so it can be shipped with jar or uberjar.

<?xml version="1.0" encoding="UTF-8"?>
<Configuration>
  <Appenders>
    <!-- console output -->
    <Console name="Console" target="SYSTEM_OUT">
      <PatternLayout pattern="%d %p %m%n" />
    </Console>

    <!-- default output -->
    <RollingFile
        name="RollingFile"
        bufferedIO="true"
        fileName="${sys:logPath:-logs}/application.log"
        filePattern="${sys:logPath:-logs}/application.%i.log.gz">
      <PatternLayout pattern="%d %p %m%n" />
      <Policies>
        <SizeBasedTriggeringPolicy size="100MB" />
      </Policies>
      <DefaultRolloverStrategy max="5" />
    </RollingFile>
    
    <!-- errors output -->
    <RollingFile
        name="RollingFileErrors"
        bufferedIO="true"
        fileName="${sys:logPath:-logs}/error.log"
        filePattern="${sys:logPath:-logs}/error.%i.log.gz">
      <PatternLayout pattern="%d %p %m%n" />
      <Policies>
        <SizeBasedTriggeringPolicy size="100MB" />
      </Policies>
      <DefaultRolloverStrategy max="5" />
    </RollingFile>
  </Appenders>

  <Loggers>
    <Root level="all" includeLocation="false">
      <AppenderRef ref="RollingFile" level="DEBUG"/>
      <AppenderRef ref="RollingFileErrors" level="ERROR"/>
      <AppenderRef ref="Console" />
    </Root>
  </Loggers>
</Configuration>

Notice that I'm using RollingFile appender for taking care of rotating logs and keeping their size to max 100 MB. Be aware that it will add overhead; for ultimate performance, File appender can be used with asynchronous logger and leave to external services like logrotate to perform file rotations. Or, use Console appender and let init service like systemd route all logs to syslog or whatever system logging facility is installed.

In my case, I could not rely that logrotate will be present on system where application is run (macOS, Windows, Linux not properly configured,...).

Nice thing about Log4j is ability to override provided log4j2.xml with custom configuration, using log4j.configurationFile property, like:

java -Dlog4j.configurationFile=custom.xml -jar app.jar

Log2j has excellent performance page explaining what to use when and their appenders page will give enough details for every appender.

Enjoy!