Configuring Rolling Logging with Logback
November 1, 2016
Tags: #logging #logback #config
A common requirement in web applications is to support a rolling log system, so that log files are rolled over on a schedule and archived after a certain point. Grails® 3 uses Logback (considered the successor to log4j) as its logging library, and it's quite simple to configure a rolling appender using Logback's Groovy config format.
The Grails framework includes a default Logback configuration at grails-app/conf/logback.groovy
. By default (as of Grails 3.2.1), this file (which follows the standard Logback groovy config format) configures a single appender, an instance of ConsoleAppender
called STDOUT
, and conditionally (when in development mode) an instance of FileAppender
called FULL_STACKTRACE
. These may then be used by logger instances, which can target specific package names and log levels, and write to one or more appenders. You have access to the Grails Environment
, so you can configure different combinations of appenders for development and production.
The default logback.groovy
file in Grails 3.2.1 is shown below.
//grails-app/confg/logback.groovy
import grails.util.BuildSettings
import grails.util.Environment
// See https://logback.qos.ch/manual/groovy.html for details on configuration
appender('STDOUT', ConsoleAppender) {
encoder(PatternLayoutEncoder) {
pattern = "%level %logger - %msg%n"
}
}
def targetDir = BuildSettings.TARGET_DIR
if (Environment.isDevelopmentMode() && targetDir != null) {
appender("FULL_STACKTRACE", FileAppender) {
file = "${targetDir}/stacktrace.log"
append = true
encoder(PatternLayoutEncoder) {
pattern = "%level %logger - %msg%n"
}
}
logger("StackTrace", ERROR, ['FULL_STACKTRACE'], false)
root(ERROR, ['STDOUT', 'FULL_STACKTRACE'])
}
else {
root(ERROR, ['STDOUT'])
}
Let's add an instance of RollingFileAppender
for our production environment. Let's say we want to split out our log files by day, and keep 30 days worth of log files around (deleting any older ones). In addition, we don't have unlimited hard drive space, so we'll also set a file size cap so that the total disk space used by our logs never exceeds 2GB.
Here's our new appender:
//grails-app/confg/logback.groovy
import ch.qos.logback.core.rolling.RollingFileAppender
import ch.qos.logback.core.rolling.TimeBasedRollingPolicy
import ch.qos.logback.core.util.FileSize
def HOME_DIR = "."
appender("ROLLING", RollingFileAppender) {
encoder(PatternLayoutEncoder) {
pattern = "%level %logger - %msg%n"
}
rollingPolicy(TimeBasedRollingPolicy) {
fileNamePattern = "${HOME_DIR}/logs/myApp-%d{yyyy-MM-dd_HH-mm}.log"
maxHistory = 30
totalSizeCap = FileSize.valueOf("2GB")
}
}
An instance of RollingFileAppender
needs two "policies", a rollingPolicy
(to define how to perform the rollover), and a triggerPolicy
(which specifies when the rollover should occur). In this case, our rollingPolicy
is TimeBasedRollingPolicy
, which happens to implement the TriggeringPolicy
interface and therefore satisfies both policy requirements. TimeBasedRollingPolicy
is one of the most common rolling policies, and it will meet the majority of rolling log requirements.
TimeBasedRollingPolicy
gets both it's rolling behavior (creating a new log file with the current date/time in the file name) and it's triggering behavior (rollover will occur based on the specified timestamp pattern) from the fileNamePattern
property.
What's In a Name?
Maybe you'd rather not have the filenames of your log files contain the trigger interval. No worries,
RollingFileAppender
also supports afile
property which can override this behavior, so your log files can be rolled over monthly (for example) without actually containing the date string in their filenames. See the documentation forRollingFileAppender
for more details.
The trigger policy is the most interesting part here - it takes an approach that bases the rollover occurrence on how specific you define the timestamp in the fileNamePattern
. So if you specify down to the month, rollover will occur each month. Specify a pattern down to the day, and it will occur daily). It's easier to understand when you see it in action, so here's some example patterns taken from Logback's documentation:
fileNamePattern = "/myApp-log.%d{yyyy-MM}.log" //Rollover at the beginning of each month, file format: myApp-log.2016-11.log
fileNamePattern = "/myApp-log.%d{yyyy-ww}.log" //Rollover at the first day of each week. Note that the first day of the week depends on the locale.
fileNamePattern = "/myApp-log.%d{yyyy-MM-dd_HH}.log" //Rollover at the top of each hour.
Note that in the above examples we are configuring the timestamp in the filename of the log files. We can also use the timestamp to create a file directory structure, like this example:
fileNamePattern = "/logs/%d{yyyy/MM}/myApp.log" //Rollover at the beginning of each month.
//Each log file will be stored in a year/month directory, e.g: /logs/2016/11/myApp.log, /logs/2016/12/myApp.log, /logs/2017/01/myApp.log
Finally, adding a zip
or gz
file extension to our fileNamePattern
will apply the selected compression to the rolled-over log files:
fileNamePattern = "/myApp-log.%d{yyyy/MM}.gz" //Rollover at the beginning of each month, compress the rolled-over file with GZIP
Going back to our previous example, we're setting a couple more properties on our TimeBasedRollingPolicy
- maxHistory
and totalSizeCap
. These are pretty simple to understand; maxHistory
sets the upper limit on how many log files to preserve (when the max is reached the oldest file is deleted), and totalSizeCap
sets a cap on how much disk space our log files are allowed to use (again, when the cap is reached the oldest files are deleted). There are other useful options you can set here, see the TimeBasedRollingPolicy
documentation for a full list and explanation.
Warning!
While the Logback docs suggest that
totalSizeCap
can be specified as a plain String (i.e, "2GB"), I've found that it needs to be specified as aFileSize
to avoid casting exceptions - so make sure to useFileSize.valueOf("2GB")
to evaluate your total size. This also applies to themaxFileSize
property used in theSizeBasedRollingPolicy
andTimeAndSizeBasedRollingPolicy
.
There are several more rolling & triggering policies that are available, including SizeBasedTriggerPolicy
and SizeAndTimeBasedTriggeringPolicy
, and FixedWindowRollingPolicy
Finally, let's specify that we want our new RollingFileAppender
to be used in production mode only, while keeping the default ConsoleAppender
for development mode.
//grails-app/confg/logback.groovy
import grails.util.BuildSettings
import grails.util.Environment
import ch.qos.logback.core.rolling.RollingFileAppender
import ch.qos.logback.core.rolling.TimeBasedRollingPolicy
import ch.qos.logback.core.util.FileSize
def HOME_DIR = "."
// See https://logback.qos.ch/manual/groovy.html for details on configuration
appender('STDOUT', ConsoleAppender) {
encoder(PatternLayoutEncoder) {
pattern = "%level %logger - %msg%n"
}
}
appender("ROLLING", RollingFileAppender) {
encoder(PatternLayoutEncoder) {
pattern = "%level %logger - %msg%n"
}
rollingPolicy(TimeBasedRollingPolicy) {
fileNamePattern = "${HOME_DIR}/logs/myApp-%d{yyyy-MM-dd_HH-mm}.log"
maxHistory = 30
totalSizeCap = FileSize.valueOf("2GB")
}
}
def targetDir = BuildSettings.TARGET_DIR
if (Environment.isDevelopmentMode() && targetDir != null) {
appender("FULL_STACKTRACE", FileAppender) {
file = "${targetDir}/stacktrace.log"
append = true
encoder(PatternLayoutEncoder) {
pattern = "%level %logger - %msg%n"
}
}
logger("StackTrace", ERROR, ['FULL_STACKTRACE'], false)
root(ERROR, ['STDOUT', 'FULL_STACKTRACE'])
}
else {
root(ERROR, ['ROLLING'])
}
And with that, we have our rolling log system. Enjoy!
Resources
- Logback documentions: https://logback.qos.ch/documentation.html
- Logback Groovy config (from the official docs): https://logback.qos.ch/manual/groovy.html
- DZone - Logback Configuration Using Groovy: https://dzone.com/articles/logback-configuration-using-groovy