Production issue comes screaming at you at 2am. Red-eyed you open your laptop and check the server logs. You see only minimal logging. After checking other areas such as database, network, machine health, etc., you wish you could turn on more debug logs. Turns out your development team did not provide you a feature to change log levels on the fly. Only way you can do that is to restart the application servers. Ok developers don’t let that happen to you or your ops team.

Lets look at a few ways to change log levels. First the simple, but not so elegant approach. Don’t get me wrong (about the elegance statement) this approach works.

Log4j 1.x API
Often applications will have custom log4j properties files. Here we define the appenders and the layouts for the appenders. Somewhere in the java code we have to initialize log4j and point it to this properties file. We can use the following API call to configure and apply the dynamic update.

• Pass it the path to the custom log4j.properties and a delay in milliseconds. Log4j will periodically check the file for changes (after passage of the configured delay time).

Log4j 2 API + Spring Boot
Log4j 2.x provides another means to change log4j properties.

• Set monitorInterval to the number of seconds after which log4j should refresh the properties.
• If you are using Spring Boot edit your application.yml and add logging.config to point to the external location of your log4j2 xml configuration file. In production you will need an NFS mount to ensure all servers see the same file, or you will have to implement some mechanism to push the file to each individual server. On AWS you can use EFS to create a common shared folder.

Spring Helpers
Spring provides ready-to-use classes to do this job. You can use the support class org.springframework.web.util.Log4jWebConfigurer.
Provide it values for log4jConfigLocation, log4jRefreshInterval. For the path you can pass either one that is relative to your web application (this means you need to deploy in expanded WAR form) or provide an absolute path. I prefer the latter; that way I can keep my WAR file warred and not expanded. This approach assumes an expanded WAR file deployment which may not be something you always want. Log4j runs a watcher thread to track changes to the configuration file. This thread does not terminate until JVM shutdown.

There is also a web application listener class org.springframework.web.util.Log4jConfigListener that you can use in the web.xml file. The actual implementation of the Spring class Log4jWebConfigurer does the call to either:

Log4j spawns a separate thread to watch the file. Make sure your application has a shutdown hook where you can org.apache.log4j.LogManager.shutdown() to shut down log4j cleanly. The thread unfortunately does not die if your application is undeployed. Thats the only downside of using Log4j configureAndWatch API. In most cases thats not a big deal so I think its fine.

JMX Approach
JMX involves some leg work. This example here was run on JBoss 4.0.5. Lets look at a simple class that will change the log level.

• Given a logger name and a level to change to this code will do just that. The code needs some error handling and can be cleaned up a little. But this works for what I am showing.
• To change the log level we get the logger for the specified loggerName and change to the new level.

My application uses Spring so the rest of the configuration is Spring related. Now we need to register this bean as an MBean into the MBeanServer running inside JBoss. Here is the Spring configuration.

• In Spring we use the MBeanExporter to register your MBeans with the containers running MBean Server.
• I provide MBeanExporter with references to beans that I want to expose via JMX. Finally my management bean is Log4jLevelChanger is registered with Spring.

Thats it. With this configuration your bean will get registered into JBoss’s MBean server. By default Spring will publish all public methods on the bean via JMX. If you need more control on what methods get published then refer to Spring documentation. I will probably cover that topic in a separate blog since I had to do all of that when i set up JMX for a project using Weblogic 8.1. With Weblogic 8.1 things are unfortunately not that straight forward as above. Thats for another day another blog.

One thing to note here is that the parameter names are p1 (for loggerName) and p2 for (level). This is because I have not provided any meta data about the parameters. When I do my blog on using JMX+Spring+CommonsAttributes under Weblogic 8.1, you will see how this can be resolved. BTW for jdk 1.4 based Spring projects you must use commons attributes tags provided by Spring to register and describe your beans as JMX beans. The initial minor learning curve will save you tons of time later.