Monitoring Log4J messages on a browser via server push

Introduction

Most of the Java projects I’ve worked on have used Log4J as the logging framework. We normally change the severity level to ERROR on lo4j.xml before deploying our web application to a production system. But when something goes wrong, you may want to enable debugging for a short time and see the messages, and then turn the level back. There are a couple of ways you can change the severity level on the fly. You can either use a jsp file as explained here or a JMX solution as explained here or here

But we also wanted to take peek at the log data in real time so when we see an error being generated we can change the level back to ERROR and look at server’s log files for debugging. Something like doing a tail –f logfile.log from the server. In some places, developers are not given SSH access to server, and needs to contact the DBA for any log files. Even if the developer has access to it, SSH access may be limited to internal traffic, and if you want to fix an issue from home, you’re out of luck. A browser based solution would be ideal in these cases. So here we are discussing such a solution, which should be very easy to implement in your own project.

Implementation

• Place the liveLogger.jar on your WEB-INF/lib folder
• Add the following to your web.xml (before the closing web-app tag)

<context-param>
<param-name>LIVE_LOGGER_MAX_CLIENTS</param-name>
<param-value>10</param-value>
</context-param>
<servlet>
<servlet-name>LiveLogger</servlet-name>
<servlet-class>uk.co.jaimon.LiveLogger</servlet-class>
</servlet>
<servlet>
<servlet-name>LiveLogFeeder</servlet-name>
<servlet-class>uk.co.jaimon.LiveLogFeeder</servlet-class>
</servlet>
<servlet>
<servlet-name>LogSelector</servlet-name>
<servlet-class>uk.co.jaimon.LogSelector</servlet-class>
</servlet>
<servlet-mapping>
<servlet-name>LiveLogger</servlet-name>
<url-pattern>/logger/*</url-pattern>
</servlet-mapping>
<servlet-mapping>
<servlet-name>LiveLogFeeder</servlet-name>
<url-pattern>/liveLogFeeder</url-pattern>
</servlet-mapping>
<servlet-mapping>
<servlet-name>LogSelector</servlet-name>
<url-pattern>/livelog</url-pattern>
</servlet-mapping>

Monitoring the logs

Once you ran your application, point the browser to, http://<<your server>>/<<your context>>/livelog to get started. I’m assuming that you’ve a valid log4j.properties or log4j.xml file in your classpath. This utility itself uses Log4J for logging with the logger name LiveLogger. Here is a sample screen shot of what you should be seeing on the first page. This is where you can select a logger to change its severity or start streaming log data.

Once you select the logger, you’ll see log data being displayed on the left hand pane as shown below. You can also use this page to change the severity level of your selected logger. Once done, change the level back and stop the monitoring by clicking on Stop button.

How does it work?

Once we select the logger, livelog.html file opens an invisible iframe connecting to LiveLogFeeder servlet to enable a comet style server push. Different styles of comet programming are well documented, so I’m not going to repeat it here. I’m using the streaming technique here. The feeder servlet adds a custom appender to our selected logger, which will keep collecting log messages on to a queue. The servlet then pushes these messages in batch to a client queue with some pause (500 milliseconds by default), so your browser doesn’t freeze up. A JavaScript function on the client side process these queues in batch and puts each message on top of existing once, so the last one gets displayed on top.

Issues with streaming a lot of data to a browser

If there is an influx of messages, then it would severely affect the performance of your application. So I’ve limited the number of messages we can receive on the queue to a 1000.

Your browser will also struggle if we keep inserting a large number of items to the list. So there is a 1000 items restriction here as well. JavaScript function which places the message will also remove any old messages.

Since we are adding an appender, it is important we remove it as soon the client stops or the connection gets lost. We’re achieving this in two ways. On the client side, we’ve added a window.onbeforeunload JavaScript event to stop the monitoring when the user closes the tab or window, or going in to another website. On the server side, most JEE servers will throw an IOException when we try to write to a closed connection. So we can remove the appender and get out of our servlet once we get an exception. OC4J behaves slightly differently though, where it takes another minute or so for it to generate an IOException, when the connection gets lost. I’ve also added a bit of code to write some characters, if we haven’t written to the stream in the last minute to make sure we generate an IOException on closed connections.

Making changes to the configuration parameters

At the moment, the only thing that is configurable via web.xml is the maximum number of clients allowed. There are a few other parameters, which I haven’t bothered to get it from web.xml. These are listed here, and I believe all of them are self explanatory.

• maxNoOfMessagesToWriteOnce (LiveLogFeeder.java)
• intervalBetweenWrites (LiveLogFeeder.java)
• maxMessageQLength (LMAppender.java)
• MAX_LOG_ITEMS_TO_DISPLAY (livelog.js)

My Netbeans project files can be downloaded from here, if you want to make any changes and create the jar file. There is also an Ant target “createjar” which generate the jar file. You’ll need to change the location of the jar file though, as it is hard coded at the moment. If you are using any other IDE, create a new project and copy/paste contents of src and web folders to relevent locations.

Making improvements

There is scope for improvement in a number of places.

On the server side, we could use a separate thread to write messages to the client to reduce some of the overheads. But as you already know, creating threads from Servlet is a no-go area. Two JSRs (236 and 237) were created as possible solutions, unfortunately none of them made it to JEE6. While JSR 236 remains inactive, last year JSR237 was withdrawn. There is a commonj project, which can be utilized as a possible solution.

On the client side, inserting and removing elements via JavaScript takes the most time. I’m sure there are better ways to implement this than using insertBefore and removeChild methods in a loop.

I would appreciate, if you could let me know of any improvements you make to this project.

Files to download

Compiled jar file : liveLogger.jar

Source code (As a Netbeans project) : LiveLoggerV01.zip