Monitoring Log4J messages on a browser via server push
Introduction
Most of the Java projects I’ve worked on have used Log4J as the logging framework. We normally change the severity level to ERROR on lo4j.xml before deploying our web application to a production system. But when something goes wrong, you may want to enable debugging for a short time and see the messages, and then turn the level back. There are a couple of ways you can change the severity level on the fly. You can either use a jsp file as explained here or a JMX solution as explained here or here
But we also wanted to take peek at the log data in real time so when we see an error being generated we can change the level back to ERROR and look at server’s log files for debugging. Something like doing a tail –f logfile.log
from the server. In some places, developers are not given SSH access to server, and needs to contact the DBA for any log files. Even if the developer has access to it, SSH access may be limited to internal traffic, and if you want to fix an issue from home, you’re out of luck. A browser based solution would be ideal in these cases. So here we are discussing such a solution, which should be very easy to implement in your own project.
- Place the liveLogger.jar on your WEB-INF/lib folder
- Add the following to your web.xml (before the closing web-app tag)
<context-param>
<param-name>LIVE_LOGGER_MAX_CLIENTS</param-name>
<param-value>10</param-value>
</context-param>
<servlet>
<servlet-name>LiveLogger</servlet-name>
<servlet-class>uk.co.jaimon.LiveLogger</servlet-class>
</servlet>
<servlet>
<servlet-name>LiveLogFeeder</servlet-name>
<servlet-class>uk.co.jaimon.LiveLogFeeder</servlet-class>
</servlet>
<servlet>
<servlet-name>LogSelector</servlet-name>
<servlet-class>uk.co.jaimon.LogSelector</servlet-class>
</servlet>
<servlet-mapping>
<servlet-name>LiveLogger</servlet-name>
<url-pattern>/logger/*</url-pattern>
</servlet-mapping>
<servlet-mapping>
<servlet-name>LiveLogFeeder</servlet-name>
<url-pattern>/liveLogFeeder</url-pattern>
</servlet-mapping>
<servlet-mapping>
<servlet-name>LogSelector</servlet-name>
<url-pattern>/livelog</url-pattern>
</servlet-mapping>
Monitoring the logs
Once you ran your application, point the browser to, http://<<your server>>/<<your context>>/livelog to get started. I’m assuming that you’ve a valid log4j.properties or log4j.xml file in your classpath. This utility itself uses Log4J for logging with the logger name LiveLogger. Here is a sample screen shot of what you should be seeing on the first page. This is where you can select a logger to change its severity or start streaming log data.
Once you select the logger, you’ll see log data being displayed on the left hand pane as shown below. You can also use this page to change the severity level of your selected logger. Once done, change the level back and stop the monitoring by clicking on Stop button.
How does it work?
Once we select the logger, livelog.html file opens an invisible iframe connecting to LiveLogFeeder servlet to enable a comet style server push. Different styles of comet programming are well documented, so I’m not going to repeat it here. I’m using the streaming technique here. The feeder servlet adds a custom appender to our selected logger, which will keep collecting log messages on to a queue. The servlet then pushes these messages in batch to a client queue with some pause (500 milliseconds by default), so your browser doesn’t freeze up. A JavaScript function on the client side process these queues in batch and puts each message on top of existing once, so the last one gets displayed on top.
Issues with streaming a lot of data to a browser
If there is an influx of messages, then it would severely affect the performance of your application. So I’ve limited the number of messages we can receive on the queue to a 1000.
Your browser will also struggle if we keep inserting a large number of items to the list. So there is a 1000 items restriction here as well. JavaScript function which places the message will also remove any old messages.
Since we are adding an appender, it is important we remove it as soon the client stops or the connection gets lost. We’re achieving this in two ways. On the client side, we’ve added a window.onbeforeunload JavaScript event to stop the monitoring when the user closes the tab or window, or going in to another website. On the server side, most JEE servers will throw an IOException when we try to write to a closed connection. So we can remove the appender and get out of our servlet once we get an exception. OC4J behaves slightly differently though, where it takes another minute or so for it to generate an IOException, when the connection gets lost. I’ve also added a bit of code to write some characters, if we haven’t written to the stream in the last minute to make sure we generate an IOException on closed connections.
Making changes to the configuration parameters
At the moment, the only thing that is configurable via web.xml is the maximum number of clients allowed. There are a few other parameters, which I haven’t bothered to get it from web.xml. These are listed here, and I believe all of them are self explanatory.
-
maxNoOfMessagesToWriteOnce (LiveLogFeeder.java)
-
intervalBetweenWrites (LiveLogFeeder.java)
-
maxMessageQLength (LMAppender.java)
-
MAX_LOG_ITEMS_TO_DISPLAY (livelog.js)
My Netbeans project files can be downloaded from here, if you want to make any changes and create the jar file. There is also an Ant target “createjar” which generate the jar file. You’ll need to change the location of the jar file though, as it is hard coded at the moment. If you are using any other IDE, create a new project and copy/paste contents of src and web folders to relevent locations.
Making improvements
There is scope for improvement in a number of places.
On the server side, we could use a separate thread to write messages to the client to reduce some of the overheads. But as you already know, creating threads from Servlet is a no-go area. Two JSRs (236 and 237) were created as possible solutions, unfortunately none of them made it to JEE6. While JSR 236 remains inactive, last year JSR237 was withdrawn. There is a commonj project, which can be utilized as a possible solution.
On the client side, inserting and removing elements via JavaScript takes the most time. I’m sure there are better ways to implement this than using insertBefore and removeChild methods in a loop.
I would appreciate, if you could let me know of any improvements you make to this project.
Files to download
Compiled jar file : liveLogger.jar
Source code (As a Netbeans project) : LiveLoggerV01.zip
Managing Vimeo video player via Javascript using Moogaloop API
Making Scrollable Tables with fixed headers
********** Please click here for an update script with additional functionalities like multiple header freezing, column freezing and multiple tables ***********
Although it is a common requirement, it’s not always straight forward to make scrollable tables with fixed headers. Here, we are going to try a few JavaScript solutions that could be suitable for most cases. I need to emphasize on that, because you might need to make a few tweaks to make it work in your environment, especially tables generated by web frameworks.
Implementation
To implement these JavaScript solutions, your tables should have an ID and width. Width can be a fixed value or a percentage. Since the width can be specified in percentage, it should also work when window get resized. It has been designed to introduce this functionality with minimal changes to your existing pages. So for the JavaScript functions to work, you only need to add the following lines to your HTML pages,
<script type="text/javascript" src="fxheader1.js"></script>
<script type="text/javascript">
fxheaderInit('dataTable',300);
</script>
Make sure you add the second JavaScript call after the table element. Replace dataTable with the ID of your table, and 300 with the height you require. Instead of specifying the height as a fixed value, there are ways to stretch the table to fill the page based on other components on the page. Last solution make use of such a method.
Browser compatibility
It’s been tested on the following browsers all running on Windows XP.
- IE6
- IE7
- IE8
- Firefox 2.0
- Firefox 3.5
- Google Chrome
The big table
Here is an example of a table we are going to introduce the scrolling functionality. Well, it is not really big, just big enough to test our functionality 🙂
http://jaimonblog.appspot.com/datascroller/bigTable.html
Solution 1: Simple CSS only scrolling without fixing headers
With the help of a bit of CSS, we can implement a simple scroller. Enclose the table element within a div container with an overflow value of auto and a fixed height. Something like
<div style=’overflow:auto;height:300px;’>
<table …. />
</div>
You can see the demo at http://jaimonblog.appspot.com/datascroller/simpleScroller.html
Solution 2: Scrolling with fixed headers using table cloning
You can see a working demo at http://jaimonblog.appspot.com/datascroller/fixedHeaderFullClone.html
Download fxheader1.js and include it in your project as explained on the implementation section
How does it work?
- First we put a scrollable div container around the table element as in our first solution.
- create a div element
- clone the table element using cloneNode(true)
- added the cloned table to this div
- replace the table element with this div
- Create a container on top of the table to place the headers.
- Attach scrollHeader function to the table container div’s onscroll event, so that we can align the heading when the table is scrolled horizontally.
- We then clone the table and place it in the header container. Container height is set, so that only the header row is visible.
- We then set a negative margin top on the original table to hide the actual header.
- Attach fxheader function to window onresize event, so it works when window get resized.
Please note that, there are more than one way to do add a div element around our table element using JavaScript. You could speed up the process by adding these divs manually on the page as in Solution 1, because cloneNode function can be expensive. Make sure to use the correct IDs, so that rest of the code can find these. Please see the function addScrollerDivs in fxheader1.js for more details.
Solution 3: Scrolling with fixed headers using fast table cloning
You can see a working demo at http://jaimonblog.appspot.com/datascroller/fixedHeaderFastClone.html
Download fxheader2.js and include it in your project as explained on the implementation section to get started.
Solution 2 will give you very accurate column widths at the expense of doing a full table cloning. If you have a table with lots of rows, this can take a while. If your rows have elements with unique IDs, you will have to change it on the cloned node to avoid ID collision. Although it will work, it will lead to a sluggish user experience on pages with really large tables.
In solution 3, we are doing a fast clone without any child elements. Then we add the first row to it and set individual column width separately based on the offsetWidth property of each cell. You’ll have to offset cell margin/padding when setting the width. In our example, I deduct 3 from offsetWidth and that works fine for our table in all browsers I’ve tested.
Everything else is similar to solution 2.
Solution 4: Adding scrolling to table generated by ADF Faces 10g with fixed header (With and without pagination support) and automatic height stretching
Although in theory it is similar to solution 3, I had to make a few changes to make it work for a table generated by ADF Faces. The problem is ADF Faces creates three HTML tables for each af:table element. First and third tables are created for pagination support. It doesn’t create an ID for the actual data table itself, so we need to access it as a childNode of the container element. If pagination is there, we need to place this above our fixed header. Since we have the scrolling functionality, I thought the second pagination controls below the data table is not necessary, and I’m removing it from the page.
When cloning table data here, we have to update ID field of each element to avoid ID collision. We also have to set the correct selected index value for pagination selector combo box. Another functionality I’ve implemented here is the ability to auto stretch the table content to fit the page. You should pass in the ID of your last container component to make it work. Please see adjustHeight function in fxheader.js for more details.
As with the second and third solutions, you could add the following line to your JSPX page to get this working.
<afh:script source="fxheader.js" />
<afh:script text="fxheaderInit('testTable','footerContent');" partialTriggers="testTable" />
Where testTable is the ID of your af:table element and footerContent is the ID of your last container component for stretching. Partial Triggers are required here if used with pagination.
You can download a fully functioning JDeveloper project from http://jaimonblog.appspot.com/datascroller/ADF10gScroller.zip
Please note that it is created with JDeveloper version 10.1.3.3. If you are using 11g, then fixed header functionality is already part of it. (Although there are a few issues with it at the time of writing this. One such issue is discussed here http://forums.oracle.com/forums/message.jspa?messageID=4032492#4032492)
As I’m not able to provide a demo link, you can see some screen shots of this solution here,
-
Recent
- Check digit calculator for barcodes and credit cards
- An Excel Addin to work with Unicode CSV files
- SimpleImageInfo – A Java class to get image size without loading the whole data
- Making Scrollable Tables with fixed headers – Updated
- Migrating 10g JSF Web project with ADF Faces components in jspx pages to 11g Facelets pages using Rich components
- Monitoring Log4J messages on a browser via server push
- Managing Vimeo video player via Javascript using Moogaloop API
- Making Scrollable Tables with fixed headers
- C# FTP Client Library
-
Links
-
Archives
- April 2013 (1)
- August 2011 (1)
- January 2011 (1)
- March 2010 (1)
- February 2010 (4)
- September 2009 (1)
-
Categories
-
RSS
Entries RSS
Comments RSS