Now it is cool to monitor your application on a regular basis.
Because I’m used to shell scripting in LINUX I present you here a scripting solution within a bash shell. But I’m sure you can do the same in Windows.
The idea is to set up a cron job ( for windows use the scheduler) and get the statuspage from your tomcat every – lets say – five minutes.
Get some of the interesting values out of this page and save them in a csv – File.
Create a new file every day. Perhaps you want to use this files as input for a spreadsheet program to create some nice charts.

But to be honest – this is a very special script – you have to adopt it to your needs.
This are the basic prerequisites:

• you need linux to run it (this script is developed on a SUSE – Distribution)
• eventually you need to install some tools (I think the w3m tool is not bundled with every linux distribution)
• this script only! works correct with a tomcat 6.0 version

#!/bin/bash
# -configure below-
HTTPPORT="8080"    # The http Port of your tomcat
JKPORT="8709"      # The Modjk Port of your tomcat (if your tomcat should be reachable from a apache webserver)
SERVER="localhost:$HTTPPORT" # you can even monitor a tomcat on another server!
USER="tomcat-oper" # the account and
PW="IwantItAll"    # password you have defined in tomcat-users.xml
APPL="CoolAppl"    # The name of your application
# -configure above-
TODAY=`date ‘+%m-%d-%y’`
TIME=`date ‘+%H:%M:%S’`
OUTFILE="tomcat-stats-${TODAY}.csv"
#
#set -x   # uncomment if you want to improve the script or if you are searching for a bug
#
# # If there is no csv-file create one and put a headerline in
if !(test -e $OUTFILE) then
  echo "Time,FreeMemory,UsedMemory,MaxMemory,HttpThreadsMax,
  HttpThreadsBusy,ModJKThreads,ModJKThreadsBusy,Sessions" > $OUTFILE
fi
# # get the statuspage from tomcat – filter it for the wanted informations – and create a temporary file
  w3m -dump http://${USER}:${PW}@${SERVER}/manager/status/all| \
  egrep "(memory|thread|Active sessions|
  ^JVM|$HTTPPORT|$JKPORT|${APPL})" >/tmp/tomcat-status
#
# # put the values in variables / uncomment to get a better understanding of the process
#
while read line
   do
      case $line in
           JVM) read line
                JVM_VALUE=`echo $line | awk ‘{print($3 "," $7 "," $11)}’`
#               uncomment below if you want to debug
#               echo "JVM_VALUE: <$JVM_VALUE>"
                ;;
            *${HTTPPORT}) read line
                HTTP_MAX_THREADS=`echo $line |awk ‘{print("," $3)}’`
#               uncomment if you want to debug
#               echo "HTTP_MAX_THREADS: <$HTTP_MAX_THREADS>"
                HTTP_BSY_THREADS=`echo $line |awk ‘{print("," $11)}’`
#               uncomment below if you want to debug
#               echo "HTTP_BSY_THREADS: <$HTTP_BSY_THREADS>"
                ;;
            *${JKPORT}) read line
                MODJK_MAX_THREADS=`echo $line |awk ‘{print("," $3)}’`
#               uncomment below if you want to debug
#               echo "MODJK_MAX_THREADS: <$MODJK_MAX_THREADS>"
                MODJK_BSY_THREADS=`echo $line |awk ‘{print("," $11)}’`
#               uncomment below if you want to debug
#               echo "MODJK_BSY_THREADS: <$MODJK_BSY_THREADS>"
                ;;
            *${APPL}) read line
                SESSION_CNT=`echo $line |awk ‘{print("," $3)}’`
#               uncomment below if you want to debug
#               echo "SESSION_CNT: <$SESSION_CNT>"
                ;;
            *) ;;
      esac
    done < /tmp/tomcat-status
# # Now is the time to write the data to the csv-file
  echo "${TIME},${JVM_VALUE}${HTTP_MAX_THREADS}${HTTP_BSY_THREADS}
  ${MODJK_MAX_THREADS}${MODJK_BSY_THREADS}${SESSION_CNT}" >>$OUTFILE
exit

*) be aware that based on word wrapping this script might not work instantly

Create a scriptfile (e.g. tomcat-check.sh) and apply the script to your tomcat. Then edit your crontab.

There should be a line like:
"*/5 * * * * cd ~/statistics; ./tomcat-check.sh 1>tomcat-check.log 2>&1"

And then the script should create the csv-files. I’m not used to excel – but even I managed to create a more or less interesting diagram from this data. Note how regular the memory behaves while there is no user session.

• Sometimes the application seems to be frozen.
• Sometimes you just want to know what the hell
  is going on with your application.

So the best way is to say:
„Please my little tomcat, tell me – how do you feel right now?“

And if you choose the right method you will get some answers.
The good news ahead – it isn’t very difficult to get the information.

You just have to enable the tomcat console. And furthermore you should enable the printout of the garbage collects.
(type "java -X" to get the right gc options for your java-version – normally something like verbose:gc. I will show you in detail in another article)

Just edit the tomcat-users.xml File.

You have to add the roles: manager and admin and add a user and a password that meets your needs. Invent your own username and password and keep it secret. Because userid and password are not encrypted (if you do not explicitly configure https as protocol) you should keep in mind that this is no high sophisticated security.

Nevertheless – save your tomcat-users.xml file and restart tomcat. If your tomcat listens on the standard port (8080) give the manager console a try:

http://yourserver-adress:8080/manager/html

You will be prompted for a username and password – if you can remember the values you had put in the tomcat-user.xml file – go on and put them in. So you can see the console – not very exciting – thats true. Here you can just see how many sessions are currently active in an application . And if you want to have fun and you are !not the responsible person for this service you can easily stop an application within the tomcat.

But to be more serious – now choose the link in the upper right corner (Server Status) and again the link in the upper right corner (Complete Server Status).
Or just enter:

http://yourserver-adress:8080/manager/status/all

Take some time to explore the page. Reload the page several times – look at how the connections fall and rise. Don’t forget to check the Memory – is enough free memory left ? What is the state of the connections – are there any connections left. How many threads are busy? How many spare threads do you have? Perhaps you have to adjust the values in the server.conf file.

Can you imagine that it is very easy to get this values extracted and in a "cvs"-file on a regular basis ? So you can get historical values – But that is subject for an own article.

A basic security can be established by a password restricted folder access, enforced by the Apache Webserver itself.

This can be done by some simple instructions added to our .htaccess file:
(In case you haven’t already a .htaccess file, just create a new file calling it .htaccess and locating it in the folder you want to protect.)

AuthType Basic
AuthName "Protected Area"
AuthUserFile FILE
require valid-user

These commands will tell the Apache webserver that the folder in question (and all below) are now protected and it needs a valid combination of username/password to access it.

• AuthName is some free form text you can specify, telling the visitor what you are asking for and Apache Webserver will show in it’s dialog box.

• AuthUserFile is the reference to the stored .htpasswd file containing all valid username/password combinations to access the folder in question (ususally and best located out of the protected directory tree).

Having that done we now need to create the necessary .htpasswd file which we can do interactive on our server with the command:

htpasswd2 -c FILE USER

Assuming html is your webroot, a valid example, for protecting a folder called images below our webroot and using demouser/demopassword as user credentials, would be:

.htaccess:
(Locate this file within the folder images and be aware to use an absolute path for AuthUserFile, as for www.domain.com would be wrong!)

AuthUserFile /home/username/html/.htpasswd
AuthType Basic
AuthName "Login for Secure Area"
require valid-user

.htpasswd
(Create a new file called .htpasswd and locate this file within the folder you’ve specified within AuthUserFile path.)

demouser:$1$OHq3K5v3$pvFrGsHjWwYjpkZhc70T9.

Once being familiar with it you will ask yourself about why switching tool when it comes to the need of creating reports in a normal web design world mostly using MySQL Databases within a "LAMP" environment.

Written as an universal reporting tool BI Publishes does offer us a more or less universal data interface based on JDBC connections. And being able to act itself as a callable web service offering WEBDAV services based on XML data exchange it anyway seems to be an excellent choice for this world.

Already Oracle’s installation instructions does give you a quick idea about setting up BI Publisher working against some MySQL Database. But acting  in a Web environment you might find it cool integration the BI Publisher within an already existing Apache Tomcat environment and for exactly this environment I will give you a short explanation about getting things together and ready for work.

First we will pick the BI Publisher software itself by copying the xmlpserver.war file from the distribution DVD and throwing it into the webapps folder of our Apache Tomcat. Waiting a few seconds until the self-deploying mechanism of the Apache Tomcat has extracted the files for us, we need to specify the access path to our project and development repository within the file xmlp-server-config.xml.

Having done that, we need to add the MySQL JDBC driver to our system and making it known within the environment of the Apache Tomcat. (The MySQL JDBC driver can be downloaded from: http://www.mysql.com/products/connector/j/.)

So with the downloaded fileready, we have to extract it to any place we like onto our disk. Just I would recommend you to put it somewhere close to your MySQL installation for you being able to later on remember it again. Finally extracted and ready to use, we have to add the information about the mysql-connector-java- -bin.jar file to the Java Classpath of the Apache Tomcat and finally restarting it:

Having this set, as a last step we have to invoke the BI Publisher itself, in which we then will configure the new JDBC data source within the Admin region of it:

And now with this tweaking ready, MySQL DB should connect, open and work for you like a charm.

Instead of downloading all content from Server A to your local system using a however clever FTP Client and later uploading it again to Server B, this whole file transfer can be done within one step using a server to server copying command.

As a basic requirement to use this feature, you have to have secure shell access onto both servers. Being logged in to one server (it doesn’t matter which one) you can use SCP command to transfer all your files. The syntax of the command is fairly easy and it is part of the command to specify source and target system and therefore is doesn’t matter if you are pulling or pushing the files.

Easy command line examples for an SCP command would be:

Copying file to host:

scp SourceFile user@host:directory/TargetFile

Copying file from host:

scp user@host:directory/SourceFile TargetFile

Using the -R option as well will allow you to transfer whole directory trees within one single copy command.

While transfering all files and perhaps several domain can take up several days depending on the size and complexity of your hosting, it can occur that parts of the original files are already changed again until the final go life.

To prevent differences in between source and new target system a final update of both system shortly before switching over does make sense. Here now the command RSYNC will help us.

With RSYNC you can syncronize files and folders between systems.

Again an easy command line example would be:

rsync -r user@host:directory/SourceFolder TargetFolder

Also here source and target system needs to be specified and therefore both sending and receiving are possible.