Running Multiple Tomcat Instances on Single Machine

Download Apache Tomcat form https://tomcat.apache.org/download-70.cgi and extract the downloaded tar.gz file. It contains the following folders

bin -  It contains all binary and script files for running tomcat.

lib - contains all shared libraries used for tomcat

conf - contains configuration information like which port tomcat can bind , etc...

logs - it contain all logging details

temp - this folder tomcat used for temporary files purpose

webapps - this folder is very important. here we put all application war files.

work - If application contain any jsp then jsp is translated and converted into servlet its stores here.

In when run the tomcat its uses 5 environment variables. They are 

CATALINA_HOME, CATALINA_BASE, CATALINA_TMPDIR, JRE_HOME/JAVA_HOME, CLASSPATH

in above list CATALINA_HOME and JAVA_HOME is mandatory environment variables. all others are optional and its can be calculated using CATALINA_HOME.

CATALINA_HOME - this environment variable should point to tomcat base folder, where tomcat binary are installed/extracted. So based on CATALINA_HOME we can get bin and lib folder

CATALINA_BASE - If we not specified then CATALINA_HOME value is set. This variable pointed to configuration and webapps folder. Based on this variable server uses conf, logs, temp, webapps, work folders.

Usual ways to run the tomcat is only set CATALINA_HOME environment variable. and run the startup.sh script file. this startup.sh file automatically calculate and assign the values of other variables what we are not set.

startup.sh file set the environment variable and  then call catalina.sh file. this files is read CATALINA_BASE value and attach conf i.e $CATALINA_BASE/conf folder and get server.xml. this file is heart of tomcat. it contains all configuration information. like which tomcat uses as shoutdown port, connector post, host name, application folder ,.. for example usually tomcat uses 8080 is a connector port, so we can access http://localhost:8080/ 

if we set the $CATALINA_BASE explicitly then tomcat uses our variable to search and get the server.xml file from our target place, what we specified in CATALINA_BASE. 

this is trick to run multiple tomcat in single machine. we don't change CATALINA_HOME value. we need to change CATALINA_BASE value before start/shutdown the tomcat.

create one folder named "tomcat-instance1" anywhere, and copy conf, temp, webapps and logs  folder from CATALINA_HOME folder and change conf/server.xml file in tomcat-instance1. we need to change 3 port shutdown port, connector port and ajp port.

shutdown port - this port is used for shutdown the tomcat. when we call the shutdown.sh script they send signal to shutdown port. this port listen by tomcat java process. if signal is received the that process then its cleanup and exit by itself.

connector Port -This port is actual port to expose the application to outside client. 

ajp port - this port is used to apache httpd server  communicate to tomcat. this port used when we setup load balanced server.

so we change these three port to different number, because once this port is binded  then other process can't bind it again. so wee bind different port. so tomcat-instance1/conf/server.xml file i configured server port =8105, connector port = 8181, ajp port = 8109.

now we can create two script file for startup and shutdown the tomcat-instance1.

startup-instance1.sh
export CATALINA_BASE= /home/ramki/tomcat-instance1
cd $CATALINA_HOME/bin
./startup.sh

shutdown-instance1.sh

export CATALINA_BASE= /home/ramki/tomcat-instance1

cd $CATALINA_HOME/bin

./shutdown.sh

here we explicitly set the CATALINA_BASE variable and point to new tomcat-instance1
the we go to CATALINA_HOME/bin folder because all binary for running tomcat is still present in CATALINA_HOME folder then startup/shutdown the script.

Based on above technique we can create many instance folder and change conf/server.xml file port values and run that instance with own newly created script files.

Source: http://www.ramkitech.com/2011/07/running-multiple-tomcat-instances-on.html

Virtual Host + Apache httpd server + Tomcat + mod_jk connector

Problem Scenario:
        In big organization they have multiple department, each department want to host their website in different machine. so these websites are accessed locally with different local IP address.

When we mapping to public address then we face the problem. We have two choice either purchase as many public address or Put one server front and delegate these request.

 We going to use 2nd option. we put Apache httpd web server in front of all department servers. so only one public IP is enough. All domain DNS entries are pointed to Apache httpd server. Then Apache server delegates these request to corresponding tomcat server. This process is completely transparent from users(browser) perspective.

Outline Structure of Virtual Host Implementation

How Apache httpd web server communicate to Tomcat server
               Before we going to detail about how communication happen between httpd server and tomcat

How many ports are bind when we start single tomcat?

·                     shutdown port

·                     http connector port

·                     https connector port (optional)

·                     ajp port

The port configuration are stored in $CATALINA_HOME/conf/server.xml file. we can change the ports when its necessary.

here AJP(Apache JServ Protocol)  is a binary protocol that can proxy inbound requests from a web server through to an application server that sits behind the web server.

Apache httpd webserver communicate to Tomcat sever through AJP protocol.

When we install Apache httpd server, It don't have inbuilt capability to support ajp protocol. so we need mod_jk module. Its add the ajp support to Apache httpd server.

Steps to Implement Virtual Host Concept in this Scenario:

1.                      Install Apache httpd Web Server

2.                      Install mod_jk connector

3.                      Configure JK Connector

4.                      Configure Apache httpd server apply virtual host concepts

Prerequisite : We already installed Tomcat in different departments and deployed the application and works fine.

Install Apache httpd web server
        We can install Apache web server in two ways.
                         - Binary module
                         - From Source

We can install Apache httpd server from distribution package manager (either apt-get or yum). Or we can download the source code and then compile and install.
Note : -
 we need to install apr, apr-util and pcre libraries because Apache httpd server depends on these libs.
But these libraries are no need to install from source. because we not going to do any customization.
so install these lib as binary using apt-get (debian/ubuntu) or yum command (redhat/fedora).
in ubuntu u can search through
sudo apt-cache search apr
we use second option. First download the httpd server source code from here . then extract it and install
./configure --prefix=/usr/local/apache    --enable-rewrite=shared  --enable-proxy=shared
make
sudo make install
here
 --prefix option to mention where the location we going to install Apache httpd server.

 --enable-rewrite and --enable-proxy options to enable these module in shared mode. These modules are not needed now. but we used in future for rewrite the URL before handover to next chain of servers and load-balancing support.

Install mod_jk connector

 Now Apache httpd server is ready. we need to add ajp support to server.

download the mod_jk connector module from here. extract it and install it

cd native

./configure    --with-apxs=/usr/local/apache/bin/apxs

make

sudo make install

here  --with-apxs option to specify where apxs module is located. so we need to give Apache httpd server location.

now mod_jk.so files is created on modules directory in apache installed location (/usr/local/apache/modules)

Configure mod_jk connector

This step have 2 sub step

·                       Create workers.properties file

·                        Load and configure the JK connector module in apache httpd.conf file

Create workers.properties file 

mod_jk connector is ready. but this connector is works based on configuration file. so we need to create configuration file called workers.properties file

this file syntax is key=value pair,

here we define the workers. i.e all department tomcat hosts IP address and ajp port for corressponding tomcat.

here entry format is look like

worker.<name>.property=<value>

for example 

worker.<name>.type=ajp13

worker.<name>.port=<ajp port>

worker.<name>.host=<tomcat ip addr>

worker.list=<name>

here 

worker.list key have all workers name separated by comma.

type = here type of the worker. we use ajp13 version 

port= we specify the ajp port (not http port ) of that server

host= IP address or host name of tomcat server

workers.properties

worker.list=department1,department2,department3

worker.department1.type=ajp13

worker.department1.port=5000

worker.department1.host=192.168.5.10

worker.department2.type=ajp13

worker.department2.port=5000

worker.department2.host=192.168.6.10

worker.department3.type=ajp13

worker.department3.port=5000

worker.department3.host=192.168.7.10

Add Entry in httpd.conf

Apache httpd server is installed. mod_jk module is installed and workers.properties file is created. but these 3 are isolated. we put together,
we need to configure the httpd server.

find the conf/httpd.conf file in Apache installed location and add these following entries to it

LoadModule    jk_module  modules/mod_jk.so

JkWorkersFile conf/workers.properties

JkLogFile     logs/mod_jk.log

JkLogLevel    emerg

JkLogStampFormat "[%a %b %d %H:%M:%S %Y] "

JkOptions     +ForwardKeySize +ForwardURICompat -ForwardDirectories

JkRequestLogFormat     "%w %V %T"

here

LoadModule    - Load mod_jk shared module to Apache httpd server (enable the mod_jk module)

JkWorkersFile - Specify the workers.properties file location

all others are logging system of mod_jk. Its boilerplate code just copy and paste.

Delegate httpd to Tomcat

now we inform to Apache httpd server how delegate the request to corresponding server.

JkMount  /department1* department1

JkMount  /department2* department2

JkMount  /department3* department3

here

JkMount - specify the if URL have /department1* pattern then that request delegate to department1 worker. that worker IP address and port is specified in workers.properties file.

Iwe changed /etc/hosts file like last post  all web site domain pointed to apache httpd web server.

if we access http://www.ramki.com/department1/index.html  how Apache httpd server process the request

Its perfectly called correct tomcat server and we got right response.

but we have 2 problem

 1. i don't want the my URL like this   http://www.ramki.com/department1/index.html 

              i want the my URL like this   http://www.ramki.com/index.html  

remove the department1 from my URL. but department1 string is very important in URL because JkMount is works based on this matching string only.

2. http://www.ramki.com/department1/index.html   is for first tomcat and second department have domain   http://www.krishnan.com then 

access second tomcat we use URL : http://www.krishnan.com/department2/index.html

but same time when we use URL      http://www.krishnan.com/department1/index.html then we access first tomcat data

(i.e )
http://www.ramki.com/department1/index.html  == http://www.krishnan.com/department1/index.html

because both these URL have department1 key word.. so JkMount is works based on these keyword. As the Result wrong interpretation. How to solve this Issue?.

Virtual Host in Apache httpd Server

we need add conditional JkMount. for example ramki.com domain asks the whereare paths like department1,department2 we need to search in that tomcat only. not other place. to add this conditioned we add virtual host entries. 

Add virtual host entry in httpd.conf file

Listen 80

NameVirtualHost *:80

<VirtualHost *:80>

    ServerName www.ramki.com

    JkMount  /department1* department1

</VirtualHost>

<VirtualHost *:80>

    ServerName www.krishnan.com

    JkMount  /department2* department1

</VirtualHost>

here

ServerName - domain name of the server

If  http://www.krishnan.com/department1/index.html URL is like this now. server matches the Server Name. here its matches 2nd virtual host entry. There are single JkMount entry is there in 2nd Virtual-Host. and there no matching department1 string.

  JkMount  /department2* department1

as the result 404 error page is responded. Its works good.

Now everythig works fine. 

now my URL is  http://www.ramki.com/department1/index.html here i don't want department1 path in my URL.

i want simply     http://www.ramki.com/index.html then we use mod_rewrite engine

Listen 80

NameVirtualHost *:80

<VirtualHost *:80>

     ServerName www.ramki.com

     RewriteEngine on

     RewriteLog logs/apache-mod_rewrite

     RewriteRule ^/(.*)$ /department1/$1 [L,PT]

     JkMount  /* <dept_name>

</VirtualHost>

here

RewriteEngine on - Turn on the Rewrite module

RewriteRule ^/(.*)$ /department1/$1 [L,PT]  - here ^/(.*)$ its matches any string it capture the value to $1 and change the URL to /department1/$1 

i.e http://www.ramki.com/index.html here  ^/(.*)$ its matches  index.html it capture to $1 and replaced to /department1/$1 ==> /department1/index.html

Rewrite is happen before delegate the request to Tomcat. so we change the URL transparently to browser.

Source: http://www.ramkitech.com/2012/03/virtual-host-apache-httpd-server-tomcat.html

Tomcat Clustering Series Part 1 : Simple Load Balancer

Why need Clustering? (Tomcat Clustering) 
          In normal production servers are running in single machine. If that's machine may be failed due to crashed or hardware defects or OutOfMemory exception then  our site can't access by anybody.

so how to solve this problem?. to add more tomcat machine to collectively (group/cluster) run as a production server. (oppose of single machine). Each tomcat has deployed the same web application. so any tomcat can process the client request. If one tomcat is failed, then other tomcat in the cluster to proceeds the request.

Here one big problem is arrive. each tomcat instances are running in dedicated physical machine or many tomcat instances are running in single machine(Check my post about Running multiple tomcat instances in single machine). so each tomcat running on different port and may be in  different IP.
the problem is in client perspective, to which tomcat we need to make the request? because there are lots of tomcat part of clustering is running. each tomcat we need to make IP and Port combination. like
http://192.168.56.190:8080/ or http://192.168.56.191:8181/

so how to solve this problem?
To add one server in-front of all tomcat clusters. to accept all the request and distribute to the cluster. so this server acts as a load balancer. There is lots of server is available with load balancing capability. here we are going to use Apache httpd web server as a load balancer. with mod_jk module.

so now all clients to access the load balancer (Apache httpd web server) and don't bother about tomcat instances. so now ur URL is http://ramkitech.com/ (Apache runs on port 80).
Apache httpd Web Server
           Here we are going to use Apache httpd web server as a Load Balancer. To provide the load balancing capability to Apache httpd server we need to include the either mod_proxy module or mod_jk module. here we are using mod_jk module.

Before continuing this post check my old post (Virtual Host Apache httpd server) about How to install the Apache httpd server and mod_jk module. and how to configure the mod_jk.
How to setup the Simple Load Balancer
         For simplicity purpose i going to run 3 tomcat instances in single machine(we can run on dedicated machine also) with Apache httpd web server. and single web application is deployed in all tomcat instances.

here we use mod_jk module as the load balancer. by default its use the round robin algorithm to distribute the requests. now we need to configure the workers.properties file like virtual host concept in Apache httpd server.

worker.list=tomcat1,tomcat2,tomcat3

worker.tomcat1.type=ajp13
worker.tomcat1.port=8009
worker.tomcat1.host=localhost

worker.tomcat2.type=ajp13
worker.tomcat2.port=8010
worker.tomcat2.host=localhost

worker.tomcat3.type=ajp13
worker.tomcat3.port=8011
worker.tomcat3.host=localhost

here i configure the 3 tomcat instances in workers.properties file. here type is ajp13 and port is ajp port (not http connector port) and host is IP address of tomcat instance machine.
there are couple of special workers we need add into workers.properties file.
First one is add load balancer worker, here the name is  balancer (u can put any name).

worker.balancer.type=lb
worker.balancer.balance_workers=tomcat1,tomcat2,tomcat3

here this worker type is lb, ie load balancer. its special type provide by load balancer. and another property is balance_workers to specify all tomcat instances like tomcat1,tomcat2,tomcat3 (comma separated)
Second one, add the status worker, Its optional. but from this worker we can get statistical of load balancer.

worker.stat.type=status

here we use special type status.

now we modify the worker.list property.

worker.list=balancer,stat

so from outside there are 2 workers are visible (balancer and stat). so all request comes to balancer. then balancer worker manage all tomcat instances.

complete workers.properties file

Now workers.properties confiuration is finished. now we need to send the all request to balancer worker.
so modify the httpd.conf file of Apache httpd server

the above code is just boiler plate code. 1st line load the mod_jk module, 2nd line to specified the worker file (workers.properties file). all others are just logging purpose.

The last 2 lines are important.
JkMount  /status  stat   means any request to match the /status then that request forward to stat worker. Its status type worker. so its shows status of load balancer.

JkMount  /*  balancer this line matches all the request, so all request is forward to balancer worker. In balancer worker is uses the round robin algorithm to distribute the request to other tomcat instances.

That's it. Now access the load balancer from the browser. each and every request is distribute to 3 tomcat instances. If one of the tomcat instances are failed then load balancer dynamically understand and stop to forward the request to that failed tomcat instances. Other tomcat instances are continue to work. If that failed tomcat is recovered from failed state to normal state then load balancer add to cluster and forward the request to that tomcat. (check the video)

Here big question is How Load balancer knows when one tomcat instance is failed or tomcat is just recovered from failed state?
Ans : when one tomcat instance is failed, load balancer don't know about that instances is failed. so its try to forward the request to all tomcat instances. If load balancer try to forward the request to failed tomcat instance, its will not respond. so load balancer understand and marked the state as a failed and forward the same request to another tomcat instances. so client perspective we not feel one tomcat instances are failed. 
when tomcat instances recovered from failed state. that time also load balancer don't know that tomcat is ready for processing. Its still marked the state is failed. In periodic interval load balancer checks the health status of all tomcat instances. (by default 60 sec). after checking health status then only load balancer came to know that tomcat instance is ready. and its update the status is OK.

Source : http://www.ramkitech.com/2012/10/tomcat-clustering-series-simple-load.html