A common design problem found in most of the servers is their need to handle hundreds and even thousands of service requests and still maintain a reasoanble performance for most of the use cases.
A good example is the Internet Information Server (IIS) found on Microsoft Windows platforms serving up ASP pages or an apache server from apache software foundation that handles incoming HTTP requests.
Problem requires Thread Recycling
Nowadays, most servers are equipped with quad-core processors and hyper-threading models with logical CPUs embedded in a core. The creation and destruction of a few threads in the context of an operating system such as MS Windows or a Linux won’t pose much of a problem for them.
However, handling a large volume of service requests eventually requires your server to spend a large portion of its time in creating and destroying threads to service these requests. You may start to notice a significant slowdown in servicing those requests. This becomes a performance bottleneck.
An expensive part of the thread creation and destruction logic is the time required to switch between user and kernel modes. As a rough approximation, on a Pentium class machine running an MS windows OS, a single API call that forces a transition from user mode to kernel mode can easily take about 600 CPU clock cycles as opposed to about 30 CPU clock cycles for a function call in an user mode.
Briefest look at Task Creation in Java
Turning your object into a Task:
If you are constructing your program using java, the built-in thread support comes in handy. Here is a rather simplified view of a class that runs as a task,
class MyClass implements Runnable{
// new incoming jobs will be placed into this bin.
List input;
MyClass(List input){
// bin setup by caller to present new jobs to our object.
this.input = input;
}
// thread's main routine
public void run(){
// loop here indefinitely looking for work!
while(true){
// new jobs ready?
while(input.isEmpty()){
// wait for a notification from the main thread to exit.
input.wait();
// found a new job in the bin
userRqst = (String) input.remove(input.size() - 1);
}
// compute the new job
doSomething(userRqst);
} // end while(true)
} // end run
} // end MyClass
Runnable Interface:Examining the above piece of code, we see, our MyClass implements an interface called Runnable which supports a lone function, run(). This API is the heart of our java thread. It is equivalent to the main() in a Java or C program.
Dynamic inputs of data into our task: The constructor of MyClass takes in a container object passed in by the calling main thread. This container may initially be empty. When a new job arrives at its door step, the main thread, which created our object can place this new job into this container. This new job may be in the form of a file name to be gzipped by our object or it can be a block of text that our object is designed to encipher etc.
Container - special consideration.If you notice, this container is shared by both our object and the calling thread who is the creator of this object. This calls for some kind of synchronization object to be present between the two. Java provides the synchronized keyword, to help with this.
Making a pool of threads in Java
In this section, we consider using a thread pool in the generation and processing of cryptographic keys unique to given user names. The class MyClass from the above section is renamed as the class UserKeyGenerator. The input to this class is a string. The output is a key generated based on the input.
The sequence diagram above shows an instance of the server class named as MikeyServer. A second class named as UserKeyGenerator represents an unit of work done by this server. The server will create and manage an array thread objects. This is the pool of threads. Each thread is supplied with a reference to the task object instance.
run routine: The run routine is the heart of the task. It typically gets called, when the main thread or the creating thread calls, thread.start(). The run routine waits indefinitely until it finds a job to process in the input container. If no jobs are found, it goes to sleep. Each task will wakeup when the next quantum of time is assigned to it.
shutdown routineNotice that the task exits when it receives an interrupt signal from the main thread.
package Servers;
import java.io.*;
import java.util.*;
// Object creates and manages the thread pool
public class MikeyServer {
public final static int THREAD_COUNT = 4; // Four threads in the pool.
private static String userRqstingKeys[] = { "user01", "user02", "user03", "user04", "user05" };
private static Vector input; // INPUT to Tasks: container used to queue incoming requests
private static Vector output; // OUTPUT from T: will be used to retrieve results
private static Thread [] jobProcessors; // array to hold the pool of threads
public static void main(String [] args){
input = new Vector();
output = new Vector();
// this object will carry out the task of generating user keys
UserKeyGenerator tuk = new UserKeyGenerator(input, output);
// Creates the Threads that'll wait for new jobs in the input queue
createAThreadPool(tuk);
// adds new jobs to the INPUT vector
addJobs();
// shuts down all the threads in the pool
shutdownAllThreads();
// get the results - this is not guaranteed to print all results - why?
printResults();
}
// a pool of threads that will work on generating user keys
private static void createAThreadPool(UserKeyGenerator tuk){
// creates an array object of type Thread to hold the thread objects.
jobProcessors = new Thread[THREAD_COUNT];
// now create the thread objects and place them into our container
for(int i = 0; i < jobProcessors.length; i++){
jobProcessors[i] = new Thread(tuk);
jobProcessors[i].start();
}
}
// adds key requests to the input queue.
private static void addJobs(){
// this block adds the strings into the INPUT array
synchronized(input){
for(int i = 0; i < userRqstingKeys.length; i++){
// add a jobs
input.add(0, userRqstingKeys[i]);
// wakes up the threads to go look for new jobs
input.notifyAll();
}
} // end synchronized
}
private static void shutdownAllThreads(){
// interrupts all waiting threads forcing them to exit run() routine.
for(int i = 0; i < jobProcessors.length; i++){
jobProcessors[i].interrupt();
}
}
private static void printResults(){
synchronized(output){
for(int i = 0; i < userRqstingKeys.length; i++){
// add a jobs
if( output.size() > 0){
String jobResult = (String) output.remove(output.size() - 1);
System.out.println(jobResult + "processed");
}
}
} // end synchronized
}
} // end MikeyServer
//------------------------------------------
// Class represents a task.
//------------------------------------------
class TUserKeyGenerator implements Runnable{
List input;
List output;
TUserKeyGenerator(List input, List output){
this.input = input;
this.output = output;
}
// thread's main routine
public void run(){
String userRqst;
// loop here indefinitely looking for work!
while(true){
// access to this block is serialized using the
// monitor from the Vector object.
synchronized(input){
// pool is empty - thread is in waiting mode
while(input.isEmpty()){
try{
// waits here for notification from the main thread to exit.
input.wait();
}
catch (InterruptedException ex){
System.out.println("thread interrupted - Ending");
}
} // end while
// pool not empty
userRqst = (String) input.remove(input.size() - 1);
} // end synchronized
// do something with the input string
Results(userRqst);
} // while(true)
}
public synchronized void Results(String jobOutput){
// this block adds the results of jobs to the OUTPUT queue
synchronized(output){
// add a jobs
output.add(0, jobOutput);
}
}
} // TUserKeyGenerator
0 comments:
Post a Comment