Saturday, September 13, 2008

Case Study of a DRM Server




Introduction to DRM Solutions in the Industry

This article is a case study to create the various building-blocks of an industry-standard DRM Server. The design and building of this is proposed as an open-source project.



Goals and Non-Goals:


1. Why a DRM Server?


Studying the various security choices available and that are being made in the existing industry-standard DRM solutions is a goal. Opportunity exists to build a DRM framework and to enable various building blocks for a DRM Solution, a Keyset Store, user authorization modules, Key Distribution module.

Some of the privacy and rights debates in the industry will have to eventually sort itself out, as the market-place mechanism for digital content establishes itself. Eventually, a level of DRM that is considered acceptable to both the content providers and the users will emerge.


2. Choice of DRM Standards to include in our case study:


The majority of the deployed DRM solutions and standards have come from a few leading industry players, Adobe, Microsoft and Apple.

There is a second group of companies that is standard-setting called Open Mobile Alliance or OMA. One very good standard that has emerged from this group is known as "SmartCard DRM profile". This is a flavour of DRM standard that supports delivering content via. broadcast/multicast and unicast medium to a mobile platform. This emerges as a leading candidate for our case study.


3. Design and Requirements considerations:


a.) If some of the requirements get too mobile specific, we will always lean towards a more generic solution or a framework to support other design choices.

b.) The early goals are to study and build prototypes for the various building blocks of a DRM server. The driving goal is to arrive at a set of high-level requirements that our DRM server will eventually satisfy.


High-Level Functional blocks of a DRM Server



A brief explanation of each function is in order and is provided here,

1. Key storage

Stores keysets used to encrypt data and even other keys. This is usually a database. Perhaps, this database design may have to consider access, authorization and IT policies. However, this store must be accessible for realtime query. Perhaps, there is a layer of data accessor functionality that manages this database. This is perhaps a vital piece of the DRM server design.

2. User Authorization

Access control policy function. Normal IT policies form the base and perhaps can be made more restrictive to protect the data from prying eyes. Perhaps, this must be supplemented by applying an additional layer for confidentiality.

3. Key Distribution

This function is used to provide keys to the clients and is part of the key management function. This may involve protocols for key exchange and retirement. This function works closely with the user authorization function.

4. Key Generation

This is a function that straddles DRM Server and the Content Server. There is a need to uniquely identify the content and assign keys. The relationship between the two needs to be recorded in a database and the generated keys need to be stored into a database.

5. Content Protection

This is the function that ensures confidentiality of the data and perhaps has policies to encipher the content such as on-the-fly or pre-packaged encrypted data. This function works closely with the Key Generation function. The content can possibly be from parties external to the network.

6. IT Policies

Any network element must be goverened by IT policies. Perhaps the existing ones can be leveraged. But, some careful thought needs to go into the aspects where certificates and keys are generated and delivered over the cloud. This is non-goal for this case study.

7. HeadEnd

Since, binding the content to an eventual transport is a consideration, there are headend equipments and tools. This is a non-goal for this case study.


Our Basic Tasks

1. Analysis of the SmartCard DRM Specification to gather specific requirements for the project.

2. Analysis of basic building blocks such as Cryptographic libraries and frameworks to use for the project.

Thursday, September 11, 2008

Hands-on Keyczar cryptographic Toolkit

In this brief article, we examine a cryptographic library for java from google, hands-on.

Basic building blocks of a DRM Server

One of my reasons to evaluate this toolkit is to identify a few basic building blocks for a DRM server, which we plan to build as an open source project. The Digital Rights Management server is in the business of creating and managing keys as well as managing a store for the keysets. The other interesting aspect is the need for a DRM server to frequently rotate the keys. The KeyCZar library documentation mentions support for key rotation and versioning.

Putting the crypto library to its most basic use

In this brief article, we'll look to put the library to its most basic use. The following figure lists a few basic use cases of interest,



Encrypting and Decrypting a String

The idea is to use the toolkit to scramble a small piece of data such as "Secret Message" in two different ways. As a first choice, we need a cipher that is quite fast. Then we'll use a cipher that has other advantages, but isn't necessarily as fast. The two chosen algorithms of interest are,

A. A cipher known as AES, the expansion of which goes like, Advanced Encryption Standard. This is a quite fast and powerful algorithm.

B. A cipher known as RSA, the expansion of which will lead to the initials of its inventors. This cipher has an interesting property that is equivalent to, an ability to lock your door with one key and then open it with a totally different key that looks nothing like the original key.

The following diagram is an illustration of the above two operations to be performed.



Brief look at keyczar's developer interfaces

Keyczar is a relatively newer library in the world of libraries like OpenSSL, BSAFE etc. The stated aim is to clearly provide a simpler alternative in certain scenarios for web and other application developers.

Examining the toolkit, it comes with about 36 java classes collected in a .JAR container. In addition, there is a command line support to invoke the library using a tool interface called as Keyczartool.

Interfacing with the Keyczartool

The first thing to figure out is how to invoke this command line interface. The documentation, while quite good, doesn't tell you how to get a handle to this tool. If you examine the class library, you suddenly realize, you have direct access to this command line interface from your java program. We will see it here,




import org.keyczar.*; // package; import the jar into your project.

// get a handle to the command line interface.
KeyczarTool kt = new KeyczarTool();

try {

// creates a keystore ( a dir. that'll hold some flat files )

String kargsCreate[] =
{"create", "--location=c:\\keyczarStore", "--purpose=crypt", "--name=Test"};

kt.main(kargsCreate);

}
catch(Exception ex){
// do your logging/handling
}


Specifying the Key Store

If you examine the above code, you'll notice that,

A.) We have specified a base directory where we want the library to place all our keysets. This happens to be c:\keyczarStore. You must have this directory created on your filesystem for this command to work.

B.) KeyStore: Notice the command line syntax that is used here. This is reasonably well-documented in the wiki or a PDF documentation from the keyczar site.



As you execute the above command, you'll see the library creates a file named "meta" or "somethingmeta" in your keysotre directory. If you examine the contents of this directory, it starts with something like this,


{"name":"Test","purpose":"DECRYPT_AND_ENCRYPT","type":"AES","versions":
...}


The name of the keystore is "Test", which is set by us. This store is meant to store keys that we add for use with the symmetric cipher, AES. There is some versioning information as well.

Adding Keys to our Key Store.

Our plan is to call the library to encrypt a common string, "Secret Message". We'll then print this scrambled message. Then we can call the library to decrypt
it back to the original string.

Now, in order to do this, we need to add some keys to the key store. These keys can be then used for these scrambling and descrambling operations we plan to do. The following snippet of code accomplishes this task and adds some keys to our key store.




try {

// creates a primary key of default size & store into our key store

String kargsaddKey[] = {"addkey", "--location=c:\\keyczarStore", "--status=primary"};

kt.main(kargsaddKey);


}
catch(Exception ex){
// do your logging/handling
}


// creates an additional active key
try {

String kargsaddKey2[] = {"addkey", "--location=c:\\keyczarStore"};

kt.main(kargsaddKey2);

}
catch(Exception ex){
}



Notice, we have changed the verb from "create" used earlier to "addkey", in the parameters passed into the command line API.

If you are curious and examine your keystore location, you'll notice a couple of more key files added. The contents of these key files will naturally look like key material. You can see a sample below...


{"aesKeyString":"j1U6ReVCwtpYYbWBk4CrTA","hmacKey":{"hmacKeyString":"i77bcM9EIqPSnOHxgonOjFaDL7gQnR8YpMvF0-RVT_Y","size":256},"mode":"CBC","size":128}



There are a few points to note here,

1. The keys that are in a printable form are base64 transformed.

2. Notice the term CBC. This expands to cipher block chaining mode. This is the key mixing mode used here. This mode is quite easy to use on small blocks of data. Perhaps the design goal here is to support enciphering/decipher of relatively small messages and that might have influenced this selection. When used with larger blocks, this mode has a disadvantage in that, the encryption has to proceed serially.

3. Finally, we also see a hmacKey being stored. This is an integrity checking solution, by which the keystore tracks the keys, so they are not compromised.

A final point here. The keys that are stored here may themselves need to be encrypted. This may prevent someone unauthorized from gaining easy access to them. The library has a way to accomplish this. We won't go into that here.

Applying the cipher and back

The following block of code simply applies the keys fetched from the keystore. Notice the keys that are used is based on the directory identification.



// Encrypting the plaintext
Encrypter encrypter = new Encrypter("c:\\keyczarStore");
String ciphertext = encrypter.encrypt("Secret message");
System.out.println(ciphertext);

// decrypting the scrambled message
Crypter crypter = new Crypter("c:\\keyczarStore");
String plaintext = crypter.decrypt(ciphertext);
System.out.println(plaintext);




Applying the asymmetric RSA cipher

Notice the commandline syntax variations here to specify the asymmetric cipher,




KeyczarTool kt2 = new KeyczarTool();

// create a private key of default size
try {

String kargscreateKey[] = {"create", "--location=C:\\keyczarStore\\rsakeys", "--purpose=crypt", "--asymmetric=rsa"};

kt2.main(kargscreateKey);

}
catch(Exception ex){
}

// creates a private key
try {

String kargsaddKey[] = {"addkey", "--location=C:\\keyczarStore\\rsakeys", "--status=primary"};

kt2.main(kargsaddKey);
}
catch(Exception ex){
}

// creates a public key
try {

String kargsaddKey[] = {"pubkey", "--location=C:\\keyczarStore\\rsakeys", "--destination=C:\\keyczarStore\\rsakeys\\rsa-publickeys"};
kt2.main(kargsaddKey);
}
catch(Exception ex){
}




Now we make use of the keys from the asymmetric key store and encipher and decipher a message,




// Initialize an Encrypter with the public keys
Encrypter encrypterpki = new Encrypter("C:\\keyczarStore\\rsakeys");
String ciphertextpki = encrypterpki.encrypt("Secret Message");
System.out.println(ciphertextpki);

// Initialize a Crypter with the private keys
Crypter crypterpki = new Crypter("C:\\keyczarStore\\rsakeys");
String plaintextpki = crypterpki.decrypt(ciphertextpki);
System.out.println(plaintextpki);




Conclusions
We only scratched the surface here. Perhaps, there is more to come. There are a few observations here,

1. The library and the API is easily accessible from a java program.
2. The commandline mode presents a set of default choices, which helps to simplify the usage.

Wednesday, July 30, 2008

Thread Pools for your Server written in Java



A common design problem found in most of the servers is their need to handle hundreds and even thousands of service requests and still maintain a reasoanble performance for most of the use cases.

A good example is the Internet Information Server (IIS) found on Microsoft Windows platforms serving up ASP pages or an apache server from apache software foundation that handles incoming HTTP requests.

Problem requires Thread Recycling

Nowadays, most servers are equipped with quad-core processors and hyper-threading models with logical CPUs embedded in a core. The creation and destruction of a few threads in the context of an operating system such as MS Windows or a Linux won’t pose much of a problem for them.

However, handling a large volume of service requests eventually requires your server to spend a large portion of its time in creating and destroying threads to service these requests. You may start to notice a significant slowdown in servicing those requests. This becomes a performance bottleneck.

An expensive part of the thread creation and destruction logic is the time required to switch between user and kernel modes. As a rough approximation, on a Pentium class machine running an MS windows OS, a single API call that forces a transition from user mode to kernel mode can easily take about 600 CPU clock cycles as opposed to about 30 CPU clock cycles for a function call in an user mode.

Briefest look at Task Creation in Java

Turning your object into a Task:
If you are constructing your program using java, the built-in thread support comes in handy. Here is a rather simplified view of a class that runs as a task,



class MyClass implements Runnable{

// new incoming jobs will be placed into this bin.
List input;

MyClass(List input){

// bin setup by caller to present new jobs to our object.
this.input = input;
}

// thread's main routine
public void run(){

// loop here indefinitely looking for work!
while(true){

// new jobs ready?
while(input.isEmpty()){

// wait for a notification from the main thread to exit.
input.wait();

// found a new job in the bin
userRqst = (String) input.remove(input.size() - 1);

}

// compute the new job
doSomething(userRqst);

} // end while(true)

} // end run

} // end MyClass




Runnable Interface:Examining the above piece of code, we see, our MyClass implements an interface called Runnable which supports a lone function, run(). This API is the heart of our java thread. It is equivalent to the main() in a Java or C program.

Dynamic inputs of data into our task: The constructor of MyClass takes in a container object passed in by the calling main thread. This container may initially be empty. When a new job arrives at its door step, the main thread, which created our object can place this new job into this container. This new job may be in the form of a file name to be gzipped by our object or it can be a block of text that our object is designed to encipher etc.

Container - special consideration.If you notice, this container is shared by both our object and the calling thread who is the creator of this object. This calls for some kind of synchronization object to be present between the two. Java provides the synchronized keyword, to help with this.

Making a pool of threads in Java


In this section, we consider using a thread pool in the generation and processing of cryptographic keys unique to given user names. The class MyClass from the above section is renamed as the class UserKeyGenerator. The input to this class is a string. The output is a key generated based on the input.



The sequence diagram above shows an instance of the server class named as MikeyServer. A second class named as UserKeyGenerator represents an unit of work done by this server. The server will create and manage an array thread objects. This is the pool of threads. Each thread is supplied with a reference to the task object instance.

run routine: The run routine is the heart of the task. It typically gets called, when the main thread or the creating thread calls, thread.start(). The run routine waits indefinitely until it finds a job to process in the input container. If no jobs are found, it goes to sleep. Each task will wakeup when the next quantum of time is assigned to it.

shutdown routineNotice that the task exits when it receives an interrupt signal from the main thread.




package Servers;
import java.io.*;
import java.util.*;

// Object creates and manages the thread pool
public class MikeyServer {
public final static int THREAD_COUNT = 4; // Four threads in the pool.
private static String userRqstingKeys[] = { "user01", "user02", "user03", "user04", "user05" };

private static Vector input; // INPUT to Tasks: container used to queue incoming requests
private static Vector output; // OUTPUT from T: will be used to retrieve results
private static Thread [] jobProcessors; // array to hold the pool of threads

public static void main(String [] args){

input = new Vector();
output = new Vector();

// this object will carry out the task of generating user keys
UserKeyGenerator tuk = new UserKeyGenerator(input, output);

// Creates the Threads that'll wait for new jobs in the input queue
createAThreadPool(tuk);

// adds new jobs to the INPUT vector
addJobs();

// shuts down all the threads in the pool
shutdownAllThreads();

// get the results - this is not guaranteed to print all results - why?
printResults();
}


// a pool of threads that will work on generating user keys
private static void createAThreadPool(UserKeyGenerator tuk){

// creates an array object of type Thread to hold the thread objects.
jobProcessors = new Thread[THREAD_COUNT];

// now create the thread objects and place them into our container
for(int i = 0; i < jobProcessors.length; i++){
jobProcessors[i] = new Thread(tuk);
jobProcessors[i].start();
}
}

// adds key requests to the input queue.
private static void addJobs(){

// this block adds the strings into the INPUT array
synchronized(input){

for(int i = 0; i < userRqstingKeys.length; i++){

// add a jobs
input.add(0, userRqstingKeys[i]);

// wakes up the threads to go look for new jobs
input.notifyAll();
}
} // end synchronized
}

private static void shutdownAllThreads(){
// interrupts all waiting threads forcing them to exit run() routine.
for(int i = 0; i < jobProcessors.length; i++){
jobProcessors[i].interrupt();
}
}

private static void printResults(){

synchronized(output){

for(int i = 0; i < userRqstingKeys.length; i++){

// add a jobs
if( output.size() > 0){
String jobResult = (String) output.remove(output.size() - 1);
System.out.println(jobResult + "processed");
}
}
} // end synchronized
}

} // end MikeyServer

//------------------------------------------
// Class represents a task.
//------------------------------------------
class TUserKeyGenerator implements Runnable{
List input;
List output;

TUserKeyGenerator(List input, List output){
this.input = input;
this.output = output;
}

// thread's main routine
public void run(){

String userRqst;

// loop here indefinitely looking for work!
while(true){

// access to this block is serialized using the
// monitor from the Vector object.
synchronized(input){

// pool is empty - thread is in waiting mode
while(input.isEmpty()){

try{
// waits here for notification from the main thread to exit.
input.wait();
}
catch (InterruptedException ex){
System.out.println("thread interrupted - Ending");
}
} // end while

// pool not empty
userRqst = (String) input.remove(input.size() - 1);

} // end synchronized

// do something with the input string
Results(userRqst);
} // while(true)

}

public synchronized void Results(String jobOutput){

// this block adds the results of jobs to the OUTPUT queue
synchronized(output){

// add a jobs
output.add(0, jobOutput);
}
}
} // TUserKeyGenerator