How slow getStackTrace() is?

Today during discussion about usage of low level Java tricks I was told that Throwable.getStackTrace() is a very heavy operation. So, I decided to check this fact myself and perform benchmark comparison of Throwable.getStackTrace() with other widely used in java operations.

I wrote loop that calls some code snippets 10 million times. I believe this statistics is good enough. Here are the results.

Action	Time (milliseconds)	Comment
System.currentTimeMillis()	149
new Throwable().getStacktrace()	19081
Thread.currentThread().getStacktrace()	24334	Added later, tested on other environment, so the value here is calculated relatively to new Throwable().getStacktrace()
throwable.getStacktrace()	343
Create object and invoke its hashCode()	548
new Thread()	8898
new Thread().start()	804*1000	Used 10000 iterations here. 10 millions take ages...
file print	1647
file print (long lines)	6673

We can see that getStackTrace() is really heavy action comparing with ordinary method call. But it can be easily compared with time that is needed to print 60 characters long line to file located on the local hard drive and it is much lighter than starting a new thread.

This means that the nice formatted log record that contains the source method name is twice heavier than log record that does not print method name. But starting new thread is 40 times heavier.

Recently I found out that there is yet another way to retrieve current stack trace: Thread.currentThread().getStacktrace(). I decided to test this method relatively to new Throwable().getStackTrace() and found that it 20% heavier. Code investigation showed that the reason is probably in yet another security check (invocation of security manager) done in code of Thread.getStackTrace().

Conclusions

Throwable.getStackTrace()is relatively heavy but there are other operations that take approximately the same or even more time. So, we can use it when we need but should be careful.


Acknowledgement
I want to thank Vladi Bar On that caused me to perform this investigation.

Find bad coding practices using regular expressions

I have been using eclipse for last 8 years and very glad to be a witness of its continuous improvement. Specifically I mean its compilation warnings. Number of warnings it is able to produce is growing from version to version and it is great. I typically enable most of them and believe that it improves my code.

Unfortunately some warnings are still absent. Sometimes people use concrete classes or too specialized interfaces where interfaces should (and almost must) be used. Here are the code examples that irritate me:

// Using concrete class to the left of the assignment
ArrayList<String> a = new ArrayList<String>();


// Using concrete class into the generics
new ArrayList<ArrayList<String>>


// Using concrete class as an method parameter or return type
HashSet<Integer> foo(ArrayList<String> list){}


I personally never write such code but I know people that do.
When I see such code I want to fix it. But how find all places? Eclipse does not create warnings about any of these problems. Fortunately Eclipse supports search using regular expressions. I decided to write several regular expressions that can help to solve this problem.

Here are the expressions:



// left side of assignment
// [A-Z]\w+(List|Set|Map|Bean|Impl)\s*(<.*?>)?\s+\w+\s*=


// nested generics
// <\w+(List|Set|Map|Bean|Impl)\b[\w,\s<>]*>


// method argument
// \s+\w+\s*\([^)]*[A-Z]\w+(List|Map|Set)\b


// return type
// \w+(List|Set|Map|Bean|Impl)\s*(<.*?>)?\s+\w+\s*\(.*\{

I used them on pretty large codebase and found useful. I tried to write pattern that will be as short as it is possible and works without false negatives and with minimum false positives.


Limitations

Obviously regular expression cannot solve 100% of problems. The method assumes specific naming conventions. If for example I call my interface Worker and the class that implements it WorkerImpl this method will find expression like WorkerImpl worker = new WorkerImpl(). But it will not work for the following code sample:

class Producer implements Runnable {
    @Override
    public void Runnable() {
        // some code
    }
}


//.............................................
Producer p = new Producer();
new Thread(p).start();

Obviously that this code sample uses the instance of Producer as Runnable only and therefore should be written:

Runnable p = new Producer();

new Thread(p).start();

Unfortunately no regular expression can find this situation. 

Here is yet another limitation. Sometimes people use "wrong" interface. Here is the code sample:

List<String> list = new ArrayList<String>();
for (String s : list) {/*do somthing*/}

List is not needed here. It should be replaced by Collection:

Collection<String> list = new ArrayList<String>();
for (String s : list) {/*do somthing*/}

Why it is important? Probably in future you will decide to store the elements in Set and still be able to iterate over elements. In this case only the right part of assignment should be chaged:

Collection<String> list = new LinkedHashSet<String>();

for (String s : list) {/*do somthing*/}

It is not a problem when code that declares variable collection and iterates over it is in the same class or even method. But if the collection is create somewhre and then is passed over several layers to code that iterates over it changing method signature of 50 methods from List to Set (or, better to Collection) may take a lot of time.

Conclusions

Regular expression can help us to locate bad coding practices. Although the method cannot find all problems and sometimes produces "false negatives" it was tested on large code base an worked well enough. But "real" solution can be implemented only on IDE level. Here is a link to bug report that I created https://bugs.eclipse.org/bugs/show_bug.cgi?id=353380. I hope eclipse team will implement this suggestion.

Automatic detection of debugger

Yesterday I was debugging test case that consists of a few similar unit tests. All unit tests call some API that executes tasks with background thread. I can verify the test results only when thread is done.

The "right" solution is to call the asynchronous API and then call wait(). The thread should call notify() when it is done. The test thread will continue and validate the results. In practice I cannot modify the background thread and add notify() there. It is too deep into the application I am working on. Moreover I do not want to modify production code to help myself writing unit tests. So I decided to use "simple" solution: each my test calls the asynchronous API, then invokes Thread.sleep(100L), then verifies the results. 100 milliseconds are enough for the asynchronous task to complete.

But when I am debugging my code that is invoked by background thread I do not want the test to be terminated. Actually I want my test to sleep infinitely. I changed the parameter of sleep() many times and thought that actually I need automatic mechanism that understands that code is being debugged now and chooses sleep period automatically. I checked system properties and did not see any difference when test is running normally or being debugged.

Other idea was to try JMX. Really, Runtime MBean can help here:

ManagementFactory.getRuntimeMXBean().getInputArguments(). According to javadoc this method:

Returns the input arguments passed to the Java virtual machinewhich does not include the arguments to the main method.

So this is exactly what I need!

Here are 2 examples of return value of this method:

• Running program with remote debugger:

[-Xdebug, -Xrunjdwp:transport=dt_socket,address=8000,server=y,suspend=n]

• Debugging program under eclipse:

[-agentlib:jdwp=transport=dt_socket,suspend=y,address=localhost:49709, -Dfile.encoding=UTF-8]



The following utility method returns true if program is running under debugger and false otherwise:


public static boolean isDebugging() {
    Pattern p = Pattern.compile("-Xdebubg|jdwp");
    for (String arg : ManagementFactory.getRuntimeMXBean().getInputArguments()) {
        if (p.matcher(arg).find()) {
            return true;
        }
    }
    return false;
}

I hope that regular expression I used here is general enough to support other IDEs.

Now we can use this code when we need different behavior of our code being executed normally or being debugged.

Performance of method invocation by reflection

One day discovering code in project I am working on I found the following code:

try {
    module.getClass().getMethod(methodName, Serializable.class).invoke(module, message);
} catch (Exception e) {
    throw e;
}        





Theoretically I knew that reflection works slower than direct invocation. But how slower?
Due to this code was found in the very performance critical part of the system I decided first to perform some benchmarking. I wrote class that contains one method foo() that does nothing and implemented 3 scenarios of invocation and ran them 1 million times:

1. direct invocation
2. invocation using reflection when getMethod() was called once
3. invocation using reflection when getMethod() was called on each loop iteration. 

And here are the results.

1. direct invocation took 5 ms
2. reflection took 38 ms
3. getMethod() + reflection invocation too 435 ms

This means that reflection generally can be used even in systems that are required to perform fast if getMethod() is not done for each invocation separately. 

File access: stream vs nio channel

Java provides 2 ways to access files: using streams and NIO channels. Streams implement blocked IO: the read() method is blocked until some content is available. Channels give us ability to read content without being blocked, It allows for example to avoid allocating special thread per data source (file, socket etc.)

While this advantage is very important when communicating over sockets it probably less relevant when reading and writing files. At least the code looks pretty the same. I decided to compare performance of streams an NIO when reading and writing files sequentially.

I wrote simple program that copies file using streams and NIO. When using NIO I used 2 types of buffers: regular and direct. The utility is able to read file without writing it back to disk. This allow to compare the reading and writing speed separately. The following table shows results I got when reading file of 23MB. Evaluation time is given in milliseconds.

Operation	NIO	NIO + Direct buffer	Stream
read + write	638	392	281
read	205	87	37
write (calculated)	433	315	244

The results show quite clearly that good old streams work much faster when accessing files sequentially, especially for reading. Reading files using streams is almost 3 times faster than doing it using NIO event utilizing direct buffer.

I was surprised seeing such results and tried to find the reason.  The short JDK code investigation explained everything. FileInputStream.read() method is declared as native. So when we call it we directly use the native mechanisms of current operating system. FileChannel used to read files is an abstract class. The real implementation is in FileChannelImpl. Its read() method calls a lot of methods implemented in java, uses synchronized blocks etc. Obviously it will work slower than native method.



Conclusions

NIO gives us a lot of advantages. But it cannot completely replace the good old streams. At least sequential reading and writing of regular files works much faster when implemented using streams.

java.util.Pattern vs regular string search

Following discussion at work where I took a role of advocate of regular expressions I decided to verify how they are really fast. I remember that I have read that once Pattern is compiled (that might take some time) it works as quickly as regular string search. So, I wrote test that calls 1 million times String.contains() and Pattern.matcher().find(). Here are the results: contains test took 102 ms while pattern test took 340.


This means that patterns are almost 3.5 times slower. Therefore although regular expressions provide very convenient way for searching within strings sometimes when pattern are very simple and code is very performance critical we should use the good old methods provided by class java.lang.String.

With that I have to say that I paid attention that method indexOf() accepts String, method contains() accepts CharSequence but is implemented as  

return indexOf(s.toString())>=0. 

Pattern at the same time works directly with CharSequence. This means that if you use StringBuilder you should be careful passing StringBuilder instance  as an argument to String methods: creating new String object may cause significant performance degradation. 

Codility certificate

Recently I found out a cool site that allows programmers to verify their programming skills and recruiters to verify the programming skills of candidates. I did an exam and got silver certificate.  
http://codility.com/cert/view/cert3D8D4M-ZZC7578U6V78B8UE
 
As far as I can see each task has 2 types of solutions: n^2 that can get silver certificate as maximum and better 0^n or ln(n) solution that has a chance to get gold certificate.

Get program entry point

Today I read the following discussion: How can I determine which class's `main` method was invoked at runtime?


The problem was to distinguish the entry point to the program, i.e. which class' main was used to start the program. The suggestion was obvious:

public static getMainClassName() {
   StackTraceElement[] elem = new Exception().getStackTrace();
   return elem[elem.length - 1].getClassName();
}

This solution is fine but it is correct for main thread only. How to detect the program entry point from other thread? Fortunately we have method getAllStackTraces() available in class Thread since java 1.5. So, the solution is to iterate over all threads, detect the main thread and return last element of its stack trace:

private static String getMainClassName() {
 Map map = Thread.getAllStackTraces();
 for (Map.Entry entry : map.entrySet()) {
  Thread thread = entry.getKey();
  if ("main".equals(thread.getName()) && "main".equals(thread.getThreadGroup().getName())) {
   StackTraceElement[] trace = entry.getValue();
   return trace[trace.length - 1].getClassName();
  }
 }
 return null;
}

This method can be called from any thread in the application. But it also has its limitation. Method getAllStackTraces() returns currently running threads. If main thread was terminated before the method is called the program entry point can not be found.

Access windows registry with pure java

Introduction

Java is a cross platform language and therefore does not support platform specific features. Windows has a special unique feature - registry. So, java cannot access registry. We have to use JNI or external processes (directly or indirectly) to do this.


Solution

But recently I found a cool Java feature: preferences implemented by class java.util.prefs.Preferences. Yes, I am eating my hat now! This feature was introduced to Java 1.4 and I did not know and have never used it! What's cool in this feature? Its implementation is platform specific. As always we can access abstract class java.util.prefs.Preferences and get its instances using several static method like userRoot(), systemRoot() and others. But the concrete implementation depends on current operating system. Class WindowsPreferences uses registry to store values. It declares several native methods: WindowsRegOpenKey, WindowsRegCloseKey, WindowsRegDeleteKey etc. Sounds good, doesn't it?

So I decided to abuse this class and implement access to registry utilizing these methods. It was not very simple task. The class WindowsPreferences uses only specific registry path: Software\Java Soft\Prefs under HKLM and HKCU. Class itself is package protected and cannot be nether inherited nor called. So, reflection is the only real option.


Implementation

I called my class Registry and decided to make it singleton. It contains static enum Hive that enumerates all standard registry hives (HKLM, HKCU, HKCR etc). It implements the following public methods:

• keys(Hive hive, String path)
• values(Hive hive, String path)
• get(Hive hive, String path, String name)
• createKey(Hive hive, String path)
• put(Hive hive, String path, String name, String value)
• removeKey(Hive hive, String path)
• removeValue(Hive hive, String path, String name)

The implementation is based on invocation of private native methods declared in WindowsPreferences class using reflection:


private <R> R call(String methodName, Class[] types, Object[] args) {
  try {
    Method m = winPrefClazz.getDeclaredMethod(methodName, types);
    m.setAccessible(true);
    return (R)m.invoke(null, args);
  } catch (Exception e) {
    throw new RuntimeException(e);
  }
}



Here is the example how this method is used. This is what I would like to write:

int hresult = WindowsRegCloseKey(handle);

This is what I have to do instead:

int result = call("WindowsRegCloseKey", new Class[] {int.class}, new Object[] {handle});

Using reflection sometimes has some benefits. For example methods that enumerate registry keys and values are very similar but they call different native methods (e.g. WindowsRegEnumValue and WindowsRegEnumKeyEx). So methods childrenNamesSpi() and keysSpi() are almost duplicate in WindowsPreferences. Using reflection allows creating one implementation for both cases.



Limitations

Careful examining of the API exposed by registry class shows that it is limited to work with string values only: method put accepts value of string type, method get returns string. Unfortunately this is the limitation of current approach. Class WindowsPreferences is not designed to work with other value types. Native method WindowsRegSetValueEx works with string values only; attempt to retrieve value of other type using WindowsRegQueryValueEx returns null.



Conclusions

Java programmers are regular to use native libraries to utilize platform specific features. But sometimes some of the features are already implemented by JVM but just not exposed as a public API. Investigation of JDK classes and utilizing of reflection allows us to get access to windows registry with simple and very compact java code. The implementation deals with string values only that is a serious limitation but still can be used because most values stored in registry are strings. The suggested technique is most useful when code size is critical. For example for applications started using JNLP.

Full source code of Registry class and unit test may be found here.


References

I found this trick myself and wrote this article. Then I googled "read windows registry with java" and found at least 2 similar implementations:

http://lenkite.blogspot.com/2008/05/access-windows-registry-using-java.html
http://www.davidc.net/programming/java/reading-windows-registry-java-without-jni

I have no idea why I did not search web before writing the code... The advantage of my solution is that I implemented full API that can be used in any application while these guys just showed that this solution is possible.

Useful abuse of API

Can abuse of API be useful? Sometimes it can be!


Word "abuse" is defined as the improper usage or treatment for a bad purpose

(from Wikipedia)

Have you ever used a knife as a screw driver? A screw driver is better but if it is not available a knife can help too.

How can this principle be applied for programming?

Implementation of Command pattern

Who has never implemented command pattern?  It is simple. Define interface Command with one no-arguments method execute, create as many implementations as you need and use it. But JDK already contains 2 command like interfaces: java.util.concurrent.Callable and java.lang.Runnable, so why not to use them? Such usage has a positive side effect. You can execute this command in separate thread or in thread pool without any modification or wrapping.


Implementation of Filter pattern

Sometimes we  have to implement filter pattern. Interfaces FileFilter and FilenameFilter already exist in JDK but they deal with files. This fact does not allow their reuse for other purposes. But there is other interface java.lang.Comparable<T>. This interface is already parametrized and declares method

int compareTo(T obj)

It is not expected to be used for filter implementation. Here is the quotation from JDK javadoc:

"This interface imposes a total ordering on the objects of each class that implements it. This ordering is referred to as the class's natural ordering, and the class's compareTo method is referred to as its natural comparison method."

But what is the difference between methods boolean accept(T) and int compareTo(T)? Only semantics and result type. I suggest to use this interface to implement external filter. 0 means true, non zero value means false. Let's assume that we have collection of strings and would like to filter only 5 character long elements.

This is the implementation of Comparable interface:

public class StringLengthFilter implements Comparable<Integer> {
    private String str; 

    public StringLengthFilter(String str) {
        this.str = str;
    }

    @Override
    public int compareTo(Integer len) {
        return str.length() - len;
    }
    
    public String getString() {
        return str;
    }
}

The following code snippet counts number of 5 characters long strings in collection:

int count = 0;
for (String s : src) {
    if (new StringLengthFilter(s).compareTo(5) == 0) {
        count++;
    }
}

Passing primitive wrappers by reference

All objects are passed by reference in java. Only primitives and their wrappers are passed by value.
Argument that is passed to method by reference may be changed by this method. For example:

Collections.sort(list);

sorts the given list "in place", i.e. the method sort() changes object passed by reference. We cannot do the same with primitives.

Value of param is not changed after the following method call:

int param = 1;
foo(param); 

// param here is still 1 

//.............
void foo(int p) {
    p *= 2; 
}

Passing immutable argument to method by reference may be implemented using AtomicInteger:

AtomicInteger arg = new AtomicInteger(5);
foo(arg);
System.out.println(arg.get()); 

This code snippet will print 10. The good side effect of this method is thread safety.


Discover available implementations

Assume that we have interface and several implementations packaged in different JARs that can be available in application classpath. Sometimes application should know all available implementations. For example to choose the best one automatically or to allow the user to choose one from drop down list.
We can use java.class.path and path.separator to find the application classes, then read classes as resources to perform self discovery and locate available implementations dynamically. The following code snippet looks for all available implementations of BSFEngine interface.

    private static Map<String, Boolean> getEngines() throws Exception {
        Map<String, Boolean> result = new HashMap<String, Boolean>();
        String[] pathElements = System.getProperty("java.class.path").split(System.getProperty("path.separator"));
        for (String pathElement : pathElements) {
            File resource = new File(pathElement);
            if (!resource.isFile()) {
                continue;
            }
            JarFile jar = new JarFile(resource);
            for (Enumeration<JarEntry> e = jar.entries(); e.hasMoreElements();) {
                JarEntry entry = e.nextElement();
                if(entry.isDirectory()) {
                    continue;
                }
                if(!entry.getName().endsWith("Engine.class")) {
                    continue;
                }
                String className = entry.getName().replaceFirst("\\.class$", "").replace('/', '.');
                try {
                    if(BSFEngine.class.getName().equals(className)) {
                        continue;
                    }
                    Class<?> clazz = Class.forName(className);
                    if(BSFEngine.class.isAssignableFrom(clazz) && !clazz.equals(BSFEngine.class)) {
                        result.put(className, true);
                    }
                } catch (NoClassDefFoundError ex) {
                    // ignore...
                    result.put(className, false);
                }
            }
        }
        return result;
    }
        

The following method call

System.out.println(getEngines().toString().replaceFirst("\\{", "").replaceFirst("\\}", "").replace(", ", "\n"));

produces the following output:

org.apache.bsf.engines.xslt.XSLTEngine=true
org.apache.bsf.engines.jacl.JaclEngine=false
org.apache.bsf.engines.javascript.JavaScriptEngine=false
org.apache.bsf.engines.jython.JythonEngine=false
org.apache.bsf.engines.netrexx.NetRexxEngine=true 

So, we have discovered all implementations of BSFEngine interface available in the application classpath.


Use AbstractCollection.toString() for debugging

Method toString() typically creates human readable text representation of the object. It is not recommended to base any logic on toString() format. But there is no rule without an exception.

Debugger is a great tool that allows viewing of variable values. Unfortunately sometimes it is not powerful enough. Let's assume that you have the linked list or set that contains 10 thousand objects. You toggle breakpoint and want to check whether specific value is on the list. Good luck to find this object :(.

To save time I typically use the following technique. I create the following watch expression:

list.toString().substring(1).replaceFirst("]$", "").repalce(" , ", "\n") 

This code produces multi-line string. Each line contains string representation of the list element. This method works fine for relatively small lists. If the quantity of elements is high I put this expression into System.out.println(), so the list is printed on STDOUT where I can perform textual search.



Copy Map to Properties

Since Java 5 is released I prefer to use parametrized collections. Although class java.util.Properties is a part of java collection framework and is typically used to store strings it is not Map<String, String> but Map<Object, Object> for backwards compatibility. So, we cannot copy elements from Map<String, String> to Properties using putAll() that expects in this case Map<Object, Object>. Sometimes I have to copy content of my parametrized collection to properties (for example to use its store() method). The regular solution is to create loop that iterates over all entries of source Map and puts them to Properties object. But sometimes we can save a couple of code lines using AbstractMap.toString() and Properties.load().

Here is an example. I defined map where both keys and values are Integers:

Map<Integer, Integer> imap = new HashMap<Integer, Integer>();
imap.put(1, 10);
imap.put(2, 20); 

Now I am copying this map to properties using the following code:

Properties props = new Properties();
props.load(new StringReader(imap.toString().substring(1).replaceFirst("}$", "").replace(", ", "\n")));

Socket based singleton application

Typically we use sockets for process-to-process communication. In most cases the processes run on different computers. Usually the processes are different too: one of them is server, other is client. This example shows how application uses socket connection to communicate with other instance of itself that runs on the same machine.

Typical example of singleton application is MS Windows Task manager. You can run only one instance of this application. If you try to run the second instance the already existing one is activated even if its window was minimized before. I have implemented class SingletonApplication. It tries to connect to predefined port. Connection succeeds if the other instance of application is already running, so my class calls System.exit(1):

new Socket().connect(new InetSocketAddress(port)); 
System.exit(1);

If connection fails we assume that this is the first instance of the application and open server socket that is listening to connections from other instances.

ServerSocket serverSocket = new ServerSocket(port);
while(true) {
    serverSocket.accept(); // blocked until client connects
    activateWindow();
}

Implementation of method activateWindow() brings us to the next abuse:


Portable window activation

The goal is to bring existing window on top and select (activate) it. Java focus system is sophisticated and not simple. There are a lot of methods that provide focus control.

Component.requestFocus works for component that is displayable, visible etc.
Window.toFront brings the window to front and may make it the focused window if this window is visible. What really happens (at least on Windows) is that the button which represents the window on toolbar starts blinking but the window itself is not focused.
Window.setAlwaysOnTop(boolean) sets whether this window should always be above other windows. It does not make this window active.

I spent a lot of time but failed to find a good portable way to put window on top of others and make it active. Finally I found the following trick. First I call window.setAlwaysOnTop(true). This brings window on top of others but does not make it active. Then I move mouse to the window header and simulate left mouse click using java.awt.Robot. Then I move mouse back. It is ugly trick but it works:


mainWindow.setAlwaysOnTop(true);
Robot robot = new Robot();
robot.mouseMove(mainWindow.getX() + 40, mainWindow.getY() + 10);
robot.mousePress(InputEvent.BUTTON1_MASK);
robot.mouseRelease(InputEvent.BUTTON1_MASK);

I have implemented generic class SingletonApplication that makes each application to behave like Task manager by adding only one line to the beginning of main(String[]) method:

new SingletonApplication(12345).start();

Where 12345 is a TCP port that will be used by our application.

Full source code of SingletonApplication is available here.


Conclusions

Typically abuse is bad. But sometimes improper usage of existing mechanism, API or technique may be very useful. "Useful" abuse may save time and produce well designed working code.


Acknowledgments

I would like to thank my former manager Avshi Avital that commented one of my design solutions as "elegant abuse." Although it was several years ago and I do not remember details of that abuse, the sentence gave me an idea for this article.

Twitter

I have just opened Twitter account: https://twitter.com/alexradzin

Hierarchical structures with Java Enums

Java enums are typically used to hold array like data. This tip shows how to use enum for hierarchical structures.

Motivation

Once upon a time I wanted to create enum that contains various operating system, i.e.

public enum OsType {
            WindowsNTWorkstation,
            WindowsNTServer,
            Windows2000Server,
            Windows2000Workstation,
            WindowsXp,
            WindowsVista,
            Windows7,
            Windows95,
            Windows98,

            Fedora,

            Ubuntu,

            Knopix,

            SunOs,

            HpUx,

}

I was not satisfied of this structure because I'd like to see a group of WindowsNT that contains WinNTWorkstation and WindNT server. All windows versions should be in super group of "windows". Fedora, Knopix and Ubuntu are distributions of Linux. All Linux distributions together with SunOs and HpUx are Unix systems. Obviously that all Windows systems have common properties. The same is about Unix systems. And I hate copy/paste programming.


Class per OS Solution

The obvious solution here is to create separate classes per operating system and several abstract classes. For example class Fedora extends class Linux that extends class Unix that extends class OperatingSystem. We can enjoy all advantages of inheritance, so all common properties of Windows OS are stored in class Windows and can be overridden by its subclasses.

But now we cannot see all operating systems together, iterate over them etc., i.e. very useful features of Java enum are missing.
No problem! Now we can create enum like previous that holds custom field of type Class:

public enum OsType {
            WindowsNTWorkstation(WindowsNTWorkstation.class),
            WindowsNTServer(WindowsNTServer.class),
            Windows2000Server(Windows2000Server.class),
            Windows2000Workstation(Windows2000Workstation.class),
            WindowsXp(WindowsXp.class),
            WindowsVista(WindowsVista.class),
            Windows7(Windows7.class),
            Windows95(Windows7.class),
            Windows98(Windows98.class),

            Fedora(Fedora.class),

            Ubuntu(Ubuntu.class),

            Knopix(Knopix.class),

            SunOs(SunOs.class),

            HpUx(HpUx.class),

            ;

            private Class clazz;

            OsType(Class clazz) {

                this.clazz = clazz;

            }

}

This solution is better but it still has disadvantages:

1. Implementation of method that retrieves all "children" of specific OS (for example all Linux distributions) is hard and ineffective.
2. Grouping is separate from enum.
3. The solution is very verbose: each OS is represented by its own class even if the class has nothing to override.

Hierarchical Enum

To create hierarchy using enum we need custom field "parent" that is initialized by constructor:
public enum OsType {
    OS(null),
        Windows(OS),
            WindowsNT(Windows),
                WindowsNTWorkstation(WindowsNT),
                WindowsNTServer(WindowsNT),
            Windows2000(Windows),
                Windows2000Server(Windows2000),
                Windows2000Workstation(Windows2000),
            WindowsXp(Windows),
            WindowsVista(Windows),
            Windows7(Windows),
            Windows95(Windows),
            Windows98(Windows),
        Unix(OS) {
                @Override
                public boolean supportsXWindows() {
                    return true;
                }
            },
            Linux(Unix),
            AIX(Unix),
            HpUx(Unix),
            SunOs(Unix),
    ;
    private OsType parent = null;

    private OsType(OsType parent) {
        this.parent = parent;
    }
}

This structure allows implementation of method "is" that works like operator instanceof for classes and interfaces. For example Windows2000 is Windows, Fedora is Linux, Windows is not Unix etc.

public boolean is(OsType other) {
    if (other == null) {
        return false;
    }
   
    for (OsType t = this;  t != null;  t = t.parent) {
        if (other == t) {
            return true;
        }
    }
    return false;
}


Sometimes we need method that returns all "children" of current nodes, e.g. all Linux systems or all variants of Windows2000. The easiest way to implement this is to hold collection of children per element and fill it from constructor:


private List<OsType> children = new ArrayList<OsType>();
private OsType(OsType parent) {
    this.parent = parent;
    if (this.parent != null) {
        this.parent.addChild(this);
    }
}

Now method "children()" that returns direct node's children is trivial:
public OsType[] children() {
    return children.toArray(new OsType[children.size()]);
}

It is not hard to implement recursive method "allChildren()" that returns all children of current node (see full source code). 

But hierarchy term is always accompanied by inheritance that allows overriding methods of parent. This is the basic feature of classes in all object oriented languages. Is it possible to implement a kind of inheritance relationship for elements of one enum?

 
Overriding parent's method

Unix systems support X Window graphical environment. MS Windows does not. We would like to be able to ask OS whether is supports X Window.

We can define boolean flag "supportsX" and boolean method
public boolean supportsX() {return suppotsX;}

Now we have to add yet another argument to OsType constructor and pass true/false for each element of the enum. But it is too verbose. Is it possible to say that Unix supports X, Windows does not support X and be sure that Fedora's supportX() returns true while Winddows95's supportX() returns false?

The implementation is pretty simple. First for simplicity let's say that X Window is supported by all Unix systems and is not supported by others.
So, we can implement method supportsXWindowSystem() at enum level as following:

public boolean supportsXWindowSystem() {
    return false;
}


Now we have to override it for all Unix systems. To implement this we change the default implementation to following:

public boolean supportsXWindowSystem() {
    if (this == OsType.OS) {
        return false;
    }
    
    for (OsType t = this;  t != null;  t = t.parent()) {
        if(!t.getClass().equals(OsType.class)) {
            try {
                return (Boolean)t.getClass().
                        getDeclaredMethod(
                            "supportsXWindowSystem").
                        invoke(t);
            } catch (SecurityException e) {
                continue;
            } catch (NoSuchMethodException e) {
                continue;
            } catch (IllegalArgumentException e) {
                throw new IllegalStateException(e);
            } catch (IllegalAccessException e) {
                throw new IllegalStateException(e);
            } catch (InvocationTargetException e) {
                throw new IllegalStateException(e);
            }
        }
    }
    return OsType.OS.supportsXWindowSystem();
}

This method invokes itself using reflection starting from the current enum element and iterating over its parents until it succeeds. The trick here is in condition !t.getClass().equals(OsType.class): the reflection call happens only if enum elements really overrides the method. So, the method of first parent in hierarchy that implements the method will be used. If no one of parents and parents of parents does not implement this method itself we call method of root element. 


Now we can say the following:

...

        Unix(OS) {
                @Override
                public boolean supportsXWindowSystem() {
                    return true;
                }
            },
            Linux(Unix),
            AIX(Unix),
            HpUx(Unix),
            SunOs(Unix),
...

The method is overridden for Unix element only and all its children will use this method.

The problem is solved. We got enum based hierarchical polymorphic structure! We can implement method in base element (using it like a super class) and then override it in any element we want.

The only disadvantage of this solution is that now we have to create similar implementation for each method we add to this enum and for all other enums that hold hierarchical structure. 


Utility that helps to call hierarchical method

The utility is implemented as static method that makes more general implementation than supportsXWindowSystem(). The main difference between common utility and method implemented for specific enum is that

1. we cannot use the enum's fields directly
2. we cannot use hard-coded method name and return type.
3. we need a way to get parent of current enum element.

To solve the first problem we pass to the utility current element, the root element and root default value element value, so the method signature looks like:

public static <R> R callHierarchicalMethod(Enum<?> enumElem, Enum<?> rootElem, R rootValue, String parentAccessor)

Utility is parametrized that allows to support any return type.
The utility uses reflection. But how to discover the "real" method name that should be called? The Throwable.getStackTrace() helps us:

String methodName = new Throwable().getStackTrace()[1].getMethodName();

Element #1 is the a "real" enum method, i.e. the method that directly called the utility.

The next difference is where to use getClass() and where getDecalaringClass(). While we are iterating over the chain of elements from current to its parents we use getClass() that returns class of enum if the element does not  redefine any method and anonymous inner class if element defines at least one method. The fallback code in the end of the utility uses getDecalaringClass() to invoke the business method that is implemented on enum level itself.

Moreover we cannot just invoke
elem.getClass().getDeclaredMethod(methodName).invoke(elem)

Although the method must be accessible (better public) the class itself may be not public. For example it happens if element is presented by anonymous inner class. In this case the method can be invoked only after calling setAccessible(true).

Method m = clazz.getDeclaredMethod(methodName);
m.setAccessible(true);  
m.invoke(elem)


Finally we need a way to access parent element. I decided to use reflection for this purpose. It is easier than creating interface with one method Enum<?> getParent(Enum<?> elem). This is classical use-case for closures, so I am waiting for Java 7 to improve this code.


Now default implementation of our business method looks like:

public boolean supportsXWindowSystem() {
    return EnumUtil.<Boolean>callHierarchicalMethod(
           this, OsType.OS, false, "parent"); 
}
 


Full source code of utility can be found here. Please find OsType enum as an usage example. JUnit test case that can be used as code example as well is available also.


Conclusions
 
Although we are regular to use enums as some kind of static arrays they also can be used to present hierarchical tree-like data structures where each node can find its parent, its children and even inherit and override parent's method almost exactly as we do with class inheritance.

Send delayed JMS messages

Very often I had to implement feature that has to do something asynchronously in a minute, day, or at 5PM next Monday. Every time I implemented some serialization mechanism (typically based on DB) and some scheduled task that runs periodically, checks the table and runs tasks that should be executed now. Sometimes more generic tools were used. For example Quartz. What bothered me to implement such tasks using JMS that is built to perform asynchronous tasks? The reason is that JMS API does not allow sending delayed messages, i.e. messages that will not be received by subscriber or receiver immediately. Occasionally I found out that some JMS implementations have proprietary implementation for delayed messages. I decided to perform some search in Internet and aggregate this information in one place. Here is a list of the most popular JMS implementations (see wikipedia):

• Apache ActiveMQ
• Apache Qpid
• FUSE Message Broker (enterprise ActiveMQ)
• Mantaray a P2P JMS implementation
• OpenJMS, from The OpenJMS Group
• JBoss Messaging from JBoss
• HornetQ from JBoss
• JORAM, from the OW2 Consortium
• Open Message Queue, from Sun Microsystems
• Sun Java System Message Queue, from Sun Microsystems, supported version of Open Message Queue
• Rabbit MQ

Due to JMS API does not define interface for delayed messages most SMS providers that support this feature implemented it using message property. You just have to say something like msg.setLongProperty(“DELAY”, delay). Some implementations require casting to specific class and invocation of proprietary method. The following table summarizes differences between implementations of different SMS providers I found.

JMS provider	Implementation
Oracle AQ	msg.setIntProperty(“JMS_OracleDelay”, delay);
JBoss	msg.setLongProperty(“JMS_JBOSS_SCHEDULED_DELIVERY”, now + delay);
ActiveMQ	msg.setLongProperty(ScheduledMessage.AMQ_SCHEDULED_DELAY, delay);
OpenJMS	((org.exolab.jms.message.MessageImpl)msg).setJMSXRcvTimestamp(now + delay);
BEA Weblogic	queueConnection = queueConnectionFactory.createQueueConnection(); QueueSession queueSession = queueConnection.createQueueSession(true, 0); QueueSender queueSender = queueSession.createSender(queue); ObjectMessage jmsMsg = queueSession.createObjectMessage(message); //Casts queueSender to weblogic.jms.extensions.WLMessageProducer interface and set delivery time ((WLMessageProducer) queueSender).setTimeToDeliver(timeToDeliver); queueSender.send(jmsMsg);

Do we have solution for JMS providers that do not have native support of delayed message delivery? Yes, we do. I would like to suggest the following solution.
Send message to special queue. Let’s call it DELAYED_QUEUE. Add to delayed message the special properties:

• JMS_DESTINATION that contains name of queue or topic where this message should be finally delivered.
• DELIVERY_TIME that contains time stamp in milliseconds (now + delay).

For each enqued delayed message create scheduled task that that will run once when message should be delivered. This scheduled task will create JMS receiver with selector that looks like DELIVERY_TIME < now (where now is the timestamp), receives all expired messages and send them to real JMS destination using property JMS_DESTINATION.
Scheduled task may be implemented as resource adapter (JCA):

BootstrapContext ctx;
ctx.getWorkManager().createTimer(). schedule(new DelayedMessageTimerTask(msg), new Date(now + delay))

This is not ideal solution. It is OK for relatively small number of messages and non persisted JMS destinations. Improvements of this solution are beyond the scope of this article.

Conclusions

Most popular JMS implementations support delayed delivery of messages. Even if this feature is not supported we can always implement it using additional queue and scheduled task.
This article is published at DZone.

Customized ValueOf

When I am writing enum I very often found myself implementing static method similar to standard enum’s valueOf() but based on field other than name:

public static TestOne valueOfDescription(String description) {
    for (TestOne v : values()) {
        if (v.description.equals(description)) {
            return v;
        }
    }
    throw new IllegalArgumentException(
    "No enum const " + TestOne.class + "@description." + description);
}

where “description” is yet another String field in my enum. And I am not alone. See for example this beautiful article.
Obviously this method is very ineffective. Every time it is invoked it iterates over all members of enum. Here is the improved version that uses cache:

 private static Map map = null;
 public static TestTwo valueOfDescription(String description) {
  synchronized(TestTwo.class) {
   if (map == null) {
    map = new HashMap();
    for (TestTwo v : values()) {
     map.put(v.description, v);
    }
   }
  }

  TestTwo result = map.get(description);
  if (result == null) {
         throw new IllegalArgumentException(
                 "No enum const " + TestTwo.class + "@description." + description);
  }

  return result;
 }

It is fine if we have only one enum and only one custom field that we use to find the enum value. But if number each one of 20 enums has 3 such fields the code will be very verbose. As far as I dislike copy/paste programming I have implemented utility that helps to create such methods. I called this utility class ValueOf. It has 2 public methods:

public static <T extends Enum<T>, V> T valueOf(Class<T> enumType, String fieldName, V value);

that finds required field in specified enum. It is implemented utilizing reflection and uses hash table initialized during the first call for better performance. Other overridden valueOf() looks like:

public static <T extends Enum<T>> T valueOf(Class<T> enumType, Comparable<T> comparable);

This method does not cache results, so it iterates over enum members on each invocation. But it is more universal: you can implement comparable as you want, so this method may find enum member using more complicated criteria.
Full code with examples and JUnit test case is available here.

Conclusions

Java Enums provide ability to locate enum member by name. This article describes utility that makes it easy to locate enum members by any other field.

Dynamic Enums

Introduction

Enums introduced to Java 1.5 is a very useful and well known feature. There are a lot of tutorials that explain enums usage in details (e.g. the official Sun’s tutorial). Java enums by definition are immutable and must be defined in code. In this article I would like to explain use case when dynamic enums are needed and how to implement them.

Motivation

There are 3 ways to use enums:

1. direct access using the enum value, e.g. Color.RED
2. access using enum name, e.g. Color.valueOf("RED")
3. get enumeration of all enum values using values() method, e.g. Color.values()

Sometimes it is very convenient to store some information about enum in DB, file etc. In this case the enum defined in code must contain appropriate values.
For example let’s discover the enum Color:
enum Color {RED, GREEN, BLUE;}
Let’s assume that we would like to allow user to create his custom colors. So, we have to handle table “Color” in database. But in this case we cannot continue using enum Color: if user adds new color (e.g. YELLOW) to DB we have to modify code and add this color to enum too.
OK, we can refuse to use enum in this case. Just rename enum to class and initialize list of colors from DB. But what if we already have 100 thousand lines of code where methods values() and valueOf() of enum Color are used? No problem: we can implement valueOf() and values() manually for the new class “Color.”
Now is the problem. What if we have to refactor 12 enums like Color? Copy/Paste the new methods to all these classes? I would like to suggest other, more generic approach.

Making enums dynamic

Enum is compile time feature. When we create enum Foo, class Foo that extends java.lang.Enum is generated for us automatically. This is the reason why enum cannot extend other class (multiple inheritance is not supported in Java). Moreover some compiler’s magic prevents attempt to manually write class that extends java.lang.Enum.
The only solution is to write yet another class similar to Enum. I wrote class DynaEnum. This class mimics functionality of Enum but it is regular class, so it can be inherited. Static hash table holds elements of all created dynamic enums.
My DynaEnum contains generic implementation of valueOf() and values() done using reflection, so users of this class do not have to implement them.
This class allows relatively easy refactoring for use case described above. Just change enum to class, inherit it from DynaEnum and write code to initialize members.

Example

Source code of examples can be found here.
I implemented 2 examples: DigitsDynaEnum that does not have any added value relatively to enum. It is implemented mostly to check that base class functionality works. DigitsDynaEnumTest contains several unit tests.
PropertiesDynaEnum is a dynamic enum that reads its members from properties file. Its subclass WritersDynaEnum reads list of famous writers from properties file WritersDynaEnum.properties.
The goal is implemented! We have class that mimics functionality of enum but is absolutely dynamic. We can change list of writers without re-compiling the code. Therefore we can actually store the enum values everywhere and change “enums” at runtime.
It is not a problem to implement something like “JdbcDynaEnum” that reads values from database but this implementation is out of scope of this article.

Limitations

The solution is not ideal. It uses static initialization that could cause problems in multi-class loaders environment. Members of dynamic enums obviously cannot be accessed directly (Color.RED) but only using valueOf() or values(). But still in some cases this technique may be very useful.

Conclusions

Java enum is a very powerful feature but it has serious limitation. Enum values are static and have to be defined in code. This article suggests trick that allows to mimic enum functionality and store “enum” values separately from code.

Discover type of parametrized class

Introduction

Generics is a beautiful feature of Java 1.5. But it has some limitations. One of them is that it is impossible to discover type of parametrized class. For example if you have generic code like:

List<?> mylist = getList();

where getList() is can return either List<String> or List<Integer> there is not convenient way to discover it. Class Class and reflection API do not provide ability to discover the type of parameter. The reason is simple: generics is the compiler’s feature. The class’ parameters are not stored in .class file and therefore cannot be accessed at runtime. Sometimes we can use workarounds. For example in case of list we can retrieve the first element and then call getClass(). Obviously this method will not work for empty lists and for null elements.

Use abstract class

Other workaround exists if we develop both sides: the discovered class and the code that discovers it. Assume that we have interface Foo:

interface Foo<T> {
    void foo(T t);
}

Assume also that we want to implement framework that deals with instances of classes implementing Foo and this framework should know the concrete type of T. One solution is to add special method that returns T.

interface Foo<T> {
    void foo(T t);
    T getType();
}

Each class that implements Foo must now implement 2 methods. I think that this design may be improved. We can move this functionality to abstract class:

public abstract class AbstractFoo<T> implements Foo<T> {
    private Class<T> t;

    protected AbstractFoo(Class<T> t) {
        this.t = t;
    }

    public Class<T> getType() {
        return t;
    }
}

The abstract class defines protected constructor that receives parameter of type Class, so all its sub classes must call it.

public class FooImpl extends AbstractFoo<String> {
    public FooImpl() {
        super(String.class);
    }

    public void foo(String s) {
        // the implementation...
    }
}

I think that this approach is better than implementing getType() in each sub-class separately because sub-class is dedicated on its own task while abstract class cares cares to provide information about the type to framework. Constructor is typically written in the beginning of the class, i.e. near the definition of class itself and therefore it is easier to see mistakes.

Real example

Here is more complicated example. Several year ago I worked for small start up company named NLayers that implemented network discovery solution. We connected to remote devices using various protocols that were represented in application by hierarchy of interfaces extended from top level interface Session. Logic of discovery itself was implemented in relatively small classes we called “probes.” All these classes implemented interface Probe. The framework decided which probe should be executed and when. The framework should know the Session type supported by probe. There were many types of session and parallel hierarchy of abstract probes: ShellSession and ShellProbe, SnmpSession and SnmpProbe etc. The years passed since that time and now I’d like to suggest improved design with only one AbstractProbe. The design uses ideas described above. The code snippets here are not stolen from my former employer but written yesterday from scratch. Real code was more complicated and contained more details irrelevant for this example.

Acknowledgments

I wold like to thank my former colleagues at NLayers: Avshi Avital, Oran Epelbaum, Chaim Linhart, Yariv Bandiel, David Resnik. I enjoyed working with them and develop product that gave me idea for this article.