Introducing NIO.2 (JSR 203) Part 5: Watch Service and Change Notification

For long time Java developers used in-house developed solutions to monitor the file system for changes. Some developed general purpose libraries to ease the task of others who deal with the same requirement.  Commercial and  free/ open source libraries like, and among others. Java 7 comes with NIO.2 or JSR 203 which provides native file system watch service.

The  watch service provided in Java 7  uses the underlying file system functionalities to watch the file system for changes, so if we are running on Windows, MacOS or Linux… we are sure that the watch service is not imposing polling overhead on our application because the underlying OS and file system provides the required functionalities to allow Java to register for receiving notification on file system changes. If the underlying file system does not provide the watch-ability, which I doubt it for any mainstream file system,  Java will fall back to some rudimentary polling mechanism to keep the code working but the performance will degrade.

From the mentioned libraries the jpathwatch API is the same as the Java 7 APIs to make it easier to migrate an IO based application from older version of Java to Java 7 when its time arrives.

The whole story starts with WatchService which we register our interest in watching a path using it. The WatchService itself is an interface with several implementatins for different file system and operating systems.
We have four class to work with when we are developing a system with file system watch capability.
  1. A Watchable: A watchable is an object of a class implementing the Watchable interface. In our case this is the Path class which is the one of the central classes in the NIO.2
  2. A set of event types: We use it to specify which types of events we are interested in. For example whether we want to receive creation, deletion, … events. In our case we will use StandardWatchEventKind which implements the WatchEvent.Kind<T>.
  3. An event modifier: An event modifier that qualifies how a Watchable is registered with a WatchService. In our case we will deal with nothing specific up to now as there is no implementation of this interface included in the JDK distribution.
  4. The Wacher: This is the watcher who watch some watchable. In our case the watcher watches the File System for changes. The abstract class is java.nio.file.WatchService but we will be using the FileSystem object to create a watcher for the File System.

Now that we know the basics, let’s see how a complete sample will look like and then we will break down the sample into different parts and discuss them one by one.

import java.nio.file.FileSystem;
import java.nio.file.FileSystems;
import java.nio.file.Path;
import java.nio.file.StandardWatchEventKind;
import java.nio.file.WatchEvent;
import java.nio.file.WatchKey;
import java.nio.file.WatchService;
import java.util.List;
import java.util.logging.Level;
import java.util.logging.Logger;

public class WatchSer {
    public static void main(String args[]) throws InterruptedException {
        try {
            FileSystem fs = FileSystems.getDefault();
            WatchService ws = null;
            try {
                ws = fs.newWatchService();
            } catch (IOException ex) {
                Logger.getLogger(WatchSer.class.getName()).log(Level.SEVERE, null, ex);
            Path path = fs.getPath("/home/masoud/Pictures");
            path.register(ws, StandardWatchEventKind.ENTRY_CREATE, StandardWatchEventKind.ENTRY_MODIFY, StandardWatchEventKind.OVERFLOW, StandardWatchEventKind.ENTRY_DELETE);

            WatchKey k = ws.take();

            List> events = k.pollEvents();
            for (WatchEvent object : events) {
                if (object.kind() == StandardWatchEventKind.ENTRY_MODIFY) {
                    System.out.println("Modify: " + object.context().toString());
                if (object.kind() == StandardWatchEventKind.ENTRY_DELETE) {
                    System.out.println("Delete: " + object.context().toString());
                if (object.kind() == StandardWatchEventKind.ENTRY_CREATE) {
                    System.out.println("Created: " + object.context().toString());
        } catch (IOException ex) {
            Logger.getLogger(WatchSer.class.getName()).log(Level.SEVERE, null, ex);

At the beginning of the code we create a FileSystem object and then create a WatchService for the file system underneath the FileSystem object we created previously.

            FileSystem fs = FileSystems.getDefault();
            WatchService ws = null;
            try {
                ws = fs.newWatchService();
            } catch (IOException ex) {
                Logger.getLogger(WatchSer.class.getName()).log(Level.SEVERE, null, ex);

At the next step we create a Path object which basically is our watchable, what we want to watch, and then register it with the WatchService object we created in the previous step.

  Path path = fs.getPath("/home/masoud/Pictures");
   path.register(ws, StandardWatchEventKind.ENTRY_CREATE, StandardWatchEventKind.ENTRY_MODIFY, StandardWatchEventKind.OVERFLOW, StandardWatchEventKind.ENTRY_DELETE);

Next we get a key which represents the registration of a watchable with a watch service.
This WatchKey object represents our access door to events propagated by the WatchService object.

After getting the key, we can poll for the events arrived at the key. An File system event may get represented by multiple WatchService events. For example a rename is represented by deletion and creation events.

 WatchKey key = ws.take();
 List> events = key.pollEvents();

Now that we have the list of events we can iterate over the events list and see whether we are intrested in the event or not.

for (WatchEvent object : events) {
                if (object.kind() == StandardWatchEventKind.ENTRY_MODIFY) {
                    System.out.println("Modify: " + path.toRealPath(true)+"/"+ object.context().toString());

                if (object.kind() == StandardWatchEventKind.ENTRY_DELETE) {
                    System.out.println("Delete: " +  path.toRealPath(true)+"/"+ object.context().toString());
                if (object.kind() == StandardWatchEventKind.ENTRY_CREATE) {
                    System.out.println("Created: " +  path.toRealPath(true)+"/"+ object.context().toString());

In our sample code we are running the watch service in the application thread while we can use multiple threads to handle the events. One reason that the WatchService is not implemented using the Listener is the urge for using multiple threads to deal with the events in cases where there are thousands of events or event processing take longer than application threshold.
You can grab the latest version of Java 7 aka Dolphin from here and from the same page you can grab the latest JavaDoc which is generated from the same source code that is used to generate the binary bits. Other entries of this series are under the NIO.2 tag of my weblog.

Introducing NIO.2 (JSR 203) Part 4: Changing File System Attributes and Permissions

This is the 4th installment of my entries covering NIO.2. In this entry I will discuss more about what we can do with attributes and permissions.  The NIO.2 lets us write the permissions and attributes of a file in addition to reading them. For example we can change the rwx permissions using the Attributes utility class and  PosixFilePermission.

You can execute the following sample code with almost no changes, the only thing you need to change is the path to our sample file and then you are ready to see how it works.

import java.nio.file.*;
import java.nio.file.attribute.*;
import java.util.*;

public class Perm2 {

    public static void main(String args[]) throws IOException {
        FileSystem fs = FileSystems.getDefault();
        Path p = fs.getPath("/home/masoud/dl");

        // Reading attributes using attributes views.
        // here we read both basic and posix attributes of a file
        Map att = (Map) p.readAttributes("posix:*, basic:*", LinkOption.NOFOLLOW_LINKS);
        // printing the attributes to output
        Set en = att.keySet();
        for (Iterator it = en.iterator(); it.hasNext();) {
            String string =;
            System.out.println(string + ": " + att.get(string));
        System.out.println("-------------Attributes printing finished----------");

        // definind a new permission set for our file
 st2 = PosixFilePermissions.fromString("rwxrwxrwx");
        // Setting the attribute using Attributes utility class
        Attributes.setPosixFilePermissions(p, st2);

        // looking up and creating a principal for the given user. If use does
        // not exists it will throws a UserPrincipalNotFoundException
        UserPrincipal up = fs.getUserPrincipalLookupService().lookupPrincipalByName("masoud");

        // Setting the owner to the owner we just looked up.
        // We should have enough permisison to change the owner otherwise it will
        // throw a FileSystemException: /home/masoud/a: Operation not permitted sort of thing
        Attributes.setOwner(p, up);

        //another way to read and write the owner value for a file is using FileOwnerAttributeView
        FileOwnerAttributeView fo = p.getFileAttributeView(FileOwnerAttributeView.class, LinkOption.NOFOLLOW_LINKS);

        //Now that we have changed the permissions lets see the permissions again:
        att = (Map) p.readAttributes("posix:permissions", LinkOption.NOFOLLOW_LINKS);
        System.out.println("New Permissions:" + ": " + att.get("permissions"));

Let’s see what we have done starting from top: at the beginning we have just initialized the FileSystem object and got a file reference to /home/masoud/dl. Then we read all of this files basic and posix attributes using the attributes view notation. The following sample code shows this step

 FileSystem fs = FileSystems.getDefault();
        Path p = fs.getPath("/home/masoud/dl");

        // Reading attributes using attributes views.
        // here we read both basic and posix attributes of a file
        Map att = (Map) p.readAttributes("posix:*, basic:*", LinkOption.NOFOLLOW_LINKS);
        // printing the attributes to output
        Set en = att.keySet();
        for (Iterator it = en.iterator(); it.hasNext();) {
            String string =;
            System.out.println(string + ": " + att.get(string));
System.out.println("-------------Attributes printing finished----------");

Next step we have use the PosixPermissions class to create a permission set using the OS permissions presentation and then applied these permissions on our file

 // definind a new permission set for our file
 st2 = PosixFilePermissions.fromString("rwxrwxrwx");
        // Setting the attribute using Attributes utility class
        Attributes.setPosixFilePermissions(p, st2);

In the next step we created a user principal for a user named masoud and changed the ownership of our file to this user. We have done this using both available methods.

// looking up and creating a principal for the given user. If use does
        // not exists it will throws a UserPrincipalNotFoundException
        UserPrincipal up = fs.getUserPrincipalLookupService().lookupPrincipalByName("masoud");

        // Setting the owner to the owner we just looked up.
        // We should have enough permisison to change the owner otherwise it will
        // throw a FileSystemException: /home/masoud/a: Operation not permitted sort of thing
        Attributes.setOwner(p, up);

        //another way to read and write the owner value for a file is using FileOwnerAttributeView
        FileOwnerAttributeView fo = p.getFileAttributeView(FileOwnerAttributeView.class, LinkOption.NOFOLLOW_LINKS);

Finally we read the file permissions to see what has changed after we applied this new permissions on our file. using the following snippet.

 //Now that we have changed the permissions lets see the permissions again:
        att = (Map) p.readAttributes("posix:permissions", LinkOption.NOFOLLOW_LINKS);
        System.out.println("New Permissions:" + ": " + att.get("permissions"));

The output of running the above sample code is shown below:

lastModifiedTime: 2010-07-06T23:15:32Z
fileKey: (dev=805,ino=1922268)
isDirectory: false
lastAccessTime: 2010-07-11T14:15:46Z
isOther: false
isSymbolicLink: false
owner: masoud
isRegularFile: true
creationTime: null
group: dip
size: 219
-------------Attributes printing finished----------

You may ask what is the attribute view notation and what is meaning of that posix:permissions or posix:*. The first one means that we want to retrieve the permissions property of the file while the second one means that we want to retrieve all attributes of the file which can be seen using the posix view.

Now you are asking where you can found list of this attributes view notations, then you question is very right. The following links shows what attributes are available to read using each one of those views. Each attribute is one of the properties included in the interface.

Sure we have other views, you can find them in the JavaDocs linked above.

Next entry will be the last entry covering File System functionalities of the NIO.2 including the watch and change notification service and the ACL list. After that we will look at new features included for IO developers. You can find other entries of this series under the NIO.2 tag of my weblog.

Introducing NIO.2 (JSR 203) Part 3: File System Attributes and Permissions support in NIO.2

In two previous entries I covered Introducing NIO.2 (JSR 203) Part 1: What are new features? and Introducing NIO.2 (JSR 203) Part 2: The Basics In this entry I will discuss Attributes introduced in NIO.2. Using attributes we can read platform specific attributes of an element in the file system. For example to hide a file system in DOS file system or to check the last access date of a file in a UNIX machine.

Using NIO.2 we can check which attributes are supported in the platform we are running on and then we can decide how to deal with the available attributes. Following sample code shows how we can detect the available attributes and then how to manipulate them.

  FileSystem fs = FileSystems.getDefault();
  Path p = fs.getPath("/home/masoud/");
 //checking available attributes:
  Set<String> supportedViews = fs.supportedFileAttributeViews();
 //We always have at least one member in the set, the basic view.
  BasicFileAttributes ba = p.getFileAttributeView(BasicFileAttributeView.class, LinkOption.NOFOLLOW_LINKS).readAttributes();
 //Printing some basic attributes
   System.out.println(p.toString() + " last access:  " + ba.lastAccessTime());
   System.out.println(p.toString() + " last modified " + ba.lastModifiedTime());
        // As I know I am in NIX machine I access the unix attributes.
        // If I didnt I should have iterate over the set to determine which
        // attributes are supported
        if (supportedViews.contains("unix")) {
            PosixFileAttributes pat = Attributes.readPosixFileAttributes(p, LinkOption.NOFOLLOW_LINKS);

I placed plethora of comments on the code so reading and understanding it get easier.

In the next snippet we will see how we can read permissions of file system element. The first step in checking permissions is using the checkAccess method as shown below. the method throw an exception if the permission is not present or it will execute with no exception if the permission is present.

 FileSystem fs = FileSystems.getDefault();
 Path p = fs.getPath("/home/masoud/");
    try {
            // A method to check the access permissin
        } catch (IOException ex) {
            Logger.getLogger(perm.class.getName()).log(Level.SEVERE, null, ex);

        // Extracting all permissions of a file and iterating over them.
        //I know that I am dealing with NIX fs so I go directly with that attributes
        // otherwise we should check which attributes are supported and then we can
        // use them.

        PosixFileAttributes patts = Attributes.readPosixFileAttributes(p, LinkOption.NOFOLLOW_LINKS);
        Set<PosixFilePermission> st = patts.permissions();
        for (Iterator<PosixFilePermission> it = st.iterator(); it.hasNext();) {

        // Using PosixFilePermissions to convert permissions to different representations

As you can see in the code we can use the helper class to convert the permission set to a simpl OS represeted permission of the element. for example the set can be translated to rwx——  if the file has owner read, write and execute permissions attached to it. The helper calss can convert the os represenation of the permission to the permissions set for later use in other nio classess or methods. In the next entry I will conver more on permissions and security by tackling the Access Control List (ACL) support in the nio.2

Introducing NIO.2 (JSR 203) Part 2: The Basics

In this part we will discuss the basic classes that we will work with them to have file system operations like copying a file, dealing with symbolic links, deleting a file, and so on. I will write a separate entry to introduce classes which are new to Java 7 for dealing with streams and file contents, watching service and directory tree walking. If you want to know what are new features in Java SE 7 for dealing with IO take a look at   Introducing NIO.2 (JSR 203) Part 1: What are new features?

Before NIO.2, dealing with file system was mainly done using the File class and no other base class was available. In NIO.2 it there are some new classes at our disposal to take advantage of their existence to do our job.

FileSystems: Everything starts with this factory class. We use this class to get an instance of the FileSystem we want to work on. The nio.2 provides a SPI to developed support for new file systems. For example an in-memory file system, a ZIP file system and so on. Following two methods are most important methods in FileSystems class.

  1. The getDefault() returns the default file system available to the JVM. Usually the operating system default files system.
  2. The getFileSystem(URI uri) returns a file system from the set of available file system providers that match the given uir schema.

Path: This is the abstract class which provides us with all File system functionalities we may need to perform over a file, a directory or a link.

FileStore: This class represents the underplaying storage. For example /dev/sda2 in *NIX machines and I think c: in windows machines. We can access the storage attributes using  FileStoreSpaceAttributes object. Available space, empty space and so on.

Following two sample codes shows how to copy a file and then how to copy it.

public class Main {

    public static void main(String[] args) {

        try {
            Path sampleFile = FileSystems.getDefault().getPath("/home/masoud/sample.txt");
            sampleFile.createFile(); // create an empty file
            sampleFile.copyTo(FileSystems.getDefault().getPath("/home/masoud/sample2.txt"), StandardCopyOption.COPY_ATTRIBUTES.REPLACE_EXISTING);
            // Creating a link
            Path dir = FileSystems.getDefault().getPath("/home/masoud/dir");
        } catch (IOException ex) {
            Logger.getLogger(Main.class.getName()).log(Level.SEVERE, null, ex);

And the next sample shows how we can use the FileStore class. In this sample we get the underlying store for a file and examined its attributes. We can an iterator over all available storages using FileSystem.getFileStores() method and examine all of them in a loop.

 public class Main {

    public static void main(String[] args) throws IOException {
        long aMegabyte = 1024 * 1024;
        FileSystem fs = FileSystems.getDefault();
        Path sampleFile = fs.getPath("/home/masoud/sample.txt");
        FileStore fstore = sampleFile.getFileStore();
        FileStoreSpaceAttributes attrs = Attributes.readFileStoreSpaceAttributes(fstore);
        long total = attrs.totalSpace() / aMegabyte;
        long used = (attrs.totalSpace() - attrs.unallocatedSpace()) / aMegabyte;
        long avail = attrs.usableSpace() / aMegabyte;
        System.out.format("%-20s %12s %12s %12s%n", "Device", "Total Space(MiB)", "Used(MiB)", "Availabile(MiB)");
        System.out.format("%-20s %12d %12d %12d%n", fstore, total, used, avail);


In next entry I will discuss how we can manage file attributes along with discussing the security features of the nio.2 file system.

Introducing OpenESB from development to administration and management.

OpenESB project initiated by Sun Microsystems to develop and deliver a high performance, and feature rich implementation of Java Business Integration (JBI) under an open source friendly license. Basic task of JBI implementations is connecting different type of resources and applications together in a standard and none intrusive way. Basic building blocks of an ESB includes the Bus which is a message driven engine and transition path which deliver the required message to designated binding components. These components are the second main set of artifacts which an ESB has. The main tasks of binding components is receiving the Bus standard format messages and deliver them to other software in a format that the particular software can use. The third set of artifacts is Service engines which can relates the Bus to other types of containers like Java EE containers, transform the messages before they enter the Bus, perform routing related activities over the Bus messages.

The Main concept behind the JBI was around for quite some times but in recent years several factors helped with this concept to further evolve. First the increasing integration problems, second the SOA acceptance level between companies and finally introduction of JSR-208 specification by JCP which provides a common interfacing mechanism for JBI artifacts.

The increasing problem is nothing but the urge for integrating different software which is running in different department of the same enterprise or different partners across enterprises. An ESB implementation can be a non intrusive enabler product for integrating different software running in different department or enterprises without or with very small amount of changes in the software itself.

The concept which helped ESB to become very well known and popular is the Service Oriented Architecture (SOA) in which an ESB can play the enabler role because of its modularity and loose coupling nature.

1 Introducing OpenESB

OpenESB project located at initiated in 2005 when Sun Microsystems released a portion of its ESB products under CDDL license. Sun Microsystems, the user community and some other third party companies are working together on OpenESB project. However, the main man power and financial support comes from Sun Microsystems.

The project goal is development and delivery of a high performance, feature rich and modular JBI compliant ESB (the runtime) and all required development phase tools which accelerates and facilitate developing applications based on OpenESB capabilities and features.

The runtime components use GlassFish application server’s components like Metro web services stack, CLI and administration console infrastructure, and so on. OpenESB uses NetBeans IDE as the foundation for providing development and design time artifacts for its it runtime components. Design time functionalities are provided in a modular manner on top of NetBeans modularity system.

1.1 Introducing JBI

The JBI defines a specification for assembling integration components to create integration solutions that enable different software (external system) to actively integrate on providing a solution based on separate capabilities which every one of that software can provide on itself. These integration components are plugged into a JBI environment and can provide or consume services messages which lead in providing or consuming a service through the shared communication channel. The shared communication channel can route messages between different components and it also can transform the messages in a pre-defined or rule based way. In addition to transformation, different other base services are provided by the JBI environment. We call the environment the Service Bus or Enterprise Service Bus.

Based on the JBI specification and architecture a component is the main part of a JBI implementation which users or developers should understand. These components can acts as consumer or producers. When a component acts as consumer it receives a message from the service bus in a normalized and standard format. Then the component delivers the message to the external system in a format that the external system understands. When a component acts as a consumer to the service bus, it receives the message in a format provided by external system and converts it to normalized XML before sending it into the service bus. We call these components as binding components because they bind a external system to the service bus.

The JBI environment

JBI defines two types of components which JBI developers can develop and plug into a JBI container. However these two types are standardized and certainly each vendor has many internal and none standard interfaces, components, and a specific component model for its internal system. These Two types of components which can be plugged into a JBI environment include:

  • Service engines provide logic in the environment, such as transformation, scripting language support, ETL and CEP support, or BPEL and so on.

Binding components are adapters which can connect an external system to the service bus. For example a JMS binding component can connect a JMS message broker to the service bus, a mail server component can bind an emailing system to the bus to make it possible for other binding components to send email or receive emails through the bus.

Figure 1 shows the overall architecture of OpenESB from the specification point of view. The architecture may differ from implementation to implementation.

Figure 1 The OpenESB architectural diagram from JBI perspective

The most important fact in the JBI environment in message delivery and content delivery usually introduces requirements like transaction support, naming support, and so on. The internal square illustrates the OpenESB overall environment while the external boxes represent systems external to the OpenESB runtime. Message format inside the square is XML while the format between the binding components and external system is external system oriented. Every component either Service engine or Binding component register the service that they provide by WSDL descriptor.

1.2 OpenESB versions and editions

OpenESB like GlassFish has several concurrent versions which are either in active development mode, stable or in maintenance mode. OpenESB version one is in maintenance mode and the only updates are possible bug fixes, no new feature will be added to this version.

OpenESB version 2 is the current active version which is under development. OpenESB version 3 named project Fuji is under active design and development. Through the project Fuji Sun Microsystems will develop the next version of OpenESB which is fully based on a modular system similar to GlassFish version 3.

Like OpenSSO and GlassFish which have which have commercially packaged version supported by Sun Microsystems, there is a commercially packaged and supported package for OpenESB named GlassFish ESB. The only different between OpenESB and GlassFish ESB is that GlassFish ESB is a package which contains stable components of the OpenESB along with Sun Microsystems support behind it.

If you are looking for latest bug fixes, latest new features and components you should go with OpenESB, if you want a stable version then you should go with GlassFish ESB. Both versions are free and you can use them commercially without paying any license fee. The difference is that Sun provides commercial level support for GlassFish ESB packages.

All versions of OpenESB including GlassFish ESB, Project Fuji, and OpenESB development builds are available at

1.3 OpenESB Architecture

OpenESB version 2 closely follows the specification and has no extendibility or pluggibility beyond the JBI proposed components. OpenESB JBI container is tightly integrated with GlassFish application server to provide the overall running ESB. The integration between GlassFish application server and OpenESB results in integrated administration console in all 3 major administration channel including JMX and AMX, CLI and web based administration console.

The most important part of an ESB is the message broker and its Web services stack. OpenESB uses Metro Web service stack which is proven enough to be used by some other vendors as their web services stack and OpenMQ as the message broker. OpenMQ can be considered one of the top level message broking systems in term of performance and capabilities.

1.4 OpenESB supported standards and features

When it come to list an ESB features and supported protocols and format we can list them under five different categories including: The over all performance, the reliability and scalability of the system, available service engines, available binding components and finally administration and management channel effectiveness.

Certainly there are other factors which can not deeply discuss here, factors like: vendor support level, quality of available components and service engines. Providing benchmarks which shows how good the product works and so on. But, to some extent, we can discuss five primary attributes briefly.


The most important part of an ESB is the messaging system; OpenESB uses OpenMQ as the underlying messaging system. In different benchmarks and uses cases OpenMQ proved to perform very well.

The other important component which affects an ESB performance is Web services stack which is used to develop the underlying WSDL-SOAP communication. The Metro framework which forms the underlying Web services stack of OpenESB is very well featured with an outstanding performance. IBM uses some part of Metro in their JDK, BEA uses Metro stack for its WebLogic application server and so on.

Available service engines

Functional feature of an ESB are directly depended on the functionality of its service engines and binding components. OpenESB has a very rich set of Service engines which are either developed directory by Sun Microsystems or one of OpenESB project partners. Table 1 shows the list of most important service engines along with a description of each service engine.

Table 1 Important service engines available through project OpenESB

Service Engine



A WS-BPEL 2.0 compliant engine that provides business process orchestration capabilities.


An engine for Complex Event Processing (CEP) and Event Stream Processing (ESP)

Java EE SE

Makes the whole EE environment available as a JBI component so that EJBs can communicate with other JBI components through the NMR and vice versa.


The Worklist Manager is a JBI based engine providing task management and human intervention in a business process.

Data Integrator / ETL SE

Performs extract – transform – load to build DataWarehouses, or can be used for Data Migration.

Data mashup

Provides a single view of data from heterogeneous sources, including static web pages and tabular data exposed as web services. Provides options to join/aggregate/clean data, and generate a response in a WebRowSet schema.


An engine for applying XSLT transformations with basic orchestration capabilities.

All of these service engines have the runtime and design time components included. Some of them are not commercially stable enough to be included in GlassFish ESB and in case of requirement we should download and install them manually. The nightly builds are available at

We will discuss BPEL SE in more details in subsequent sections.

Available binding components

Binding components provide the functionalities required to connect an external system to the service bus. We have two expressions about binding components; a binding component acts as consumer or provider. For example an email component which acts as consumer sends a message which receives through the service bus to an email address. And when it injects the incoming emails into the bus it is acting as a provider.

Table 2 lists important binding components available through the OpenESB components project. These components are either developed by Sun Microsystems or one of its partners. Each binding component has a runtime and a design time part. The runtime part should be deployed into OpenESB while the design time part is a NetBeans module which we should install.

Table 2 Important binding components available through project OpenESB



eMail BC

A Binding Component for sending and receiving emails

Database BC

Reads messages from and writes messages to a database over JDBC


A JBI component for sending and receiving messages over HTTP


Receives messages from and sends messages to a remote service through JMS


Reads messages from and writes messages to an LDAP server


Provides functionality for interacting with a SAP R/3 system within a JBI environment.


Integrates media capabilities from communications networks using the SIP protocol (Session Initiation Protocol)


BC Polls for files on an FTP server, and provides for uploading files to an FTP server


BC Polls for files on the file system, and provides for writing files to the file system


Number of Binding components reaches 30 and covers variety of standards and protocols. Table 2 lists the most widely used binding components regardless of either it are in beta or stable state. You can find a list of all components at The list contains all necessary information like states, in maintenance mode or deprecated components and so on.

Administration and management

One important feature of a middleware is the possibility to administrate, monitor, and manage the middleware in ac effective and preferment way. OpenESB follows the same path that GlassFish does and provides very profound administration functionalities including a CLI and web based administration. Administration console provides monitoring information for service engines and binding components and deployed artifact.

2 Getting OpenESB running

Every big software need some sort of manual for installing and running the application, the manual includes information about compatible architecture, operating systems and runtime platforms. In our case we can install OpenESB on any operating system which there is a Java 5 runtime environment available for it. The supported platforms include Windows, Linux flavors, Solaris flavors, and Mac OSX flavors.

2.1 Downloading and installing

OpenESB is based on GlassFish and therefore it has the same packaging policy and schema for different operating systems. In our exploration we will use GlassFish ESB version 2 which is available at Make sure that you select the right version for your operating system and underlying architecture.

GlassFish ESB bundle contains the runtime bits which is consisting of JBI components and service engines along with the JBI container itself. The distribution package also contains the development time components which include NetBeans IDE and additional modules for supporting bundler JBI runtime components.

The installation process is quite simple; the installer asks for installation path and configuration information like GlassFish instance port, administration credentials and so on. By now you should be very good with the installation process. Before starting the installation process, ensure that you have something around 600MB of free space in the partition that you want to install the package.

Installing both GlassFish ESB runtime and NetBeans IDE inside one parent directory is a good practice as we have all development related artifacts in the same place.

2.2 Getting familiar with GlassFish ESB

I assume that we installed GlassFish ESB in a folder named glassfishesb_home, content of this folder is shown in figure 2


Figure 2 GlassFish ESB installation directory’s content

The content is very straight forward and cleans for such a massive development and deployment system. The glassfish folder includes GlassFish version 2ur2 along with stable versions of OpenESB container and components while the netbeans directory contains NetBeans IDE along with all necessary modules to support GlassFish ESB functionalities in design time.

We can start the server separately by running start_glassfish_domain1 script for times that we do not need the development environment running. Otherwise we can start NetBeans IDE by running start_netbeans script and then letting NetBeans to start the server when required.

3 Developing and testing a simple application

NetBeans IDE is a very feature rich IDE for developing Java EE applications and inclusion of SOA and Composite application development functionalities which targets OpenESB make the IDE a perfect development tools for developing composite applications and integration components. In this section we create a very simple BPEL based application to show how the overall process works.

3.1 The sample scenario

It is important to understand what we want to build and what features of OpenESB artifacts we want to use to implement the required functionalities. We have a portal like system which users can add a whether Portlet to see the current weather of their home city. We want to store some statistics which later on helps us determine which country and cities has largest number of users which use the Portlet weather.

To achieve this goal we used two web services, the first one translate the IP address of our client to its location including country and city and the second service provides us with current weather condition. We should somehow weave these services together to achieve the desired result. We also need to log all access to these web services with details related to country, city and date in a text file or database table to perform the statistical analysis and extract the usage portion for each country and each city in each country.

We will only use two BEPL components including the Database BC and BPEL SE. with these two component we can achieve a complete solution for our problem which otherwise could take more time to implement with much less flexibility in term of applying new and updated business rules.

3.2 Creating the BPEL Module project

Run NetBeans IDE and switch to Services view (CTRL+5), expand the database node and right click on the Java DB instance and select Start Server from the context menu. Expand the drivers’ node and right click on Java DB (Network) node and select Connect Using.

When the New Database Connection opened, change different attributes of the connection as follow:

  • Database URL: jdbc:derby://;create=true

  • User: APP

Password: APP

Click OK and NetBeans will create a new connection to our database. Expand the connection node, right click on the Tables node and select Create Table item. Create a table named wRequestLog with fields listed in the table 3.

Table 3 The wRequestLog table’s fields

Field Name

Field Type








The composite application runs inside the container and for accessing our WeatherInfo database we need to create a connection pool and data source in GlassFish application server. Create a connection pool for the above database and then create a data source named jdbc/winfo.

To create the connection pool you can use following asadmin command.

create-jdbc-connection-pool --datasourceclassname org.apache.derby.jdbc.ClientDataSource --restype javax.sql.XADataSource --property portNumber=1527:password=APP:user=APP:serverName=localhost:databaseName=WeatherInfo WinfoPool

I am sure that you remember how we used to create data sources but I provide the corresponding asadmin command if you have just forget the spelling ;-).

create-jdbc-resource --connectionpoolid WinfoPool jdbc/winfo

Now we are ready to take one step further and create the project and its related files like copy of WSDL documents, BPEL process file and so on.

Run the SOA enabled NetBeans IDE and from the main menu select File>New>Project and from categories select SOA and from projects list select BEPL Module click next and change the project name to WeatherInfo.

Right click on project node in project view (CTRL+1) and select New>Other, a window will open, from the categories select XML and from file types select External WSDL Document and click next. We want to create a WSDL from a URL therefore use as the URL and click finish. NetBeans will create a reference to the WSDL file along with a local copy of the WSDL file which we may edit to tailor it to our requirement. This web service provides us with location of a given IP address. The IP2Geo web service is provided by CDYNE Corporation and I get a trial license for using the web service in the book.

Follow the same procedure and create a new WSDL file from This web service takes a city name along with the country name and returns the current weather. The Global Weather web service is provided by WebserviceX.

We have two more steps and then we are finished with creating our project’s basic artifacts. The first artifact is a WSDL descriptor which let us use Database Binding Component to record all requests and the second artifact is the WSDL descriptor which describe input and output types of our provided service which is our BPEL process.

To create the database binding component, right click on the project node and select New>WSDL Document the wizard will open and guide you through the process. In the first page of wizard use the following information:

  • Change the name to DBLOGBC

  • Change the WSDL Type to Concrete WSDL Document; two new attributes will become available.

Change the Binding to Database and change the Type to Prepared Statement.

Click next, in this page we should determine which database connection we want to use, select the connection URL that we made in previous step and click Next. In this step we need to enter a valid prepared statement which the binding component will execute against the database. Enter the following snippet as the prepared statement value and then press Discover Parameters.

insert into APP.WREQUESTLOG (rdate,country,city) values (?,?,?)

Change the parameters name according to figure 3 and click Next then click Finish and we are done with defining a database binding component which can integrate into the solution and insert our records into database.


Figure 3 Database binding component’s prepared statement definition

The OpenESB version 2 design time modules does not works very good for naming the attributes and operations. We need to rename them manually to have easier mapping and navigation inside the project artifact later. Open the DBLOGBC.wsdl in the source editor and replace all instance of newuntitled_Operation with log_Operation. Also look for jdbc/__defaultDS and replace it with jdbc/winfo. Depending on your NetBeans and OpenESB design time component the default data source name may differ or even the wizard may asks you for the data source name but you can always manually edit the jdbc:address element of the DatabaseBC’s wsdl file to make the correct configuration.

You remember that we changed the parameters name, but your OpenESB design time modules may forget to apply the naming in the XSD file and therefore we will face some ambiguity problems during variables mapping in next steps. So open the DBLOGBC.xsd in the source editor and make following changes:

  • param1 should change to rDate

  • param2 should change to Country

param3 should change to City

Here comes the last step before creating the BPEL process, in this step we define the process communication interface which external clients will use to invoke our process and utilize its service.

Right click on the project node and select New>WSDL Document, when the wizard opened change the name to wInfoProcess; for the WSDL type select Concrete WSDL Document; for the binding choose SOAP and for the type attribute select RPC Literal press Next. Change the input message part name to IPAddress and change the output message part name to weatherInfo.

We are finished with defining all basic artifacts prior to designing the process, now we should just continue with designing the BPEL process itself. To do so, right click on the project node and select New>BPEL Process, change the name to wInfoProcess and click Finish. Now NetBeans should open the BPEL process in the BPEL designer which is a design time module of OpenESB.

In the right side of the IDE you can see components plat and in the left side the project explorer is in place. First let’s create the partner links which connect the BPEL process to other involved parties like external web services and our database binding component. The BPEL designer has three sections, left right and the middle section, the left and right side are used for partner links. Therefore drag and drop the WSDL files from project explorer into the design are as listed in table 4. When you drop a WSDL into the BPEL designer left or right side a window will open which allows you to change the partner link attribute and characteristics. We accept default values for all attributes expect for the partner link name which are listed in the table.

Table 4 adding WSDLs to BPEL process designer

WSDL name

BPEL designer area

Changes required for partner link


Right Side

Change the name to Positioning


Right Side

Change the name to Weather


Right Side

Change the name to Logger


Left Side

Change the name to WInfoProcess

By now your designer window should be similar to figure 5. Partner Links in the left side meant to be internal to the BPEL process and those in the right side are external to the BPEL process.

Figure 5 The BPEL designer after adding all partner links.

Now we need to add BPEL elements to the process to implement the business logic that we discussed. From top to button we need to add the following elements to the BPEL designer window. The final view of designer window after we finish adding elements will be similar to figure 6.

 Figure 6 BPEL designer view adding all necessary elements

Table 5 shows the complete list of all elements which we should add to the designer. To add the elements we should drag and drop them from the right side Palette into the middle area of the designer. Designer will show you all possible places for dropping the element by shoeing an orange circle in that position. To rename an element we can simply select the element and press F2 or double click on the element name. To connect an element to one of the partners, click on the element, an envelop will appear, drag and connect the element to the partner link mentioned in the table 5

Table 5 List of required BPEL elements which we should add to the designer window

Element Type

Element Name

Partner Link













Flow (Structured Activity)


Invoke (Inside Flow)



Invoke (Inside Flow)









Now we need to add variables and variables assignment and our work with the process and the process designer is finished. In this step we will create all variables and then we will move toward assigning these variables to each other. To create the variables:

  • Double click on GetWeatherInfo and for the input variable click create, accept the defaults and click OK buttons as much as required to see the designer window again.

  • Double click on InvokeToResolveIP and create both output and input variable the same way that you create the variable for the GetWeatherInfo.

  • Double click on GetWeather and make sure that you select GetWeather as the operation and create the input and output variables similar to previous steps.

  • Double click on LogInformation and create input and output variables, accept defaults for variable names and other attributes.

Finally double click on the reply button and for the normal response create a variable and accept the defaults for its name and other attributes.

Now we should be able to assign the variables to each other to make the data flow possible in the process. Thankfully NetBeans come with a very nice and easy to use Assign variable editor called Mapper which we can use to assign the variables to each other. Double click on the AssignIP element and use the following procedure to assign the received IP in the previous step to input variable of InvokeToResolveIP.

In the left side of the Mapper window select output button to ensure that it is filtered for output variables and in the right side select input button to ensure that we are looking at input variables.

Expand the WInfoProcessOperationIn from the left side and expand ResolveIPIn from the right side, make sure that you expand them as much as it does not expand anymore. Now from the left side select IPAddress and connect it to ipAddress in the right side. From the Mapper top menu bar select String and add a String Literal to the middle section of the Mapper window. Change its value to 0 and connect it to licenseKey variable in the right side. Now we are finished with the first assign element. Now switch to the designer view to finish the mapping for AssignCityAndCountry element.

Double click on the AssignCityAndCountry element and assign the variables as follow:

Let’s assign the city name and country name to the GetWeather invoke element. To do so, from the left side expand ResolveIPOut and from the right side expand GetWeatherIn. From the left side, assign city and country parameters of the ResolveIPOut to similar parameters of the GetWeatherIn in the right side. Now we need to assign the same information to our database log operation therefore, expand the Log_operationIn and assign the city and country from the left side to its parameters. We have an rDate parameter which we should use a Current Date & Time predicate. So, from the top menu bar select Date And Time and drop a Current Date & Time predicate into the middle section of the Mapper window. Connect the new predicate to rDate variable of the Log_OperationIn. We are finished with the second variable assignment and one last item remains which we should complete. Figure 6 shows the Mapper window after assigning all variables.


Figure 7 Mapper window with complete AssignCityAndCountry assignment

One more variable assignment and we are finished with the BPEL process development. Double click on AssignWeather element and assign the output of GetWeatherOut parameter to WInfoProcessOperationOut parameter. Ok, the name looks unlikely and you may thing that it is an output variable but believe me it is an input variable.

3.2 Creating the Composite Application project

A composite application is the project type that we can use to deploy a composite application into the JBI container. Select File>New Project; from the categories select SOA and from Projects select Composite application and click next. Change the project name to WinfoCA and click finish. The composite application is created and is ready for adding JBI modules.

Right click on the project node and select Add JBI Module, select WeatherInfo project. The CASA editor opens which shows the BEPL module without any relation. Right click on WinfoCA project node and select Clean and Build. NetBeans will compile the composite application and its included JBI modules. When we issue a build command NetBeans refresh the CASA editor and we can add new module, change a data flow between components and so on.

3.4 Deploying and testing sample project

Deploying and testing a composite application is very easy using the well integrated Netbeans, GlassFish and OpenESB. To deploy the application, right click on the WinfoCA node and select deploy. NetBeans will deploy the assembly to OpneESB runtime. To create a test case for the composite application, expand the WinfoCA node and right click on Test node. A wizard will start that help us generate the test case.

In the first page, wizard asks for the test case name, it can be WinfoTest at the second step it asks us to choose which WSDL file we want to test. Expand the WeatherInfo node and select winfoprocess.wsdl press next. Now we should select which operation we want to test, select winfProrocessOperation and click finish. Now we have a test case created, expand the test case in project view and double click on input node to edit the test case input values. Double clicking the input node results in opening an XML file in the editor. The XML file is a SOAP message which we should change its parameters value to suite our test requirement. Therefore change the value of IPAddress element to which is one the Google IP addresses. Save the file and we are finished with creating the test case.

Right click on WinfoTest and select run. IDE will invoke the operation defined in the WSDL file by passing the SOAP message we created to it. Then the output result will appear under the output node of the test case. The output should be information about current weather in Mountain View in the United States.

You can debug a BPEL process and generally a JBI application using NetBeans. The IDE allows you to view the variables during different steps of BPEL execution. The simplest way that we can use to debug the BPEL module is adding our required breakpoint and then executing the test using debug mode.

3.5 OpenESB administration from within NetBeans IDE

NetBeans IDE is very well integrated with OpenESB for not only for developing and testing applications but also for administration and monitoring purposes. All of these functionalities are hidden inside the NetBeans integration adapter for OpenESB.

Press CTRL+5 to open the servers tab and then expand the servers node, expand the OpenESB server instance and finally expand JBI node. Now your server view should be similar to figure 8. Now press CTRL+SHIFT+7 to open the Properties window and then select JBI node. It is wonderful that you can change configuration of the JBI container from within the IDE to continue your work without leaving the IDE and switching between different windows.


Figure 8 The OpenESB node in NetBeans Server view

Expand the Service Engines node and select sun-bpel-engine node, the Properties view shows you the entire configuration and read only attributes of the BPEL engine. You can enable the monitoring by checking the checkbox or you can enable the persisting feature to have your BPEL process recovered after any possible crash.

Right click on the sun-bpel-engine node and explore what operations you can perform on the engine from context menu. We can start, stop, upgrade, uninstall and perform any other lifecycle related task using the context menu. We can view the different endpoints used in BPEL process deployed in the BPEL engine using the Show Endpoints Statistics menu item. This menu item helps us to understand whether our services are called during our test period or not and if they are called what was the response semantic and quality.

4 OpenESB administration

The same administration channels which we saw for GlassFish are available for OpenESB and GlassFish ESB. OpenESB CLI commands are integrated with GlassFish CLI utility and follow the same principals. And the Web Based administration console uses GlassFish administration console to present required administration capabilities.

4.1 The CLI utility

OpenESB CLI administration is completely integrated with GlassFish administration and has the same set of characteristics. The only major difference is lack of some commands as GlassFish ESB version 2 is based on GlassFish version 2 and any new functionality that is included in GlassFish version 3 is not present in version 2.

Service assembly administration commands

JBI container like Java EE container has its own application type which is called a JBI service assembly or simply an assembly. Usually the overall procedure of creating and deploying an assembly is done using build systems or IDE but there some commands in the OpenESB CLI which let us administrate the assemblies. All of the OpenESB CLI commands are remote with all characteristics of a remote command like default target server, and so on.

Deploying and undeploying a service assembly are two most basic tasks related to OpenESB administration. In the previous step we developed and deployed an application using NetBeans, now lets undeploy the application and then deploy it using the CLI commands.

To underply the application we can use undeploy-jbi-service-assembly command. For example to undeploy the WinfoCA composite application we can use:

asadmin> undeploy-jbi-service-assembly WinfoCA

Now we can deploy the application archive using the deploy-jbi-service-assembly command. NetBeans puts the built packages into a directory named dist inside the project directory. Let’s deploy the WinfoCA application from that particular directory.

deploy-jbi-service-assembly /opt/article/WinfoCA/dist/

This command deploy the application to the default target server with default attributes like application name and so on.

To list all deployed service assembly we can use list-jbi-service-assemblies command. Now that we have one application deployed into the container this command should show a list containing WinfoCA similar to:

asadmin> list-jbi-service-assemblies


Command list-jbi-service-assemblies executed successfully.

Each service assembely contains one or more service units which connect services, binding components, and service engines along with any deployment artifacts like XSLT documents, BPEL process and so on. To view these service units we can use show-jbi-service-assembly command. Figure 9 shows how we can use the command to view detailed information for WinfoCA composite application.

Figure 9 viewing the detailed information for an assembly

We can stop, start and forcefully shutdown a service assembly when required by using the OpenESB provided commands.

  • To stop the WinfoCA service assembly: stop-jbi-service-assembly WinfoCA

  • To start the WinfoCA service assembly if it is stopped: start-jbi-service-assembly WinfoCA

To forcefully shutdown the WinfoCA service assembly if required: shut-down-jbi-service-assembly WinfoCA

There are more JBI service assembly commands in the OpenESB CLI which you can find them using ./asadmin help command.

Components administration commands

There two major types of JBI components including service engines and binding components which we may need to deal with. These two types are accompanied with another type named shared library which contains common library classes for several components. Table 6 shows important commands in this set along with an small explanation about each one of them.

Table 6 Important JBI components administration commands





These commands can be used to start and stop binding components. Stopping a binding component result in any related service unit to stop.




Showing a list of deployed JBI components




Show detailed information about a given JBI component which is deployed in the target container.




Installing different JBI components into the target container.



Uninstalling given JBI component from the target server.



Shutdown the given component. Any deployed service unit on the target component will shutdown consequently

Although there are many other command which an administrator may need during his daily tasks but we learned the most important commands in the section. In the next section we will take a look at web based administration console features for OpenESB.

4.2 Administration console

One good point about GlassFish and its companion product is the unified administration channels including CLI, JMX and web based administration console which let us use a unified look and feel and familiar environment to administrate all the infrastructure.

All of JBI administration capabilities are listed under the JBI node in the navigation tree. When you expand all nodes you can see something similar to figure 10. As you can see in figure 10 the JBI node also helps us have a big picture view of our service units’ deployment on different service engines and binding components.


Figure 10 OpenESB administration node in the GlassFish administration console

The JBI node itself allows us to configure the container settings like Logging, general statistics, and general configuration related to JBI artifacts installation. Subsequent nodes let us administrate the deployed applications, binding components, service engines and shared libraries.

Service assembly administration

Administration console provides deploy, undeploy, start, stop and shutdown operations for service assemblies. We also can view monitoring information for a deployed service assembles.

For each service assembly we may have one or more service units, we can view service units included in a service assembly along with accessing some monitoring information about them.

JBI components administration

Installing, uninstalling, starting, stopping and shutting down a JBI component which can be either a service engine or a biding component provided in the Components node. Administration console provides us with capabilities which allow us to change the component configuration; its loggers logging level, viewing the descriptor file for the component, and most important thing viewing detailed monitoring information about requests and request processing by each component.

We can see view which service units from which service assemblies are deployed on a component and navigate to the service unit or service assembly configuration.

JBI shared libraries administration

For the shared libraries, we can install, uninstall and view the depended components on each shared libraries to ensure that uninstalling the library will not cause unexpected result in other components.

4.3 Installing new components

OpenESB project provides many components out of the box but there are tens of components available in the OpenESB website at From this page we can navigate to each component WIKI page and download the required and install them into OpenESB and NetBeans. But another effort is undergo to let the developers to download one single installer for each component and let the installer install all required artifacts like shared libraries, runtime engines and design time NetBeans modules. Latest build of installers for all components are available at

Installing the components using the installers available at the above page is very straight forward. the installer asks for NetBeans and OpenESB (GlassFish) directory and complete the installation.

5 Summary

In this article we discussed almost all basics of OpenESB administration and application development in a crush course mode. We saw how JBI and OpenESB are related, how OpenESB and GlassFish ESB are similar products with different naming. We developed a sample JBI service assembly which utilizing several binding components and service engines and finally we deployed the application into OpenESB and performed functional testing using NetBeans IDE.

We discussed how we can install and administrate OpenESB. We discussed CLI and administration console for managing OpenESB and we learned that OpenESB administration channels uses similar rules and look and feel that GlassFish administration and CLI uses.

GlassFish Security book FAQ 1: Custom Security Realm in GlassFish

I decided to write down the answer for some questions which my book’s readers email me or ask me via twitter in my weblog so everyone can benefit from the answers. Here is the answer to the first question which involves custom security realms.

GlassFish supports 5 types of security realms out of the box which are sd follow:

  1. File Realm: Usefull for development and testing purposes. GlassFish provids a user/ group management interface for this realm. We can add user and groups using the administration console. When using this realm all usernames, passwords and groups are stored in a plain text file.
  2. JDBC Realm: In production environment we store user information including but not limited to username, passwords and groups in an RDBMS and then configure a JDBC realm to authenticate the given credentials againts the information stored in the datase.
  3. LDAP Realm: Sometimes we have all user details stored in an LDAP like Active Directory or Redhat Directory Server, OpenDS or Sun Java System Directory Server Enterprise Edition.
  4. Solaris Realm: This realm is used to authenticate users with a Solaris user directory.
  5. Certificate Realm: The certificate realm allows us to conduct mutual SSL authentication based on the client certificates.

Sometimes our users information is stored in a silo different than all of this mentioned storages and we need to use that source for authentication and access control. For example assume that we have our users information including username, passwords and group membership stored in an Object Database and we need to authenticate our enterprise application’s users with that storage. In such times we should either think about having a synchronized RDMBS keeping update user information and use JDBC realm for authentication and authorization or we should develop  a custom security realm which uses the object database as a source for authentication.

Setting up synchronization between the e.g object database and RDBS can be tricky while developing a custom authentication realm is much easier using GlassFish provided SPIs.

Second chapter of GlassFish security book discusses GlassFish security realms in details and discuss a sample application which uses these realms for authenticating and authorizing users. In the same chapter, developing custom security realms is discussed along with developing a sample realm.

In the same chapter GlassFish support for  JSR-196 (Java Authentication Service Provider Interface for Containers) is discussed to complete the ring of authentication and authorization in Java EE in general and GlassFish application server in particular.