JavaZone 2010, a very brief report.

Yesterday the two days JavaZone 2010 conference finished, the conference was as good as I expected it to be. The sessions were selected very carefully to cover all sort of interest from architecture and NoSQL to security and core java APIs.

A fair portion of the participants, both in speakers group and in the audiences group were loyal participants which at least attended one of the previous conferences in past 10 years. I was one of those who participated for the first time, so I can not compare this year’s event with the previous one but those who have seen the previous events agreed that the quality of each year’s conference is better than previous year, and for this year the quality increases is way more than previous ones.

For few instances I heard people complaining about why more speakers are using English or small number of elementary level sessions (tutorials perhaps) but I believe no one found another reason  for a complain.

The JourneyZone is what I am looking forward to experience in the next two days. It is the first time in the 10 years history of JavaZone that the program committee is organizing this event for the JZ participants. I am going to check out of the hotel and go for the JourneyZone.

Following album includes few photos from my session, thanks to Alexis for taking the photos. The slides is available here I know that all videos for the sessions will be available at some point in time, but I do not know when it will become available, maybe in few days maybe in few months.

Introducing NIO.2 (JSR 203) Part 6: Filtering directory content and walking over a file tree

In this part we will look at how the directory tree walker and the directory stream reader works. These two features are another couple of long requested features which was not included in the core java before Java 7.

First, lets see what directory stream reader is, this API allows us to filter content of a directory on the file system and extract the file names that matches our filter criteria. The feature works for very large folders with thousands of files.

For filtration we can use PathMatcher expression matching the file name or we can filter the directory content based on the different file attributes. for example based on the file permissions or the file size.

Following sample code shows how to use the DirectoryStream along with filtering. For using the PathMatcher expression we can just use another overload of the newDirectoryStream method which accepts the PathMatcher expression instead of the filter.

public class DirectoryStream2 {

    public static void main(String args[]) throws IOException {
	//Getting default file system and getting a path
        FileSystem fs = FileSystems.getDefault();
        Path p = fs.getPath("/usr/bin");

	//creating a directory streamer filter
        DirectoryStream.Filter
 filter = new DirectoryStream.Filter
() {

            public boolean accept(Path file) throws IOException {
                long size = Attributes.readBasicFileAttributes(file).size();
                String perm = PosixFilePermissions.toString(Attributes.readPosixFileAttributes(file).permissions());
                if (size > 8192L && perm.equalsIgnoreCase("rwxr-xr-x")) {
                    return true;
                }
                return false;
            }
        };
	// creating a directory streamer with the newly developed filter
        DirectoryStream
 ds = p.newDirectoryStream(filter);
        Iterator
 it = ds.iterator();
        while (it.hasNext()) {

            Path pp = it.next();

            System.out.println(pp.getName());
        }
    }
}

The above code is self explaining and I will not explain it any further than the in-line comments.

Next subject of this entry is directory tree walking or basically file visitor API. This API allows us to walk over a file system tree and execute any operation we need over the files we found. The good news is that we can scan down to any depth we require using the provided API.

With the directory tree walking API, the Java core allows us to register a vaster class with the directory tree walking API and then for each entry the API come across, either a file or a folder, it calls our visitor methods. So the first thing we need is a visitor to register it with the tree walker. Following snippet shows a simple visitor which only prints the file type using the Files.probeContentType() method.

class visit extends SimpleFileVisitor
 {

    @Override
    public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) {
        try {
            System.out.println(Files.probeContentType(file));

        } catch (IOException ex) {
            Logger.getLogger(visit.class.getName()).log(Level.SEVERE, null, ex);
        }
        return super.visitFile(file, attrs);
    }

    @Override
    public FileVisitResult postVisitDirectory(Path dir, IOException exc) {
        return super.postVisitDirectory(dir, exc);
    }

    @Override
    public FileVisitResult preVisitDirectory(Path dir) {
        return super.preVisitDirectory(dir);
    }

    @Override
    public FileVisitResult preVisitDirectoryFailed(Path dir, IOException exc) {
        return super.preVisitDirectoryFailed(dir, exc);
    }

    @Override
    public FileVisitResult visitFileFailed(Path file, IOException exc) {
        return super.visitFileFailed(file, exc);
    }
}

As you can see we extended the SimpleFileVisitor and we have visitor methods for all possible cases.

Now that we have the visitor class, the rest of the code is straight forward. following sample shows how to walk over /home/masoud directory down to two levels.

public class FileVisitor {

    public static void main(String args[]) {

        FileSystem fs = FileSystems.getDefault();
        Path p = fs.getPath("/home/masoud");
        visit v = new visit();
        Files.walkFileTree(p, EnumSet.allOf(FileVisitOption.class), 2, v);

    }
}

You can grab the latest version of Java 7 aka Dolphin from here and from the same page you can grab the latest JavaDoc which is generated from the same source code that is used to generate the binary bits. Other entries of this series are under the NIO.2 tag of my weblog.

Introducing NIO.2 (JSR 203) Part 5: Watch Service and Change Notification

For long time Java developers used in-house developed solutions to monitor the file system for changes. Some developed general purpose libraries to ease the task of others who deal with the same requirement.  Commercial and  free/ open source libraries like http://jnotify.sourceforge.net/, http://jpathwatch.wordpress.com/ and  http://www.teamdev.com/jxfilewatcher among others. Java 7 comes with NIO.2 or JSR 203 which provides native file system watch service.

The  watch service provided in Java 7  uses the underlying file system functionalities to watch the file system for changes, so if we are running on Windows, MacOS or Linux… we are sure that the watch service is not imposing polling overhead on our application because the underlying OS and file system provides the required functionalities to allow Java to register for receiving notification on file system changes. If the underlying file system does not provide the watch-ability, which I doubt it for any mainstream file system,  Java will fall back to some rudimentary polling mechanism to keep the code working but the performance will degrade.

From the mentioned libraries the jpathwatch API is the same as the Java 7 APIs to make it easier to migrate an IO based application from older version of Java to Java 7 when its time arrives.

The whole story starts with WatchService which we register our interest in watching a path using it. The WatchService itself is an interface with several implementatins for different file system and operating systems.
We have four class to work with when we are developing a system with file system watch capability.
  1. A Watchable: A watchable is an object of a class implementing the Watchable interface. In our case this is the Path class which is the one of the central classes in the NIO.2
  2. A set of event types: We use it to specify which types of events we are interested in. For example whether we want to receive creation, deletion, … events. In our case we will use StandardWatchEventKind which implements the WatchEvent.Kind<T>.
  3. An event modifier: An event modifier that qualifies how a Watchable is registered with a WatchService. In our case we will deal with nothing specific up to now as there is no implementation of this interface included in the JDK distribution.
  4. The Wacher: This is the watcher who watch some watchable. In our case the watcher watches the File System for changes. The abstract class is java.nio.file.WatchService but we will be using the FileSystem object to create a watcher for the File System.

Now that we know the basics, let’s see how a complete sample will look like and then we will break down the sample into different parts and discuss them one by one.

import java.io.IOException;
import java.nio.file.FileSystem;
import java.nio.file.FileSystems;
import java.nio.file.Path;
import java.nio.file.StandardWatchEventKind;
import java.nio.file.WatchEvent;
import java.nio.file.WatchKey;
import java.nio.file.WatchService;
import java.util.List;
import java.util.logging.Level;
import java.util.logging.Logger;

public class WatchSer {
    public static void main(String args[]) throws InterruptedException {
        try {
            FileSystem fs = FileSystems.getDefault();
            WatchService ws = null;
            try {
                ws = fs.newWatchService();
            } catch (IOException ex) {
                Logger.getLogger(WatchSer.class.getName()).log(Level.SEVERE, null, ex);
            }
            Path path = fs.getPath("/home/masoud/Pictures");
            path.register(ws, StandardWatchEventKind.ENTRY_CREATE, StandardWatchEventKind.ENTRY_MODIFY, StandardWatchEventKind.OVERFLOW, StandardWatchEventKind.ENTRY_DELETE);

            WatchKey k = ws.take();

            List> events = k.pollEvents();
            for (WatchEvent object : events) {
                if (object.kind() == StandardWatchEventKind.ENTRY_MODIFY) {
                    System.out.println("Modify: " + object.context().toString());
                }
                if (object.kind() == StandardWatchEventKind.ENTRY_DELETE) {
                    System.out.println("Delete: " + object.context().toString());
                }
                if (object.kind() == StandardWatchEventKind.ENTRY_CREATE) {
                    System.out.println("Created: " + object.context().toString());
                }
            }
        } catch (IOException ex) {
            Logger.getLogger(WatchSer.class.getName()).log(Level.SEVERE, null, ex);
        }
    }
}

At the beginning of the code we create a FileSystem object and then create a WatchService for the file system underneath the FileSystem object we created previously.

            FileSystem fs = FileSystems.getDefault();
            WatchService ws = null;
            try {
                ws = fs.newWatchService();
            } catch (IOException ex) {
                Logger.getLogger(WatchSer.class.getName()).log(Level.SEVERE, null, ex);
            }

At the next step we create a Path object which basically is our watchable, what we want to watch, and then register it with the WatchService object we created in the previous step.

  Path path = fs.getPath("/home/masoud/Pictures");
   path.register(ws, StandardWatchEventKind.ENTRY_CREATE, StandardWatchEventKind.ENTRY_MODIFY, StandardWatchEventKind.OVERFLOW, StandardWatchEventKind.ENTRY_DELETE);

Next we get a key which represents the registration of a watchable with a watch service.
This WatchKey object represents our access door to events propagated by the WatchService object.

After getting the key, we can poll for the events arrived at the key. An File system event may get represented by multiple WatchService events. For example a rename is represented by deletion and creation events.

 WatchKey key = ws.take();
 List> events = key.pollEvents();

Now that we have the list of events we can iterate over the events list and see whether we are intrested in the event or not.

for (WatchEvent object : events) {
                if (object.kind() == StandardWatchEventKind.ENTRY_MODIFY) {
                    System.out.println("Modify: " + path.toRealPath(true)+"/"+ object.context().toString());

                }
                if (object.kind() == StandardWatchEventKind.ENTRY_DELETE) {
                    System.out.println("Delete: " +  path.toRealPath(true)+"/"+ object.context().toString());
                }
                if (object.kind() == StandardWatchEventKind.ENTRY_CREATE) {
                    System.out.println("Created: " +  path.toRealPath(true)+"/"+ object.context().toString());
                }
            }

In our sample code we are running the watch service in the application thread while we can use multiple threads to handle the events. One reason that the WatchService is not implemented using the Listener is the urge for using multiple threads to deal with the events in cases where there are thousands of events or event processing take longer than application threshold.
You can grab the latest version of Java 7 aka Dolphin from here and from the same page you can grab the latest JavaDoc which is generated from the same source code that is used to generate the binary bits. Other entries of this series are under the NIO.2 tag of my weblog.

I will be speaking at JavaZone 2010 about NIO.2

Well, my abstract for a session in JavaZone 2010 accepted and I will be speaking about NIO.2, you may call it new new IO, more new IO or technically JSR-203,on 8th of September in saloon 1. I will be the second speaker in the first day of the conference so the timing will be something like 10:15 to 11:15. The session will be in English and I will give the audience an overview of IO features in Java 7 and its differences improvements over old IO framework. I estimated that  I will be speaking for 55 minutes to present something around 35 slides which I have prepared their sketch.

JavaZone 2010, Oslo
JavaZone 2010, JavaZone logo

Majority of the slides will discuss the file system capabilities included in the NIO.2, some slides will cover the differences between I/O with Java 7 and I/O in older releases. Some slides will discuss Asynchronous I/O, and other cool features of NIO.2. Almost all slides come with a sample code showing the concept in action. I will try to include more about my presentation and slides while I am completing the sketch and the first draft.

Introducing NIO.2 (JSR 203) Part 4: Changing File System Attributes and Permissions

This is the 4th installment of my entries covering NIO.2. In this entry I will discuss more about what we can do with attributes and permissions.  The NIO.2 lets us write the permissions and attributes of a file in addition to reading them. For example we can change the rwx permissions using the Attributes utility class and  PosixFilePermission.

You can execute the following sample code with almost no changes, the only thing you need to change is the path to our sample file and then you are ready to see how it works.

import java.io.IOException;
import java.nio.file.*;
import java.nio.file.attribute.*;
import java.util.*;

public class Perm2 {

    public static void main(String args[]) throws IOException {
        FileSystem fs = FileSystems.getDefault();
        Path p = fs.getPath("/home/masoud/dl");

        // Reading attributes using attributes views.
        // here we read both basic and posix attributes of a file
        Map att = (Map) p.readAttributes("posix:*, basic:*", LinkOption.NOFOLLOW_LINKS);
        // printing the attributes to output
        Set en = att.keySet();
        for (Iterator it = en.iterator(); it.hasNext();) {
            String string = it.next();
            System.out.println(string + ": " + att.get(string));
        }
        System.out.println("-------------Attributes printing finished----------");

        // definind a new permission set for our file
        Set
 st2 = PosixFilePermissions.fromString("rwxrwxrwx");
        // Setting the attribute using Attributes utility class
        Attributes.setPosixFilePermissions(p, st2);

        // looking up and creating a principal for the given user. If use does
        // not exists it will throws a UserPrincipalNotFoundException
        UserPrincipal up = fs.getUserPrincipalLookupService().lookupPrincipalByName("masoud");

        // Setting the owner to the owner we just looked up.
        // We should have enough permisison to change the owner otherwise it will
        // throw a FileSystemException: /home/masoud/a: Operation not permitted sort of thing
        Attributes.setOwner(p, up);

        //another way to read and write the owner value for a file is using FileOwnerAttributeView
        FileOwnerAttributeView fo = p.getFileAttributeView(FileOwnerAttributeView.class, LinkOption.NOFOLLOW_LINKS);
        fo.setOwner(up);

        //Now that we have changed the permissions lets see the permissions again:
        att = (Map) p.readAttributes("posix:permissions", LinkOption.NOFOLLOW_LINKS);
        System.out.println("New Permissions:" + ": " + att.get("permissions"));
    }
}

Let’s see what we have done starting from top: at the beginning we have just initialized the FileSystem object and got a file reference to /home/masoud/dl. Then we read all of this files basic and posix attributes using the attributes view notation. The following sample code shows this step

 FileSystem fs = FileSystems.getDefault();
        Path p = fs.getPath("/home/masoud/dl");

        // Reading attributes using attributes views.
        // here we read both basic and posix attributes of a file
        Map att = (Map) p.readAttributes("posix:*, basic:*", LinkOption.NOFOLLOW_LINKS);
        // printing the attributes to output
        Set en = att.keySet();
        for (Iterator it = en.iterator(); it.hasNext();) {
            String string = it.next();
            System.out.println(string + ": " + att.get(string));
        }
System.out.println("-------------Attributes printing finished----------");

Next step we have use the PosixPermissions class to create a permission set using the OS permissions presentation and then applied these permissions on our file

 // definind a new permission set for our file
        Set
 st2 = PosixFilePermissions.fromString("rwxrwxrwx");
        // Setting the attribute using Attributes utility class
        Attributes.setPosixFilePermissions(p, st2);

In the next step we created a user principal for a user named masoud and changed the ownership of our file to this user. We have done this using both available methods.

// looking up and creating a principal for the given user. If use does
        // not exists it will throws a UserPrincipalNotFoundException
        UserPrincipal up = fs.getUserPrincipalLookupService().lookupPrincipalByName("masoud");

        // Setting the owner to the owner we just looked up.
        // We should have enough permisison to change the owner otherwise it will
        // throw a FileSystemException: /home/masoud/a: Operation not permitted sort of thing
        Attributes.setOwner(p, up);

        //another way to read and write the owner value for a file is using FileOwnerAttributeView
        FileOwnerAttributeView fo = p.getFileAttributeView(FileOwnerAttributeView.class, LinkOption.NOFOLLOW_LINKS);
        fo.setOwner(up);

Finally we read the file permissions to see what has changed after we applied this new permissions on our file. using the following snippet.

 //Now that we have changed the permissions lets see the permissions again:
        att = (Map) p.readAttributes("posix:permissions", LinkOption.NOFOLLOW_LINKS);
        System.out.println("New Permissions:" + ": " + att.get("permissions"));

The output of running the above sample code is shown below:

lastModifiedTime: 2010-07-06T23:15:32Z
fileKey: (dev=805,ino=1922268)
isDirectory: false
lastAccessTime: 2010-07-11T14:15:46Z
isOther: false
isSymbolicLink: false
owner: masoud
permissions: [OTHERS_READ, OWNER_WRITE, GROUP_WRITE, OWNER_READ, GROUP_READ, OTHERS_EXECUTE, OWNER_EXECUTE, GROUP_EXECUTE, OTHERS_WRITE]
isRegularFile: true
creationTime: null
group: dip
size: 219
-------------Attributes printing finished----------
Permissions:: [OTHERS_READ, OWNER_WRITE, GROUP_WRITE, OWNER_READ, GROUP_READ, OTHERS_EXECUTE, OWNER_EXECUTE, GROUP_EXECUTE, OTHERS_WRITE]

You may ask what is the attribute view notation and what is meaning of that posix:permissions or posix:*. The first one means that we want to retrieve the permissions property of the file while the second one means that we want to retrieve all attributes of the file which can be seen using the posix view.

Now you are asking where you can found list of this attributes view notations, then you question is very right. The following links shows what attributes are available to read using each one of those views. Each attribute is one of the properties included in the interface.

Sure we have other views, you can find them in the JavaDocs linked above.

Next entry will be the last entry covering File System functionalities of the NIO.2 including the watch and change notification service and the ACL list. After that we will look at new features included for IO developers. You can find other entries of this series under the NIO.2 tag of my weblog.

Introducing NIO.2 (JSR 203) Part 3: File System Attributes and Permissions support in NIO.2

In two previous entries I covered Introducing NIO.2 (JSR 203) Part 1: What are new features? and Introducing NIO.2 (JSR 203) Part 2: The Basics In this entry I will discuss Attributes introduced in NIO.2. Using attributes we can read platform specific attributes of an element in the file system. For example to hide a file system in DOS file system or to check the last access date of a file in a UNIX machine.

Using NIO.2 we can check which attributes are supported in the platform we are running on and then we can decide how to deal with the available attributes. Following sample code shows how we can detect the available attributes and then how to manipulate them.

  FileSystem fs = FileSystems.getDefault();
  Path p = fs.getPath("/home/masoud/netbeans-6.9-ml-linux.sh");
 //checking available attributes:
  Set<String> supportedViews = fs.supportedFileAttributeViews();
 //We always have at least one member in the set, the basic view.
  BasicFileAttributes ba = p.getFileAttributeView(BasicFileAttributeView.class, LinkOption.NOFOLLOW_LINKS).readAttributes();
 //Printing some basic attributes
   System.out.println(p.toString() + " last access:  " + ba.lastAccessTime());
   System.out.println(p.toString() + " last modified " + ba.lastModifiedTime());
        // As I know I am in NIX machine I access the unix attributes.
        // If I didnt I should have iterate over the set to determine which
        // attributes are supported
        if (supportedViews.contains("unix")) {
            PosixFileAttributes pat = Attributes.readPosixFileAttributes(p, LinkOption.NOFOLLOW_LINKS);
            System.out.println(pat.group().getName());
            System.out.println(pat.owner().getName());
        }

I placed plethora of comments on the code so reading and understanding it get easier.

In the next snippet we will see how we can read permissions of file system element. The first step in checking permissions is using the checkAccess method as shown below. the method throw an exception if the permission is not present or it will execute with no exception if the permission is present.

 FileSystem fs = FileSystems.getDefault();
 Path p = fs.getPath("/home/masoud/netbeans-6.9-ml-linux.sh");
    try {
            // A method to check the access permissin
            p.checkAccess(AccessMode.EXECUTE);
        } catch (IOException ex) {
            Logger.getLogger(perm.class.getName()).log(Level.SEVERE, null, ex);
        }

        // Extracting all permissions of a file and iterating over them.
        //I know that I am dealing with NIX fs so I go directly with that attributes
        // otherwise we should check which attributes are supported and then we can
        // use them.

        PosixFileAttributes patts = Attributes.readPosixFileAttributes(p, LinkOption.NOFOLLOW_LINKS);
        Set<PosixFilePermission> st = patts.permissions();
        for (Iterator<PosixFilePermission> it = st.iterator(); it.hasNext();) {
            System.out.println(it.next().toString());
        }

        // Using PosixFilePermissions to convert permissions to different representations
        System.out.println(PosixFilePermissions.toString(st));

As you can see in the code we can use the helper class to convert the permission set to a simpl OS represeted permission of the element. for example the set can be translated to rwx——  if the file has owner read, write and execute permissions attached to it. The helper calss can convert the os represenation of the permission to the permissions set for later use in other nio classess or methods. In the next entry I will conver more on permissions and security by tackling the Access Control List (ACL) support in the nio.2