Introducing NIO.2 (JSR 203) Part 6: Filtering directory content and walking over a file tree

In this part we will look at how the directory tree walker and the directory stream reader works. These two features are another couple of long requested features which was not included in the core java before Java 7.

First, lets see what directory stream reader is, this API allows us to filter content of a directory on the file system and extract the file names that matches our filter criteria. The feature works for very large folders with thousands of files.

For filtration we can use PathMatcher expression matching the file name or we can filter the directory content based on the different file attributes. for example based on the file permissions or the file size.

Following sample code shows how to use the DirectoryStream along with filtering. For using the PathMatcher expression we can just use another overload of the newDirectoryStream method which accepts the PathMatcher expression instead of the filter.

public class DirectoryStream2 {

    public static void main(String args[]) throws IOException {
	//Getting default file system and getting a path
        FileSystem fs = FileSystems.getDefault();
        Path p = fs.getPath("/usr/bin");

	//creating a directory streamer filter
        DirectoryStream.Filter
 filter = new DirectoryStream.Filter
() {

            public boolean accept(Path file) throws IOException {
                long size = Attributes.readBasicFileAttributes(file).size();
                String perm = PosixFilePermissions.toString(Attributes.readPosixFileAttributes(file).permissions());
                if (size > 8192L && perm.equalsIgnoreCase("rwxr-xr-x")) {
                    return true;
                }
                return false;
            }
        };
	// creating a directory streamer with the newly developed filter
        DirectoryStream
 ds = p.newDirectoryStream(filter);
        Iterator
 it = ds.iterator();
        while (it.hasNext()) {

            Path pp = it.next();

            System.out.println(pp.getName());
        }
    }
}

The above code is self explaining and I will not explain it any further than the in-line comments.

Next subject of this entry is directory tree walking or basically file visitor API. This API allows us to walk over a file system tree and execute any operation we need over the files we found. The good news is that we can scan down to any depth we require using the provided API.

With the directory tree walking API, the Java core allows us to register a vaster class with the directory tree walking API and then for each entry the API come across, either a file or a folder, it calls our visitor methods. So the first thing we need is a visitor to register it with the tree walker. Following snippet shows a simple visitor which only prints the file type using the Files.probeContentType() method.

class visit extends SimpleFileVisitor
 {

    @Override
    public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) {
        try {
            System.out.println(Files.probeContentType(file));

        } catch (IOException ex) {
            Logger.getLogger(visit.class.getName()).log(Level.SEVERE, null, ex);
        }
        return super.visitFile(file, attrs);
    }

    @Override
    public FileVisitResult postVisitDirectory(Path dir, IOException exc) {
        return super.postVisitDirectory(dir, exc);
    }

    @Override
    public FileVisitResult preVisitDirectory(Path dir) {
        return super.preVisitDirectory(dir);
    }

    @Override
    public FileVisitResult preVisitDirectoryFailed(Path dir, IOException exc) {
        return super.preVisitDirectoryFailed(dir, exc);
    }

    @Override
    public FileVisitResult visitFileFailed(Path file, IOException exc) {
        return super.visitFileFailed(file, exc);
    }
}

As you can see we extended the SimpleFileVisitor and we have visitor methods for all possible cases.

Now that we have the visitor class, the rest of the code is straight forward. following sample shows how to walk over /home/masoud directory down to two levels.

public class FileVisitor {

    public static void main(String args[]) {

        FileSystem fs = FileSystems.getDefault();
        Path p = fs.getPath("/home/masoud");
        visit v = new visit();
        Files.walkFileTree(p, EnumSet.allOf(FileVisitOption.class), 2, v);

    }
}

You can grab the latest version of Java 7 aka Dolphin from here and from the same page you can grab the latest JavaDoc which is generated from the same source code that is used to generate the binary bits. Other entries of this series are under the NIO.2 tag of my weblog.

Introducing NIO.2 (JSR 203) Part 3: File System Attributes and Permissions support in NIO.2

In two previous entries I covered Introducing NIO.2 (JSR 203) Part 1: What are new features? and Introducing NIO.2 (JSR 203) Part 2: The Basics In this entry I will discuss Attributes introduced in NIO.2. Using attributes we can read platform specific attributes of an element in the file system. For example to hide a file system in DOS file system or to check the last access date of a file in a UNIX machine.

Using NIO.2 we can check which attributes are supported in the platform we are running on and then we can decide how to deal with the available attributes. Following sample code shows how we can detect the available attributes and then how to manipulate them.

  FileSystem fs = FileSystems.getDefault();
  Path p = fs.getPath("/home/masoud/netbeans-6.9-ml-linux.sh");
 //checking available attributes:
  Set<String> supportedViews = fs.supportedFileAttributeViews();
 //We always have at least one member in the set, the basic view.
  BasicFileAttributes ba = p.getFileAttributeView(BasicFileAttributeView.class, LinkOption.NOFOLLOW_LINKS).readAttributes();
 //Printing some basic attributes
   System.out.println(p.toString() + " last access:  " + ba.lastAccessTime());
   System.out.println(p.toString() + " last modified " + ba.lastModifiedTime());
        // As I know I am in NIX machine I access the unix attributes.
        // If I didnt I should have iterate over the set to determine which
        // attributes are supported
        if (supportedViews.contains("unix")) {
            PosixFileAttributes pat = Attributes.readPosixFileAttributes(p, LinkOption.NOFOLLOW_LINKS);
            System.out.println(pat.group().getName());
            System.out.println(pat.owner().getName());
        }

I placed plethora of comments on the code so reading and understanding it get easier.

In the next snippet we will see how we can read permissions of file system element. The first step in checking permissions is using the checkAccess method as shown below. the method throw an exception if the permission is not present or it will execute with no exception if the permission is present.

 FileSystem fs = FileSystems.getDefault();
 Path p = fs.getPath("/home/masoud/netbeans-6.9-ml-linux.sh");
    try {
            // A method to check the access permissin
            p.checkAccess(AccessMode.EXECUTE);
        } catch (IOException ex) {
            Logger.getLogger(perm.class.getName()).log(Level.SEVERE, null, ex);
        }

        // Extracting all permissions of a file and iterating over them.
        //I know that I am dealing with NIX fs so I go directly with that attributes
        // otherwise we should check which attributes are supported and then we can
        // use them.

        PosixFileAttributes patts = Attributes.readPosixFileAttributes(p, LinkOption.NOFOLLOW_LINKS);
        Set<PosixFilePermission> st = patts.permissions();
        for (Iterator<PosixFilePermission> it = st.iterator(); it.hasNext();) {
            System.out.println(it.next().toString());
        }

        // Using PosixFilePermissions to convert permissions to different representations
        System.out.println(PosixFilePermissions.toString(st));

As you can see in the code we can use the helper class to convert the permission set to a simpl OS represeted permission of the element. for example the set can be translated to rwx——  if the file has owner read, write and execute permissions attached to it. The helper calss can convert the os represenation of the permission to the permissions set for later use in other nio classess or methods. In the next entry I will conver more on permissions and security by tackling the Access Control List (ACL) support in the nio.2

Introducing NIO.2 (JSR 203) Part 2: The Basics

In this part we will discuss the basic classes that we will work with them to have file system operations like copying a file, dealing with symbolic links, deleting a file, and so on. I will write a separate entry to introduce classes which are new to Java 7 for dealing with streams and file contents, watching service and directory tree walking. If you want to know what are new features in Java SE 7 for dealing with IO take a look at   Introducing NIO.2 (JSR 203) Part 1: What are new features?

Before NIO.2, dealing with file system was mainly done using the File class and no other base class was available. In NIO.2 it there are some new classes at our disposal to take advantage of their existence to do our job.

FileSystems: Everything starts with this factory class. We use this class to get an instance of the FileSystem we want to work on. The nio.2 provides a SPI to developed support for new file systems. For example an in-memory file system, a ZIP file system and so on. Following two methods are most important methods in FileSystems class.

  1. The getDefault() returns the default file system available to the JVM. Usually the operating system default files system.
  2. The getFileSystem(URI uri) returns a file system from the set of available file system providers that match the given uir schema.

Path: This is the abstract class which provides us with all File system functionalities we may need to perform over a file, a directory or a link.

FileStore: This class represents the underplaying storage. For example /dev/sda2 in *NIX machines and I think c: in windows machines. We can access the storage attributes using  FileStoreSpaceAttributes object. Available space, empty space and so on.

Following two sample codes shows how to copy a file and then how to copy it.

public class Main {

    public static void main(String[] args) {

        try {
            Path sampleFile = FileSystems.getDefault().getPath("/home/masoud/sample.txt");
            sampleFile.deleteIfExists();
            sampleFile.createFile(); // create an empty file
            sampleFile.copyTo(FileSystems.getDefault().getPath("/home/masoud/sample2.txt"), StandardCopyOption.COPY_ATTRIBUTES.REPLACE_EXISTING);
            // Creating a link
            Path dir = FileSystems.getDefault().getPath("/home/masoud/dir");
            dir.deleteIfExists();
            dir.createSymbolicLink(sampleFile);
        } catch (IOException ex) {
            Logger.getLogger(Main.class.getName()).log(Level.SEVERE, null, ex);
        }
    }

And the next sample shows how we can use the FileStore class. In this sample we get the underlying store for a file and examined its attributes. We can an iterator over all available storages using FileSystem.getFileStores() method and examine all of them in a loop.

 public class Main {

    public static void main(String[] args) throws IOException {
        long aMegabyte = 1024 * 1024;
        FileSystem fs = FileSystems.getDefault();
        Path sampleFile = fs.getPath("/home/masoud/sample.txt");
        FileStore fstore = sampleFile.getFileStore();
        FileStoreSpaceAttributes attrs = Attributes.readFileStoreSpaceAttributes(fstore);
        long total = attrs.totalSpace() / aMegabyte;
        long used = (attrs.totalSpace() - attrs.unallocatedSpace()) / aMegabyte;
        long avail = attrs.usableSpace() / aMegabyte;
        System.out.format("%-20s %12s %12s %12s%n", "Device", "Total Space(MiB)", "Used(MiB)", "Availabile(MiB)");
        System.out.format("%-20s %12d %12d %12d%n", fstore, total, used, avail);

            }

In next entry I will discuss how we can manage file attributes along with discussing the security features of the nio.2 file system.