Masoud Kalali's Blog

My thoughts on software engineering and beyond…

By

My last blog in 2012: My 2013 wishes and predictions…

Last year I wrote a wish list and prediction for 2012 and this year I am going to do the same for 2013. So for 2013 in technology realm specially when focused on Java is as follow:

  • Java SE 8 will be an awesome release despite the Jigsaw setback.
  • Java EE 7 will bring more ease of use and clarity into the community.
  • GlassFish 4 will be awesome and and more people will benefit from it’s modular and extensible architecture…
  • In late 2013 NetBeans IDE 8 will rock!
  • IBM will push the idea of how cool Rational set of IDEs are and how good Websphere is and people will believe it until the are caught with no way to return.
  • RIM seems to be pulling it together and it is likely to keep its own operating system rather than adopting Android.
  • Google Chrome will continue eating other browsers marketshare as fast as browserly possible.
  • Some of the new cool boys in the JVM town that are claiming to be the next Java will vanish/start vanishing without any trace
  • I wish for a very thin and edge to edge tablet and cell phone on top of android so I could switch to another phone. This will be something that Google_Moto will do.
  • Maybe I see someone with a Windows Mobile phone somewhere other than advertisements.

What I wish for during 2013, unrelated to technology

  • No more war and instead of that some peace and quiet time around the globe.
  • No disasters like what we had in 2011 and instead some ground breaking scientific discoveries in medicine, energy and space travel.
  • No economy breakdown anywhere in the world.
  • To win more bets against my nemesis.

Other predictions for 2013 which I truly like to be proven wrong for some parts:

  • Iranian government will not go away and will not change to a sane governing body.
  • Pakistan army and ISI will continue supporting /training and harboring  Al Qaeda and Taliban and continue destabilizing Afghanistan southern and central provinces.
  • Iranian government will continue meddling in other countries affair specially in Afghanistan and Arab countries.
  • It is highly likely that Syrian dictatorship loose the battle for capital city and leave the capital but they will remain a player in the country and wreck havoc for the time being.

I wish everyone a happy new year with lots of joys and success.

By

My thoughts on JSR 351, Java Identity API

Identity, something that we hear more often these days with the whole web 2.0 and social services and more and more web based public services growing around us. The identity notion is an integral part of a security system in distributed services. Developing effective software system require an effective security and access control system which java provides, not exactly in the way that it should be in 2011 but it does provide what is the bare bone necessity to develop applications and frameworks on top of it and  benefit from its presence. The identity API is going to ease the interaction between the identity providers and those who consume the identity and trust the identity providers in addition to governing and managing the identity attributes.

I was studying the JSR details and it seems to be covering everything required for the identity attributes governance and the required API for both ends of the usage including the client API the governing/producing API. The identity producing and consuming is not new and there are fair number of public identity producers like facebook, twitter, etc. and also products that system integrators can use  like OpenAM as an open source product or gorilla Commercial software products like ORACLE identity management  or IBM tivoli identity management software, etc.

In a very simple set of words, the JSR 351: The Java Identity API will be as successful as it is going to be adopted. No adoption and it will endup dying some dark corner…  Design a simple and elegant API and try to ship it with some free and easy to use service implementations and it may get some momentum, otherwise it will be a goner and people will stick with what they have. I like the new features that it is going to introduce in the decision making or authorization part but we should see how well it will be adopted by identity providers to develop the services that provides the interaction point between the JSR interface and their repositories.  Pushing it as JSR wont really do that much without a wide adoption in the community. Look at how many implementation of the JSR 115 and JSR 196 exits to be plugged into application servers supporting the contract and you will get what I am referring to by community adoption.

By

A walkthrough for the fork/join framework introduced in Java SE 7

Java SE 7 brought some neat features on the table for Java developers, one of these features is the fork/join framework or basically the new parallel programming framework we can use to easily to implement our divide and conquer solutions. The point is that a solution to a problem should be devised with the following characteristics to use the fork/join framework effectively:

  • The problem domain whether it is a file, list, etc to be processed or a computation should be dividable to smaller subtasks.
  • Processing chunks should be possible without requiring the result of other chunks.

To summarize it, solving or processing the problem domain should require no self-feedback to make it possible to use the framework. For example if you want to process a list and processing each element in the list require the result of processing previous element then it is impossible to use any parallel computing for doing that job. If you want to apply some FFT over a sound stream which require feedback for processing each pulse from the previous pulses it is not possible to speedup the processing using the fork/join framework, etc.

Well, before we start learning the fork/join framework we better know what it is and what it is not: What fork/join framework is:

  • A parallel programming framework for Java
  • Part of Java SE 7
  • Suitable for implementing parallel processing solutions, mostly data intensive with small or no shared resources between the workers who process the data chunks.
  • Suitable when no synchronization is required between the workers

What fork/join framework is not:

  • It is not a magic that turns your code to run fast on machines with multiple processors, you need to think and implement your solutions in a parallel manner.
  • It is not hard and obscure like other frameworks, MPI for example. Using the framework is way easier than anything I used before.

If you want to learn the mechanics behind the fork/join framework you can read the original article written by Doug Le which explains the motive and the design. The article is available at http://gee.cs.oswego.edu/dl/papers/fj.pdf. If you want to see how we can use the framework then continue on reading this article.

First let’s see what are the important classes that one need to know in order to implement a divide and conquer solution using fork/join framework and then we will start using those classes.

  • The ForkJoinPool: This is the workers pool where you can post your ForkJoinTask to be executed. The default parallelism level is the number of processors available to the runtime.
  • The RecursiveTask<V>: This is a task, subclass of the ForkJoinTask which can return some value of type V. For example processing a list of DTOs and returning the result of process.
  • The RecursiveAction: Another subclass of the ForkJoinTask without any return value, for example processing an array…

I looked at this new API mainly for data pipelining in which I need to process a pretty huge list of object and turn it to another format to keep the processing result of one library consumable for the next one in the data flow and I am happy with the result pretty easy and straight forward.

Following is an small sample showing how to process a list of Row objects and convert them a list of Entity Objects. In my case it was something similar with processing Row objects and turning them to OData OEntity objects.


import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.ForkJoinPool;
import java.util.concurrent.RecursiveTask;

/**
 *
 * @author Masoud Kalali <mkalali>
 */
class RowConverter extends RecursiveTask<List<Entity>> {

    //if more than 5000 we will use the parallel processing
    static final int SINGLE_TREAD_TOP = 5000;
    int begin;
    int end;
    List<Row> rows;

    public RowConverter(int begin, int end, List<Row> rows) {
        this.begin = begin;
        this.end = end;
        this.rows = rows;
    }

    @Override
    protected List<Entity> compute() {

        if (end - begin <= SINGLE_TREAD_TOP) {
            //actual processing happens here
            List<Entity> preparedEntities = new ArrayList<Entity>(end - begin);
            System.out.println("  beging: " + begin + " end: " + end);
            for (int i = begin; i < end; ++i) {
                preparedEntities.add(convertRow(rows.get(i)));
            }
            return preparedEntities;
        } else {
            //here we do the dividing the work and combining the results
            // specifies the number of chunks you want to break the data to
            int divider = 5000;
            // one can calculate the divider based on the list size and the number of processor available 
            // using the http://download.oracle.com/javase/7/docs/api/java/lang/Runtime.html#availableProcessors()
            // decrease the divider number and examine the changes.

            RowConverter curLeft = new RowConverter(begin, divider, rows);
            RowConverter curRight = new RowConverter(divider, end, rows);
            curLeft.fork();
            List<Entity> leftReslt = curRight.compute();
            List<Entity> rightRes = curLeft.join();
            leftReslt.addAll(rightRes);
            return leftReslt;
        }
    }

    //dummy converted method converting one DTO to another
    private Entity convertRow(Row row) {

        return new Entity(row.getId());
    }
}

// the driver class which own the pool 
public class Fjf {

    public static void main(String[] args) {

        List<Row> rawData = initDummyList(10000);
        ForkJoinPool pool = new ForkJoinPool();
        System.out.println("number of worker threads: " + pool.getParallelism());


        List<Entity> res = pool.invoke(new RowConverter(0, rawData.size(), rawData));

        // add a breakpoint here and examine the pool object. 
        //check how the stealCount, which shows number of subtasks taken on by available workers, 
        //changes when you use an smaller divider and thus produce more tasks
        System.out.println("processed list: " + res.size());

    }

    /**
     * creates a dummy list of rows
     * 
     * @param size number of rows int he list
     * @return the list of @see Row objects
     */
    private static List<Row> initDummyList(int size) {

        List<Row> rows = new ArrayList<Row>(size);

        for (int i = 0; i < size; i++) {
            rows.add(new Row(i));
        }
        return rows;
    }
}

//dummy classes which should be converted from one form to another
class Row {

    int id;

    public Row(int id) {
        this.id = id;
    }

    public int getId() {
        return id;
    }
}

class Entity {

    int id;

    public Entity(int id) {
        this.id = id;
    }

    public int getId() {
        return id;
    }
}

Just copy and paste the code into your IDE and try running and examining it to get deeper understanding of how the framework can be used. post any comment and possible questions that you may have here and I will try to help you own with them.

By

JavaZone 2010, a very brief report.

Yesterday the two days JavaZone 2010 conference finished, the conference was as good as I expected it to be. The sessions were selected very carefully to cover all sort of interest from architecture and NoSQL to security and core java APIs.

A fair portion of the participants, both in speakers group and in the audiences group were loyal participants which at least attended one of the previous conferences in past 10 years. I was one of those who participated for the first time, so I can not compare this year’s event with the previous one but those who have seen the previous events agreed that the quality of each year’s conference is better than previous year, and for this year the quality increases is way more than previous ones.

For few instances I heard people complaining about why more speakers are using English or small number of elementary level sessions (tutorials perhaps) but I believe no one found another reason  for a complain.

The JourneyZone is what I am looking forward to experience in the next two days. It is the first time in the 10 years history of JavaZone that the program committee is organizing this event for the JZ participants. I am going to check out of the hotel and go for the JourneyZone.

Following album includes few photos from my session, thanks to Alexis for taking the photos. The slides is available here I know that all videos for the sessions will be available at some point in time, but I do not know when it will become available, maybe in few days maybe in few months.

By

Introducing NIO.2 (JSR 203) Part 6: Filtering directory content and walking over a file tree

In this part we will look at how the directory tree walker and the directory stream reader works. These two features are another couple of long requested features which was not included in the core java before Java 7.

First, lets see what directory stream reader is, this API allows us to filter content of a directory on the file system and extract the file names that matches our filter criteria. The feature works for very large folders with thousands of files.

For filtration we can use PathMatcher expression matching the file name or we can filter the directory content based on the different file attributes. for example based on the file permissions or the file size.

Following sample code shows how to use the DirectoryStream along with filtering. For using the PathMatcher expression we can just use another overload of the newDirectoryStream method which accepts the PathMatcher expression instead of the filter.

The above code is self explaining and I will not explain it any further than the in-line comments.

Next subject of this entry is directory tree walking or basically file visitor API. This API allows us to walk over a file system tree and execute any operation we need over the files we found. The good news is that we can scan down to any depth we require using the provided API.

With the directory tree walking API, the Java core allows us to register a vaster class with the directory tree walking API and then for each entry the API come across, either a file or a folder, it calls our visitor methods. So the first thing we need is a visitor to register it with the tree walker. Following snippet shows a simple visitor which only prints the file type using the Files.probeContentType() method.

As you can see we extended the SimpleFileVisitor and we have visitor methods for all possible cases.

Now that we have the visitor class, the rest of the code is straight forward. following sample shows how to walk over /home/masoud directory down to two levels.

You can grab the latest version of Java 7 aka Dolphin from here and from the same page you can grab the latest JavaDoc which is generated from the same source code that is used to generate the binary bits. Other entries of this series are under the NIO.2 tag of my weblog.

By

JavaZone 2010 sessions I am going to attend

I will be attending JavaZone 2010 both as an speaker presenting NIO.2 and as an atendee sitting and learning new technologies other speakers kindly share. After looking at the list of sessions, here is the sessions I decided to attend. It is really hard to decided on which speaker and subject to choose. All speakers are known engineers and developers and selected subjects are exceptionally good.

I may change few of this items as JZ2010 staff updates the agenda but majority of the sessions will be what I have already selected. See you at JZ2010 and the JourneyZone afterward the conference itself.

8th of September:

  1. Emergent Design — Neal Ford
  2. JRuby: Now With More J! — Nick Sieger
  3. Howto: Implement Collaborative Filtering with Map/Reduce – Ole-Martin Mørk
  4. Building a scalable search engine with Apache Solr, Hibernate Shards and MySQL – Aleksander Stensby, Jaran Nilsen
  5. The not so dark art of Performance Tuning — Dan Hardiker, Kirk Pepperdine
  6. Creating modular applications with Apache Aries and OSGi — Alasdair Nottingham
  7. Java 7 NIO.2: The I/O API for Future — Masoud Kalali -> I am Speaking here
  8. The evolution of data grids, from local caching to distributed computing – Bjørn Vidar Bøe

9th of September:

  1. NOSQL with Java — Aslak Hellesøy
  2. CouchDB and the web — Knut O. Hellan
  3. Decision Making in Software Teams — Tim Berglund
  4. A Practical Introduction to Apache Buildr — Alex Boisvert
  5. Architecture Determines Performance — Randy Stafford
  6. Surviving Java Maintenance – A mini field manual — Mårten Haglind
  7. Using the latest Java Persistence API 2.0 features — Arun Gupta

JavaZone 2010, Oslo

JavaZone 2010, JavaZone logo