Masoud Kalali's Blog

My thoughts on software engineering and beyond…

By

Getting started with CouchDB and MongoDB, Part II: Beginning with Security

In the first part of this series I discussed how can we perform CRUD operation in MongoDB and CouchDB. In this part we will look into security features of MongoDB and CouchDB. In the next two installments of this series I will go into how to apply the configuration that are described here and how to write codes that work with them.

Basic security requirements

There are some basic features and functionalities which are required by different applications, in addition to tons of other more advanced fine grained features, which one can expect a semi-structured data storage server to provide. If the not provided, other measures should be applied to limit the effect of missing features risks. The basic features are as listed below:

  • Authentication: Authentication refers to ensuring that if a token is presented to our system the system can verify that the token is valid, for example a username can be verified using the password that accompany it.
  • Access control: To limit the access to specific resources/ set of resources to an authenticated client or a group of authenticated clients sharing common characteristics like a assigned role_name.
  • Transport safety: To ensure that the data is being transferred over the network safely without possibility of tampering or reading depending on the required level of confidentiality.
  • On-Disk/In-memory Data Encryption: To keep the data encrypted on disk to prevent RAW access to the data without presence of the decryption key/token. Same applies to the systems that are data memory intensive, the memory data in memory should be protected from memory dump/views.

General Security precautions

  • Do not run the server on default network interface which it is configured, usually 0.0.0.0 which means all available interfaces. Imagine you having a LAN and WAN interface and you want your NoSQL storage to only listen on the LAN interface while by default it listens on all interfaces (including your outbound WAN).
  • Do not use the default port numbers, change the port numbers that your backend application server, database, NoSQL storage, ABCD network server listen on. No need to wait for intrusion on the default port number.
  • Do not run your network server, e.g NoSQL server on root or any other privileged user.  Create a user which is relaxed enough in disk/network access to let the network server perform its task and not relaxer!
  • Do not let the above user to have access to any portion of file server other than the one that it requires.
  • Enable disk quota!
  • If on-disk/in-memory encryption is supported depending on sensitivity of the data, enable it.
  • Enable transport security. One way or another transport security is required to prevent others peeping, tampering the data or request responses!
  • Make sure you enable your iptable and limit the above user to only have access to port/interfaces that it meant to have access nothing more.
  • Disable any sample welcome page, any default “I am UP” message, any default “Welcome page” or anything else that your network server may have out of the box. This will prevent people from peeping into seeing what version of what network server you are running.
  • Disable any extra interface/communication channel that your network server has and you are not using it. E.g In an application server we just don’t need the JMX interface and thus we disable it rather than having a door into the application server, although the door is locked but…
  • Do not use anything default, no port, no default username, no default password, no default welcome message, no default… Nothing default!

MongoDB and Security

Below is some description on how MongoDB cover the basic security requirements.

  • Authentication: Supports two types of access. Either no authentication is enabled and anonymous users can do read/write or authentication is enabled (per database) and users are authenticated and then checked for their authorization before accessing the database. At the token level, CouchDB supports OAuth, cookie based authentication and http basic authentication. We will go into details of these in the next installments of this blog series.
  • Authorization: Supports role based access control. There are several roles which can be assigned to users. each role add some more privilege to the user(remember unauthenticated users cannot interact with the database when authentication is enabled). Details of the roles MongoDB access control documentation.
  • Transport Security: In the community edition there is no build-in transport security out of the box, you can use the MongoDB enterprise or you need to build it from the source to get SSL included in your non-enterprise copy. Procedure described at Configure SSL.
  • On-disk/In-memory encryption: I am not aware of any built-in support for any of these. One possibility is to use OS level disk encryption and/or use application level encryption for the sensitive fields, which imposes limitation on indexing and similarity searches, in the document.

CouchDB and Security

Below is some description on how Apache CouchDB cover the basic security requirements.

  • Authentication: By default any connection to CouchDB has access to perform CRUD on the server (creating/reading/deleting databases) and also CRUD operations on documents stored in the databases. When authentication is enabled, we go into how each one of these tasks can be performed in next blog entry, the user’s credentials is checked to let it perform CRUD on server or on databases or to perform read action on the database.
  • Authorization: There are 3 roles in CouchDB, the server admin role which can do anything and everything possible in the server, the database admin role on a database and the read-only role on a database.
  • Transport: Apache CouchDB has built-in supports for transport security using SSL and thus all communication that requires transport security (encryption, integrity) can be made over HTTPS.
  • On-Disk/In-memory encryption: I am not aware of any built-in support for any of these. One possibility is to use OS level disk encryption and/or use application level encryption for the sensitive fields in the document. Application level encryption will imposes limitation on indexing and similarity searches.
This blog entry is wrote based on CouchDB 1.3.1 and MongoDB 2.4.5 and thus the descriptions are valid for these versions and they may differ in other versions. The sample codes should work with the mentioned versions and any newer release.

By

Getting started with CouchDB and MongoDB, Part I: CRUD operations

In a series of blog I am going to write to cover CouchDB and MongoDB as two popular document databases. In each one of the entries I will show how can you do similar tasks with each one of these two so that you get some understanding of the differences and similarities between them. I am not going to do any comparison here but reading the series you can draw conclusion on which one seems more suitable for your case based judging by how different tasks can be accomplished using these two.

Some technicalities:

  • All of the source codes for these series is available at https://github.com/kalali/nosql which you can get and run the samples.
  • All sample codes assume default port numbers unless I mention otherwise in the blog entry itself.
  • All the blog entries use CouchDB 1.3.1 and MongoDB 2.4.5 and thus I expect them to work fine with this version and any newer ones.

A little about these two databases:

  • MongoDB and CouchDB: Both of them are document databases which lets you store freeform data into the storage in form of a document with key/value fields of data. Just look at JSON to see how a document would look like.
  • Apache CouchDB: Provides built-in support for reading/writing JSON documents over a REST interface. Result of a query is JSON and the data you store is in form of JSON all through the HTTP interface. There are tens of access API implemented, each with its own unique characteristics, around the REST interface in different languages which one can use to avoid using the REST interface directly. List of these implementations are available at: Related Projects. Apache CouchDB has more emphasis on availability rather than consistency if you look at it from CAP theorem perspective. CouchDB is cross platform and developed in ErLang.
  • MongoDB: MongoDB does not provide a built-in REST interface to interact with the document store and it does not provide direct support for storing and retrieving JSON documents however it provides a fast native protocol which can be access using the MongoDB Drivers and Client Libraries which are available for almost any language you can think of, and also there are some bridges which one can use to create a REST interface on top of the native interface. MongoDB has more emphasis on consistency rather than availability if you look at it from CAP theorem perspective. MongoDB is cross platform and developed in C++.

Downloading and starting:

  • MongoDB: You can download MongoDB from  the download page and after extracting it you can start it by navigating to bin directory and running ./mongo to start the document store. The default administration port is 28017 and the access port number is 27017 by default.
  • Apache CouchDB: You can download Apache CouchDB from the download page or if you are running some Linux distribution you can get it from the package repositories or you can just build it from the code. After installation run sudo couchdb to start the server.

The basic scenario for this blog entry is to create a database, add a document, find that document back, update the document and read it back from the store, delete the document and finally to delete the database to make the sample code’s execution repeatable.

The CouchDB sample code for the sample scenario:

I used JAX-RS and Some JSON-P in the code to add some salt to it rather than using plan http connection and plain-text content, but that aside the most important thing that you may want to remember is that we are using HTTP verbs for different actions, for example, PUT to create, DELETE to delete, POST to update and GET to query and read. The other important factor that you may have noticed is the _rev or the revision attribute of the document. You see that in order to update the document I specified the revision string, _rev in the document, also to delete the document I use the _rev id. This is required because at each point in time there might be multiple revisions of the document living in the database and you may read an older revision and that is the revision you can update (you can update a document revision because someone else might have updated the document while you were doing the update and thus your revision is old and you are updating your own revision).

The POM file for the couchDB sample is as follow:

Now, doing a clean install for the CouchDB project sample we get:

If you pay attention you see that the _rev attribute has changed after updating the document, that is because by default read operation reads the latest revision of the document and in the second read after we updated the document the revision is changed and document that is read is the latest revision.

The MongoDB code for the sample scenario:

As you can see in the code we are using the Mongo API rather than using direct HTTP communication and that is because MongoDB does not provide a built-in REST interface but rather it has its own native protocol as explained above. The other important part which will be in comparison with CouchDB is absence of revision or _rev or an attribute with similar semantic role. That is because MongoDB does not keep multiple version of the document Look at CAP theorem for more information on consistency and availability attributes of these two databases.

Now, the POM file for the MongoDB sample is as follow:

The output for doing a clean install on the MongoDB sample will be as shown below:

The sample code for this blog entry is available at GutHub:Kalali

This blog entry is wrote based on CouchDB 1.3.1 and MongoDB 2.4.5 and thus the descriptions are valid for these versions and they may differ in other versions. The sample codes should work with the mentioned versions and any newer release.


By

My last blog in 2012: My 2013 wishes and predictions…

Last year I wrote a wish list and prediction for 2012 and this year I am going to do the same for 2013. So for 2013 in technology realm specially when focused on Java is as follow:

  • Java SE 8 will be an awesome release despite the Jigsaw setback.
  • Java EE 7 will bring more ease of use and clarity into the community.
  • GlassFish 4 will be awesome and and more people will benefit from it’s modular and extensible architecture…
  • In late 2013 NetBeans IDE 8 will rock!
  • IBM will push the idea of how cool Rational set of IDEs are and how good Websphere is and people will believe it until the are caught with no way to return.
  • RIM seems to be pulling it together and it is likely to keep its own operating system rather than adopting Android.
  • Google Chrome will continue eating other browsers marketshare as fast as browserly possible.
  • Some of the new cool boys in the JVM town that are claiming to be the next Java will vanish/start vanishing without any trace
  • I wish for a very thin and edge to edge tablet and cell phone on top of android so I could switch to another phone. This will be something that Google_Moto will do.
  • Maybe I see someone with a Windows Mobile phone somewhere other than advertisements.

What I wish for during 2013, unrelated to technology

  • No more war and instead of that some peace and quiet time around the globe.
  • No disasters like what we had in 2011 and instead some ground breaking scientific discoveries in medicine, energy and space travel.
  • No economy breakdown anywhere in the world.
  • To win more bets against my nemesis.

Other predictions for 2013 which I truly like to be proven wrong for some parts:

  • Iranian government will not go away and will not change to a sane governing body.
  • Pakistan army and ISI will continue supporting /training and harboring  Al Qaeda and Taliban and continue destabilizing Afghanistan southern and central provinces.
  • Iranian government will continue meddling in other countries affair specially in Afghanistan and Arab countries.
  • It is highly likely that Syrian dictatorship loose the battle for capital city and leave the capital but they will remain a player in the country and wreck havoc for the time being.

I wish everyone a happy new year with lots of joys and success.

By

My sessions in JavaOne 2012

It is long since last I blogged and I go forward I find less and less time to put into tech blogging. I thought to post in my blog that I will be presenting three sessions in Java one along or accompanying another speaker.  As you may expect two sessions evolve around security and one session covers some interesting features and APIs provided by Java EE 7. The sessions are as follow:

Top 10 OWASP security concerns and how to address them with Java EEMarkus Eisele and I will be presenting this session.

The hottest topic for Enterprise Java applications out there is security which Java EE provides fair deal of flexibility to benefit from. This session focuses on how the top 10 OWASP application security concerns including “Broken Authentication and Session Management”, “Failure to Restrict URL Access” and “Security Misconfiguration” can be addressed using Java EE and GlassFish application server. The session covers how and where to integrate the solution for each OWASP concerns through a demo application. The demo application is developed considering the relevant best practices/design patterns involved with developing a secure application while avoiding the common pitfalls.

Server Sent Events, Async Servlet, Web Sockets and JSON; born to work together!: This BOF session is hosted by a very experienced and long standing Metro/ GlassFish team member, Bhakti Mehta and me.

This session focuses on how Java EE 7 provides extensive set of new and enhanced features to support standards like HTML5, WebSockets, and Server Sent Events among others.In this session we will show how these new features are designed and matched to work together for developing lightweight solutions matching end users high expectation from a web application’s responsiveness. The session will cover best practices and design patterns governing application development using JAX-RS 2.0, Async Servlet, and JSON-P (among others) as well as iterating over the pitfalls that should be avoided. During the session we will show code snippets and block diagrams that clarify use of APIs coming from the demo application we will show at the end.

Utilize the Full Power of GlassFish Server and Java EE Security: I will be you host for this session.

In this session, learn how to utilize Java EE security and what GlassFish Server technology provides to address your security requirements. The presentation explains a two-phase authentication mechanism.

If you are attending JavaOne you may like to join this sessions and if you are not attending and still interested in this sessions, Keep an eye on @MasoudKalali or check this blog sometime after JavaOne 2012 to get the slides.

By

From NetBeans to IntelliJ IDEA, Week 1

I have been using NetBeans for a long time now; Because it is easy to use, easy to understand and explain to others, it has impressive Java EE support, etc. I was working on Java EE projects or projects  involving NetBeans RCP applications with 40-50 modules (1m loc or so) which are considered small applications. I didn’t need to have all modules opened or even when I needed NetBeans handled it pretty well. After joining GlassFish team things started to change a little. GlassFish codebase was new to me and the number of maven modules in GlassFish project was way larger than what I could easily open in NetBeans IDE to be able to navigate around in the code, to find the usages of a method, or to locate all of the implementing classes of a particular interface, etc.

I opened a RFE in NetBeans bug-tracking system but till that RFE got addressed I needed to work and thus I decided to give IntelliJ a try and see how it fares with GlassFish codebase and how easy it will be for me to adopt it as the IDE of choice for my day to day work on GlassFish.

First days impressions (Positive):

  • Can import the entire GlassFish codebase and start using it, no delay in code assisting popup and amazingly no lag after the initial scanning and indexing of the entire codebase.
  • Find usage works almost perfectly, sometimes it says that the class/es are outside of the codebase which might be my fault when importing the project
  • Code completion assistant is fast, when I say fast I mean really fast.
  • I like the fact that I can choose between different available LAFs. [The CDE/Motif LAF is still part of the JRE which denotes the overzealousness of keeping the backward compatibility of the platform?]
  • I like the “Store Current Layout as Default” feature which let me store a layout and use it when the layout get too messed-up.
  • I like the “Autoscroll to source” and “Autoscroll from source” though I’d rather have the action to do it manually when the mentioned options are disabled and I want to locate a class in the project view

First days impression (Not positive ones): Some of what I am missing might be available somewhere in the IDE but I haven’t find them yet….

  • I was not able to switch between different profiles for the module, for example IDE, release, etc. Maybe it is somewhere but I have not found it yet.
  • I think the color schema is hard to read because some of the used colors are low contrast, for example the comments, annotations, are hard to read and required some adjustment to be easier to read.
  • I opened some views that I cannot close, IDETalk, commander, Ant Build… they are all on auto-hide in the vertical bar in the left side without any close button or close action in the context (right click) menu to remove the tab from the side bar. Same goes for view tabs appearing in the bottom bar and left bar. But the good thing is we can rearrange them for better accessibility.
  • I think the local history feature in NetBeans has a better UI and usability than IDEA. In NetBeans the local history is shown in the same tab as the code and not in an extra window same goes for  diff window, etc.
  • The code formatter works better in NetBeans (SHIFT+CTRL+F) compared to IntelliJ (CMD+ALT+L). I like the NetBeans hotkey better as it has something from Formatting in the hotkey. (I tried couple of unindented unglized code snippet in both).
  • The Favorites view does not allow me to add a file system directory to a favorites list which limits the favorites view usage to project members (classes, packages, modules) …

I will post another blog entry after I used IntelliJ for another two weeks to give you a more in depth comparison of how they work for Mavan projects (my comparison will be mostly on Editors and Project/ navigation and refactoring features. Nothing on other languages support, Java EE, JavaFx, application server support, etc.)

By

Using GlassFish domain templates to easily create several customized domains

It might have happened to you to require some customization the GlassFish behavior after you create the domain in order to make the domain fit the  basic requirements that you have in your organization or for your development purpose. Some of the files that we usually manipulate to customize GlassFish includes logging.properties, keystore.jks, cacert.jks, default-web.xml, server.policy and domain.xml. These files can be customized through different asadmin commands, or JDK commands like keytool, policytool or manually using a text editor after you created the domain in the config directory of the domain itself.  But repeating the steps for multiple domains is a laborious task which can be prevented by changing the template files that GlassFish domains are created using them. The templates are located atAnd we can simply open them and edit the properties to make them more fit to our needs.

The benefit of modifying the templates rather than copy pasting the config directory of one domain to another is the domain specific behaviors like port numbers which have placeholders in the domain.xml to be filled by asadmin command. An example of a placeholder is %%%JMS_PROVIDER_PORT%%% which will be replaced by JMS provider port by asadmin command.