Getting started with CouchDB and MongoDB, Part II: Beginning with Security

In the first part of this series I discussed how can we perform CRUD operation in MongoDB and CouchDB. In this part we will look into security features of MongoDB and CouchDB. In the next two installments of this series I will go into how to apply the configuration that are described here and how to write codes that work with them.

Basic security requirements

There are some basic features and functionalities which are required by different applications, in addition to tons of other more advanced fine grained features, which one can expect a semi-structured data storage server to provide. If the not provided, other measures should be applied to limit the effect of missing features risks. The basic features are as listed below:

  • Authentication: Authentication refers to ensuring that if a token is presented to our system the system can verify that the token is valid, for example a username can be verified using the password that accompany it.
  • Access control: To limit the access to specific resources/ set of resources to an authenticated client or a group of authenticated clients sharing common characteristics like a assigned role_name.
  • Transport safety: To ensure that the data is being transferred over the network safely without possibility of tampering or reading depending on the required level of confidentiality.
  • On-Disk/In-memory Data Encryption: To keep the data encrypted on disk to prevent RAW access to the data without presence of the decryption key/token. Same applies to the systems that are data memory intensive, the memory data in memory should be protected from memory dump/views.

General Security precautions

  • Do not run the server on default network interface which it is configured, usually 0.0.0.0 which means all available interfaces. Imagine you having a LAN and WAN interface and you want your NoSQL storage to only listen on the LAN interface while by default it listens on all interfaces (including your outbound WAN).
  • Do not use the default port numbers, change the port numbers that your backend application server, database, NoSQL storage, ABCD network server listen on. No need to wait for intrusion on the default port number.
  • Do not run your network server, e.g NoSQL server on root or any other privileged user.  Create a user which is relaxed enough in disk/network access to let the network server perform its task and not relaxer!
  • Do not let the above user to have access to any portion of file server other than the one that it requires.
  • Enable disk quota!
  • If on-disk/in-memory encryption is supported depending on sensitivity of the data, enable it.
  • Enable transport security. One way or another transport security is required to prevent others peeping, tampering the data or request responses!
  • Make sure you enable your iptable and limit the above user to only have access to port/interfaces that it meant to have access nothing more.
  • Disable any sample welcome page, any default “I am UP” message, any default “Welcome page” or anything else that your network server may have out of the box. This will prevent people from peeping into seeing what version of what network server you are running.
  • Disable any extra interface/communication channel that your network server has and you are not using it. E.g In an application server we just don’t need the JMX interface and thus we disable it rather than having a door into the application server, although the door is locked but…
  • Do not use anything default, no port, no default username, no default password, no default welcome message, no default… Nothing default!

MongoDB and Security

Below is some description on how MongoDB cover the basic security requirements.

  • Authentication: Supports two types of access. Either no authentication is enabled and anonymous users can do read/write or authentication is enabled (per database) and users are authenticated and then checked for their authorization before accessing the database. At the token level, CouchDB supports OAuth, cookie based authentication and http basic authentication. We will go into details of these in the next installments of this blog series.
  • Authorization: Supports role based access control. There are several roles which can be assigned to users. each role add some more privilege to the user(remember unauthenticated users cannot interact with the database when authentication is enabled). Details of the roles MongoDB access control documentation.
  • Transport Security: In the community edition there is no build-in transport security out of the box, you can use the MongoDB enterprise or you need to build it from the source to get SSL included in your non-enterprise copy. Procedure described at Configure SSL.
  • On-disk/In-memory encryption: I am not aware of any built-in support for any of these. One possibility is to use OS level disk encryption and/or use application level encryption for the sensitive fields, which imposes limitation on indexing and similarity searches, in the document.

CouchDB and Security

Below is some description on how Apache CouchDB cover the basic security requirements.

  • Authentication: By default any connection to CouchDB has access to perform CRUD on the server (creating/reading/deleting databases) and also CRUD operations on documents stored in the databases. When authentication is enabled, we go into how each one of these tasks can be performed in next blog entry, the user’s credentials is checked to let it perform CRUD on server or on databases or to perform read action on the database.
  • Authorization: There are 3 roles in CouchDB, the server admin role which can do anything and everything possible in the server, the database admin role on a database and the read-only role on a database.
  • Transport: Apache CouchDB has built-in supports for transport security using SSL and thus all communication that requires transport security (encryption, integrity) can be made over HTTPS.
  • On-Disk/In-memory encryption: I am not aware of any built-in support for any of these. One possibility is to use OS level disk encryption and/or use application level encryption for the sensitive fields in the document. Application level encryption will imposes limitation on indexing and similarity searches.
This blog entry is wrote based on CouchDB 1.3.1 and MongoDB 2.4.5 and thus the descriptions are valid for these versions and they may differ in other versions. The sample codes should work with the mentioned versions and any newer release.

Getting started with CouchDB and MongoDB, Part I: CRUD operations

In a series of blog I am going to write to cover CouchDB and MongoDB as two popular document databases. In each one of the entries I will show how can you do similar tasks with each one of these two so that you get some understanding of the differences and similarities between them. I am not going to do any comparison here but reading the series you can draw conclusion on which one seems more suitable for your case based judging by how different tasks can be accomplished using these two.

Some technicalities:

  • All of the source codes for these series is available at https://github.com/kalali/nosql which you can get and run the samples.
  • All sample codes assume default port numbers unless I mention otherwise in the blog entry itself.
  • All the blog entries use CouchDB 1.3.1 and MongoDB 2.4.5 and thus I expect them to work fine with this version and any newer ones.

A little about these two databases:

  • MongoDB and CouchDB: Both of them are document databases which lets you store freeform data into the storage in form of a document with key/value fields of data. Just look at JSON to see how a document would look like.
  • Apache CouchDB: Provides built-in support for reading/writing JSON documents over a REST interface. Result of a query is JSON and the data you store is in form of JSON all through the HTTP interface. There are tens of access API implemented, each with its own unique characteristics, around the REST interface in different languages which one can use to avoid using the REST interface directly. List of these implementations are available at: Related Projects. Apache CouchDB has more emphasis on availability rather than consistency if you look at it from CAP theorem perspective. CouchDB is cross platform and developed in ErLang.
  • MongoDB: MongoDB does not provide a built-in REST interface to interact with the document store and it does not provide direct support for storing and retrieving JSON documents however it provides a fast native protocol which can be access using the MongoDB Drivers and Client Libraries which are available for almost any language you can think of, and also there are some bridges which one can use to create a REST interface on top of the native interface. MongoDB has more emphasis on consistency rather than availability if you look at it from CAP theorem perspective. MongoDB is cross platform and developed in C++.

Downloading and starting:

  • MongoDB: You can download MongoDB from  the download page and after extracting it you can start it by navigating to bin directory and running ./mongo to start the document store. The default administration port is 28017 and the access port number is 27017 by default.
  • Apache CouchDB: You can download Apache CouchDB from the download page or if you are running some Linux distribution you can get it from the package repositories or you can just build it from the code. After installation run sudo couchdb to start the server.

The basic scenario for this blog entry is to create a database, add a document, find that document back, update the document and read it back from the store, delete the document and finally to delete the database to make the sample code’s execution repeatable.

The CouchDB sample code for the sample scenario:

import javax.json.JsonObject;
import javax.json.JsonReader;
import javax.ws.rs.client.Client;
import javax.ws.rs.client.ClientBuilder;
import javax.ws.rs.client.Entity;
import javax.ws.rs.core.Form;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.Response;
import java.io.StringReader;
import java.util.UUID;

/**
 * @author Masoud Kalali
 */

public class App {
    private static final String DB_BASE_URL = "http://127.0.0.1:5984/";
    private static final String STORE_NAME = "book_ds";
    private static Client c = ClientBuilder.newClient();

    public static void main(String[] args) {

        createDocumentStore();

        addingBookToDocumentStore();

        JsonObject secBook = readBook();

        String revision = secBook.getStringValue("_rev");
        updateBook(revision);

        JsonObject secBookAfterUpdate = readBook();
        deleteBookDocument(secBookAfterUpdate);

        deleteDocumentStore();
    }

    private static void createDocumentStore() {
        System.out.println("Creating Document Store...");
        javax.ws.rs.core.Response r = c.target(DB_BASE_URL).path(STORE_NAME).request().accept(MediaType.APPLICATION_JSON_TYPE).put(Entity.form(new Form()));
        if (!r.getStatusInfo().getFamily().equals(Response.Status.Family.SUCCESSFUL)) {
            throw new RuntimeException("Failed to create document storage: " + r.getStatusInfo().getReasonPhrase());
        }
    }

    private static void addingBookToDocumentStore() {
        System.out.println("Library document store created. Adding a book to the library...");
        javax.ws.rs.core.Response r = c.target(DB_BASE_URL).path(STORE_NAME).path("100").request(MediaType.APPLICATION_JSON_TYPE).put(Entity.text("{\n" +
                "   \"name\": \"GlassFish Security\",\n" +
                "   \"author\": \"Masoud Kalali\"\n" +
                "}"));

        if (!r.getStatusInfo().getFamily().equals(Response.Status.Family.SUCCESSFUL)) {
            throw new RuntimeException("Failed to create book document in the store: " + r.getStatusInfo().getReasonPhrase());
        }
    }

    private static JsonObject readBook() {

        System.out.println("Reading the book from the library...");
        javax.ws.rs.core.Response r = c.target(DB_BASE_URL).path(STORE_NAME).path("100").request(MediaType.APPLICATION_JSON_TYPE).get();
        if (!r.getStatusInfo().getFamily().equals(Response.Status.Family.SUCCESSFUL)) {
            throw new RuntimeException("Failed to read the book document from the data store: " + r.getStatusInfo().getReasonPhrase());

        }
        String ec = r.readEntity(String.class);
        System.out.println(ec);
        JsonReader reader = new JsonReader(new StringReader(ec));
        return reader.readObject();
    }

    private static void updateBook(String revision) {
        System.out.println("Updating the GlassFish Security book...");
        javax.ws.rs.core.Response r = c.target(DB_BASE_URL).path(STORE_NAME).path("100").request(MediaType.APPLICATION_JSON_TYPE).put(Entity.text(

                "{\"_rev\":\"" + revision + "\",\"name\":\"GlassFish Security And Java EE Security\",\"author\":\"Masoud Kalali\"}"
        ));

        if (!r.getStatusInfo().getFamily().equals(Response.Status.Family.SUCCESSFUL)) {
            throw new RuntimeException("Failed to update book document in the data store: " + r.getStatusInfo().getReasonPhrase());
        }
    }

    private static void deleteBookDocument(JsonObject secBook) {
        System.out.println("Deleting the the GlassFish Security book's document...");
        javax.ws.rs.core.Response r = c.target(DB_BASE_URL).path(STORE_NAME).path("100").queryParam("rev", secBook.getStringValue("_rev")).request().buildDelete().invoke();
        if (!r.getStatusInfo().getFamily().equals(Response.Status.Family.SUCCESSFUL)) {
            throw new RuntimeException("Failed to delete book document in the data store: " + r.getStatusInfo().getReasonPhrase());
        }
    }

    private static void deleteDocumentStore() {
        System.out.println("Deleting the document store...");
        javax.ws.rs.core.Response r = c.target(DB_BASE_URL).path(STORE_NAME).request().accept(MediaType.APPLICATION_JSON_TYPE).delete();
        if (!r.getStatusInfo().getFamily().equals(Response.Status.Family.SUCCESSFUL)) {
            throw new RuntimeException("Failed to delete document storage: " + r.getStatusInfo().getReasonPhrase());
        }
    }
}

I used JAX-RS and Some JSON-P in the code to add some salt to it rather than using plan http connection and plain-text content, but that aside the most important thing that you may want to remember is that we are using HTTP verbs for different actions, for example, PUT to create, DELETE to delete, POST to update and GET to query and read. The other important factor that you may have noticed is the _rev or the revision attribute of the document. You see that in order to update the document I specified the revision string, _rev in the document, also to delete the document I use the _rev id. This is required because at each point in time there might be multiple revisions of the document living in the database and you may read an older revision and that is the revision you can update (you can update a document revision because someone else might have updated the document while you were doing the update and thus your revision is old and you are updating your own revision).

The POM file for the couchDB sample is as follow:

  

    4.0.0

    me.kalali.couchdb
    couchdb_ch01
    1.0-SNAPSHOT
    jar

    couchdb_ch01
    http://maven.apache.org

    
        UTF-8
    

    
        
            org.glassfish.jersey.media
            jersey-media-moxy
            2.2
        
        
            org.glassfish.jersey.core
            jersey-client
            2.2
        
        
            org.glassfish
            javax.json
            1.0-b02
        
    

    
        

            
                org.codehaus.mojo
                exec-maven-plugin
                1.2.1
                
                    
                        run-main
                        package
                        
                            java
                        
                    
                
                
                    me.kalali.couchdb.ch01.App
                
            
        
    


Now, doing a clean install for the CouchDB project sample we get:

Creating Document Store...
Library document store created. Adding a book to the library...
Reading the book back from the library...
{"_id":"100","_rev":"1-99fb78c8607deb8ff966e2e2b3f3f7b3","name":"GlassFish Security","author":"Masoud Kalali"}

Updating the GlassFish Security book...
Book details after update:
{"_id":"100","_rev":"2-27c86bc501856c8bbe301b5a27f8da02","name":"GlassFish Security And Java EE Security","author":"Masoud Kalali"}
Deleting the the GlassFish Security book's document...
Deleting the document store...

If you pay attention you see that the _rev attribute has changed after updating the document, that is because by default read operation reads the latest revision of the document and in the second read after we updated the document the revision is changed and document that is read is the latest revision.

The MongoDB code for the sample scenario:

import com.mongodb.BasicDBObject;
import com.mongodb.DB;
import com.mongodb.DBCollection;
import com.mongodb.DBObject;
import com.mongodb.MongoClient;
import com.mongodb.WriteResult;

import java.net.UnknownHostException;

/**
 * @author Masoud Kalali
 */
public class App {
    private static final String STORE_NAME = "book_ds";
    private static MongoClient mongoClient;

    public static void main(String[] args) throws UnknownHostException {

        mongoClient = new MongoClient("127.0.0.1", 27017);

        DBCollection collection = createDocumentStore();

        addBookToCollection(collection);

        DBObject secBook = readBook(collection);
        System.out.println("Book details read from document store: \n" + secBook.toString());

        updateBook(collection);

        DBObject secBookAfterUpdate = readBook(collection);
        System.out.println("Book details after update: \n" + secBookAfterUpdate.toString());

        deleteBookDocument(collection);

        deleteDocumentStore();
    }

    private static DBCollection createDocumentStore() {
        System.out.println("Creating Document Store...");
        DB db = mongoClient.getDB(STORE_NAME);
        DBCollection collection = db.getCollection("books");
        return collection;
    }

    private static void addBookToCollection(DBCollection collection) {
        System.out.println("Library document store created. Adding a book to the library...");
        DBObject object = new BasicDBObject();
        object.put("name", "GlassFish Security");
        object.put("_id", "100");
        object.put("author", "Masoud Kalali");
        collection.insert(object);
    }

    private static DBObject readBook(DBCollection collection) {

        DBObject secBook = new BasicDBObject();
        secBook.put("_id", "100");
        return collection.findOne(secBook);
    }

    private static void updateBook(DBCollection collection) {
        System.out.println("Updating the GlassFish Security book...");
        DBObject matchings = collection.findOne(new BasicDBObject("_id", "100"));
        DBObject updateStmt = new BasicDBObject();
        updateStmt.put("name", "GlassFish Security And Java EE Security");
        updateStmt.put("_id", "100");
        updateStmt.put("author", "Masoud Kalali");
        WriteResult result = collection.update(matchings, updateStmt);
        System.out.println("Updated documents count: " + result.getN());
    }

    private static void deleteBookDocument(DBCollection collection) {
        DBObject object = new BasicDBObject();
        object.put("_id", "100");
        WriteResult result = collection.remove(object);
        System.out.println("Deleted documents count: " + result.getN());
    }

    private static void deleteDocumentStore() {
        System.out.println("Deleting the document store...");
        mongoClient.dropDatabase(STORE_NAME);
    }
}

As you can see in the code we are using the Mongo API rather than using direct HTTP communication and that is because MongoDB does not provide a built-in REST interface but rather it has its own native protocol as explained above. The other important part which will be in comparison with CouchDB is absence of revision or _rev or an attribute with similar semantic role. That is because MongoDB does not keep multiple version of the document Look at CAP theorem for more information on consistency and availability attributes of these two databases.

Now, the POM file for the MongoDB sample is as follow:

 

    4.0.0

    me.kalali.mongodb
    mongodb_ch01
    1.0-SNAPSHOT
    jar

    mongodb_ch01
    http://maven.apache.org

    
        UTF-8
    

    
        
            org.mongodb
            mongo-java-driver
            2.10.1
        
    

    
        
            
                org.codehaus.mojo
                exec-maven-plugin
                1.2.1
                
                    
                        run-main
                        package
                        
                            java
                        
                    
                
                
                    me.kalali.mongodb.ch01.App
                
            
        
    

The output for doing a clean install on the MongoDB sample will be as shown below:

Creating Document Store...
Library document store created. Adding a book to the library...
Book details read from document store:
{ "_id" : "100" , "name" : "GlassFish Security" , "author" : "Masoud Kalali"}
Updating the GlassFish Security book...
Updated documents count: 1
Book details after update:
{ "_id" : "100" , "name" : "GlassFish Security And Java EE Security" , "author" : "Masoud Kalali"}
Deleting the the GlassFish Security book's document...
Deleting the document store...

The sample code for this blog entry is available at GutHub:Kalali

This blog entry is wrote based on CouchDB 1.3.1 and MongoDB 2.4.5 and thus the descriptions are valid for these versions and they may differ in other versions. The sample codes should work with the mentioned versions and any newer release.


Configuring DHCP server in Solaris

A Dynamic Host Configuration Protocol (DHCP) server leases IP address to clients connected to the network and has DHCP client enabled on their network interface.
Before we can setup a start the DHCP server we need to install DHCP configuration packages. Detail information about installing packages in provided in recipe of chapter 1. But to save the time we can use the following command to install the packages.
# pkg install SUNWdhcs
After installing these packages we can continue with the next step.
How to do it…
First thing to setup the DHCP server is creating the storage and initial settings for the DHCP server. Following command does the trick for us.
# dhcpconfig -D -r SUNWfiles -p /fpool/dhcp_fs -a 10.0.2.254 -d domain.nme -h files -l 86400
In the above command we used several parameters and options, each one of these options are explained below.
  • The -D specifies that we are setting up a new instance of the DHCP service.
  • The -r SUNWfiles specifies the storage type. Here we are using plain-text storage while SUNWbinfiles and SUNWnisplus are available as well.
  • The -p /fpool/dhcp_fs specifies the absolute path to where the configuration files should be stored.
  • The -a 10.0.2.15 specifies the DNS server to use on the LAN. We can multiple comma separated addresses for DNS servers.
  • The -d domain.nme specifies the network domain name.
  • The -h files specifies where the host information should be stored. Other values are nisplus and dns.
  • The -l 86400 specifies the lease time in seconds.
Now that the initial configuration is created we should proceed to the next step and create a network.
# dhcpconfig -N 10.0.2.0 -m 255.255.255.0  -t 10.0.2.1
Parameters we used in the above command are explained below.
  • The -N 10.0.2.0 specifies the network address.
  • The -m 255.255.255.0 specifies the network mask to use for the network
  • The -t 10.0.2.1 specifies the default gateway
All configurations that we created are stored in DHCP server configuration files. We can manage the configurations using the dhtadm command. For example to view all of the current DHCP server configuration assemblies we can use the following command.
# dhtadm -P
This command’s output is similar to the following figure.
Each command we invoked previously is stored as a macro with a unique name in the DHCP configuration storage. Later on we will use these macros in subsequent commands.
Now we need to create a network of addresses to lease. Following command adds the addresses we want to lease.
# pntadm -C 10.0.2.0
If we need to reserve an address for a specific host or a specific interface in a host we should add the required configuration to the system to ensure that our host or interface receives the designated IP address. For example:
# pntadm -A 10.0.2.22 -f MANUAL -i 01001BFC92BC10 -m 10.0.2.0 -y 10.0.2.0
In the above command we have:
  • The -A 10.0.2.22 adds the IP address 10.0.2.22.
  • The -f MANUAL sets the flag MANUAL in order to only assign this IP address to the MAC address specified.
  • The -i 01001BFC92BC10 sets the MAC address for the host this entry assigned  to it.
  • The -m 10.0.2.0 specifies that this host is going to use the 192.168.0.0 macro.
  • The –y asks the command to verify that the macro entered actually exists.
  • The 10.0.2.0 Specifies the network the address is assigned to.
Finally we should restart the DHCP server in order for all the changes to take effect. Following command restarts the corresponding service.
#  svcadm restart dhcp-server
When we setup the DHCP service, we store the related configuration in the storage of our choice. When we start the service, it reads the configuration from the storage and wait dormant until it receives a request for leasing an IP address. The service checks the configuration and if an IP was available for lease, it leases the IP to the client.
Prior to leasing the IP, DHCP service checks all leasing conditions like leasing a specific IP address to a client to ensure that it leases the right address to a client, etc.
We can use the DHCP Manager GUI application to configure a DHCP server. The DHCP manager can migrate the DHCP storage from one format to another. To install the DHCP manager package we can use the following command.
# pkg install SUNWdhcm
Now we can invoke the DHCP manager using the following command which opens the DHCP Manager welcome page shown in the following figure.
# dhcpmgr

My refcard for Oracle Berkeley DB Java edition published by DZone

The refcard discuss the following items: 

  1. The BDB Family : An introduction to different DBD family members, including BDB Base Edition, BDB XML edition and BDB Java Edition with tables comparing their features.
  2. Key Features: Key features of BDB family members and BDB Java Edition exclusive features are explained here
  3. Introducing Berkeley DB Java Edition includin:

  • Installation: How to install the BDB JE and which JAR files are required to be in class path.
  • Description of Access APIs: Three different types of access APIs are explained and a working sample for each type of access APIs is included. Discussed APIs are Base API (Key/Value pair store and retrieval), DPL (Direct Persistence Layer for object persistence), Collections API(Using the BDB JE as an on-disk collection

        4. BDB Java Edition environment anatomy: How BDB JE store information on hard disk and what is directory and file layout of BDB JE is discussed here
5. BDB Java Edition APIs characteristics: What makes BDB JE different and how these differences can help developers in different scenarios.
6. Transaction Support: explanation and sample code for using Transaction with Base API and DPL
7. Persisting Complex Object Graph using DPL: How to define relation between objects using annotation and how to persist and retrieve related objects is explained here
8. BDB JE Backup/Recovery and Tuning: How to create backup, restore the backup and what tuning factor can be used to fine tune the BDB JE is explained here
9. Helper Utilities: Helper utilities for exporting and importing data are explained here
You can download the refcard free of charge from here.

 

You will find several tables and illustrations to make understanding the the whole concept of BDB JE and differences between BDB JE, BDB XML Edition and BDB Base Edition better. All included sample codes are fully explained.