Tug’s Blog

My journey in Big Data, Hadoop, NoSQL and MapR

How to Get the Latest Document by Date/time Field?

| Comments

I read this question on Twitter, let me answer the question in this short article.

First of all you need to be sure your documents have an attribute that contains a date ;), something like :

To get the “latest hired employee” you need to create a view, and emit the hire date as key. The important part is to check that this date is emitted in a format that is sorted properly, for example an array of value using dateToArray function, or the time as numerical value. In the following view I am using the date as an array like that I will be able to do some grouping but this is another topic. The view looks like the following:

Now that you have a view. You can now query it using the parameters:

  • descending = true
  • limit = 1

If you use Java SDK the code will look like the following :

Finally it is important when you work with views to understand how the index are managed by the server so be sure your read the chapter “Index Updates and the stale Parameter”.

Introduction to Collated Views With Couchbase 2.0

| Comments

Most of the applications have to deal with “master/detail” type of data:

  • breweries and beer
  • department and employees
  • invoices and items

This is necessary for example to create application view like the following:

With Couchbase, and many of the document oriented databases you have different ways to deal with this, you can:

  • Create a single document for each master and embed all the children in it
  • Create a master and child documents and link them using an attribute.

In the first case when all the information are stored in a single document it is quite easy to use the entire set of data and for example create a screen that shows all the information, but what about the second case?

In this post I am explaining how it is possible to use Couchbase views to deal with that an make it easy to create master/detail views.

As an ex-Oracle employee, I am using the infamous SCOTT schema with the DEPT and EMP tables, as the first example. Then at the end I will extend this to the beer sample data provided with Couchbase.

The Data

Couchbase is a schema-less database, and you can store “anything you want” into it, but for this you need to use JSON documents and create 2 types of document : “department” and “employee”.

The way we usually do that is using a technical attribute to type the document. So the employee and department document will look as follow :

Department

1
2
3
4
5
6
{
  "type": "dept",
  "id": 10,
  "name": "Accounting",
  "city": "New York"
}

Employee

1
2
3
4
5
6
7
8
9
{
  "type": "emp",
  "id": 7782,
  "name": "Blake",
  "job": "Clark",
  "manager": 7839,
  "salary": 2450,
  "dept_id": "dept__10"
}

This shows just the document, in Couchbase you have to associate a document to a key. For this example I am using a simple pattern : type__id, for these documents the keys will look like the following:

  • dept__10
  • emp__20

You can use any pattern to create a key, for example for the employee you could chose to put an email.

Note the “dept_id” attribute in the employee document. This is the key of the department; you can see that as the “foreign key”. But remember, the relationship between the department and employee documents are managed entirely by the application, Couchbase Server does not enforce it.

I have created a Zip file that contains all the data, you can download it from here; and import the data into Couchbase using the cbdocloader utility. To import the data run the following command from a terminal window:

1
./cbdocloader -n 127.0.0.1:8091 -u Administrator -p password -b default ~/Downloads/emp-dept.zip

You can learn more about the cbdocloader tool in the documentation.

The View

Queries inside Couchbase are based on views; and views build indexes, so we have to create a view, a “collated view” to be exact.

The idea behing a collated view is to produce an index where the keys are ordered so that a parent id appears first followed by its children. So we are generating an index that will look like:

1
2
3
4
5
6
7
DEPT_10, Accounting
DEPT_10, Blake
DEPT_10, Miller
DEPT_20, Research
DEPT_20, Adams
DEPT_20, Ford
...

This is in fact quite easy to do with Couchbase views. The only trick here is to control the order and be sure the master is always the first one, just before its children.

So to control this we can create an compound key that contains the department id, a “sorting” element and the name (beer or brewery)

So the map function of the view looks like the following:

The key is composed of:

  • the department id extracted from the department document itself or from the employee document depending of the type of document
  • an arbitrary number that is used to control the ordering. I put 0 for the department, 1 for the employee
  • the name of the department or the employee, this also allows to sort the result by name

In addition to the key, this view is used to emit some information about the salary of the employees. The salary is simply the sum of the salary plus thecommission when exists. The result of the view looks like:

With this view you can now use the result of the view to build report for your application. It is also possible to use parameters in your query to see only a part of the data, for example by departement, using for example startkey=["dept__20",0]&endkey=["dept__20",2] to view only the data -Department and Employees- of the deparment 20-Research.

The Beer Sample Application

You can create an equivalent view for the beer sample application where you print all the breweries and beers in the same report. The view is called “all_with_beers” in the design document “brewery”. The view looks like:

Once you have publish it in production you can use it in the Beer Sample application, for this example I have modified the Java sample application.

Create a servlet to handle user request and on the /all URI.

The “BreweryAndBeerServlet” that calls the view using the following code :

The result of the query is set into the HttpRequest and the all.jsp page is executed. The JSP uses JSTL to print the information using the following code:

The JSP gets the items from the HTTP Request and loops on each items, then based on the type of the item the information is printed. The final result looks like :

This extension to the Beer Sample application is available here :https://github.com/tgrall/beersample-java/tree/BreweriesAndBeers

Getting Started With Couchbase and node.js on Windows

| Comments

In a previous post I have explained how to use Couchbase and Node.js on OS X. Since it is quite different on Windows here another article about it.

Install Couchbase Server 2.0

If you have not installed Couchbase Server already, do the following :

  1. Download Couchbase Server from here
  2. Run the installer
  3. Configure the database at http://localhost:8091 (if you have issue take a look to this article)

These steps are documented in the Couchbase Server Manual.

Install Node

Install latest version of node

It is quite easy to install Node.js using the Windows installer provided at http://nodejs.org.

Once you have installed node, you can test is using the command line interface:

1
2
3
node
> console.log(process.version);
v0.8.16

Node is installed. So far so good !

Install Couchnode

Couchnode, the Couchbase Client Library for Node.js, is a native module. The tool used to install native modules is node-gyp. So to install couchnode you need to install :

  • node-gyp
  • python
  • Visual Studio to have access to a C/C++ compiler

Install node-gyp

The node-gyp module is easy to install and you can install it using npm using the following command:

1
npm install -g node-gyp

The -g parameter indicates that this module will be installed globally and added to your %PATH%.

Install Python

GYP uses Python to generate the project, so you need to install it on your environment. I have installed Python 2.7.3 using the Windows installer.

Install Visual Studio

Finally you need a C/C++ compiler, the best way to get it is to install Visual Studio. As you probably know I am not a Windows expert and I do not know a lot about Microsoft development tools. I have downloaded Visual Studio Express the free development tools from here; it was sufficient.

Install Libcouchbase for Windows

Couchnode uses libcouchbase the C client library, so before running the npm install for Couchbase, you need to install libcouchbase itself.

You can download it from Couchbase site. The Windows versions are located in the left menu of the page. Download the zip file, that matches your environment. I have downloaded the “Windows, 64-bit MSVC 10”.

Node-gyp will look for all the dependencies (DLL, library headers) into c:\couchbase directory, so you need to unzip the file in this folder. This location comes from the binding.gyp file of the couchnode project.

Install and Test Couchnode itself!

Let’s check what we have done so far; we have installed:

  • Node
  • node-gyp
  • Python
  • Visual Studio
  • Libcouchbase

We are now ready to install and use couchnode itself. For this we can create a new node project.

1
2
3
mkdir my-app
cd my-app
npm install couchbase

The install command will:

  • Create a node_modules folder and put couchbase client library in it
  • When installing/building couchnode on Windows I had the following warning :

C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V110\Microsoft.CppBuild.targets(1138,5): warning MSB8012: TargetExt(.dll) does not match the Linker’s Output File property value (.node). This may cause your project to build incorrectly. To correct this, please make sure that $(OutDir), $(TargetName) and $(TargetExt) property values match the value specified in %(Link.OutputFile). [C:\Users\tgrall\node\node_modules\couchbase\build\couchbase_impl.vcxproj]

This is only a warning and as far as I know, it is not a blocker. At the end of the log you should see:

1
2
3
couchbase@0.0.10 node_modules\couchbase
├── bindings@1.0.0
└── request@2.11.4

You have successfully installed couchnode.

Let’s now write a small test. Create a test.js file with the following content:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
var  driver = require('couchbase');

driver.connect({
  "username": "",
  "password": "",
  "hostname": "localhost:8091",
  "bucket": "default"},
  function(err, cb) {
    if (err) {
      throw (err)
    }

    var key = 'foo';
    cb.set(key, '{"server" : "couchbase", "version" : 2 }' , function (err, meta) {
      if (err) { console.log(err); }
      cb.get(key, function(err, doc) {
        if (err){ console.log(err);}
        console.log(doc);
      });
    });
  });

Run this with the command:

1
node test.js

You should see the following text in your console :

1
{ server: 'couchbase', version: 2 }

Conclusion

In this article you have learned how to:

  • Install Couchbase
  • Install Node
  • Install and configure node-gyp
  • Install and use Couchbase and Node

all this on Windows 7.

Couchbase 101: Create Views (MapReduce) From Your Java Application

| Comments

When you are developing a new applications with Couchbase 2.0, you sometimes need to create view dynamically from your code. For example you may need this when you are installing your application, writing some test, or you can also use that when you are building frameworks, and wants to dynamically create views to query data. This post shows how to do it.

Prerequisites

If you are using Maven you can use the following information in your pom.xml to add the Java Client library:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
<repositories>
  <repository>
    <id>couchbase</id>
    <name>Couchbase Maven Repository</name>
    <layout>default</layout>
    <url>http://files.couchbase.com/maven2/</url>
    <snapshots>
      <enabled>false</enabled>
    </snapshots>
  </repository>
</repositories>

<dependencies>
  <dependency>
    <groupid>couchbase</groupid>
    <artifactid>couchbase-client</artifactid>
    <version>1.1.0</version>
    <type>jar</type>
  </dependency>
</dependencies>

See online at https://gist.github.com/4337172

Create and Manage Views From Java

The full Maven project is available on Github.

Connect to Couchbase Cluster

The first thing to do when you want to create a view from Java is obviously to connect to the cluster.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
import com.couchbase.client.CouchbaseClient;
...

List<uri> uris = new LinkedList<uri>();
uris.add(URI.create("http://127.0.0.1:8091/pools"));
CouchbaseClient client = null;
try {
  client = new CouchbaseClient(uris, "beer-sample", "");

  // put your code here

  client.shutdown();

  } catch (Exception e) {
    System.err.println("Error connecting to Couchbase: " + e.getMessage());
    System.exit(0);
  }
...
  1. Create a list of URIs to different nodes of the cluster - lines 5-6. (In this example I am working on a single node)
  2. Connect to the bucket, in our case beer-sample -line 9. You can include the password if the bucket is protected ( this is not the case here so I am sending an empty string)

If you are looking for more information about Couchbase and Java, you can read this article from DZone : Hello World with Couchbase and Java.

Let’s now talk about Couchbase views. You use views/map-reduce functions to index and query data from Couchbase Server based on the content of the JSON document you store inside Couchbase. For more information about views you can look at the “view basics” chapter of the Couchbase Server Manual.

Create Views from Java

Creating a view from Java is really easy : the Java Client Library contains all the classes and methods to do it. As a concrete use case we will use the Application that is described in the Couchbase Java Tutorial.

When you follow this tutorial, you need to manually create some views, as you can see here. In this example, we will create our map function and directly in our Java code and then store it to Couchbase Server. The tutorial asks you to create the following artifacts:

  • a view named “by_name”
  • in the design document named "dev_beer" (development mode)
  • and the map function which looks like the following :
1
2
3
4
5
function (doc, meta) {
  if(doc.type && doc.type == "beer") {
    emit(doc.name, null);
  }
}

The following code allows you to do it from Java:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
import com.couchbase.client.protocol.views.DesignDocument;
import com.couchbase.client.protocol.views.ViewDesign;
...
DesignDocument designDoc = new DesignDocument("dev_beer");

String viewName = "by_name";
String mapFunction =
"function (doc, meta) {\n" +
"  if(doc.type && doc.type == \"beer\") {\n" +
"    emit(doc.name);\n" +
"  }\n" +
"}";

ViewDesign viewDesign = new ViewDesign(viewName,mapFunction);
designDoc.getViews().add(viewDesign);
client.createDesignDoc( designDoc );
...
  • Create a design document using the com.couchbase.client.protocol.views.DesignDocument class - line 4.
  • Create a view using com.couchbase.client.protocol.views.ViewDesign class with a name and the map function - line 14.
  • You can add this view to a design document - line 15
  • Finally save the document into the cluster using the CouchbaseClient.createDesignDoc method.

If you need to use a reduce function (built-in or custom) you just need to pass to the ViewDesign constructor as 3rd parameter.

When developing view, from Java or from any other tool/language be sure you understand what are the best practices, and the life cycle of the index. This is why I am inviting you to take a look to the following chapters in the Couchbase documentation:

Using the view

First of all, the view that you just created is in “development mode”, and by default the Java client SDK will only access the view when it is in “production mode”. This means that when you are calling a view from your application it will search it into the production environment. So before connecting to Couchbase cluster you need to setup the viewmode to development.

This is done using the viewmode environment variable from the Java SDK, that could be set using the following methods:

  • In your code, add this line before the client connects to the cluster : System.setProperty("viewmode", "development");
  • At the command line -Dviewmode=development
  • In a properties file viewmode=development

Once it is done you can call the view using the following code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
import import com.couchbase.client.protocol.views.*;

...
System.setProperty("viewmode", "development"); // before the connection to Couchbase
...
View view = client.getView("beer", "by_name");
Query query = new Query();
query.setIncludeDocs(true).setLimit(20);
query.setStale( Stale.FALSE );
ViewResponse result = client.query(view, query);
for(ViewRow row : result) {
  row.getDocument(); // deal with the document/data
}
...

This code queries the view you just created. This means Couchbase Server will generate an index based on your map function, will query the server for results. In this case, we specifically want to set a limit of 20 results and also get the most current results by setting Stale.FALSE.

  • Set the viewmode to development - line 4
  • Get the view using the CouchbaseClient.getView() method -line 6-. As you can see I just use the name beer for the design document (and not dev_beer, Couchbase will know where to search since I am in development mode)
  • Create a query and set a limit (20) and ask the SDK to return the document itself setIncludeDocs(true) -line 8- The document will be returned from Couchbase server in the most efficient way
  • Ask the system to update the index before returning the result using query.setStale( Stale.FALSE );-line 9-. Once again be careful when you use setStale method. Just to be sure here is the documentation about it : Index Updates and the stale Parameter
  • Execute the query - line 10
  • And use the result - lines 11-13

Conclusion

In this article, you have learned:

  • How to create Couchbase views from Java
  • Call this view from Java
  • Configure development/production mode views from Couchbase Java Client Library

This example is limited to the creation of a view, you can take a look to the other methods related to design documents and views if you want to manage your design documents : getDesignDocument(), deleteDesignDocument(), …

What to Do if Your Couchbase Server Does Not Start?

| Comments

Working with the Couchbase 2.0.0 release you may have issues when trying to access the Web Admin Console or simply starting the server. This is due to the way Couchbase Server uses the IP address/hostname during the installation process. So when you have one of the following errors :

  • On Windows, Server is not working at all, even after installation. You can access the sever on port 8092 (Couchbase API port), but cannot on port 8091
  • You have the following error when accessing the console "[10:02:02] IP address seems to have changed. Unable to listen on 'ns_1@10.0.2.15'"

  • When you try to restart the server it does not start and you have the following error message in the error log : "Configured address '10.0.2.15' seems to be invalid. Will refuse to start for safety reasons"

Some of these issues are related to a known issue on Windows ( see MB-7417 that will be fixed in 2.0.1) or the fact that Couchbase server does not support change of the IP address after installation. This is documented in the section “Using Couchbase in the Cloud: Handling Changes in IP Addresses” of the Couchbase Server Manual. This article explains what should be done when configuring Couchbase Server on Windows, but you can do equivalent steps on any platform using the shell scripts available on Linux and/or Mac OS X.

Once you have installed Couchbase, you can see in the console that the IP address of your server is used :

Typically the address 192.168.0.97 is stored in the configuration of Couchbase. If your server receives a new address from the DHCP server, Couchbase will not work anymore. In this article you will see how you can configure Couchbase to use another IP address or Hostname.

Important: The steps that follow will completely destroy any data and configuration from the node, so it is best to start with a fresh Couchbase install. If you can not, you should backup your data using the file based backup-restore documented here.

New Adventure…

| Comments

Yesterday was my last day at eXo… I have been working at eXo since 2008, and we have achieved many exciting things such as building eXo Platform, the open source social platform, and the Cloud-IDE allowing developers to build, test, and deploy applications online.

It was a great experience for me, but it is time for me to jump into a new adventure…

I am joining Couchbase as a Technical Evangelist for the EMEA Region. When I have started to work with NoSQL engines (starting with Google BigTable for Resultri) I really enjoyed the experience and the flexibility that it gives to the developers. Then I choose to work on a documented oriented database because it looks very natural to me, and evaluate the clustering capabilities. This is how I discovered Couchbase and its community.

This new job is a great opportunity for me to do the things that I really like about software:

  • Coding applications using different languages and frameworks
  • Understanding the sysops/devops related challenges when using a new product/technology
  • And finally probably the most important part; sharing it with others!

I look forward to sharing what I like about NoSQL and Couchbase and discuss with others about their experience or needs around NoSQL. 

See you soon online and in real life during conferences and meetups!

"Couchbase Logo"

Couchbase : Create a Large Dataset Using Twitter and Java

| Comments

An easy way to create large dataset when playing/demonstrating Couchbase -or any other NoSQL engine- is to inject Twitter feed into your database.

For this small application I am using:

In this example I am using Java to inject Tweets into Couchbase, you can obviously use another langage if you want to.

The sources of this project are available on my Github repository Twitter Injector for Couchbase you can also download the Binary version here, and execute the application from the command line, see Run The Application paragraph. Do not forget to create your Twitter oAuth keys (see next paragraph)

Create oAuth Keys

The first thing to do to be able to use the Twitter API is to create a set of keys. If you want to learn more about all these keys/tokens take a look to the oAuth protocol : http://oauth.net/

1- Log in into the Twitter Development Portal : https://dev.twitter.com/

2- Create a new Application

Click on the “Create an App” link or go into the “User Menu > My Applications > Create a new application”

3- Enter the Application Details information

4- Click “Create Your Twitter Application” button

Your application’s OAuth settings are now available :

5- Go down on the Application Settings page and click on the “Create My Access Token” button

You have now all the necessary information to create your application:

  • Consumer key
  • Consumer secret
  • Access token
  • Access token secret

These keys will be uses in the twitter4j.properties file when running the Java application from the command line

Create the Java Application

The following code is the main code of the application:

Some basic explanation:

  • The setUp() method simply reads the twitter4j.properties file from the classpath to build the Couchbase connection string.
  • The injectTweets opens the Couchbase connection -line 76- and calls the TwitterStream API.
  • A Listener is created and will receive all the onStatus(Status status) from Twitter. The most important method is onStatus() that receive the message and save it into Couchbase.
  • One interesting thing : since Couchbase is a JSON Document database it allows your to just take the JSON String and save it directly. cbClient.add(idStr,0 ,twitterMessage);

Packaging

To be able to execute the application directly from the Jar file, I am using the assembly plugin with the following informations from the pom.xml:

1
2
3
4
5
6
7
8
9
10
  ...
<archive>
<manifest>
<mainclass>com.couchbase.demo.TwitterInjector</mainclass>
</manifest>
<manifestentries>
<class-path>.</class-path>
</manifestentries>
</archive>
...

Some information:

  • The mainClass entry allows you to set which class to execute when running java -jar command.
  • The Class-Path entry allows you to set the current directory as part of the classpath where the program will search for the twitter4j.properties file.
  • The assembly file is also configure to include all the dependencies (Twitter4J, Couchbase client SDK, …)

If you do want to build it from the sources, simply run :

1
mvn clean package

This will create the following Jar file ./target/CouchbaseTwitterInjector.jar

Run the Java Application

Before running the application you must create a twitter4j.properties file with the following information :

1
2
3
4
5
6
7
8
9
10
twitter4j.jsonStoreEnabled=true

oauth.consumerKey=[YOUR CONSUMER KEY]
oauth.consumerSecret=[YOUR CONSUMER SECRET KEY]
oauth.accessToken=[YOUR ACCESS TOKEN]
oauth.accessTokenSecret=[YOUR ACCESS TOKEN SECRET]

couchbase.uri.list=http://127.0.0.1:8091/pools
couchbase.bucket=default
couchbase.password=

Save the properties file and from the same location run:

1
jar -jar [path-to-jar]/CouchbaseTwitterInjector.jar

This will inject Tweets into your Couchbase Server. Enjoy !

Create a Simple Node.js and Couchbase Application… On OS X

| Comments

NOTE: The Couchbase Node.js Client Library is currently changing. I will update this article and source code once the API is stable.

I am currently playing a little bit with Node.js . It is quite fun! In this article I won’t go in a a very complex application but just give you the basic steps to create your first Node.js+Couchbase application… on Mac OS X.

Installation

Couchbase 2.0 Beta:

You can take a look the first steps of my previous article to install Couchbase. The basics steps are:

  • Download Couchbase 2. 0 Beta
  • Start the server (Run the “Couchbase Server” application)
  • Configure the server

Other Components :

  • Node.js
  • Couchbase Client Library (C version)

Couchbase 101 : Install, Store and Query Data

| Comments

Introduction

In this post I just want to show how easily is to get started with Couchbase, and also explain how to “query” the data. The basic steps of this tutorial are:

  1. Install Couchbase
  2. Create Data
  3. Query Data

I will try to post more articles, if I have time to show how to use Couchbase from your applications (starting with Java).

Prerequisites :

  • Could not be simpler : Couchbase 2.0 available here. (Currently in Developer Preview)

Couchbase 101 : Insert and Query data

Installation

I am using Couchbase on Mac OS X, so let me describe the installation in this environment. If you are using other operating system just take a look to the Couchbase documentation.

Couchbase installation is very (very!) fast:

  1. Download the Mac OS X Zip file.
  2. Double-click the downloaded Zip installation file to extract the contents. This will create a single file, the Couchbase.app application.
  3. Drag and Drop the Couchbase.app to your chosen installation folder, such as the system Applications folder.

Start and Configure Couchbase Server

To start Couchbase Server, just double click on the Couchbase Server. Once the server is started, a new icon is added in the OS X Menu to indicate that the Server is up and running.

You can now configure your Couchbase instance, for this you just need to access the Admin Console, available at the following location http://127.0.0.1:8091 (change the IP address if needed) or simply by going in the Couchbase menu and click on Open Admin Console entry.

  1. Welcome Screen : Click Setup
  2. Set the disk and cluster configuration. On my instance I keep the default location for the on disk storage. Just configure the size of the memory usage for your instance, for example 800Mb. So far, we have a single instance, so no need to join a cluster.
  3. Choose to generate sample data. This will be interesting to learn more about data and views.
  4. Create the default bucket (use for testing only). A bucket is used by Couchbase to store data. It could be compared to a “database” in RDBMS world.
  5. Configure update notifications to be alerted when new version of Couchbase is released
  6. Configure the server with a final step with the administrator username and password
  7. When this is done you are automatically redirected to the Admin Console.

This is it! You are ready to use your Couchbase server.

Couchbase has many interesting features, especially around scalability and elasticity but for not in this article let’s focus on the basics :

  • Insert some data and query them

Insert Data

Couchbase has many ways to manipulate data from you favorite programming language using the different client libraries : Java, Python, PHP, Ruby, .Net, C. For now let’s use the Admin Console to create and query data.

Couchbase can store any type of data, but when you need to manipulate some data with a structure the best way is to use JSON Documents. So let’s use the console and create documents.

To create new documents in your database, click on the “Data Buckets” tab. If you have installed the sample you see 2 buckets: default and gamesim-sample.

Let’s create a new documents in the default bucket:

  1. Click on Documents button
  2. Click on Create Document
  3. Since each document must have an id for example 100.
  4. Couchbase save the document and add some metadata such as _rev, $flags, expiration
  5. Add new attributes to the document that describe an employee : Name, Departement and Salary, then save it. You just need to update the JSON object with values
1
2
3
4
5
6
{
      "_id": "100",
      "name": "Thomas",
      "dept": "Sales",
      "salary": 5000
  }

Repeat the operation with some other employees :

1
2
3
4
5
  200,Jason,Technology,5500
  300,Mayla,Technology,7000
  400,Nisha,Marketing,9500
  500,Randy,Technology,6000
  501,Ritu,Accounting,5400

You have now a list of employees in your database. That was easy isn’t? Let’s now query them.

Query Data

Access document directly from its ID

First of all you can quickly access a document using a simple HTTP request using its id. For example to access the Mayla with the id 300 just enter the following URL:

  • http://127.0.0.1:8092/default/300

In this URL you have :

  • 8092 is the Couch API REST port used to access data (where 8091 is the port for the Admin console)
  • default is the bucket in which the document is stored
  • 300 is the id of the document

Search your data with queries

So we have seen how you can access one document. But what if my need is :

  • “Give me all the employee of the Technology department”

To achieve such query it is necessary to create views. The views are used by Couchbase server to index and search your data. A view is a Map function written in JavaScript, that will generate a key value list that is compliant with logic you put in the Map function. Note that this key,value is now indexed and sorted by key. This is important when you query your data.

So let’s create a new view from the Admin Console:

  1. Click on the Views tab (be sure you are on the default bucket)
  2. Click on the “Create Development View”
  3. Enter the Document and View name:
  4. Document Name : _design/dev_dept
  5. View Name : dept
  6. Cick Save
  7. Click on your View to edit it

Since we need to provide the list of employees that are part of a the Technology department, we need to create a view that use the department as key, so the map function looks like :

1
2
3
function (doc) {
    emit(doc.dept, null);
}

Save the view

This function takes the document and create a list that contains the “dept” as key and null as value. The value itself is not that important in our case. A simple rule will be : do not put too much data in the value since at the end Couchbase server creates an index with this map. Will see that Couchbase allows developer to easily get the document information when accessing a view.

Click on the “Show Results” button, the result will look like:

1
2
3
4
5
6
7
8
9
{"total_rows":6,"rows":[
  {"id":"501","key":"Accounting","value":null},
  {"id":"400","key":"Marketing","value":null},
  {"id":"100","key":"Sales","value":null},
  {"id":"200","key":"Technology","value":null},
  {"id":"300","key":"Technology","value":null},
  {"id":"500","key":"Technology","value":null}
  ]
}

As we have seen in earlier it is possible to access the document using a single URL, it is the same for views. You can for example access the view we have just created using the following URL:

Now it is possible to use query parameter to filter the results using the key parameter with the value entered using a JSON String :

The result of this query is now :

1
2
3
4
5
6
{"total_rows":6,"rows":[
  {"id":"200","key":"Technology","value":null},
  {"id":"300","key":"Technology","value":null},
  {"id":"500","key":"Technology","value":null}
  ]
}

You have many other parameters you can use when accessing a view to control the size, the time out, …. One of them is quite interesting is include_docs that ask Couchbase to include the full content of the document in the result. So if you call :

The result is :

1
2
3
4
5
6
{"total_rows":6,"rows":[
  {"id":"200","key":"Technology","value":null,"doc":  {"_id":"200","_rev":"1-1de6e6751608eada0000003200000000","$flags":0,"$expiration":0,"name":"Jason","dept":"Technology","salary":5500}},
  {"id":"300","key":"Technology","value":null,"doc":{"_id":"300","_rev":"1-f3e44cee742bfae10000003200000000","$flags":0,"$expiration":0,"name":"Mayla","dept":"Technology","salary":7000}},
  {"id":"500","key":"Technology","value":null,"doc":  {"_id":"500","_rev":"1-05780359aac8f3790000003200000000","$flags":0,"$expiration":0,"name":"Randy","dept":"Technology","salary":6000}}
]
}

Let’s now create a little more complicated view to answer the following business requirement:

  • “Give me all the employee with a salary between 5000 and 6000”

So now you know that you need to create a new view with the salary as key let’s with the following Map function:

1
2
3
function (doc) {
  emit(doc.salary, null);
}

Couchbase is automatically sorting the key when creating/updating the index so, let’s use the startkey and endkey parameter when calling the view. So let’s call the view with from the following URL:

The result is :

1
2
3
4
5
6
7
{"total_rows":6,"rows":[
 {"id":"100","key":5000,"value":null,"doc":{"_id":"100","_rev":"1-0f33580d780014060000002e00000000","$flags":0,"$expiration":0,"name":"Thomas","dept":"Sales","salary":5000}},
 {"id":"501","key":5400,"value":null,"doc":{"_id":"501","_rev":"1-b1fe5bc79637720e0000003100000000","$flags":0,"$expiration":0,"name":"Ritu","dept":"Accounting","salary":5400}},
 {"id":"200","key":5500,"value":null,"doc":{"_id":"200","_rev":"1-1de6e6751608eada0000003200000000","$flags":0,"$expiration":0,"name":"Jason","dept":"Technology","salary":5500}},
 {"id":"500","key":6000,"value":null,"doc":{"_id":"500","_rev":"1-05780359aac8f3790000003200000000","$flags":0,"$expiration":0,"name":"Randy","dept":"Technology","salary":6000}}
]
}

Conclusion

In this short article you have learn how to:

  • Install Couchbase* Create data using the Admin Console
  • Query data with views

When I get more time I will write another article that do the same from Java, and other languages.


Note from @ingenthr

Nice blog! Note that while querying the REST interface directly is okay, we’ve really tried to make it easy by having high-level language support for queries in each of the official client libraries. They’re all listed over at http://www.couchbase.com/develop

eXo Platform: Internationalization of Content

| Comments

In this screencast I explain the support of multi-lingual content of eXo Platform 3.5. The different features that you can see in this video are:

  • Support of multiple languages from URL
  • Configure the eXo File Explorer to add support for multi-lingual content (new button)
  • Content translation
  • Add new language to the platform