Node.js, MongoDB, and You: an Intro in Parts

A few years ago, two technologies were born. One was a platform for building network applications, the other a database. Both were slightly rebellious and shared a fascination with javascript. Their courtship was inevitable.

In the years since, both technologies have grown up. Node.js and MongoDB have matured as a couple and if you haven’t had a chance to get to know them, its high time you took a few minutes and familiarized yourself.

While hype doesn’t make a language or a database, both technologies have their benefits. Node.js delivers relatively good concurrency and performance by making asynchronous IO the norm, and wraps it all up with the best package manager ever. MongoDB serves as a simplified persistence layer that eschews the overhead of strict schema design for a document based approach and has a simple query interface to make working with your data easy. The fact that both have great communities that are actively making things better is a nice cherry on top. Certainly, no technology is perfect for all uses cases, but node.js and MongoDB make it fairly easy to get start building production network applications.

Getting Started

Make sure you download and install Node.js and npm. You will also need access to a MongoDB databases. If you don’t want to go through the struggle of setting up your own, spend five minutes and create a free database on MongoHQ.

Start by creating a directory for your work:

mkdir node_playground && cd node_playground

Next, use npm (the node package manager) to create a template package.json file. This file is used to specify package metadata and dependencies. npm init will create one for us by asking some questions, use the defaults if you’re in a hurry.

And last, we need a way to connect to our running mongo instance, for that, we will install the mongodb driver.

npm install mongodb --save #--save will add the dependency to our package.json

With that, its time to write some code.

The Basics

To get started, lets connect to the database, write a document, update it, and then read it back (if the wall of code is intimidating, start with the inline comments):

// basic-mongo.js
// import the mongo driver that we installed above and grab
// the MongoClient that helps us connect
var MongoClient = require("mongodb").MongoClient
// connecting requires passing a mongodb uri
// these can take lots of options, but here we keep it simple 
// and connect to the superhero db with a username and password
// you will need to change this!
MongoClient.connect("mongodb://addison:supersecure@paulo.mongohq.com:10000/superhero", function(err, db) {
  // if we didn't connect, throw the error
  if (err) throw err

  // in mongo, documents are grouped in collection (like a table)
  // lets make one!
  var superheroes = db.collection("superheroes")
  // inserting a new document is easy, just pass arbitrary json
  superheroes.insert({name: "Addison", superpower: "insert"}, 
    function(err, result) {
      if (err) throw err

      // all documents in mongo get assigned a unique id, _id
      // we use this to find the document we just inserted
      var _id = result[0]._id

      // to update, we write a 'selector', and then the update
      // notice the use of $set, it is a special operator 
      // that sets the fields in the document, otherwise, we would
      // wipe out the existing document
      superheroes.update({_id: _id}, {$set: {superpower: "update"}}, function(err) {
        if (err) throw err

        // finding a document needs a selector like above
        superheroes.findOne({_id: _id}, function(err, doc) {
          if (err) throw err
          console.log(doc.name + " has the power to " + doc.superpower)
          // close our database so the process will die
          db.close()
        }) 
      })

   })
})

To run it, we just do:

node basic-mongo.js

If all went well we should see the following:

> Addison has the power to update

This is a dumb example and kinda ugly, but it gives a basic example of interacting with mongo from node. There are a few problems here, though, that we should fix. A lot of people getting started with node have the tendency to nest lots of anonymous functions, creating the dreaded “pyramid of doom”. By breaking out some functions and naming them, we can make it a lot cleaner and easier to follow.

Also, a script with no interaction is boring, let’s tweak things a bit and turn this into a small command line app that allows us to read and update our list of heroes and superpowers. We’ll also introduce a few other important MongoDB concepts.

// super-cli.js
var MongoClient = require("mongodb").MongoClient

// we move connecting into a function to avoid the "pyramid of doom"
function getConnection(cb) {
  MongoClient.connect("mongodb://addison:supersecure@paulo.mongohq.com:10000/superhero", function(err, db) {
    if (err) return cb(err)
    var superheroes = db.collection("superheroes")
    // because we are searching by name, we need an index! without an index, things can get slow
    // if you want to learn more: http://docs.mongodb.org/manual/core/indexes-introduction/
    superheroes.ensureIndex({name: true}, function(err) {
      if (err) return cb(err)
      cb(null, superheroes)
    })
  })
}

// an upsert will create a new record OR update an existing record, which makes things easier
// in mongo, we can do this with a findAndModify and passing the upsert option
// to have the updated document returned, we pass the new option as well
function upsertPower(collection, name, power, cb) {
  collection.findAndModify({name: name}, {}, {$set: {superpower: power}}, {upsert: true, new: true}, cb)
}

// instead of finding just one hero, we can list all of the
// documents by passing an empty selector.
// This returns a 'cursor', which allows us to walk through the documents
// look at how we do this in processHeroes
function readAll(collection, cb) {
  collection.find({}, cb)
}

function readPower(collection, name, cb) {
  collection.findOne({name: name}, cb)
}

function printHero(hero) {
  // make sure we found our hero!
  if (!hero) {
    console.log("couldn't find the hero you asked for!")
  }
  console.log(hero.name + " has the power of " + hero.superpower)
}

// the each method allows us to walk through the result set, notice the callback, as every time the callback
// is called, there is another chance of an error
function printHeroes(heroes, cb) {
  heroes.each(function(err, hero) {
    if (err) return cb(err)
    printHero(hero)
  })
}

function superPowerCli(operation, name, superpower, cb) {
  getConnection(function(err, collection) {
    if (err) return cb(err)

    // we need to make sure to close the database, otherwise, the process
    // won't stop
    function processHero(err, hero) {
      if (err) return cb(err)
      printHero(hero)
      collection.db.close()
      cb()
    }

    // use this function when dealing with lots of heroes
    function processHeroes(err, heroes) {
      if (err) return cb(err)
      // the callback to each is called for every result, once it returns a null, we know
      // the result set is done
      heroes.each(function(err, hero) {
        if (err) return cb(err)
        if (hero) {
          printHero(hero)
        } else {
          collection.db.close()
          cb()
        }
      })
    }

    if (operation === "list") {
      readAll(collection, processHeroes)
    } else if (operation === "update") {
      upsertPower(collection, name, superpower, processHero)
    } else if (operation === "read" ){
      readPower(collection, name, processHero)
    } else {
      return cb(new Error("unknown operation!"))
    }
  })
}

var operation = process.argv[2]
var name = process.argv[3]
var superpower = process.argv[4]

superPowerCli(operation, name, superpower, function(err) {
  if (err) {
    console.log("had an error!", err)
    process.exit(1)
  }
})

Now, interaction!:

node super-cli.js update Addison puns
> Addison has the power of puns
node super-cli.js update Ian coding
> Ian has the power of coding
node super-cli read Addison
> Addison has the power of puns
node super-cli list
> Addison has the power of puns
> Ian has the power of coding

This is better! We cleaned up the code, got rid of those pesky pyramids, and also learned a bit about indexes, upserts, and cursors.

With these simple MongoDB operations and a bit of Javascript code structuring knowledge, you have the basics to dig and and start getting a bit more familiar with these two technologies.

In the coming weeks, expect another article on how to use MongoDB with Express and Mongoose to help build a simple API service.

In the mean time, if you want to learn more, make sure to look at the docs for the Node.js MongoDB driver.

  • samof76

    Great intro to using Mongodb driver with nodejs, hoping to see more. Thanks.

  • http://www.ceclinux.org/ ceclinux

    nice post.More examples and details would be better