After last week’s blog post where we connected an Internet of Things temperature sensor to record data in MongoHQ’s MongoDB instances, one Twitter follower commented that it would have been even easier if we had used Node-RED. We were going to write about Node-RED anyway but were keeping that previous post to the bare essentials.
Things like this are best demonstrated though. Recently I was asked to set up a database sample of Twitter tweets about the floods in the UK and I decided to use Node-RED. If you want to learn how to install Node-RED in detail consult our how-to page. Suffice to say that once installed, to begin working with a running Node-RED, we just need to point our browser at port 1880 on the host it’s running on. Once there we get to see the Node-RED user interface.
On the left is a palette of nodes we can drag into the workspace in the centre. On the right are two tabs, one which shows information and help about the currently selected node (either in the palette or in the workspace) and another for displaying debugging information. For our Flood tweets, we scroll drown and drag a Twitter input into the workspace. The red Deploy button lights up to show we’ve made changes we haven’t deployed yet. We can now double click on the dropped node to set it up.
This lets us select Twitter credentials (and handles authorisation with Twitter), select what feed we want to search (public, followers or direct messages) and enter what we want to search for. I’ve put in “#flood” for this example. There’s also a name field to give the node its own name. How will we know thats working? If we drag a debug node onto the workspace and then drag a connection from our Twitter node to the debug node, we’ll be able to see what is being generated. To start it running, click the Deploy button. After a while – you’ll be waiting for someone to tweet – the debug window should start showing the messages being generated.
Now to get those messages to MongoDB – we drag a MongoDB out node onto the workspace and double click that to configure it. We can configure a MongoDB server by selecting “add new mongodb” where we can enter the host, port, database and, if needed, username and password. We can also name the configuration. But there was a catch… MongoHQ requires authenticated clients but, as we were writing this, the released Node-RED didn’t have support for username and password authentication. That, by the way would have stopped us using it for last week’s Internet of Things project. When we mentioned this to the developers on Twitter they pushed support for authentication into the development version of Node-RED within an hour so everyone will be able to make use of it – our page on setting up Node-RED shows you how to get and install the development version so you can use Node-RED with MongoHQ’s sandboxes and production servers now. We can now set up our MongoDB node to save the incoming messages to a floodtweetsarchive collection in our database.
Once the database is set up, we can set the collection the message will be saved in, whether it should be saved, inserted or used to remove a document, and we can, if we want, limit it to the payload field. As the Twitter in node populates the message with all the Twitter information and we want to save that, we’ll not set that checkbox. We can now connect the Twitter in node to the MongoDB out node, leaving the debug connection in place and click Deploy.
We can then hop over to the MongoHQ admin console and look at the database and find a tweet already saved in the database. It’s that simple. The Node-RED server will continue to run the flow even after we’ve browsed away. This is probably the simplest Node-RED setup we could come up with. Say we needed something which did a some more processing, where we want to remove a lot of the extraneous data from the tweets and perform some evaluation of the text. Implementing this as a flow gives us…
Although Node-RED is a great tool for prototyping ideas, it is important to remember it is still a version 0.5 and as such omits things which would be essential for full-scale production like authentication and authorisation for the web front end and a separation of the run-time functionality from that front end. But development has been rapid and, as an open source project, Node-RED will see these features sooner rather than later. Right now though, it offers a powerful way of graphically wiring the world of data flows together, from social media and the Internet of Things or from more traditional sources, and populating your MongoDB databases in novel ways.