Nest Thermostat monitoring pack: Part I: How did I get data?

This blog post is part of a series check out the other posts in this series:

I recently bought myself a Nest Thermostat. For those who are not familiar with this device. It’s basically an internet connected thermostat which learns your heating patterns with different variables. It’s of course packaged in a nice futuristic design… Cool…

More info here: https://nest.com/thermostat/life-with-nest-thermostat/

Although it is not officially released in Europe nor supported it’s actually quite easy to install. If you are reading this and need more info please make sure to check out this blog post: http://www.fousa.be/blog/nest-thermostat

So I survived the installation…

This round eye thing as my wife refers to it is doing its job nicely. I get a lot of feedback from the app and even on the website. Everything looks shiny and cool… But… What if… What if I could get this data in SCOM and see whether I can monitor and control it from there… Imagine the fun.

Well this blog post will document my way of getting data into SCOM. To give you a high level idea this is actually what I did:

§ Retrieve data from the nest.com website (not from the actual device as my first attempts failed miserably)

§ Get this data into a MySQL dbase

This is a schematic overview of what components were used:

In this blog post we’ll discuss how to get data from the nest into a MySQL dbase

Retrieving the data from nest

So how was I going to get data out of this nice circle on the wall which is controlling my heating… My first attempt was to sniff all the traffic which came from the IP of the nest thermostat and see whether there were valuable entries in the traffic between the device and the Nest web service. Well a lot of data was sent but it would take a lot of work to make sure everything was discovered correctly. I opened Google and Bing (always nice to compare both results) and started searching on the web. A lot of users had the same question how to get data out but not many had the answer. Until i stumble upon the github library and did a search there.

I found this API of Guillaume Boudreau on github: https://github.com/gboudreau/nest-api

This nice gentlemen wrote a PHP based api which could retrieve all the info and return it with a simple REST API call. I’m not a dev so this was unknown territory for me but I gave it a shot. My aim was to get this data stored locally so I can have full control over how it’s stored and retrieved by SCOM. This to make my life afterwards a bit easier.

I could inject it into a SQL dbase but this would require my server to be switched on all the time. I would also need to have a machine which is running a webserver to take care of all the different php requests to the API and such. Sure I can build a dedicated machine on my hyper-v host but the server had to be on all the time for the entire process to work not a good idea with current energy rates. But wait a minute. I have another device running all the time. My Synology DS412+. Up until now it only served as a storage device for both entertainment and iscsi luns for hyper-v. But this thing is capable of doing much more. I checked the different fan blogs / forums and realized right away that this device was the way to go. Without any knowledge of MySQL and or php I set of on an adventure..

Note: All this is configured on the beta version of DSM 5.0 so screenshots may differ from previous versions of DSM.

Configure the Synology NAS to store the data in MySQL and run the PHP settings.

I had a fun time configuring the NAS to get it a point that everything was working. I’ll link to some blog posts which got me going as this will take me too far. If you have issues please do not hesitate to comment or connect on twitter @dieterwijckmans to get more specific on this topic:

Install Mysql and phpadmin on synology nas

Install winscp to browse your Synology build in ftp server (via scp actually)

Install Putty to get a terminal access to the linux distro underneath your synology OS Note: Just install it you do not have to setup a key chain to access your synology.

A couple of quick pointers (I will write a more detailed process if I ever find the time or if there’s a huge demand for it)

When you connect to your synology station with Winscp use the SCP protocol on port 22 instead of the FTP or SFTP
To connect to your device via putty or winscp use the root user. The password is actually the admin password.
Set a password on your MySQL install for the root password because some scripts do not allow to run it with password no option.

Get this data in Mysql dbase

Again this will be a fairly straight forward process if you’ve done this before. For me all this was quite new as a 99% windows oriented tech (ok it’s out in the open I admit it) but it proofed to be a challenging but rewarding route.

First things first. These were my resources to get everything in Mysql. I started down a different path to write my own scripts when I stumbled upon a post on github by chriseng. He actually wrote a nice front end webpage to get data of Nest in a comprehenable way… He writes the data to a mysql dbase first. Exactly what I needed!

https://github.com/chriseng/nestgraph

So follow the instructions on there and get the api + script in place.

If all goes well open up your phpmyadmin by opening your browser and surfing to: http://<address of your synology>/phpmyadmin . You should see a dbase created and the first line of data in there:

I know this is a high level and quick manual but this is in fact how I got it running without any knowledge of mysql / php and so on.

Now that we have 1 line in the dbase (mine has more in there but I have already automated the process) we need to schedule the command to poll the web service on a regular basis.

This took me quite some time to figure out so if you made it this far I’ll safe you the time of figuring it out yourself.

High level steps:

Create an extra line in the cron
Restart the cron

Create an extra line in the cron

The cron is basically the Linux equivalent of task scheduler. You need to put in different parameters and Linux will run the command for you on a regular interval. More info can be found here: http://en.wikipedia.org/wiki/Cron

On the Synology the cron is located in folder: /etc and the file is called crontab.

It should look like this:

Some considerations which got me searching for quite a while why it wasn’t working:

Make sure to separate the columns by TABS instead of spaces.
Always use the root user or the job will be deleted when the NAS will reboot
Update the path of your php bin. Normally on a synology it’s located in /usr/bin/php
Update the path of your php scripts. Mine are stored in /volume1/web/nestgraph/insert.php
Verify your command by running it in a putty session like: “php /volume1/web/nestgraph/insert.php”. You will not receive a visual confirmation that everything went successful but you will notice an extra line in your mysql dbase.

So… You wait patiently to see the data coming into your dbase. But nothing is happening. You check again a while later. Nope. I see myself again struggling with this. Turned out you still need to restart the cron service on your Linux distro. All so complicated for us windows guys… sigh…

To do this on DSM 4X run:

/usr/syno/etc/rc.d/S04crond.sh stop
/usr/syno/etc/rc.d/S04crond.sh start

To do this on DSM 5 run:

/usr/syno/sbin/synoservicectl –restart crond

Do this in a putty session as these are both Linux related commands:

After this the data should be flowing in…

Still with me?

Next thing we need to do is get the data queried out of our MySQL dbase and into a property bag to pass onto to SCOM.

Check out part 2 (link in top of blog post)

Go back to the blog

Blog