Using the
 Davis Vantage Pro
 Weather Station
 on Linux

Table Of Contents

Summary

Commentary

The Hardware

The Software

Data File Variables

Real Time Data

Highs/Lows Data

Graph Data

Download

Summary

The Davis Instruments Vantage Pro Weather Station is a relatively low cost, professional weather station designed for home use. This article describes the hardware and software necessary to get weather data from a Davis Vantage Pro Weather Station to one or more html web pages via a Linux server.

Commentary

The Vantage Pro is actually my third station. I started out with an Oregon Scientific WMR-918 and then moved to the WMR-968 because of numerous problems. It turned out that the 968 wasn't much better. Compared to Davis, the Oregon Scientific units are crap. They a cheaply made and fail quite frequently. If you are thinking about buying a weather station, go with Davis. Furthermore, go with the cabled unit rather than the wireless. The wireless unit will give you lots of headaches with poor and varying range, bad batteries (especially in cold weather), and frequent firmware upgrades. The cabled units are rock solid and easy to install. Davis uses standard RJ flat phone cable for the sensor arrays, so the cable can be tucked away and tacked into any little crack or crevice. Just buy a cheap RJ-11 crimper at Radio Shack and a roll of phone cable and you can custom make the wire to fit any installation. In my case, the 50ft cable that came with the Vantage Pro was actually too long, so I cut it down to size just crimped on a new connector to one end.

The Hardware

Let's go through this block diagram piece by piece to see how the hardware works.

Davis Vantage Pro console
Any weather data you want to send or receive from the Vantage Pro must come from the console. But guess what? the console does not have a serial port on it.

WeatherLink
In order to get RS-232 serial data in and out of the console, you have to buy the WeatherLink Windows software. In the package is a small potted module with a proprietary connector on one end and an RJ-11 jack on the other. The module plugs into a slot inside the battery compartment on the back of the console. The module has a phone jack that runs to a DB-9 connector. This is what gives you RS-232 communication capability. Note that there is no way to get data out of the console without this hardware module. It is not a simple adapter but actually contains an EEPROM and possibly its own mcu. So what you have to do is buy the WeatherLink Windows Software and then throw away the software part of it. It sounds stupid. But if it makes you feel any better, you can think of this as buying the hardware and they included some free Windows test software in the box.

Troy XCD Serial Server
Okay, so now we have an RS-232 interface we can talk to. If you're like me, your Linux server is not sitting in the middle of the living room next to the Vantage Pro console. And you don't want to run 200 feet of RS-232 cable to your server that is tucked away in an upstairs closet.

The Troy XCD Serial Server is a low cost serial to Ethernet port server used primarily for sharing printers, but it also makes an excellent general-purpose unit for sending and receiving raw RS-232 data to and from Ethernet.

What I like about this unit is it's also an http mini-server, which let's you configure the thing through your web browser. You can even ftp binary files to it and the bytes of the files will be sent out through the RS-232 port.

The output from the WeatherLink interface ties into the Troy Serial Server. The Ethernet connector out of Troy unit can then be plugged into your hub or switch.

Note that if you want to plug your console directly into a serial port on your server, you can still use my software, just leave out the Troy Serial Server.

Linux Server
Your Linux server requires no hardware. It will just listen on the IP address and port address assigned to the Troy Serial Server.

The Software

REMSERIAL
Download: http://lpccomp.bc.ca/remserial/

The remserial program, written by Paul Davis, allows the Troy Serial Server to appear as a local tty port on your Linux server. This is what makes it possible to access the weather station console remotely over an Ethernet connection. You run the remserial program as a background task. Any program that accesses the tty port controlled by remserial will actually be talking to the remote Troy serial server.

The remserial program talks to the 'master' side of a pseudo-tty device and a TCP/IP port on the other. The vproweather program (described below) opens the slave side of the pseudo-tty, thinking it is talking to a serial port, when it is actually talking to the remserial program. It passes the traffic through to the remote TCP/IP address and port of the Troy Serial Server. On Linux, the master side goes from ptyp0 to ptyp9 then ptypa to ptypf and then ptypq* etc. Some other Unix systems use ptyp0 to ptyp255. The slave side of a pseudo-tty is named /dev/tty* where * matches the master side as given above. So for example, if remserial opens the master port /dev/ptyp9, then the serial port that the program will need to use would be /dev/ttyp9.

As an example, if the Troy Serial Server is set up with a static IP address of 192.168.0.5 with its default communication port of 9100, the remserial program can be started as:

remserial -r 192.168.0.5 -p 9100 /dev/ptyp9

This will cause the slave tty device (/dev/ttyp9) on your Linux Server to act just as if the weather station was plugged into a local serial port. However, you probably want to run remserial in the background by putting the following line in your rc.local file:

remserial -d -r 192.168.0.5 -p 9100 /dev/ptyp9 &

The -d parameter and the ampersand (&) character will run remserial as a background task. This allows remserial to be always listening for incoming data and ready to send any outgoing data that's written to port /dev/ttyp9.

The selection of which pseudo-tty port you use may be important. Some pseudo-tty ports will be used for ssh, telnet, and other remote logins. If opening the pseudo-tty port seems to fail for no reason, run lsof and see if it is being used by another process. I would recommend picking a high number like ptyp9 to avoid these problems. Also, start it in rc.local when no pseudo-tty ports will be in use yet.

See the README file in the remserial package for more information.

VPROWEATHER
Download: vproweather-0.6.tgz (57K)

VPROWEATHER is the heart of this whole project. This program is responsible for communicating with the Davis Vantage Pro Weather Station console. It reads the various sections of EEPROM and real time data, and outputs its finding to stdout. It also allows you to control different aspects of the weather station. vproweather provides the following major functions:

Real Time Data   This option allows you to retrieve the real time data from the console. This is most of the information you see on the LCD display.

Highs/Lows Data   This option retrieves highs and lows data as related to weather items such as temperature extremes, highest rain rate, lowest wind chill, etc.

Graph Data   This option retrieves data sets suitable for graphing such as rainfall or temperature over the last 24 hours, 24 days, or 24 months.

How the vproweather program outputs its three types of data sets (Real Time, High/Lows, or Graphs) is through stdout. Normally you would redirect the output to a file so that another script or program can process it (see vproreplace below). For example, to get the Highs/Lows data to a file called hilodata from the slave pseudo-tty port /dev/ttyp0, you would use the following syntax:

vproweather -l /dev/ttyp0 > /tmp/hilodata

This would create a file hilodata in the /tmp directory containing the data. The hilodata file will be a plain ASCII text file with entries like the following:

hlWindHiMonth = 28
hlWindHiYear = 56
hlOutTempHiDay = 78.8
hlOutTempLoDay = 68.9
hlOutTempHiTime = 4:26PM

The above example says that the Highest wind for the month (hlWindHiMonth) is 28 mph. The highest wind speed for the year (hlWindHiYear) is 56 mph. The highest outside temperature so far for today is 78.8 F degrees (hlOutTempHiDay), which occurred at 4:26PM (hlOutTempHiTime). You get the idea. A complete description of every variable/value pair for each data output command is listed in the Data File Variables section.

Time functions   Allows you to synchronize the clock in the weather station console with your server's system clock. Also, you can read the clock to a file, which can be used in html or logging files to record the time of weather events.

Backlite Control   Allows you to turn the LCD backlite on or off. You could for example, use a cron job to turn the backlite on and off in the morning and evening, or flash the backlite during some severe weather event.

ID Functions   Allows you to query the weather station for firmware version and model number/name.

The help screen for the vproweather program is as follows:

Usage: vproweather [options] device
Options:
-x, --get-realtime    Get real time weather data.
-l, --get-highlow     Get Highs/Lows data.
-g, --get-graph       Get graph data.
-t, --get-time        Get weather station time.
-s, --set-time        Set weather station time to system time.
-o, --bklite-on       Turn backlite on.
-f, --bklite-off      Turn backlite off.
-r, --version         Query for Davis firmware version string.
-m, --model           Query for weather station model name.
-d, --delay=num       Cmd Delay in 1/10ths seconds. Default is 10 (1 sec).
-v, --verbose         Verbose mode.
device                Serial Device. Required parameter.

Examples:
vproweather --get-realtime /dev/ttyp0 > rtwdata.txt
Gets real time data set to file rtwdata.txt from serial device ttyp0
vproweather --verbose --bklite-on /dev/ttyp0
Turns the LCD backlite On, illuminating the display.

Options Descriptions

-x --get-realtime    Sends the Real Time data set to stdout.

-l --get-highlow     Sends the Highs/Lows data set to stdout.

-g --get-graph       Sends the Graph data sets to stdout.

-t --get-time        Gets the current time set in the console. The time is returned as a string in this form:

DavisTime = Monday, July 29, 2004 06:49 PM

You could redirect this output string to any of the data files or its own file so that the time can be used by other programs.

-s --set-time        Sets the clock in the weather station console to the time set in your Linux machine. Note that there are no parameters to this option. The time is taken from the system clock. In this version of vproweather, there is no way to set the time to anything other than the system time.

-o bklite-on         Turns the LCD backlite on and off.
-f bklite-off

-r --version         Returns the version string from the console. This command is normally used during testing so you don‘t have to run a lot of data back and forth. The contents of the string returned will be a time and date with the word 'OK' indicating the version of the internal firmware.

-m --model           Returns a string with the model name of your weather station. Again, this is good for testing and verifying communications. Note that if the model is not recognized as the Vantage Pro, vproweather will output a warning about compatibility issues with the program.

-d --delay=num       This parameter specifies the amount of time to wait for data to be sent back to the server. Since RS-232 communications are asynchronous, there is no way to tell when characters will appear at the output after a command is executed. If you don't use this option, the default will be 1 second, which should be fine for most installations. However, if you frequently get wrong data counts or not enough data (you will get warnings to stderr) you may increase the delay value to compensate for this. The value specified is in 1/10ths of a second. The default value is 10 (1 second) and you may specify any number from 0 to 255. Again, you probably don't need to mess with this at all so don't use this option unless you have to.

-v --verbose         Detailed descriptions of what is happening. Useful for debugging. You would probably never want to use the -v option when you are using any of the data set options( -x, -l , or -g) because the verbose messages will just wind up scattered throughout stdout and ultimately in your data files.

Note that vproweather sets the Baud rate of the serial port to 19200, which is the default Baud rate of the Vantage Pro console. Make sure that baud rate of the console has not been changed to another value.

VPROREPLACE
Download: vproreplace-0.1.tgz (18K)

So now that we've got all this data in a file, we have to get it to some html web pages for display. This is where the vproreplace program comes in. It takes the variable/value pairs in the generated data files and replaces html comment tags in html files with the data. For example, suppose we have an html file called tmpl.html with the following line in it:

cat tmpl.html
The outside temperature is <!-- rtOutsideTemp --> degrees.

This will serve as our "template" file or the input to the vproreplace program. We also have a data file generated by vproweather at /tmp/rtdata that has the following line in it:

cat /tmp/rtdata
rtOutsideTemp = 78.6

Now we run vproreplace as follows:

vproreplace -t tmpl.html -d /tmp/rtdata > final.html

This takes the template file (tmpl.html) and the data file (/tmp/rtdata) and produces an output file called final.html. This file will contain the following:

cat final.html
The outside temperature is 78.6 degrees.

What vproreplace does is go through your template file and replace any variable references with the actual data from the vproweather generated data file.

There are a few rules when creating your template files. You can have any amount of whitespace (spaces or tabs) before and after the html comment tags, so that:

<!--rtOutsideTemp-->
<!--      rtOutsideTemp--     >

Will both work fine. However, do not split html comment tags across lines. Also, If you have any other html comment tags in your template files, vproreplace will not disturb them and will copy them verbatim to stdout.

The graph data files contain lists of data. These variables are generated with the -g option of vproweather. For example, running the command:

vproweather -g /dev/ttyp0 > grdata

will produce the file grdata which will have entries in it like this:

grRainRateHiByDays = 2.22,2.03,1.18,1.86,0.31,1.43,5.33,0.01, ...
grRainRateHiByDaysTime = 2:07PM,3:50PM,12:58PM,2:22PM, ...
grTimeRefDays = 6/30,7/1,7/2,7/3,7/4,7/5,7/6,7/7,7/8, ...

The variables always consist of 24 comma separated values. If the template file contains a line like this:

<p>On <!-- grTimeRefDays[1] --> at <!-- grRainRateHiByDaysTime [1] -->, The rain rate was <!-- grRainRateHiByDays[1] -->> inches per hour.</p>

The vrporeplace program will produce the following output:

<p>On 7/1 at 3:50PM, The rain rate was 2.03 inches per hour.</p>

For graph data, you specify the index of the data you want by placing a number in brackets after the variable name. The index can range from [0] to [23]. Note that to get the first entry, you specify a value of zero and not 1. There must not be any spaces between the variable name and the brackets. If you leave out the brackets, the entire string of comma separated values will be returned. So that:

<p>The rain rate was <!-- grRainRateHiByDays --> inches per hour.</p>

Would produce the string:

<p>The rain rate was 2.22,2.03,1.18,1.86,0.31,1.43,5.33,0.01,0.00,0.00,4.80, 0.65,0.11,0.00,1.26,0.00,0.00,2.53,0.00,0.00,0.00,0.00,0.00,0.00 inches per hour.</p>

Which is probably not what you intended.

Metric Converter
Download: metconv.tar (10K)

The native output format of the data from the Vantage Pro is in english units. If you need to convert to metric, Mike Kelly of Sydney, Australia (mike_at_kirrawee.net) has written a perl script that takes the output of vproweather and converts all the data to metric. This neat perl script converts Fahrenheit to Celsius, inches to millimeters, miles to kilometers, inches of mercury to millibars, and even swaps the date and month. Simply run the vproweather generated files through the script via stdin and stdout to convert the data.

The Glue
Download BASH Scripts: weather-0.1.tgz (41K)
Download Sample Web Pages: weatherweb.tgz (192K)

Finally, to tie this all together, I use a few BASH scripts.

rtweather.sh    This script runs as a cron job every 60 seconds. It calls vproweather and gets the real time data to the /tmp directory. It also gets the weather station time (using the -t flag) so I can display the actual weather station time along with the data. The script then calls vproreplace and passes the real time data file (/tmp/rtdata) and the html template file (realtime.tmpl) to produce a file called rt.hml that web browsers can view.

The good thing about this method is that there is no delay in getting the weather station data to a web browser. Since the rt.html file is created (and overwritten) every 60 seconds via the cron job, your web browsing guest will always view the latest data without having to wait for downloads from the weather station. Of course, an alternative would be to do a perl or other cgi-bin script and call up the weather station on each request, but it can take up to 15 seconds in some cases to download the data from the console. In this world of impatient internet users, 15 seconds is a long time.

exweather.sh    This script runs as a cron job every 10 minutes. This produces a single data file with both the Highs/Lows data and Graph data in it. This is processed by vproreplace and creates four html files (grain.html, grtemp,html, grwind,html and hl.html) for browsers to view. I generate my graphs using a simple javascript that creates bar graphs by using colored table elements. Take a looks at any of these template files (grwind.tmpl, grain.tmpl, or grtemp.tmpl) for details on how to do this. It's a quick dirty way to make graphs without resorting to GD::Graph under perl or create images on the fly or other methods.

settclock.sh    The settclock.sh script sets the weather station clock to the system time. I actually run this in rc.local because the system and weather station console time do not get out of sync very much. I suppose you could also run this as a cron job, perhaps once a month.

weatherfunc.sh    This script just holds a bunch of functions used by rtweather.sh and exweather.sh. Note that before you use any of these scripts, you should modify the variables at the top of each script file for your particular needs.

An important thing to remember when setting up cron jobs is that the serial port is not a multi-tasking, multi-threaded entity and if you happen to run two copies of vproweather at the same time, they will both fail as the serial port will get clobbered with data from two different sources. To work around this, my scripts rtweather.sh and exweather.sh check to see if vproweather is already running and if so, it waits up to 20 seconds for it to finish. It can take upward of 10-15 seconds to get a lot of data from the weather station so you need to be careful that multiple copies of vproweather can't run at the same time. I will probably address this in future versions via some sort of lock file mechanism but for now, you need to guard against this occurrence. Using my scripts as an example for your own will get around this.

The cron job for rtweather.sh runs every minute except on the 10, and the exweather.sh script runs every 10 minutes. Here is what my cron lines look like for these two programs:

1-9,11-19,21-29,31-39,41-49,51-59 * * * * /root/weather/rtweather.sh
0,10,20,30,40,50 * * * * /root/weather/exweather.sh

The other thing I do in my scripts is to check for the existence of the remserial program, and if it's not running, start it up as a background task. I have had AC mains power failures where remserial wound up with a broken pipe and terminated itself. The code in the script makes sure that if this happens, remserial will be restarted again.

Data File Variables

The output of vproweather produces lists of data containing the following definitions:

Real Time Data   Real Time Data output is invoked by using the -x option with vproweather. The following lines are generated:

Download