Strange rain readings

[yorky]

Hi all,

I have had this happening for a while (year or so). I thought it was my aging PC but after a brand new Windows 7 Pro 64bit from XP it still does it.

Cumulus records rain suddenly, at random times with random amounts from a few mm to 10's of mm's. It doesn't do it when the PC is off, only when it is running and gathering per minute data.

To fix it each time I close Cumulus and use the Cumulus Toolbox to edit the rain out. It happens around once a week sometimes more often. Using the spike removal I can make it not record record rain rates but it still thinks it rained, and also changes the hourly rain and rain after midnight.

Sometimes but not always, the Rainfall counter goes DOWN then UP, the difference being what it records as rain. Again this never ever happens with the PC off/Cumulus not running. It's getting to a stage that I'll just run Cumulus once a day to download data and upload, then close Cumulus. Today it added 8.1mm but the rainfall counter didn't move.

The display on the Fine Offset WA2081 does not display the rain when this error occurs.

[steve]

Yes, this is a problem with some Fine Offset stations. I've added a lot of code to Cumulus to try to cater for it, far more than just the spike removal settings. If it's happening a lot to yours, then perhaps you are getting interference from somewhere, or your batteries need changing. You are using the latest version of Cumulus, presumably?

[yorky]

Knew I forgot to mention something, yes v1.9.4 a couple of builds old. Thanks for quick response Steve.

[steve]

If you zip up the diags folder and attach it, I'll see if I can see any pattern to when it happens.

[yorky]

Sure, does it matter when the issue happens? Would it be easier to give it to you just after the event happens?

[steve]

Yes, that would probably be best.

[NZian]

I have had this also, as probably a lot of us have, and am starting to suspect that wind gusts are the problem.
Had a day show up in records of 20M (meters) in a day! Unlikely....
It happens frequently although after adjusting the location/position/height above roof etc, it has calmed down it seems.
regards
Ian

[steve]

I think it's been demonstrated in the past that electrical interference on the anemometer wire can cause spikes in not just the wind but also the rain and/or temperature readings. Erroneous jumps in the rain total are tricky to handle sensibly (and things like rain in the last hour and last 24 hours particularly so). The current code does (or should) try to handle large sudden increases in the rain counter, when running with live data, but not (currently) when processing logger data.

[guesswhatigotforxmas]

I seem to get odd behaviour all round when there are spiders webs in the rain sensor

[yorky]

Ok it just happened again 15min ago. However its slightly different to how I originally had the issue, the rainfall counter didn't actually change at all, it just put a phantom 3.6mm rainfall so far in (as well as rail since midnight). It also did the same thing yesterday.

Attached is the diags folder.

The only thing that I noticed is the rain Start= is today.ini is 3.6 less than it should be. I have to manually change this from 1557 back to 1560.6 so somehow its changed that by itself. I have spike removal of max rain rate and max hourly rain set so it doesn't trigger them/record them.

[steve]

The rain counter did change; it went down from 1560.6 to 1557. Cumulus ignored the change for a few readings to make sure it wasn't a one-off glitch, but as it was apparently permanent, it reset the start of day counter to compensate. Shortly afterwards the counter went back up to 1560.6, so 3.6 mm of rain was recorded.

4/11/2014 14:53:00.942 : Writing today.ini, LastUpdateTime = 4/11/2014 2:53:00 PM raindaystart = 1560.59997558594 rain counter = 1560.59997558594
4/11/2014 14:53:00.942 : Latest reading: 28C0: Data: 03 2D FC 00 33 F0 00 7B 27 36 4E 00 02 52 14 80
4/11/2014 14:53:54.908 : Humidity difference greater than specified; reading ignored
4/11/2014 14:53:54.916 : Old value = 52 New value = 83 EWhumiditydiff = 10.0
4/11/2014 14:53:54.922 : 2:53:54 PM ****Rain reset? First chance: raindaystart = 1560.59997558594, raintotal = 1557
4/11/2014 14:53:54.922 : Leaving counter at 1560.59997558594
4/11/2014 14:53:54.922 : Temp difference greater than specified; reading ignored
4/11/2014 14:53:54.927 : Old value = 23.9 New value = 15.1 EWtempdiff = 2.0
4/11/2014 14:53:54.932 : Pressure difference greater than specified; reading ignored
4/11/2014 14:53:54.936 : Old value = 1017.9 New value = 1020.9 EWpressurediff = 2.0
4/11/2014 14:54:01.032 : Writing today.ini, LastUpdateTime = 4/11/2014 2:54:00 PM raindaystart = 1560.59997558594 rain counter = 1560.59997558594
4/11/2014 14:54:01.032 : Latest reading: 28D0: Data: 05 33 D1 00 53 97 00 9A 27 0E 22 00 02 46 14 80
4/11/2014 14:54:04.885 : Humidity difference greater than specified; reading ignored
4/11/2014 14:54:04.890 : Old value = 52 New value = 83 EWhumiditydiff = 10.0
4/11/2014 14:54:04.897 : 2:54:04 PM ****Rain counter reset confirmed: raindaystart = 1560.59997558594, raintotal = 1557
4/11/2014 14:54:04.897 : Setting raindaystart to 1557
4/11/2014 14:54:04.897 : Temp difference greater than specified; reading ignored
4/11/2014 14:54:04.901 : Old value = 23.9 New value = 15.1 EWtempdiff = 2.0
4/11/2014 14:54:04.906 : Pressure difference greater than specified; reading ignored
4/11/2014 14:54:04.912 : Old value = 1017.9 New value = 1020.9 EWpressurediff = 2.0
4/11/2014 14:54:14.908 : Humidity difference greater than specified; reading ignored
4/11/2014 14:54:14.915 : Old value = 52 New value = 83 EWhumiditydiff = 10.0
4/11/2014 14:54:14.920 : Temp difference greater than specified; reading ignored
4/11/2014 14:54:14.925 : Old value = 23.9 New value = 15.1 EWtempdiff = 2.0
4/11/2014 14:54:14.930 : Pressure difference greater than specified; reading ignored
4/11/2014 14:54:14.934 : Old value = 1017.9 New value = 1020.9 EWpressurediff = 2.0
4/11/2014 14:55:00.114 : Writing today.ini, LastUpdateTime = 4/11/2014 2:55:00 PM raindaystart = 1557 rain counter = 1560.59997558594

It appears that the station is moving the current memory location pointer on to the next location too soon - well before it has actually started writing data to the next location. In this case, it was using location 28C0, and had been for the last 3 minutes (the first byte of the data gives the number if minutes since that location started being used):

4/11/2014 14:53:00.942 : Latest reading: 28C0: Data: 03 2D FC 00 33 F0 00 7B 27 36 4E 00 02 52 14 80

It then changed the counter to 28D0, but the minute counter in that entry is still 5, so this is almost certainly old data from last time around:

4/11/2014 14:54:01.032 : Latest reading: 28D0: Data: 05 33 D1 00 53 97 00 9A 27 0E 22 00 02 46 14 80

When the location changes, Cumulus skips a reading to allow for the fact that the station changes the pointer just before it writes new data to the new location. But in this case it didn't write the new data to the new location, even after skipping a reading. The change in readings then triggered your spike removal settings and caused the rain counter reset.

Some time during the next minute, the station actually started writing new data to location 28D0, so all of the readings returned to normal (note the zero in the first byte):

4/11/2014 14:55:00.114 : Latest reading: 28D0: Data: 00 2D FC 00 34 EF 00 7C 27 30 55 00 0A 52 14 80

So, how did this happen? 28D0 is 28C0 with an extra bit set, so perhaps the memory holding the current location became corrupted, i.e the bit flipped somehow. When you had the same problem yesterday, it appears that exactly the same thing happened - the station moved the current location pointer on to the next location a minute before it should have done. But in this case the locations in question were 1710 and 1720 (hex), and this requires two bits to flip. So it looks less like a memory corruption and more like a bug where it actually did move the counter on, but it did it a minute too soon, i.e. before it actually started using that location.

It's perhaps relevant that the problem happened at around the same time yesterday (15:13) and today (14:53). Looking back, you had a 'sensor contact lost' at 15:25 on 31/10. Does your station have a radio-controlled clock?

[yorky]

Not sure what you mean by radio controlled clock, it was set and forget kind of thing. I might check if its in sync.

[steve]

Some FO stations have clocks which are supposed to be kept to the correct time by receiving radio signals; in Europe we have one in the UK and one in Germany. I don't know whether you have one in Australia or whether your station is one of the ones which has that facility. It may not be relevant anyway, it was just a thought that the problem occurs when the station sets its clock. (Edit: It appears that the WA2081 does not have a radio-controlled clock).

I looked again at your diags logs and I can see a few other places where the current location counter is being moved on well before new data is being written to it, at other times of day. I also checked the operation of the code on Cumulus which skips a reading when the location changes, and it does appear to be working. Effectively, this allows at least ten seconds between the current location changing and new data being written to it. Your station (and perhaps others of the same model?) is apparently changing the location more than ten seconds before it starts using it.

[steve]

Having had a chat with Jim Easterbrook (who also writes software for Fine Offset stations and has more understanding of the way the FO firmware does things than I do) it appears that the way the 'current location' pointer is handled is slightly different to how I thought it worked. It seems that the console advances the pointer on the minute (according to the console clock), but (obviously) doesn't write new data to the new location until the next data arrives from the sensors, up to 48 seconds later.

To fill in the gap, it apparently copies the data from the old location to the new location. This is what allows the 'skip a read' mechanism in Cumulus to work. It appears that your console is not doing this part of the operation. What is not clear is whether it is some kind of issue with your console, or whether this is standard for the WA2081 and similar models. You say this started happening in the last year, so this would suggest that the problem is not a standard feature. When did you last do a full reset of the console?

[yorky]

Thanks Steve, yes I'd tend to agree there isn't such a radio controlled thingy in Aus anyway. Can't remember the last time I reset it, most likely when the batteries needed changing which would have been one year plus ago since its running off USB power most of the day except 8 hours at night.

Do you recommend a complete power off and reset of the console?