Troubleshooting WUnderground non-upload

bpsmicro · Post by **bpsmicro** » Sun 02 Nov 2008 1:28 pm

After getting everything else working, I've noticed that Cumulus is not uploading to my Weather Underground account at all. I turned on debug logging, and I can see the log entry where it appears to be sending th data to the wunderground URL, and everything in it looks right and no errors are logged.
I've tried both with and without RapidFile (I'd prefer without), but my page at http://www.wunderground.com/weatherstat ... ONTARIO122 shows no activity since the time I turned off WeatherLink and turned on Cumulus last week.
Any suggestions on how I can troubleshoot this?

Brad.

bpsmicro · Post by **bpsmicro** » Sun 02 Nov 2008 1:50 pm

As a test, I copied the entire URL out of the debug log and pasted into my browser, and that worked (so there's now one entry in the wunderground chart). So we know the URL generation is correct. I just think it's not actually doing the HTTP connection.

Brad.

Post by **steve** » Sun 02 Nov 2008 4:31 pm

I assume there were no error messages in the log, so that rules out a timeout condition.

Could you do a web update from the file menu, and see if it says 'update success'?

Steve

bpsmicro · Post by **bpsmicro** » Sun 02 Nov 2008 5:41 pm

The "Web Update" appears to FTP up all the various files to my FTP server, and it has no errors. That particular dialog doesn't reference the Wunderground stuff though.

Post by **steve** » Sun 02 Nov 2008 6:53 pm

Oh - yes. The wunderground update code logs messages to that window, but the 'web update' from the menu doesn't do the wunderground upload.

Anyway, I've added some more diagnostics to the Wunderground code - I haven't needed any before because it's always 'just worked' before. It'll be in the next beta upload, probably tomorrow.

Steve

bpsmicro · Post by **bpsmicro** » Sun 02 Nov 2008 7:15 pm

Cool, I'll keep an eye out for it and let you know as soon as I have more info.

Everything else is working great. Next week I'm going to start playing with the customization of the web pages (just for fun).

Post by **steve** » Mon 03 Nov 2008 10:04 am

I've uploaded the new build. You will need to edit cumulus.ini to turn on some of the extra logging. To the [Wunderground] section, add:

Code: Select all

Logging=1

This will create a file called httplog.txt

I've also added a bit more to the debug log, so please turn that on also.

Thanks,
Steve

bpsmicro · Post by **bpsmicro** » Mon 03 Nov 2008 11:57 am

Well, this is interesting. With RapidFire on or off, the log gets:

0139.240 : 06:43:13 Response Code: 0 Response = <HEAD><TITLE>403 Forbidden</TITLE></HEAD>
<BODY><H1>403 Forbidden</H1>
<P>The request was not properly formatted. A possible security risk detected.</P>
</BODY>[/color]

I wondered if it was coming from my virus scanner (Avast) so I turned it off with no effect. It's odd that I can copy/paste the URL right out of the log into the browser and not get that error though.

Brad.

Post by **steve** » Mon 03 Nov 2008 12:43 pm

Very odd. I think that response is from Wunderground, so maybe something is happening to the string before it gets there. It's a shame Wunderground isn't more helpful. Does the http log have anything interesting?

Steve

bpsmicro · Post by **bpsmicro** » Mon 03 Nov 2008 12:49 pm

Depends on your definition of "interesting"

:

07:44:42:558 SendRequest Done
07:44:42:558 00E87E60 TriggerDataSent 580
07:44:42:558 00E87E60 TriggerDataSent 580
07:44:42:568 Data available. Len=349
07:44:42:568 >|HTTP/1.0 403 Forbidden|
07:44:42:568 FCloseReq=1
07:44:42:568 >|Proxy-agent: Ositis-WinProxy|
07:44:42:568 >|Connection: close|
07:44:42:568 >|Pragma: no-cache|
07:44:42:568 >|Cache-Control: no-cache|
07:44:42:568 >|Content-Type: text/html|
07:44:42:568 >|Content-Encoding: 7bit|
07:44:42:568 >|Content-Length: 161|
07:44:42:568 >||
07:44:42:568 End of header
07:44:42:568 State = httpWaitingBody
07:44:42:568 GetBodyLineNext begin
07:44:42:568 DocBegin
07:44:42:568 GetBodyLineNext FBodyDataLen=161
07:44:42:568 End of document
07:44:42:568 State = httpBodyReceived
07:44:42:568 7bit content uncompressed from 161 bytes to 161 bytes
07:44:42:568 DocEnd
07:44:42:568 GetBodyLineNext end
07:44:42:568 Data available. Len=0
07:44:42:568 **data available. Len=0
07:44:42:568 SessionClosed Error: 0
07:44:42:568 00E87E60 TCustomWSocket.Shutdown 1 580
07:44:42:568 State = httpReady
07:44:42:568 PrepareNTLMAuth end, FStatusCode = 403 FProxyAuthNTLMState=0 FAuthNTLMState=0
07:44:42:568 PrepareNTLMAuth end, FStatusCode = 403 FProxyAuthNTLMState=0 FAuthNTLMState=0
07:44:42:568 RequestDone
07:46:42:629 State = httpNotConnected
07:46:42:629 Login weatherstation.wunderground.com
07:46:42:629 State = httpDnsLookup
07:46:42:639 State = httpDnsLookupDone
07:46:42:639 connect to 64.243.174.125/80
07:46:42:639 TWSocket will connect to 64.243.174.125:80
07:46:42:649 SessionConnected
07:46:42:649 State = httpConnected
07:46:42:649 State = httpWaitingHeader
07:46:42:649 4 header lines to send
GET /weatherstation/updateweatherstation.php?ID=IONTARIO122&PASSWORD=*****&dateutc=2008-11-03+12:46:42&winddir=207&windspeedmph=3&windgustmph=10&humidity=81&tempf=39.4&rainin=0.00&dailyrainin=0.00&baromin=30.30&dewptf=34.0&softwaretype=Cumulus v1.8.3&action=updateraw HTTP/1.0
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*
User-Agent: Mozilla/4.0 (compatible; ICS)
Host: weatherstation.wunderground.com

Post by **steve** » Mon 03 Nov 2008 12:56 pm

Something to do with your proxy?

Steve

bpsmicro · Post by **bpsmicro** » Mon 03 Nov 2008 1:03 pm

Hard to say. The Proxy is actually on the gateway machine that drives my entire network. I poked through the settings there to see if I could spot anything awry, but it's really mostly for connection sharing across the LAN. *Everything* goes through it, and it's not really something I can bypass without some major reworking of the LAN. There's nothing particularly unusual about the URL you're using that would concern me (other than it's really long). I'm going to keep poking at stuff though.

Post by **steve** » Mon 03 Nov 2008 1:12 pm

http://www.redline-software.com/eng/sup ... ssages.php

"403 Forbidden Commands

[...]

The request was not properly formatted. WinProxy will permit requests only with properly formatted strings. This error is usually generated by programs that do not strictly follow the http 1.0 or 1.1 specification."

So maybe I'm not following (or the http component I use isn't following) the http 1.0 specification?

Steve

Post by **steve** » Mon 03 Nov 2008 1:16 pm

Also found this:

Incorrectly formatted http requests, URL strings, and unsupported protocols
Some of the more common reasons for getting it are incorrectly formatted http requests, incorrectly formatted URL strings, and unsupported protocols. WinProxy is pretty strict about the format of statements going through, and won't pass those that are incorrectly formatted. A failure going out will get you a 403, a failure coming in is invisible. Some pages with lists of files or sites have whole sections of URLs that will get a 403; most often the problem is a space in the URL.

There is a space between Cumulus and v1.8.3 - I guess I should be encoding this as %20?

Steve

bpsmicro · Post by **bpsmicro** » Mon 03 Nov 2008 1:29 pm

I wonder if, when pasting the URL into IE and sending that way, IE itself is reformatting to more strictly follow the spec, which is why it seems to work that way.
WinProxy has it's own (albeit crappy) logging. I'll turn that on and see if it yields any clues. But the stuff you posted sure *seems* like a likely suspect.

Cumulus Support

Troubleshooting WUnderground non-upload

Troubleshooting WUnderground non-upload

Re: Troubleshooting WUnderground non-upload

Re: Troubleshooting WUnderground non-upload

Re: Troubleshooting WUnderground non-upload

Re: Troubleshooting WUnderground non-upload

Re: Troubleshooting WUnderground non-upload

Re: Troubleshooting WUnderground non-upload

Re: Troubleshooting WUnderground non-upload

Re: Troubleshooting WUnderground non-upload

Re: Troubleshooting WUnderground non-upload

Re: Troubleshooting WUnderground non-upload

Re: Troubleshooting WUnderground non-upload

Re: Troubleshooting WUnderground non-upload

Re: Troubleshooting WUnderground non-upload

Re: Troubleshooting WUnderground non-upload