Setting up CumulusUtils (a beginners thread)

[alexvanuxem]

Hi all

I'm a first user of the utils feature, and I have some beginners issue I can't work out.

I don't find a CUserdataRECENT.json file generated?

I don't see my inside temp and humidity displayed in the gauges?

I have extra sensors (soil, leaf station,(davis) and lightning (ecowitt): whatever i try I can't generate the web extra sensor page.

Also I don't have extrasensorscharts.txt, and extrasensorsrealtime.txt and extrasensorsdata.json remains empty.

Please advise, attached is my ini file
Grtz

[AndyKF650]

Hi there

From your message it seems that you have not come across the CUtils ExtraSensors wiki as per the following link. https://www.cumuluswiki.org/a/Extra_Sensors

You need to generate the missing files by using the cmd extrasensors and cmd useraskeddata and then cmd website to start the process. I will admit it is a bit of a fiddle but once you get there all becomes a lot more visible and easy. Dont forget to set up your sensors in CMX as CUtils also needs this.

[HansR]

Hi Alex,

Let's take this one step at a time. CumulusUtils is a modular utility and the first step will be to have the CUtils website operate at its standard level without modifying much.

I advise you to read the wiki carefully on the subjects you are working on although I think you did already as the website is up.

First thing is that the inifile parameters DoLibraryIncludes and GeneratejQueryInclude are default defined as false. However in your inifile they are:

Code: Select all

DoLibraryIncludes=true
GeneratejQueryInclude=true

You need to set those to false!

I'm a first user of the utils feature, and I have some beginners issue I can't work out.

I don't find a CUserdataRECENT.json file generated?

I have extra sensors (soil, leaf station,(davis) and lightning (ecowitt): whatever i try I can't generate the web extra sensor page.

Also I don't have extrasensorscharts.txt, and extrasensorsrealtime.txt and extrasensorsdata.json remains empty.

We will leave the ExtraSensors until the basic website is functioning

I don't see my inside temp and humidity displayed in the gauges?

Your parameter is

Code: Select all

ShowInsideMeasurements=true

that is correct. However, you must activate the inside data in CMX as well.
I don't know which version of CMX you use because that has changed somewhat over the successive versions. In CMX you can configure which datafields you wish to see in the dashboard and on the website. If you want to see fields on the website it will be stored in the datafile (tempdata.json and transferred to the website.

I would first like to focus on the internal Temp/Humidity (which must show) but above all on the charts which should be available in the right part of the site (I call that the reportview). Once the site is configured for the regular things we will look at the ExtraSensors.

I noticed the site is generally working and so is the AirLink, that is good

To get things going, could you post the logfile from the run with the commandline parameter website? Thanks.

[alexvanuxem]

i done all the wiki stuff but i m lost now , apologies

started from scratch agin, i lost almost anything, also the airlink , noaa , etc

lost the CUUserdata json files.

on the plus side i got soilmoist and soiltemp jsons uploaded,

i must be doing something wrong

[AndyKF650]

Hi there

It seems to me that your dayfile date stream is causing issues. You will need to address these before going forward. Any editing needs to be done either on your RPi or in notepad++ or other good editor.
You could put your dayfile onto this site as a zip so that we can identify the problem.

[HansR]

i done all the wiki stuff but i m lost now , apologies

started from scratch agin, i lost almost anything, also the airlink , noaa , etc

lost the CUUserdata json files.

on the plus side i got soilmoist and soiltemp jsons uploaded,

i must be doing something wrong

OK. There is a crash after reading the dayfile. That probably has to do with the RecordsBeganDate parameter.
In your inifile that is defined as:

Code: Select all

RecordsBeganDate= 15/10/2019

you should change that to:

Code: Select all

RecordsBeganDate=15/10/19

.

In the wiki it gives as a format: 01/01/20 (od dd/MM/yy); I will make that more clear but above all, I will prevent it for crashing because the data is by definition user entered and a format error can be made. That is for a next release, editing the date to the correct format should give the workaround.

Please modify, rerun and post the new logfile.

[alexvanuxem]

thx for reply!

yes i noticed it too

got some things working again: noaa reports and graphs (although the rain graph seems to stop at 10 march?)

got my extra sensors and and the airklink working again.

not working inside temp and humidity

im using the latest build Cumulus MX v.3.24.1 build 3234

no right pain also.

[HansR]

got some things working again: noaa reports and graphs (although the rain graph seems to stop at 10 march?)

The rain chart stops at 10 march because that data is not there in your dayfile (see YADR data report which shows the detailed data)
I don't know why it is not there because the dayfile comes straight from CMX. You should investigate what goes wrong there (you are missing more days). Assuming your logfiles are OK, it would be wise if you run CreateMissing.exe (see wiki). Note that maybe CreateMissing has not been updated to the latest version of CMX in which case it will not run. We'll have to ask @mcrossley to create a newer version in that case.

got my extra sensors and and the airklink working again.

I don't see any ExtraSensors. Note that you better leave UserModificationExtraSensorCharts=false (the default) until we are sure everything works fine.

not working inside temp and humidity

im using the latest build Cumulus MX v.3.24.1 build 3234

no right pain also.

We leave this until the charts (the Home menu choice) are available because that is where the setting for the inside is transferred to the gauges. The version 3.24.1 for CMX is OK (should be OK )

Note that your URL in section [Maps] has a typo: htps://sint-katelijne-waver-meteo.be
Note that your path for FTP may be UploadDir=wwwroot (the trailing slash is not required and leads to a double slash (see FTP.log)

Note that when getting the usermap from the meteo-wagenborgen server there is a fail as well. I don't have a solution at the moment. It is not critical as CUtils conitnues sending all result files to your site. So we leave it atm.

Main problem is the creation of the Home charts, the charts of the RECENT history. And then researching that I find that, although you do a 'website run', the index.html is not generated. I miss the line in the the logfile like:

Code: Select all

<Datetime> Generating Website: Start
<DateTime> Generating Website: Generating index.html

Also I see that you have a menu based on the CutilsMenu.def file on the site but I do not see the generation of that in the logfile. So it seems the logfile is a mix and is not the run resulting in what I see on the site. If CutilsMenu.def and/or CutilsCharts.def are in the utils directory I suggest you remove those (backup for later use).

OK, step by step we'll get there.

Please make changes above and run again. Supply resulting logfiles and cumulusutils.ini. Thanks.

[HansR]

The menu seems a mix from other sites. It is as if the index.html is not generated by you but as if it is fetched from another site where you only supply the additional files. Especially the menu looks strange. Where does that come from?

[alexvanuxem]

indeed the index.html is not updated

got some progress

see sint-katelijne-waver-meteo.be

i'm missing some files like CUUserdaily.json and the like.

made all the adjustments asked

the index.html will not update, for some reason.

i'm running the mono utility on windows 11 pro

each time I need to CD to CumulusMX and then run mono cumulusutils.exe
thx!!!!

[alexvanuxem]

and now the attachments

the website is also offline since yesterday,

i changed the CuCharts.def and the Cumenu.def from default to this..

[HansR]

indeed the index.html is not updated

got some progress

see sint-katelijne-waver-meteo.be

i'm missing some files like CUUserdaily.json and the like.

made all the adjustments asked

the index.html will not update, for some reason.

i'm running the mono utility on windows 11 pro

each time I need to CD to CumulusMX and then run mono cumulusutils.exe
thx!!!!

OK. Progress.

On Windows you do NOT need to run mono (mono is for Linux/Unix). CMX and CUtils will run just like that on windows. Just go to your CMX directory and run the executables (CUtils with the path to utils\bin)

The logfile says:

Code: Select all

2023-03-13 19:19:46.795 SystemStatus : starting
2023-03-13 19:19:46.827 Information GetUrlData Start: URL - http://localhost:8998/api/tags/process.json?version&build&ProgramUpTime&NewBuildAvailable&NewBuildNumber&CpuCount&CPUTemp 
2023-03-13 19:19:51.078 Error GetUrlData : Exception - An error occurred while sending the request
2023-03-13 19:19:51.078 Error GetUrlData: Inner Exception: System.Net.WebException: Error: ConnectFailure (Kan geen verbinding maken omdat de doelcomputer de verbinding actief heeft geweigerd.
) ---> System.Net.Sockets.SocketException: Kan geen verbinding maken omdat de doelcomputer de verbinding actief heeft geweigerd.

This indicates that either CMX is not running or it cannot connect.

Anyway, that may have to do with using mono. As said don't use mono on Windows.
I assume CMX is running.

1. Is CMX already configured and running on this computer?
2. Your FTP seems to be correctly running and all files are uploaded
3. There is one thing which bothered me since the beginning : the datafiles are named with a prepend wwwroot but I think I now know why: you filled in the parameter

   Code: Select all

   CumulusRealTimeLocation=wwwroot

   That parameter should not be used in first instance. The parameter is intended if you have more than one website and want to use only one location for the realtime file. Ignore this at the moment and just empty the parameter.

Modify and run again. Please repost the logfiles and the ini afterwards.

NOTE: you only need to run website other things come later. And please never do more runs within the same minute because then the same logfile is used and that is confusing.

NOTE: do not modify or fill in inifile parameters if you don't know what they do. It wil disrupt the functioning of CUtils. Then when the whole thing does what you want it to do you can start modifying.

[alexvanuxem]

thx for your help!!

yep the mono thing was the culprip

check sint-katelijne-waver-meteo.be

I also have some other sites :

sint-katelijne-waver-meteo.be/pwsWD
sint-katelijne-waver-meteo.be/template
sint-katelijne-waver-meteo.be/weewx
sint-katelijne-waver-meteo.be/wd
sint-katelijne-waver-meteo.be/Weather34
sint-katelijne-waver-meteo.be/Cumulus

I still need to play with maps ....

had to upload the noaa reports manually in the ini its set to ./reports/ also in CMX upload to wwwroot/reports (here it might be also ./reports ?)

One question please: i'm using also the cumulus multilingual site; can I use both (utils and ml?), because in the CMX remote ftp upload folder one or the other becomes offline (wwwroot/ or wwwroot/Cumulus/)

thx again, great help!!!

[HansR]

thx for your help!!

yep the mono thing was the culprip

check sint-katelijne-waver-meteo.be

That is good
You probably wanted too many things at once. From here you can go deeper step by step.

I also have some other sites :

sint-katelijne-waver-meteo.be/pwsWD
sint-katelijne-waver-meteo.be/template
sint-katelijne-waver-meteo.be/weewx
sint-katelijne-waver-meteo.be/wd
sint-katelijne-waver-meteo.be/Weather34
sint-katelijne-waver-meteo.be/Cumulus

That is OK, should give no problems. Cutils generates an index.html file and if other websites use different index files you could mix those websites. For clarity I advise to give CUtils its own subdirectory.

Nice collection overview btw

I still need to play with maps ....

Yes, you are on the map but it is unclear to me why it does not show the URL as the one in the inifiles you supplied seems definitely correct.
Or did you remove the URL? Out of the many sites you maintain, it would be logic to add the URL to the CUtils site of course
I would advise to include the index file in the URL so:

Code: Select all

https://sint-katelijne-waver-meteo.be/<subdirectory>/index.html

had to upload the noaa reports manually in the ini its set to ./reports/ also in CMX upload to wwwroot/reports (here it might be also ./reports ?)

That probably is a CMX thing: the upload of the NOAA report is done by CMX, CUtils only reads them. You will have to configure the reports directory in the cumulusutils.ini with a similar location issue as the realtime files (see below) as you work with one instance to handle multiple sites so there is only one reports location.

One question please: i'm using also the cumulus multilingual site; can I use both (utils and ml?), because in the CMX remote ftp upload folder one or the other becomes offline (wwwroot/ or wwwroot/Cumulus/)

I noticed that. Apparently you have all your sites - see above - in a subdirectory. I would advise to put the CumulusUtils site in the subdirectory utils (for the following, let's assume this one) or CUtils. To do so, you will of course have to create that directory first and change your upload destination. That should be no issue.

The main thing here is the question where you have your realtime files and the datafiles (the JSON's). I don't know if you can configure the realtime files in the cumulus multilingual site but if not, you can in CUtils:

1. Determine where you want to have your realtime files and configure CMX to upload your files there
2. Work out the relative location of those files to the utils directory (note this is HTTP relative, not FTP so the wwwroot becomes irrelevant). if you use the cumulus directory and you want to access that from the utils directory then the relative location becomes ../cumulus
3. You set the parameter: CumulusRealTimeLocation=../cumulus
4. Rerun:

   Code: Select all

   utils\bin\cumulusutils.exe website

That should do the trick

thx again, great help!!!

You're welcome.
In addition I have the following remarks:

1. Your normal values for temperature seem much too high. For rain that is difficult to see.
2. You need to add the following command (or variations thereof) to your scheduler on an interval which you use for ftp (e.g. 10 or 15 minutes):

   Code: Select all

   utils\bin\cumulusutils.exe SysInfo UserAskedData UserReports

3. Your record for minimum pressure is 0, that can't be. What may help here is to set the start of data at 01/01/2020
4. Start editing your menu
5. The ExtraSensors still have no charts, the file extrasensorscharts.txt is not present. That means something goes wrong in the ExtraSensors. Please run again and supply the logfile.
6. You need to define your channel(s) for the Davis WLL in the parameter Tx= which needs a string of comma separated integers representing the channels you use.
7. You may have noticed I renamed the thread: it covers the content better and may help other new users to find this thread more easily.
8. Etc...

[HansR]

Still in the flow or stuck somewhere