CMXv4 Memory usage

[broadstairs]

My Linux system has 16gb memory also and I'm guessing a memory leak would take a loooong time

Stuart

[PaulMy]

Mark wrote:
It's for all OS.

Just rechecking Task Manager and my Windows 10 system (also 16 gb) mostly still the same as before, from 30 to 60 mb for the three CMXv3 and one CMXv4 installs, except one CMXv4 is now at about 340 Mb and was at 548 Mb for a bit. This v4 is using Davis Weatherlink Console for data source. Checking all these with Dashboard > Charts > Select-A

Code: Select all

Mark wrote:
Did anyone try the runtime config file I posted above? Any change?

This is what I get in that CMXv4 with the high memory

Code: Select all

{
  "runtimeOptions": {
    "tfm": "net8.0",
    "framework": {
      "name": "Microsoft.NETCore.App",
      "version": "8.0.0"
    },
    "configProperties": {
      "System.GC.gen0size": 5242880,
      "System.GC.Server": true,
      "System.GC.Concurrent": true,
      "System.GC.HeapHardLimit": 209715200,
      "System.GC.ConserveMemory": 5,
      "System.Threading.ThreadPool.MinThreads": "10",
      "System.Threading.ThreadPool.MaxThreads": "30",
      "System.Runtime.Serialization.EnableUnsafeBinaryFormatterSerialization": false
    }
  }
}

Enjoy,
Paul

[Dador]

I can confidently say that I have improved significantly. Memory consumption, during normal operation and cyclical sending of files to the server, currently ranges from 50-130MB.
Now the RAM consumption is only 6x what CMXv3 consumes (~20MB), but fortunately not 22x as before. A really big plus for me.

I have found that last config file gives out of memory errors when you request a lot of data from the select-a-period graphs

I checked it. The computer gets out of breath at very long times, but RAM consumption does not exceed 200MB.

[ConligWX]

It's for all OS.

I have found that last config file gives out of memory errors when you request a lot of data from the select-a-period graphs

Third version will be incoming soon....

so far not noticed any increase in memory usage even though I loaded the select-charts heavily.

here is the difference from htop at the start and then 3 hours later:

Screenshot 2024-04-09 181132.png

Screenshot 2024-04-09 213727.png

as you see memory is less than what it was.


anyone wanting an overview of htop please checkout the video here: https://www.youtube.com/watch?v=bKWZJ3_5ODc

[weatherbee]

I'm running Windows 11. After installing CumulusMX.runtimeconfig.json my usage has gone from 278 MB to 87 MB.
Using ver.4.0.0 build 4019.
Tom

[AndyKF650]

I have monitored HTOP memory use for a short while and the results are as follows

Uptime 03:000:00 mem use 506M/3.70G
Uptime 05:34:42 mem use 512M/3.70G
Uptime 18:36:45 mem use 519M/3.70G

Steadily increasing but at that rate my ram will take a very long time to flood the available space.

[Dador]

I can write that installing .NET 8.0.4, in my case, did not change anything when it comes to memory consumption. What it was like (avg ~120MB) is still the same.

[PaulMy]

This morning:
CMXv3 with WLL 63.4 mb
CMXv3 with Wifilogger 55.9 mb
CMXv3 with Davis Weatherlink Cloud 40.4 mb
CMXv4 with WLL 46.5 mb
CMXv4 with Davis Weatherlink Cloud 36.0 mb
These all do fluctuate but usually no more than 5% higher or lower
There is little difference if the local dashboard is changed to any of the other tabs - i.e. Today/Yesterday, Reports, and not much change in any of them during the normal 5-minute interval processing and uploads.
Looks good here.

Enjoy,
Paul

[Dador]

CMXv3 with WLL 63.4 mb
CMXv3 with Wifilogger 55.9 mb
CMXv3 with Davis Weatherlink Cloud 40.4 mb
CMXv4 with WLL 46.5 mb
CMXv4 with Davis Weatherlink Cloud 36.0 mb

Nice

I am very curious how Davis+WiFiLogger will work with CMXv4 in my case.
I'll find out when CMXv4 comes out of Beta.

[broadstairs]

Mark I am curious to know if your zip file attached in this thread was included in the 4020 full download. My reason for asking is that my CMX V4 is running varying between 650mb and 910mb pretty much constantly and was at or near this level right after a restart. OK so I have 15gb memory so memory per se is not an issue but just curious as this is somewhat more than I remember in V3 and although memory use seems high it shows no classic signs of memory leak. I have the latest Linux dotnet version installed 8.0.4.

Stuart

[AndyKF650]

Hi there

To add to this conversation, I have been monitoring the system memory on my RPi v4 which runs CMX V4.0.0 with dotnet 8.0.4. CMX is the only software running permanently on the RPI

The memory test is using "free -h" via a crontab script every 20 mins, this shows the Total Memory, Used Memory, Free Memory, Shared Memory, Buffer cache, and Available memory.

The results for the past few weeks is attached and shows that the total used memory is largely stable in the period between 695Mb to 815Mb, rebooting the RPi from time to time reduces the used memory a bit but as you can see the free memory is 1.3Gb until I started writing this post when it suddenly peaked out.

[philpugh]

Her's my htop for RPi 5 8Gb 64bit Debian 12 running three instances of CumulusMX 4b4019. Not seen any memory problems wit this setup. System was 'rebooted' this am as we had a power cut!

[mcrossley]

I have been doing some more testing of this and the next build will use a different config, plus a forced memory clean-up every 10 minutes - though that does not seem to do very much.

[broadstairs]

That's interesting you have a lot less unique PID tasks running, I have 23 unique PIDs!

Stuart

[mcrossley]

Do you both use PHP upload? That creates a fair-sized thread pool. This is mine...
.

Screenshot 2024-04-26 123730.png