Cannot change or move any files using WinSCP

[KenKenyon]

Hello,
My first post here.

Rather than waste people's time asking hundreds of questions I've spent about 10 days trying to learn about Raspberry Pi, Linux, Putty, WinSCP etc. etc. I've found the Wikis and the forum absolutely excellent and a gem of a resource. Generally with the help of previous posters I've made better progress than I expected.

Basically my probelm is that I downloaded the Raspberry Pi image and managed to set it up using the Wiki advice and at first all was running pretty well to my complete astonishment. I started with a Pi Zero W and had a lot of problems with Mono but got through and everything seemed to be working. After 1 successful creation of NOAA monthly reports the second day was corrupt and no further updates occurred. I read Mark talking about MXDiags so had a look in there and there was a clear error in Jan24log.txt. I again followed advice I read from Mark to try and correct a single line in the Jan24log.txt using WinSCP. I failed completely with "permission denied Error code 3".

With help from my programmer son in London and Chatgpt (I'm in Stockport, UK) I tried numerous methods to move from rw permission being limited to the owner (root) and group and others having only r permission. I've tried to change the permissions in WinSCP and Putty using chmod, chown, nano and numerous other things I don't really understand. Always the result was that I had no permission or access was denied.

In the end out of total frustration I bought a Raspberry Pi 4 and started again with the 64 bit image. After adding an SSH.txt file (which caught me out for a good while but the Wikis again came to the rescue) I've ended up in exactly the same position with no access to any files using WinSCP.

I've used Cumulus 1 since 2012 with an old, cheap Fine Offset station but that gave up the ghost a couple of months ago and I'm currently using An ecowitt Wittboy GW 2001 which seems excellent. I've got cumulusmx windows version working perfectly but with no console/logger I decided I couldn't afford the electricity and went down the Raspberry Pi route.

Any help would be most gratefully received as I'm sure you can imagine

[freddie]

Sounds to me like you're running CumulusMX as the root user, so any files that it creates are owned by the root user. So even if you change permissions or ownership any new files created will be owned by root and will not have global permissions.

To change permissions on existing files you will need to prefix any of the commands you use with sudo. So instead of typing

chmod 666 name-of-file

you should use

sudo chmod 666 name-of-file

Ultimately you will need to do two things:

Change ownership of all CumulusMX files to the user you use to connect to the pi with winscp. You will need to use the following command from the directory in which your 'CumulusMX' directory is located - typically under /opt or /home/pi:

sudo chown -R pi:pi CumulusMX

Then you will need to be careful to only start the application when you are user pi.

If you start CumulusMX as a service then you will need to edit the service config file, which can be found under /etc/systems/system, and it will be called cumulusmx.service. The file should be edited so that the 'pi' user is used to run the program. Note you will need to open your edit using sudo again.

If this is all double Dutch to you then let me know and I will write you a script that will do it all for you.

[KenKenyon]

Hello Freddie and many thanks for your very quick reply which is most helpful.

I don't really want to put you to any trouble so will speak to my son over the next couple of days and ask if he thinks I'll be able to follow your instructions. From recent experience I've found that I invariably hit unexpected problems which then take me hours to resolve (that's if I do resolve them). I'm therefore thinking that your kind offer of a script might be a life saver but I don't want to ask you to do this just yet. But if you don't mind I may well take you up on the offer in a few days.

On a different note I wondered if you have any idea how I got into this mess in the first place and whether reformatting the SD card and starting again might help. On the basis that I've reformatted and started again several times already I'm quite happy with that. It just worries me though, that unless I change something in the procedure, I'll inevitably end up with the same ownership problem again.

Thanks again for your help.
Ken

[Cortmalaw]

Freddie is on the right lines, but an important thing is how you setup your system. You said "downloaded the Raspberry Pi image", and I'm guessing you mean the CumulusMX Pi image, rather than an ordinary Pi image then adding CumulusMX for yourself.

That's important, because the "CumulusMX Pi image" puts the program files in /opt/CumulusMX , and runs the program (automatically) as 'root' user. The permissions in /opt/CumulusMX are set accordingly.

But WinSCP cannot connect as 'root' (at least, I cannot make it do so, even if I change the password etc). So it cannot change files in /opt/CumulusMX unless you change something.

What you can do is change all the files in /opt/CumulusMX so that ANYONE can read or write them (which will then include you when you log in as user 'cmx' through WinSCP).

The command to do this (eg, using Putty) is simply:

Code: Select all

sudo chmod -R 777 /opt/CumulusMX

That is, as superuser, change access permissions (recursively) to allow root or any user to Read, Write or eXecute any file in /opt/CumulusMX or any subdirectory thereof. Then you should be able to use WinSCP as you like.

[freddie]

@Cortmalaw two issues with your approach:
1. Security risk
2. You have to keep repeating the procedure each time MX creates a new file

Far better to change the ownership of the installed files and change cumulusmx.service to run the application as that same (normal unprivileged) user.

P.S: loved your "on the right lines" comment - I do this for a living

[HansR]

And it might be a good idea to set up the CMX image for pi and not root (i.e. ownership and groups and also the daemon running as pi).

[freddie]

Very true, so it's all correct from the moment you download. I think Dazza looks after the image.

[KenKenyon]

Many thanks to all for your replies and help. I was about to reformat and reinstall the image this morning so at least I now know that would be a waste of time. Cortmalaw is right that I downloaded the CumulusMX image not the Rasperry Pi image - I meant to say the CumulusMX image for the Rasperry Pi but didn't make that clear. He is also right that all the files are in /opt/CumulusMX/.

At first I thought the solution from Cortmalaw seemed nice and easy but now realise from what Freddie says it probably isn't the best solution.

I also think that HansR makes a good point as clearly I will not be the only "beginner" affected by this. What surprised me a lot was that despite searching the forum and internet more widely looking for people with my exact same problem I didn't really turn anything up. This made me assume it was something I must have done wrong.

If you could provide me with the script as you offered Freddie I'd be most grateful. I've been in touch with my son and he thought that was the option I should go for and says applying the script (which I wouldn't know how to do) is straightforward and he will help me with that.

Thanks again to all.
Ken

ps One last thought. I don't know how much work is involved in writing a script Freddie and if it's excessive I suppose the only other option open to me is to reformat, install the Raspberry Pi OS and then download and install CumulusMX. I presume that would work but I have to say bearing in mind the problems I've had so far I fear that it may be beyond my pay grade! That does also seem a bit OTT bearing in mind that the CumulusMX image has been produced to open up the Rasperry Pi world to people like me.

[Cortmalaw]

I agree with freddie and HansR - I was trying to give a easy solution to the immediate need, for the existing pre-built image.

[freddie]

Hi Ken,

Rather than run a script, as it is only a few commands to run you may as well do it manually. Copy and paste the following command into your terminal window (you will need to be logged on to the Pi):

Code: Select all

sudo systemctl stop cumulusmx.service

This stops CumulusMX. Now you can safely run the other commands to make the changes. Copy and paste this:

Code: Select all

sudo chown -R pi:pi /opt/CumulusMX

All the changes to ownership should be done, so you now need to edit the service config file to ensure CumulusMX runs as the correct user from now on. The next command will open an editor to allow you to make this change:

Code: Select all

sudo nano /etc/systemd/system/cumulusmx.service

Look for the following lines:

Code: Select all

User=root
Group=root

Change them to:

Code: Select all

User=pi
Group=pi

Save the file and exit the editor. You do this by holding down the "Ctrl" key and then pressing "x".

Finally restart your pi with the following command:

Code: Select all

reboot

Et voila - that should be you sorted.

If you see any error messages as you work through the commands, it is best to stop and copy the error detail into a reply to this topic and we will help you out.

[KenKenyon]

Thanks Freddie

My job for today! I'll report back whatever the outcome.
Ken

[KenKenyon]

Hi Freddie

I've just tried your suggestion but have fallen at the first hurdle I'm afraid! The first command after stopping the cumulusmx service came up with chown:invalid user: 'pi:pi'

Am I right in thinking that I should be logged into Putty as cmx when I start this process (that's what I've done)? I did also try logging in as pi but for some reason my password was declined
Ken
ps I did copy the error from the Putty terminal window but for some reason couldn't paste it into this post. Is there a trick to that or is it not allowed?

[freddie]

Hi Ken,

I'm sorry, I thought the user was called pi. You should use the username you logged on to the pi with. So if the user is cmx then the first command will be:

Code: Select all

sudo chown -R cmx:cmx /opt/CumulusMX

Where I asked you to change user and group in the cumulusmx.service file that bit should now be:

Code: Select all

User=cmx
Group=cmx

If you log on as something else then you should change the above accordingly.

Let me know how it goes.

[freddie]

Am I right in thinking that I should be logged into Putty as cmx when I start this process (that's what I've done)? I did also try logging in as pi but for some reason my password was declined
Ken
ps I did copy the error from the Putty terminal window but for some reason couldn't paste it into this post. Is there a trick to that or is it not allowed?

Yes, you should be logging in as whatever user you normally use.
No trick to copying into a post reply - it normally just works. All my command line instructions were copied and pasted in to my posts.

[KenKenyon]

Et voila!

It's worked Freddie. You won't believe the relief I'm feeling. I've spent so many hours on this trying to avoid being a pain to others but following your brilliant instructions sorted it out in 10 minutes. I now have the required ownership and permissions and managed to correct the error I found in MXdiags. I'm hoping that the NOAA reports will now start populating again.

Just one last thing if I may. I have a bit of a problem with rain no longer recording from the ecowitt piezo sensor in the pi zero (it does in cumulusmx in both my new pi 4 and in windows. I did pick up a thread in the forum on this a week or so ago but didn't pursue it as I had this current problem on my plate already. If I fail to sort that out am I right in thinking I should post a new message under that subject heading?

Many thanks again Freddie. One last point I've just thought of. You're clearly an old (perhaps I should say experienced!) hand at cumulusmx and will, I'm sure be able to answer this question.. I know it's open source software and that donations will therefore be apprecieated. I think I probably made a donation back in 2012 when this journey started but I think a further donation is now due, especially in view of the support I've just had. Can you point me in the right direction for making one?

Ken