tl;dr – Either a plesk update or CentOS update borked an OpenSSHd install such that the sshd_config was pointing to an sftp-server that didn’t exist where it was looking for it, but existed in two other locations.
Fix was to update the sshd_config while investigating the possibility of removing the duplicate install.Plesk updates are strange. The upgrade process broke on a 1and1 VPS, so I had to use the manual update from the Plesk site. After the update, I then updated the CentOS install. SSH and other stuff seemed to work fine, so I called it a day and logged off.
Now, something happened that an openssh update seems to have been installed in a chroot – either Plesk updated the OpenSSH install as part of the upgrade, and it did the upgrade in a chroot, or something happened with the VPS update. But either way, I ended up with 2 copies of /var/www/vhosts/chroot/usr/libexec/openssh/sftp-server and /usr/libexec/openssh/sftp-server, and a non-functional sshd_config which was looking for /usr/libexec/sftp-server.
The first sign of trouble was FileZilla refusing to connect:
Status: Connected to 2kinds.com Error: Connection closed by server with exitcode 127 Error: Could not connect to server
But OpenSSH was up and running – I could login fine. Second symptom was sftp-server not running. locate sftp-server pointed at two locations, so I looked at /etc/ssh/sshd_config to see what it was using, and updated it to point at the new location.
The investigation as to why sshd_config was pointing at the wrong location was pretty straight-forward – there was an sshd_config.rpmnew wiith a correct location, so it looked like an ssh update was installed and updated the location of the sftp-server binary.
The question as to why there are two sftp-server binaries was a bit more strange – there seemed to be a chroot in the webserver folders. The timestamp of the two binaries was the same, so it looked like both had been updated at the same time, implying the same update. However, the chroot was incomplete – it only had a barebones directory structure, only containing sftp-server and a few other files. This is why I believe it was installed by a Plesk update. It’s the only reason I can think of that would have a chroot created in the webserver root.
Of course, no proof that it was a malicious attack, it might be an internal Plesk thing, and I wasn’t asked to fix the duplicates, so I left it there. SFTP works now though, which was the entire point of fixing it.
#1 by Henry on September 26, 2013 - 10:49 am
The same thing happened to me, still trying to figure out if sever was hacked, or was a Plesk update thing?
The fixe I used was putting a sym link in each chrooted dir
cd /usr/libexec
ln -s openssh/sftp-server .
I also did the same in the /var/www/vhosts/chroot/usr/libexec so any future sites added would work.
Hope this helps!
#2 by AEL on October 15, 2013 - 11:31 pm
Hey thanks for doing this post, it helped me fix the problem on my system too.