I’ve been recently investigating a poorly maintained box (granted, it was just a workstation used for surfing the web) which was behaving oddly. In fact you could not longer login via SSH. I quickly noticed that there was a non-working sshd on it, in /usr/local/bin. So I thought - why would a badly maintained box have a non-standard SSH on it?

This made me very suspicious, so I immedeately ran chkrootkit. It didn’t find anything except it told me that someone had tampered with the wtmp file. The “tampering dates” aligned with the creation times of the ssh. A quick “find” run came up with the tool to do so, too.

A quick check in the logfiles - which were fine, since they were not standard syslog, probably - showed that a SSH scanner had managed to login into an unprivileged account with a weak password, then used a kernel exploit from january to gain root privileges. Apparently he was unsatisfied with the sshd on the box, so he tried to put a different version on it, which, well, didn’t work and he could no longer logon to the box himself.

A rootkit was apparently not yet installed (verified after a clean boot), he was just about to setup his own sshd… maybe he had noticed that the box was just a stupid surfing box and then didn’t care enough to cover his tracks (or just shoot himself in the foot by breaking the sshd)

So if you are running boxes on the internet:

  • Make sure you don’t allow shell logins with weak passwords
  • Have your system and kernel up to date (note: you need to reboot for activating a new kernel…)
  • Don’t run an sshd unless you really need to
  • Maybe setup a recent-match filter to stop ssh scanners
  • Scan your log files for unusual entries such as logins by users who shouldn’t really exist in the first place… Use logcheck.
  • Don’t allow PHP scripts to run binaries on your system …
  • Or better, don’t allow PHP at all

(PHP guys: yes, I know that there are some good PHP scripts. But there are tons of badly written ones, too… and PHP is a major intrusion vector, and the privilege escalation I’ve seen here would have worked just fine with a PHP installation not using safe mode and safe_mode_exec_dir)

A firewall I maintain at our university has been tracking ssh connections using the recent match for quite some time, and a nice side effect is that it reduces all their “spam” in your logs, too (in case you bother to read them. Do at least read the logcheck results!)