UBB.threads PHP Forum Software Community Forums General Off Topic Hacked server - anyone any suggestions?

Hi,

Sorry I have not been around recently - not a good time

Anyway to add to my problems, my server has been hacked.

I have a mgmt company - who do not seem able to fully assist So thought I would ask here for any suggestions on sources of help etc.

Essentially the following has happened....

1) We were attacked on one clients site, which had an out of date script. I patched the site, and got rid of all the crap. That site seems fine now. No further evidence of problems.

2) We keep getting files written to /tmp - we get some dos-xxx.xxx.xxx.xxx where xxx is an ip - we get 3 or 4 of these a day. I am not sure if this is an issue - all owned by apache.

3) We keep getting txt files written to /tmp - these are a problem, as each one contains a script. I delete them, when I see them. We usually get 1 or 2 a day - usually late evening UK time. Again all owned by apache.

4) I have found some old hacking files - for example in the sessions folder for threads - these I have deleted. I am guessing these were from ubbthreads 6.x, but can't remember the dates now. I found this whilst looking for files over 1000K in size.

I can find no other evidence of anything else, yet these files keep appearing within /tmp

I pay my mgmt company each month - and whilst they will respond 24/7, I am not very confident on this recent issue (usually they are very good and helpful). All the issues I have tracked down after many many hours of searching (I have not slept properly for the last few nights). And all they can now say is 'From looking into some of the abnormal activity I have found several root owned files. This being the case the only remaining solution is to restore your server from remotlely stored backups.'

But I have looked at bash.history and I can't see any evidence of them searching or deleting any problematic files - in fact all the problematic files to date, I have found and deleted. So I am not at all confident.

I am at a total loss, and do not know what to do.

Over the last couple of weeks, I have considered several options - some of which are drastic, but none of which are really going to solve anything or enhance my life.

I know that I have friends here - none of whom I have met over the years - and so I am coming on my knees asking for some guidance as to what I should do. I can't take much more

Thanks for listening.

I find most of these management companies suck, don't do anything more than what you can hire someone "one time" to do...

Cleanup after a hack is quite tricky; in all honesty it can take a lot of time to cleanup...

There are a few things you can do, for starters, monitor "top" (it's a command that shows in real time what commands are being run) for anything suspicious, especially if owned by the apache user.

You can also check cron to ensure nothing "out of the ordinary" is set to run.

Now, the fun part, finding out what script has been installed that people can regain access; usually it's something mudayne and just sits around waiting for them to call it; and it can be anywhere web-accessable (likely within the web-home of the site which got hacked).

Lot of places to look, lots to check... I used to do security sweeps nightly when I worked for a webhost, there can be quite a few common locations to stash things, and a quite a few things that get installd (and quite a few locations to install them in!).

If you have slocate set to update daily (I myself do, I find it an excellent way to keep files in the db updated and ready when I want to be a paranoid freak) and do this: locate /...

Why 3 dots? Well, these dolts want you to think "oh, if .. brings me down a directory, and . brings me to the same directory i'm in, the ... isn't out of the ordinary"; likely you'll end up finding a bnc or five installed on the server, eggdrop bot's, pretty much anything your datacenter doesn't allow ...

A lot of differant groups target differant scripts, so that may also be a decent start...

</end 5.20a ramble>

Can't help you with the server thing, Ian .. but just wanted to tell you that I'm sorry you're having such a difficult time just lately.

Chin up .. things DO get better, although probably hard to see right now. ((hugs))

Thanks - already tried the ... - all it brought up was a few jpgs uploaded to the forums that have additonal dots after them. (oh and isn't it just 'locate ...' ? - no /

The site that was hacked in clean - nothing suspicious in there, as afar as I can tell. Checked dates etc.

Also checked the crons - again nothing suspicious under root, other users such as apache, admin etc., have nothing set up.

Basically I have checked everything, I can think of and so am in need of a spark of wisdom from somewhere LOL

With a bit of investigating, you can probably find the culprit. You say that there are tmp files being written all own by apache. That means it's a web script that's probably been uploaded somewhere. Soo, what you need to do is write down the timestamps that the files in the tmp/ directory have.

Once you have those, you need to look through all of your webserver access logs, for all domains. Scan through them and look for the dates that you have written down. It takes some time for sure, especially if you have a lot of domains. More than likely it's going to be a POST log entry, but could be a GET. Eventually you'll find an entry with the same timestamp, maybe a second or two off depending on how long it takes to write the tmp file, that looks a bit odd.

When you find the entry, you'll know where the script is at.

/... will specifically look for directories beginning with ...

Sorry i cant help either, but i'm watching with interest.

Chin up fella