Yes, I’ve been hacked, and it wasn’t fun! In this post I will go through some of the lessons learned. But before that, let’s shed some light on what happened.
It began when a friend of mine notified me that my DotNetArabi blog, which is WordPress blog, has new suspicious and unrelated posts. I rushed to my admin page, deleted these posts, and then changed my password to a stronger one.
I wasn’t so much afraid of the impact; after all this is an Arabic podcast blog while the posts were English. In addition to that, most likely the audience who saw these posts are few (since the posts were recent), and those who saw it would excuse me and understand that something went wrong (I like my audience :P).
After deleting these posts I also thought maybe I should check my folders and files, and indeed when I did, I found that there are hundreds and hundreds of files that aren’t part of WordPress files, most of them created in a single day. Deleting these wasn’t as easy as deleting the posts though; they were many files, they were in different folders, I didn’t know all the WordPress files to distinguish them from these files, my host provider does not provide file management system, and the files didn’t have much in common to find a single rule to delete them by (maybe the date was a good indicative, but wasn’t good enough).
Fair enough, since the harm is quarantined for now (or so I thought!), I decided to take this task on ease by deleting these files in bunches, this decision was also influenced by the fact that FileZilla kept disconnecting; I couldn’t just select many suspicious files and delete them.
Days pass by and I receive an email from my host provider informing me that I have been a victim to a hack; the email listed couple of files as a sample of many files (_the_ files) that are sending spam to others. I already knew about the files, but I didn’t know about the “sending spam” part, of course I should have known better; why would these files exist in the first place?! Duh!
Anyway, my host provider urged me to take action but he didn’t mention any thing about taking measures if I don’t, so I kept doing what I was doing: deleting files on ease, even though that I have received probably another same email or two from my host provider.
A week or so after, my Google Analytics numbers flattened to 0! being lazy (actually I was in the middle of moving houses so I shouldn’t bash myself here :P) I didn’t check what the reason was; I thought I can check it in couple of days, maybe it was the mobile app I am using to read my analytics rather than the analytics themselves.
And then a different email reaches my inbox: “your website have been suspended for the last 3 days because it’s been a source of spam”! This is when I freaked out; it’s true that I don’t make money of the hits to my blog, but being down for that long is bad bad bad for reputation.
I instantly sent them an email explaining to them how angry I was because of their inadequate notification/action protocol; their initial notifications didn’t mention any threat of closing down the website, and their last notification of closing down the website came 4 days after they have closed it down!
I demanded them to put it up again ASAP, but I also promised to remove the malicious files. They refused! No go live again before we delete all the files.
Being under the pressure, I had to try all sorts of stuff, to the extent that I tried the Windows Explorer’s built-in FTP client, and to my surprise, it worked better than FileZilla! I was happy seeing that green progress bar deleting all these awful files. After I made sure I have deleted everything that looked suspicious to me, I sent the host provider an email again informing them that everything is fine now and my website is ready to go up again (yes, they don’t have chat-support, only email).
Hours and hours later, I receive an email from them again saying that I still have malicious files and “Here is a sample”, the website will not be up until this is solved. This time, though, they provided me with two options: either deleting the whole website and uploading from a backup I have (which is potentially infected as well), or pay for a service on hourly basis to fix the problem for me.
I decided to go with the first option first, but rather than deleting the whole website, I asked them to delete the suspicious folder only. Hours and hours after we managed to do this, and finally my website is up again (I went through more problems after that but maybe we can save this for the list of lessons below).
Not a short story looking at the narration above, now let’s look into the lessons learned and how I can relate things together.
You have a website? You are already a target
Security hasn’t been something I neglected, but it was something that I miscalculated; the hacked part of my website was my podcast DotNetArabi’s blog, and my thinking has always been “Why would someone hack my podcast blog? My audience is very specific; it does not host any sensitive information, the ROI of hacking it is little compared to other sites…, so the possibility of being a victim of hacking is very minimal.
But they weren’t after my website, the content, or my audience; they were after the resources on which my website runs on! My website became a platform to annoy others. I agree, I should’ve known better, but the comfort of not doing a lot to secure my website along with the “low possibility” of being a target made me feel good about not securing my website!
Do you have a website that you manage? GO SECURE IT NOW!! Do all what is necessary to secure it, if it is a WordPress blog check the points below, if not look how to secure it. YOU ARE A TARGET…RUN… NOW!
Don’t be Lazy
One of the reasons why I ended up in a bad situation is that I was a little lazy; I know I was moving houses and was too busy, but I also knew about having the malicious files before, and I took it easy, tsk tsk tsk Emad, bad!
Windows Explorer’s FTP client VS FileZilla
For a long time I looked down to Windows Explorer’s FTP client, especially if compared to products that have been in the market for a long time like FileZilla. To my surprise, for the specific task of deleting files, WE’s FTP client out-performed FileZilla; no disconnections at all. If deleting files wasn’t so difficult task due to the bad tool, I might have been in a better position.
Don’t put all your eggs in one basket
I have one site account with my host in which I put 3 websites; the resources these websites need were really minimal so I just created sub folders and created a web app in each folder: one for my personal blog emadashi.com, one for my DotNetArabi podcast, and a blog for the same podcast. This was made possible by some URL Rewriting tricks.
The plague didn’t hit all of them, it only hit the blog of the podcast, but when the host decided to take the website down it took them all simply because to my host it’s a single website.
Regardless of my host’s decision to take the website down, there are so many things that can go wrong to a website which might affect all the subsites. Separation is good in this case.
Manage your backups
Like I said, I had 3 websites with 3 folders, and so I didn’t manage the backup by the entirety of the website, instead I managed the backups separately. Makes sense? Well, I also had a web.config in the root in which I laid the URL rewriting rules, without which the internal links to my blog posts will be broken (shout out to Maher for his help and notifications). And you guessed right my dear reader, I didn’t backup this one up, in fact I did back it up, but by mere coincidence! *slaps self’s hand*. So make sure you backup your website entirely.
Also, I thought I knew where my backups were, I was wrong! I was disappointed that I had to look for my backups! Are they in the external drive? Are they on my personal computer? Are they in my personal VM on my work computer?
Your host’s influence
This is very important; let’s see:
- Communication: It was good of my host to notify my of the hack, but also they didn’t give me a clear message on what I should specifically do, and the potential outcomes if I didn’t. Instead of sending me sample files of those malicious files, they could have sent me a list of all the malicious files, saving me (and them) the time and effort to look these up. I can hear you say that this is not their problem, but considering the wasted effort and time they had to give away by the back and forth communication, and spam inflicting their servers …due to all that I reckon it was better if they had just sent me the list of all files.
Also, they didn’t make it clear that they will shut me down if I don’t delete these files on timely manner, if they did I would have been more active and keen to delete them. My impression was that the effect of these files was minimal.
- Response Time: my host does not provide chat support, only email; this meant long latency before we could cooperate and solve the problem. Especially the notification of putting my website down after 3 days.
- To their credit, in their last email after the problem was solved, they suggested couple of points on how to secure a WordPress blog; nothing fancy or detailed, but it was good of them, I guess.
Use scan service?
I deliberately put a question mark at the end of this title; I am not sure how good such services are, my host advised me to use sitelock, but don’t consider this as an advice as I haven’t tried it yet; I just think it’s worth mentioning here.
There are numerous content on the web talking about securing WordPress blog, here is one. But without being too sophisticated, this most important things to do:
- Make sure that the engine is up to date
- Make sure the plugins are up to date
- Make sure you use a strong password
- FTP access: to be able to upload media content to your blog you might need to provide an FTP access (if the installation didn’t do that). If you are hosting your WordPress on Linux, DO NOT GIVE 777 permission!
It was all about me belittling the possibility of being hacked! So let me ask this again: do you have a website? You are already a target, don’t be lazy and go secure it NOW!
Sorry to hear that my friend. I am glad that the hack did not cause any permanent damage, and that you were able to recover and to contain the infection.
Great points and lessons ( maybe not the one about Windows Explorer) I agree with all of them. It astonishes me how we always undermine the importance of security. “Who would put effort to hack my website?”, “There’s nothing in my computer that I need to worry about.”, “I don’t want to have 16 characters password. 123abc is more than enough”, “Let me write the wireless password on the router lest I forget it”. We always realize how serious this is when it is too late. So, thanks for posting this!