A Hack By Any Other Name — Part 1
Posted on 21 February 2014 by Bob Lacatena
Joshua (computer program): Shall we play a game?
— Wargames (1983) —
Introduction
Our society romanticizes hackers. Books and movies invariably present them as the good guys, the nerdy heroes, or at worst pit a good hacker against an evil hacker. There’s something intriguing about that lone individual, armed with brains and an arcane, almost magical power over the preeminent technology of our day, granted him (or her) by the massive, interconnected and insanely complex world of global computing. It’s that heroic ability, when used wisely, to take down huge, nefarious government agencies, corporations, or anyone who isn’t considered “the little guy.”
This is the two year anniversary of the first of the days that the Skeptical Science web site was seriously hacked, and while from a security standpoint my attitude has always been that the less people know about things, the better — safer — the site is, I think it’s important to also establish that there is an ongoing, active war against Skeptical Science.
Regular visitors may have noticed that the site was down for much of the day a few weeks back. That was a result of a concerted “denial of service” attack, an effort where individuals or bots attempt to overwhelm our servers in order to specifically bring the site to its knees. This is one more example of the Subterranean War that is being waged on climate science. Skeptical Science is simply a volunteer group, organized by one person, to try to counteract the persistent and easily debunked myths that are incessantly repeated and pushed, no matter how often they’ve been refuted, and no matter how directly contradictory those arguments are.
But because none of those arguments can really stick, because they are so easily debunked with facts, observations, and most importantly the peer-reviewed literature, “they” resort to attacks. Some attacks involve a state’s attorney general relentlessly hounding an established researcher for no good reason. Some involve possible libel and slander in the media against climate scientists. Some involve the invasion and theft of correspondence like the CRU hack. Some involve death threats, litigation, threats of litigation, the muzzling of scientists, the destruction of scientific material, and more.
Some are even so petty as to hack into a private, volunteer web site like Skeptical Science. Who knows what their intention actually was — to disrupt the site, alter data, plant an electronic time bomb? What they did do was to discover that the team behind Skeptical Science uses a private forum to discuss the science and climate science denial, but more importantly to review and discuss everything that is published at Skeptical Science, in an effort to make it as accurate as possible. That private forum was what the hacker or hackers ultimately found valuable, although it really was only of interest to cranks fond of conspiracy theories or just plain making fun of other people, because the only real content in the forum is honest, candid discussions about how best to present the actual climate science to people.
So how did the hack happen? Again, I am reluctant to share too many details, because every bit of information is a potential avenue for the next attack, and Skeptical Science does ward off frequent denial of service attacks, like the one previously mentioned, and SQL injection attacks. A SQL injection attack is a rather common hacking attempt to gain access to the web site’s database, although it is one often performed automatically and randomly by bots, in an effort to find entry into any site that might be worth hacking for profit. Many of these originate from China, Russia and Eastern Europe. It’s of passing interest that one SQL injection attack that was detected (and thwarted) actually happened during the period of the successful hack. A private forum thread discussing that hack may have motivated the successful hacker to change his tactics, as you’ll see below.
SID 6.7: I'm a fifty terrabyte, self-evolving, neural network, double backflip off the high platform. I'm not a swan dive.
— Virtuosity (1995) —
February 21, 2012 — 6:52 AM AEDT — The German
It was February 20, 8:52 PM CET, the local time in Germany, when The German, or so I’ll call him, first hacked his way into the Skeptical Science web site. If it had happened in America in the nineties, beside his keyboard would have been a can of Coca Cola and a few Twinkies. I guess today the drink would be a Red Bull. I’m not sure what a German might choose.
To mask his identity, he fired up a Tor browser. Tor, despite the titlecase spelling, is actually an acronym for The Onion Router.
The Onion Router was first conceived in 1996, initially funded by the U.S. Office of Naval Research, a department of the U.S. military, and later supplemented by DARPA, the U.S. Defense Advanced Research Projects Agency. The purpose of the project was to provide security and anonymity in Internet communications. The concept was basically that the secure network within the larger, unsecured network (the Internet) would include a number of cooperating nodes. When a message (an Internet browser request) was to be sent to a particular server, then an Onion Routing Proxy, a sort of master server, would select a random path through the available onion routers. The data would be encrypted multiple times, and each node in the path would decrypt one layer (like the layers of an onion), until the final, exit node had the real, completely unencrypted transmission.
In this way, it would be impossible to track a message, since its contents would “change” (due to the encryption/decryption) along the way, and the path the message would take would be completely random. Each node would only know about its neighbors, never the entire path, and never the contents of the message. The destination server that receives the message knows the contents, obviously, and the exit node from which it came, but that’s all. Since the onion router proxy randomly chooses the path, that exit node (for a session) can also change from time to time, so the server has a hard time keeping track of the client.
For a hacker, the advantages of this are obvious. A hacked server will never see his real IP address, and will in fact see a changing series of IP addresses, making it even harder to piece together his various prods and probes.
Safe in his room, buried under layers of onions, focused on his screen, probably with a caffeinated drink and a snack beside his keyboard on his desk, the hacker comfortably and earnestly began his work.
David Lightman: Hey, I don't believe that any system is totally secure.
— Wargames (1983) —
March 24, 2012 — 1:06 AM AEDT — SkS was hacked
More than a month passed, certainly not uneventfully, but without any knowledge on our part that we'd been hacked.
It was an unseasonably warm and sunny, late March, Friday morning outside of Boston, although it was considerably cooler than the record 82?F just two days earlier. That spring was a polar opposite (see what I just did there?) to the current New England winter. Temperatures were like midsummer, when one is usually pained by the last assaulting cold snaps of winter.
I was at my desk, at home, where I work. Since I'm self-employed, Fridays are a day when I can usually take my foot off the gas. I tend on Fridays to let my mind wander to various personal projects, if I'm not either behind or otherwise engrossed in my paying work.
As it turned out, I was annoyingly behind on my paying project and so mostly trying to get some real work done anyway, but I was distracted enough by approaching "weekend mode" to pop over to the private Skeptical Science contributor forum to see if there was anything of interest to read or quip about. I figured I could maybe even take a few minutes to peruse an upcoming blog post or two, to offer some criticisms. Nit-picking other people's work is one of the simplest and most satisfying of pass-times. Ask any pseudo-skeptic blogger.
The forum is used by Skeptical Science authors and contributors to discuss climate science, upcoming posts, site issues, and now and then just to blow off steam. One of the most critical roles is to provide a sort of “peer review” for upcoming posts, to check not only grammar but also scientific facts, balance and tone. It’s a critical part of producing the high quality articles that make Skeptical Science an increasingly valuable and reliable resource.
It’s also used to plan special projects, to discuss software improvements for the Skeptical Science web site, and any other communication among the site’s contributors. Last, but not least, it’s also used just to chat, whether about the latest science, the antics of certain “climate personalities,” or even the latest cricket matches. Cricket is a surprisingly popular topic.
One thread immediately grabbed my attention. I clicked it and read the following on March 23, 2012, about 10:30 AM, EST:
grypo: "It looks like it happened"
The title of the thread was "SkS was hacked."
That was the first that I and most others knew that the Skeptical Science web site had been hacked. We'd been talking about the possibility for a while within the forum. After all, Skeptical Science had already been hacked once in the past, CRU had been hacked, and just a month before this the Heartland Institute had embarrassingly succumbed to something much simpler, an ordinary phishing attack. I had frequently explained that most hacks worked just like phishing. They preyed on human failings (such as naively responding to a phishing request) rather than actual hacking. Most hacks rely on human, not computer, flaws.
Real hacking takes considerable knowledge, training, skill and experience if the target has been properly secured, and with all of the hacking that you've read about in the past decade, most sites must be pretty well secured, right? But that's where human nature comes in. People don't really wake up to their dangers until the danger is staring at them with hungry, angry eyes.
I quickly followed the link supplied by grypo to a comment on a backwater pseudo-skeptic blog announcing the hack. I hastened to read the hacker's comment, inwardly grimacing at the cartoonishly ridiculous assertions there, including a line with such comic irony that it could only have been written by Lewis Carroll:
I will consider stepping bravely forward if I get caught.
Huh? I proceeded to quickly download the first of the files so that I could look at them myself. I noted as I clicked the link that it was unsurprisingly being hosted on a server in Russia (now where have we seen that tactic before?), as evidenced by the .ru extension on the domain name.
Dr. Walter Gibbs: Won't that be grand? Computers and the programs will start thinking and the people will stop.
— Tron (1982) —
March 23, 2012 — 10:52 AM, EST — The Contents
I downloaded and unzipped the hack file, racing through the contents to get a high level idea of what was there. Right away I was confused. What I saw didn't make any sense. It looked a lot like the private forum I'd just been using, where grypo had announced the attack. The formatting was the same, from the colors to the column layouts.
The hacked files looked mostly like the actual forum. It certainly had real posts in it, posts as recent as a day or two old that I'd remembered reading myself. Yet it was wrong. The colors were right. The formatting was mostly right. But it had people's full, real names, and their IP addresses and e-mail addresses right there, embedded beside each and every comment.
The real forum didn't have that. In fact, I don't know of a single forum on the Internet that does that. Why would you? People use handles for a reason, and everyone recognizes them. IP addresses and e-mail addresses (and real names) are kept strictly private and confidential, even within small groups that otherwise know each other. After all, I know who Dikran Marsupial is because he told me, and if I don't know who grypo or Albatross is, that's because he's made that choice and we all respect it. I maintained my own anonymity, for my own reasons, for many years until very recently. Why would the hacker bother to do such a thing?
This was not quite the forum, but it clearly was. It was weird.
I looked through the files. There were other differences. The forum discussion threads were in a single folder named "forum," which was further, accurately broken into folders mirroring the different categories that existed within the real forum, and then within that every separate thread had its own HTML file listing all of comments in the thread, in order, named with each thread name.
But for a thread in the real forum, only 50 comments were listed on a page, while in this case every file contained one entire thread full of comments from start to finish, no matter how large. The page header had been altered, too. It was close, but not quite the same. And then there were the names, e-mails and IP addresses.
There was no way that someone just got into the forum, went into every thread, and saved the web pages. You couldn't generate the data released in the hack that way. Someone would have had to put a lot of work into editing the pages, to merge them all and to change the presentation. It would take a lot of work, too, to cross reference every user with their full name, e-mail and IP address, to insert those. It would also require access to data that wasn’t available unless you’d hacked into the database. You can’t look at other people’s personal profiles through the application.
At first I wondered if John Cook had a secret admin version of the pages that matched the downloaded files, although I wondered why in the world he'd waste his time programming such a useless thing. It seemed very unlikely.
Someone had gone to a lot of trouble to reorganize and rebuild the forum to their own liking.
Part 2 describes the earliest encounters, known and unknown, with the hack.
One way or the other, however, I had a problem that was going to suck a lot of time out of my life.
To be continued...
Looking forward to part two!
One thing I think you should stop doing is holding passwords in plain text. A better way is to use a one way encryption algorithm and to only store the encrypted password. That way you never know my password and so no one else can get it from you. If I forget my password you issue a new one and require me to change it.
[BL] Passwords are not and have never been stored in the database as clear text. They are and always have been encrypted, and they are never decrypted. Rather, the password sent by the user is encrypted, and that encrypted password is compared to the encrypted password stored for the user. If they match, then the password supplied by the user is valid.
[BL] Correction, I just looked at the code, and passwords are decrypted in the "Forgot your password" function -- but that doesn't represent much of a security hole, because it can't be used to breach the system, and it can only be used to steal passwords if you already have the password and so can change a user's e-mail, or otherwise have access to that person's e-mail.
Either way, that particular flaw doesn't represent a pressing issue, at least compared to the effort it would take to correct.
Passwords should always be stored as salted hashes:
https://crackstation.net/hashing-security.htm
Private forums on a vpn?
I too, am looking forward to part 2, of course. Reminds me of how Real Climate was hacked at one point. That was back at the beginning of Climategate.
Appropriately named, that. A break-in was used to illegally obtain documents intended to be used for the purpose of systematic smear campaigns. But the similarity breaks down soon after that.
In this case, the crooks got away. Then the smear campaign actually worked for a while, with the press working often as unwitting accomplices, putting the victims of the attack on trial rather than trying to uncover those who were behind the criminal act.
With the break-in at Real Climate, the perpetrators uploaded a zip of the files they thought they could best spin for their campaign to the Real Climate server. They were planning on making a post that would include a link to the file.
But at the time Gavin Schmidt was in the system. He noticed that someone had broken in, then soon realized they were still there. At that point he shut down the server. He said something about it a while back, I believe in an interview.
A great read Bob!
What's wrong with discussions about cricket? Particularly with the form Mitchell Johnson is showing lately :-)
Highly interesting - so long as it doesn't become thread creep!
Would it be possible to set up a server to deny requests from TOR-servers, if their IP address were known? (and would it be possible to map TOR servers by using it oneself?).
[BL] Yes to blocking Tor IP addresses, and many web sites do, but building the map is a huge task. Generally you have to pay to get a good Tor IP list, or you can get a less reliable and complete list for free. Just google it.
It's something we've discussed, but again, there's a lot to do and not many people available to do it, so it's a lower priority task.
1) It would be nice to know the IP that hit you and the time it hit you so we could check if it was really Tor here https://exonerator.torproject.org/
2)It's trivially easy to block Tor. The IPs are publicly available for free.
To be more complete:
http://torstatus.blutmagie.de/
or:
https://check.torproject.org/cgi-bin/TorBulkExitList.py?ip=198.41.222.255
after ip= you can put the IP of your server to get a list of IPs whose exit policies hit your server.
Bob@2
Thanks for the response. Of course you need to determine your own enhancement priorities. I only make the point that the fact that you can decrypt the passwords means that the method for doing so is somewhere in your code and in principle at least hackable. That's one reason why secure sites use one way encryption and only ever send out new, temporary passwords which have to be changed on first use. That's why I was surprised when I received my old forgotten password in plain text via Email.
[BL] The thing is, a site like this shouldn't need all that much security. All you can really do (now) by stealing someone's password is to post comments using their user ID... annoying, but it's not like stealing credit card info.
Of course (as you'll learn in future posts) at the time of the hack, that wasn't the case.
And I am all in favor of salting passwords -- and we have on the new forum -- but salting is protection against dictionary attacks, rainbow tables and other intricate password hacking schemes. Our DoS protections would pretty much also thwart a dictionary attack or brute-force attack.
If I had the time, and for any site that I set up from scratch, salts are easy and painless. Working with a site that's been in existence for 7 years, and has evolved considerably over that time, however, presents a much greater coding problem.
It is an interesting read so far, but protecting against SQL injection should be as simple as making everything that can come from outside a parameter in the statements to be executed. That way there is no way the server can receive data that look like commands; data always look like data.
Once you've got the name dictionary out of the database, you can substitute pen names for real names everywhere with a short script using tools as simple as awk and sed. Concatenating files with a predictable path name is pretty simple fair as well. This isn't common knowledge, but it isn't exactly a high degree of skill or labor either. I guess I'm chaffing a little at building the hacker up into something more impressive than I've seen evidence for so far.
Nonetheless, this is interesting. It sounds like something motivated by emotion rather than profit. I suspect that they may have actually thought they'd find something nefarious and be able to attach a name to it. I mean, look at the way some people cling to the word "trick"; Mann supposedly "tricked" people by describing in the text how he "hid the decline". Telling people exactly what you've done is a curious way to hide it from them; nonetheless, there are still people who think that. I think that has more to do with their emotional response than rational thought.
P.S. I sympathise with having to work on code that has probably grown beyond anything the original architect envisioned.
I was wondering when 'they' would resort to something like DOS.
It points to the fact that such people are more interested in politics than science and truth.
It was always going to be the case that the weaker 'deniers' got the more aggresive they would get.