Chapter 4: h3X’s Adventures in Networkland – Stealing The Network

Chapter 4 h3X’s Adventures in Networkland

by FX

h3X is a hacker, or to be more precise, she is a hackse (from hexe, the German word for witch). Currently, h3X is on the lookout for some printers. Printers are the best places to hide files and share them with other folks anonymously. And since not too many people know about that, h3X likes to store exploit codes and other kinky stuff on printers, and point her buddies to the Web servers that actually run on these printers. She has done this before…

Over the centuries, witches have either been admired for their mysterious capabilities or hunted down and burned by the male members of the society who feared them. h3X is convinced that there is no such thing as secret, esoteric knowledge. It’s all learning things and applying your experience in a specific way, no matter if you build something as beneficial as the microwave oven or find your way into some organization’s printers. But if you do the things you do right, or even worse, use your imagination to do them differently with greater effect, there will always be people fearing you. Her approach, together with her taste for lower-level network communication, led to her h3X handle.

First, h3X checks her list of big university networks. Collecting this information has required some effort. She has spent some time surfing the Web and querying the Google.com search engine and the whois databases, but she knows that it always pays to have vital data gathered in advance. The network in question should be at least class B sized, which means up to 65,535 systems in theory, and it should not have any firewalls in place to protect the internal networks. University networks usually fit the bill perfectly.

Male 31337 hackers would now probably fire up a port scanner such as nmap and scan the whole class B network for systems that could possibly be printers, but not h3X. She opens a Web browser. The university of choice today is bszh.edu. The first step is to go to the campus Web site and look for the IT department pages. These usually reside on their own Web server and contain all the answers to those stupid questions students usually ask the poor administrators. She digs through a ton of “How do I send e-mail?” and “Where do I get an account for this-and-that system?” questions, and finally finds the support pages that deal with printing. Here, she can choose between pages on how to set up a UNIX-based print server, and pages for those poor students using Apple Macintosh or, even worse, Windows systems.

These support pages turn out to be a gold mine. They are filled with information on where to download the driver for which printer and what to put in the fields. h3X checks for the section that details the installation of the Hewlett-Packard (HP) network printer client. Somewhere in the lower middle of the page, h3X finds the information she was looking for:

“In the field with the name Remote Printer, please enter the number that corresponds to the printer you want to use according to the table below.”

Following this entry is a table with printer names such as ChemLabColor and DeanDesk, their models, and their IP addresses—all presented to her on a silver platter.

Now, h3X runs a ping sweep to see which of the printers are online. In fact, she copies and pastes the IP addresses listed on the Web page into a text file and uses it as input for the almighty scanner nmap, this time with option -sP for a ping scan. As expected, most of these printers are responding to her pings, and nearly all of the HP printers run Web servers. She already knows which models they are, but if she didn’t, she could have found this information on the printer’s own Web pages, served directly off the box itself.

All the HP printers have at least 4MB of RAM, which can be used to store files—more than enough for the average-sized exploit code. But RAM means that when the printers are switched off, the files are gone. A far better solution for storing files on printers is flash memory. This memory keeps the information, even after a cold start. And the printers with flash memory have other capabilities of interest to h3X.

But in general, it’s not complicated to use a printer as her personal storage. HP invented a printing protocol called the Printer Job Language, or PJL. This language is a combination of escape sequences and clear text commands, and it is generally used to format your print job. You tell the printer things like:

  1. Look printer, a print job starts right here.
  2. Get me some size A4 paper, in portrait.
  3. Use the ECO print mode.
  4. I want it in 600 dots per inch (dpi).
  5. And here comes the data.
  6. That’s it. Now please proceed and print it.
  7. End of transmission.

But the same PJL also supports commands to handle files on the local file system on the printer. Smaller printer models see their RAM as a file system; the bigger ones also use the flash memory. It pretty much looks like an old MS-DOS system, since the so-called volumes are numbered from 0 on and are designated by a colon after the number (for example, 0:). On these volumes, you can create files and directories.

If h3X puts her files and directories in places not inspected by the printer’s firmware, she can be pretty sure they won’t be touched. This is why h3X likes to place her files on printers. There is simply no better offsite storage a hacker can use. So, she selects the 10 printers in the desired model range from the list, which contains about 60 entries, and checks the device’s Web pages.

Three of the printers are entirely open, which is typical. Five others ask her for an administrator password when she tries to enter the configuration menus on the device’s Web server, but that is only a minor problem. The other two don’t react correctly. Well, these printer Web servers aren’t exactly Apache Group software, and they occasionally crash. But for the hackse, it would be a waste of valuable resources to ignore these two little devices.

She considers port-scanning the printers, but decides against it. Although universities rarely have an IDS, a port scan can be spotted by all kinds of people and devices. Sometimes, administrators will notice the decreased performance and see a bunch of TCP SYN packets in the tcpdump output. Other times, the scanned devices are not in the best shape and simply crash or behave oddly, which often alerts the support personal and spoils the whole hide-behind-a-printer idea.

What h3X does check is access to the AppSocket port: TCP 9100. This port is the one that talks PJL to her system, right through a TCP connection. This port is her golden key to the network. She doesn’t want to be ready to go, just to find out later that the damn port is filtered out. On her system, h3X opens yet another shell, and types:

She does this manual check for all 10 printers, since she has had bad experiences with these 9100 ports. She always waits for a while to see if the connection is closed by the printer. This would mean there are access lists configured on the device, which would mildly complicate matters. After a while, h3X presses Ctrl+C to terminate the connection. But at one of these checks, h3X lets go of the Ctrl key just a split second too early and transmits the character c. Without realizing this, she presses Ctrl+C again and closes the connection.

Satisfied that the ports are all accessible, she goes on to take over the five “protected” printers. The Simple Network Management Protocol, or SNMP, has been her friend for years. Version 1 of this protocol authenticates with clear text community strings that resemble passwords. Nearly all network equipment supports SNMP, mostly version 1. And most network equipment comes with a standard community string for read access: public.

This brings another smirk to h3X’s face. The bug in some HP printer firmware versions has been known for quite a while, and nobody bothers to update the printers. Why? It’s just a printer and can’t do any harm, can it? She laughs at her own joke. The object ID h3X requested reveals the administrator password in hexadecimal. It’s not a surprise with a handle like hers that she can read hex instantly, globe as a password … how silly, she thinks.

The trick works on only two of the five protected printers, but hey, that’s life. But the silly password on those two turns out to work on the other three protected ones as well. h3X leans back a bit on her couch and puts the laptop to the side for a minute or two to think about that. Suddenly, she grabs the laptop again and enters:

Ha, ha, ha! globe is not only the administration password for the printers, but also the SNMP read/write community string—the one that lets h3X change settings of the printer via SNMP. Well, these dudes at the university are seriously hopeless, and one of their printers just got relocated several levels underground to serve Satan’s printing needs. Now h3X can fix the two broken printers, assuming the community string works there as well. And it does.

Now the printer reboots. h3X doesn’t like to do that, but rebooting not only helps with most Windows-based systems, but also can fix printers. After all, they are not too different. But after a while, the ping still doesn’t show any answer from the rebooted printer. What’s wrong?

h3X checks that she is still pinging the IP address of the printer and finds this to be true. Now, what the heck happened to this damn piece of HP technology? And how is she supposed to find out if the godforsaken piece of hardware does not get back up? She is angry. Why did that happen? Why always to her? The hackse lets some more time pass, and then decides that this particular target just got KIA.

Since it’s about one in the morning (CET) on a Thursday (actually, it’s Friday already), h3X decides to pay the local house club a visit and see if there is a nice piece of meat to play with in place of the printer. She puts the freshly discovered devices in her list file and makes a note about that one particular go-and-never-return box. Then it’s time for DJs, vodka-lemon, and possibly some dude with a decent body and half a brain—though she knows that’s a hard-to-find combination.

Halfway Around the Globe at bszh.edu

Dizzy shows up for work on a cloudy Friday morning. Dizzy isn’t his real name, but since no one seems to be able to pronounce his last name, and for some reason his first name doesn’t do the trick, everyone refers to him as Dizzy.

Dizzy isn’t actually what you call an early bird. He is more like the late bird that finally gets the worm because the early bird was eaten by a fox. But that’s okay. As an administrator at a major university, you aren’t really expected to report for work at oh seven hundred sharp.

The first thing Dizzy does when he comes to work is unlock his personal system, a Sun UltraSparc, and check e-mail. For Dizzy, mutt does nicely. He can’t really understand all those dudes clicking around in Outlook Express, Netscape Mail, or whatever. The next thing is to join some Internet Relay Chat (IRC—yes, admins do that too) and greet some friends.

Then Dizzy gets a call from one of the student labs. “Hi, this is Professor Tarhanjan. I’m giving a lecture at the mathematics computer lab, and my students can’t print. I tried to print myself, but it doesn’t work. I even power-cycled the printer, but it still doesn’t work.”

“Sure thing, prof, I’ll come over and see what I can do.” Frowning, Dizzy locks his screen and starts the long walk to the lab.

In the lab, most students behave as if their entire career now depends on the ability to print in the next 10 seconds, but Dizzy is used to that. He trots over to the HP 8150 and looks at the one piece of letter-sized paper in the output tray. It contains a single character: c. Dizzy finds that kind of weird and asks if anyone has printed this page. Apparently, each lab student tried to print before calling the professor to report the problem. Nobody knows who could have printed this page.

On the printer’s front panel, Dizzy uses the painfully slow menu interface to check the IP address of the device. “Hmm… I’m not sure, but I don’t think this is the IP address the printer is supposed to have. Did you change it?” he asks the teacher. The professor is astonished by the question and doesn’t know if he did. Probably not, Dizzy decides. He grabs the phone and calls his colleague: “James, are we having any issues with BOOTP today?”

BOOTP is a bootstrap protocol. Devices can use it before they have an IP address. In fact, they often get their IP addresses and other stuff from the BOOTP server. Most people think that this is what the Dynamic Host Configuration Protocol (DHCP) is for, but DHCP is actually just an extension to BOOTP.

“Wait a minute buddy, I’ll check. Yep, the bootpd is crying all over the log files. What’s the problem?” James asks. “Well, one of the printers got a funny IP. Can you fix the BOOTP for me?” Dizzy hears James hammer away on his keyboard. James always sounds like a roach racing from one corner of the keyboard to the other and back, because of his blazing typing speed.

“Dizzy, found the problem. Some moron tried to be smart in the bootptab. It should work now.”

Dizzy turns off the printer and then switches it back on. Voil$aG! It gets an IP address from the correct network. He quickly walks over to the professor’s workstation and checks the settings. At this very moment, the printer spits out several Windows test-page sheets and all kinds of other documents spooled by the print server. Well, obviously, it works.

Exploring the Prey

The previous night didn’t get any better for h3X after that printer didn’t return. The only half-smart guy she met began boasting about his magic Internet knowledge and telling her how cool KaZaA is. She couldn’t stand it any longer and left him alone. At least she had a decent time with the other women.

But today is another day. It’s now Friday afternoon, a good time to continue where she stopped last night. To her surprise, the dead printer got reanimated somehow and responds to pings again, but h3X decides to leave this one alone for now. She wants to explore the others a bit. Now is the time for port 9100 magic. The hackse starts pft, a tool to communicate with a printer in its PJL language, and connects to the first printer.

It’s the standard setup for an 8150n. The good news is that it has plenty of space to store even larger files. h3X creates an HTML file in vi and fills it with some pretty cool exploit code she got off a friend in IRC. Then she puts it into the printer’s Web server directory 0: \webServer\home, using pft. If someone asks her for the code, she can pass him the URL to the printer and impress the guy. Cool, eh? And the best thing is that nobody can connect her to the exploit activity, since she is passing on a URL to a device that doesn’t even remotely belong to her. In some countries, the university is responsible for the content and will face a criminal charge.

But the printer’s disappearance from last night still bothers her. What happened? Well, let’s find out. She goes back to this particular printer’s Web server and checks the network configuration. Aha, the printer gets its IP address off a BOOTP server. That probably didn’t work last night for some reason. But wait a minute, a few lines below the IP address settings is something that really worries h3X: there is a syslog server configured.

Configured Syslog Server

Damn! She should have checked that before. The printer logs whatever it does to the server. Not that it would immediately lead to her, since most actions like connecting to the Web server or browsing the file system using a PJL port 9100 connection never show up. But the reboot sure as hell does.

h3X considers herself a careful hacker. She really doesn’t like the idea of log entries lurking around on another box and being a tattletale to her presence. So, the next target is the syslog server. If she takes this one over, she can remove the evidence. And besides that, it’s probably a good training exercise to attack a common operating system again. So, why not?

A quick port scan of the server in question using nmap reveals that it is a Linux system with just a few ports open. Among these are 21 (FTP), 22 (SSH), 23 (telnet), and 80 (HTTP). The Web server hasn’t received much attention since this box was set up, since it still says “It worked! The Apache Web Server is Installed on this Web Site.” h3X finds this amusing. The box is not a standard installation of a major Linux distribution, because it has either not enough or too many ports open for that. And no Linux distro h3X knows would install the Apache Web server with its after-install page.

And why is it that people install secure shell (SSH) on a system and still leave telnet open? It’s not the first time she’s seen that one, but it still gives h3X the creeps. Speaking of which, the SSH daemon is the next thing to check:

Oh well, the SSH daemon is not in any better shape than the Web server. This version is extremely well known for being vulnerable and shouldn’t be a problem. The hackse has the right magic (tools) to take care of this vulnerability:

This should be a short game, h3X thinks. Her box starts and tries the information from the target file on the remote SSH daemon, one attack at a time. h3X likes the way this exploit intelligently figures out one memory address after another. She would like to meet the guy who wrote it and see if he deserves some h3Xtended attention. The process actually takes quite some time.

After about an hour, h3X starts to think of alternative ways to get the box, since it doesn’t look like 7350ssh is going to make anything happening in the next few centuries. Fuck, h3X thinks, it’s one of those days when every damn thing goes wrong one way or another. You know, one day, you have the magic fingers of a digital David Copperfield, and the next day the stuff behaves as if you have pure concentrated and distilled shit on your hands.

So, the SSH exploit is not going to work. Well, h3X would love to know why, but this is a little bit over her head. While she hates to admit that, it would be stupid to behave as if she knows. Okay, back to square one. What was the thing she didn’t check? Oh yeah, the FTP daemon on the box.

Cool! At least some luck is left today. It’s funny people still use the Washington University FTP server. It has had security-relevant bugs in nearly every version. Some hackers have suggested that this particular service was implemented only to have every possible kind of bug in one code tree. It might make the coders, who spend the time to write this thing, feel bad; but face it, there is some truth to it.

Even in the world of hacking, there are brands. And brands suggest some key message to you. One message that many brands try to convey is the image of quality. If you managed that one, you can be sure of a fairly stable customer base, since people who are after quality are rarely the ones thinking too much about money. In the world of hacking, money is generally not an issue. Well, some people try that, but it doesn’t taste good. But a large happy customer base of your tools and exploits grants fame, and hell, most people like fame.

h3X has plenty of different wu-ftpd exploits at her disposal. Her own repository, together with stuff publicly available off http://www.packetstorm-security.org, gives her about 10 exploits for this single version of wu-ftpd. She is on the lookout for quality brands, since she has a choice. It’s kind of like shopping, actually. The one exploit in Java sure looks like fun, but it’s not going to be The One. After quickly checking the code, she goes for 7350wu.

Now that h3X has root on the box, she can relax a bit. All the hassles with the system and the printer from last night are gone. There is nothing like getting root on some box, no matter how complicated or, as in this case, simple it was. Root=done deal. Again, our hackse does not follow what most script-kiddies would see as the standard procedure. She does not install the next best rootkit on the box and move on. Why? Oh, that has some history to it.

One time, at a hacker conference in Las Vegas, h3X watched a young guy—barely 18 years’ old—take over a box. The guy thought h3X was a scene whore with next to no hacking skills. As usual, the dude figured he was going to impress her with his speed. So, after getting root on the box, he switched to another xterm and FTPed a rootkit over. Seconds after the package arrived at the target box, he fired up the prepared script, named 31337kit.sh, and was convinced he had shown his superior hacking skills. h3X, witnessing the whole procedure, smiled at the guy, who nearly jumped out of his chair and probably made plans for that night, tomorrow, and the rest of their lives. But, despite his extremely hopeful wishes, her smile was not an invitation to populate the world with future hacker generations.

Still smiling, h3X asked, “May I?” The guy looked puzzled but had no objections and moved slightly to the right, so she could touch the keyboard. When she leaned over, her hair brushed the cheeks of the guy, who hardly had any eyes for the rooted system. But instead of hacking away on the box, h3X only entered two letters, pressed the Enter key slowly, and took a step backward, to make sure this dude could concentrate on the screen instead of on her shape. When the happy hacker looked at the screen, he did not understand what he saw there:

“Well, dude,” h3X said, “do you know what a dynamic linker is?” The guy, realizing that something was not quite right, looked dumbfounded at the screen. h3X considered checking his vital functions to see if he was still alive, but the guy was just shocked. So she continued, “Your rootkit replaced the binaries, which were dynamically liked with the libraries on the system. Unfortunately, your rootkit binaries were not linked to the libs available on this system but to an older version. You broke the binary. You didn’t hide your presence. Instead, you announced it as loud as possible, since even basic system administration and operation will now fail. You can’t fix that, and the system will undergo a forensic analysis in … let’s say 24 hours.”

Dude junior-hacker could hardly look less happy. But then, his expression changed, and he felt a little anger in his chest. He slammed the laptop closed, took it under his right arm like a school book, and walked out of the room to do what most of the guys his age did: look for scene whores with less intelligence. (He didn’t succeed for the next four years.)

But h3X learned an important lesson from this fairly funny encounter. It’s not too hard to totally screw up a hack after you’ve already become root. Since then, h3X has a preference for another way of keeping her access rights the level they are. She grabs the password hashes from the shadow file and throws them in her crack program of choice: John the Ripper. The idea is that a logon with a known and existing username, which may even belong to the “wheel” group, looks less suspicious than connections to funny inbound ports. A lot less can go wrong, and the procedure is passive, which adds to the appeal. Of course, it’s far less sexy than installing a loadable kernel module (LKM), but a lack of sexiness isn’t h3X’s problem.

So there are at least two guys regularly using this box. A good assumption would be that those two are administrators. She drops the password hashes in John the Ripper and lets it start its work. h3X has a decent laptop, but it will take some time. Anyway, as long as she has this session running, she wants to find out what was and was not logged about her printer activity. She doesn’t really care if her actions on this box are observed later or not. She can accept the loss of a small-ass Linux system. But being caught with some sweet exploits on a printer would reveal this nice little storage strategy to people who she would rather not know about it. The Honeynet project did a fairly good job in setting up catch-the-script-kiddy boxes, but they still don’t have a printer in their setup.

h3X narrows her eyes, and her expression changes from one second to the next. It’s an interesting setup. What kind of guy puts all of the messages arriving via syslog in one file? He has to have some reason, because a stupid idiot wouldn’t bother to change the syslog configuration at all. And the guy also prefers to watch things in real time, which is the only explanation h3X has for the last line. Sending all syslog output to a console? The idea is kind of neat actually. With all the messages in one file, he can use any combination of UNIX command-line power to parse, dissect, and work the magic on the whole bunch of data at once. It’s not everyone’s favorite setup, but it’s still fairly effective if he can use it. h3X sure as hell can and silently thanks the guy for making her life a bit easier. Isn’t that what admins are paid for?

As suspected, the syslog file contains some serious evidence that she was here. h3X checks the remaining disk space on the system. When she fires up vi to modify the messages file, she doesn’t want to exceed the free space with the swap file created by the editor. It sure would look stupid when a swap file from the syslog messages is the one that fills the file system beyond its capacity and make all kinds of things go terribly wrong.

But there is enough space, and she goes straight to the edit of the messages file. Some minutes and several globally applied POSIX regular expressions later, the log file doesn’t contain any more evidence that she played with the printers. All those suspicious SSH connections with CRC errors are also now gone.

At that moment, the doorbell rings, and h3X leaves the computer for a minute to check who’s there. It turns out to be some of the gals she regularly hangs out with. They planned on some swimming pools (the cocktails, that is) today. “Hey bitch, turn your stupid computers off and let’s have some fun,” one of the visitors says.

“Yeah, just a fucking second, okay?”

‘Babe, when you say just a second,’ that usually means we get to hear at least two or three CDs before you get your sweet ass moving. Don’t do this guy thing to us again. I’m thirsty, and you can take over the Pentagon tomorrow. Move!”

h3X gives her friend a strange look and goes back to her machine. She needs to at least check that the remaining information on this box isn’t too bad. Since the syslog file is still open, she checks for leftover trash from her FTP attack and deletes lines that could give away things. In fact, since she is in a rush, she deletes every indication of FTP activity in the last two hours without checking what it is.

“Girl, if you don’t stop hacking around in the next minute, we’re going without you,” her visitor insists.

“Yeah, I’m done.” h3X logs off the system known to its administrators as tombstone, but leaves her own laptop on to run the password cracking, and puts it in the corner. Then she changes from her baggy pants and T-shirt into something more appropriate for hanging out: tight, black pants and a top that reveals the little piercing in her belly. Then they head out for a good measure of pure feminine fun.

The cocktail bar turns out to be the right place in more than one way. At first, h3X had some decent drinks, and then she even meets a guy. He is approximately one head taller than she is, not exactly in perfect athletic shape, but he’s still attractive. They talk a little, and she finds out that he works with computers, but the topic doesn’t come up again during the rest of the night. He’s the kind of guy you talk to and feel kind of cool. He knows a lot about music and bands and all that, keeps drinking strong beverages without slurring his words and staring at her breasts, and is overall pretty nice.

Despite the fact that they just met, they get into some serious personal discussions, and end up in each other’s arms for a good amount of kissing and fumbling. Unfortunately, the guy is from another city and just here for a business trip with his colleague, who looks like a total computer nerd. So, the encounter will be remembered by h3X as some serious drinking, a pretty good one night stand, and a panicking guy leaving her place and returning three times because he forgot all kinds of things (like his wallet, car keys, cigarettes, and some funny looking badge for the place he was supposed to be at an hour ago).

D-Day

So it’s Saturday, and h3X is alone again. She gives her friends a call and finds out that their night was a lot less eventful than hers. After that, it’s time to check the laptop and, of course, check on the box she took over yesterday. The laptop’s cooling fan vent no longer hums, and she unlocks the console to see what John the Ripper found. The screen reads:

This day is off to an awesome start, h3X thinks. She had an excellent night, and in the morning, as if ordered from room service, she gets toast, coffee, tomato juice, and the passwords of the guys for breakfast. She consumes them in order. First, it’s time to eat something and regain some of the energy lost in the past eight hours. Then h3X goes online and sees if the box from yesterday is still there. It is.

Although most hackers have several bounce points and other systems they can use to hide their traces in the land of the Internet, h3X does not possess such assets and, quite frankly, she doesn’t care a bit about that. In theory, most, if not all, hackers are traceable one way or another. But in reality, most system administrators don’t have the skills and are not going to hire an expensive consulting company to track her down. Even if they did, or their people actually know their kung fu, next to nobody contacts the FBI at the right time or files a civil charge against a guy (or gal) living halfway around the globe in a completely different jurisdiction. Forget it. So h3X fires up SSH and goes directly for the box. She tries to log in directly as root, and it works.

It’s time to explore the system a bit more, since the hackse assumes the admins will find out about her being on the machine shortly, and there might still be things of interest. But, at first glance, it’s just a syslog server. The Web server h3X saw the night before is really just that—an installed and forgotten Apache. It was compiled from source on that system, which, by the way, turns out to be a Slackware installation. There is not much running besides the usual stuff, the already known services and the SSH and related processes. So, h3X goes for the home directories of people or things on the box. There is not much there either. The home directory of the user James is pretty much an exact copy of /etc/skel and does not yield any useful information.

On all the systems h3X has owned over the years, reading the shell history has always been one of her favorite activities. In addition to the syslog, assuming the competent superusers of the boxes had enabled the histories and not fumbled too much with the configuration, they provided a lot of entertainment, and sometimes, even some cool command-line tricks she used later. But the majority of the people, even the ones fairly fluent in UNIX shell commands, leave quite messy histories. Lord, what has she seen? One guy didn’t know the difference between the command killall on different system types like Sun Solaris and Linux and tried to do a killall httpd on a Solaris box, followed by a hard power-off and reboot shortly after that. Well, at least it did exactly what the name suggested.

Another one had found out about disk space problems on his box, a database server. After checking all available devices and discovering a seemingly empty disk partition, he created a file system on that one and moved some of the bigger home directories there. What was funny about this particular box was the history file of another guy, obviously responsible for the Oracle database, trying to figure out what could have possibly happened to the raw device holding all the data. She imagined the database administrator (DBA) was seriously mad at the other guy when he finally found out.

She checks Dizzy’s home directory next. It’s pretty much empty, but the .bash_history file is large and sure as hell is a good read. The guy keeps calling the same shell script.

The next logical move is second nature to h3X. Of course, she looks at the shell script itself:

“Cool,” h3X says aloud to herself. “These guys use this box for the configuration management of the routers. This is going to be fun.” A broad smile appears on her face. She can pretty much see that this network is going to be her playground for the time being. You don’t leave a chance like this unused. As the next step to reflect the changed priorities, h3X leaves the computer, gets some Coke out of the fridge, powers on the stereo, puts a good DJ set on, and cranks the knob with the label “Volume” to the right. Then she heads back to her laptop.

Back on tombstone, h3X checks the /etc/inetd.conf to see where the Trivial File Transfer Protocol (TFTP) daemon writes its files. There’s a good reason. Most people would not see anything terribly interesting in the shell script she just found. But she is not “most people.” h3X knows exactly what this shell script does. It instructs the Cisco router, actually the Internetwork Operating System (IOS) on it, to place its current configuration on the TFTP server mentioned—this very box—and tells it how to name the file. So she got the whole nine yards, since the configuration files have to be here on the box. And Cisco configuration files contain interesting information, such as the firewall configuration (so-called access control lists) or leak thereof, the routes and network sizes, and passwords, which are not even really encrypted.

The line for TFTP in the inetd configuration file doesn’t mention a directory, which tells h3X it’s probably the default. As far as she remembers, that should be /tftpboot. The next sound in her room is a slap against her forehead. “Bright little girl,” she says. “It’s right in front of your eyes in the script.” So, she changes into the /tftpboot directory and sees about 50 files lying around, all ending with -confg. Excellent. Following a gut feeling, she also checks the cron table, which lists programs that are supposed to be executed on a regular basis. This table on tombstone actually contains a list of calls to the getconfig.sh script, so that the box will go out at night and get a backup of the configuration used on all the routers.

h3X uses the secure shell copy program (scp) to get the files down to her box. Having a collection of the router configuration of some place, even a university, on your system is kind of cool, especially if you aren’t supposed to have it. The passwords are encrypted with a trivial algorithm that is based on some exclusive OR (XOR) function that is considered secure—unless someone finds out how it works, and that would never happen. Well, it has, h3X thinks. Security by obscurity never makes sense, because sooner or later the information will leak. The more interesting the information is and the more value it loses over time, such as an exploit, the faster the secret spreads.

An idea pops into her head when two formerly unrelated synapses made a sudden decision to join their forces: Douglas Adams should have made spaceships travel by 0day exploits instead of bad news. Oh, wrong script, and a bad idea anyway, since the resource 0day exploits is very limited, while there is a nearly infinite supply of bad news. So much for spontaneous synaptic action. But the mentioned Cisco algorithm really wasn’t a good idea. It was quite some work for the guy who discovered it in the first place, since he had to wade through tons of absolutely unrelated binary data before finding the key. But after he found it, people could write instant crack programs in nearly every programming language. You could get these programs for Palm handheld computers, and even mobile phones can do it these days.

The hackse knows the rules. You don’t protect a computer system by relying on the fact that nobody can get the information about how you did it. You’re better off telling everyone you work with and seeing if someone can come up with a way to defeat your protection. If everyone who needs to rely on the security of whatever you did has a chance to check it out first, you get an army of testers and ideas applied to your mechanism. Sometimes, it takes years until the first one says “Eureka!” and tells you how he broke it. In the ideal case, this never happens. Then, you’ve got a good concept. Otherwise, you are back to square one.

Back to work, h3X thinks, and uses the power of bash, her shell of choice, to find out how many different passwords are used on the Cisco routers.

This isn’t an ideal query, but sufficient for h3X right now. So they use only one user password for all the boxes. Cisco IOS commonly uses two different types of local password encryption. One of them is called the enable secret password and is a genuine MD5 (message digest 5) hash function, and h3X can’t do anything about that. The MD5 hash is a one-way trap function. It’s easy to perform in one way but nearly impossible to undo, pretty much like cutting your head off. The difference here is that brute force will never get your head back on your shoulders, while a high-end computer can search the entire possible or likely key space for the MD5 hash to crack it.

The other encryption is this broken, old, funny algorithm they keep using for whatever compatibility reason. This encryption just revealed the password to at least user-level access to h3X. Now, the only thing she needs is a router she can connect to and find out if her discovery is correct. The best way to do this is to follow the path your traffic takes when it tries to reach one of the systems in this network, because this path will cross the routers.

One of the first things h3X learned when playing with the Internet in general, and routers in particular, is that the best way to think in these networks is to sit on a packet. If you can make your mind settle down and feel comfortable on a 1500-byte frame as much as on a $1,500 couch, you’ve got the right mindset. Then buckle up and await being dropped on the cable and instantly accelerated to nearly the speed of light until the next hop—another router. Get off the packet as fast as you can (it might become corrupted, and you don’t want to risk that for yourself) and see what happens to it. Usually, it is parked for ages compared to the time on the cable, and is then disassembled and reassembled with some of the data changed. Now, get back on and enjoy the next leg of your journey.

So h3X performs a trace to the Linux box she owns now and checks the results:

Well, the last hop before the little Linux box sure looks like a router. Now h3X can see if the password is worth all the trouble or if she just stumbled across an old repository of Cisco router configurations nobody uses anymore.

“Yes, user level access on the routers achieved,” h3X reports to the empty room. And it’s always good to award something to yourself when you’ve finished a piece of work, so she raises from her office-type chair and walks over to the kitchen to get some coffee and a cigarette. Now, the only problem is the enable secret password. Cisco routers have 15 different privilege levels. Usually, only levels 1 and 15 are used, and guess what, 15 is the superuser. Only with level 15, commonly referred to as enable access, can she reconfigure the box and have some serious fun with it. Let’s try that, h3X thinks.

“God is a girl!” h3X cries out. The enable password is exactly the same as the easily decrypted user-level access key. “Dude,” she says to the screen, actually addressing the administrator of these boxes, “the command-line interface even warns you when you do that. Guess why?” But truth be told, most people overlook the fact that not only the password itself is important, but also where it is used. If you have a strong password of about 10 characters, and you use it all over the place, you risk a domino effect. Assume that someone uses his password for the company account and also for all those Web pages he subscribes to. Now, on those Web pages, or to be more precise, on the database behind the Web page, the password is stored in clear text. This, in turn, means that his company account password is stored in clear text on a database in some Web farm. Now, doesn’t the company account also allow remote virtual private network (VPN) access? Yes, and it’s still the same password, protected by some probably flawed Web-based system. The same concept holds true for the Cisco configuration. When you got two different security levels of encryption: stupid and proven, and you use the same password in both, what’s the value?

The hackse wants to make sure that the enable password is the same for all the boxes. It’s really bad if you find out in the middle of doing something exciting that all your plans are toast, just because you didn’t prove a theory completely. She uses the grep command to get all the enable secret strings out of the configurations and puts them with the configuration filename as username in a file.

Now, she supplies the word list and her fake shadow file to John the Ripper. Most of the passwords are cracked right away, since the second word in this unbelievable extensive word list is the assumed correct one. John does not return right away, but instead tries to crack two other passwords. h3X isn’t actually happy about that outcome. Apart from those two routers, she has the whole network nailed down. But these two have a different enable secret password. She checks if they have a different user password as well, but (unfortunately for her) they are all the same. Well, she will need a different way to get these two. They are called inetup1 and inetup2. So, there is at least some special protection for the Internet uplink boxes, h3X thinks.

Right then, her mobile phone rings. “Yep,” h3X takes the call. It’s the guy from last night. He just wanted to say good bye’ for the weekend and doesn’t want her to think he’s an asshole or something. He apologizes for leaving in such a chaotic way this morning. Actually, he sounds like he is in chaotic mode again, being in the car and alternatively talking to her and shouting politically incorrect terms at the other drivers around him. The phone call goes smoothly, and they agree to stay in contact … for whatever that’s worth, h3X doesn’t add.

Just when she presses the red button on her phone and wants to get back to enjoying her new little networking fun, the phone rings again. It’s another hackse, who regularly gives h3X a call to see what’s up and occasionally ask some questions.

“Hey h3X, question: How do I convert an IP address to its binary form in C?”

“What do you want to do with it?”

“Don’t ask. I just need the IP address as a binary number, and don’t fucking tell me to use a calculator”

“Well, I would use some left-shifting in a loop. Something like for k from 0 to 31, left-shift IP address and see if the current number AND 0x80000000 is 1, then write 1; otherwise, write 0.”

“Great, thanks, I didn’t understand shit. Could you send me an e-mail with that a bit more verbosely explained? I need it.”

“Babe, do you need that for some hacking?”

“Not exactly, but why is that important?”

“Because I get the impression that I do your damn homework!”

“Come on h3X, don’t bitch at me. Can you send me that e-mail or not?”

“Oh well, yes, I can. Check your mail in half an hour or so.”

“Thanks. And how is life in general?”

They go on and chat a little about the guy from last night, how they met, how they spent the evening and the night, and so on. h3X doesn’t mention a single word about the bszh.edu network. Later, she probably will.

h3X needs to get a handle on how this particular network works. Having the configuration files of the routers in this network is one thing. Finding out what they are is another. The thing is, the administrators are probably not the brightest in the world, but if you connect to each and every device with a Cisco Systems label on it, they’ll notice sooner or later. But h3X has the configuration files. Now, such a file contains a lot more information than just the passwords.

The top line, version, shows the operating system version used to write this configuration file. Except for a very few weird situations, this is the version running on the device. That’s the first critical piece of information. Earlier versions indicate a network where nobody cares about the routers and opens the possibility for some exploitation attempts, but h3X doesn’t need that since she has only 4 percent of the routers left to take over. A higher IOS version is much better in that situation, because it supports more features, including features h3X plans to use.

Other elements of the configuration file contain implicit information. The number of interfaces in the box gives a good indication to what kind of device it is. If you include some interesting side effects in the configuration, you don’t want the device to slow to a crawl. Just because it can theoretically do something doesn’t mean it has enough CPU power for the job. Devices with one or more controller statements in the interface list are usually bigger. If it just knows one Ethernet device and one BRI (Basic Rate Interface, or just plain ISDN), it’s probably not one of the Internet’s core routers.

Inspecting about 50 different Cisco router configurations for hints on the application of this particular black or blue box is as boring as it sounds. You need to proceed methodically and stay concentrated, and this basically sucks, since you don’t see real progress being made. It’s the same for h3X, but females are sometimes a lot better at concentrating than males, and so she spends the better part of the night trying to figure out interconnections and other facts about this network. After that, she barely has enough energy left to sit on the couch and watch some TV before she dozes off. The phone rings several times in an attempt to make this attractive, young member of society participate in what people call nightlife, but it goes unheard.

Trainees First

Christian is a trainee at bszh.edu. He received his chris@bszh.edu e-mail address two months ago, when he came over from what his colleagues call “Yorope” to spend half a year or so there at the campus and see some serious computing equipment. So far, he can handle all the stuff they have given him, but he doesn’t want to become the Windows administrator of this place. That’s what they try to put on my shoulders, but no way I buy in, he thinks.

It’s a Saturday, and he is not required to be at work. But Dizzy has told him that he can touch the other production systems on weekends, if he is careful. Dizzy and Christian agree that you can’t learn about being a system administrator on nonproduction play-around boxes. Therefore, Christian got the root password to work with the real things. And since the root password is kind of complicated, he wrote it down on a piece of paper and put it in his wallet. Nobody is ever going to find it.

Since it is probably going to be one of his next tasks, Christian checks the syslog server. It’s a Linux machine. He has Linux systems at home, so he knows his way around. Dizzy has told him to check the syslog file and make himself familiar with all the devices dropping information on this host. He looks around for a while and sees several strange boxes, but the Domain Name Service (DNS) is his friend and tells him mostly what they are. For some other devices, he has to check the documentation on the intranet server. After a while, Christian sees several messages from a really unknown device. They are not very recent, about a week old, and they look kind of strange. Intranet, DNS, and his own text files don’t yield any information. “So, who do I call on a Saturday to find that out without getting killed?” Christian asks himself. He has an idea. By checking who logged in last on the box, he can reduce the number of people on his call list down to a few.

Christian issues the command last. It’s supposed to tell him who logged in and how long the session took. Also, it will tell him where they came from IP-wise, but that’s not of any interest to him right now. Unfortunately, several thousand lines of names flash by, listing every user logging in since the existence of the universe, or at least of this box. Damn it, Christian thinks, I forgot the command-line switch.

Instead of limiting the number of people on the command line, and this is surely supported here, he scrolls up in the window and looks at the names. Well, there aren’t many people using this system with their own usernames—only James and Dizzy, in fact. But a lot of people log in as root, since the root password is pretty well known to the computer people on the campus. So he has no choice but to call Dizzy. “Yeah.” “Hey, sorry, this is Christian.”

“Hey Chris, what’s up?”

“Sorry to call you on the weekend”

“Yeah, yeah, stop that. It’s okay. What’s your problem?”

“The device with the IP address … 194.95.254.17 … what’s that?”

“Oh, that’s easy. It was a test. We got this little router for testing, a Juniper box, and I connected it to the network to see how it works. Kind of cool, actually. Why are you asking?”

“Oh, just checking the syslog system as you told me. There’s a lot of stuff in here.”

“Yep, but cool that you check it.”

“Okay man, see you Monday then.”

“Bye.”

Christian hangs up and wonders what to do next. There is this little quake server he wants to build for himself and connect it to the big Internet pipe available here. While thinking idly about the next moves for today, Christian scrolls down the user list he just produced. Weird, he thinks, who is logging into this box from outside campus? If he knew what a whois database is, he could have figured out where this particular connection came from, but he doesn’t. Instead he considers calling Dizzy again. Well, he thinks, someone probably had a reason to do this. Maybe it’s one of Dizzy’s tests. Who knows? He logs out of the system to configure his quake server.

Secret Service(s)

Now, the obvious question is, what can a hacker do with a bunch of Cisco routers at her disposal. You can hardly install an IRC client on them, although it would have some coolness value to it coming into a channel on IRC from a Cisco box. Maybe I’ll work on that one later this life, h3X thinks. But you definitely own the infrastructure this particular network runs on. Therefore, you can redirect traffic in any way possibly supported by IOS. You can filter out specific packets and connections, like the syslog traffic going from the printers to the syslog host. This way, nobody would ever notice things happening with the printers. But, on the other hand, a halfway competent admin would surely notice the total absence of messages.

You can also have some serious fun with the routing. Just set some routes on the routers so they point to each other, and watch the packets jump back and forth until one of the boxes gets tired, and while decreasing the time to live (TTL) value on the packet, simply converts it to heat and blows it out of the fan instead of the interface. But again, it doesn’t make too much sense. It just causes the administrators to track down the problem and see if they can find it. And you can be pretty sure that even a total moron would eventually figure out that this route does not belong there and start wondering how it got there in the first place.

No, the absolutely best thing you can do with routers is a transparent traffic redirection. The technique here is called GRE sniffing, after the Generic Router Encapsulation protocol it uses. Information on a network normally flows in fairly direct lines. If that’s not the case, someone made a mistake or really needs some training. Every single hop decides on where the journey goes next. Assume that two computers on the bszh.edu campus want to talk to each other. The first one finds a poor, little router to pass the problem (the packet) to. On most systems, that setting is simply the default gateway.

Routing in the Internet works pretty much like the (mis)management of a problem in a bureaucracy or a big company, and there is not much of a difference between the two anyway. One guy has a problem, often created by himself. That’s the sending host with the packet that must be delivered to the destination. To not risk his promotion and prevent any unnecessary work, or work at all, he looks for some other guy to pass the problem on to. Ironically, the next hop (default gateway) is usually his team leader. He has a lot more contacts (connections) at his disposal and knows more or less what to do with the problem (packet). But usually, it’s passed on to the head of the department. After some of those up-the-ladder-pushing operations, the problem (packet) reaches a fairly high level. On this level, it’s transported to another department (backbone). From there, the problem descends down a comparable ladder until it hits some poor guy right in the face, and he needs to solve it or start the process from the beginning in an attempt to make it SEP (someone else’s problem).

But, if the self-generated problem is something trivial, the next hop will always handle it himself. Let’s say two people in one team have a problem with each other. This is one case that (hopefully) is not kicked up the whole ladder but solved by the team leader. He smashes their heads together, or something along those lines. Problem solved.

h3X now has the problem that she is not a member of this department, but she wants to know what’s going on. The only way to achieve that is to find a shortcut into the department’s social system—for example, by talking to the guys on a regular basis or by reading the e-mail of the boss. The idea is to do the latter.

Because routing works the same way as the described locally handled department problems inside bszh.edu, h3X needs a shortcut, or actually, a longcut. When two systems on the campus want to talk to each other, there is no need to send the packets all over the Internet. But h3X needs to teach the routers to do exactly that, so she can read every single packet going from point A to B. The solution to this problem is GRE sniffing. The generic router encapsulation is a tunnel. Packets coming into the router are not forwarded directly, but they are put into yet another packet with a completely different destination. This packet is sent on its way, and after several hops, it reaches the destination—again, a router. This router knows that there is another packet in the packet, and it takes the outer hull off. The inner packet doesn’t feel anything.

It’s like using your company internal snail mail system and sending a letter to your buddy in another location. It’s transported like everything else inside the building by your company mail people. But when they discover that its destination is outside your building, they put it into a sack and hand it over to UPS, who will sure as hell lose it (hence, the name). But if the UPS people don’t lose it, they will perform a comparable “routing” procedure to get the sack to the other company building, where a company mail person will take your letter out and continue the internal routing until it finally makes it to your buddy’s desk. For your company’s mail people, the whole UPS procedure is transparent, and they don’t care about the routing UPS itself does. They just throw it in at one side, and it magically appears on the other. And here we are: a tunnel.

Of course, when you are smart enough, you can make your company’s mail people use UPS to send a letter to the guy in the office next to you. And that’s exactly what h3X plans to do. It’s just a bit more technical in nature than sending letters around the office. First, she logs into one of the routers. She selects one in the technical department, judging from the name, to capture interesting traffic. Then she configures a GRE tunnel back to the little Cisco 1600 router at her place:

The IP address range in the 1.1.1.0 network is kept from a world starving for IP address space, but that’s just fine for h3X. Using an RFC1918 network here would be risky. It could be that some of the internal networks in this campus actually use these as test addresses, and she doesn’t want to give away this little remote sniffing by creating a total routing mess. Now, she needs to tell her own box to actually react on these GRE tunnel packets and reflect them back to where they came from; otherwise, it would break communication by making the information go around the globe and never come back.

“Okay,” h3X says, “let’s see if we can talk IP here.”

“Cool. Now for the tricky part.” There is an interesting feature in IOS that’s called a route map. h3X thinks about a route map as deliberately breaking the rules of TCP/IP routing. You can basically tell any logical interface to ignore everything it got taught in the code about how routing should work but forward the packet in absolutely unexpected ways. That’s what she aims for:

The last part is to configure the router at bszh.edu to use the same feature to send all the traffic to h3X. She does this last, since otherwise she would probably also lose her connection to the box by basically cutting down the tree branch she’s sitting on. Here she goes:

Now, let’s verify it works, h3X thinks. She telnets from another router in the tech department to the one she just adjusted the configuration on and checks her own router’s GRE processing:

“Yep, done. I own you.” She doesn’t bother with trying to send the traffic into her own network. This would just interfere with the network and some of the experiments she’s running here. She takes one of her spare machines and hooks it up to the outside segment of her little Cisco router. It’s always nice to have a hub in every network segment you are using, she thinks. Firing off the sniffer Ethereal on this machine finishes the trick. Ethereal is smart enough to know about GRE encapsulation and just proceed with the inner packet as if it were sent directly and not encapsulated. Now, h3X can sniff traffic that is traveling in a network several thousand miles from where she is. She watches the traffic going by, but sees only some boring packets like the TCP keepalive messages for some proprietary protocol.

Since the whole sniffing business is automated and clogs up her DSL connection quite fully, it’s time to do something completely different. She calls some of her friends to find out what party is going on tonight. Some of them are just being couch potatoes today, watching TV and stuffing unhealthy things in their mouths. But h3X teams up with a faction of them to go to some club party. It turns out to be a former restaurant stripped of all the features of such a place, including the wallpaper and other decoration, with nothing more than a DJ spinning and an improvised bar. But it’s nice to hang out with her girlfriends, look at people, and decide who deserves the observation, “What an ass”—in whatever respect.

Discovery

Dizzy is on the road. It’s Monday at his current position on earth, and he is on a business trip. His boss has decided that he should go to some event a router vendor put up. As he was told, he is sitting at the airport oh eight hundred sharp, waiting for his economy class flight to some sales pitch. Out of pure boredom, Dizzy calls James to see what’s up on the campus network.

“Hey James, it’s Dizzy, what’s up?”

“Hey, enjoy the airport?”

“Yeah, sure. Kiss a politically incorrect place of your choice on my body. So what’s happening at the campus?”

“Well, not much. It’s the usual Monday morning crap. Refilling paper on printers, checking the backups, and so on. You know the drill.”

“Anything interesting besides that stuff?”

“Oh, yeah, one thing. The MRTG traffic shapes look kind of funny on two different boxes. Since Sunday, the amount of traffic doubled on those. No idea where it went. Could easily go to the Internet, I don’t know.”

“Got any idea what it is?”

“Not really. Chris is looking at it, but he’s seeing MRTG for the first time.”

MRTG—Multi Router Traffic Grapher—is a tool that collects values off one or more devices and plots a graph about it. As typical for open-source software, it doesn’t really matter what type of device you use MRTG on. One guy actually makes MRTG graphs about the wave height on the shore in front of his house. But most people use it for collecting traffic statistics on their routers, so they can see how many bytes these moved from point A to point B.

“James, can you set up a sniffer on the segment and find out what’s wrong?”

“Well, yeah, if I find the cabling plans for that. You know what the patch panels look like. It’s a mess.”

Damn it, Dizzy thinks, I could find them way faster than James, but, of course, I have to sit at the airport and wait for some cattle car to haul me to a sales show.”

Dizzy hates flying around. Not that he is afraid of flying itself; that’s actually something he enjoys, but it’s the process of getting there. You’re standing in more lines than are required in some poor countries to get your food vouchers. Your stuff is taken apart several times, just to make sure you aren’t a terrorist. And onboard, it’s not a bit better. Just to make sure it doesn’t end there, you need to hunt down your luggage on arrival. It’s even worse on international flights, when you’re required to tell the immigration officer why you’re going to spend money in his country and why you sure as hell will leave again when your return flight is due. But the worst thing about all the airlines and airports is the unbelievable amount of lies. Every “Hope you enjoyed …” is a slap in the face of the passenger. Actually, you could die of starvation and rot away right there in front of the gold members lounge, and nobody would care.

“Okay, James. I’ll be back tomorrow. Please, if you find time, check on the router thing. It could be a bug in the routers, and I don’t want them to explode on me in the middle of the night.”

“Yeah, I’ll try to find out what’s going on there.”

“Okay, bye.”

Dizzy hangs up the phone and thinks about the issue. They had problems with routers before, but there has never been such an increase in traffic, at least not doubling the traffic. First, he considers some system in the network being too stupid and fragmenting the packets to a high degree. But that would not explain the 100 percent increase James talked about. So what is it? And what if it gets worse? Well, on the Internet uplink routers, nobody is going to notice the increase in traffic. The students use the network to trade copies of full movies, so whatever happens, it’s not going to be a significant increase in the Internet traffic shape. But what traffic would go out to the Internet here? It’s just one segment James said, right? Dizzy checks his watch. Well, it’s time to move from his seat to yet another line: boarding.

Three hours and several queues later, Dizzy is at the place where the show is taking place. A sales assistant is talking to him about the vendor’s routers and why they are so much better than anyone else’s. Dizzy barely listens. He still thinks about the increase in traffic James reported. When the presentation starts, he sits in the last row and discovers that these guys have a public WLAN set up for the show. His neighbor is surfing CNN. He fires up his laptop and checks if he can reach the system named tombstone, and he can. It has its merits that they don’t close the shop like a fortress. Checking the SSH key fingerprint, Dizzy logs in.

In contrast to what h3X discovered, the Web server on tombstone is actually used for something, namely serving the MRTG-generated graphs. Dizzy checks them out and discovers something really interesting. Some time yesterday, the amount of traffic on average doubled from one moment to the next. He has no idea why. But he can reduce the possible time frame pretty well. Dizzy goes for the syslog file and checks for any messages that could give him an indication of what happened. About half an hour later, he sees something that gives him a sudden, cold chill.

“Oh shit!” Dizzy says aloud, and the whole group of people politely listening to the presentation turn and look at him. He blushes a little, but doesn’t spend too much time worrying about these people. Lord he thinks, someone from outside changed the configuration on our routers! Dizzy leaves the room and calls James.

“Hey buddy, did you fumble around the routers during the weekend from home?”

“No, why should I? I was at my mother’s place, and she doesn’t even have a computer, let alone Internet access. It’s a pain when you can’t check e-mails and …”

Dizzy cuts him off. “Someone did.” The line is silent for several seconds.

“Are you sure? How do you know?”

“Well, the logs say it loud and clear. Check with Chris if he did something, but he shouldn’t even know the password.”

James puts the phone aside and talks to Christian. As expected, he doesn’t know what happened to the routers, and he sure doesn’t know the password. “Dizzy, Chris say’s he doesn’t know and I believe him.”

“Yeah, me too.”

“So what do we do man?”

“I don’t know. I think one of the students has sniffed the password when we telnet’d to one of the routers and is now playing around with the routers from home. What do you think?”

“Sounds reasonable. I can’t imagine someone finding out our password. But what do we do about it?”

Dizzy thinks about the possible countermeasures: We could just change the password, but that’s only a temporary solution. If one of the students really sniffs passwords on a regular basis, it would help only until one of the administrators logs in to a router the next time. And how do you change the password? Via telnet, so it’s chicken and egg in modern communications.

He gets back on the phone to James. “Hey, leave it as it is right now and please investigate if we can use SSH on the Ciscos.”

“Okay, will do. But what about the traffic?”

“Fuck the traffic. We’ve got other problems,” Dizzy says and hangs up.

He can’t believe it. After all, bszh.edu is not interesting computing-wise. Heck, if they had anything interesting on their boxes, Dizzy would know about it; well, and download it, too. After all, they don’t do much research there, since research needs funding and Corporate America believes only in funding things it can sell, not things that improve education. Dizzy is outraged and astonished at the same time. Sure he reads BugTraq, who doesn’t? And yes, there are bugs in next to everything. But why should someone attack his little Class B campus network? His thoughts are no longer centered on actually finding the threat he just discovered. Instead, he begins to wonder about the thing as a whole. Good Lord, this is unbelievable. We aren’t the Lawrence Berkeley Laboratories. This stuff happens to astronomers, not to real sys admins. I’m sure as hell not Cliff Stoll. And I don’t have line printers to connect to my Cisco routers either.

Like most system administrators, Dizzy didn’t consider the data on his systems critical or classified. What’s the point on hacking around in our Ciscos? The student who got in there is probably just playing a joke on me. Why didn’t he hack the servers? Oh yes, we use SSH there, so he couldn’t sniff the password. But what did the guy do to the routers to increase the traffic so much?

It feels very strange when someone else takes over a system that, by configuration, belongs to you. It’s a feeling of being helpless and betrayed. You start thinking about all the things that are on the system, what it is used for, and which bits of information on the system are actually important and/or confidential. A friend of his had the experience once. Someone broke into his system and used it as a warez server. They traded software and movies on the box, and his friend had to pick up the tab for several gigabytes of Internet traffic. This is plain fraud. But, he wonders, why would you take over a router?

He waits impatiently for the sales presentation to finish, and then runs off the place as fast as possible. Back at the airport, Dizzy experiences a flood of “Sorry sir” and “I can’t help you” apologies, while trying to get an earlier flight back to the campus. Hanging out in the public waiting area, he thinks about the countermeasures he will take when he gets back to the systems.

Since he can usually think better when someone else is listening, he calls James again. Of course, the topic of the conversation is already agreed on.

“What should we do? Well, first off, we have to change the router password. But the attacker can sniff them off the wire as soon as we use them again.”

James was not idle either since their last talk. “Hey buddy, I checked on the SSH for Cisco router stuff. Man, that’s not as easy as configure, make, make install. They actually have different IOS images for that one. And guess what, they want money for it.”

“Really, oh … why is that?”

“Maybe because they’re a company?” James suggests.

“But the security of our entire network is at risk, and that’s only because the standard package doesn’t include secure administration? What a joke!” Dizzy can’t believe they charge you for security. “Next time, we have to pay extra for password support or what?”

“Hey, my name is not John Chambers, so please don’t be mad at me.”

“Yeah, sorry. So the department has to buy these secure-my-ass licenses, and we install them, and that’s it? Sounds okay to me.”

“Well, it’s not that easy. Most of the crypto images—that is, the ones with SSH support—need more RAM or more flash or both. So we first have to find out which routers need upgrades of one type or another and order these parts. Then, we can proceed and install the crypto image.”

Dizzy doesn’t like the information he is getting here, but it makes sense. SSH is supported only by newer IOS versions, and these are more memory-hungry than the older ones. On some Cisco presentations on troubleshooting, he has seen the memory management information: 40 bytes per allocated memory block overhead. Here goes all the memory.

“But wait a minute, James. Are these SSH images newer than 11.0 or 11.1?”

“Yes, sure man. You can’t just plug it into an older version.”

“Yes, I know. But this means we can’t just install them, even if the hardware supports it. Some commands changed, and we have to be careful when porting the configs. This ain’t no copy-and-paste!”

“You’re saying we can’t fix the whole thing today?” James asks.

“Hell, no. As you said, we need upgrades for some of the routers and the new IOS images in the first place, and then we have to port the configuration. And what about all these smaller routers we have? What about the Ascend MAX we got for dial-in, does this thing even support SSH?”

“I dunno, we’ll have to check. But don’t hold your breath.” James did not sound very encouraging.

They didn’t say anything for the next minute or two, but both stayed on the line. Dizzy started again. “But then, the attacker came in over the Internet and probably won’t risk playing with the routers while on campus.” Sniffing would also work for the administrators. A network IDS is basically an automated administrator with a tcpdump in front of it. If the attacker was on the campus and played with the routers, he risked other students or even the administrators seeing the traffic in the sniffer, and that would surely get him an appointment with the dean.

“So, we can install access lists on the routers and make sure you can only telnet in from the campus network itself. We could even limit it to the administration network.”

“Yeah, good idea, but you can’t limit it to the admin network. When we’ve got a problem in building A and you’re in building G, you have to be able to talk to the router.”

“We can SSH into tombstone and telnet from there. We can do this and limit the exposure. What’s the dude going to do with a password he can’t enter anywhere?” Dizzy actually likes the idea. If the routers don’t talk to you, there is no password prompt, and without a prompt, you can’t make any use of the password.

They chat for a while and agree on making the change at night. First of all, they have to telnet to every router and change the password. Doing this at night means they are going to check out who’s logged in on the router right after they connected. They would have preferred to make the change during the day, but that had the risk of the attacker (or worse, another new attacker) watching the traffic and learning the new password. On the other hand, at night, the guy could be on the boxes already.

Back at bszh.edu several hours later, Dizzy and James get ready to reconfigure the routers. James had done a little testing and decided that it would make sense to bind the access list only to the telnet service (vty). On Cisco routers, you can create various access control lists, give them a number, and assign them by number to an interface or service. The reason James prefers the binding to the telnet service instead of all the interfaces is performance. Instead of consulting a sequential list every time a packet crosses the router, it would only be inspected when someone makes a telnet connection to the box.

After that, he goes ahead and changes the telnet and enable passwords, as well as the SNMP communities. Now, that everything is access-controlled and all the passwords are changed, Dizzy feels tired and just wants a beer, or several of them. It’s two in the morning, and he really wants to go home and feel safe. James is still around and looks slightly better. Well, he didn’t have a flight-around-the-country type of day after all.

In his innocent style, James looks at Dizzy with a satisfied expression and asks, “Now that we closed the bastard out, what do you want to do about the traffic increase?”

“Oh shit!” Dizzy sits up straight, or as straight as his current state of fitness permits, and looks at James. He had forgotten the modified configuration and what it did over all the changes they pulled off today. “Damn, I forgot about these! Did you take a look at what it is?”

“No, I just asked around if everything seems to work fine.”

“Great, so we still run a configuration supplied by someone we really don’t know. Which routers are affected after all?”

“Dunno, according to the graph, it’s just the two routers. How did you find out about that whole business anyway?”

“I found the line in the …” Dizzy doesn’t finish the sentence. He is logging in to the two routers and checks the configuration. “Uh, what’s that? I sure as hell never did this configuration. Wait, what are these tunnel interfaces for? Uh oh. Why on earth should we send our traffic through a GRE tunnel? And where is this location? Ah … I’ve got an idea.”

James doesn’t understand anything, but doesn’t feel like asking questions right now. He is just too tired and hangs out in his office chair. Dizzy goes ahead and analyzes the configuration. When he finds it a bit too complex to dissect right now, he saves it via copy-and-paste and reconfigures the routers using the old configuration still available on tombstone. Then, he changes the passwords and makes up the same access list they did the whole night. After that’s done, Dizzy performs another rather critical task: He gets himself another cup of coffee.

Getting back to his computer, he logs into tombstone and checks the syslog file again. Sure, the entry is still there. This single line saying that someone else—someone evil—has reconfigured his router. Now, he uses grep on the whole syslog file, trying to find all occurrences of this particular alien IP address. He sees the two lines from the two routers in question with the statement that someone has configured them coming from this IP address. But the worst part is this one line that keeps showing up several times:

“Uh oh!” Dizzy says. “Not good,” he continues and starts typing furiously. First, check the last log. “Damn.” Then go to the command history file, but no luck here.

Dizzy suddenly stops typing and slowly raises his head to face James. “Dude,” he says very slowly, “someone just owned our ass.”

“What’s that mean?”

“He got root on tombstone.” It’s not even said as a remarkable fact. It’s just a simple statement, so it takes about five seconds for James to react. “Fuck.”

“Yeah, that pretty much sums it up.”

They stare at each other in disbelief and shock. “We can’t take it offline, so we have to stay with this system for a while. We can only try to close shop as good a possible and watch it.” Dizzy’s knack for crisis management kicks in. If it’s a small snafu type of situation, he might get a bit annoyed. But for a full-blown, 500-square-mile, global killer disaster, you want someone like him around. Keeping his calm, he goes down the list of services on the box.

“The SSH daemon is vulnerable to some attacks. We forgot to patch it that time when we did all the other systems on the campus. The telnet service isn’t the latest, and we can switch that off. Same for FTP. Who needs FTP anyway when we’ve got SCP. We need the Web server, but I’m pretty sure it’s not the Web server, so we’ll keep it up and just restrict access to the campus IP range and assign a password. Anything else?”

James doesn’t know what to say. His mind is still flying close circles around the fact that someone else has root on his system. Someone he doesn’t know. The routers were kind of unreal to him. It can’t hurt that much having some guy playing with it. It felt not so bad. But this one feels seriously crappy. It feels like watching someone else walking around your house, opening drawers and lockers, looking at this and that, shuffling through your papers on the desk, and you can’t do anything to stop him.

While James is still nursing his mental wound, Dizzy has already disabled all the services and is in the process of recompiling SSH, a newer version this time. Then, he halts the process again and looks at James. “The log says root, doesn’t it?”

“Yeah, so we figured he got root on the box. And?”

“James, it’s late but please try to be with me here. When wtmp logged a user as root, he provided the right password. Ergo, the hacker got our root password off this box. Luckily, it’s not the campus-wide password.”

“Yeah, but root123 isn’t really hard to guess.”

But Dizzy continues, “From all the boxes he could have owned, why this? Or did he own more?”

They go ahead and change the root password on tombstone. Just to be sure, they also change their own passwords, because you never know. Then they check about 20 boxes in the proximity of tombstone for signs of break-ins or other potential misuse. No such signs were found. Both system administrators have a very bad gut feeling about the whole issue. Dizzy still wonders why the hacker has taken over only this single box, and James thinks about getting fired for the bad job they were doing in terms of security. After several hours of fruitless searches for more hacker evidence, they decide to call it a day and go home, straight to bed without any more thoughts for beer.

The Girl Is Back in the House

h3X is coding. The sound system is active and reproduces some vinyl spinning from DJ C-MOS at DefCon, which is pretty much the absolute best sound for coding you can get as far as h3X is concerned. A buddy of hers had asked if she could write a little client to a Web-based system that keeps track of his working hours. He said something along the lines of the people writing the application being total morons and the whole thing working only in Internet Explorer. Now, this particular guy prefers systems with command lines, much like h3X, but he still lacks the appropriate coding skills. She does him the favor of putting together a Perl script that will automatically send the right requests when called with start and end times on the command line—much easier to use than grabbing the mouse or fingering around with the little rubber pointer control element on laptops, commonly referred to as clitoris.

When the script is finished and her buddy has to delete several interesting looking entries in his workbook from all those tests she did, h3X decides to pay her little remote-sniffing experiment a visit. But there are no more packets coming in from this other end, and the router reports the interface tunnel0 to be down. Argh, that was fast, she thinks. Then, she leans back and says to herself, “It was clear that they would shut me out sooner or later, but not so fast.”

The sniffer got several megabytes of data, but it turns out to be of very limited use. Most of it is simple stuff like SNMP status queries between hosts or syslog messages traveling the campus network. In fact, there is pretty much nothing serious in there. Then, at the bottom of all these packets, there is a telnet connection going on. h3X uses the Ethereal feature Follow TCP Stream and looks at the data going back and forth. “Looks like he got it,” she says. It is clearly visible from the trace, up to the point where it disappears and everything else with it, what the guy was doing. The last command she sees reads:

So, at least he’s not a total idiot, she thinks. She tries to connect to the routers, but the connection gets dropped every time the initial TCP handshake is completed. h3X starts to become annoyed. She had gone to a lot of trouble to get the routers set up this way, and the guy just slammed the door in her face. “Oh well, let’s take it back then. All your Cisco are belong to us.” She tries to log into tombstone and realizes that it doesn’t work. h3X never mistypes a password. Connection attempts to port 22, 23, and 21 finish the picture. She’s out. They closed the box down. “Fuck!” Maybe she should have used a rootkit. After all, they aren’t too bad, if you watch the linked-library stuff. Well, now it’s too late to be sorry.

Wait a minute, h3X thinks, if they had firewalled me off, I wouldn’t get a connection there. But now, I get TCP reset packets as if they closed the telnet port. Let’s check that. She port-scans one of the Cisco routers completely to make sure there is no other service listening that could be used for configuration. Maybe those guys configured SSH on every router and moved to some strange port. But it turns out that every single port is reported closed and none of them filtered. SNMP requests don’t produce any responses either. The problem with this is that you never know if the community string was wrong or the service is filtered, because the result is the same: nothing, nada, zip. But those TCP reset packets tell her a different story: “Hee hee,” she laughs, “That’s something. Guys, I think you overlooked something.”

h3X checks her printer file from bszh.edu. Didn’t they have some of those 8150 printers there? Yes, here they are. She quickly checks if she still has PJL access to them, and yes, she has. Now it’s time to use some of the charm that is genetically more dominant in females and get some code. She could have written that herself, but she knows someone who has a bit more experience with it, and why reinvent the wheel?

h3X grabs the phone. “Hey dude, how are you doing?”

“Hey h3X, what’s up?”

“Got a Q for ya. Didn’t you write one of these transparent proxy services for the HP printers once?”

“Yeah, everyone seems to want it.”

“So why don’t you just publish it?”

“Well, it’s rather cool to have it.”

“Okay, fine. Sooo, does it support UDP as well?”

“Actually, no. It’s just for TCP. Who needs UDP support for it anyway”

“I do.”

“But you don’t have it.”

“Right, but I could do the UDP support for it without reinventing the whole thing. I mean it’s not like there is a big secret behind socket code.”

“True. Look, if you pass this on, I will be after your sweet ass. But fine, check mail in a few.”

“Thanks dude. So, when is the next coding party?”

“What about a private one?”

“How private?”

“Just you and me”

“Can it.”

“Okay, it was worth a try. Byte.”

“Bye.”

This worked out quite well. Not that h3X is exceptionally happy about the fact that she has to fix the damn thing, but at least the TCP proxy part works. After a few tries, the command for getting mail messages actually produces more output than “No mail for h3X.” and she gets the code down. It turns out to be a fairly small Java program, designed to run on printers with the ChaiVM. It’s nice that they ship printers with Java virtual machines (JVMs), so sweet little hacksen can use them. Who else would need a JVM on a damn printer?

First, she has to check if this thing actually works. After little less than 20 full eons, she gets this Java code compiled and is once again happy about how cool C compilers work compared to this resource-hungry beast of a javac. Then she goes for the printer.

Now, the only thing h3X needs to do is add the classes to the configuration file of the ChaiVM, so they will be loaded into the process space next time the services start. So, she switches to another xterm and adds the some lines to the csconfig file:

Back at the pft window, she uploads the modified configuration file to the printer:

What’s left is to reset the printer, but that’s just a simple SNMP write, and here it goes. This time, h3X has taken care of the printer using a manual IP configuration to prevent the disaster she experienced last time playing with it. When the printer comes back up, she uses her beloved Lynx Web browser to connect to http://194.95.31.3/device/hp/h3x.bnc and configures a port-forwarding to one of the Cisco routers. Now, whenever she connects to the printer on port 31337, it will open a connection to the Cisco router’s telnet service and forward every byte one way or another. And voil$aG, she can again telnet to the routers. But right away, h3X realizes that the password doesn’t work anymore.

“Hee hee, dude, and here comes the h3X!”. She disconnects from the whole setup and gets back to another virtual desktop with the Java code of the printer proxy open. A few changes and several lookups in the class documentation later, the whole thing does UDP as well. The code was already there, so the changes for UDP were marginal.

It takes her a full hour from the first line changed in the code until the whole thing runs on the printer. “Now it’s time to teach this admin jockey how we deal with things in the network land,” she says to the screen and starts typing the final lines of her revenge:

The idea she is following is based on the fact that Cisco routers default to a specific naming convention for their configuration files, and as she has seen on the TFTP server on tombstone, this naming convention is followed at bszh.edu. The newly introduced access restrictions on the TFTP server prevent her from directly accessing these configurations. But on the other hand, TFTP doesn’t use any authentication. Therefore, she just needs to make sure that she is coming from a system within the address space of the campus, and the printer is the one doing this for her. By running a transparent UDP proxy on the printer, the printer will talk to her and the TFTP server on the campus, thereby circumventing the access restrictions.

h3X smiles to herself and says, “Now boy, I will make your day a bit more interesting.” She considers logging in to the routers and trashing their configuration or configuring the routing loop from hell, but this kind of behavior isn’t something h3X finds amusing. Instead, she aims at publicly showing the whole campus that the network administrators screwed up. She decrypts the new router password, smiles at the result, and fires off the pft printer tool again, this time for a longer session.

Aftermath

Dizzy and James are at work really late today. Fixing the whole network and making sure everything is the way it was before took all the resources they could muster. Back at the campus, Christian has a stack of things that need their attention. Of course, today a backup didn’t work, some elements of their homegrown network management software had a really bad time checking the routers, and a lot of other things just waited for a day like this to go wrong.

While James fixes the network management software by telling it the new SNMP read community, Dizzy walks over to the boss of the department to tell him the story. The boss is predictably not very happy about the whole thing, but in contrast to James’ fears, he does not even consider any disciplinary actions. Rather, he congratulates the two admins to the well-done job of recovering without any loss. He, too, has read Cliff Stoll and appreciates that they don’t try to catch hackers for the next year but rather concentrate on the tasks ahead.

Back in his office, Dizzy is about to check his remaining e-mails and answer a few of them concerning things he didn’t do in the last two days, when the phone rings: “Professor Tarhanjan here. Say, what’s the deal with all these messages on the printers?”

“What are you talking about?”

“Look, I know you find this funny, but it’s not so nice to distract all those students from their work. They have better things to do than play your little game.”

“Prof, again, what are you talking about?”

“You really don’t know? Then, come over to the C block and see for yourself.” The teacher hangs up, obviously annoyed about whatever it is.

Dizzy gets the feeling that something isn’t right. He walks over to the C building. On the way, he meets several excited students from the computer science and math groups. They appear to be running around playing some kind of scavenger hunt game. He stops one of them he knows on the floor and asks what this is about.

“Look Dizzy, that was a cool idea.”

“What was a cool idea?”

“You mean it’s not you?”

“No, damn it. What is it?”

“Ha, someone left messages on all the printer displays in the C building. It’s a sentence and we’re trying to puzzle it together. Can you tell me where the other printers in this building are? We already covered the ones in the lab and the auditorium.”

“What? What’s the sentence?”

“We’re trying to find out. It’s always two words per printer. So far, we’ve got this.” The student hands a piece of paper to Dizzy. It says:

Dizzy stands there and stares at the paper. This hacker played a joke on him—a bad one this time. But what is he supposed to do? When the student starts moving again in the direction the others went, Dizzy follows him. First, he walks slowly, and then he starts running to catch up with the crowd. Arriving at the next printer right in the dean’s office, he finds several students trying to convince the dean to take a look at it. The dean isn’t really happy, but one of the students catches a glimpse of the display and says to the others, “Capital S … three … c … capital U … capital R … n … three … seven. What does this mean?”

One of the students notices, “This is only one word, so it’s probably the last. Now, let’s try to find out what it means.”

Dizzy wonders how long it will take before the students find out that the last word is actually the new password to the routers. At this very moment, the dean finally comes out of the office with a piece of paper from the printer in his hand. He tells the students to evaporate into thin air and asks Dizzy into his office.

Dizzy and the dean talk for three hours straight. In the first hour, it looks like James’ fears about getting fired will finally come true, but then the tension eases a little, and they talk about network security. In the third hour, the dean approves the money necessary to purchase SSH-enabled IOS versions and the required hardware upgrades for the routers. More time or another intern to relieve Dizzy from the day-to-day work is not approved, and Dizzy must promise to look more seriously after security, without preventing the researchers, teachers, and students from using the systems conveniently. Dizzy agrees with a hushed “Yeah, sure.” At the end, the dean hands Dizzy the paper from the printer. It reads: