/kevinbehr/home

Bleep Bleep

I had just finished almost ten solid minutes of uninterrupted work on my bosses’ pet project.  I had three ssh sessions and about 25 other windows going on my two monitors.

The phone caller-ID read Tom Lispon, an infosec Ninja, and a great friend of mine from Information Security.

“What’s the haps Tom?” I sighed into the phone.

“Dude, don’t sound so enthusiastic just for me.  I am just delivering the friendly pre-atom-bomb-clue-stick to you.”

“Oh, yeah?” I managed while putting the finishing touches on a script and saving it.

“Yeah, remember those old servers you found in the datacenter last week?”

As I secure copied the script to a server, su-d and ran it I remembered, Tom was speaking of two servers that had no visible identification and had multiple nics in them.

At the time, I thought they were some sort of experiment by the security team.  They certainly fit the mold of a skunk works project, made of retired server hardware and located in the back corner rack of row one of the old data center sans any physical asset tags and or labels of any kind. So naturally I queried my buddy Tom and acted like I was in the know just to spook him.

Turns out nobody owned these boxes and they had been here forever.

They were Microsoft boxes and still had their own domain setup on them, we had migrated to Active Directory eons ago so either these boxes were for some app that was not happy with AD or they were just missed during the migration.

“Yeah, I remember.  Did you ever figure out who they belonged to or even what they are doing in there?”

“Um, yeah, and boatloads more. So check this out, remember we couldn’t find them in any docs? Well turns out when we did that whole reverse merger years back they were the original Exchange and Outlook Web Access servers for the company we bought.”

I started to laugh one of those ‘this-is-not-really-funny-but-everyday-here-is-like-a-masters-program-in-how-not-to-manage-anything’ laughs, but then it hit me.

If the servers were from the company we bought in the reverse merger many of the accounts on it would have been from the now senior executives of our company. Holy crap, this was getting interesting.

“Tell me the boxes are dormant dude.  Please tell me.” I quietly prayed. 

The other reason I was sure that the boxes were a security project had to do with the fact the multiple network cards and their connections actually connected to several different networks.  I was pretty sure that one of the NICs plugged directly into a DMZ or non-firewalled VLAN.

“I wish I could, but they are still being used. Only by about a dozen accounts as far as I can tell”

“You found some creds to get in?”

“Yep, actually we didn’t need them.”

“What?  Was the console just logged in with no time out after like 4 years?”

“No, the boxes had all been owned for, like, many years, dude. They had reactivated and modified very handy backup-administrator accounts, removed the passwords, and gave them full administrator privileges”

This was getting worse by the minute.

“Looks like they used an old IIS hack and from there they just owned the box.  It’s got all kinds of tar files in a temp directory.  Including a nice rootkit and a bunch of tools to explore the inside network from the servers. Oh yeah we found some old CDC stuff and a old Back-Orifice app listening for connections.”

God, the thought of someone opening and closing the CD-ROM drive of the old servers in our datacenter, not to mention all of the data… Oh man.

“Well at least not many users are on the box, right Tom?

“True, but it is the entire executive management team minus like two of our original execs.  But that list includes the CEO, CFO, CMO, COO, Chief legal counsel, and several VPs.  This is not pretty my friend.”

“Indeed.  Have you told anyone yet?” I asked.

“I had to.  I scheduled a quick meeting with our new interim CIO.  He was less than pleased.  I guess he has been on the job all of about four days now.  Poor guy, I actually felt bad for him, unlike the last guy who let this happen on his watch.”

“Yeah, I hear you.”  I chimed in.

I could count the CIOs I had met or even read about that understood security let alone operations with just a couple of fingers.

“So you wanna hear the real kicker we found?” Tom baited.

“Probably not, but go ahead.  Tell me.”

“The Outlook Web App box had a capture app intercepting the usernames and passwords as everyone was logging in.”

“Oh man they were harvesting creds from that box.  Good thing we have a policy mandating different creds than your internal ones for external facing web apps like that”

“Well… not back then we didn’t.  All of the account names and passwords were owned for years. When you consider that people usually recycle three different passwords for mandatory passwords over the course of a year they have a good chunk of our execs creds. We are still trying to figure out what you can see from these boxes.  Even though they are not part of the Active Directory they have access to a lot of stuff via legacy domain trust relationships with other old stuff. ”

This whole scenario was getting out of control.  There would be no way to put a lid on this, as we would need to interrupt the senior exec’s email to resolve the problem.