Dexter: New Point-Of-Sale Malware

Seculert has discovered a new piece of malware called Dexter which is designed to infect retail sales workstations and steal credit card information.

How it operates:

• It scans the process list of an infected system and looks for Point-Of-Sale software.
• It scans the memory segments of the POS software and pulls out the credit card data.
• Communicates data back to a C&C server.

Infection vector:

• Looks like its targets Windows systems, including Window Server systems. 
• 50% of the infected systems are Windows XP
• Most targets are in western countries.
• How a system gets infected is still unknown.

Seculert is suggesting that because Windows Server systems are getting infected then drive-by downloads or web-based social engineering is not likely.  They're also theorizing that the credit card data is being used to clone credit cards.
 
SpiderLabs has a great analysis of how the C&C communication actually works.  The credit card data and other information is base-64 encoded and XOR encrypted and sent to the C&C server.  It looks like there are several domain names involved as the C&C servers.  The server sends back instructions, again base-64 encoded and XOR encrypted, in a cookie.

Volatile Labs had a list of domain names that the program uses.  They're just .com names of random jibberish, and they can probably change frequently.  But look out for domains like this going through your firewall:

• 11e2540739d7fbea1ab8f9aa7a107648.com
• 7186343a80c6fa32811804d23765cda4.com
• e7dce8e4671f8f03a040d08bb08ec07a.com
• e7bc2d0fceee1bdfd691a80c783173b4.com
• 815ad1c058df1b7ba9c0998e2aa8a7b4.com
• 67b3dba8bc6778101892eb77249db32e.com
• fabcaa97871555b68aa095335975e613.com

The good news is that if it's just Windows systems being infected then you shouldn't need to worry if you use stand-alone terminals with a direct Ethernet connection.  Chase Paymentech uses these a fair amount, and I'm pretty certain they're not using Windows on those little devices.


Here are the original Seculert posting and some additional research from Trustwave SpiderLabs and Volatile Labs.

Home Wireless Networks

A question I get asked often is how to setup multiple wireless access points in a house to get better wireless coverage.  When your Internet provider sets up your connection, it makes sense to put your wireless router at the same physical location.  If you're in an apartment or small bungalow that's generally not a problem for wireless coverage.  If you have a bigger living space then coverage becomes more of an issue.  Unfortunately most wireless routers only come with instructions on how to set them up as the only wireless router.  To make wireless routers function together there are several things to keep in mind.

We'll use this diagram for our discussions:

First, setup each wireless router with a different IP address on that same network.  If you're new to networking, then stick with something simple like in the diagram.  The first router would be 192.168.0.1, then second will be 192.168.0.2, and so on...


Second, each wireless router must use a separate wireless channel.  This is very important.  A lot of people think the complete opposite and try to setup all the wireless routers on the same channel.  When the routers are on the same channel, they just interfere with each other and you'll be lucky if you get anything working.  Your wireless devices are made so they can easily switch channels on the same wireless network without any dropped connections.

Third, each wireless router must use the same wireless security settings. The whole point of this is so you don't lose your wireless connection as you wander around the house.  If you have different security settings, you need separate SSIDs and you'd be dropping your connection all the time when you move around.  So make sure your security mode, algorithms, keys, key renewals, and beacon settings are all the same, whatever they might be.  (I recommend WPA2 with AES)

Fourth, only one routers should run DHCP.  I recommend the router that is connected to your Internet connection for this function.  The Dynamic Host Configuration Protocol is used to give unique IP addresses to your various devices.  If you have multiple DHCP servers and you don't set them up properly then you can end up with IP address conflicts or dropped connections as you move around the house.  When setting up DHCP, make sure to set a range that doesn't overlap with any of the router IP addresses you set from the first step.  Use a range like 192.168.0.100 to 192.168.0.200 should avoid any problems.

Fifth, there should only be one firewall.  The only firewall you should have on your network is the one that connects to the Internet connection.  If you can, I recommend disabling the firewalls on all the other routers.  While the last step will ensure there are no problems if you can't turn off all the firewalls, it's better if you do.

Lastly, only your router that connects to the Internet should have its "Internet" or "WAN" port connected to anything.  For any of the other routers, don't plug anything into the Internet ports.  When you connect your routers together, always use the "LAN" ports.  This keeps them all on the same network and prevents any firewalls from interfering with communications between your local devices.  You will still have the one firewall you setup in step five to protect you from anything on the Internet.

IPv6: When will it really happen?

I was looking through some old drafts in this blog, stuff I had never posted, and noticed one I had written about IPv6 in 2010.  I had started to learn about the details of IPv6 in 2010.  I know it's been coming for a while, but there really hasn't been a lot of pressure.  Need perhaps, but not pressure.

IPv6 is a scary thing to most people.  People are bad enough with the 4 octets of IPv4, but even some of the shorter IPv6 addresses can be downright frightening to them:

4102:44F0:7F0::1/64

This would invoke a WTF from most people.  Thankfully those types of  people will never have
to deal with this.  DNS in all its wonder and glory will be needed more than ever for the ordinary person.

The real question is when will we actually use IPv6?  I know why we need it, but using it is another thing.  I hear from my Windows admin coworkers that when they call in for a support problem to Microsoft and there is a suspected network issue, one of the first things they want is for you to turn off IPv6 (if you can).  Not a thrilling endorsement.

Even worse is that one of the fastest growing consumers of IPv6, mobile phones.  Mobile phones are not really ready for IPv6.  Last check, my Blackberry doesn't support IPv6.  Supposedly Apple has been supporting it since iOS version 4.1, and Android since 4.2 but I hear it's a bit buggy and not actually useful.  So, right now IPv6 is not deployed on a lot of carriers simply because the handsets aren't ready.

So if the mobile operating systems haven't completely gotten their act together, then how long will it be before developers actually start worrying about IPv6?  It's sad to say that we're still a few years away.  I know IPv6 launched this June 6th but I believe the adoption rate is only about 1%.  Some Internet transit providers I've called don't even offer IPv6 feeds and a lot of the residential ISPs still don't offer IPv6.

Packets Don't Lie

My nickname was given to me a while back, thepacketmaster.  It was because I love using packet tracers to solve problems.  I sometimes wonder if I'm unique in this sense or if other network/security admins use packet tracers as often as I do.  Time and again I solve problems using packet dumps when nothing else can work.  It's not that using packet tracers is an action of last resort.  It's simply that you can't argue with a packet traces.

Packets don't lie. I'm not saying that people are lying when they come to me with problems that they can't resolve. However, I've seen far too many instances where an administrator of an application, database, or server will come to me with a problem that is based on a false premise given to them by a poorly written error message.  More often than not it starts with, "I think there's a problem with the network".  Fair enough, it could be there is a problem with the network.  We in the network/security realm are not without problems and I never have a problem admitting that.  In my case I'd have to say the security side comes up more than the network side, simply because I prefer firewall rules that are as strict as possible and deny everything that isn't necessary.  Most of the other administrators aren't necessarily appreciative of that fact.

After making sure I understand the problem and checking that it isn't related to the network or security systems, I move onto trying to help the person understand what other problems it could be.  In many instances it could simply be that they've gotten some cryptic error message from a piece of software that says "Network error".  The bane of my existence are developers that write software with generic error messages that always mention the network.  Let this be a lesson to developers to always be specific in your error messages.  Instead of saying "Network error" put something useful like "Cannot bind to port" or "Unable to establish TCP connection to remote system".  Anything but "Network error"!

Once I have a feel for the problem I often try to suggest what other causes it could be.  In my previous lives, before I was a Network and Security Architect, I have been a UNIX admin, Network admin, Windows admin, Application admin, Security admin, database admin, and even a backup admin.  I've written in assembler, C, Perl, C++, php, bash, BASIC and COBOL.  In short, I have a holistic view of IT.  That's not to say I know it all.  Nobody, NOBODY, ever knows it all.  Those that claim to are full-of-it.

If a solution to the problem is still eluding detection, that's generally when I fallback to the packet capture.  This can tell so many things about what's actually going on.  Packet captures are mind altering experiences.  Previous conclusions that seemed unbreakable can be completely shattered or reinforced by a packet capture.  Problems and solutions never even imagined can be envisioned with complete clarity.  Packet captures can be a humbling experience.

The great thing is that nowadays you don't even need help interpreting what a packet capture says.  Tools like Wireshark are so user friendly that I can put it up on a projector to an entire room of people that have never seen a packet capture in their lives, show them what's I'm seeing and what is happening, and they'll understand the problem completely.

Fun with Gingerbread

Merry Christmas to all!  Work recently sponsored a gingerbread contest.   I was immediately interested in building a Cisco Nexus or a Juniper SRX.  I was a little disappointed to find out the  kits were pre-made gingerbread houses.  How much fun is it to just decorate a house that's the same as all the other houses?  (At least that was my thought).  Other coworkers did a great job of decorating their houses, very fancy and neat.  I have to say the attempt by myself and several accomplices was less that neat, but we made up for it with originality.  I will also caveat this by saying we only used the materials we were given.  Many people brought in "extra" decorations and frosting, which I feel went against the spirit of the competition, but I'll defer to the judges on that... (Next year, I'll get a freestyle kit and rock the competition!)

So without further ado, I give you the Gingerbread wireless router...

QRadar detecting ZeroAccess Trojan

Many of my co-workers have been witness to my recent love affair with QRadar.  It's an easy to use, yet powerful SIEM that allows a security team to operate very efficiently.  When I first started using it my hope was to detect possible hacking attempts into the system.  I envisioned seeing probes coming from the Internet to our systems, attempted logins, and possibly exploits.  While I did see all this, QRadar pointed out much more just by simply examining our firewall logs.

My general rule for configuring firewall logs is to log every connection and every event.  While this may seem to be overkill for some, experience has shown me over and over again that this level of logging proves invaluable.  How many times have I been able to answer the questions of application or system administrators regarding why something is not working.  "Let me check the firewall logs", "Yes, I can see that traffic going through, but I don't see any return traffic", "Must be a connection blocked on their firewall", "Oh, we didn't have a rule for that", etc, etc.

With this level of detail being given to QRadar it was able to do what it does best, correlate and alert on possible threats.  I was interested to see a couple of workstations that were trying to establish connections to what seemed to be random hosts on the Internet.  Furthermore, it seemed to be attempting connections to just certain UDP ports, namely 16471. I was able to get hold of one of these workstations and track down what process was using these ports.  It seemed to be the explorer.exe process.

A simple Google of "16471 and explorer.exe" turned up several sites that discussed the ZeroAccess trojan.  One of the more helpful sites was Kindsight.  The research material mentions the malware sends out packets that are 16 bytes in size, and filled with encrypted data.  A packet capture confirmed that this is what was happening.  

We had a difficult time actually finding the infection files on each computer.  The location of the malware was very random.  Symantec had some recommendations, as did other sites, but we found cleaning the infection to be difficult in some cases.  McAfee system would sometimes catch it and prevent or clean the infection, and in some cases not.  Sophos had some success, while Spybot Search and Destroy was ineffective.  We were never able to determine why certain systems would be impossible to clean, and required the "nuke from orbit" response to rid the system of the infection.

In the end, we were able to clean all our infected workstations.  We wrote a rule in QRadar to send an alert every time the telltale traffic was seen, and an email automatically alerted our friends in Helpdesk.  Thanks to QRadar we were able to detect a piece of malware that nothing else was catching.

FireEye Update: A class act

Companies live and die by the customer base.  Too many times I've seen a company with a great product but poor customer support, and eventually the product will just fade away.  (See yesterday's post on HP TippingPoint for an example of a downward spiral).  Those companies that flourish are the ones with a good product and back it up with great customer service.

FireEye is one such company that has a good product and great customer service.  After some recent hardware issues with the product,  FireEye took no chances with the network stability.  They sent us a brand new unit to replace the failed unit.  When they realized they didn't have our exact model in stock current,apparently because it's so popular, they went above-and-beyond by offering a better replacement unit that was a slight upgrade (redundant hard drive unit).  This ensured we were up and running as fast as possible.

The support team was in contact with me daily, if not more, to ensure that every was under control and proceeding as we needed.  The replacement device was delivered quickly, and FireEye offered to answer any questions I might have and provide any documentation I might be missing.  Because we couldn't simply swap out the hard drive of our old unit into the replacement unit, due to the upgrade, they offered their services to assist with the migration.

The helpful service didn't just stop at their support team.  The account rep and reseller both got involved to help sooth management's worries about their investment in the product.  While some techies may not see this as necessary, there is no doubt that Layer 8 (The political layer) is alive and well in the network, and we are ultimately answerable to management.  So I appreciate the account rep stepping up and answering all the tough questions.

FireEye is a perfect example of a company that will do well.  Their product fills an important roll in the security landscape, catching malware threats that other appliances don't catch.  Rather than just coasting on a great product, they listen to their customers about their needs, because in the end you can't sell a product if nobody wants it.  I have every confidence that the new unit will be back inline and blocking malware (after the Christmas freeze).  For now, it's just been switched over to tap mode, which pretty much gives us the same protection.


Thank you FireEye!


P.S. Just in case anyone was wondering, FireEye's great response was not influenced by my earlier post.  They jumped on this without any prompting from me.  Another good sign!

TippingPoint IPS: Yes, we've reached the tipping point

TippingPoint was once considered one of the best network IPS.  Gartner's Magic Quadrant for Network IPS of 2008 and 2009 put it in the the top right quadrant, meaning the product was full of features and was backed by a solid company.  The 2009 Magic Quadrant showed it surpassed every other network IPS.  While I don't put 100% faith in Gartner, it is a useful source of information.

My first experience with TippingPoint was in a proof-of-concept bake-off between TippingPoint and McAfee.  TippingPoint boldly said their IPS could be put inline and with the base profiles would provide valuable network protection without interrupting network traffic.  I was impressed with the features, protection and GUI.  McAfee's GUI was painful at the time and it had some other things that made the TippingPoint preferable.  So TippingPoint won.

TippingPoint served us well enough but since the HP acquisition things have been going down hill. Gartner has downgraded TippingPoint to the point where they are almost out of the top right quadrant.  This isn't because of HP's inability to execute (although it should be).  This is because when compared with other network IPS they are falling regarding capabilities.

Nowadays, I consider TippingPoint to be mud and I place the responsibility solely on HP.  At the start of 2012 they put out a new version of the TippingPoint management console. It was a big disaster to say the least.  TippingPoint support said in an email to customers, "At this time we recommend that you do not upgrade to Security Management System (SMS) 3.3.  Our customers have reported several issues with this release during and after the upgrade process.   For those customers who have already upgraded to SMS 3.3 we are working on an additional patch to address the issues"  This situation didn't resolve itself for months.

Then came the time for our support renewal.  We asked our reseller for a support quote a full two months before the end of the contract.  We then spent the next few weeks following up with the reseller only to be told that HP wasn't getting back to them with the support quote.   They kept hounding HP for a quote and received silence.  On the off chance that the reseller was having issues with HP, we tried another reseller only to find the same story.  We became concerned that HP, with all its other business woes, that it would do something unfortunate like announce they were discontinuing TippingPoint. Finally, a week before our support was supposed to expire, we received quotes from both reseller.

The lesson to vendors is this: Support contracts are just paperwork so there shouldn't be any huge delays in renewing... If you delay giving your customers a quote, they'll get worried that something is wrong and go elsewhere.  This is especially true when your product is not top of the heap anymore and the competition is cheaper (i.e. Palo Alto  firewalls).

FireEye: An helpful product that needs to grow up

When I first heard of FireEye I thought this was an ideal product to help combat zero-day malware infections.  It takes a snapshot of user downloads such as applications, PDFs, and other things that might be capable of “running” on a system.  It runs them in an unpatched virtual machine to see what malicious activities take place, and reports and blocks any malware trying to call back to command-and-control.   You can simply hang it off your network in monitor mode, or you can put it inline so it can block traffic when needed.  It will catch things that no other products will catch.

It can also bring down your network at random times of the day.  This isn’t an issue of misconfiguration, or coming under attack by a DDoS.  This is a known flaw in the hardware that the vendor has recently discovered.  The way it was described to me, it sounds almost like a quality control issue.  The FireEye network card that is used to tap into the network traffic has a problem.  From my experience, this problem is not immediately evident, but can become worse over time.  It’s almost like a circuit on the network card is slowly degrading over time, gradually causing the problem to occur more frequently.

This hardware issue results in the FireEye randomly dropping the network link.  If you saw enough potential in the FireEye to put it inline of your network traffic, this means your network link will drop for a brief moment.  If you put the FireEye in the recommended location directly between your firewall’s internal interfaces and the network switches, this means your firewall will lose its network connection.  You have a firewall cluster, so you’re okay with it handling the failover?  Hopefully you don’t have a Cisco ASA, with version 8.2 or less, and running a dynamic routing protocol.  If this is the case, your firewall may not failover so gracefully and you’ll be left with a longer outage until you get the Cisco’s routing protocol stabilized.  This is a known caveat in these versions of Cisco ASA, best to upgrade to the later versions soon.

The most unfortunate part is FireEye’s reactions:  Rather than reaching out to all their customers to mention there is a problem, very likely with all their equipment, they have been very quiet.  To compound matters, this is not an easy situation to detect.  FireEye does not send operational error messages to syslog, like network links going up or down.  It has to be dug up by FireEye’s support staff using specialized license keys that allow them deeper access into the machine.  Since the customer can’t see the FireEye issues this can lead to a lot of frustration because you may suspect other devices like firewalls and switches that are putting out link alerts and showing signs of some sort of problem.   This inability to send basic operational messages shows a lack of maturity in the product.  Yes, it is a very new product, but these sort of operational features are around in any enterprise products.  It shouldn’t have been left out of the FireEye, especially with its critical positioning in the network right behind the firewalls.

I was worried that this was the end of a wonderful product.  Word from FireEye was that there was  no workaround for this.  FireEye's best feature, preventing malware callbacks, would becomes useless.  You can continue to use it in a monitor mode, but that means your workstations will be getting infected more often.  At least you’ll known it has a zero-day infection, and that will  be enough to save the FireEye by some measure.  

Thankfully word has come down from the FireEye management that they do have replacement hardware to solve the problem.  FireEye has made a really useful product that stands alone in its capabilities.  If you suspect the FireEye might be causing network issues, I highly recommend contacting their support team immediately.  There are still far too many users willing to click on things they really shouldn’t, and you'll need FireEye's protection.