The Challenge

A large, multinational organization was alerted by US-CERT/FBI that it had been the source of a number of credit cards and details being leaked/sold on underground (carding) forums. After an initial investigation, the organization's security team discovered a compromised credit-card processing server but, having insufficient resources and skills in dealing with the incident, called in OSEC.

The Solution

OSEC sent a team of analysts, including Incident Response, Crisis Management, and Digital Forensics personnel to the organization's head office and data centers to deal with the incident. Once there, the team initiated full incident response based on the information supplied by the organization itself as well as law enforcement/authorities.

Planning - After The Fact

The first task was understanding what measures were in place to deal with the incident. Unfortunately while the organization had an incident response plan, it had not undertaken the first step of Incident Response - preparation. OSEC's incident response manager, along with the team, got to work coming up with a strategy: analyzing the available information, using it to understand the extent of the compromise, and the incident, and working out how to contain and eradicate it. All the while, information to the rest of the organization and the world at large had to be controlled, due to the possible legal and regulatory implications.

Detection And Analysis

Containment required understanding what data had been exfiltrated, and working back from there to the compromised resources, as well as examining the rest of the environment for other footholds that the attackers had. Quickly gaining an understanding of the network and segmentation, as well as rapidly implementing network behavioral analysis and performing content inspection between the payment processing infrastructure and external networks, OSEC detected connections back to command and control servers that were known to be operated by organized criminal elements ('carders'). From there, we started performing analysis of the compromised systems using forensics techniques to determine how and what vulnerabilities had been exploited to gain access, correlating that with available logging information, all the while monitoring network flows to both ensure that no additional card information was being exfiltrated for the purposes of understanding what machines were under their control, all without alerting the bad guys.

Within a short amount of time, OSEC determined that a third-party web application/site that was vulnerable to SQL injection had been initially compromised, and then used as a "base of operations" to penetrate further into the network, ultimately gaining access to the payment processing segments. By targeting administrators using social engineering attacks in combination with an Internet Explorer vulnerability, they had then stolen credentials that could be used to authenticate to payment processing servers, and utilized privilege escalation vulnerabilities on the servers themselves to harvest credit card numbers as they were being processed. In addition, they had installed customized malware that communicated with the command and control servers and exfiltrated data through encrypted tunnels, in bursts, to evade detection.

Containment And Eradication

OSEC then went about stopping the spread of the malware and compromise, and expelling the attackers from the network. Once we had determined that the malware installed would not respond negatively to loss of connectivity to command and control servers, we quickly: ensured the initial point of compromise (SQL injection) was corrected scanned for similar common vulnerabilities in externally-visible systems, and ensured any identified issues were corrected reset all relevant authentication credentials blocked the attackers at the network perimeter

We then set about isolating and cleaning each of the compromised hosts as quickly as we could, in coordination with IT personnel, to ensure that the processing systems were impacted as little as possible. In most cases, we were able to wipe hosts and perform recovery to ensure all traces of malware were eradicated, but a number of systems required manual cleaning, which we undertook with the relevant organizational resources, and initiated extensive monitoring to ensure no undetected issues remained.

Finally, once the full extent of the breach was understood - particularly what and how much data had been stolen, OSEC coordinated with PR and Legal personnel to manage client and other regulatory-body notifications.

Post-Incident Activity

Once the immediate incident had been dealt with, OSEC performed a post-mortem analysis of the incident, the organization's response, and compared it to OSEC's internally-developed IR processes, procedures, and frameworks to identify what needed to be done to ensure IR, vulnerability management, as well as overall Information Security Management process and procedures were improved such that future incidents would be minimized We then sat down with the various stakeholders in the organization that had been involved and discussed the incident and response, explaining the relevant issues, identifying organizational problems that also needed to be corrected, as well as future strategies for avoiding incidents and dealing with them when they occurred, communicating our recommended incident response strategy and implementation to the organization's senior levels.

Having reviewed OSEC's recommendations, the organization then asked us back to assist with implementing them. Over a 3 month period, OSEC led a number of efforts, including implementing protection mechanisms at the host, application, and network layers; establishing a functioning vulnerability management within the overall information security management program, verifying processes, helping with staffing and training, and performing incident response drills to test the final product.

The Result

Twelve months after implementing the recommendations, and achieving a practical incident response program, the organization has not suffered any subsequent breaches. In addition, it has gained the assurance, through incident response drills, that should a breach occur, response will be swift and effective.

Back to case studies.

Copyright ©2013. All Rights Reserved.