Malware Defense

Effective and Efficient Malware Detection at the End Host

C. Kolbitsch, P. Milani, C. Kruegel, E. Kirda, X. Zhou and X. Wang. In Proceeding of the USENIX Security Symposium (Security 2009)
Malware is one of the most serious security threats on the Internet today. In fact, most Internet problems such as spam e-mails and denial of service attacks have malware as their underlying cause. That is, computers that are compromised with malware are often networked together to form botnets, and many attacks are launched using these malicious, attacker-controlled networks. With the increasing significance of malware in Internet attacks, much research has concentrated on developing techniques to collect, study, and mitigate malicious code.

Without doubt, it is important to collect and study malware found on the Internet. However, it is even more important to develop mitigation and detection techniques based on the insights gained from the analysis work. Unfortunately, current host-based detection approaches (i.e., anti-virus software) suffer from ineffective detection models. These models concentrate on the features of a specific malware instance, and are often easily evadable by obfuscation or polymorphism. Also, detectors that check for the presence of a sequence of system calls exhibited by a malware instance are often evadable by system call reordering. In order to address the shortcomings of ineffective models, several dynamic detection approaches have been proposed that aim to identify the behavior exhibited by a malware family. Although promising, these approaches are unfortunately too slow to be used as real-time detectors on the end host, and they often require cumbersome virtual machine technology.

In this paper, we propose a novel malware detection approach that is both effective and efficient, and thus, can be used to replace or complement traditional anti-virus software at the end host. Our approach first analyzes a malware program in a controlled environment to build a model that characterizes its behavior. Such models describe the information flows between the system calls essential to the malware’s mission, and therefore, cannot be easily evaded by simple obfuscation or polymorphic techniques. Then, we extract the program slices responsible for such information flows. For detection, we execute these slices to match our models against the runtime behavior of an unknown program. Our experiments show that our approach can effectively detect running malicious code on an end user’s host with a small overhead.

AGIS: Automatic Generation of Infection Signatures

Z. Li, X. Wang, Z. Liang and M. K. Reiter. In Proceedings of the 38th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN 2008)
An important yet largely uncharted problem in malware defense is how to automate generation of infection signatures for detecting compromised systems, i.e., signatures that characterize the behavior of malware residing on a system. To this end, we develop AGIS, the first host-based technique that detects infections by novel malware and automatically generates an infection signature of the malware. AGIS monitors the runtime behavior of suspicious code according to a set of security policies to detect a previously undetected infection, and then identifies its characteristic behavior in terms of system or API calls. AGIS then statically analyzes the corresponding executables to extract the instructions important to the infection’s mission. These instructions can be used to build a template for a static-analysis-based scanner, or a regular-expression signature for legacy scanners. AGIS also detects encrypted malware and generates a signature from its plaintext decryption loop. We implemented AGIS on Windows XP and evaluated it against real-life malware, including keyloggers, mass-mailing worms, and a well-known mutation engine. The experimental results demonstrate the effectiveness of our technique in detecting new infections and generating high-quality signatures.

PRECIP: Towards Practical and Retrofittable Confidential Information Protection

X. Wang, Z. Li, N. Li and J. Choi.In Proceedings of the 15th Annual Network and Distributed System Security Symposium (NDSS 2008)
A grand challenge in information protection is how to preserve the confidentiality of sensitive information under spyware surveillance. This problem has not been well addressed by the existing access-control mechanisms which cannot prevent the spyware already in a system from monitoring an authorized party’s interactions with sensitive data. Our answer to this challenge is PRECIP, a new security policy model which takes a first step towards practical and retrofittable confidential information protection. This model is designed to offer efficient online protection for commercial applications and operating systems. It intends to be retrofitted to these applications and systems without modifying their code. To this end, PRECIP addresses several practical issues critical to containing spyware surveillance, which however are not well handled by the previous work in access control and information-flow security. Examples include the models for human input devices such as keyboard whose sensitivity level must be dynamically determined, other shared resources such as clipboard and screen which must be accessed by different processes, and the multitasked processes which work on public and sensitive data concurrently. We applied PRECIP toWindows XP to protect the applications for editing or viewing sensitive documents and browsing sensitive websites. We demonstrate that our implementation works effectively against a wide spectrum of spyware, including keyloggers, screen grabbers and file stealers. We also evaluated the overheads of our technique, which are shown to be very small.

Spyshield: Preserving privacy from spyware add-ons

Z. Li, X. Wang and J. Choi.In Proceedings of the Recent Advance in Intrusion Detection (RAID 2007)
Spyware infections are becoming extremely pervasive, posing a grave threat to Internet users’ privacy. Control of such an epidemic is increasingly difficult for the existing defense mechanisms, which in many cases rely on detection alone. In this paper, we propose SpyShield, a new containment technique, to add another layer of defense against spyware. Our technique can automatically block the visions of untrusted programs in the presence of sensitive information, which preserves users’ privacy even after spyware has managed to evade detection. It also enables users to avoid the risks of using free software which could be bundled with surveillance code. As a first step, our design of SpyShield offers general protection against spy add-ons, an important type of spyware. This is achieved through enforcing a set of security policies to the channels an add-on can use to monitor its host application, such as COM interfaces and shared memory, so as to block unauthorized leakage of sensitive information.We prototyped SpyShield under Windows XP to protect Internet Explorer and also evaluated it using real plug-ins. Our experimental study shows that the technique can effectively disrupt spyware surveillance in accordance with security policies and introduce only a small overhead.

Packet Vaccine: Black-box Exploit Detection and Signature Generation

X. Wang, Z. Li, J. Xu, M. Reiter, C. Kil and J. Choi. In Proceedings of the 13th ACM Conference on Computer and Communication Security (CCS 2006).
In biology, a vaccine is a weakened strain of a virus or bacterium that is intentionally injected into the body for the purpose of stimulating antibody production. Inspired by this idea, we propose a packet vaccine mechanism that randomizes address-like strings in packet payloads to carry out fast exploit detection, vulnerability diagnosis and signature generation. An exploit with a randomized jump address be- haves like a vaccine: it will likely cause an exception in a vulnerable program's process when attempting to hijack the control flow, and thereby expose itself. Taking that exploit as a template, our signature generator creates a set of new vaccines to probe the program, in an attempt to uncover the necessary conditions for the exploit to happen. A signature is built upon these conditions to shield the underlying vulnerability from further attacks. In this way, packet vaccine detects and alters exploits in a black-box fashion, i.e., avoiding the expense of tracking the program's execution flow. We present the design of the packet vaccine mechanism and an example of its application. We also describe our proof-of-concept implementation and the evaluation of our technique using real exploits.

Side Channel

Thwarting Wi-Fi Side-Channel Analysis through Traffic Demultiplexing

F. Zhang, W. He, Y. Chen, Z. Li, X. Wang, S. Chen and X. Liu. Submitted to IEEE Transactions on Dependable and Secure Computing.

Peeping Tom in the Neighborhood: Keystroke Eavesdropping on Multi-user Systems

K. Zhang and X. Wang. In Proceedings of the USENIX Security Symposium (Security 2009).
A multi-user system usually involves a large amount of information shared among its users. The security implications of such information can never be underestimated. In this paper, we present a new attack that allows a malicious user to eavesdrop on other users’ keystrokes using such information. Our attack takes advantage of the stack information of a process disclosed by its virtual file within procfs, the process file system supported by Linux. We show that on a multi-core system, the ESP of a process when it is making system calls can be effectively sampled by a "shadow" program that continuously reads the public statistical information of the process. Such a sampling is shown to be reliable even in the presence of multiple users, when the system is under a realistic workload. From the ESP content, a keystroke event can be identified if they trigger system calls. As a result, we can accurately determine inter-keystroke timings and launch a timing attack to infer the characters the victim entered.

We developed techniques for automatically analyzing an application's binary executable to extract the ESP pattern that fingerprints a keystroke event. The occurrences of such a pattern are identified from an ESP trace the shadow program records from the application's runtime to calculate timings. These timings are further analyzed using a Hidden Markov Model and other public information related to the victim on a multi-user system. Our experimental study demonstrates that our attack greatly facilitates password cracking and also works very well on recognizing English words.

Authorization failure

Towards Automatic Reverse Engineering of Software Security Configuration

R. Wang, X. Wang, K. Zhang and Z. Li.Proceedings of the 15th ACM Conference on Computer and Communication Security (CCS 2008)
The specifications of an application’s security configuration are crucial for understanding its security policies, which can be very helpful in security-related contexts such as misconfiguration detection. Such specifications, however, are often ill-documented, or even close because of the increasing use of graphic user interfaces to set program options. In this paper, we propose ConfigRE, a new technique for automatic reverse engineering of an application’s access-control configurations. Our approach first partitions a configuration input into fields, and then identifies the semantic relations among these fields and the roles they play in enforcing an access control policy. Based upon such knowledge, ConfigRE automatically generates a specification language to describe the syntactic relations of these fields. The language can be converted into a scanner using standard parser generators for scanning configuration files and discovering the security policies specified in an application. We implemented ConfigRE in our research and evaluated it against real applications. The experiment results demonstrate the efficacy of our approach.

Privacy Preserving

Panalyst: Privacy-Aware Remote Error Analysis on Commodity Software

R. Wang, X. Wang and Z. Li.In Proceedings of the 17th USENIX Security Symposium (Security 2008)
Remote error analysis aims at timely detection and remedy of software vulnerabilities through analyzing runtime errors that occur on the client. This objective can only be achieved by offering users effective protection of their private information and minimizing the performance impact of the analysis on their systems without undermining the amount of information the server can access for understanding errors. To this end, we propose in the paper a new technique for privacy-aware remote analysis, called Panalyst. Panalyst includes a client component and a server component. Once a runtime exception happens to an application, Panalyst client sends the server an initial error report that includes only public information regarding the error, such as the length of the packet that triggers the exception. Using an input built from the report, Panalyst server performs a taint analysis and symbolic execution on the application, and adjusts the input by querying the client about the information upon which the execution of the application depends. The client agrees to answer only when the reply does not give away too much user information. In this way, an input that reproduces the error can be gradually built on the server under the client’s consent. Our experimental study of this technique demonstrates that it exposes a very small amount of user information, introduces negligible overheads to the client and enables the server to effectively analyze an error.