Malware variant detection using similarity search over sets of control flow graphs

Cesare, Silvio and Xiang, Yang 2011, Malware variant detection using similarity search over sets of control flow graphs, in TRUSTCOM 2011 : International Conference on Trust, Security and Privacy in Computing and Communications, IEEE, [Changsha, China], pp. 181-189.

Attached Files
Name Description MIMEType Size Downloads

Title Malware variant detection using similarity search over sets of control flow graphs
Author(s) Cesare, Silvio
Xiang, Yang
Conference name International Conference on Trust, Security and Privacy in Computing and Communications (10th : 2011 : Changsha, China)
Conference location Changsha, China
Conference dates 16-18 Nov. 2011
Title of proceedings TRUSTCOM 2011 : International Conference on Trust, Security and Privacy in Computing and Communications
Editor(s) [Unknown]
Publication date 2011
Conference series IEEE International Conference on Trust, Security and Privacy in Computing and Communications
Start page 181
End page 189
Total pages 9
Publisher IEEE
Place of publication [Changsha, China]
Keyword(s) computer security
malware classification
static analysis
control flow
structuring
decompilation
Summary Static detection of polymorphic malware variants plays an important role to improve system security. Control flow has shown to be an effective characteristic that represents polymorphic malware instances. In our research, we propose a similarity search of malware using novel distance metrics of malware signatures. We describe a malware signature by the set of control flow graphs the malware contains. We propose two approaches and use the first to perform pre-filtering. Firstly, we use a distance metric based on the distance between feature vectors. The feature vector is a decomposition of the set of graphs into either fixed size k-sub graphs, or q-gram strings of the high-level source after decompilation. We also propose a more effective but less computationally efficient distance metric based on the minimum matching distance. The minimum matching distance uses the string edit distances between programs' decompiled flow graphs, and the linear sum assignment problem to construct a minimum sum weight matching between two sets of graphs. We implement the distance metrics in a complete malware variant detection system. The evaluation shows that our approach is highly effective in terms of a limited false positive rate and our system detects more malware variants when compared to the detection rates of other algorithms.
ISBN 145772135X
9781457721359
Language eng
Field of Research 080503 Networking and Communications
Socio Economic Objective 890201 Application Software Packages (excl. Computer Games)
HERDC Research category E1 Full written paper - refereed
Copyright notice ©2011, IEEE
Persistent URL http://hdl.handle.net/10536/DRO/DU:30042195

Document type: Conference Paper
Collection: School of Information Technology
Connect to link resolver
 
Unless expressly stated otherwise, the copyright for items in DRO is owned by the author, with all rights reserved.

Versions
Version Filter Type
Citation counts: Scopus Citation Count Cited 4 times in Scopus
Google Scholar Search Google Scholar
Access Statistics: 83 Abstract Views, 6 File Downloads  -  Detailed Statistics
Created: Tue, 14 Feb 2012, 15:14:04 EST

Every reasonable effort has been made to ensure that permission has been obtained for items included in DRO. If you believe that your rights have been infringed by this repository, please contact drosupport@deakin.edu.au.