Evasion and Countermeasures Techniques to Detect Dynamic Binary Instrumentation Frameworks

Dynamic Binary Instrumentation (DBI) is a dynamic analysis technique that allows arbitrary code to be executed when a program is running. DBI frameworks have started to be used to analyze malicious applications. As a result, different approaches have merged to detect and avoid them. Commonly referred to as split personality malware or evasive malware are pieces of malicious software that incorporate snippets of code to detect when they are under DBI framework analysis and thus mimic benign behavior. Recent studies have questioned the use of DBI in malware analysis, arguing that it increases the attack surface. In this paper, we examine the anti-instrumentation techniques that abuse desktop-based DBI frameworks and existing countermeasures to determine if it is possible to reduce the exploitable attack surface introduced by these DBI frameworks. In particular, we review the related literature to identify (i) the existing set of DBI framework evasion techniques and (ii) the existing set of countermeasures to avoid them. We also analyze and compare the taxonomies introduced in the literature, and propose a new taxonomy that expands and completes previous taxonomies. We also note some relevant issues and outline ways of future research in the use of DBI frameworks for security purposes

BAHK: Flexible Automated Binary Analysis Method with the Assistance of Hardware and System Kernel

To protect core functions, applications often utilize the countermeasure techniques such as antidebugging to avoid analysis by outsiders, especially the malware. Dynamic binary instrumentation is commonly used in the analysis of binary programs. However, it can be easily detected and has stability and applicability problems as it involves program rewriting and just-in-time compilation. This paper proposes a new lightweight analysis method for binary programs with the assistance of hardware features and the operating system kernel, named BAHK, which can automatically analyze the target program by stealth and has wide applicability. With the support of underlying infrastructures, this paper designs several optimization strategies and specific analysis approaches at instruction level to reduce the impact of fine-grained analysis on the performance of target program so that it can be well applied in practice. The experimental results show that the proposed method has good stealthiness, low memory consumption, and positive user experience. In some cases, it shows better analysis performance than the traditional dynamic binary instrumentation method. Finally, the real case studies further show its feasibility and effectiveness.