Home » Articles posted by kaiser

Author Archives: kaiser

DIRECT : A Transformer-based Model for Decompiled Identifier Renaming

Vikram Nitin, Anthony Saieva, Baishakhi Ray and Gail Kaiser. DIRECT : A Transformer-based Model for Decompiled Identifier Renaming. 1st Workshop on Natural Language Processing for Programming (NLP4Prog), co-located with ACL-IJCNLP, Virtual, August 2021, pp. 48-57. http://dx.doi.org/10.18653/v1/2021.nlp4prog-1.6.

Decompiling binary executables to high-level code is an important step in reverse engineering scenarios, such as malware analysis and legacy code maintenance. However, the generated high-level code is difficult to understand since the original variable names are lost. In this paper, we leverage transformer models to reconstruct the original variable names from decompiled code. Inherent differences between code and natural language present certain challenges in applying conventional transformer-based architectures to variable name recovery. We propose DIRECT, a novel transformer-based architecture customized specifically for the task at hand. We evaluate our model on a dataset of decompiled functions and find that DIRECT outperforms the previous state-of-the-art model by up to 20%. We also present ablation studies evaluating the impact of each of our modifications. We make the source code of DIRECT available to encourage reproducible research.

@inproceedings{direct,
author = {Vikram Nitin and Anthony Saieva and Baishakhi Ray and Gail Kaiser},
title = {{DIRECT : A Transformer-based Model for Decompiled Identifier Renaming}},
booktitle = {{1st Workshop on Natural Language Processing for Programming (NLP4Prog), co-located with ACL-IJCNLP}},
month = {August},
year = {2021},
pages = {48-57},
location={Virtual},
url = {http://dx.doi.org/10.18653/v1/2021.nlp4prog-1.6},
}

Metamorphic Detection of Repackaged Malware

Shirish Singh and Gail Kaiser. Metamorphic Detection of Repackaged Malware. 6th International Workshop on Metamorphic Testing (MET), co-located with ICSE, Virtual, June 2021, pp. 9-16. https://doi.org/10.1109/MET52542.2021.00009

Machine learning-based malware detection systems are often vulnerable to evasion attacks, in which a malware developer manipulates their malicious software such that it is misclassified as benign. Such software hides some properties of the real class or adopts some properties of a different class by applying small perturbations. A special case of evasive malware hides by repackaging a bonafide benign mobile app to contain malware in addition to the original functionality of the app, thus retaining most of the benign properties of the original app. We present a novel malware detection system based on metamorphic testing principles that can detect such benign-seeming malware apps. We apply metamorphic testing to the feature representation of the mobile app, rather than to the app itself. That is, the source input is the original feature vector for the app and the derived input is that vector with selected features removed. If the app was originally classified benign, and is indeed benign, the output for the source and derived inputs should be the same class, i.e., benign, but if they differ, then the app is exposed as (likely) malware. Malware apps originally classified as malware should retain that classification, since only features prevalent in benign apps are removed. This approach enables the machine learning model to classify repackaged malware with reasonably few false negatives and false positives. Our training pipeline is simpler than many existing ML-based malware detection methods, as the network is trained end-to-end to jointly learn appropriate features and to perform classification. We pre-trained our classifier model on 3 million apps collected from the widely-used AndroZoo dataset. 1 We perform an extensive study on other publicly available datasets to show our approach’s effectiveness in detecting repackaged malware with more than 94% accuracy, 0.98 precision, 0.95 recall, and 0.96 F1 score.

@INPROCEEDINGS{9477656,
author={Singh, Shirish and Kaiser, Gail},
booktitle={IEEE/ACM 6th International Workshop on Metamorphic Testing (MET), co-located with ICSE},
title={{Metamorphic Detection of Repackaged Malware}},
year={2021},
month = {June},
volume={},
number={},
pages={9-16},
location={Virtual},
url = {https://doi.org/10.1109/MET52542.2021.00009},
}

BASTION (Binary AnalysiS and TransformatION)

Seeking students interested in static, load-time and dynamic program analysis and manipulation of C/Linux binaries for layer-flattening optimizations and offloading segments to hardware accelerators. Must be familiar with ELF format and DWARF metadata, and be comfortable working with x86 and ARM instruction sets.

Contact Professor Gail Kaiser to discuss opportunities: kaiser@cs.columbia.edu.

Outreach Service

Are you an unpaid volunteer for a non-profit organization in the New York City area? Have you completed COMS W4156 Advanced Software Engineering (or equivalent) with grade B+ or higher? Does your work at the non-profit (or could your work there) leverage your software engineering skills to develop and maintain software important for the non-profit’s mission (not just their website)? Would you like to receive academic credit for your work (in 3998, 4901, or 6901)?

Contact Professor Gail Kaiser to discuss opportunities: kaiser@cs.columbia.edu.

Replay without Recording of Production Bugs for Service Oriented Applications

Presented by Jonathan Bell at 33rd ACM/IEEE International Conference on Automated Software Engineering (ASE) in September 2018. https://doi.org/10.1145/3238147.3238186.

Software available at https://github.com/Programming-Systems-Lab/Parikshan. The software is not maintained.

Short time-to-localize and time-to-fix for production bugs is extremely important for any 24×7 service-oriented application (SOA). Debugging buggy behavior in deployed applications is hard, as it requires careful reproduction of a similar environment and workload. Prior approaches for automatically reproducing production failures do not scale to large SOA systems. Our key insight is that for many failures in SOA systems (e.g., many semantic and performance bugs), a failure can automatically be reproduced solely by relaying network packets to replicas of suspect services, an insight that we validated through a manual study of 16 real bugs across five different systems. This paper presents Parikshan, an application monitoring framework that leverages user-space virtualization and network proxy technologies to provide a sandbox “debug” environment. In this “debug” environment, developers are free to attach debuggers and analysis tools without impacting performance or correctness of the production environment. In comparison to existing monitoring solutions that can slow down production applications, Parikshan allows application monitoring at significantly lower overhead.

Binary Quilting to Generate Patched Executables without Compilation

Anthony Saieva and Gail Kaiser. Binary Quilting to Generate Patched Executables without Compilation. ACM Workshop on Forming an Ecosystem Around Software Transformation (FEAST), Virtual, November 2020, pp. 3-8. https://doi.org/10.1145/3411502.3418424

When applying patches, or dealing with legacy software, users are often reluctant to change the production executables for fear of unwanted side effects. This results in many active systems running vulnerable or buggy code even though the problems have already been identified and resolved by developers. Furthermore when dealing with old or proprietary software, users can’t view or compile source code so any attempts to change the application after distribution requires binary level manipulation. We present a new technique we call binary quilting that allows users to apply the designated minimum patch that preserves core semantics without fear of unwanted side effects introduced either by the build process or by additional code changes. Unlike hot patching, binary quilting is a one-time procedure that creates an entirely new reusable binary. Our case studies show the efficacy of this technique on real software in real patching scenarios.

@inproceedings{10.1145/3411502.3418424,
author = {Saieva, Anthony and Kaiser, Gail},
title = {{Binary Quilting to Generate Patched Executables without Compilation}},
year = {2020},
month = {November},
url = {https://doi.org/10.1145/3411502.3418424},
doi = {10.1145/3411502.3418424},
booktitle = {ACM Workshop on Forming an Ecosystem Around Software Transformation (FEAST)},
pages = {3–-8},
location = {Virtual},
}

Binary Quilting to Generate Patched Executables without Compilation

Presented by Anthony Saieva at ACM Workshop on Forming an Ecosystem Around Software Transformation (FEAST) on November 13, 2020. https://doi.org/10.1145/3411502.3418424

When applying patches, or dealing with legacy software, users are often reluctant to change the production executables for fear of unwanted side effects. This results in many active systems running vulnerable or buggy code even though the problems have already been identified and resolved by developers. Furthermore when dealing with old or proprietary software, users can’t view or compile source code so any attempts to change the application after distribution requires binary level manipulation. We present a new technique we call binary quilting that allows users to apply the designated minimum patch that preserves core semantics without fear of unwanted side effects introduced either by the build process or by additional code changes. Unlike hot patching, binary quilting is a one-time procedure that creates an entirely new reusable binary. Our case studies show the efficacy of this technique on real software in real patching scenarios.

Ad hoc Test Generation Through Binary Rewriting

Presented by Anthony Saieva at 20th IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM) on September 27, 2020. https://doi.org/10.1109/SCAM51674.2020.00018

Software available at https://github.com/Programming-Systems-Lab/ATTUNE.

When a security vulnerability or other critical bug is not detected by the developers’ test suite, and is discovered post-deployment, developers must quickly devise a new test that reproduces the buggy behavior. Then the developers need to test whether their candidate patch indeed fixes the bug, without breaking other functionality, while racing to deploy before attackers pounce on exposed user installations. This can be challenging when factors in a specific user environment triggered the bug. If enabled, however, record-replay technology faithfully replays the execution in the developer environment as if the program were executing in that user environment under the same conditions as the bug manifested. This includes intermediate program states dependent on system calls, memory layout, etc. as well as any externally-visible behavior. Many modern record-replay tools integrate interactive debuggers, to help locate the root cause, but don’t help the developers test whether their patch indeed eliminates the bug under those same conditions. In particular, modern record-replay tools that reproduce intermediate program state cannot replay recordings made with one version of a program using a different version of the program where the differences affect program state. This work builds on record-replay and binary rewriting to automatically generate and run targeted tests for candidate patches significantly faster and more efficiently than traditional test suite generation techniques like symbolic execution. These tests reflect the arbitrary (ad hoc) user and system circumstances that uncovered the bug, enabling developers to check whether a patch indeed fixes that bug. The tests essentially replay recordings made with one version of a program using a different version of the program, even when the the differences impact program state, by manipulating both the binary executable and the recorded log to result in an execution consistent with what would have happened had the the patched version executed in the user environment under the same conditions where the bug manifested with the original version. Our approach also enables users to make new recordings of their own workloads with the original version of the program, and automatically generate and run the corresponding ad hoc tests on the patched version, to validate that the patch does not break functionality they rely on.

Ad hoc Test Generation Through Binary Rewriting

Anthony Saieva, Shirish Singh and Gail Kaiser. Ad hoc Test Generation Through Binary Rewriting. 20th IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM), Virtual, September 2020, pp. 115-126. https://doi.org/10.1109/SCAM51674.2020.00018.

Software available at https://github.com/Programming-Systems-Lab/ATTUNE.

When a security vulnerability or other critical bug is not detected by the developers’ test suite, and is discovered post-deployment, developers must quickly devise a new test that reproduces the buggy behavior. Then the developers need to test whether their candidate patch indeed fixes the bug, without breaking other functionality, while racing to deploy before attackers pounce on exposed user installations. This can be challenging when factors in a specific user environment triggered the bug. If enabled, however, record-replay technology faithfully replays the execution in the developer environment as if the program were executing in that user environment under the same conditions as the bug manifested. This includes intermediate program states dependent on system calls, memory layout, etc. as well as any externally-visible behavior. Many modern record-replay tools integrate interactive debuggers, to help locate the root cause, but don’t help the developers test whether their patch indeed eliminates the bug under those same conditions. In particular, modern record-replay tools that reproduce intermediate program state cannot replay recordings made with one version of a program using a different version of the program where the differences affect program state. This work builds on record-replay and binary rewriting to automatically generate and run targeted tests for candidate patches significantly faster and more efficiently than traditional test suite generation techniques like symbolic execution. These tests reflect the arbitrary (ad hoc) user and system circumstances that uncovered the bug, enabling developers to check whether a patch indeed fixes that bug. The tests essentially replay recordings made with one version of a program using a different version of the program, even when the the differences impact program state, by manipulating both the binary executable and the recorded log to result in an execution consistent with what would have happened had the the patched version executed in the user environment under the same conditions where the bug manifested with the original version. Our approach also enables users to make new recordings of their own workloads with the original version of the program, and automatically generate and run the corresponding ad hoc tests on the patched version, to validate that the patch does not break functionality they rely on.

@INPROCEEDINGS{9252025,
author={Anthony {Saieva} and Shirish {Singh} and Gail {Kaiser}},
booktitle={IEEE 20th International Working Conference on Source Code Analysis and Manipulation (SCAM)},
title={{Ad hoc Test Generation Through Binary Rewriting}},
month = {September},
year={2020},
location = {Virtual},
volume={},
number={},
pages={115–126},
url = {https://doi.org/10.1109/SCAM51674.2020.00018},
}

Replay without Recording of Production Bugs for Service Oriented Applications

Nipun Arora, Jonathan Bell, Franjo Ivančić, Gail Kaiser and Baishakhi Ray. Replay without Recording of Production Bugs for Service Oriented Applications. 33rd ACM/IEEE International Conference on Automated Software Engineering (ASE), Montpellier, France, September 2018, pp. 452-463. https://doi.org/10.1145/3238147.3238186.

Software available at https://github.com/Programming-Systems-Lab/Parikshan. The software is not maintained.

Short time-to-localize and time-to-fix for production bugs is extremely important for any 24×7 service-oriented application (SOA). Debugging buggy behavior in deployed applications is hard, as it requires careful reproduction of a similar environment and workload. Prior approaches for automatically reproducing production failures do not scale to large SOA systems. Our key insight is that for many failures in SOA systems (e.g., many semantic and performance bugs), a failure can automatically be reproduced solely by relaying network packets to replicas of suspect services, an insight that we validated through a manual study of 16 real bugs across five different systems. This paper presents Parikshan, an application monitoring framework that leverages user-space virtualization and network proxy technologies to provide a sandbox “debug” environment. In this “debug” environment, developers are free to attach debuggers and analysis tools without impacting performance or correctness of the production environment. In comparison to existing monitoring solutions that can slow down production applications, Parikshan allows application monitoring at significantly lower overhead.

@inproceedings{Arora:2018:RWR:3238147.3238186,
 author = {Arora, Nipun and Bell, Jonathan and Ivan\v{c}i\'{c}, Franjo and Kaiser, Gail and Ray, Baishakhi},
 title = {Replay Without Recording of Production Bugs for Service Oriented Applications},
 booktitle = {Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering},
 series = {ASE 2018},
 year = {2018},
 isbn = {978-1-4503-5937-5},
 location = {Montpellier, France},
 pages = {452--463},
 numpages = {12},
 url = {http://doi.acm.org/10.1145/3238147.3238186},
 doi = {10.1145/3238147.3238186},
 acmid = {3238186},
 publisher = {ACM},
 address = {New York, NY, USA},
 keywords = {Fault reproduction, live debugging},
}