Introduction to Malware Reverse Engineering

(3-0-1-4)

CMPE Degree: This course is Selected Elective for the CMPE degree.

EE Degree: This course is Selected Elective for the EE degree.

Lab Hours: 0 supervised lab hours and 1 unsupervised lab hours.

Technical Interest Groups / Course Categories: Threads / ECE Electives

Course Coordinator: Brendan D Saltaformaggio

Prerequisites: ECE 2035 [min C] or ECE 2036 [min C] or CS 2200 [min C]

Catalog Description

This course exposes students to an immersive, hands-on experience in the dissection and analysis of the code, structure, and functionality of malicious software.

Textbook(s)

Course Outcomes

Identify and disarm common anti-analysis behaviors in malware samples

Statically reverse engineer malware samples in a disassembler

Decide upon and employ appropriate reverse engineering tools for a range of malware analysis cases

Reverse engineer exploit inputs for benign program binaries

Dynamically unpack malware in a debugger and extract clean disassemblies

Strategic Performance Indicators (SPIs)

N/A

Topic List

  1. Reverse Engineering Principles
    1. Intro. to Malware and Assembly Language
    2. Background on Malware
  2. Low-Level Software
    1. Overview of Intel Assembly Language
    2. Virtual Machines for Interpreted High-Level Languages
    3. Representation of Compiled High-Level Language Structures in Assembly
    4. Operating Systems Background
    5. Executable File Formats
  3. PE Files
    1. Import Address Table
    2. Analysis of Malicious Software
    3. System Monitoring Tools
    4. Dynamic Tracing: System Calls, Filesystem, and Registry
    5. Compiler Issues
    6. Debuggers
    7. Disassemblers
    8. Memory Analysis to Support Reverse Engineering
  4. Advanced Reverse Engineering Techniques
    1. Encrypted/Packed Executables
    2. Anti-Debugging Techniques
    3. Anti-VM Techniques
    4. Code Obfuscation
  5. Remediation of Advanced Persistent Threats
    1. Determination of Malicious Behaviors
    2. Analysis of Decompiled Source Code
    3. Revelation of Command and Control Functionalities