Unicorn

Callfuscated: Reverse Engineering a Stack-Based VM Protected by MBA, Opaque Predicates and Call-Based Jumps

A detailed technical write-up examines the HackTheBox challenge Callfuscated, which combines virtualization, mixed Boolean-arithmetic expressions, opaque predicates and junk instructions. The author first converts call-pop sequences into direct jumps by proving that register r8 is never read, then removes the resulting NOPs to obtain clean code. The resulting binary reveals a classic stack-based virtual machine whose handlers invoke heavily obfuscated functions. Triton is used to lift these functions into AST form, after which the CoBRA solver simplifies the MBA expressions into the original operations: addition, subtraction, multiplication, XOR, OR and AND. With semantics recovered, the author captures the bytecode at runtime, writes a C translator that emits readable operations, compiles the output and obtains a clean decompiled routine that checks the flag.