Engineer Tackles Jane Street ASIC Reverse Engineering Puzzle with Custom Simulator and Verilog Extraction

A detailed technical account describes how one engineer spent weeks reverse engineering an ASIC from GDS files provided in a Jane Street puzzle. The process began with parsing the GDS layout using the gdstk Python library to identify 27 cells in the warmup challenge and thousands of elements in the main task. The engineer built a custom logic simulator backed by SQLite, developed a domain-specific hardware description language, and eventually extracted a netlist that could be converted into Verilog for simulation. Key components identified included shift registers, an adder, and a comparator named comparitor496 in the warmup round. In the full challenge, nearly 10,000 instances of 81 different sky130 standard cells were processed, revealing an unexpected floating net that prompted a bug report to Jane Street. The effort combined manual schematic tracing, graph-based connectivity analysis, and waveform inspection with Surfer to confirm functional behavior.

HabrTranslated from Russian

Read at source

An experienced engineer has published a comprehensive write-up detailing the months-long effort required to solve Jane Street’s “Can you reverse engineer an ASIC?” puzzle. The challenge supplied a GDS file describing the physical layout of a custom ASIC along with a VCD simulation trace, tasking participants with recovering the device’s intended function and any hidden message.

Instead of relying on existing EDA tools, the author constructed a complete custom toolchain. After extracting cell names prefixed with sky130_fd_sc_hd__ using the gdstk library, a graph-based connectivity algorithm was implemented to merge overlapping polygons across layers and produce a netlist. This netlist was then emitted as Verilog, enabling functional simulation that ultimately revealed two shift registers, an adder, and a comparator checking for the value 496.

The main puzzle proved substantially larger, containing almost 10,000 instances of 81 distinct standard cells. Performance optimizations reduced netlist extraction time from minutes to under a second per iteration, yet the author still encountered an un-driven net connected only to two input pins—an anomaly reported back to Jane Street as a possible layout bug.

Throughout the project the engineer also created supporting utilities: an SQLite-backed logic simulator, a simple hardware description language with its parser, a test harness, and a raylib-based GDS viewer. Although these side tools were eventually abandoned in favor of Surfer Waveform Viewer, they illustrate the depth of custom infrastructure required when standard reverse-engineering flows are unavailable.

The final simulation confirmed that a specific input sequence produces the expected output, completing the warmup stage and demonstrating a viable path for the full challenge despite the author’s self-imposed rule of avoiding established commercial EDA suites.

Meridian digestThe week's most consequential security stories, distilled into one briefing.See the digest

More in Malware & Botnets

Yoba Campaign Uses Signed ChatGPT and Claude Binaries for DLL Sideloading and XMRig Deployment
uCreator Spyware Infiltrates Modified Telegram, WeChat and GCash Builds on Android
Researchers Deploy Cowrie Honeypot Disguised as NVIDIA H100 Server to Study SSH Botnet Tactics