[Intermediate] — working backward from a compiled program to understand what it does, why it’s a distinct skill, and the legal line that governs it. Assumes the authorization framing in rules of engagement.

The kid version first

You have a program — an .exe, a phone app, a suspicious file from an email — and no source code. All you have is the compiled machine code. Reverse engineering is figuring out what it does anyway: what it talks to, what it changes, whether it’s malicious, how a protocol works.

It’s the same skill whether you’re analysing malware to defend against it, solving a CTF challenge, checking a program does what it claims, or working out an undocumented file format so other software can read it.

Why it exists as a discipline

Compilation is lossy and (deliberately or not) one-way. Source code has variable names, comments, structure, types — the compiler throws almost all of it away, producing machine code that runs but doesn’t explain itself:

   source.c  ──compile──►  machine code
   (names, types,          (registers, jumps, raw bytes —
    comments, structure)    names GONE, structure IMPLICIT)

Reverse engineering rebuilds the lost understanding from what survives. You’ll never recover the original source, but you can recover what it does, which is usually what you need. It’s the flip side of compilation — and understanding how compilers work is a large part of being good at reversing → assembly.

The legitimate uses — this is defensive and educational work

Being explicit, because the techniques overlap with attacking:

  • Malware analysis — the primary use in this folder. You reverse a malicious sample to learn what it does, write detection for it, and defend → security operations. This is defensive work
  • CTF rev challenges — a whole competition category: reverse a binary to find the flag → CTF platforms
  • Vulnerability research — finding bugs in closed-source software to report and fix, feeding exploitation
  • Interoperability — understanding an undocumented format or protocol so other software can work with it (legally protected in many jurisdictions for exactly this)
  • Verifying claims — does this app actually encrypt your data? Does this “free” tool phone home?
  • Legacy recovery — the source is lost but the binary must be maintained

The two approaches

Every reversing task uses one or both, and knowing which to reach for is half the skill:

Static analysis — examine without running it.

Read the code, strings, structure and imports of the file as it sits on disk. Safe (nothing executes), complete (you see all code paths, not just the ones that ran), but slow and defeated by obfuscation and packing → static analysis.

Dynamic analysis — run it and watch.

Execute the program (safely, in a sandbox) and observe its behaviour: files touched, network connections, registry changes, memory. Fast, cuts through obfuscation (the code must un-obfuscate itself to run), but you only see the paths that actually executed — and running malware is dangerous if you get the isolation wrong → dynamic analysis.

They’re complementary. Static tells you what code exists; dynamic tells you what it does. Real analysis alternates: static to find the interesting function, dynamic to watch it run, static to understand what you saw.

The formats you’ll meet

Reversing means reading executable file formats, and the three big ones:

  • PE (Portable Executable) — Windows .exe/.dll. Most malware. Headers, sections, the import table (which OS functions it calls — a huge tell)
  • ELF — Linux/Unix binaries → the ABI
  • Mach-O — macOS/iOS

Plus bytecode formats that are much easier to reverse because they decompile cleanly: Java .class/JVM, .NET/CLR (dnSpy near-perfectly recovers C#), Python .pyc, and Android DEX. Managed/bytecode languages reverse far more easily than native C/C++ — a reason attackers targeting reversibility avoid them, and a reason your own IP in them is exposed → C#, Java.

The mindset

Reversing is patient, iterative detective work, not a linear procedure:

  • Start at the edges. Strings, imports, and the file’s metadata tell you a lot before you read a single instruction — an import of CreateRemoteThread and WriteProcessMemory screams “code injection”
  • Follow the interesting thread. You don’t read every instruction; you find the function that matters (the one handling the network, the one checking the licence) and focus there
  • Rename as you understand. Ghidra lets you rename FUN_00401260 to decrypt_config as you work it out — the act of naming is the act of understanding, and a well-annotated disassembly is the deliverable
  • Hypothesise and test. “I think this checks a password” → set a breakpoint, run it, confirm

Reverse engineering’s legality depends on jurisdiction, purpose, and what you agreed to:

  • Malware analysis, CTF, security research, and interoperability are broadly protected or accepted, and interoperability specifically has legal carve-outs in many places (the EU, US DMCA exemptions)
  • EULAs often prohibit reversing the software you licensed — a contractual restriction, separately enforceable from copyright
  • Anti-circumvention laws (DMCA §1201 and equivalents) restrict bypassing copy protection, with research exemptions that are narrower than people assume
  • Analysing malware you received, or software you own, for defence or research, is the safe centre. Cracking commercial software’s licensing to pirate it is not

The rule, as with all of this domain: the technique is neutral; the purpose and authorization are what matter. When in doubt about commercial software, check the licence and the local law.

Key insight

Reverse engineering rebuilds what a program does from compiled code that discarded how it was written — and it’s a detective skill, not a procedure: start at the edges (strings, imports), follow the one thread that matters, and rename as you understand. Static analysis (safe, complete, defeated by obfuscation) and dynamic analysis (fast, cuts through obfuscation, only sees executed paths) are complementary halves you alternate between. The whole discipline is defensive and educational at its centre — malware analysis, CTF, research — with a legal edge governed by purpose and the licence you agreed to.

Source: [reference] — Aug 2026. Defensive/CTF/research framing; analyse only what you’re authorised to.

↑ Back to Malware Analysis and Reverse Engineering