Hi All,
I'm collecting little tricks that will stymie a disassembler (that is,
prevent it from disassembling the code correctly) to use in a book
project I'm working on ("The Art of Disassembly"). I've collected a
bunch of tricks over the years (OhMyGosh, it's getting to be decades
now), but chances are pretty good that I've missed some pretty good
ones.

Here are some of the ideas I'm using in the book:

1. Burying data in the code stream
2. Placing code in the middle of data objects (a variant of [1]).
3. Arithmetic expressions involving two relocatable addresses (e.g.,
lbl1-lbl2)
4. Burying instructions within the opcodes of other instructions
5. Using alignment operations in code and data
6. Writing code that does not have well-defined procedure/function
boundaries
7. Overlapping data tables and, in general, making data boundaries
fuzzy.
8. Using unions and variant types to make it difficult to infer a data
object's type
9. Writing interpreters that allow a mixture of 80x86 and interpretive
code in the code stream
10. Using the breakpoint (int 3) and trace flag facilities within the
application
11. Using the machine instructions that correspond to a copyright
notice (or other string) do useful computations within the program.
12. Using the data at some location as both program data and executable
machine code (a generalization of [11]).
13. Using lots of dynamically-linked libraries to make it difficult (or
even impossible) for a disassembler to infer much about the external
code.
14. Creating wrappers for system APIs to make it difficult for
heuristic analysis to make any headway processing those calls.

My interest in this subject is duomorphic. I want to be able to discuss
how to overcome these problems when using (or writing) a disassembler;
I also want to discuss how to help obfuscate object code to make it
difficult to disassemble. Any and all constructive comments,
suggestions, and examples are welcome.
Cheers,
Randy Hyde

randyhyde@earthlink.net wrote:

Perhaps falling into #6 is the technique of using ret for various
branches (push address, ret). Combine that with some computation on
the address, and you can even do conditional branches. For example:

; Branch to 456 or 579 based on carry:
setc al
movzx, eax,al
mul dword_const_123
add eax,456
push eax
ret


Similarly, you can replace all the calls with "push return address"
followed by a jump. But when you do this, insert a byte or two of
garbage after the "call", and adjust the return address appropriately.
For example:

push return_addr
jmp subroutine
db 0x0f ; the beginning of a multi-byte instruction
return_addr:
...next real instruction

Alternatively, leave the call intact, but include an "inc dword ptr
[esp]" just before the ret in the subroutine.

In general computed branch targets of all sorts potentially confuse
disassemblers.

randyhyde@earthlink.net wrote:

And you failed to mention self-modifying code at all. For confusing a
disassembler, changing the opcode so that you have a different length
instruction will throw things off, so will mucking with a branch
address. Of course the ability to write self-modifying code may be
restricted on your platform of choice.

In comp.lang.asm.x86 robertwessel2@yahoo.com <spamtrap@crayne.org> wrote in part:
These bytes of garbage could be carefully chosen to decode
into following "real" opcodes and reduce the self-syncing of
most disassemblers. Something like

jmp [ebx]
db 0Fh
jmp [ebx]

The bytes might even include actual instructions that kept
the disasm in sync, but generating false instructions.
Used judiciously, the disasm might not even realize it's
being fooled. False sync.

Or far more complex alterations of the addr on the stack.

Agreed. Spagetti code jumps are likely to be more confusing
than call/ret unless you have no intention of actually returning.
Call/ret otherwise betrays the structure of the pgm. Since you
probably don't actually want to eliminate structure, you could use
something like

mov eax, subr_addr
mov ecx, return_addr
jmp [eax]
...
jmp [ecx]

SMC of all kinds is also obfuscating, but not allowed on
many modern OSes

-- Robert

robertwessel2@yahoo.com wrote:
Actually, it comes under (12), but I really should have explicitly
mentioned it.

Yes, the main thing that messes up an automatic disassembler that
relies upon control-flow or data-flow analysis is computing target
addresses in a manner than cannot be divined statically at disassembly
time.

Cheers,
Randy Hyde

Robert Redelmeier wrote:

Excellent idea. Never considered the idea of false sync at all.
Thanks a lot for that idea.
Cheers,
Randy Hyde

Robert Redelmeier wrote:
Could you provide a reference for that statement?

http://en.wikipedia.org/wiki/W%5EX

W^X is a BSD feature that allows you to write or execute a particular
memory page, but not both. As far as I know, under Linux or Windows XP,
you are able to run self-modifying code. I'm not sure if XP requires
the same hoops that windows 95 did, however.

There are always major implications to the paging and cache structure
when you begin modifying your own code, so the operating system can be
heavily involved.

randyhyde@earthlink.net wrote:
One of the most irritating things I've had to
crack^H^H^H^H^Hdisassemble was a game that used "compiled sprites" --
the game's sprites were packed using an RLE scheme for which the output
was actually x86 code to draw the sprite. A section of code was loaded
at the same time the sprites were loaded, and was even called a
"sprite" name (like "actor22m" or something), and just like the sprites
had relocatable addresses altered. When this section of code was
executed, it was done so using the sprite portion of the graphics
code... sneaky and irritating.

This is an oldie but goodie, yes. JMP to inside a 2-4 byte opcode.

Yep, very irritating and hard to patch around because INT 3 is a
special-case 1-byte opcode. INTO was the only "easy" way around this
(the only other 1-byte, IIRC)

"Jim Leonard" <spamtrap@crayne.org> writes:

I did something like that when I was writing games for a 4.77 MHz
8088. It was a good way to increase performance, if I recall
correctly. It wasn't intentionally obfuscatory, just faster.

(Cross-thread connection: is this "self-modifying code"?)
--
Ben Pfaff
email: blp@cs.stanford.edu
web: http://benpfaff.org

Ben Pfaff wrote:
I do the same. I just don't hide other parts of the program in with
the sprites to be obscure :-)

Just curious, what games? I'm doing some 8088 programming and would
like to check out your work.

"Jim Leonard" <spamtrap@crayne.org> writes:

[about what I worked on]
I never did anything commercially or even widely released
anything. This was circa 1990. I doubt I have any of it around
anymore.
--
Ben Pfaff
email: blp@cs.stanford.edu
web: http://benpfaff.org

randyhyde@earthlink.net wrote:
Closely related are tricks to prevent breakpoints and single-stepping
from being effective. (Just studying the disassembly of a very large
binary will take much time, but understanding is sped up greatly with
clever use of breakpoints.)

In any case, self-modifying code is essential. For example, code
might be gibberish until decrypted, preferably with a decryption key
that is not readily visible. Re-gibberishize the code after executing
it so a post-mortem disassembly won't help. (Especially nice, though
I've never seen it used, would be encrypted code where the
pre-decryption
form is also a valid routine which needs to be executed.)

Unusual linkages will make setting breakpoints hard. As a simple
example:
foo();
bar();
The enemy will expect foo to return and then bar to be invoked.
Instead let foo jump to the middle of bar, or somewhere else entirely.

Yes, if your code uses the breakpoint and step mecahnisms itself,
it makes it much harder for the enemy to use them.

Many years ago I investigated some of these mechanisms. One example
was the early Jazz for Macintosh which had strong anti-debug measures
to interfere with attempts to break its copy-protection. One thing
done
during Jazz's initialization seemd impressive enough that I wrote
it up on my website:

http://freepages.genealogy.rootsweb....esson9.htm#p98

James Dow Allen

(Robert Redelmeier wrote "[Self-Modifying Code] of all kinds is also
obfuscating, but not allowed on many modern OSes." I then asked for
a reference.)

ldb wrote:
I think it is possible to write SMC in OpenBSD.

<quote Theo de Raadt>
Furthermore, there are difference in approach between W^X and PAX
which are so fundamental that it is clear we did not copy from PAX!
Like, our idea that mprotect should still permit a user to request
a page that is PROT_EXEC|PROT_WRITE; by default the PAX people
prefer to deny such requests.
</quote>

http://cert.uni-stuttgart.de/archive.../msg00272.html

(AFAICT, mprotect() is defined in POSIX.)

http://www.opengroup.org/onlinepubs/.../mprotect.html

Note: Implementations seem to be allowed to deny a W|X request.

randyhyde@earthlink.net wrote:
Knowing that all the best disassemblers have been cracked, is it
reasonable to think that there is a way to protect from disassembling ?

If these guys cant protect their codes, who can ?

Jacques

Ben Pfaff wrote:

http://groups.google.com/group/comp....70a9ecc1a6fbf2

In my opinion, on-the-fly code generation should not be considered
self-modifying code. For my personal knowledge, how are on-the-fly
code generation and just-in-time compilation related?

http://en.wikipedia.org/wiki/Just-in-time_compilation

In article <1140212405.765667.152780@g14g2000cwa.googlegroups .com>, "ldb" <spamtrap@crayne.org> writes:
When running on the more recent Intel and AMD x86 processors that
have separate read and execute bits, Windows XP SP2 and later
support non-executable pages. Microsoft refers to this as "Data
Execution Prevention".

By default, though, DEP runs in "OptIn" mode, so only binaries that
explicitly request DEP get it.

http://support.microsoft.com/kb/875352

--
Michael Wojcik michael.wojcik@microfocus.com

He smiled and let his gaze fall to hers, so that her cheek began to
glow. Ecstatically she waited until his mouth slowly neared her own.
She knew only one thing: rdoeniadtrgove niardgoverdgovnrdgog.

Grumble wrote:

I'm not sure that there's a bright line separating the two, but
on-the-fly code generation would usually be what's done by a program to
optimize some operation it had to perform repeatedly, one which was not
well defined at compile time. A classic example is a
comparison/selection routine generated by a sort program or database
engine to handle a request. Another common example is a generated
BitBlt operation.

Just-in-time compilation would apply to executing a program written in
a traditional manner, and actually generating the executable code at
run time. The generation might be the whole program at once, or in
smaller pieces, perhaps routines, lines or statements, and the program
is often (but certainly not always) compiled to an intermediate form by
something approximating a traditional compiler.

In comp.lang.asm.x86 Nudge <spamtrap@crayne.org> wrote in part:
You are correct, and I was mistaken. I had assumed that
..text pages were in some way different to facilitate sharing,
but it seems they are just plain ordinary CoW like .data,
..bss and .stack. Usually they don't go CoW, but they can.

I don't see this as a very big security risk. Nothing nearly
as big as letting a variable length read clobber a stack
return address. And smaller than an executable stack.

-- Robert

Erdemal wrote:
StarForce.
http://www.star-force.com/