Pentester Academy – x86_64 Assembly Language and Shellcoding on Linux

This course focuses on teaching the basics of 64-bit assembly language for the x86_64 family of processors on the Linux platform and applying it to Infosec. Once we are through with the basics, we will look at writing shellcode, encoders, decoders, crypters, and other advanced low-level applications.

The course outline follows the exact same outline of our 32-bit course but all the topics will be taught with x86_64 64-bit assembly.

A non-exhaustive list of topics to be covered include:

• Computer Architecture Basics
• x86/x86_64 Family
• Compilers, Assemblers and Linkers
• CPU Modes and Memory Addressing
• Tools of the trade
  • Nasm, Ld, Objdump, Ndisasm etc.
• x86_64 Assembly Language
  • Registers and Flags
  • Program Structure for use with nasm
  • Data Types
  • Data Movement Instructions
  • Arithmetic instructions
  • Reading and Writing from memory
  • Conditional instructions
  • Strings and Loops
  • Interrupts, Traps and Exceptions
  • Procedures, Prologues and Epilogues
  • Syscall structure and ABI for Linux
  • Calling standard library functions
  • FPU instructions
  • MMX, SSE, SSE2 etc. instruction sets
• Shellcoding on Linux
  • Execution environment
  • Exit and Execve shellcode
  • Bind Shell and Reverse TCP
  • Staged Shellcode
  • Egg Hunter
  • Using 3rd party shellcode
  • Simulating shellcode
    • locating syscalls
    • graphing shellcode execution
• Encoders, Decoders, and Crypters on Linux
  • Purpose of encoding and encrypting
  • XOR encoders
  • Custom encoding
    • Random sequencing and scrambling
    • mapping functions
  • Crypters
• Polymorphism
  • Why polymorphism?
  • Polymorphic engines
  • Techniques and Tools

Pentester Academy – x86 Assembly Language and Shellcoding on Linux