14. Hello, Ptrace

Written by Derek Selander

As alluded to in the introduction to this book, debugging is not entirely about just fixing stuff. Debugging is the process of gaining a better understanding of what’s happening behind the scenes. In this chapter, you’ll explore the foundation of debugging, namely, a system call responsible for a process attaching itself to another process: ptrace.

In addition, you’ll learn some common security tricks developers use with ptrace to prevent a process from attaching to their programs. You’ll also learn some easy workarounds for these developer-imposed restrictions.

System calls

Wait wait wait… ptrace is a system call. What’s a system call?

A system call is a powerful, lower-level service provided by the kernel. System calls are the foundation for user-land frameworks, such as C’s stdlib, Cocoa, UIKit, or even your own brilliant frameworks are built upon.

macOS Mojave Sierra has about 533 system calls. Open a Terminal window and run the following command to get a very close estimate of the number of systems calls available in your system:

sudo dtrace -ln 'syscall:::entry' | wc -l

This command uses an incredibly powerful tool named DTrace to inspect system calls present on your macOS machine.

Note: Remember, you’ll need to disable SIP (See Chapter 1) if you want to use DTrace. In addition, you’ll also need sudo for the DTrace command since DTrace can monitor processes across multiple users, as well as perform some incredibly powerful actions. With great power comes great responsibility — that’s why you need sudo.

You’ll learn more about how to bend DTrace to your will in the 5th section of this book. For now you’ll use simple DTrace commands to get system call information out of ptrace.

The foundation of attachment, ptrace

You’re now going to take a look at the ptrace system call in more depth. Open a Terminal console. Before you start, make sure to clear the Terminal console by pressing ⌘ + K. Next, execute the following DTrace inline script in Terminal to see how ptrace is called:

sudo dtrace -qn 'syscall::ptrace:entry { printf("%s(%d, %d, %d, %d) from %s\n", probefunc, arg0, arg1, arg2, arg3, execname); }'

This creates a DTrace probe that will execute every time the ptrace function executes; it will spit out the arguments of the ptrace system call as well as the executable responsible for calling.

Don’t worry about the semantics of this DTrace script; you’ll become uncomfortably familiar with this tool in a later set of chapters. For now, just focus on what’s returned from the Terminal.

Create a new tab in Terminal with the shortcut ⌘ + T.

Note: If you haven’t disabled Rootless yet, you’ll need to check out Chapter 1 for more information on how to disable it, or else ptrace will fail when attaching to Finder and your DTrace scripts will not work.

Once Rootless is disabled, type the following into the new Terminal tab:

lldb -n Finder

Once you’ve attached to the Finder application, the DTrace probe you set up on your first Terminal tab will spit out some information similar to the following:

ptrace(14, 459, 0, 0) from debugserver

It seems a process named debugserver is responsible for calling ptrace and attaching to the Finder process. But how was debugserver called? You attached to Finder using LLDB, not debugserver. And is this debugserver process even still alive?

Time to answer these questions. Create a new tab in Terminal (⌘ + T). Next, type the following into the Terminal window:

pgrep debugserver

Provided LLDB has attached successfully and is running, you’ll receive an integer output representing debugserver’s process ID, or PID, indicating debugserver is alive and well and running on your computer.

Since debugserver is currently running, you can find out how debugserver was started. Type the following:

ps -fp `pgrep -x debugserver`

Be sure to note that the above commands uses backticks, not single quotes, to make the command work.

This will give you the full path to the location of debugserver, along with all arguments used to launch this process.

You’ll see something similar to the following:

/Applications/Xcode-beta.app/Contents/SharedFrameworks/LLDB.framework/Resources/debugserver --native-regs --setsid --reverse-connect 127.0.0.1:59297

Cool! This probably makes you wonder how the functionality changes when you subtract or modify certain launch arguments. For instance, what would happen if you got rid of --reverse-connect 127.0.0.1:59297?

So which process launched debugserver? Type the following:

ps -o ppid= $(pgrep -x debugserver)

This will dump out the parent PID responsible for launching debugserver. You’ll get an integer similar to the following:

82122

As always when working with PIDs, they will very likely be different on your computer (and from run-to-run) than what you see here.

All right, numbers are interesting, but you’re dying to know the actual name associated with this PID. You can get this information by executing the following in Terminal, replacing the number with the PID you discovered in the previous step:

ps -a 82122

You’ll get the name, fullpath, and launch arguments of the process responsible for launching debugserver:

PID   TT  STAT      TIME COMMAND
82122 s000  S+     0:05.35 /Applications/Xcode.app/Contents/Developer/usr/bin/lldb -n Finder

As you can see, LLDB was responsible for launching the debugserver process, which then attached itself to Finder using the ptrace system call. Now you know where this call is coming from, you can take a deeper dive into the function arguments passed into ptrace.

ptrace arguments

You’re able to infer the process and arguments executed when ptrace was called. Unfortunately, they’re just numbers, which are rather useless to you at the moment. It’s time to make sense of these numbers using the <sys/ptrace.h> header file.

To do this, you’ll use a macOS application to guide your understanding.

Open up the helloptrace application, which you’ll find in the resources folder for this chapter. This is a macOS Terminal command application and is as barebones as they come. All it does is launch then complete with no output to stdout at all.

The only thing of interest in this project is a bridging header used to import the ptrace system call API into Swift.

Open main.swift and add the following code to the end of the file:

while true {
  sleep(2)
  print("helloptrace")
}

Next, position Xcode and the DTrace Terminal window so they are both visible on the same screen.

Build and run the application. Once your app has launched and debugserver has attached, observe the output generated by the DTrace script.

Take note of the DTrace Terminal window. Two new ptrace calls will happen when the helloptrace process starts running. The output of the DTrace script will look similar to this:

ptrace(14, 50121, 0, 0) from debugserver
ptrace(13, 50121, 5891, 0) from debugserver

Use Xcode’s Open Quickly feature (⌘ + Shift + O) and type /usr/include/sys/ptrace.h. A look in ptrace.h gives the following function prototype for ptrace:

int ptrace(int _request, pid_t _pid, caddr_t _addr, int _data);

The first parameter is what you want ptrace to do. The second parameter is the PID you want to act upon. The third and fourth parameters depend on the first parameter.

Take a look back at your earlier DTrace output. Your first line of output was something similar to the following:

ptrace(14, 50121, 0, 0) from debugserver

Compare the first parameter to ptrace.h header and you’ll see the first parameter, 14, actually stands for PT_ATTACHEXC. What does this PT_ATTACHEXC mean? To get information about this parameter, first, open a Terminal window. Finally, type man ptrace and search for PT_ATTACHEXC.

Note: You can perform case-sensitive searches on man pages by pressing /, followed by your search query. You can search downwards to the next hit by pressing N or upwards to the previous hit by pressing Shift + N.

You’ll find some relevant info about PT_ATTACHEXC with the following output obtained from the ptrace man page:

This request allows a process to gain control of an otherwise unrelated process and begin tracing it. It does not need any cooperation from the to-be-traced process. In this case, `pid` specifies the process ID of the to-be-traced process, and the other two arguments are ignored.

With this information, the reason for the first call of ptrace should be clear. This call says “hey, attach to this process”, and attaches to the process provided in the second parameter.

Onto the next ptrace call from your DTrace output:

ptrace(13, 50121, 5891, 0) from debugserver

This one is a bit trickier to understand, since Apple decided to not give any man documentation about this one. This call relates to the internals of a process attaching to another one.

If you look at the ptrace API header, 13 stands for PT_THUPDATE and relates to how the controlling process, in this case, debugserver, handles UNIX signals and Mach messages passed to the controlled process; in this case, helloptrace. The kernel needs to know how to handle signal passing from a process controlled by another process, as in the Signals project from Section 1. The controlling process could say it doesn’t want to send any signals to the controlled process.

This specific ptrace action is an implementation detail of how the Mach kernel handles ptrace internally; there’s no need to dwell on it. Fortunately, there are other documented signals definitely worth exploring through man. One of them is the PT_DENY_ATTACH action, which you’ll learn about now.

Creating attachment issues

A process can actually specify it doesn’t want to be attached to by calling ptrace and supplying the PT_DENY_ATTACH argument. This is often used as an anti-debugging mechanism to prevent unwelcome reverse engineers from discovering a program’s internals.

You’ll now experiment with this argument. Open main.swift and add the following line of code before the while loop:

ptrace(PT_DENY_ATTACH, 0, nil, 0)

Build and run, keep on eye on the debugger console and see what happens.

The program will exit and output the following to the debugger console:

Program ended with exit code: 45

Note: You may need to open up the debug console by clicking View ▸ Debug Area ▸ Activate Console (or ⌘ + Shift + Y if you’re one of those cool, shortcut devs) to see this.

This happened because Xcode launches the helloptrace program by default with LLDB automatically attached. If you execute the ptrace function with PT_DENY_ATTACH, LLDB will exit early and the program will stop executing.

If you were to try and execute the helloptrace program, and tried later to attach to it, LLDB would fail in attaching and the helloptrace program would happily continue execution, oblivious to debugserver’s attachment issues.

There are numerous macOS (and iOS) programs that perform this very action in their production builds. However, it’s rather trivial to circumvent this security precaution. Ninja debug mode activated!

Getting around PT_DENY_ATTACH

Once a process executes ptrace with the PT_DENY_ATTACH argument, making an attachment greatly escalates in complexity. However, there’s a much easier way of getting around this problem.

Typically a developer will execute ptrace(PT_DENY_ATTACH, 0, 0, 0) somewhere in the main executable’s code — oftentimes, right in the main function.

Since LLDB has the -w argument to wait for the launching of a process, you can use LLDB to “catch” the launch of a process and perform logic to augment or ignore the PT_DENY_ATTACH command before the process has a chance to execute ptrace!

Open a new Terminal window and type the following:

sudo lldb -n "helloptrace" -w 

This starts an lldb session and attaches to the helloptrace program, but this time -w tells lldb to wait until a new process with the name helloptrace has started.

You need to use sudo due to an ongoing bug with LLDB and macOS security when you tell LLDB to wait for a Terminal program to launch.

In the Project Navigator, open the Products folder and right click on the helloptrace executable. Next, select Show in Finder.

Next, drag the helloptrace executable into a new Terminal tab. Finally, press Enter to start the executable.

Now, open the previously created Terminal tab, where you had LLDB sit and wait for the helloptrace executable.

If everything went as expected, LLDB will see helloptrace has started and will launch itself, attaching to this newly created helloptrace process.

In LLDB, create the following regex breakpoint to stop on any type of function containing the word ptrace:

(lldb) rb ptrace -s libsystem_kernel.dylib

This will add a breakpoint on the userland gateway to the actual kernel ptrace function. Next, type continue into the Terminal window.

(lldb) continue

You’ll break right before the ptrace function is about to be executed. However, you can simply use LLDB to return early and not execute that function. Do that now like so:

(lldb) thread return 0

Next, simply just continue:

(lldb) continue

Although the program entered the ptrace userland gateway function, you told LLDB to return early and not execute the logic that will execute the kernel ptrace system call.

Navigate to the helloptrace output tab and verify it’s outputting “helloptrace” over and over. If so, you’ve successfully bypassed PT_DENY_ATTACH and are running LLDB while still attached to the helloptrace command!

In a couple chapters, you’ll explore an alternative method to crippling external functions like ptrace by inspecting Mach-O’s __DATA.__la_symbol_ptr section along with the lovely DYLD_INSERT_LIBRARIES environment variable.

Other anti-debugging techniques

Since we’re on the topic of anti-debugging, let’s put iTunes on the spot: for the longest time, iTunes actually used the ptrace’s PT_DENY_ATTACH. However, the current version of iTunes (12.7.0 at the time of writing) has opted for a different technique to prevent debugging.

iTunes will now check if it’s being debugged using the powerful sysctl function, then kill itself if true. sysctl is another kernel function (like ptrace) that gets or sets kernel values. iTunes repeatedly calls sysctl while it’s running using a NSTimer to call out to the logic.

Below is a simplified code example in Swift of what iTunes is doing:

let mib = UnsafeMutablePointer<Int32>.allocate(capacity: 4)
mib[0] = CTL_KERN
mib[1] = KERN_PROC
mib[2] = KERN_PROC_PID
mib[3] = getpid()

var size: Int = MemoryLayout<kinfo_proc>.size
var info: kinfo_proc? = nil

sysctl(mib, 4, &info, &size, nil, 0)

if (info.unsafelyUnwrapped.kp_proc.p_flag & P_TRACED) > 0 {
  exit(1)
}

I am not going to go into the details of the expected params for sysctl yet, we’ll save that for a different chapter. Just know that there is more than one way to skin a cat.

Where to go from here?

With the DTrace dumping script you used in this chapter, explore parts of your system and see when ptrace is called.

If you’re feeling cocky, read up on the ptrace man pages and see if you can create a program that will automatically attach itself to another program on your system.

Still have energy? Go man sysctl. That will be some good night-time reading.

Remember, having attachment issues is not always a bad thing!