Debugging with GDB: Tracepoints

[ < ] | [ > ] |  | [ << ] | [ Up ] | [ >> ] |  |  |  |  | [Top] | [Contents] | [Index] | [ ? ]

# 13. Tracepoints

In some applications, it is not feasible for the debugger to interrupt the program's execution long enough for the developer to learn anything helpful about its behavior. If the program's correctness depends on its real-time behavior, delays introduced by a debugger might cause the program to change its behavior drastically, or perhaps fail, even when the code itself is correct. It is useful to be able to observe the program's behavior without interrupting it.

Using GDB's `trace` and `collect` commands, you can specify locations in the program, called tracepoints, and arbitrary expressions to evaluate when those tracepoints are reached. Later, using the `tfind` command, you can examine the values those expressions had when the program hit the tracepoints. The expressions may also denote objects in memory--structures or arrays, for example--whose values GDB should record; while visiting a particular tracepoint, you may inspect those objects as if they were in memory at that moment. However, because GDB records these values without interacting with you, it can do so quickly and unobtrusively, hopefully not disturbing the program's behavior.

The tracepoint facility is currently available only for remote targets. See section 19. Specifying a Debugging Target. In addition, your remote target must know how to collect trace data. This functionality is implemented in the remote stub; however, none of the stubs distributed with GDB support tracepoints as of this writing. The format of the remote packets used to implement tracepoints are described in E.6 Tracepoint Packets.

It is also possible to get trace data from a file, in a manner reminiscent of corefiles; you specify the filename, and use `tfind` to search through the file. See section 13.4 Using Trace Files, for more details.

This chapter describes the tracepoint commands and features.

> 13.1 Commands to Set Tracepoints |  |
> 13.2 Using the Collected Data |  |
> 13.3 Convenience Variables for Tracepoints |  |
> 13.4 Using Trace Files |  |

[ < ] | [ > ] |  | [ << ] | [ Up ] | [ >> ] |  |  |  |  | [Top] | [Contents] | [Index] | [ ? ]

## 13.1 Commands to Set Tracepoints

Before running such a trace experiment, an arbitrary number of tracepoints can be set. A tracepoint is actually a special type of breakpoint (see section 5.1.1 Setting Breakpoints), so you can manipulate it using standard breakpoint commands. For instance, as with breakpoints, tracepoint numbers are successive integers starting from one, and many of the commands associated with tracepoints take the tracepoint number as their argument, to identify which tracepoint to work on.

For each tracepoint, you can specify, in advance, some arbitrary set of data that you want the target to collect in the trace buffer when it hits that tracepoint. The collected data can include registers, local variables, or global data. Later, you can use GDB commands to examine the values these data had at the time the tracepoint was hit.

Tracepoints do not support every breakpoint feature. Ignore counts on tracepoints have no effect, and tracepoints cannot run GDB commands when they are hit. Tracepoints may not be thread-specific either.

Some targets may support fast tracepoints, which are inserted in a different way (such as with a jump instead of a trap), that is faster but possibly restricted in where they may be installed.

Regular and fast tracepoints are dynamic tracing facilities, meaning that they can be used to insert tracepoints at (almost) any location in the target. Some targets may also support controlling static tracepoints from GDB. With static tracing, a set of instrumentation points, also known as markers, are embedded in the target program, and can be activated or deactivated by name or address. These are usually placed at locations which facilitate investigating what the target is actually doing. GDB's support for static tracing includes being able to list instrumentation points, and attach them with GDB defined high level tracepoints that expose the whole range of convenience of GDB's tracepoints support. Namely, support for collecting registers values and values of global or local (to the instrumentation point) variables; tracepoint conditions and trace state variables. The act of installing a GDB static tracepoint on an instrumentation point, or marker, is referred to as probing a static tracepoint marker.

`gdbserver` supports tracepoints on some target systems. See section Tracepoints support in `gdbserver`.

This section describes commands to set tracepoints and associated conditions and actions.

> 13.1.1 Create and Delete Tracepoints |  |
> 13.1.2 Enable and Disable Tracepoints |  |
> 13.1.3 Tracepoint Passcounts |  |
> 13.1.4 Tracepoint Conditions |  |
> 13.1.5 Trace State Variables |  |
> 13.1.6 Tracepoint Action Lists |  |
> 13.1.7 Listing Tracepoints |  |
> 13.1.8 Listing Static Tracepoint Markers |  |
> 13.1.9 Starting and Stopping Trace Experiments |  |
> 13.1.10 Tracepoint Restrictions |  |

[ < ] | [ > ] |  | [ << ] | [ Up ] | [ >> ] |  |  |  |  | [Top] | [Contents] | [Index] | [ ? ]

### 13.1.1 Create and Delete Tracepoints

`trace location`
  The `trace` command is very similar to the `break` command. Its argument location can be a source line, a function name, or an address in the target program. See section 9.2 Specifying a Location. The `trace` command defines a tracepoint, which is a point in the target program where the debugger will briefly stop, collect some data, and then allow the program to continue. Setting a tracepoint or changing its actions takes effect immediately if the remote stub supports the ``InstallInTrace'` feature (see install tracepoint in tracing). If remote stub doesn't support the ``InstallInTrace'` feature, all these changes don't take effect until the next `tstart` command, and once a trace experiment is running, further changes will not have any effect until the next trace experiment starts. In addition, GDB supports pending tracepoints---tracepoints whose address is not yet resolved. (This is similar to pending breakpoints.) Pending tracepoints are not downloaded to the target and not installed until they are resolved. The resolution of pending tracepoints requires GDB support--when debugging with the remote target, and GDB disconnects from the remote stub (see disconnected tracing), pending tracepoints can not be resolved (and downloaded to the remote stub) while GDB is disconnected.

  Here are some examples of using the `trace` command:

      (gdb) trace foo.c:121    // a source file and line number

      (gdb) trace +2           // 2 lines forward

      (gdb) trace my_function  // first source line of function

      (gdb) trace *my_function // EXACT start address of function

      (gdb) trace *0x2117c4    // an address

  You can abbreviate `trace` as `tr`.

`trace location if cond`
  Set a tracepoint with condition cond; evaluate the expression cond each time the tracepoint is reached, and collect data only if the value is nonzero--that is, if cond evaluates as true. See section Tracepoint Conditions, for more information on tracepoint conditions.

`ftrace location [ if cond ]`
  The `ftrace` command sets a fast tracepoint. For targets that support them, fast tracepoints will use a more efficient but possibly less general technique to trigger data collection, such as a jump instruction instead of a trap, or some sort of hardware support. It may not be possible to create a fast tracepoint at the desired location, in which case the command will exit with an explanatory message.

  GDB handles arguments to `ftrace` exactly as for `trace`.

  On 32-bit x86-architecture systems, fast tracepoints normally need to be placed at an instruction that is 5 bytes or longer, but can be placed at 4-byte instructions if the low 64K of memory of the target program is available to install trampolines. Some Unix-type systems, such as GNU/Linux, exclude low addresses from the program's address space; but for instance with the Linux kernel it is possible to let GDB use this area by doing a `sysctl` command to set the `mmap_min_addr` kernel parameter, as in

      sudo sysctl -w vm.mmap_min_addr=32768

  which sets the low address to 32K, which leaves plenty of room for trampolines. The minimum address should be set to a page boundary.

`strace location [ if cond ]`
  The `strace` command sets a static tracepoint. For targets that support it, setting a static tracepoint probes a static instrumentation point, or marker, found at location. It may not be possible to set a static tracepoint at the desired location, in which case the command will exit with an explanatory message.

  GDB handles arguments to `strace` exactly as for `trace`, with the addition that the user can also specify `-m marker` as location. This probes the marker identified by the marker string identifier. This identifier depends on the static tracepoint backend library your program is using. You can find all the marker identifiers in the ``ID'` field of the `info static-tracepoint-markers` command output. See section Listing Static Tracepoint Markers. For example, in the following small program using the UST tracing engine:

      main ()
      {
        trace_mark(ust, bar33, "str %s", "FOOBAZ");
      }

  the marker id is composed of joining the first two arguments to the `trace_mark` call with a slash, which translates to:

      (gdb) info static-tracepoint-markers
      Cnt Enb ID         Address            What
      1   n   ust/bar33  0x0000000000400ddc in main at stexample.c:22
               Data: "str %s"
      [etc...]

  so you may probe the marker above with:

      (gdb) strace -m ust/bar33

  Static tracepoints accept an extra collect action -- `collect $_sdata`. This collects arbitrary user data passed in the probe point call to the tracing library. In the UST example above, you'll see that the third argument to `trace_mark` is a printf-like format string. The user data is then the result of running that formating string against the following arguments. Note that `info static-tracepoint-markers` command output lists that format string in the ``Data:'` field.

  You can inspect this data when analyzing the trace buffer, by printing the $_sdata variable like any other variable available to GDB. See section Tracepoint Action Lists.

  The convenience variable `$tpnum` records the tracepoint number of the most recently set tracepoint.

`delete tracepoint [num]`
  Permanently delete one or more tracepoints. With no argument, the default is to delete all tracepoints. Note that the regular `delete` command can remove tracepoints also.

  Examples:

      (gdb) delete trace 1 2 3 // remove three tracepoints

      (gdb) delete trace       // remove all tracepoints

  You can abbreviate this command as `del tr`.

[ < ] | [ > ] |  | [ << ] | [ Up ] | [ >> ] |  |  |  |  | [Top] | [Contents] | [Index] | [ ? ]

### 13.1.2 Enable and Disable Tracepoints

These commands are deprecated; they are equivalent to plain `disable` and `enable`.

`disable tracepoint [num]`
  Disable tracepoint num, or all tracepoints if no argument num is given. A disabled tracepoint will have no effect during a trace experiment, but it is not forgotten. You can re-enable a disabled tracepoint using the `enable tracepoint` command. If the command is issued during a trace experiment and the debug target has support for disabling tracepoints during a trace experiment, then the change will be effective immediately. Otherwise, it will be applied to the next trace experiment.

`enable tracepoint [num]`
  Enable tracepoint num, or all tracepoints. If this command is issued during a trace experiment and the debug target supports enabling tracepoints during a trace experiment, then the enabled tracepoints will become effective immediately. Otherwise, they will become effective the next time a trace experiment is run.

[ < ] | [ > ] |  | [ << ] | [ Up ] | [ >> ] |  |  |  |  | [Top] | [Contents] | [Index] | [ ? ]

### 13.1.3 Tracepoint Passcounts

`passcount [n [num]]`
  Set the passcount of a tracepoint. The passcount is a way to automatically stop a trace experiment. If a tracepoint's passcount is n, then the trace experiment will be automatically stopped on the n'th time that tracepoint is hit. If the tracepoint number num is not specified, the `passcount` command sets the passcount of the most recently defined tracepoint. If no passcount is given, the trace experiment will run until stopped explicitly by the user.

  Examples:

      (gdb) passcount 5 2 // Stop on the 5th execution of
                                         // tracepoint 2

      (gdb) passcount 12  // Stop on the 12th execution of the
                                         // most recently defined tracepoint.
      (gdb) trace foo
      (gdb) pass 3
      (gdb) trace bar
      (gdb) pass 2
      (gdb) trace baz
      (gdb) pass 1        // Stop tracing when foo has been
                                          // executed 3 times OR when bar has
                                          // been executed 2 times
                                          // OR when baz has been executed 1 time.

[ < ] | [ > ] |  | [ << ] | [ Up ] | [ >> ] |  |  |  |  | [Top] | [Contents] | [Index] | [ ? ]

### 13.1.4 Tracepoint Conditions

The simplest sort of tracepoint collects data every time your program reaches a specified place. You can also specify a condition for a tracepoint. A condition is just a Boolean expression in your programming language (see section Expressions). A tracepoint with a condition evaluates the expression each time your program reaches it, and data collection happens only if the condition is true.

Tracepoint conditions can be specified when a tracepoint is set, by using ``if'` in the arguments to the `trace` command. See section Setting Tracepoints. They can also be set or changed at any time with the `condition` command, just as with breakpoints.

Unlike breakpoint conditions, GDB does not actually evaluate the conditional expression itself. Instead, GDB encodes the expression into an agent expression (see section F. The GDB Agent Expression Mechanism) suitable for execution on the target, independently of GDB. Global variables become raw memory locations, locals become stack accesses, and so forth.

For instance, suppose you have a function that is usually called frequently, but should not be called after an error has occurred. You could use the following tracepoint command to collect data about calls of that function that happen while the error code is propagating through the program; an unconditional tracepoint could end up collecting thousands of useless trace frames that you would have to search through.

    (gdb) trace normal_operation if errcode > 0

[ < ] | [ > ] |  | [ << ] | [ Up ] | [ >> ] |  |  |  |  | [Top] | [Contents] | [Index] | [ ? ]

### 13.1.5 Trace State Variables

A trace state variable is a special type of variable that is created and managed by target-side code. The syntax is the same as that for GDB's convenience variables (a string prefixed with "$"), but they are stored on the target. They must be created explicitly, using a `tvariable` command. They are always 64-bit signed integers.

Trace state variables are remembered by GDB, and downloaded to the target along with tracepoint information when the trace experiment starts. There are no intrinsic limits on the number of trace state variables, beyond memory limitations of the target.

Although trace state variables are managed by the target, you can use them in print commands and expressions as if they were convenience variables; GDB will get the current value from the target while the trace experiment is running. Trace state variables share the same namespace as other "$" variables, which means that you cannot have trace state variables with names like `$23` or `$pc`, nor can you have a trace state variable and a convenience variable with the same name.

`tvariable $name [ = expression ]`
  The `tvariable` command creates a new trace state variable named `$name`, and optionally gives it an initial value of expression. expression is evaluated when this command is entered; the result will be converted to an integer if possible, otherwise GDB will report an error. A subsequent `tvariable` command specifying the same name does not create a variable, but instead assigns the supplied initial value to the existing variable of that name, overwriting any previous initial value. The default initial value is 0.

`info tvariables`
  List all the trace state variables along with their initial values. Their current values may also be displayed, if the trace experiment is currently running.

`delete tvariable [ $name ... ]`
  Delete the given trace state variables, or all of them if no arguments are specified.

[ < ] | [ > ] |  | [ << ] | [ Up ] | [ >> ] |  |  |  |  | [Top] | [Contents] | [Index] | [ ? ]

### 13.1.6 Tracepoint Action Lists

`actions [num]`
  This command will prompt for a list of actions to be taken when the tracepoint is hit. If the tracepoint number num is not specified, this command sets the actions for the one that was most recently defined (so that you can define a tracepoint and then say `actions` without bothering about its number). You specify the actions themselves on the following lines, one action at a time, and terminate the actions list with a line containing just `end`. So far, the only defined actions are `collect`, `teval`, and `while-stepping`.

  `actions` is actually equivalent to `commands` (see section Breakpoint Command Lists), except that only the defined actions are allowed; any other GDB command is rejected.

  To remove all actions from a tracepoint, type ``actions num'` and follow it immediately with ``end'`.

      (gdb) collect data // collect some data

      (gdb) while-stepping 5 // single-step 5 times, collect data

      (gdb) end              // signals the end of actions.

  In the following example, the action list begins with `collect` commands indicating the things to be collected when the tracepoint is hit. Then, in order to single-step and collect additional data following the tracepoint, a `while-stepping` command is used, followed by the list of things to be collected after each step in a sequence of single steps. The `while-stepping` command is terminated by its own separate `end` command. Lastly, the action list is terminated by an `end` command.

      (gdb) trace foo
      (gdb) actions
      Enter actions for tracepoint 1, one per line:
      > collect bar,baz
      > collect $regs
      > while-stepping 12
        > collect $pc, arr[i]
        > end
      end

`collect[/mods] expr1, expr2, ...`
  Collect values of the given expressions when the tracepoint is hit. This command accepts a comma-separated list of any valid expressions. In addition to global, static, or local variables, the following special arguments are supported:

  `$regs`
    Collect all registers.

  `$args`
    Collect all function arguments.

  `$locals`
    Collect all local variables.

  `$_ret`
    Collect the return address. This is helpful if you want to see more of a backtrace.

  `$_probe_argc`
    Collects the number of arguments from the static probe at which the tracepoint is located. See section 5.1.10 Static Probe Points.

  `$_probe_argn`
    n is an integer between 0 and 11. Collects the nth argument from the static probe at which the tracepoint is located. See section 5.1.10 Static Probe Points.

  `$_sdata`
    Collect static tracepoint marker specific data. Only available for static tracepoints. See section Tracepoint Action Lists. On the UST static tracepoints library backend, an instrumentation point resembles a `printf` function call. The tracing library is able to collect user specified data formatted to a character string using the format provided by the programmer that instrumented the program. Other backends have similar mechanisms. Here's an example of a UST marker call:

         const char master_name[] = "$your_name";
         trace_mark(channel1, marker1, "hello %s", master_name)

    In this case, collecting `$_sdata` collects the string ``hello $yourname'`. When analyzing the trace buffer, you can inspect ``$_sdata'` like any other variable available to GDB.

  You can give several consecutive `collect` commands, each one with a single argument, or one `collect` command with several arguments separated by commas; the effect is the same.

  The optional mods changes the usual handling of the arguments. `s` requests that pointers to chars be handled as strings, in particular collecting the contents of the memory being pointed at, up to the first zero. The upper bound is by default the value of the `print elements` variable; if `s` is followed by a decimal number, that is the upper bound instead. So for instance ``collect/s25 mystr'` collects as many as 25 characters at ``mystr'`.

  The command `info scope` (see section info scope) is particularly useful for figuring out what data to collect.

`teval expr1, expr2, ...`
  Evaluate the given expressions when the tracepoint is hit. This command accepts a comma-separated list of expressions. The results are discarded, so this is mainly useful for assigning values to trace state variables (see section 13.1.5 Trace State Variables) without adding those values to the trace buffer, as would be the case if the `collect` action were used.

`while-stepping n`
  Perform n single-step instruction traces after the tracepoint, collecting new data after each step. The `while-stepping` command is followed by the list of what to collect while stepping (followed by its own `end` command):

      > while-stepping 12
        > collect $regs, myglobal
        > end
      >

  Note that `$pc` is not automatically collected by `while-stepping`; you need to explicitly collect that register if you need it. You may abbreviate `while-stepping` as `ws` or `stepping`.

`set default-collect expr1, expr2, ...`
  This variable is a list of expressions to collect at each tracepoint hit. It is effectively an additional `collect` action prepended to every tracepoint action list. The expressions are parsed individually for each tracepoint, so for instance a variable named `xyz` may be interpreted as a global for one tracepoint, and a local for another, as appropriate to the tracepoint's location.

`show default-collect`
  Show the list of expressions that are collected by default at each tracepoint hit.

[ < ] | [ > ] |  | [ << ] | [ Up ] | [ >> ] |  |  |  |  | [Top] | [Contents] | [Index] | [ ? ]

### 13.1.7 Listing Tracepoints

`info tracepoints [num...]`
  Display information about the tracepoint num. If you don't specify a tracepoint number, displays information about all the tracepoints defined so far. The format is similar to that used for `info breakpoints`; in fact, `info tracepoints` is the same command, simply restricting itself to tracepoints.

  A tracepoint's listing may include additional information specific to tracing:

  - its passcount as given by the `passcount n` command

  - the state about installed on target of each location

      (gdb) info trace
      Num     Type           Disp Enb Address    What
      1       tracepoint     keep y   0x0804ab57 in foo() at main.cxx:7
              while-stepping 20
                collect globfoo, $regs
              end
              collect globfoo2
              end
              pass count 1200
      2       tracepoint     keep y   <MULTIPLE>
              collect $eip
      2.1                         y     0x0804859c in func4 at change-loc.h:35
              installed on target
      2.2                         y     0xb7ffc480 in func4 at change-loc.h:35
              installed on target
      2.3                         y     <PENDING>  set_tracepoint
      3       tracepoint     keep y   0x080485b1 in foo at change-loc.c:29
              not installed on target
      (gdb)

  This command can be abbreviated `info tp`.

[ < ] | [ > ] |  | [ << ] | [ Up ] | [ >> ] |  |  |  |  | [Top] | [Contents] | [Index] | [ ? ]

### 13.1.8 Listing Static Tracepoint Markers

`info static-tracepoint-markers`
  Display information about all static tracepoint markers defined in the program.

  For each marker, the following columns are printed:

  Count
    An incrementing counter, output to help readability. This is not a stable identifier.
  ID
    The marker ID, as reported by the target.
  Enabled or Disabled
    Probed markers are tagged with ``y'`. ``n'` identifies marks that are not enabled.
  Address
    Where the marker is in your program, as a memory address.
  What
    Where the marker is in the source for your program, as a file and line number. If the debug information included in the program does not allow GDB to locate the source of the marker, this column will be left blank.

  In addition, the following information may be printed for each marker:

  Data
    User data passed to the tracing library by the marker call. In the UST backend, this is the format string passed as argument to the marker call.
  Static tracepoints probing the marker
    The list of static tracepoints attached to the marker.

      (gdb) info static-tracepoint-markers
      Cnt ID         Enb Address            What
      1   ust/bar2   y   0x0000000000400e1a in main at stexample.c:25
           Data: number1 %d number2 %d
           Probed by static tracepoints: #2
      2   ust/bar33  n   0x0000000000400c87 in main at stexample.c:24
           Data: str %s
      (gdb)

[ < ] | [ > ] |  | [ << ] | [ Up ] | [ >> ] |  |  |  |  | [Top] | [Contents] | [Index] | [ ? ]

### 13.1.9 Starting and Stopping Trace Experiments

`tstart`
  This command starts the trace experiment, and begins collecting data. It has the side effect of discarding all the data collected in the trace buffer during the previous trace experiment. If any arguments are supplied, they are taken as a note and stored with the trace experiment's state. The notes may be arbitrary text, and are especially useful with disconnected tracing in a multi-user context; the notes can explain what the trace is doing, supply user contact information, and so forth.

`tstop`
  This command stops the trace experiment. If any arguments are supplied, they are recorded with the experiment as a note. This is useful if you are stopping a trace started by someone else, for instance if the trace is interfering with the system's behavior and needs to be stopped quickly.

  Note: a trace experiment and data collection may stop automatically if any tracepoint's passcount is reached (see section 13.1.3 Tracepoint Passcounts), or if the trace buffer becomes full.

`tstatus`
  This command displays the status of the current trace data collection.

Here is an example of the commands we described so far:

    (gdb) trace gdb_c_test
    (gdb) actions
    Enter actions for tracepoint #1, one per line.
    > collect $regs,$locals,$args
    > while-stepping 11
      > collect $regs
      > end
    > end
    (gdb) tstart
    	[time passes ...]
    (gdb) tstop

You can choose to continue running the trace experiment even if GDB disconnects from the target, voluntarily or involuntarily. For commands such as `detach`, the debugger will ask what you want to do with the trace. But for unexpected terminations (GDB crash, network outage), it would be unfortunate to lose hard-won trace data, so the variable `disconnected-tracing` lets you decide whether the trace should continue running without GDB.

`set disconnected-tracing on`
`set disconnected-tracing off`
  Choose whether a tracing run should continue to run if GDB has disconnected from the target. Note that `detach` or `quit` will ask you directly what to do about a running trace no matter what this variable's setting, so the variable is mainly useful for handling unexpected situations, such as loss of the network.

`show disconnected-tracing`
  Show the current choice for disconnected tracing.

When you reconnect to the target, the trace experiment may or may not still be running; it might have filled the trace buffer in the meantime, or stopped for one of the other reasons. If it is running, it will continue after reconnection.

Upon reconnection, the target will upload information about the tracepoints in effect. GDB will then compare that information to the set of tracepoints currently defined, and attempt to match them up, allowing for the possibility that the numbers may have changed due to creation and deletion in the meantime. If one of the target's tracepoints does not match any in GDB, the debugger will create a new tracepoint, so that you have a number with which to specify that tracepoint. This matching-up process is necessarily heuristic, and it may result in useless tracepoints being created; you may simply delete them if they are of no use.

If your target agent supports a circular trace buffer, then you can run a trace experiment indefinitely without filling the trace buffer; when space runs out, the agent deletes already-collected trace frames, oldest first, until there is enough room to continue collecting. This is especially useful if your tracepoints are being hit too often, and your trace gets terminated prematurely because the buffer is full. To ask for a circular trace buffer, simply set ``circular-trace-buffer'` to on. You can set this at any time, including during tracing; if the agent can do it, it will change buffer handling on the fly, otherwise it will not take effect until the next run.

`set circular-trace-buffer on`
`set circular-trace-buffer off`
  Choose whether a tracing run should use a linear or circular buffer for trace data. A linear buffer will not lose any trace data, but may fill up prematurely, while a circular buffer will discard old trace data, but it will have always room for the latest tracepoint hits.

`show circular-trace-buffer`
  Show the current choice for the trace buffer. Note that this may not match the agent's current buffer handling, nor is it guaranteed to match the setting that might have been in effect during a past run, for instance if you are looking at frames from a trace file.

`set trace-buffer-size n`
`set trace-buffer-size unlimited`
  Request that the target use a trace buffer of n bytes. Not all targets will honor the request; they may have a compiled-in size for the trace buffer, or some other limitation. Set to a value of `unlimited` or `-1` to let the target use whatever size it likes. This is also the default.

`show trace-buffer-size`
  Show the current requested size for the trace buffer. Note that this will only match the actual size if the target supports size-setting, and was able to handle the requested size. For instance, if the target can only change buffer size between runs, this variable will not reflect the change until the next run starts. Use `tstatus` to get a report of the actual buffer size.

`set trace-user text`

`show trace-user`

`set trace-notes text`
  Set the trace run's notes.

`show trace-notes`
  Show the trace run's notes.

`set trace-stop-notes text`
  Set the trace run's stop notes. The handling of the note is as for `tstop` arguments; the set command is convenient way to fix a stop note that is mistaken or incomplete.

`show trace-stop-notes`
  Show the trace run's stop notes.

[ < ] | [ > ] |  | [ << ] | [ Up ] | [ >> ] |  |  |  |  | [Top] | [Contents] | [Index] | [ ? ]

### 13.1.10 Tracepoint Restrictions

There are a number of restrictions on the use of tracepoints. As described above, tracepoint data gathering occurs on the target without interaction from GDB. Thus the full capabilities of the debugger are not available during data gathering, and then at data examination time, you will be limited by only having what was collected. The following items describe some common problems, but it is not exhaustive, and you may run into additional difficulties not mentioned here.

- Tracepoint expressions are intended to gather objects (lvalues). Thus the full flexibility of GDB's expression evaluator is not available. You cannot call functions, cast objects to aggregate types, access convenience variables or modify values (except by assignment to trace state variables). Some language features may implicitly call functions (for instance Objective-C fields with accessors), and therefore cannot be collected either.

- Collection of local variables, either individually or in bulk with `$locals` or `$args`, during `while-stepping` may behave erratically. The stepping action may enter a new scope (for instance by stepping into a function), or the location of the variable may change (for instance it is loaded into a register). The tracepoint data recorded uses the location information for the variables that is correct for the tracepoint location. When the tracepoint is created, it is not possible, in general, to determine where the steps of a `while-stepping` sequence will advance the program--particularly if a conditional branch is stepped.

- Collection of an incompletely-initialized or partially-destroyed object may result in something that GDB cannot display, or displays in a misleading way.

- When GDB displays a pointer to character it automatically dereferences the pointer to also display characters of the string being pointed to. However, collecting the pointer during tracing does not automatically collect the string. You need to explicitly dereference the pointer and provide size information if you want to collect not only the pointer, but the memory pointed to. For example, `*ptr@50` can be used to collect the 50 element array pointed to by `ptr`.

- It is not possible to collect a complete stack backtrace at a tracepoint. Instead, you may collect the registers and a few hundred bytes from the stack pointer with something like `*(unsigned char *)$esp@300` (adjust to use the name of the actual stack pointer register on your target architecture, and the amount of stack you wish to capture). Then the `backtrace` command will show a partial backtrace when using a trace frame. The number of stack frames that can be examined depends on the sizes of the frames in the collected stack. Note that if you ask for a block so large that it goes past the bottom of the stack, the target agent may report an error trying to read from an invalid address.

- If you do not collect registers at a tracepoint, GDB can infer that the value of `$pc` must be the same as the address of the tracepoint and use that when you are looking at a trace frame for that tracepoint. However, this cannot work if the tracepoint has multiple locations (for instance if it was set in a function that was inlined), or if it has a `while-stepping` loop. In those cases GDB will warn you that it can't infer `$pc`, and default it to zero.

[ < ] | [ > ] |  | [ << ] | [ Up ] | [ >> ] |  |  |  |  | [Top] | [Contents] | [Index] | [ ? ]

## 13.2 Using the Collected Data

After the tracepoint experiment ends, you use GDB commands for examining the trace data. The basic idea is that each tracepoint collects a trace snapshot every time it is hit and another snapshot every time it single-steps. All these snapshots are consecutively numbered from zero and go into a buffer, and you can examine them later. The way you examine them is to focus on a specific trace snapshot. When the remote stub is focused on a trace snapshot, it will respond to all GDB requests for memory and registers by reading from the buffer which belongs to that snapshot, rather than from real memory or registers of the program being debugged. This means that all GDB commands (`print`, `info registers`, `backtrace`, etc.) will behave as if we were currently debugging the program state as it was when the tracepoint occurred. Any requests for data that are not in the buffer will fail.

> 13.2.1 `tfind n` |  | How to select a trace snapshot
> 13.2.2 `tdump` |  | How to display all data for a snapshot
> 13.2.3 `save tracepoints filename` |  | How to save tracepoints for a future run

[ < ] | [ > ] |  | [ << ] | [ Up ] | [ >> ] |  |  |  |  | [Top] | [Contents] | [Index] | [ ? ]

### 13.2.1 `tfind n`

The basic command for selecting a trace snapshot from the buffer is `tfind n`, which finds trace snapshot number n, counting from zero. If no argument n is given, the next snapshot is selected.

Here are the various forms of using the `tfind` command.

`tfind start`
  Find the first snapshot in the buffer. This is a synonym for `tfind 0` (since 0 is the number of the first snapshot).

`tfind none`
  Stop debugging trace snapshots, resume live debugging.

`tfind end`
  Same as ``tfind none'`.

`tfind`
  No argument means find the next trace snapshot.

`tfind -`
  Find the previous trace snapshot before the current one. This permits retracing earlier steps.

`tfind tracepoint num`
  Find the next snapshot associated with tracepoint num. Search proceeds forward from the last examined trace snapshot. If no argument num is given, it means find the next snapshot collected for the same tracepoint as the current snapshot.

`tfind pc addr`
  Find the next snapshot associated with the value addr of the program counter. Search proceeds forward from the last examined trace snapshot. If no argument addr is given, it means find the next snapshot with the same value of PC as the current snapshot.

`tfind outside addr1, addr2`
  Find the next snapshot whose PC is outside the given range of addresses (exclusive).

`tfind range addr1, addr2`
  Find the next snapshot whose PC is between addr1 and addr2 (inclusive).

`tfind line [file:]n`
  Find the next snapshot associated with the source line n. If the optional argument file is given, refer to line n in that source file. Search proceeds forward from the last examined trace snapshot. If no argument n is given, it means find the next line other than the one currently being examined; thus saying `tfind line` repeatedly can appear to have the same effect as stepping from line to line in a live debugging session.

The default arguments for the `tfind` commands are specifically designed to make it easy to scan through the trace buffer. For instance, `tfind` with no argument selects the next trace snapshot, and `tfind -` with no argument selects the previous trace snapshot. So, by giving one `tfind` command, and then simply hitting `RET` repeatedly you can examine all the trace snapshots in order. Or, by saying `tfind -` and then hitting `RET` repeatedly you can examine the snapshots in reverse order. The `tfind line` command with no argument selects the snapshot for the next source line executed. The `tfind pc` command with no argument selects the next snapshot with the same program counter (PC) as the current frame. The `tfind tracepoint` command with no argument selects the next trace snapshot collected by the same tracepoint as the current one.

In addition to letting you scan through the trace buffer manually, these commands make it easy to construct GDB scripts that scan through the trace buffer and print out whatever collected data you are interested in. Thus, if we want to examine the PC, FP, and SP registers from each trace frame in the buffer, we can say this:

    (gdb) tfind start
    (gdb) while ($trace_frame != -1)
    > printf "Frame %d, PC = %08X, SP = %08X, FP = %08X\n", \
              $trace_frame, $pc, $sp, $fp
    > tfind
    > end

    Frame 0, PC = 0020DC64, SP = 0030BF3C, FP = 0030BF44
    Frame 1, PC = 0020DC6C, SP = 0030BF38, FP = 0030BF44
    Frame 2, PC = 0020DC70, SP = 0030BF34, FP = 0030BF44
    Frame 3, PC = 0020DC74, SP = 0030BF30, FP = 0030BF44
    Frame 4, PC = 0020DC78, SP = 0030BF2C, FP = 0030BF44
    Frame 5, PC = 0020DC7C, SP = 0030BF28, FP = 0030BF44
    Frame 6, PC = 0020DC80, SP = 0030BF24, FP = 0030BF44
    Frame 7, PC = 0020DC84, SP = 0030BF20, FP = 0030BF44
    Frame 8, PC = 0020DC88, SP = 0030BF1C, FP = 0030BF44
    Frame 9, PC = 0020DC8E, SP = 0030BF18, FP = 0030BF44
    Frame 10, PC = 00203F6C, SP = 0030BE3C, FP = 0030BF14

Or, if we want to examine the variable `X` at each source line in the buffer:

    (gdb) tfind start
    (gdb) while ($trace_frame != -1)
    > printf "Frame %d, X == %d\n", $trace_frame, X
    > tfind line
    > end

    Frame 0, X = 1
    Frame 7, X = 2
    Frame 13, X = 255

[ < ] | [ > ] |  | [ << ] | [ Up ] | [ >> ] |  |  |  |  | [Top] | [Contents] | [Index] | [ ? ]

### 13.2.2 `tdump`

This command takes no arguments. It prints all the data collected at the current trace snapshot.

    (gdb) trace 444
    (gdb) actions
    Enter actions for tracepoint #2, one per line:
    > collect $regs, $locals, $args, gdb_long_test
    > end

    (gdb) tstart

    (gdb) tfind line 444
    #0  gdb_test (p1=0x11, p2=0x22, p3=0x33, p4=0x44, p5=0x55, p6=0x66)
    at gdb_test.c:444
    444        printp( "%s: arguments = 0x%X 0x%X 0x%X 0x%X 0x%X 0x%X\n", )

    (gdb) tdump
    Data collected at tracepoint 2, trace frame 1:
    d0             0xc4aa0085       -995491707
    d1             0x18     24
    d2             0x80     128
    d3             0x33     51
    d4             0x71aea3d        119204413
    d5             0x22     34
    d6             0xe0     224
    d7             0x380035 3670069
    a0             0x19e24a 1696330
    a1             0x3000668        50333288
    a2             0x100    256
    a3             0x322000 3284992
    a4             0x3000698        50333336
    a5             0x1ad3cc 1758156
    fp             0x30bf3c 0x30bf3c
    sp             0x30bf34 0x30bf34
    ps             0x0      0
    pc             0x20b2c8 0x20b2c8
    fpcontrol      0x0      0
    fpstatus       0x0      0
    fpiaddr        0x0      0
    p = 0x20e5b4 "gdb-test"
    p1 = (void *) 0x11
    p2 = (void *) 0x22
    p3 = (void *) 0x33
    p4 = (void *) 0x44
    p5 = (void *) 0x55
    p6 = (void *) 0x66
    gdb_long_test = 17 '\021'

    (gdb)

`tdump` works by scanning the tracepoint's current collection actions and printing the value of each expression listed. So `tdump` can fail, if after a run, you change the tracepoint's actions to mention variables that were not collected during the run.

Also, for tracepoints with `while-stepping` loops, `tdump` uses the collected value of `$pc` to distinguish between trace frames that were collected at the tracepoint hit, and frames that were collected while stepping. This allows it to correctly choose whether to display the basic list of collections, or the collections from the body of the while-stepping loop. However, if `$pc` was not collected, then `tdump` will always attempt to dump using the basic collection list, and may fail if a while-stepping frame does not include all the same data that is collected at the tracepoint hit.

[ < ] | [ > ] |  | [ << ] | [ Up ] | [ >> ] |  |  |  |  | [Top] | [Contents] | [Index] | [ ? ]

### 13.2.3 `save tracepoints filename`

This command saves all current tracepoint definitions together with their actions and passcounts, into a file ``filename'` suitable for use in a later debugging session. To read the saved tracepoint definitions, use the `source` command (see section 23.1.3 Command Files). The `save-tracepoints` command is a deprecated alias for `save tracepoints`

[ < ] | [ > ] |  | [ << ] | [ Up ] | [ >> ] |  |  |  |  | [Top] | [Contents] | [Index] | [ ? ]

## 13.3 Convenience Variables for Tracepoints

`(int) $trace_frame`
  The current trace snapshot (a.k.a. frame) number, or -1 if no snapshot is selected.

`(int) $tracepoint`
  The tracepoint for the current trace snapshot.

`(int) $trace_line`
  The line number for the current trace snapshot.

`(char []) $trace_file`
  The source file for the current trace snapshot.

`(char []) $trace_func`
  The name of the function containing `$tracepoint`.

Note: `$trace_file` is not suitable for use in `printf`, use `output` instead.

Here's a simple example of using these convenience variables for stepping through all the trace snapshots and printing some of their data. Note that these are not the same as trace state variables, which are managed by the target.

    (gdb) tfind start

    (gdb) while $trace_frame != -1
    > output $trace_file
    > printf ", line %d (tracepoint #%d)\n", $trace_line, $tracepoint
    > tfind
    > end

[ < ] | [ > ] |  | [ << ] | [ Up ] | [ >> ] |  |  |  |  | [Top] | [Contents] | [Index] | [ ? ]

## 13.4 Using Trace Files

In some situations, the target running a trace experiment may no longer be available; perhaps it crashed, or the hardware was needed for a different activity. To handle these cases, you can arrange to dump the trace data into a file, and later use that file as a source of trace data, via the `target tfile` command.

`tsave [ -r ] filename`
`tsave [-ctf] dirname`
  Save the trace data to filename. By default, this command assumes that filename refers to the host filesystem, so if necessary GDB will copy raw trace data up from the target and then save it. If the target supports it, you can also supply the optional argument `-r` ("remote") to direct the target to save the data directly into filename in its own filesystem, which may be more efficient if the trace buffer is very large. (Note, however, that `target tfile` can only read from files accessible to the host.) By default, this command will save trace frame in tfile format. You can supply the optional argument `-ctf` to save date in CTF format. The Common Trace Format (CTF) is proposed as a trace format that can be shared by multiple debugging and tracing tools. Please go to @indicateurl{http://www.efficios.com/ctf} to get more information.

`target tfile filename`
`target ctf dirname`
  Use the file named filename or directory named dirname as a source of trace data. Commands that examine data work as they do with a live target, but it is not possible to run any new trace experiments. `tstatus` will report the state of the trace run at the moment the data was saved, as well as the current trace frame you are examining. filename or dirname must be on a filesystem accessible to the host.

      (gdb) target ctf ctf.ctf
      (gdb) tfind
      Found trace frame 0, tracepoint 2
      39            ++a;  /* set tracepoint 1 here */
      (gdb) tdump
      Data collected at tracepoint 2, trace frame 0:
      i = 0
      a = 0
      b = 1 '\001'
      c = {"123", "456", "789", "123", "456", "789"}
      d = {{{a = 1, b = 2}, {a = 3, b = 4}}, {{a = 5, b = 6}, {a = 7, b = 8}}}
      (gdb) p b
      $1 = 1

[ << ] | [ >> ] |  |  |  |  |  | [Top] | [Contents] | [Index] | [ ? ]

This document was generated by GNAT Mailserver on April, 17 2014 using texi2html