You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
195 lines
8.0 KiB
195 lines
8.0 KiB
.TH argdist 8 "2016-02-11" "USER COMMANDS"
|
|
.SH NAME
|
|
argdist \- Trace a function and display a histogram or frequency count of its parameter values. Uses Linux eBPF/bcc.
|
|
.SH SYNOPSIS
|
|
.B argdist [-h] [-p PID] [-z STRING_SIZE] [-i INTERVAL] [-d DURATION] [-n COUNT] [-v] [-T TOP] [-H specifier] [-C specifier] [-I header]
|
|
.SH DESCRIPTION
|
|
argdist attaches to function entry and exit points, collects specified parameter
|
|
values, and stores them in a histogram or a frequency collection that counts
|
|
the number of times a parameter value occurred. It can also filter parameter
|
|
values and instrument multiple entry points at once.
|
|
|
|
Since this uses BPF, only the root user can use this tool.
|
|
.SH REQUIREMENTS
|
|
CONFIG_BPF and bcc.
|
|
.SH OPTIONS
|
|
.TP
|
|
\-h
|
|
Print usage message.
|
|
.TP
|
|
\-p PID
|
|
Trace only functions in the process PID.
|
|
.TP
|
|
\-z STRING_SIZE
|
|
When collecting string arguments (of type char*), collect up to STRING_SIZE
|
|
characters. Longer strings will be truncated.
|
|
.TP
|
|
\-i INTERVAL
|
|
Print the collected data every INTERVAL seconds. The default is 1 second.
|
|
.TP
|
|
\-d DURATION
|
|
Total duration of trace in seconds.
|
|
.TP
|
|
\-n NUMBER
|
|
Print the collected data COUNT times and then exit.
|
|
.TP
|
|
\-v
|
|
Display the generated BPF program, for debugging purposes.
|
|
.TP
|
|
\-T TOP
|
|
When collecting frequency counts, display only the top TOP entries.
|
|
.TP
|
|
\-H specifiers, \-C specifiers
|
|
One or more probe specifications that instruct argdist which functions to
|
|
probe, which parameters to collect, how to aggregate them, and whether to perform
|
|
any filtering. See SPECIFIER SYNTAX below.
|
|
.TP
|
|
\-I header
|
|
One or more header files that should be included in the BPF program. This
|
|
enables the use of structure definitions, enumerations, and constants that
|
|
are available in these headers. You should provide the same path you would
|
|
include in the BPF program, e.g. 'linux/blkdev.h' or 'linux/time.h'. Note: in
|
|
many cases, argdist will deduce the necessary header files automatically.
|
|
.SH SPECIFIER SYNTAX
|
|
The general specifier syntax is as follows:
|
|
|
|
.B {p,r,t,u}:{[library],category}:function(signature)[:type[,type...]:expr[,expr...][:filter]][#label]
|
|
.TP
|
|
.B {p,r,t,u}
|
|
Probe type \- "p" for function entry, "r" for function return, "t" for kernel
|
|
tracepoint, "u" for USDT probe; \-H for histogram collection, \-C for frequency count.
|
|
Indicates where to place the probe and whether the probe should collect frequency
|
|
count information, or aggregate the collected values into a histogram. Counting
|
|
probes will collect the number of times every parameter value was observed,
|
|
whereas histogram probes will collect the parameter values into a histogram.
|
|
Only integral types can be used with histogram probes; there is no such limitation
|
|
for counting probes.
|
|
.TP
|
|
.B [library]
|
|
Library containing the probe.
|
|
Specify the full path to the .so or executable file where the function to probe
|
|
resides. Alternatively, you can specify just the lib name: for example, "c"
|
|
refers to libc. If no library name is specified, the kernel is assumed.
|
|
.TP
|
|
.B category
|
|
The category of the kernel tracepoint. For example: net, sched, block.
|
|
.TP
|
|
.B function(signature)
|
|
The function to probe, and its signature.
|
|
The function name must match exactly for the probe to be placed. The signature,
|
|
on the other hand, is only required if you plan to collect parameter values
|
|
based on that signature. For example, if you only want to collect the first
|
|
parameter, you don't have to specify the rest of the parameters in the signature.
|
|
When capturing kernel tracepoints, this should be the name of the event, e.g.
|
|
net_dev_start_xmit. The signature for kernel tracepoints should be empty. When
|
|
capturing USDT probes, this should be the name of the probe, e.g. reloc_complete.
|
|
The signature for USDT probes should be empty.
|
|
.TP
|
|
.B [type[,type...]]
|
|
The type(s) of the expression(s) to capture.
|
|
This is the type of the keys in the histogram or raw event collection that are
|
|
collected by the probes.
|
|
.TP
|
|
.B [expr[,expr...]]
|
|
The expression(s) to capture.
|
|
These are the values that are assigned to the histogram or raw event collection.
|
|
You may use the parameters directly, or valid C expressions that involve the
|
|
parameters, such as "size % 10".
|
|
Tracepoints may access a special structure called "args" that is formatted
|
|
according to the tracepoint format (which you can obtain using tplist).
|
|
For example, the block:block_rq_complete tracepoint can access args->nr_sector.
|
|
USDT probes may access the arguments defined by the tracing program in the
|
|
special arg1, arg2, ... variables. To obtain their types, use the tplist tool.
|
|
Return probes can use the argument values received by the
|
|
function when it was entered, through the $entry(paramname) special variable.
|
|
Return probes can also access the function's return value in $retval, and the
|
|
function's execution time in nanoseconds in $latency. Note that adding the
|
|
$latency or $entry(paramname) variables to the expression will introduce an
|
|
additional probe at the function's entry to collect this data, and therefore
|
|
introduce additional overhead.
|
|
.TP
|
|
.B [filter]
|
|
The filter applied to the captured data.
|
|
Only parameter values that pass the filter will be collected. This is any valid
|
|
C expression that refers to the parameter values, such as "fd == 1 && length > 16".
|
|
The $entry, $retval, and $latency variables can be used here as well, in return
|
|
probes.
|
|
The filter expression may also use the STRCMP pseudo-function to compare
|
|
a predefined string to a string argument. For example: STRCMP("test.txt", file).
|
|
The order of arguments is important: the first argument MUST be a quoted
|
|
literal string, and the second argument can be a runtime string.
|
|
.TP
|
|
.B [label]
|
|
The label that will be displayed when printing the probed values. By default,
|
|
this is the probe specifier.
|
|
.SH EXAMPLES
|
|
.TP
|
|
Print a histogram of allocation sizes passed to kmalloc:
|
|
#
|
|
.B argdist -H 'p::__kmalloc(u64 size):u64:size'
|
|
.TP
|
|
Print a count of how many times process 1005 called malloc with an allocation size of 16 bytes:
|
|
#
|
|
.B argdist -p 1005 -C 'p:c:malloc(size_t size):size_t:size:size==16'
|
|
.TP
|
|
Snoop on all strings returned by gets():
|
|
#
|
|
.B argdist -C 'r:c:gets():char*:$retval'
|
|
.TP
|
|
Print a histogram of read sizes that were longer than 1ms:
|
|
#
|
|
.B argdist -H 'r::__vfs_read(void *file, void *buf, size_t count):size_t:$entry(count):$latency > 1000000'
|
|
.TP
|
|
Print frequency counts of how many times writes were issued to a particular file descriptor number, in process 1005:
|
|
#
|
|
.B argdist -p 1005 -C 'p:c:write(int fd):int:fd'
|
|
.TP
|
|
Print a histogram of error codes returned by read() in process 1005:
|
|
#
|
|
.B argdist -p 1005 -H 'r:c:read()'
|
|
.TP
|
|
Print a histogram of buffer sizes passed to write() across all processes, where the file descriptor was 1 (STDOUT):
|
|
#
|
|
.B argdist -H 'p:c:write(int fd, const void *buf, size_t count):size_t:count:fd==1'
|
|
.TP
|
|
Count fork() calls in libc across all processes, grouped by pid:
|
|
#
|
|
.B argdist -C 'p:c:fork():int:$PID;fork per process'
|
|
.TP
|
|
Print histogram of number of sectors in completing block I/O requests:
|
|
#
|
|
.B argdist -H 't:block:block_rq_complete():u32:nr_sector'
|
|
.TP
|
|
Aggregate interrupts by interrupt request (IRQ):
|
|
#
|
|
.B argdist -C 't:irq:irq_handler_entry():int:irq'
|
|
.TP
|
|
Print the functions used as thread entry points and how common they are:
|
|
#
|
|
.B argdist -C 'u:pthread:pthread_start():u64:arg2' -p 1337
|
|
.TP
|
|
Print histograms of sleep() and nanosleep() parameter values:
|
|
#
|
|
.B argdist -H 'p:c:sleep(u32 seconds):u32:seconds' -H 'p:c:nanosleep(struct timespec *req):long:req->tv_nsec'
|
|
.TP
|
|
Spy on writes to STDOUT performed by process 2780, up to a string size of 120 characters:
|
|
#
|
|
.B argdist -p 2780 -z 120 -C 'p:c:write(int fd, char* buf, size_t len):char*:buf:fd==1'
|
|
.TP
|
|
Group files being read from and the read sizes from __vfs_read:
|
|
#
|
|
.B argdist -C 'p::__vfs_read(struct file *file, void *buf, size_t count):char*,size_t:file->f_path.dentry->d_iname,count:file->f_path.dentry->d_iname[0]!=0'
|
|
.SH SOURCE
|
|
This is from bcc.
|
|
.IP
|
|
https://github.com/iovisor/bcc
|
|
.PP
|
|
Also look in the bcc distribution for a companion _examples.txt file containing
|
|
example usage, output, and commentary for this tool.
|
|
.SH OS
|
|
Linux
|
|
.SH STABILITY
|
|
Unstable - in development.
|
|
.SH AUTHOR
|
|
Sasha Goldshtein
|