Go 1.10 added inlined calls to debug_info, this commit adds support
for DW_TAG_inlined_call to delve, both for stack traces (where
inlined calls will appear as normal stack frames) and to correct
the behavior of next, step and stepout.
The calls to Next and Frame of stackIterator continue to work
unchanged and only return real stack frames, after reading each line
appendInlinedCalls is called to unpacked all the inlined calls that
involve the current PC.
The fake stack frames produced by appendInlinedCalls are
distinguished from real stack frames by having the Inlined attribute
set to true. Also their Current and Call locations are treated
differently. The Call location will be changed to represent the
position inside the inlined call, while the Current location will
always reference the real stack frame. This is done because:
* next, step and stepout need to access the debug_info entry of
the real function they are stepping through
* we are already manipulating Call in different ways while Current
is just what we read from the call stack
The strategy remains mostly the same, we disassemble the function
and we set a breakpoint on each instruction corresponding to a
different file:line. The function in question will be the one
corresponding to the first real (i.e. non-inlined) stack frame.
* If the current function contains inlined calls, 'next' will not
set any breakpoints on instructions that belong to inlined calls. We
do not do this for 'step'.
* If we are inside an inlined call that makes other inlined
functions, 'next' will not set any breakpoints that belong to
inlined calls that are children of the current inlined call.
* If the current function is inlined the breakpoint on the return
address won't be set, because inlined frames don't have a return
address.
* The code we use for stepout doesn't work at all if we are inside
an inlined call, instead we call 'next' but instruct it to remove
all PCs belonging to the current inlined call.
* Extend the "frame" command to set the current frame.
Command
frame 3
sets up so that subsequent "print", "set", "whatis" command
will operate on frame 3.
frame 3 print foo
continues to work.
Added "up", "down". They move the current frame up or down.
Implementation note:
This changes removes "scopePrefix" mode from the terminal/command.go and instead
have the command examine the goroutine/frame value to see if it is invoked in a
scoped context.
* Rename Command.Frame -> Command.frame.
updates vendored version of x86asm, adds a symbol lookup function to
pass to the disassembler.
This will show global symbol names in the disassembly like go tool
objdump does.
Registers XMM1 and XMM2 get sometimes clobbered between the time we set
them and the panic. There is no guarantee that they won't in the go
spec so we shouldn't expect any register to keep its value. However
since this seems to only affect 1 and 2 let's try to use 9 and 10
instead.
Every time we read an empty string we accidentally issue a read for 0
bytes at address 0, this is fine for real memory but the core file
reader doesn't like it.
Fixes an issue reported on the mailing list.
If a breakpoint is hit close to process death on a thread that isn't
the group leader the process could die while we are trying to stop it.
This can be easily reproduced by having the goroutine that's executing
main.main (which will almost always run on the thread group leader)
wait for a second goroutine before exiting, then setting a breakpoint
on the second goroutine and stepping through it (see TestIssue1101 in
proc_test.go).
When stepping over the return instruction of main.f the deferred
wg.Done() call will be executed which will cause the main goroutine to
resume and proceed to exit. Both the temporary breakpoint on wg.Done
and the temporary breakpoint on the return address of main.f will be in
close proximity to main.main calling os.Exit() and causing the death of
the thread group leader.
Under these circumstances the call to native.(*Thread).waitFast in
native.(*Thread).halt can hang forever due to a bug similar to
https://sourceware.org/bugzilla/show_bug.cgi?id=12702 (see comment in
native.(*Thread).wait for an explanation).
Replacing waitFast with a normal wait work in most circumstances,
however, besides the performance hit, it looks like in this
circumstances trapWait sometimes receives a spurious SIGTRAP on the
dying group leader which would cause the subsequent call to wait in
halt to accidentally reap the process without noting that it did exit.
Instead this patch removes the call to wait from halt and instead calls
trapWait in a loop in setCurrentBreakpoints until all threads are set
to running=false. This is also a better fix than the workaround to
ESRCH error while setting current breakpoints implemented in 94b50d.
Fixes#1101
If the last entry of the package path contains a '.' the corresponding
DIEs for its types will replace the '.' character with '%2e'. We must
do the same when resolving the package path of the concrete type of an
interface variable.
Fixes#1137
debug_info entries can use DW_AT_abstract_origin to inherit the
attributes of another entry, supporting this attribute is necessary to
support DW_TAG_inlined_subroutine.
Go, starting with 1.10, emits DW_TAG_inlined_subroutine entries when
inlining is enabled.
* Handle race between fork and task_for_pid
On macOS a call to fork and a subsequent call to task_for_pid will race each other. This is because the macOS kernel assigns a new proc_t structure early but the new task, thread and uthread come much later. The function exec_mach_imgact in the XNU sources contains this logic.
In a system under load or one with delays in fork processing (i.e. various security software), task_for_pid as currently called by Delve often returns the parent task. This can be seen by printing out the task number around line 86. In a normal system we would see three calls:
-> ~/go/bin/dlv --listen=localhost:59115 --headless=true --api-version=2 --backend=native exec ./___main_go --
Task: 9731
Task: 9731
Task: 9731
API server listening at: 127.0.0.1:59115
This is the result on a system where the race is lost:
-> ~/go/bin/dlv --listen=localhost:59115 --headless=true --api-version=2 --backend=native exec ./___main_go --
Task: 8707
Task: 10499
Task: 10499
could not launch process: could not get thread count
In this latter case, task 8707 is the parent task. The child task of 10499 was desired and hence the error.
This code change checks to make sure the returned task is not that of the parent. If it is, it retries. It's possible other macOS reported Delve issues are the result of this failed race.
* proc: correct formatting
Much like the bug in issue #1031 and commit
f6f6f0bf13e4c708cb501202b83a6327a0f00e31 pointers can also escape to
the heap and then have a zero address (and no children) when we
autodereference.
1. Mark autodereferenced escaped variables with a 0 address as
unreadable.
2. Add guards to the pretty printers for unsafe.Pointer and pointers.
Fixes#1075
Depending on how the runtime schedules our goroutines we can get
unlucky and have the first call to runtime.newstack we intercept be for
a different goroutine (usually the garbage collector).
Only check stacktraces that happen on the same goroutine that executed
main.main.
The runtime calls into g0 in many places, not necessarily using
runtime.systemstack or runtime.asmcgocall.
One example of this is the call to runtime.newstack inside
runtime.morestack.
If we stop the process while one goroutine is executing
runtime.newstack we would be unable to fully scan its stack because we
don't know that we have to switch back to the goroutine stack after
runtime.newstack.
Instead of tracking down every possible way that the runtime switches
to g0 we switch to the goroutine stack immediately after the top of the
stack, unless cgo is being executed on the systemstack.
Fixes#1066
Adds a configuration option (show-location-expr) that when activated
will cause the whatis command to also print the DWARF location
expression for a variable.
On macOS, externally linked programs will have an abbrev for
DW_TAG_subprogram without the haschildren flag set. We should handle
this case instead of expecting all DW_TAG_subprogram entries to have
list of children.
Fixes#1034
When creating a stack trace we should switch between the goroutine
stack and the system stack (where cgo code is executed) as appropriate
to reconstruct the logical stacktrace.
Goroutines that are currently executing on the system stack will have
the SystemStack flag set, frames of the goroutine stack will have a
negative FrameOffset (like always) and frames of the system stack will
have a positive FrameOffset (which is actually just the CFA value for
the frame).
Updates #935
Either the CPU or the kernel may not support the calls we do when
retrieving floating point registers, this isn't an error we should
propagate.
Also improve the error reporint of pkg/proc/native.fpRegisters.
Fixes#1022
StepBreakpoints are set on CALL instructions, when they are hit we
disassemble the current instruction, figure out the destination address
and set a breakpoint after the prologue of the called function.
In order to disassemble the current instruction we disassemble the area
of memory starting from PC and going to PC+15 (because 15 bytes is the
maximum length of one instruction on AMD64). This means that we won't
just disassemble one instruction but also a few instructions following
it ending with one truncated instruction.
This usually works fine but sometimes the disassembler will panic with
an array out of bounds error when trying to disassemble a truncated
instruction. To avoid this problem this commit changes the funciton
disassemble to take one extra parameter, singleInstr, when singleInstr
is set disassemble will quit after disassembling a single instruction.
Replace the unsafe.Pointer type of the buf field of channels with the
appropriate array type, allow expressions accessing member field of the
channel struct.
Fixes#962
Conditional breakpoints with unmet conditions would cause next and step
to skip the line.
This breakpoint changes the Kind field of proc.Breakpoint from a single
value to a bit field, each breakpoint object can represent
simultaneously a user breakpoint and one internal breakpoint (of which
we have several different kinds).
The breakpoint condition for internal breakpoints is stored in the new
internalCond field of proc.Breakpoint so that it will not conflict with
user specified conditions.
The breakpoint setting code is changed to allow overlapping one
internal breakpoint on a user breakpoint, or a user breakpoint on an
existing internal breakpoint. All other combinations are rejected. The
breakpoint clearing code is changed to clear the UserBreakpoint bit and
only remove the phisical breakpoint if no other bits are set in the
Kind field. ClearInternalBreakpoints does the same thing but clearing
all bits that aren't the UserBreakpoint bit.
Fixes#844
Move some duplicate code, related to breakpoints, that was in both
backends into a single place.
This is in preparation to solve issue #844 (conditional breakpoints
make step and next fail) which will make this common breakpoint code
more complicated.
Instead of only tracking a few cherrypicked registers in stack.go track
all DWARF registers.
This is needed for cgo code and for the locationlists emitted by go in
1.10:
* The debug_frame sections emitted by C compilers can not be used
without tracking all registers
* the loclists emitted by go1.10 need all registers of a frame to be
interpreted.
A thread could terminate between the point when we stop for a
breakpoint and the point where we send a stop signal to all threads, if
this happens setCurrentBreakpoints will fail with an error.
We should tolerate this.
For some reason this happens very frequently when running delve on
processes with the race detector enabed.
Adds test command line flag to compile target fixtures using the -race flag.
Multiple tests will fail because of https://github.com/golang/go/issues/22600
but eventually this should work.
