Go 1.12 introduced a change to the internal map representation where
empty map cells can be marked with a tophash value of 1 instead of just
0.
Fixes#1531
RestoreRegisters on linux would also restore FS_BASE and GS_BASE, if
the target goroutine migrated to a different thread during the call
injection this would result in two threads of the target process
pointing to the same TLS area which would greatly confuse the target
runtime, leading to fatal panics with nonsensical stack traces.
Other backends are unaffected:
- native/windows doesn't store the TLS in the same CONTEXT struct as
the other register values.
- native/darwin doesn't support function calls (and wouldn't store the
TLS value in the same struct)
- gdbserial/rr doesn't support function calls (because it's a
recording)
- gsdbserial/lldb extracts the value of TLS by executing code in the
target process.
* *: use loglevel to control what gets logged instead of output redirection
This stops logrus from doing all the formatting just to discard it
immediately afterwards.
* logflags: replace default formatter of logrus
The default formatter of logrus emits logs in two different formats
depending on whether or not the output is going to a terminal. The
output format for non-terminals is indented to be machine readable, but
we mostly read logs ourselves and the excessive quoting makes that
format unreadable.
When outputting to terminals it uses ANSI escape codes unconditionally,
without checking whether the terminal it is connected to actually
supports colors.
This commit replaces the default formatter with a much simpler
formatter that always uses a more readable format, doesn't use colors
and places the key-value pairs at the beginning of the line (which is a
better match for how we use them).
* cmd/dlv: add command line options to redirect logs
Adds two options, --log-to-file and --log-to-fd, to redirect logs to a
file or to a file descriptor.
When one of those two options is specified the "API server listening
at:" message will also be redirected to the specified file/file
descriptor.
This allows clients that want to use the "API server listening at:"
message to do so even if they want to redirect the target's stdout to
another file or device.
Implements #1179, #1523
Adds initial support for plugins, this is only the code needed to keep
track of loaded plugins on linux (both native and gdbserial backend).
It does not actually implement support for debugging plugins on linux.
Updates #865
Like we do with unrecovered panics, create a default breakpoint to
catch runtime errors that will cause the program to terminate.
Primarily intended to give users the opportunity to examine the state
of a deadlocked process.
runtime.clone (on some operating systems?) work similarly to fork:
when a thread calls runtime.clone a new thread is created. For a
short period of time both the parent thread and the child thread
appear to be running the same goroutine, until the child thread
adjusts its TLS to point to the correct goroutine.
This means that proc.GetG for a thread that's currently running
'runtime.clone' could be wrong and, consequently, the field
proc.(G).thread of a G struct returned by GoroutinesInfo could be
also wrong. And, finally, that FindGoroutine could sometimes return
a *G with a bad associated thread if the goroutine of interest
recently called 'runtime.clone'.
To work around this problem this commit makes two changes:
1. proc.GetG will return nil for all threads executing runtime.clone.
2. FindGoroutine will return the selected goroutine as long as the
ID matches the one requested.
Change (1) takes care of the 'runtime.clone' problem. If we stop
the target process shortly after a thread executed the SYSCALL
instruction in 'runtime.clone' there are three possibilities:
a. Both the parent thread and the child thread are stopped inside
'runtime.clone'. In this case the state we report is slightly
incorrect, because both threads will be reported as not running any
goroutine when we do know which goorutine one of them (the parent)
is running. This doesn't actually matter since runtime.clone is
always called on the system stack and therefore the goroutine in
runtime.allgs will have the correct location.
b. The child thread managed to exit 'runtime.clone' but the parent
thread didn't. This is similar to (a) but in this case GetG on the
child thread will return the correct goroutine. GetG on the parent
thread will still return (incorrectly) nil but this doesn't matter
for the samer reason as described in (a).
c. The parent thread managed to exit 'runtime.clone' but the child
thread didn't. In this case GetG will return the correct goroutine
both for the parent thread (because it's not executing runtime.clone)
and the child thread.
Change (2) means that even if a thread has a completely nonsensical
TLS (for example because it's set through cgo) evaluating a variable
with a valid GoroutineID will still work as long as it's the current
goroutine (which is the most common case). This change also doubles
as an optimization for FindGoroutine.
Fixes#1469
When compression is applied by default running the DWZ tool on the
resulting binary will crash.
The actual default compression code will look and see if compression
makes any difference and if so replace the normal `.debug_*` section
with `.zdebug_*`. This is why it may not have been hit before. On one of
my workstations I build with 1.12rc1 and no compression happens, but on
a Fedora VM I build and the binary results in compressed DWARF sections.
Adding this flag will make this test more consistent overall.
FindGoroutine can be slow when there are many goroutines running. This
can not be fixed in the general case, however:
1. Instead of getting the entire list of goroutines at once just get a
few at a time and return as soon as we find the one we want.
2. Since FindGoroutine is mostly called by ConvertEvalScope and users
are more likely to request informations about a goroutine running on a
thread, look within the threads first.
The repository is being switched from the personal account
github.com/derekparker/delve to the organization account
github.com/go-delve/delve. This patch updates imports and docs, while
preserving things which should not be changed such as my name in the
CHANGELOG and in TODO comments.
Some build environments (such as when building RPMs) enjoy symlinking
things. This unfortunately causes our tests to fail as we record the
path of fixtures and use that when looking up file:line information.
However, the debug info in the binary records the original file
location, not the location of the symlink.
When casting an integer into a struct pointer we make a fake pointer
variable that doesn't have an address, maybeDereference and
structMember should still work on this kind of Variable.
Fixes#1432
The size of the TLS memory arena needs to be aligned to pointer sized
boundaries on 86x64 architectures, otherwise some programs using cgo
will not have the correct offset for the g struct.
No tests because reproducing this problem depends on behavior of the
GNU ld linker caused by unclear influences.
Fixes#1428.
Goroutine id == 0 is special (there can be many goroutines with id 0).
If the caller of FindGoroutine asks for gid==0 and current thread is
running a goroutine 0 (i.e. either no goroutine or a special
goroutine) return whatever goroutine is running on the current thread.
Updates #1428
Minidumps are the windows equivalent of unix core files.
This commit updates pkg/proc/core so that it can open and read windows
minidumps.
Updates #794
If a function can be inlined it will appear as two entries in
debug_info. A DW_TAG_subprogram entry with DW_AT_inlined = true (that
will be used as the abstract origin) and a second DW_TAG_subprogram
entry with an abstract origin.
To retrieve the name of this second entry we must load its abstract
origin.
If proc.Step encounters a CALL instruction that points to an address
that isn't associated with any function it should still follow the
CALL.
The circumstances creating this problem do not normally occur, it was
encountered in the process of fixing a bug created by Go1.12.
It was never true that return variables were in the inverse order.
Instead in Go1.11 return variables are saved in debug_info in an
arbitrary order and inverting them just happened to work for this
specific example.
This bug was fixed in Go 1.12, regardless we should attempt to
rearrange return variables anyway.
When a location expression requests a register check that we have as
many bytes in the register as requested and if we don't report the
error.
Updates #1416
Instead of unconditionally returning all present goroutines,
GoroutinesInfo now allows specifying a range (start and count). In
addition to the array of goroutines and the error, it now also returns
the next goroutine to be processed, to be used as 'start' argument on
the next call, or 0 if all present goroutines have already been
processed.
This way clients can avoid eating large amounts of RAM while debugging
core dumps and processes with a exceptionally high amount of goroutines.
Fixes#1403
This patch is a slight refactor to share more code used for genericprocess initialization. There will always be OS/backend specificinitialization, but as much as can be shared should be to preventduplicating of any logic (setting internal breakpoints, loading bininfo,etc).
Users can create sparse maps in two ways, either by:
a) adding lots of entries to a map and then deleting most of them, or
b) using the make(mapType, N) expression with a very large N
When this happens reading the resulting map will be very slow
because loadMap needs to scan many buckets for each entry it finds.
Technically this is not a bug, the user just created a map that's
very sparse and therefore very slow to read. However it's very
annoying to have the debugger hang for several seconds when trying
to read the local variables just because one of them (which you
might not even be interested into) happens to be a very sparse map.
There is an easy mitigation to this problem: not reading any
additional buckets once we know that we have already read all
entries of the map, or as many entries as we need to fulfill the
MaxArrayValues parameter.
Unfortunately this is mostly useless, a VLSM (Very Large Sparse Map)
with a single entry will still be slow to access, because the single
entry in the map could easily end up in the last bucket.
The obvious solution to this problem is to set a limit to the
number of buckets we read when loading a map. However there is no
good way to set this limit.
If we hardcode it there will be no way to print maps that are beyond
whatever limit we pick.
We could let users (or clients) specify it but the meaning of such
knob would be arcane and they would have no way of picking a good
value (because there is no objectively good value for it).
The solution used in this commit is to set an arbirtray limit on
the number of buckets we read but only when loadMap is invoked
through API calls ListLocalVars and ListFunctionArgs. In this way
`ListLocalVars` and `ListFunctionArgs` (which are often invoked
automatically by GUI clients) remain fast even in presence of a
VLSM, but the contents of the VLSM can still be inspected using
`EvalVariable`.
The linux version of proc/native and proc/core contained largely
overlapping implementations of the register handling code, deduplicate
it by moving it into proc/linutil.
This patch allows the `trace` CLI subcommand to display return values of
a function. Additionally, it will also display information on where the
function exited, which could also be helpful in determining the path
taken during function execution.
Fixes#388
Some libraries (for example steam_api64.dll) will send this exception
code to set the thread name on Microsoft VisualC.
In theory it should be fine to send the exception back to the target,
which is responsible for setting a handler for it, in practice in some
cases (steam_api64.dll) this will crash the program. So we'll mask it
instead.
Fixes#1383
Continue did not resume execution after a call to CallFunction if the
point where the process was stopped, before the call CallFunction, was
a breakpoint.
Fixes#1374
Support for position independent executables (PIE) on the native linux
backend, the gdbserver backend on linux and the core backend.
Also implemented in the windows native backend, but it can't be tested
because go doesn't support PIE on windows yet.
On macOS 10.14 Apple changed the command line tools so that system
headers now need to be manually installed.
Instead of adding one extra install step to the install procedure add a
build tag to allow compilation of delve without the native backend on
macOS. By default (i.e. when using `go get`) this is how delve will be
compiled on macOS, the make script is changed to enable compiling the
native backend if the required dependencies have been installed.
Insure that both configuration still build correctly on Travis CI and
change the documentation to describe how to compile the native backend
and that it isn't normally needed.
Fixes#1359
With this syntax users do not need to type the concrete type of an
interface variable to access its contents. This also sidesteps the
problem where the serialization of a type by go/printer is different
from the one used for debug_info type names.
Updates #1328
Go allows converting a single integer value to string, resulting in a
string containing a single unicode rune with the same code as the value
of the integer.
Allow the same conversion to happen.
Fixes#1322
Add a flag to Stackframe that indicates where the stack frame is the
bottom-most frame of the stack. This allows clients to know whether the
stack trace terminated normally or if it was truncated because the
maximum depth was reached.
Add a truncation message to the 'stack' command.
Fncall.go was written with the assumption that the object returned by
proc.Thread.Registers does not change after we call
proc.Thread.SetPC/etc.
This is true for the native backend but not for gdbserial. I had
anticipated this problem and introduced the Save/SavedRegisters
mechanism during the first implementation of fncall.go but that's
insufficient.
Instead:
1. clarify that the object returned by proc.Thread.Registers could
change when the CPU registers are modified.
2. add a Copy method to Registers that returns a copy of the registers
that are guaranteed not to change when the CPU registers change.
3. remove the Save/SavedRegisters mechanism.
This solution leaves us the option, in the future, to cache the output
of proc.(Thread).Registers, avoiding a system call every time it's
called.
Normally variables that have a named struct as a type will get a
typedef entry as their type, sometimes however the Go linker will
decide to use the DW_TAG_structure_type entry instead.
For consistency always wrap a struct type into a typedef when we are
creating a new variables (see comment in newVariable for exceptions).
This fixes a bug where it would be impossible to call methods on a
global variable.
Evaluates var.method expressions into a variable holding the
corresponding method with the receiver variable as a child, in
preparation for extending CallFunction so that it can call methods.
Changes (*Variable).setValue so that it can be used in CallFunction to
set up the argument frame for the function call, adding the ability to:
- nil nillable types
- set strings to the empty string
- copy from one structure to another (including strings and slices)
- convert any interface type to interface{}
- convert pointer shaped types (map, chan, pointers, and structs
consisting of a single pointer field) to interface{}
This covers all cases where an assignment statement can be evaluated
without allocating memory or calling functions in the target process.
This patch makes it so inlined functions are returned in the
function
list, and also allows users to set breakpoints on the call site of
inlined functions.
Fixes#1261
If we send a process to foreground while the headless instance may get
a SIGTTOU/SIGTTIN, if not ignored this signal will stop the headless.
It's not clear why this only happens the second time we do this but
that's how it is.
Also removes the direct syscall to TIOCSPGRP and lets the go runtime do
it instead.
Fixes#1279
Adds -defer flag to the stack command that decorates the stack traces
by associating each stack frame with its deferred calls.
Reworks proc.next to use this feature instead of using proc.DeferPC,
laying the groundwork to implement #1240.
There is no guarantee that files will end up stored contiguously in the
debug_line section which makes this optimization wrong in the general
case.
In particular with recent versions of go1.11 and a go.mod file present
the go compiler seems to sometimes produce executables that actually
violate this assumption.
Implements the function call injection protocol introduced in go 1.11
by https://go-review.googlesource.com/c/go/+/109699.
This is only the basic support, see TODO comments in pkg/proc/fncall.go
for a list of missing features.
Updates #119
The JSON-RPC layer doesn't like non-nil error that return an empty string
when the Error method is called and when this happens it shuts down the
connection to the server.
Since we can return a ThreadBlockedError to the client it can't have an
empty string as return value.
Fixes#1251
Core files created by gdb can have sections missing that would be
present in OS created core files.
We work around this by first reading PT_LOAD entries from the exe and
then reading them from the core.
Fixes#1121
Setting the Level field of a logrus logger doesn't actually do anything
since the Level field simply reports the log level of the last log
message emitted on the logger.
The right way to do that is to set logger.Logger.Level.
Also cleans up newline characters from log messages emitted through
logrus and fixes the direction of the arrows in the messages emitted by
rpccommon, which was inconsistent with the arrows of gdbserial.
This pull request makes several changes to delve to allow headless
instancess that are started with the --accept-multiclient flag to
keep running even if there is no connected client. Specifically:
1. Makes a headless instance started with --accept-multiclient quit
after one of the clients sends a Detach request (previously they
would never ever quit, which was a bug).
2. Changes proc/gdbserial and proc/native so that they mark the
Process as exited after they detach, even if they did not kill the
process during detach. This prevents bugs such as #1231 where we
attempt to manipulate a target process after we detached from it.
3. On non --accept-multiclient instances do not kill the target
process unless we started it or the client specifically requests
it (previously if the client did not Detach before closing the
connection we would kill the target process unconditionally)
4. Add a -c option to the quit command that detaches from the
headless server after restarting the target.
5. Change terminal so that, when attached to --accept-multiclient,
pressing ^C will prompt the user to either disconnect from the
server or pause the target process. Also extend the exit prompt to
ask if the user wants to keep the headless server running.
Implements #245, #952, #1159, #1231
Go1.11 switched to the zlib-gnu compression format for debug sections.
Change proc and and a test in dwarf/line to support this change.
Also deletes some dead code from pkg/proc/bininfo.go that hadn't been
used in a long time.
Implements structured logging via Logrus. This gives us a logger per
boundry that we care about, allowing for easier parsing of logs if users
have more than one log option enabled. Also, cleans up a lot of
conditionals in the code by simply silencing the logger at creation as
opposed to conditionally logging everywhere.
To save disk space, some distributions strip the debugging information
from the binaries, putting it in separate files, usually distributed in
separate packages.
To locate the file containing the debug information for a certain
binary, an ELF note named ".note.gnu.build-id" is added to the latter,
which contains a header and a build identification. This identification
can be used to compose a path with this form:
/usr/lib/debug/.build-id/BUILDID[:2]/BUILDID[2:].debug
With this patch, if Delve can't find the debug information in the main
binary, it'll try to locate and parse ".note.gnu.build-id", to compose
and attempt to open a path with the format described above.
Fixes#1206
A user complained on the mailing list about having continuous
"optimized function warnings" on non-optimized functions when using 1.9.
This commit fixes the problem by disabling optimized function detection
on 1.9 and earlier (where it's impossible) and adds a test so we don't
break it again in the future.
If the application being debugged imports two packages with the same
name (but different paths) there was no way to disambiguate the two,
since the character '/' can not appear inside a go identifier.
By allowing users to use a string literal as the package name a package
path can be specified.
Displays the return values of the current function when we step out of
it after executing a step, next or stepout command.
Implementation of this feature is tricky: when the function has
returned the return variables are not in scope anymore. Implementing
this feature requires evaluating variables that are out of scope, using
a stack frame that doesn't exist anymore.
We can't calculate the address of these variables when the
next/step/stepout command is initiated either, because between that
point and the time where the stepout breakpoint is actually hit the
goroutine stack could grow and be moved to a different memory address.
Add a new method "Common" to proc.Process that returns a pointer to a
struct that pkg/proc can use to store its things, independently of the
backend.
This is used here to replace the AllGCache typecasts, it will also be
used to store the return values of the stepout breakpoint and the state
for injected function calls.
go1.11 adds a new extended attribute to all type DIEs containing the
address of the corresponding runtime._type struct, use this attribute
to find the DIE of the concrete type of interface variables when
available.
Go1.11 uses the is_stmt flag of .debug_line to communicate which
assembly instructions are good places for breakpoints, we should
respect this flag.
These changes were introduced by:
* https://go-review.googlesource.com/c/go/+/102435/
Additionally when setting next breakpoints ignore all PC addresses that
belong to the same line as the one currently under at the cursor. This
matches the behavior of gdb and avoids stopping multiple times at the
heading line of a for statement with go1.11.
Change: https://go-review.googlesource.com/c/go/+/110416 adds the
prologue_end flag to the .debug_line section to communicate the end of
the stack-split prologue. We should use it instead of pattern matching
the disassembly when available.
Fixes#550
type of interfaces
'c7cde8b'.
Maps were always loaded with using the default configuration during a
reslice. This is probably a remnant from when we didn't let clients
configure the load parameters.
If dwz binary is available in the system, test delve's ability to find
deduplicated symbols in the DWARF information.
dwzcompression.go contains a small C function (void fortytwo()) which
calls glibc's fprintf with stdin as first argument. Normally, stdin
will be present as a DW_TAG_variable as part of a DW_TAG_compile_unit
named dwzcompression.cgo2.c.
After running dwz on the binary, stdin is moved to a
DW_TAG_partial_unit, which is imported from dwzcompression.cgo2.c with
a DW_TAG_imported_unit.
This test verifies that delve is able to find stdin symbol's type, as a
way to confirm it understands dwz's compressed/deduplicated DWARF
information.
The EnableDWZCompression flag allows tests to request BuildFixture to
run "dwz" on the Fixture's resulting binary to compress/deduplicate its
DWARF sections.
'dwz' is a tool that reduces the size of DWARF sections by
deduplicating symbols. The deduplicated symbols are moved from their
original 'compile unit' to a 'partial unit', which is then referenced
from its original location with an 'imported unit' tag.
In the case of Go binaries, all symbols are located in a single
'compile unit', and the name of each symbol contains a reference to its
package, so 'dwz' is not able to deduplicate them. But still, some C
symbols included in the binary are deduplicated, which also alters the
structure of the DWARF sections, making delve unable to parse them
(crashing in the attempt).
While it would've been possible to simply ignore the C symbols, or
blindly loading all then into BinaryInfo members (packageVars,
Functions...), for correctness sake this change tries to do the right
thing, staging symbols into temporary partialUnit objects, moving them
to BinaryInfo when they are actually requested by a 'imported unit'
tag.
Change the linux verison of proc/native and proc/gdbserial (with
debugserver) so that they let the target process use the terminal when
delve is launched in headless mode.
Windows already worked, proc/gdbserial (with rr) already worked.
I couldn't find a way to make proc/gdbserial (with lldb-server) work.
No tests are added because I can't think of a way to test for
foregroundness of a process.
Fixes#65
