deref'd
Fix infinite recursion if escapeCheck, at some point during its
recursion, creates an unreadable variable.
The deeper reason for this is that we evaluate function calls in a very
weird order so that we can always have stack space to store
intermediate evaluation results.
The variable 'value' happens to be stored in a register when we try to
make the call and because of our weird evaluation strategy registers
are no longer available to us when we evaluate 'value'.
This is not a complete fix for the issue, the real fix would be to
evaluate everything in its natural order, storing intermediate values
in Delve's memory instead of the target's stack. To do this we need a
mechanism to pin heap allocated objects, which at the moment does not
exist.
Updates #3310
The compiler produces ABI compatibility wrappers for some functions.
We have changed the support for breakpoints to allow a single logical
breakpoint to correspond to multiple physical breakpoints, take
advantage of that to set breakpoints on both the ABI wrapper and the
real function.
Fixes#3296
Delve no longer compiles on Go1.12 and earlier, we don't test it on
these versions and they are 4 years old and unsupported. Remove some
code related to Go 1.12 and earlier, mostly from tests.
Previously it was only possible to extract a value of type `any` using
an attribute name. This poses challenges when fields are allowed to have
different classes, and it is ambiguous how to handle them.
Introduces a new TargetGroup abstraction that can be used to manage
multiple related targets.
No actual management of child processes is implemented here, this is
just a refactoring to make it possible to do that in the future.
Updates #2551
debugCallV2 for amd64 has a bug where it corrupts the flags registers
every time it is called, this commit works around that problem by
restoring flags one extra time to its original value after stepping out
of debugCallV2.
Fixes#2985
* _scripts/test_linux.sh,_scripts/test_windows.ps1: always return exit code 0 when testing on tip
Same as what we do for test_mac.sh
* proc: support function call injection on arm64
Support function call injection on arm64 with go1.19
Normally calls can't be performed on recorded processes, becuase the
future instructions executed by the target are predetermined. The rr
debugger however has a mechanism that allows this by taking the current
state of the recording and allowing it to diverge from the recording,
temporarily.
This commit adds support for starting and ending such diversions around
function calls.
Note: this requires rr version 5.5 of later to work, see:
https://github.com/rr-debugger/rr/pull/2748
Change debug_info type reader and proc to convert parametric types into
their real types by reading the corresponding dictionary entry and
using the same method used for interfaces to retrieve the DIE from a
runtime._type address.
'2586e9b1'.
When the function we are calling is an autogenerated stub (because, for
example, we are calling it through a function pointer) the declaration
line of variables is meaningless and could cause us to discard valid
return arguments.
* proc: support new Go 1.17 panic/defer mechanism
Go 1.17 will create wrappers for deferred calls that take arguments.
Change defer reading code so that wrappers are automatically unwrapped.
Also the deferred function is called directly by runtime.gopanic, without going through runtime.callN which means that sometimes when a panic happens the stack is either:
0. deferred function call
1. deferred call wrapper
2. runtime.gopanic
or:
0. deferred function call
1. runtime.gopanic
instead of always being:
0. deferred function call
1. runtime.callN
2. runtime.gopanic
the isPanicCall check is changed accordingly.
* test: miscellaneous minor test fixes for Go 1.17
* proc: resolve inlined calls when stepping out of runtime.breakpoint
Calls to runtime.Breakpoint are inlined in Go 1.17 when inlining is
enabled, resolve inlined calls in stepInstructionOut.
* proc: add support for debugCallV2 with regabi
This change adds support for the new debug call protocol which had to
change for the new register ABI introduced in Go 1.17.
Summary of changes:
- Abstracts over the debug call version depending on the Go version
found in the binary.
- Uses R12 instead of RAX as the debug protocol register when the binary
is from Go 1.17 or later.
- Creates a variable directly from the DWARF entry for function
arguments to support passing arguments however the ABI expects.
- Computes a very conservative stack frame size for the call when
injecting a call into a Go process whose version is >=1.17.
Co-authored-by: Michael Anthony Knyszek <mknyszek@google.com>
Co-authored-by: Alessandro Arzilli <alessandro.arzilli@gmail.com>
* TeamCity: enable tests on go-tip
* goversion: version compatibility bump
* TeamCity: fix go-tip builds on macOS/arm64
Co-authored-by: Michael Anthony Knyszek <mknyszek@google.com>
We told clients that further loading of variables can be done by
specifying a type cast using the address of a variable that we
returned.
This does not work for registerized variables (or, in general,
variables that have a complex location expression) because we don't
give them unique addresses and we throw away the compositeMemory object
we made to read them.
This commit changes proc so that:
1. variables with location expression divided in pieces do get a unique
memory address
2. the compositeMemory object is saved somewhere
3. when an integer is cast back into a pointer type we look through our
saved compositeMemory objects to see if there is one that covers the
specified address and use it.
The unique memory addresses we generate have the MSB set to 1, as
specified by the Intel 86x64 manual addresses in this form are reserved
for kernel memory (which we can not read anyway) so we are guaranteed
to never generate a fake memory address that overlaps a real memory
address of the application.
The unfortunate side effect of this is that it will break clients that
do not deserialize the address to a 64bit integer. This practice is
contrary to how we defined our types and contrary to the specification
of the JSON format, as of json.org, however it is also fairly common,
due to javascript itself having only 53bit integers.
We could come up with a new mechanism but then even more old clients
would have to be changed.
Delve represents registerized variables (fully or partially) using
compositeMemory, implementing proc.(*compositeMemory).WriteMemory is
necessary to make SetVariable and function calls work when Go will
switch to using the register calling convention in 1.17.
This commit also makes some refactoring by moving the code that
converts between register numbers and register names out of pkg/proc
into a different package.
evalFunctionCall needs to remove the breakpoint from the current thread
after starting the function call injection, otherwise Continue will
think that the thread is stopped at a breakpoint and return to the user
instead of continuing the call injection.
Adds a flag that distinguishes the return values of an injected
function call from the return values of a function call executed by the
target program.
On linux we can not read memory if the thread we use to do it is
occupied doing certain system calls. The exact conditions when this
happens have never been clear.
This problem was worked around by using the Blocked method which
recognized the most common circumstances where this would happen.
However this is a hack: Blocked returning true doesn't mean that the
problem will manifest and Blocked returning false doesn't necessarily
mean the problem will not manifest. A side effect of this is issue
#2151 where sometimes we can't read the memory of a thread and find its
associated goroutine.
This commit fixes this problem by always reading memory using a thread
we know to be good for this, specifically the one returned by
ContinueOnce. In particular the changes are as follows:
1. Remove (ProcessInternal).CurrentThread and
(ProcessInternal).SetCurrentThread, the "current thread" becomes a
field of Target, CurrentThread becomes a (*Target) method and
(*Target).SwitchThread basically just sets a field Target.
2. The backends keep track of their own internal idea of what the
current thread is, to use it to read memory, this is the thread they
return from ContinueOnce as trapthread
3. The current thread in the backend and the current thread in Target
only ever get synchronized in two places: when the backend creates a
Target object the currentThread field of Target is initialized with the
backend's current thread and when (*Target).Restart gets called (when a
recording is rewound the currentThread used by Target might not exist
anymore).
4. We remove the MemoryReadWriter interface embedded in Thread and
instead add a Memory method to Process that returns a MemoryReadWriter.
The backends will return something here that will read memory using
the current thread saved by the backend.
5. The Thread.Blocked method is removed
One possible problem with this change is processes that have threads
with different memory maps. As far as I can determine this could happen
on old versions of linux but this option was removed in linux 2.5.
Fixes#2151
* proc: use argument position for addr only when injecting function calls
We can not, in general, use the argument position to determine the
address of a formal parameter, it will not work in presence of
optimizations or inlining. In those cases formal arguments could be
stored in registers.
Fixes#2176
* Travis-CI: add ignorechecksum option to chocolatey command
Looks like a configuration problem on chocolatey's end.
Co-authored-by: a <a@kra>
Since proc is supposed to work independently from the target
architecture it shouldn't use architecture-dependent types, like
uintptr. For example when reading a 64bit core file on a 32bit
architecture, uintptr will be 32bit but the addresses proc needs to
represent will be 64bit.
* proc: start variable visibility one line after their decl line
In most cases variables shouldn't be visible on their declaration line
because they won't be initialized there.
Function arguments are treated as an exception.
This fix is only applied to programs compiled with Go 1.15 or later as
previous versions of Go did not report the correct declaration line for
variables captured by closures.
Fixes#1134
* proc: silence go vet error
* Makefile: enable PIE tests on windows/Go 1.15
* core: support core files for PIEs on windows
* goversion: add Go 1.15 to supported versions
* proc: fix function call injection for Go 1.15
Go 1.15 changed the call injection protocol so that the runtime will
execute the injected call on a different (new) goroutine.
This commit changes the function call support in delve to:
1. correctly track down the call injection state after the runtime
switches to a different goroutine.
2. correctly perform the escapeCheck when stack values can come from
multiple goroutine stacks.
* proc: miscellaneous fixed for call injection under macOS with go 1.15
- create copy of SP in debugCallAXCompleteCall case because the code
used to assume that regs doesn't change
- fix automatic address calculation for function arguments when an
argument has a spurious DW_OP_piece at entry
Changes implementations of proc.Registers interface and the
op.DwarfRegisters struct so that floating point registers can be loaded
only when they are needed.
Removes the floatingPoint parameter from proc.Thread.Registers.
This accomplishes three things:
1. it simplifies the proc.Thread.Registers interface
2. it makes it impossible to accidentally create a broken set of saved
registers or of op.DwarfRegisters by accidentally calling
Registers(false)
3. it improves general performance of Delve by avoiding to load
floating point registers as much as possible
Floating point registers are loaded under two circumstances:
1. When the Slice method is called with floatingPoint == true
2. When the Copy method is called
Benchmark before:
BenchmarkConditionalBreakpoints-4 1 4327350142 ns/op
Benchmark after:
BenchmarkConditionalBreakpoints-4 1 3852642917 ns/op
Updates #1549
Instead of rescanning debug_info every time we want to read a function
(either to find inlined calls or its variables) cache the tree of
dwarf.Entry that we would generate and use that.
Benchmark before:
BenchmarkConditionalBreakpoints-4 1 5164689165 ns/op
Benchmark after:
BenchmarkConditionalBreakpoints-4 1 4817425836 ns/op
Updates #1549
Implement debugging function for 386 on linux with reference to AMD64.
There are a few remaining problems that need to be solved in another time.
1. The stacktrace of cgo are not exactly as expected.
2. Not implement `core` for now.
3. Not implement `call` for now. Can't not find `runtime·debugCallV1` or
similar function in $GOROOT/src/runtime/asm_386.s.
Update #20
Removes the restriction that the DWARF type for the receiver of a method
must be a TypeDef. This seems reasonable in practice, but it turns out
Go DWARF does not consider
```
type X int
```
to be a typedef. This patch also allows for calling a method where the
receiver is not used or passed in, such as:
```
func (_ X) Method() { println("why") }
```
* pkg/proc: Introduce Target
* pkg/proc: Remove Common.fncallEnabled
Realistically we only block it on recorded backends.
* pkg/proc: Move fncallForG to Target
* pkg/proc: Remove CommonProcess
Remove final bit of functionality stored in CommonProcess and move it to
*Target.
* pkg/proc: Add SupportsFunctionCall to Target
Trust argument order to determine argument frame layout when calling
functions, this allows calling optimized functions and removes the
special cases for runtime.mallocgc.
Fixes#1589
Moves EvalScope methods to the proper file and organizes everything
together. Also makes some EvalScope methods no longer methods and just
pure functions.
Add variables flag to mark variables that are allocated on a register
(and have no address) and variables that we read as result of a
function call (and are allocated on a stack that no longer exists when
we show them to the user).
* proc: allow simultaneous call injection to multiple goroutines
Changes the call injection code so that we can have multiple call
injections going on at the same time as long as they happen on distinct
goroutines.
* proc: fix EvalExpressionWithCalls for constant expressions
The lack of address of constant expressions would confuse EvalExpressionWithCalls
Fixes#1577
Allow changing the value of a string variable to a new literal string,
which requires calling runtime.mallocgc to allocate the string into the
target process.
This means that a command like:
call f("some string")
is now supported.
Additionally the command:
call s = "some string"
is also supported.
Fixes#826
* proc: support nested function calls
Changes the code in fncall.go to support nested function calls.
This changes delays argument evaluation until after we have used
the call injection protocol to allocate an argument frame. When
evaluating the parse tree of an expression we'll initiate each
function call we find on the way down and then complete the function
call on the way up.
For example. in:
f(g(x))
we will:
1. initiate the call injection protocol for f(...)
2. progress it until the point where we have space for the arguments
of 'f' (i.e. when we receive the debugCallAXCompleteCall message
from the target runtime)
3. inititate the call injection protocol for g(...)
4. progress it until the point where we have space for the arguments
of 'g'
5. copy the value of x into the argument frame of 'g'
6. finish the call to g(...)
7. copy the return value of g(x) into the argument frame of 'f'
8. finish the call to f(...)
Updates #119
* proc: bugfix: closure addr was wrong for non-closure functions
The initial implementation of the 'call' command required the
function call to be the root expression, i.e. something like:
double(3) + 1
was not allowed, because the root expression was the binary operator
'+', not the function call.
With this change expressions like the one above and others are
allowed.
This is the first step necessary to implement nested function calls
(where the result of a function call is used as argument to another
function call).
This is implemented by replacing proc.CallFunction with
proc.EvalExpressionWithCalls. EvalExpressionWithCalls will run
proc.(*EvalScope).EvalExpression in a different goroutine. This
goroutine, the 'eval' goroutine, will communicate with the main
goroutine of the debugger by means of two channels: continueRequest
and continueCompleted.
The eval goroutine evaluates the expression recursively, when
a function call is encountered it takes care of setting up the
function call on the target program and writes a request to the
continueRequest channel, this causes the 'main' goroutine to restart
the target program by calling proc.Continue.
Whenever Continue encounters a breakpoint that belongs to the
function call injection protocol (runtime.debugCallV1 and associated
functions) it writes to continueCompleted which resumes the 'eval'
goroutine.
The 'eval' goroutine takes care of implementing the function call
injection protocol.
When the expression is fully evaluated the 'eval' goroutine will
write a special message to 'continueRequest' signaling that the
expression evaluation is terminated which will cause Continue to
return to the user.
Updates #119
This change splits the BinaryInfo object into a slice of Image objects
containing information about the base executable and each loaded shared
library (note: go plugins are shared libraries).
Delve backens are supposed to call BinaryInfo.AddImage whenever they
detect that a new shared library has been loaded.
Member fields of BinaryInfo that are used to speed up access to dwarf
(Functions, packageVars, consts, etc...) remain part of BinaryInfo and
are updated to reference the correct image object. This simplifies this
change.
This approach has a few shortcomings:
1. Multiple shared libraries can define functions or globals with the
same name and we have no way to disambiguate between them.
2. We don't have a way to handle library unloading.
Both of those affect C shared libraries much more than they affect go
plugins. Go plugins can't be unloaded at all and a lot of name
collisions are prevented by import paths.
There's only one problem that is concerning: if two plugins both import
the same package they will end up with multiple definition for the same
function.
For example if two plugins use fmt.Printf the final in-memory image
(and therefore our BinaryInfo object) will end up with two copies of
fmt.Printf at different memory addresses. If a user types
break fmt.Printf
a breakpoint should be created at *both* locations.
Allowing this is a relatively complex change that should be done in a
different PR than this.
For this reason I consider this approach an acceptable and sustainable
stopgap.
Updates #865
* *: use loglevel to control what gets logged instead of output redirection
This stops logrus from doing all the formatting just to discard it
immediately afterwards.
* logflags: replace default formatter of logrus
The default formatter of logrus emits logs in two different formats
depending on whether or not the output is going to a terminal. The
output format for non-terminals is indented to be machine readable, but
we mostly read logs ourselves and the excessive quoting makes that
format unreadable.
When outputting to terminals it uses ANSI escape codes unconditionally,
without checking whether the terminal it is connected to actually
supports colors.
This commit replaces the default formatter with a much simpler
formatter that always uses a more readable format, doesn't use colors
and places the key-value pairs at the beginning of the line (which is a
better match for how we use them).
* cmd/dlv: add command line options to redirect logs
Adds two options, --log-to-file and --log-to-fd, to redirect logs to a
file or to a file descriptor.
When one of those two options is specified the "API server listening
at:" message will also be redirected to the specified file/file
descriptor.
This allows clients that want to use the "API server listening at:"
message to do so even if they want to redirect the target's stdout to
another file or device.
Implements #1179, #1523
The repository is being switched from the personal account
github.com/derekparker/delve to the organization account
github.com/go-delve/delve. This patch updates imports and docs, while
preserving things which should not be changed such as my name in the
CHANGELOG and in TODO comments.
Users can create sparse maps in two ways, either by:
a) adding lots of entries to a map and then deleting most of them, or
b) using the make(mapType, N) expression with a very large N
When this happens reading the resulting map will be very slow
because loadMap needs to scan many buckets for each entry it finds.
Technically this is not a bug, the user just created a map that's
very sparse and therefore very slow to read. However it's very
annoying to have the debugger hang for several seconds when trying
to read the local variables just because one of them (which you
might not even be interested into) happens to be a very sparse map.
There is an easy mitigation to this problem: not reading any
additional buckets once we know that we have already read all
entries of the map, or as many entries as we need to fulfill the
MaxArrayValues parameter.
Unfortunately this is mostly useless, a VLSM (Very Large Sparse Map)
with a single entry will still be slow to access, because the single
entry in the map could easily end up in the last bucket.
The obvious solution to this problem is to set a limit to the
number of buckets we read when loading a map. However there is no
good way to set this limit.
If we hardcode it there will be no way to print maps that are beyond
whatever limit we pick.
We could let users (or clients) specify it but the meaning of such
knob would be arcane and they would have no way of picking a good
value (because there is no objectively good value for it).
The solution used in this commit is to set an arbirtray limit on
the number of buckets we read but only when loadMap is invoked
through API calls ListLocalVars and ListFunctionArgs. In this way
`ListLocalVars` and `ListFunctionArgs` (which are often invoked
automatically by GUI clients) remain fast even in presence of a
VLSM, but the contents of the VLSM can still be inspected using
`EvalVariable`.
Support for position independent executables (PIE) on the native linux
backend, the gdbserver backend on linux and the core backend.
Also implemented in the windows native backend, but it can't be tested
because go doesn't support PIE on windows yet.
Fncall.go was written with the assumption that the object returned by
proc.Thread.Registers does not change after we call
proc.Thread.SetPC/etc.
This is true for the native backend but not for gdbserial. I had
anticipated this problem and introduced the Save/SavedRegisters
mechanism during the first implementation of fncall.go but that's
insufficient.
Instead:
1. clarify that the object returned by proc.Thread.Registers could
change when the CPU registers are modified.
2. add a Copy method to Registers that returns a copy of the registers
that are guaranteed not to change when the CPU registers change.
3. remove the Save/SavedRegisters mechanism.
This solution leaves us the option, in the future, to cache the output
of proc.(Thread).Registers, avoiding a system call every time it's
called.
