Supports viewing local variables and evaluating expressions correctly
when range-over-func is used.
The same limitations that the previous commit on this line had still
apply (no inlining, wrong way to identify the range parent in some
cases).
Updates #3733
* proc: generalize escapeCheck and allocString
Generalizes the function for checking for escaping pointers so that it
can be used to iterate on all pointers of a variable.
Also generalizes the string allocation opcodes so that in the future we
can use it to call other special runtime functions (for example: map
access, channel send/receive, etc).
* review changes
Instead of having a different version for each architecture have a
single version that uses an architecture specific list of registers.
Also generalize it so that, if we want, we can extend the workaround to
other runtime functions we might want to call (for example the channel
send/receive functions).
We have used a goroutine to keep track of some of the expression
evaluation status across target resumes during call injections.
Now that the expression interpreter has been rewritten to use a stack
machine we can move what little state is left into the stack machine
and get rid of the goroutine-and-channel mechanism.
* proc: use stack machine to evaluate expressions
This commit splits expression evaluation into two parts. The first part (in
pkg/proc/evalop/evalcompile.go) "compiles" as ast.Expr into a list of
instructions (defined in pkg/proc/evalop/ops.go) for a stack machine
(defined by `proc.(*evalStack)`).
The second part is a stack machine (implemented by `proc.(*EvalScope).eval`
and `proc.(*EvalScope).evalOne`) that has two modes of operation: in the
main mode it executes inteructions from the list (by calling `evalOne`), in
the second mode it executes the call injection protocol by calling
`funcCallStep` repeatedly until it either the protocol finishes, needs more
input from the stack machine (to set call arguments) or fails.
This approach has several benefits:
- it is now possible to remove the goroutine we use to evaluate expression
and the channel used to communicate with the Continue loop.
- every time we resume the target to execute the call injection protocol we
need to update several local variables to match the changed state of the
target, this is now done at the top level of the evaluation loop instead of
being hidden inside a recurisive evaluator
- using runtime.Pin to pin addresses returned by an injected call would
allow us to use a more natural evaluation order for function calls, which
would solve some bugs #3310, allow users to inspect values returned by a
call injection #1599 and allow implementing some other features #1465. Doing
this with the recursive evaluator, while keeping backwards compatibility
with versions of Go that do not have runtime.Pin is very hard. However after
this change we can simply conditionally change how compileFunctionCall works
and add some opcodes.
* review round 1
* review round 2
* enable func call injection on delve for ppc64le
* Function call injection on Delve/ppc64le, modified DWARF encoding and decoding for floating point registers to make floatsum test work
* Function call injection on Delve/ppc64le cleanup
* skip PIE tests for function call injection on other packages
* Address review comments
* accounted for additional skipped PIE tests for function call injection
* Code cleanup and undoing revert of previous commit
* Enable function call injection only on 1.22 and above and some cleanup
* additional cleanup, go fmt run
* Debug function call tests fail on ppc64le/PIE mode adjusted the backup_test_health.md file accordingly
* proc: correctly update local variables after continue
At various point during the execution of the call injection protocol
the process is resumed and the call injection goroutine could migrate
to a different thread, we must make sure to update our local variables
correctly after every point where the target program is resumed.
'fncall122debug_clean' on 'f469a0a5'.
* go.mod: update golang.org/x/tools
Go 1.22 broke golang.org/x/tools/packages
* cmd/dlv: disable TestStaticcheck with go1.22
Go 1.22 is not yet supported by staticcheck.
We used to parse the .gopclntab section but removed support in favor of
simply using DWARF debug information, due to lack of C symbols among
other reasons. This makes it impossible to debug stripped binaries,
which some distrubutions ship by default.
Add back in basic support for .gopclntab which survives if the binary
is stripped, allowing for rudimentary debugging such as basic
program navigation, tracing, etc...
Go 1.21 renamed runtime._type to internal/abi.Type and changed the name
of its fields. Update Delve so that it uses the new names for loading
interfaces and generic type parameters.
deref'd
Fix infinite recursion if escapeCheck, at some point during its
recursion, creates an unreadable variable.
The deeper reason for this is that we evaluate function calls in a very
weird order so that we can always have stack space to store
intermediate evaluation results.
The variable 'value' happens to be stored in a register when we try to
make the call and because of our weird evaluation strategy registers
are no longer available to us when we evaluate 'value'.
This is not a complete fix for the issue, the real fix would be to
evaluate everything in its natural order, storing intermediate values
in Delve's memory instead of the target's stack. To do this we need a
mechanism to pin heap allocated objects, which at the moment does not
exist.
Updates #3310
The compiler produces ABI compatibility wrappers for some functions.
We have changed the support for breakpoints to allow a single logical
breakpoint to correspond to multiple physical breakpoints, take
advantage of that to set breakpoints on both the ABI wrapper and the
real function.
Fixes#3296
Delve no longer compiles on Go1.12 and earlier, we don't test it on
these versions and they are 4 years old and unsupported. Remove some
code related to Go 1.12 and earlier, mostly from tests.
Previously it was only possible to extract a value of type `any` using
an attribute name. This poses challenges when fields are allowed to have
different classes, and it is ambiguous how to handle them.
Introduces a new TargetGroup abstraction that can be used to manage
multiple related targets.
No actual management of child processes is implemented here, this is
just a refactoring to make it possible to do that in the future.
Updates #2551
debugCallV2 for amd64 has a bug where it corrupts the flags registers
every time it is called, this commit works around that problem by
restoring flags one extra time to its original value after stepping out
of debugCallV2.
Fixes#2985
* _scripts/test_linux.sh,_scripts/test_windows.ps1: always return exit code 0 when testing on tip
Same as what we do for test_mac.sh
* proc: support function call injection on arm64
Support function call injection on arm64 with go1.19
Normally calls can't be performed on recorded processes, becuase the
future instructions executed by the target are predetermined. The rr
debugger however has a mechanism that allows this by taking the current
state of the recording and allowing it to diverge from the recording,
temporarily.
This commit adds support for starting and ending such diversions around
function calls.
Note: this requires rr version 5.5 of later to work, see:
https://github.com/rr-debugger/rr/pull/2748
Change debug_info type reader and proc to convert parametric types into
their real types by reading the corresponding dictionary entry and
using the same method used for interfaces to retrieve the DIE from a
runtime._type address.
'2586e9b1'.
When the function we are calling is an autogenerated stub (because, for
example, we are calling it through a function pointer) the declaration
line of variables is meaningless and could cause us to discard valid
return arguments.
* proc: support new Go 1.17 panic/defer mechanism
Go 1.17 will create wrappers for deferred calls that take arguments.
Change defer reading code so that wrappers are automatically unwrapped.
Also the deferred function is called directly by runtime.gopanic, without going through runtime.callN which means that sometimes when a panic happens the stack is either:
0. deferred function call
1. deferred call wrapper
2. runtime.gopanic
or:
0. deferred function call
1. runtime.gopanic
instead of always being:
0. deferred function call
1. runtime.callN
2. runtime.gopanic
the isPanicCall check is changed accordingly.
* test: miscellaneous minor test fixes for Go 1.17
* proc: resolve inlined calls when stepping out of runtime.breakpoint
Calls to runtime.Breakpoint are inlined in Go 1.17 when inlining is
enabled, resolve inlined calls in stepInstructionOut.
* proc: add support for debugCallV2 with regabi
This change adds support for the new debug call protocol which had to
change for the new register ABI introduced in Go 1.17.
Summary of changes:
- Abstracts over the debug call version depending on the Go version
found in the binary.
- Uses R12 instead of RAX as the debug protocol register when the binary
is from Go 1.17 or later.
- Creates a variable directly from the DWARF entry for function
arguments to support passing arguments however the ABI expects.
- Computes a very conservative stack frame size for the call when
injecting a call into a Go process whose version is >=1.17.
Co-authored-by: Michael Anthony Knyszek <mknyszek@google.com>
Co-authored-by: Alessandro Arzilli <alessandro.arzilli@gmail.com>
* TeamCity: enable tests on go-tip
* goversion: version compatibility bump
* TeamCity: fix go-tip builds on macOS/arm64
Co-authored-by: Michael Anthony Knyszek <mknyszek@google.com>
We told clients that further loading of variables can be done by
specifying a type cast using the address of a variable that we
returned.
This does not work for registerized variables (or, in general,
variables that have a complex location expression) because we don't
give them unique addresses and we throw away the compositeMemory object
we made to read them.
This commit changes proc so that:
1. variables with location expression divided in pieces do get a unique
memory address
2. the compositeMemory object is saved somewhere
3. when an integer is cast back into a pointer type we look through our
saved compositeMemory objects to see if there is one that covers the
specified address and use it.
The unique memory addresses we generate have the MSB set to 1, as
specified by the Intel 86x64 manual addresses in this form are reserved
for kernel memory (which we can not read anyway) so we are guaranteed
to never generate a fake memory address that overlaps a real memory
address of the application.
The unfortunate side effect of this is that it will break clients that
do not deserialize the address to a 64bit integer. This practice is
contrary to how we defined our types and contrary to the specification
of the JSON format, as of json.org, however it is also fairly common,
due to javascript itself having only 53bit integers.
We could come up with a new mechanism but then even more old clients
would have to be changed.
Delve represents registerized variables (fully or partially) using
compositeMemory, implementing proc.(*compositeMemory).WriteMemory is
necessary to make SetVariable and function calls work when Go will
switch to using the register calling convention in 1.17.
This commit also makes some refactoring by moving the code that
converts between register numbers and register names out of pkg/proc
into a different package.
evalFunctionCall needs to remove the breakpoint from the current thread
after starting the function call injection, otherwise Continue will
think that the thread is stopped at a breakpoint and return to the user
instead of continuing the call injection.
Adds a flag that distinguishes the return values of an injected
function call from the return values of a function call executed by the
target program.
On linux we can not read memory if the thread we use to do it is
occupied doing certain system calls. The exact conditions when this
happens have never been clear.
This problem was worked around by using the Blocked method which
recognized the most common circumstances where this would happen.
However this is a hack: Blocked returning true doesn't mean that the
problem will manifest and Blocked returning false doesn't necessarily
mean the problem will not manifest. A side effect of this is issue
#2151 where sometimes we can't read the memory of a thread and find its
associated goroutine.
This commit fixes this problem by always reading memory using a thread
we know to be good for this, specifically the one returned by
ContinueOnce. In particular the changes are as follows:
1. Remove (ProcessInternal).CurrentThread and
(ProcessInternal).SetCurrentThread, the "current thread" becomes a
field of Target, CurrentThread becomes a (*Target) method and
(*Target).SwitchThread basically just sets a field Target.
2. The backends keep track of their own internal idea of what the
current thread is, to use it to read memory, this is the thread they
return from ContinueOnce as trapthread
3. The current thread in the backend and the current thread in Target
only ever get synchronized in two places: when the backend creates a
Target object the currentThread field of Target is initialized with the
backend's current thread and when (*Target).Restart gets called (when a
recording is rewound the currentThread used by Target might not exist
anymore).
4. We remove the MemoryReadWriter interface embedded in Thread and
instead add a Memory method to Process that returns a MemoryReadWriter.
The backends will return something here that will read memory using
the current thread saved by the backend.
5. The Thread.Blocked method is removed
One possible problem with this change is processes that have threads
with different memory maps. As far as I can determine this could happen
on old versions of linux but this option was removed in linux 2.5.
Fixes#2151
* proc: use argument position for addr only when injecting function calls
We can not, in general, use the argument position to determine the
address of a formal parameter, it will not work in presence of
optimizations or inlining. In those cases formal arguments could be
stored in registers.
Fixes#2176
* Travis-CI: add ignorechecksum option to chocolatey command
Looks like a configuration problem on chocolatey's end.
Co-authored-by: a <a@kra>
Since proc is supposed to work independently from the target
architecture it shouldn't use architecture-dependent types, like
uintptr. For example when reading a 64bit core file on a 32bit
architecture, uintptr will be 32bit but the addresses proc needs to
represent will be 64bit.
* proc: start variable visibility one line after their decl line
In most cases variables shouldn't be visible on their declaration line
because they won't be initialized there.
Function arguments are treated as an exception.
This fix is only applied to programs compiled with Go 1.15 or later as
previous versions of Go did not report the correct declaration line for
variables captured by closures.
Fixes#1134
* proc: silence go vet error
* Makefile: enable PIE tests on windows/Go 1.15
* core: support core files for PIEs on windows
* goversion: add Go 1.15 to supported versions
* proc: fix function call injection for Go 1.15
Go 1.15 changed the call injection protocol so that the runtime will
execute the injected call on a different (new) goroutine.
This commit changes the function call support in delve to:
1. correctly track down the call injection state after the runtime
switches to a different goroutine.
2. correctly perform the escapeCheck when stack values can come from
multiple goroutine stacks.
* proc: miscellaneous fixed for call injection under macOS with go 1.15
- create copy of SP in debugCallAXCompleteCall case because the code
used to assume that regs doesn't change
- fix automatic address calculation for function arguments when an
argument has a spurious DW_OP_piece at entry
Changes implementations of proc.Registers interface and the
op.DwarfRegisters struct so that floating point registers can be loaded
only when they are needed.
Removes the floatingPoint parameter from proc.Thread.Registers.
This accomplishes three things:
1. it simplifies the proc.Thread.Registers interface
2. it makes it impossible to accidentally create a broken set of saved
registers or of op.DwarfRegisters by accidentally calling
Registers(false)
3. it improves general performance of Delve by avoiding to load
floating point registers as much as possible
Floating point registers are loaded under two circumstances:
1. When the Slice method is called with floatingPoint == true
2. When the Copy method is called
Benchmark before:
BenchmarkConditionalBreakpoints-4 1 4327350142 ns/op
Benchmark after:
BenchmarkConditionalBreakpoints-4 1 3852642917 ns/op
Updates #1549
Instead of rescanning debug_info every time we want to read a function
(either to find inlined calls or its variables) cache the tree of
dwarf.Entry that we would generate and use that.
Benchmark before:
BenchmarkConditionalBreakpoints-4 1 5164689165 ns/op
Benchmark after:
BenchmarkConditionalBreakpoints-4 1 4817425836 ns/op
Updates #1549
Implement debugging function for 386 on linux with reference to AMD64.
There are a few remaining problems that need to be solved in another time.
1. The stacktrace of cgo are not exactly as expected.
2. Not implement `core` for now.
3. Not implement `call` for now. Can't not find `runtime·debugCallV1` or
similar function in $GOROOT/src/runtime/asm_386.s.
Update #20
Removes the restriction that the DWARF type for the receiver of a method
must be a TypeDef. This seems reasonable in practice, but it turns out
Go DWARF does not consider
```
type X int
```
to be a typedef. This patch also allows for calling a method where the
receiver is not used or passed in, such as:
```
func (_ X) Method() { println("why") }
```
* pkg/proc: Introduce Target
* pkg/proc: Remove Common.fncallEnabled
Realistically we only block it on recorded backends.
* pkg/proc: Move fncallForG to Target
* pkg/proc: Remove CommonProcess
Remove final bit of functionality stored in CommonProcess and move it to
*Target.
* pkg/proc: Add SupportsFunctionCall to Target
Trust argument order to determine argument frame layout when calling
functions, this allows calling optimized functions and removes the
special cases for runtime.mallocgc.
Fixes#1589
Moves EvalScope methods to the proper file and organizes everything
together. Also makes some EvalScope methods no longer methods and just
pure functions.
Add variables flag to mark variables that are allocated on a register
(and have no address) and variables that we read as result of a
function call (and are allocated on a stack that no longer exists when
we show them to the user).
