In order for DAP to support halting the program (either manually or on a breakpoint) performing some action and then resuming execution, there needs to be a way to stop the program without clearing the internal breakpoints. This is necessary for log points and stopping the program to set breakpoints.
The debugging UI makes it seem like a user should be able to set or clear a breakpoint at any time. Adding this ability to complete synchronous requests while the program is running is thus important to create a seamless user experience.
This change just adds a configuration to determine whether the target should clear the stepping breakpoints, and changes the server to use this new mode. Using the new mode means that the DAP server must determine when it expect the next to be canceled and do this manually.
* terminal,service: add way to see internal breakpoints
Now that Delve has internal breakpoints that survive for long periods
of time it will be useful to have an option to display them.
* proc,terminal,service: support stack watchpoints
Adds support for watchpoints on stack allocated variables.
When a stack variable is watched, in addition to the normal watchpoint
some support breakpoints are created:
- one breakpoint inside runtime.copystack, used to adjust the address
of the watchpoint when the stack is resized
- one or more breakpoints used to detect when the stack variable goes
out of scope, those are similar to the breakpoints set by StepOut.
Implements #279
* proc: move breakpoint condition evaluation out of backends
Moves breakpoint condition evaluation from the point where breakpoints
are set, inside ContinueOnce, to (*Target).Continue.
This accomplishes three things:
1. the breakpoint evaluation method needs not be exported anymore
2. breakpoint condition evaluation can be done with a full scope,
containing a Target object, something that wasn't possible before
because ContinueOnce doesn't have access to the Target object.
3. moves breakpoint condition evaluation out of the critical section
where some of the threads of the target process might be still
running.
* proc/native: handle process death during stop() on Windows
It is possible that the thread dies while we are inside the stop()
function. This results in an Access is denied error being returned by
SuspendThread being called on threads that no longer exist.
Delay the reporting the error from SuspendThread until the end of
stop() and only report it if the thread still exists at that point.
Fixes flakyness with TestIssue1101 that was exacerbated by moving
breakpoint condition evaluation outside of the backends.
* pkg/proc: Prefer throw instead of fatalthrow
Currently there is a breakpoint set at runtime.fatalthrow to catch any
situation where the runtime crashes (e.g. deadlock).
When we do this, we go up a frame in order to parse the crash reason.
The problem is that there is no guarentee the "s" variable we attempt to
parse will still be considered "live".
Since runtime.fatalthrow is never called directly, set a breakpoint on
runtime.throw instead and prevent having
to search up a stack frame in order to
get the throw reason.
Fixes#2602
* service/dap: Fix TestFatalThrowBreakpoint
* Reenable TestFatalThrow DAP test
* service/dap: Don't skip test on < 1.17
* service/dap: Update test constraint for 1.16
* pkg/proc: Reinstate runtime.fatalthrow as switchstack exception
Changes Breakpoint to allow multiple overlapping internal breakpoints
on the same instruction address.
This is done by changing the Breakpoint structure to contain a list of
"breaklets", each breaklet has a BreakpointKind and a condition
expression, independent of the other.
A breakpoint is considered active if any of its breaklets are active.
A breakpoint is removed when all its breaklets are removed.
We also change the terminology "internal breakpoint" to "stepping
breakpoint":
HasInternalBreakpoints -> HasSteppingBreakpoints
IsInternal -> IsStepping
etc...
The motivation for this change is implementing watchpoints on stack
variables.
Watching a stack variable requires also setting a special breakpoint to
find out when the variable goes out of scope. These breakpoints can not
be UserBreakpoints because only one user breakpoint is allowed on the
same instruction and they can not be internal breakpoints because they
should not be cleared when a next operation is completed (they should
be cleared when the variable watch is cleared).
Updates #279
* proc: support new Go 1.17 panic/defer mechanism
Go 1.17 will create wrappers for deferred calls that take arguments.
Change defer reading code so that wrappers are automatically unwrapped.
Also the deferred function is called directly by runtime.gopanic, without going through runtime.callN which means that sometimes when a panic happens the stack is either:
0. deferred function call
1. deferred call wrapper
2. runtime.gopanic
or:
0. deferred function call
1. runtime.gopanic
instead of always being:
0. deferred function call
1. runtime.callN
2. runtime.gopanic
the isPanicCall check is changed accordingly.
* test: miscellaneous minor test fixes for Go 1.17
* proc: resolve inlined calls when stepping out of runtime.breakpoint
Calls to runtime.Breakpoint are inlined in Go 1.17 when inlining is
enabled, resolve inlined calls in stepInstructionOut.
* proc: add support for debugCallV2 with regabi
This change adds support for the new debug call protocol which had to
change for the new register ABI introduced in Go 1.17.
Summary of changes:
- Abstracts over the debug call version depending on the Go version
found in the binary.
- Uses R12 instead of RAX as the debug protocol register when the binary
is from Go 1.17 or later.
- Creates a variable directly from the DWARF entry for function
arguments to support passing arguments however the ABI expects.
- Computes a very conservative stack frame size for the call when
injecting a call into a Go process whose version is >=1.17.
Co-authored-by: Michael Anthony Knyszek <mknyszek@google.com>
Co-authored-by: Alessandro Arzilli <alessandro.arzilli@gmail.com>
* TeamCity: enable tests on go-tip
* goversion: version compatibility bump
* TeamCity: fix go-tip builds on macOS/arm64
Co-authored-by: Michael Anthony Knyszek <mknyszek@google.com>
ConvertEvalScope() attempts to find the scope for the specified
goroutine id and frame index. If the goroutine that is found is nil,
then it falls back to the threads stack trace to find the scope.
This fix makes sure that the frame id is taken into account for
thread strack traces as well.
We told clients that further loading of variables can be done by
specifying a type cast using the address of a variable that we
returned.
This does not work for registerized variables (or, in general,
variables that have a complex location expression) because we don't
give them unique addresses and we throw away the compositeMemory object
we made to read them.
This commit changes proc so that:
1. variables with location expression divided in pieces do get a unique
memory address
2. the compositeMemory object is saved somewhere
3. when an integer is cast back into a pointer type we look through our
saved compositeMemory objects to see if there is one that covers the
specified address and use it.
The unique memory addresses we generate have the MSB set to 1, as
specified by the Intel 86x64 manual addresses in this form are reserved
for kernel memory (which we can not read anyway) so we are guaranteed
to never generate a fake memory address that overlaps a real memory
address of the application.
The unfortunate side effect of this is that it will break clients that
do not deserialize the address to a 64bit integer. This practice is
contrary to how we defined our types and contrary to the specification
of the JSON format, as of json.org, however it is also fairly common,
due to javascript itself having only 53bit integers.
We could come up with a new mechanism but then even more old clients
would have to be changed.
While Go still mostly uses DWARF v4, newer versions of GCC will emit
DWARF v5 by default. This patch improves support for DWARF v5 by parsing
the .debug_line_str section and using that during file:line lookups.
This patch only includes support for files, not directories.
Co-authored-by: Derek Parker <deparker@redhat.com>
* pkg/proc: implement support for hit count breakpoints
* update comment
* udpate hitcount comment
* update HitCond description
* add test for hit condition error
* respond to review
* service/dap: add support for hit count breakpoints
* use amendbps to preserve hit counts
* update test health doc
* fix failing test
* simplify hit conditions
* REmove RequestString, use name instead
* update backend_test_health.md
* document hit count cond
* fix tests
A RequestManualStop received while the target program is stopped can
induce a crash when the target is restarted.
This is caused by the phantom breakpoint detection that was introduced
in PR #2179 / commit e69d536.
Instead of always interpreting an unexplained SIGTRAP as a phantom
breakpoint memorize all possible unreported breakpoint hits and only
act on it when the thread hasn't moved from one.
Also clarifies the behavior of the halt command when it is received
while the target is stopped or in the process of stopping.
Adds the low-level support for watchpoints (aka data breakpoints) to
the native linux/amd64 backend.
Does not add user interface or functioning support for watchpoints
on stack variables.
Updates #279
There seems to be a problem where debugserver will leave a zombie
process instead of detaching correctly, we are sending the right
commands, it doesn't seem to be a problem with Delve.
This adds a workaround for the bug described at:
https://github.com/golang/go/issues/25841
Because dsymutil running on PIE does not adjust the address of
debug_frame entries (but adjusts debug_info entries) we try to do the
adjustment ourselves.
Updates #2346
- remove github workflow for testing macOS/amd64 that is now covered by
TeamCity
- fix DeepSource glob patterns to actually match what they are intended
to match (did the interpretation change?)
- disable some cgo tests on darwin/arm64
Delve represents registerized variables (fully or partially) using
compositeMemory, implementing proc.(*compositeMemory).WriteMemory is
necessary to make SetVariable and function calls work when Go will
switch to using the register calling convention in 1.17.
This commit also makes some refactoring by moving the code that
converts between register numbers and register names out of pkg/proc
into a different package.
* TeamCity: add linux/arm64/tip configuration
So that it can be tested when we make the next-version-support-branch.
* tests: disable failing cgo tests on arm64
* proc/core: off-by-one error reading ELF core files
core.(*splicedMemory).ReadMemory checked the entry interval
erroneously when dealing with contiguous entries.
* terminal,service,proc/*: adds dump command (gcore equivalent)
Adds the `dump` command that creates a core file from the target process.
Backends will need to implement a new, optional, method `MemoryMap` that
returns a list of mapped memory regions.
Additionally the method `DumpProcessNotes` can be implemented to write out
to the core file notes describing the target process and its threads. If
DumpProcessNotes is not implemented `proc.Dump` will write a description of
the process and its threads in a OS/arch-independent format (that only Delve
understands).
Currently only linux/amd64 implements `DumpProcessNotes`.
Core files are only written in ELF, there is no minidump or macho-o writers.
# Conflicts:
# pkg/proc/proc_test.go
Add a helper method for collecting line table file references that
does the correct thing for DWARF 5 vs DWARF 4 (in the latter case you
have an implicit 0 entry which is the comp dir, whereas in the former
case you do not). This is to avoid out-of-bounds errors when examining
the file table section of a DWARF 5 compilation unit's line table.
Included is a new linux/amd-only test that includes a precompiled C
object file with a DWARF-5 section that triggers the bug in question.
Fixes#2319
Changs TestClientServer_FullStacktrace and
Test1ClientServer_FullStacktrace to log more information, also removes
code from TestFrameEvaluation that could mask the error.
Updates #2231
Fix bug in DAP test: TestEvaluateCallRequest.
In Go 1.15 the call injection will be executed on a different goroutine
from the goroutine where it was started on to avoid confusing the
garbage collector, the test must be aware of this fact and use the
goroutine ID from the stopped response instead of assuming 1 is the
currently selected goroutine.
Disables TestAttachDetach when running in Github Actions.
Disable some coredump tests when running in Github Actions (core size
limits?).
On linux we can not read memory if the thread we use to do it is
occupied doing certain system calls. The exact conditions when this
happens have never been clear.
This problem was worked around by using the Blocked method which
recognized the most common circumstances where this would happen.
However this is a hack: Blocked returning true doesn't mean that the
problem will manifest and Blocked returning false doesn't necessarily
mean the problem will not manifest. A side effect of this is issue
#2151 where sometimes we can't read the memory of a thread and find its
associated goroutine.
This commit fixes this problem by always reading memory using a thread
we know to be good for this, specifically the one returned by
ContinueOnce. In particular the changes are as follows:
1. Remove (ProcessInternal).CurrentThread and
(ProcessInternal).SetCurrentThread, the "current thread" becomes a
field of Target, CurrentThread becomes a (*Target) method and
(*Target).SwitchThread basically just sets a field Target.
2. The backends keep track of their own internal idea of what the
current thread is, to use it to read memory, this is the thread they
return from ContinueOnce as trapthread
3. The current thread in the backend and the current thread in Target
only ever get synchronized in two places: when the backend creates a
Target object the currentThread field of Target is initialized with the
backend's current thread and when (*Target).Restart gets called (when a
recording is rewound the currentThread used by Target might not exist
anymore).
4. We remove the MemoryReadWriter interface embedded in Thread and
instead add a Memory method to Process that returns a MemoryReadWriter.
The backends will return something here that will read memory using
the current thread saved by the backend.
5. The Thread.Blocked method is removed
One possible problem with this change is processes that have threads
with different memory maps. As far as I can determine this could happen
on old versions of linux but this option was removed in linux 2.5.
Fixes#2151
Due to a missing check TestCgoStacktrace2 didn't actually check
anything. Enable it and then skip it on linux/386 and linux/arm64 where
it's broken.
Co-authored-by: a <a@kra>
* proc/tests: keep track of tests skipped due to backend problems
Mark tests skipped due to backend problems and add a script to keep
track of them.
* Travis-CI: add ignorechecksum option to chocolatey command
Looks like a configuration problem on chocolatey's end.
Since proc is supposed to work independently from the target
architecture it shouldn't use architecture-dependent types, like
uintptr. For example when reading a 64bit core file on a 32bit
architecture, uintptr will be 32bit but the addresses proc needs to
represent will be 64bit.
Adds features to support default file descriptor redirects for the
target process:
1. A new command line flag '--redirect' and '-r' are added to specify
file redirects for the target process
2. New syntax is added to the 'restart' command to specify file
redirects.
3. Interactive instances will check if stdin/stdout and stderr are
terminals and print a helpful error message if they aren't.
An internal breakpoint condition shouldn't ever error:
* use a ThreadContext to evaluate conditions if a goroutine isn't
available
* evaluate runtime.curg to a fake g variable containing only
`goid == 0` when there is no current goroutine
Fixes#2113
Commit 1ee8d5c reviewed in Pull Request #1960 relaxed some tests using
goroutinestackprog but missed others.
Fixes some test flakiness that isn't relevant.
* proc: start variable visibility one line after their decl line
In most cases variables shouldn't be visible on their declaration line
because they won't be initialized there.
Function arguments are treated as an exception.
This fix is only applied to programs compiled with Go 1.15 or later as
previous versions of Go did not report the correct declaration line for
variables captured by closures.
Fixes#1134
* proc: silence go vet error
* Makefile: enable PIE tests on windows/Go 1.15
* core: support core files for PIEs on windows
* goversion: add Go 1.15 to supported versions
* proc: fix function call injection for Go 1.15
Go 1.15 changed the call injection protocol so that the runtime will
execute the injected call on a different (new) goroutine.
This commit changes the function call support in delve to:
1. correctly track down the call injection state after the runtime
switches to a different goroutine.
2. correctly perform the escapeCheck when stack values can come from
multiple goroutine stacks.
* proc: miscellaneous fixed for call injection under macOS with go 1.15
- create copy of SP in debugCallAXCompleteCall case because the code
used to assume that regs doesn't change
- fix automatic address calculation for function arguments when an
argument has a spurious DW_OP_piece at entry
The file:line information for the entrypoint is more acccurate than the
file:line information at a return point, which could be affected by a
compiler bug.
Fixes#2086
Recent change #2061:
292f5c69f0c769fd32c2e8b1e7153b56e908efd7
proc: step into unexported runtime funcs when already inside runtime
means that TestIssue414 (which tries to step repeatedly until the
program exits) can now steps through way more runtime code than it ever
did before. This causes this test to occasionally fail. Stepping
blindly through runtime code has never been particularly safe as the
runtime can switch to a different goroutine causing Delve to misbehave.
This change restores the previous behavior of TestIssue414 where Step
behaved like Next inside runtime code.
On platforms other than macOS this doesn't matter but on macOS a
segmentation fault will cause ContinueOnce to return an error, before
returning it we should still fix the current thread and selected
goroutine values.
Fixes#2078
Changes implementations of proc.Registers interface and the
op.DwarfRegisters struct so that floating point registers can be loaded
only when they are needed.
Removes the floatingPoint parameter from proc.Thread.Registers.
This accomplishes three things:
1. it simplifies the proc.Thread.Registers interface
2. it makes it impossible to accidentally create a broken set of saved
registers or of op.DwarfRegisters by accidentally calling
Registers(false)
3. it improves general performance of Delve by avoiding to load
floating point registers as much as possible
Floating point registers are loaded under two circumstances:
1. When the Slice method is called with floatingPoint == true
2. When the Copy method is called
Benchmark before:
BenchmarkConditionalBreakpoints-4 1 4327350142 ns/op
Benchmark after:
BenchmarkConditionalBreakpoints-4 1 3852642917 ns/op
Updates #1549
