PenaDevops/delve
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

terminal: adds embedded scripting language (#1466)

Browse Source 
If the argument of 'source' ends in '.star' it will be interpreted as a
starlark script.
If the argument of 'source' is '-' an interactive starlark repl will be
started.

For documentation on how the starlark execution environment works see
Documentation/cli/starlark.md.

The starlark API is autogenerated from the JSON-RPC API by
script/gen-starlark-bindings.go.
In general for each JSON-RPC API a single global starlark function is
created.
When one of those functions is called (through a starlark script) the
arguments are converted to go structs using reflection. See
unmarshalStarlarkValue in pkg/terminal/starbind/conv.go.
If there are no type conversion errors the JSON-RPC call is executed.
The return value of the JSON-RPC call is converted back into a starlark
value by interfaceToStarlarkValue (same file):

* primitive types (such as integers, floats or strings) are converted
  by creating the corresponding starlark value.
* compound types (such as structs and slices) are converted by wrapping
  their reflect.Value object into a type that implements the relevant
  starlark interfaces.
* api.Variables are treated specially so that their Value field can be
  of the proper type instead of always being a string.

Implements #1415, #1443
This commit is contained in:
Alessandro Arzilli 2019-07-02 19:55:27 +02:00 committed by Derek Parker
parent 116b9631dc
commit ed35dce7a3
43 changed files with 15947 additions and 6 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
Documentation/cli/README.md
View File

@ -346,6 +346,10 @@ See [Documentation/cli/expr.md](//github.com/go-delve/delve/tree/master/Document
Executes a file containing a list of delve commands
source <path>
If path ends with the .star extension it will be interpreted as a starlark script. See [Documentation/cli/starlark.md](//github.com/go-delve/delve/tree/master/Documentation/cli/starlark.md) for the syntax.
If path is a single '-' character an interactive starlark interpreter will start instead. Type 'exit' to exit.
## sources

263
Documentation/cli/starlark.md Normal file
View File

@ -0,0 +1,263 @@
# Introduction
Passing a file with the .star extension to the `source` command will cause delve to interpret it as a starlark script.
Starlark is a dialect of python, a [specification of its syntax can be found here](https://github.com/google/starlark-go/blob/master/doc/spec.md).
In addition to the normal starlark built-ins delve defines [a number of global functions](#Starlark-built-ins) that can be used to interact with the debugger.
After the file has been evaluated delve will bind any function starting with `command_` to a command-line command: for example `command_goroutines_wait_reason` will be bound to `goroutines_wait_reason`.
Then if a function named `main` exists it will be executed.
Global functions with a name that begins with a capital letter will be available to other scripts.
# Starlark built-ins
<!-- BEGIN MAPPING TABLE -->
Function | API Call
---------|---------
amend_breakpoint(Breakpoint) | Equivalent to API call [AmendBreakpoint](https://godoc.org/github.com/go-delve/delve/service/rpc2#RPCServer.AmendBreakpoint)
ancestors(GoroutineID, NumAncestors, Depth) | Equivalent to API call [Ancestors](https://godoc.org/github.com/go-delve/delve/service/rpc2#RPCServer.Ancestors)
attached_to_existing_process() | Equivalent to API call [AttachedToExistingProcess](https://godoc.org/github.com/go-delve/delve/service/rpc2#RPCServer.AttachedToExistingProcess)
cancel_next() | Equivalent to API call [CancelNext](https://godoc.org/github.com/go-delve/delve/service/rpc2#RPCServer.CancelNext)
checkpoint(Where) | Equivalent to API call [Checkpoint](https://godoc.org/github.com/go-delve/delve/service/rpc2#RPCServer.Checkpoint)
clear_breakpoint(Id, Name) | Equivalent to API call [ClearBreakpoint](https://godoc.org/github.com/go-delve/delve/service/rpc2#RPCServer.ClearBreakpoint)
clear_checkpoint(ID) | Equivalent to API call [ClearCheckpoint](https://godoc.org/github.com/go-delve/delve/service/rpc2#RPCServer.ClearCheckpoint)
raw_command(Name, ThreadID, GoroutineID, ReturnInfoLoadConfig, Expr, UnsafeCall) | Equivalent to API call [Command](https://godoc.org/github.com/go-delve/delve/service/rpc2#RPCServer.Command)
create_breakpoint(Breakpoint) | Equivalent to API call [CreateBreakpoint](https://godoc.org/github.com/go-delve/delve/service/rpc2#RPCServer.CreateBreakpoint)
detach(Kill) | Equivalent to API call [Detach](https://godoc.org/github.com/go-delve/delve/service/rpc2#RPCServer.Detach)
disassemble(Scope, StartPC, EndPC, Flavour) | Equivalent to API call [Disassemble](https://godoc.org/github.com/go-delve/delve/service/rpc2#RPCServer.Disassemble)
eval(Scope, Expr, Cfg) | Equivalent to API call [Eval](https://godoc.org/github.com/go-delve/delve/service/rpc2#RPCServer.Eval)
find_location(Scope, Loc) | Equivalent to API call [FindLocation](https://godoc.org/github.com/go-delve/delve/service/rpc2#RPCServer.FindLocation)
function_return_locations(FnName) | Equivalent to API call [FunctionReturnLocations](https://godoc.org/github.com/go-delve/delve/service/rpc2#RPCServer.FunctionReturnLocations)
get_breakpoint(Id, Name) | Equivalent to API call [GetBreakpoint](https://godoc.org/github.com/go-delve/delve/service/rpc2#RPCServer.GetBreakpoint)
get_thread(Id) | Equivalent to API call [GetThread](https://godoc.org/github.com/go-delve/delve/service/rpc2#RPCServer.GetThread)
is_multiclient() | Equivalent to API call [IsMulticlient](https://godoc.org/github.com/go-delve/delve/service/rpc2#RPCServer.IsMulticlient)
last_modified() | Equivalent to API call [LastModified](https://godoc.org/github.com/go-delve/delve/service/rpc2#RPCServer.LastModified)
breakpoints() | Equivalent to API call [ListBreakpoints](https://godoc.org/github.com/go-delve/delve/service/rpc2#RPCServer.ListBreakpoints)
checkpoints() | Equivalent to API call [ListCheckpoints](https://godoc.org/github.com/go-delve/delve/service/rpc2#RPCServer.ListCheckpoints)
dynamic_libraries() | Equivalent to API call [ListDynamicLibraries](https://godoc.org/github.com/go-delve/delve/service/rpc2#RPCServer.ListDynamicLibraries)
function_args(Scope, Cfg) | Equivalent to API call [ListFunctionArgs](https://godoc.org/github.com/go-delve/delve/service/rpc2#RPCServer.ListFunctionArgs)
functions(Filter) | Equivalent to API call [ListFunctions](https://godoc.org/github.com/go-delve/delve/service/rpc2#RPCServer.ListFunctions)
goroutines(Start, Count) | Equivalent to API call [ListGoroutines](https://godoc.org/github.com/go-delve/delve/service/rpc2#RPCServer.ListGoroutines)
local_vars(Scope, Cfg) | Equivalent to API call [ListLocalVars](https://godoc.org/github.com/go-delve/delve/service/rpc2#RPCServer.ListLocalVars)
package_vars(Filter, Cfg) | Equivalent to API call [ListPackageVars](https://godoc.org/github.com/go-delve/delve/service/rpc2#RPCServer.ListPackageVars)
registers(ThreadID, IncludeFp) | Equivalent to API call [ListRegisters](https://godoc.org/github.com/go-delve/delve/service/rpc2#RPCServer.ListRegisters)
sources(Filter) | Equivalent to API call [ListSources](https://godoc.org/github.com/go-delve/delve/service/rpc2#RPCServer.ListSources)
threads() | Equivalent to API call [ListThreads](https://godoc.org/github.com/go-delve/delve/service/rpc2#RPCServer.ListThreads)
types(Filter) | Equivalent to API call [ListTypes](https://godoc.org/github.com/go-delve/delve/service/rpc2#RPCServer.ListTypes)
process_pid() | Equivalent to API call [ProcessPid](https://godoc.org/github.com/go-delve/delve/service/rpc2#RPCServer.ProcessPid)
recorded() | Equivalent to API call [Recorded](https://godoc.org/github.com/go-delve/delve/service/rpc2#RPCServer.Recorded)
restart(Position, ResetArgs, NewArgs) | Equivalent to API call [Restart](https://godoc.org/github.com/go-delve/delve/service/rpc2#RPCServer.Restart)
set_expr(Scope, Symbol, Value) | Equivalent to API call [Set](https://godoc.org/github.com/go-delve/delve/service/rpc2#RPCServer.Set)
stacktrace(Id, Depth, Full, Defers, Cfg) | Equivalent to API call [Stacktrace](https://godoc.org/github.com/go-delve/delve/service/rpc2#RPCServer.Stacktrace)
state(NonBlocking) | Equivalent to API call [State](https://godoc.org/github.com/go-delve/delve/service/rpc2#RPCServer.State)
dlv_command(command) | Executes the specified command as if typed at the dlv_prompt
read_file(path) | Reads the file as a string
write_file(path, contents) | Writes string to a file
cur_scope() | Returns the current evaluation scope
default_load_config() | Returns the current default load configuration
<!-- END MAPPING TABLE -->
## Should I use raw_command or dlv_command?
There are two ways to resume the execution of the target program:
raw_command("continue")
dlv_command("continue")
The first one maps to the API call [Command](https://godoc.org/github.com/derekparker/delve/service/rpc2#RPCServer.Command). As such all the caveats explained in the [Client HowTo](../api/ClientHowto.md).
The latter is equivalent to typing `continue` to the `(dlv)` command line and should do what you expect.
In general `dlv_command("continue")` should be preferred, unless the behavior you wish to produces diverges significantly from that of the command line's `continue`.
# Creating new commands
Any global function with a name starting with `command_` will be made available as a command line command. If the function has a single argument named `args` all arguments passed on the command line will be passed to the function as a single string.
Otherwise arguments passed on the command line are interpreted as starlark expressions. See the [expression arguments](#expression-arguments) example.
If the command function has a doc string it will be used as a help message.
# Working with variables
Variables of the target program can be accessed using `local_vars`, `function_args` or the `eval` functions. Each variable will be returned as a [Variable](https://godoc.org/github.com/go-delve/delve/service/api#Variable) struct, with one special field: `Value`.
## Variable.Value
The `Value` field will return the value of the target variable converted to a starlark value:
* integers, floating point numbers and strings are represented by equivalent starlark values
* structs are represented as starlark dictionaries
* slices and arrays are represented by starlark lists
* maps are represented by starlark dicts
* pointers and interfaces are represented by a one-element starlark list containing the value they point to
For example, given this variable in the target program:
```
type astruct struct {
A int
B int
}
s2 := []astruct{{1, 2}, {3, 4}, {5, 6}, {7, 8}, {9, 10}, {11, 12}, {13, 14}, {15, 16}}
```
The following is possible:
```
>>> s2 = eval(None, "s2").Variable
>>> s2.Value[0] # access of a slice item by index
main.astruct {A: 1, B: 2}
>>> a = s2.Value[1]
>>> a.Value.A # access to a struct field
3
>>> a.Value.A + 10 # arithmetic on the value of s2[1].X
13
>>> a.Value["B"] # access to a struct field, using dictionary syntax
4
```
For more examples see the [linked list example](#Print-all-elements-of-a-linked-list) below.
# Examples
## Listing goroutines and making custom commands
Create a `goroutine_start_line` command that prints the starting line of each goroutine, sets `gsl` as an alias:
```
def command_goroutine_start_line(args):
gs = goroutines().Goroutines
for g in gs:
line = read_file(g.StartLoc.File).splitlines()[g.StartLoc.Line-1].strip()
print(g.ID, "\t", g.StartLoc.File + ":" + str(g.StartLoc.Line), "\t", line)
def main():
dlv_command("config alias goroutine_start_line gsl")
```
Use it like this:
```
(dlv) source goroutine_start_line.start
(dlv) goroutine_start_line
1 /usr/local/go/src/runtime/proc.go:110 func main() {
2 /usr/local/go/src/runtime/proc.go:242 func forcegchelper() {
17 /usr/local/go/src/runtime/mgcsweep.go:64 func bgsweep(c chan int) {
18 /usr/local/go/src/runtime/mfinal.go:161 func runfinq() {
(dlv) gsl
1 /usr/local/go/src/runtime/proc.go:110 func main() {
2 /usr/local/go/src/runtime/proc.go:242 func forcegchelper() {
17 /usr/local/go/src/runtime/mgcsweep.go:64 func bgsweep(c chan int) {
18 /usr/local/go/src/runtime/mfinal.go:161 func runfinq() {
```
## Expression arguments
After evaluating this script:
```
def command_echo(args):
print(args)
def command_echo_expr(a, b, c):
print("a", a, "b", b, "c", c)
```
The first commnad, `echo`, takes its arguments as a single string, while for `echo_expr` it will be possible to pass starlark expression as arguments:
```
(dlv) echo 2+2, 2-1, 2*3
"2+2, 2-1, 2*3"
(dlv) echo_expr 2+2, 2-1, 2*3
a 4 b 1 c 6
```
## Creating breakpoints
Set a breakpoint on all private methods of package `main`:
```
def main():
for f in functions().Funcs:
v = f.split('.')
if len(v) != 2:
continue
if v[0] != "main":
continue
if v[1][0] >= 'a' and v[1][0] <= 'z':
create_breakpoint({ "FunctionName": f, "Line": -1 }) # see documentation of RPCServer.CreateBreakpoint
```
## Switching goroutines
Create a command, `switch_to_main_goroutine`, that searches for a goroutine running a function in the main package and switches to it:
```
def command_switch_to_main_goroutine(args):
for g in goroutines().Goroutines:
if g.currentLoc.function != None and g.currentLoc.function.name.startswith("main."):
print("switching to:", g.id)
raw_command("switchGoroutine", GoroutineID=g.id)
break
```
## Repeatedly executing the target until a breakpoint is hit.
Repeatedly call continue and restart until the target hits a breakpoint.
```
def command_flaky(args):
"Repeatedly runs program until a breakpoint is hit"
while True:
if dlv_command("continue") == None:
break
dlv_command("restart")
```
## Print all elements of a linked list
```
def command_linked_list(args):
"""Prints the contents of a linked list.
linked_list <var_name> <next_field_name> <max_depth>
Prints up to max_depth elements of the linked list variable 'var_name' using 'next_field_name' as the name of the link field.
"""
var_name, next_field_name, max_depth = args.split(" ")
max_depth = int(max_depth)
next_name = var_name
v = eval(None, var_name).Variable.Value
for i in range(0, max_depth):
print(str(i)+":",v)
if v[0] == None:
break
v = v[next_field_name]
```
## Find an array element matching a predicate
```
def command_find_array(arr, pred):
"""Calls pred for each element of the array or slice 'arr' returns the index of the first element for which pred returns true.
find_arr <arr> <pred>
Example use (find the first element of slice 's2' with field A equal to 5):
find_arr "s2", lambda x: x.A == 5
"""
arrv = eval(None, arr).Variable
for i in range(0, arrv.Len):
v = arrv.Value[i]
if pred(v):
print("found", i)
return
print("not found")
```

9
_fixtures/create_breakpoint_main.star Normal file
View File

@ -0,0 +1,9 @@
def main():
for f in functions().Funcs:
v = f.split('.')
if len(v) != 2:
continue
if v[0] != "main":
continue
if v[1][0] >= 'a' and v[1][0] <= 'z':
create_breakpoint({ "FunctionName": f, "Line": -1 }) # see documentation of RPCServer.CreateBreakpoint

2
_fixtures/echo_expr.star Normal file
View File

@ -0,0 +1,2 @@
def command_echo_expr(a, b, c):
print("a", a, "b", b, "c", c)

17
_fixtures/find_array.star Normal file
View File

@ -0,0 +1,17 @@
def command_find_array(arr, pred):
"""Calls pred for each element of the array or slice 'arr' returns the index of the first element for which pred returns true.
find_arr <arr>, <pred>
Example use:
find_arr "s2", lambda x: x.A == 5
"""
arrv = eval(None, arr).Variable
for i in range(0, arrv.Len):
v = arrv.Value[i]
if pred(v):
print("found", i)
return
print("not found")

9
_fixtures/goroutine_start_line.star Normal file
View File

@ -0,0 +1,9 @@
def command_goroutine_start_line(args):
"prints the line of source code that started each currently running goroutine"
gs = goroutines().Goroutines
for g in gs:
line = read_file(g.StartLoc.File).splitlines()[g.StartLoc.Line-1].strip()
print(g.ID, "\t", g.StartLoc.File + ":" + str(g.StartLoc.Line), "\t", line)
def main():
dlv_command("config alias goroutine_start_line gsl")

16
_fixtures/linked_list.star Normal file
View File

@ -0,0 +1,16 @@
def command_linked_list(args):
"""Prints the contents of a linked list.
linked_list <var_name> <next_field_name> <max_depth>
Prints up to max_depth elements of the linked list variable 'var_name' using 'next_field_name' as the name of the link field.
"""
var_name, next_field_name, max_depth = args.split(" ")
max_depth = int(max_depth)
next_name = var_name
v = eval(None, var_name).Variable.Value
for i in range(0, max_depth):
print(str(i)+":",v)
if v[0] == None:
break
v = v[next_field_name]

6
_fixtures/starlark_map_iteration.star Normal file
View File

@ -0,0 +1,6 @@
v = eval(None, "m1").Variable
n = 0
for k in v.Value:
n = n + 1
print(k)
print(n)

6
_fixtures/switch_to_main_goroutine.star Normal file
View File

@ -0,0 +1,6 @@
def command_switch_to_main_goroutine(args):
for g in goroutines().Goroutines:
if g.CurrentLoc.Function != None and g.CurrentLoc.Function.Name_.startswith("main."):
print("switching to:", g.ID)
raw_command("switchGoroutine", GoroutineID=g.ID)
break

9
_fixtures/testvariables2.go
View File

@ -101,6 +101,11 @@ type Block struct {
cache *Cache
}
type List struct {
N int
Next *List
}
func main() {
i1 := 1
i2 := 2
@ -275,6 +280,8 @@ func main() {
},
}
ll := &List{0, &List{1, &List{2, &List{3, &List{4, nil}}}}}
var amb1 = 1
runtime.Breakpoint()
for amb1 := 0; amb1 < 10; amb1++ {
@ -282,5 +289,5 @@ func main() {
}
runtime.Breakpoint()
fmt.Println(i1, i2, i3, p1, amb1, s1, s3, a1, p2, p3, s2, as1, str1, f1, fn1, fn2, nilslice, nilptr, ch1, chnil, m1, mnil, m2, m3, up1, i4, i5, i6, err1, err2, errnil, iface1, iface2, ifacenil, arr1, parr, cpx1, const1, iface3, iface4, recursive1, recursive1.x, iface5, iface2fn1, iface2fn2, bencharr, benchparr, mapinf, mainMenu, b, b2, sd, anonstruct1, anonstruct2, anoniface1, anonfunc, mapanonstruct1, ifacearr, efacearr, ni8, ni16, ni32, pinf, ninf, nan, zsvmap, zsslice, zsvar, tm, errtypednil, emptyslice, emptymap, byteslice, runeslice, bytearray, runearray, longstr, nilstruct, as2, as2.NonPointerRecieverMethod, s4, iface2map, issue1578)
fmt.Println(i1, i2, i3, p1, amb1, s1, s3, a1, p2, p3, s2, as1, str1, f1, fn1, fn2, nilslice, nilptr, ch1, chnil, m1, mnil, m2, m3, up1, i4, i5, i6, err1, err2, errnil, iface1, iface2, ifacenil, arr1, parr, cpx1, const1, iface3, iface4, recursive1, recursive1.x, iface5, iface2fn1, iface2fn2, bencharr, benchparr, mapinf, mainMenu, b, b2, sd, anonstruct1, anonstruct2, anoniface1, anonfunc, mapanonstruct1, ifacearr, efacearr, ni8, ni16, ni32, pinf, ninf, nan, zsvmap, zsslice, zsvar, tm, errtypednil, emptyslice, emptymap, byteslice, runeslice, bytearray, runearray, longstr, nilstruct, as2, as2.NonPointerRecieverMethod, s4, iface2map, issue1578, ll)
}

18
cmd/dlv/dlv_test.go
View File

@ -247,8 +247,7 @@ func TestGeneratedDoc(t *testing.T) {
var generatedBuf bytes.Buffer
commands := terminal.DebugCommands(nil)
commands.WriteMarkdown(&generatedBuf)
cliDocFilename := "Documentation/cli/README.md"
checkAutogenDoc(t, cliDocFilename, "scripts/gen-cli-docs.go", generatedBuf.Bytes())
checkAutogenDoc(t, "Documentation/cli/README.md", "scripts/gen-cli-docs.go", generatedBuf.Bytes())
// Checks gen-usage-docs.go
tempDir, err := ioutil.TempDir(os.TempDir(), "test-gen-doc")
@ -261,6 +260,21 @@ func TestGeneratedDoc(t *testing.T) {
docFilename := "Documentation/usage/" + doc.Name()
checkAutogenDoc(t, docFilename, "scripts/gen-usage-docs.go", slurpFile(t, tempDir+"/"+doc.Name()))
}
runScript := func(args ...string) []byte {
a := []string{"run"}
a = append(a, args...)
cmd := exec.Command("go", a...)
cmd.Dir = projectRoot()
out, err := cmd.CombinedOutput()
if err != nil {
t.Fatalf("could not run script %v: %v (output: %q)", args, err, string(out))
}
return out
}
checkAutogenDoc(t, "pkg/terminal/starbind/starlark_mapping.go", "'go generate' inside pkg/terminal/starbind", runScript("scripts/gen-starlark-bindings.go", "go", "-"))
checkAutogenDoc(t, "Documentation/cli/starlark.md", "'go generate' inside pkg/terminal/starbind", runScript("scripts/gen-starlark-bindings.go", "doc/dummy", "Documentation/cli/starlark.md"))
}
func TestExitInInit(t *testing.T) {

2
go.mod
View File

@ -1,6 +1,7 @@
module github.com/go-delve/delve
require (
github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e // indirect
github.com/cosiner/argv v0.0.0-20170225145430-13bacc38a0a5
github.com/cpuguy83/go-md2man v1.0.8 // indirect
github.com/inconshreveable/mousetrap v1.0.0 // indirect
@ -12,6 +13,7 @@ require (
github.com/sirupsen/logrus v0.0.0-20180523074243-ea8897e79973
github.com/spf13/cobra v0.0.0-20170417170307-b6cb39589372
github.com/spf13/pflag v0.0.0-20170417173400-9e4c21054fa1 // indirect
go.starlark.net v0.0.0-20190225160109-1174b2613e82
golang.org/x/arch v0.0.0-20171004143515-077ac972c2e4
golang.org/x/crypto v0.0.0-20180614174826-fd5f17ee7299 // indirect
golang.org/x/sys v0.0.0-20180614134839-8883426083c0

12
go.sum
View File

@ -1,3 +1,5 @@
github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e h1:fY5BOSpyZCqRo5OhCuC+XN+r/bBCmeuuJtjz+bCNIf8=
github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI=
github.com/cosiner/argv v0.0.0-20170225145430-13bacc38a0a5 h1:rIXlvz2IWiupMFlC45cZCXZFvKX/ExBcSLrDy2G0Lp8=
github.com/cosiner/argv v0.0.0-20170225145430-13bacc38a0a5/go.mod h1:p/NrK5tF6ICIly4qwEDsf6VDirFiWWz0FenfYBwJaKQ=
github.com/cpuguy83/go-md2man v1.0.8 h1:DwoNytLphI8hzS2Af4D0dfaEaiSq2bN05mEm4R6vf8M=
@ -20,6 +22,16 @@ github.com/spf13/cobra v0.0.0-20170417170307-b6cb39589372 h1:eRfW1vRS4th8IX2iQey
github.com/spf13/cobra v0.0.0-20170417170307-b6cb39589372/go.mod h1:1l0Ry5zgKvJasoi3XT1TypsSe7PqH0Sj9dhYf7v3XqQ=
github.com/spf13/pflag v0.0.0-20170417173400-9e4c21054fa1 h1:7bozMfSdo41n2NOc0GsVTTVUiA+Ncaj6pXNpm4UHKys=
github.com/spf13/pflag v0.0.0-20170417173400-9e4c21054fa1/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4=
go.starlark.net v0.0.0-20181108041844-f4938bde4080 h1:PynO3TmUXWWlWQ1FHArWPoFcoQR3oCaMm0l+d6rbjeo=
go.starlark.net v0.0.0-20181108041844-f4938bde4080/go.mod h1:c1/X6cHgvdXj6pUlmWKMkuqRnW4K8x2vwt6JAaaircg=
go.starlark.net v0.0.0-20181213015430-66ac3a2d309f h1:8Fs2MRAY3gcipeW5YuV7PR0PYZ30uyRurWJSON4km/I=
go.starlark.net v0.0.0-20181213015430-66ac3a2d309f/go.mod h1:c1/X6cHgvdXj6pUlmWKMkuqRnW4K8x2vwt6JAaaircg=
go.starlark.net v0.0.0-20181213164318-58464401628a h1:zSGIMfylaSMaCEjwVKLMO8Hb/YXDHik8Z+j6Yb1/MJc=
go.starlark.net v0.0.0-20181213164318-58464401628a/go.mod h1:c1/X6cHgvdXj6pUlmWKMkuqRnW4K8x2vwt6JAaaircg=
go.starlark.net v0.0.0-20190106165729-93b8d146431e h1:iAWNmlXeb8B9A+QSPlujsxdd5paZFxRA2RSGQE8qvJ4=
go.starlark.net v0.0.0-20190106165729-93b8d146431e/go.mod h1:c1/X6cHgvdXj6pUlmWKMkuqRnW4K8x2vwt6JAaaircg=
go.starlark.net v0.0.0-20190225160109-1174b2613e82 h1:vAHCTDREx7LqPeWqNeTvCKWcURJztP3wBKnF2Q0sW7Q=
go.starlark.net v0.0.0-20190225160109-1174b2613e82/go.mod h1:c1/X6cHgvdXj6pUlmWKMkuqRnW4K8x2vwt6JAaaircg=
golang.org/x/arch v0.0.0-20171004143515-077ac972c2e4 h1:TP7YcWHbnFq4v8/3wM2JwgM0SRRtsYJ7Z6Oj0arz2bs=
golang.org/x/arch v0.0.0-20171004143515-077ac972c2e4/go.mod h1:cYlCBUl1MsqxdiKgmc4uh7TxZfWSFLOGSRR090WDxt8=
golang.org/x/crypto v0.0.0-20180614174826-fd5f17ee7299 h1:zxP+xTjjk4kD+M5IFPweL7/4851FUhYkzbDqbzkN1JE=

16
pkg/terminal/command.go
View File

@ -12,6 +12,7 @@ import (
"math"
"os"
"os/exec"
"path/filepath"
"reflect"
"regexp"
"sort"
@ -297,7 +298,11 @@ Move the current frame down by <m>. The second form runs the command on the give
Executes the specified command (print, args, locals) in the context of the n-th deferred call in the current frame.`},
{aliases: []string{"source"}, cmdFn: c.sourceCommand, helpMsg: `Executes a file containing a list of delve commands
source <path>`},
source <path>
If path ends with the .star extension it will be interpreted as a starlark script. See $GOPATH/src/github.com/go-delve/delve/Documentation/cli/starlark.md for the syntax.
If path is a single '-' character an interactive starlark interpreter will start instead. Type 'exit' to exit.`},
{aliases: []string{"disassemble", "disass"}, cmdFn: disassCommand, helpMsg: `Disassembler.
[goroutine <n>] [frame <m>] disassemble [-a <start> <end>] [-l <locspec>]
@ -1566,6 +1571,15 @@ func (c *Commands) sourceCommand(t *Term, ctx callContext, args string) error {
return fmt.Errorf("wrong number of arguments: source <filename>")
}
if filepath.Ext(args) == ".star" {
_, err := t.starlarkEnv.Execute(args, nil, "main", nil)
return err
}
if args == "-" {
return t.starlarkEnv.REPL()
}
return c.executeFile(t, args)
}

43
pkg/terminal/command_test.go
View File

@ -17,6 +17,7 @@ import (
"github.com/go-delve/delve/pkg/config"
"github.com/go-delve/delve/pkg/goversion"
"github.com/go-delve/delve/pkg/logflags"
"github.com/go-delve/delve/pkg/proc/test"
"github.com/go-delve/delve/service"
"github.com/go-delve/delve/service/api"
@ -29,12 +30,15 @@ var testBackend, buildMode string
func TestMain(m *testing.M) {
flag.StringVar(&testBackend, "backend", "", "selects backend")
flag.StringVar(&buildMode, "test-buildmode", "", "selects build mode")
var logConf string
flag.StringVar(&logConf, "log", "", "configures logging")
flag.Parse()
test.DefaultTestBackend(&testBackend)
if buildMode != "" && buildMode != "pie" {
fmt.Fprintf(os.Stderr, "unknown build mode %q", buildMode)
os.Exit(1)
}
logflags.Setup(logConf != "", logConf, "")
os.Exit(test.RunTestsWithFixtures(m))
}
@ -70,14 +74,49 @@ func (ft *FakeTerminal) Exec(cmdstr string) (outstr string, err error) {
return
}
func (ft *FakeTerminal) ExecStarlark(starlarkProgram string) (outstr string, err error) {
outfh, err := ioutil.TempFile("", "cmdtestout")
if err != nil {
ft.t.Fatalf("could not create temporary file: %v", err)
}
stdout, stderr, termstdout := os.Stdout, os.Stderr, ft.Term.stdout
os.Stdout, os.Stderr, ft.Term.stdout = outfh, outfh, outfh
defer func() {
os.Stdout, os.Stderr, ft.Term.stdout = stdout, stderr, termstdout
outfh.Close()
outbs, err1 := ioutil.ReadFile(outfh.Name())
if err1 != nil {
ft.t.Fatalf("could not read temporary output file: %v", err)
}
outstr = string(outbs)
if logCommandOutput {
ft.t.Logf("command %q -> %q", starlarkProgram, outstr)
}
os.Remove(outfh.Name())
}()
_, err = ft.Term.starlarkEnv.Execute("<stdin>", starlarkProgram, "main", nil)
return
}
func (ft *FakeTerminal) MustExec(cmdstr string) string {
outstr, err := ft.Exec(cmdstr)
if err != nil {
ft.t.Errorf("output of %q: %q", cmdstr, outstr)
ft.t.Fatalf("Error executing <%s>: %v", cmdstr, err)
}
return outstr
}
func (ft *FakeTerminal) MustExecStarlark(starlarkProgram string) string {
outstr, err := ft.ExecStarlark(starlarkProgram)
if err != nil {
ft.t.Errorf("output of %q: %q", starlarkProgram, outstr)
ft.t.Fatalf("Error executing <%s>: %v", starlarkProgram, err)
}
return outstr
}
func (ft *FakeTerminal) AssertExec(cmdstr, tgt string) {
out := ft.MustExec(cmdstr)
if out != tgt {
@ -870,3 +909,7 @@ func TestIssue1493(t *testing.T) {
}
})
}
func findStarFile(name string) string {
return filepath.Join(test.FindFixturesDir(), name+".star")
}

695
pkg/terminal/starbind/conv.go Normal file
View File

@ -0,0 +1,695 @@
package starbind
import (
"errors"
"fmt"
"math"
"reflect"
"strconv"
"go.starlark.net/starlark"
"github.com/go-delve/delve/service/api"
)
// autoLoadConfig is the load configuration used to automatically load more from a variable
var autoLoadConfig = api.LoadConfig{false, 1, 1024, 64, -1}
// interfaceToStarlarkValue converts an interface{} variable (produced by
// decoding JSON) into a starlark.Value.
func (env *Env) interfaceToStarlarkValue(v interface{}) starlark.Value {
switch v := v.(type) {
case uint8:
return starlark.MakeUint64(uint64(v))
case uint16:
return starlark.MakeUint64(uint64(v))
case uint32:
return starlark.MakeUint64(uint64(v))
case uint64:
return starlark.MakeUint64(v)
case uintptr:
return starlark.MakeUint64(uint64(v))
case uint:
return starlark.MakeUint64(uint64(v))
case int8:
return starlark.MakeInt64(int64(v))
case int16:
return starlark.MakeInt64(int64(v))
case int32:
return starlark.MakeInt64(int64(v))
case int64:
return starlark.MakeInt64(v)
case int:
return starlark.MakeInt64(int64(v))
case string:
return starlark.String(v)
case map[string]uint64:
// this is the only map type that we use in the api, if we ever want to
// add more maps to the api a more general approach will be necessary.
var r starlark.Dict
for k, v := range v {
r.SetKey(starlark.String(k), starlark.MakeUint64(v))
}
return &r
case nil:
return starlark.None
case error:
return starlark.String(v.Error())
default:
vval := reflect.ValueOf(v)
switch vval.Type().Kind() {
case reflect.Ptr:
if vval.IsNil() {
return starlark.None
}
vval = vval.Elem()
if vval.Type().Kind() == reflect.Struct {
return structAsStarlarkValue{vval, env}
}
case reflect.Struct:
return structAsStarlarkValue{vval, env}
case reflect.Slice:
return sliceAsStarlarkValue{vval, env}
}
return starlark.String(fmt.Sprintf("%v", v))
}
}
// sliceAsStarlarkValue converts a reflect.Value containing a slice
// into a starlark value.
// The public methods of sliceAsStarlarkValue implement the Indexable and
// Sequence starlark interfaces.
type sliceAsStarlarkValue struct {
v reflect.Value
env *Env
}
var _ starlark.Indexable = sliceAsStarlarkValue{}
var _ starlark.Sequence = sliceAsStarlarkValue{}
func (v sliceAsStarlarkValue) Freeze() {
}
func (v sliceAsStarlarkValue) Hash() (uint32, error) {
return 0, fmt.Errorf("not hashable")
}
func (v sliceAsStarlarkValue) String() string {
return fmt.Sprintf("%#v", v.v)
}
func (v sliceAsStarlarkValue) Truth() starlark.Bool {
return v.v.Len() != 0
}
func (v sliceAsStarlarkValue) Type() string {
return v.v.Type().String()
}
func (v sliceAsStarlarkValue) Index(i int) starlark.Value {
if i >= v.v.Len() {
return nil
}
return v.env.interfaceToStarlarkValue(v.v.Index(i).Interface())
}
func (v sliceAsStarlarkValue) Len() int {
return v.v.Len()
}
func (v sliceAsStarlarkValue) Iterate() starlark.Iterator {
return &sliceAsStarlarkValueIterator{0, v.v, v.env}
}
type sliceAsStarlarkValueIterator struct {
cur int
v reflect.Value
env *Env
}
func (it *sliceAsStarlarkValueIterator) Done() {
}
func (it *sliceAsStarlarkValueIterator) Next(p *starlark.Value) bool {
if it.cur >= it.v.Len() {
return false
}
*p = it.env.interfaceToStarlarkValue(it.v.Index(it.cur).Interface())
it.cur++
return true
}
// structAsStarlarkValue converts any Go struct into a starlark.Value.
// The public methods of structAsStarlarkValue implement the
// starlark.HasAttrs interface.
type structAsStarlarkValue struct {
v reflect.Value
env *Env
}
var _ starlark.HasAttrs = structAsStarlarkValue{}
func (v structAsStarlarkValue) Freeze() {
}
func (v structAsStarlarkValue) Hash() (uint32, error) {
return 0, fmt.Errorf("not hashable")
}
func (v structAsStarlarkValue) String() string {
if vv, ok := v.v.Interface().(api.Variable); ok {
return fmt.Sprintf("Variable<%s>", vv.SinglelineString())
}
return fmt.Sprintf("%#v", v.v)
}
func (v structAsStarlarkValue) Truth() starlark.Bool {
return true
}
func (v structAsStarlarkValue) Type() string {
if vv, ok := v.v.Interface().(api.Variable); ok {
return fmt.Sprintf("Variable<%s>", vv.Type)
}
return v.v.Type().String()
}
func (v structAsStarlarkValue) Attr(name string) (starlark.Value, error) {
if r, err := v.valueAttr(name); err != nil || r != nil {
return r, err
}
r := v.v.FieldByName(name)
if r == (reflect.Value{}) {
return starlark.None, fmt.Errorf("no field named %q in %T", name, v.v.Interface())
}
return v.env.interfaceToStarlarkValue(r.Interface()), nil
}
func (v structAsStarlarkValue) valueAttr(name string) (starlark.Value, error) {
if v.v.Type().Name() != "Variable" || (name != "Value" && name != "Expr") {
return nil, nil
}
v2 := v.v.Interface().(api.Variable)
if name == "Expr" {
return starlark.String(varAddrExpr(&v2)), nil
}
return v.env.variableValueToStarlarkValue(&v2, true)
}
func varAddrExpr(v *api.Variable) string {
return fmt.Sprintf("(*(*%q)(%#x))", v.Type, v.Addr)
}
func (env *Env) variableValueToStarlarkValue(v *api.Variable, top bool) (starlark.Value, error) {
if !top && v.Addr == 0 && v.Value == "" {
return starlark.None, nil
}
switch v.Kind {
case reflect.Struct:
if v.Len != 0 && len(v.Children) == 0 {
return starlark.None, errors.New("value not loaded")
}
return structVariableAsStarlarkValue{v, env}, nil
case reflect.Slice, reflect.Array:
if v.Len != 0 && len(v.Children) == 0 {
return starlark.None, errors.New("value not loaded")
}
return sliceVariableAsStarlarkValue{v, env}, nil
case reflect.Map:
if v.Len != 0 && len(v.Children) == 0 {
return starlark.None, errors.New("value not loaded")
}
return mapVariableAsStarlarkValue{v, env}, nil
case reflect.String:
return starlark.String(v.Value), nil
case reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64, reflect.Int:
n, _ := strconv.ParseInt(v.Value, 0, 64)
return starlark.MakeInt64(n), nil
case reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uint, reflect.Uintptr:
n, _ := strconv.ParseUint(v.Value, 0, 64)
return starlark.MakeUint64(n), nil
case reflect.Float32, reflect.Float64:
switch v.Value {
case "+Inf":
return starlark.Float(math.Inf(+1)), nil
case "-Inf":
return starlark.Float(math.Inf(-1)), nil
case "NaN":
return starlark.Float(math.NaN()), nil
default:
n, _ := strconv.ParseFloat(v.Value, 0)
return starlark.Float(n), nil
}
case reflect.Ptr, reflect.Interface:
if len(v.Children) > 0 {
v.Children[0] = *env.autoLoad(varAddrExpr(&v.Children[0]))
}
return ptrVariableAsStarlarkValue{v, env}, nil
}
return nil, nil
}
func (env *Env) autoLoad(expr string) *api.Variable {
v, err := env.ctx.Client().EvalVariable(api.EvalScope{-1, 0, 0}, expr, autoLoadConfig)
if err != nil {
return &api.Variable{Unreadable: err.Error()}
}
return v
}
func (v structAsStarlarkValue) AttrNames() []string {
typ := v.v.Type()
r := make([]string, 0, typ.NumField()+1)
for i := 0; i < typ.NumField(); i++ {
r = append(r, typ.Field(i).Name)
}
return r
}
// structVariableAsStarlarkValue converts an api.Variable representing a
// struct variable (in the target process) into a starlark.Value.
// The public methods of structVariableAsStarlarkValue implement the
// starlark.HasAttrs and starlark.Mapping interfaces.
type structVariableAsStarlarkValue struct {
v *api.Variable
env *Env
}
var _ starlark.HasAttrs = structVariableAsStarlarkValue{}
var _ starlark.Mapping = structVariableAsStarlarkValue{}
func (v structVariableAsStarlarkValue) Freeze() {
}
func (v structVariableAsStarlarkValue) Hash() (uint32, error) {
return 0, fmt.Errorf("not hashable")
}
func (v structVariableAsStarlarkValue) String() string {
return fmt.Sprintf("%s", v.v.SinglelineString())
}
func (v structVariableAsStarlarkValue) Truth() starlark.Bool {
return true
}
func (v structVariableAsStarlarkValue) Type() string {
return v.v.Type
}
func (v structVariableAsStarlarkValue) Attr(name string) (starlark.Value, error) {
for i := range v.v.Children {
if v.v.Children[i].Name == name {
v2 := v.env.autoLoad(varAddrExpr(&v.v.Children[i]))
return v.env.variableValueToStarlarkValue(v2, false)
}
}
return nil, nil // no such field or method
}
func (v structVariableAsStarlarkValue) AttrNames() []string {
r := make([]string, len(v.v.Children))
for i := range v.v.Children {
r[i] = v.v.Children[i].Name
}
return r
}
func (v structVariableAsStarlarkValue) Get(key starlark.Value) (starlark.Value, bool, error) {
skey, ok := key.(starlark.String)
if !ok {
return starlark.None, false, nil
}
r, err := v.Attr(string(skey))
if r == nil && err == nil {
return starlark.None, false, nil
}
if err != nil {
return starlark.None, false, err
}
return r, true, nil
}
type sliceVariableAsStarlarkValue struct {
v *api.Variable
env *Env
}
var _ starlark.Indexable = sliceVariableAsStarlarkValue{}
var _ starlark.Sequence = sliceVariableAsStarlarkValue{}
func (v sliceVariableAsStarlarkValue) Freeze() {
}
func (v sliceVariableAsStarlarkValue) Hash() (uint32, error) {
return 0, fmt.Errorf("not hashable")
}
func (v sliceVariableAsStarlarkValue) String() string {
return fmt.Sprintf("%s", v.v.SinglelineString())
}
func (v sliceVariableAsStarlarkValue) Truth() starlark.Bool {
return v.v.Len != 0
}
func (v sliceVariableAsStarlarkValue) Type() string {
return v.v.Type
}
func (v sliceVariableAsStarlarkValue) Index(i int) starlark.Value {
if i >= v.Len() {
return nil
}
v2 := v.env.autoLoad(fmt.Sprintf("%s[%d]", varAddrExpr(v.v), i))
r, err := v.env.variableValueToStarlarkValue(v2, false)
if err != nil {
return starlark.String(err.Error())
}
return r
}
func (v sliceVariableAsStarlarkValue) Len() int {
return int(v.v.Len)
}
func (v sliceVariableAsStarlarkValue) Iterate() starlark.Iterator {
return &sliceVariableAsStarlarkValueIterator{0, v.v, v.env}
}
type sliceVariableAsStarlarkValueIterator struct {
cur int64
v *api.Variable
env *Env
}
func (it *sliceVariableAsStarlarkValueIterator) Done() {
}
func (it *sliceVariableAsStarlarkValueIterator) Next(p *starlark.Value) bool {
if it.cur >= it.v.Len {
return false
}
s := sliceVariableAsStarlarkValue{it.v, it.env}
*p = s.Index(int(it.cur))
it.cur++
return true
}
type ptrVariableAsStarlarkValue struct {
v *api.Variable
env *Env
}
var _ starlark.HasAttrs = ptrVariableAsStarlarkValue{}
var _ starlark.Mapping = ptrVariableAsStarlarkValue{}
func (v ptrVariableAsStarlarkValue) Freeze() {
}
func (v ptrVariableAsStarlarkValue) Hash() (uint32, error) {
return 0, fmt.Errorf("not hashable")
}
func (v ptrVariableAsStarlarkValue) String() string {
return fmt.Sprintf("%s", v.v.SinglelineString())
}
func (v ptrVariableAsStarlarkValue) Truth() starlark.Bool {
return true
}
func (v ptrVariableAsStarlarkValue) Type() string {
return v.v.Type
}
func (v ptrVariableAsStarlarkValue) Attr(name string) (starlark.Value, error) {
if len(v.v.Children) == 0 {
return nil, nil // no such field or method
}
if v.v.Children[0].Kind == reflect.Struct {
// autodereference pointers to structs
x := structVariableAsStarlarkValue{&v.v.Children[0], v.env}
return x.Attr(name)
} else if v.v.Kind == reflect.Interface && v.v.Children[0].Kind == reflect.Ptr {
// allow double-autodereference for iface to ptr to struct
vchild := &v.v.Children[0]
if len(vchild.Children) > 0 {
vchild.Children[0] = *v.env.autoLoad(varAddrExpr(&vchild.Children[0]))
}
v2 := ptrVariableAsStarlarkValue{vchild, v.env}
return v2.Attr(name)
}
return nil, nil
}
func (v ptrVariableAsStarlarkValue) AttrNames() []string {
if v.v.Children[0].Kind != reflect.Struct {
return nil
}
// autodereference
x := structVariableAsStarlarkValue{&v.v.Children[0], v.env}
return x.AttrNames()
}
func (v ptrVariableAsStarlarkValue) Get(key starlark.Value) (starlark.Value, bool, error) {
if ikey, ok := key.(starlark.Int); ok {
if len(v.v.Children) == 0 {
return starlark.None, true, nil
}
if idx, _ := ikey.Int64(); idx == 0 {
r, err := v.env.variableValueToStarlarkValue(&v.v.Children[0], false)
if err != nil {
return starlark.String(err.Error()), true, nil
}
return r, true, nil
}
return starlark.None, false, nil
}
if len(v.v.Children) == 0 || v.v.Children[0].Kind != reflect.Struct {
return starlark.None, false, nil
}
// autodereference
x := structVariableAsStarlarkValue{&v.v.Children[0], v.env}
return x.Get(key)
}
type mapVariableAsStarlarkValue struct {
v *api.Variable
env *Env
}
var _ starlark.IterableMapping = mapVariableAsStarlarkValue{}
func (v mapVariableAsStarlarkValue) Freeze() {
}
func (v mapVariableAsStarlarkValue) Hash() (uint32, error) {
return 0, fmt.Errorf("not hashable")
}
func (v mapVariableAsStarlarkValue) String() string {
return fmt.Sprintf("%s", v.v.SinglelineString())
}
func (v mapVariableAsStarlarkValue) Truth() starlark.Bool {
return true
}
func (v mapVariableAsStarlarkValue) Type() string {
return v.v.Type
}
func (v mapVariableAsStarlarkValue) Get(key starlark.Value) (starlark.Value, bool, error) {
var keyExpr string
switch key := key.(type) {
case starlark.Int:
keyExpr = key.String()
case starlark.Float:
keyExpr = fmt.Sprintf("%g", float64(key))
case starlark.String:
keyExpr = fmt.Sprintf("%q", string(key))
case starlark.Bool:
keyExpr = fmt.Sprintf("%v", bool(key))
case structVariableAsStarlarkValue:
keyExpr = varAddrExpr(key.v)
default:
return starlark.None, false, fmt.Errorf("key type not supported %T", key)
}
v2 := v.env.autoLoad(fmt.Sprintf("%s[%s]", varAddrExpr(v.v), keyExpr))
r, err := v.env.variableValueToStarlarkValue(v2, false)
if err != nil {
if err.Error() == "key not found" {
return starlark.None, false, nil
}
return starlark.None, false, err
}
return r, true, nil
}
func (v mapVariableAsStarlarkValue) Items() []starlark.Tuple {
r := make([]starlark.Tuple, 0, len(v.v.Children)/2)
for i := 0; i < len(v.v.Children); i += 2 {
r = append(r, mapStarlarkTupleAt(v.v, v.env, i))
}
return r
}
func mapStarlarkTupleAt(v *api.Variable, env *Env, i int) starlark.Tuple {
keyv := env.autoLoad(varAddrExpr(&v.Children[i]))
key, err := env.variableValueToStarlarkValue(keyv, false)
if err != nil {
key = starlark.None
}
valv := env.autoLoad(varAddrExpr(&v.Children[i+1]))
val, err := env.variableValueToStarlarkValue(valv, false)
if err != nil {
val = starlark.None
}
return starlark.Tuple{key, val}
}
func (v mapVariableAsStarlarkValue) Iterate() starlark.Iterator {
return &mapVariableAsStarlarkValueIterator{0, v.v, v.env}
}
type mapVariableAsStarlarkValueIterator struct {
cur int
v *api.Variable
env *Env
}
func (it *mapVariableAsStarlarkValueIterator) Done() {
}
func (it *mapVariableAsStarlarkValueIterator) Next(p *starlark.Value) bool {
if it.cur >= 2*int(it.v.Len) {
return false
}
if it.cur >= len(it.v.Children) {
v2 := it.env.autoLoad(fmt.Sprintf("%s[%d:]", varAddrExpr(it.v), len(it.v.Children)))
it.v.Children = append(it.v.Children, v2.Children...)
}
if it.cur >= len(it.v.Children) {
return false
}
keyv := it.env.autoLoad(varAddrExpr(&it.v.Children[it.cur]))
key, err := it.env.variableValueToStarlarkValue(keyv, false)
if err != nil {
key = starlark.None
}
*p = key
it.cur += 2
return true
}
// unmarshalStarlarkValue unmarshals a starlark.Value 'val' into a Go variable 'dst'.
// This works similarly to encoding/json.Unmarshal and similar functions,
// but instead of getting its input from a byte buffer, it uses a
// starlark.Value.
func unmarshalStarlarkValue(val starlark.Value, dst interface{}, path string) error {
return unmarshalStarlarkValueIntl(val, reflect.ValueOf(dst), path)
}
func unmarshalStarlarkValueIntl(val starlark.Value, dst reflect.Value, path string) (err error) {
defer func() {
// catches reflect panics
ierr := recover()
if ierr != nil {
err = fmt.Errorf("error setting argument %q to %s: %v", path, val, ierr)
}
}()
converr := func(args ...string) error {
if len(args) > 0 {
return fmt.Errorf("error setting argument %q: can not convert %s to %s: %s", path, val, dst.Type().String(), args[0])
}
return fmt.Errorf("error setting argument %q: can not convert %s to %s", path, val, dst.Type().String())
}
if _, isnone := val.(starlark.NoneType); isnone {
return nil
}
for dst.Kind() == reflect.Ptr {
if dst.IsNil() {
dst.Set(reflect.New(dst.Type().Elem()))
}
dst = dst.Elem()
}
switch val := val.(type) {
case starlark.Bool:
dst.SetBool(bool(val))
case starlark.Int:
switch dst.Kind() {
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uintptr:
n, ok := val.Uint64()
if !ok {
return converr()
}
dst.SetUint(n)
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
n, ok := val.Int64()
if !ok {
return converr()
}
dst.SetInt(n)
default:
return converr()
}
case starlark.Float:
dst.SetFloat(float64(val))
case starlark.String:
dst.SetString(string(val))
case *starlark.List:
if dst.Kind() != reflect.Slice {
return converr()
}
r := []reflect.Value{}
for i := 0; i < val.Len(); i++ {
cur := reflect.New(dst.Type().Elem())
err := unmarshalStarlarkValueIntl(val.Index(i), cur, path)
if err != nil {
return err
}
r = append(r, cur)
}
case *starlark.Dict:
if dst.Kind() != reflect.Struct {
return converr()
}
for _, k := range val.Keys() {
if _, ok := k.(starlark.String); !ok {
return converr(fmt.Sprintf("non-string key %q", k.String()))
}
fieldName := string(k.(starlark.String))
dstfield := dst.FieldByName(fieldName)
if dstfield == (reflect.Value{}) {
return converr(fmt.Sprintf("unknown field %s", fieldName))
}
valfield, _, _ := val.Get(starlark.String(fieldName))
err := unmarshalStarlarkValueIntl(valfield, dstfield, path+"."+fieldName)
if err != nil {
return err
}
}
case structAsStarlarkValue:
rv := val.v
if rv.Kind() == reflect.Ptr {
rv = rv.Elem()
}
dst.Set(rv)
default:
return converr()
}
return nil
}

203
pkg/terminal/starbind/repl.go Normal file
View File

@ -0,0 +1,203 @@
package starbind
// Code in this file is derived from go.starlark.net/repl/repl.go
// Which is licensed under the following copyright:
//
// Copyright (c) 2017 The Bazel Authors. All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// 1. Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
//
// 2. Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the
// distribution.
//
// 3. Neither the name of the copyright holder nor the names of its
// contributors may be used to endorse or promote products derived
// from this software without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
import (
"fmt"
"io"
"os"
"go.starlark.net/starlark"
"go.starlark.net/syntax"
"github.com/peterh/liner"
)
// REPL executes a read, eval, print loop.
func (env *Env) REPL() error {
thread := env.newThread()
globals := starlark.StringDict{}
for k, v := range env.env {
globals[k] = v
}
rl := liner.NewLiner()
defer rl.Close()
for {
if err := isCancelled(thread); err != nil {
return err
}
if err := rep(rl, thread, globals); err != nil {
if err == io.EOF {
break
}
return err
}
}
fmt.Println()
return env.exportGlobals(globals)
}
const (
normalPrompt = ">>> "
extraPrompt = "... "
exitCommand = "exit"
)
// rep reads, evaluates, and prints one item.
//
// It returns an error (possibly readline.ErrInterrupt)
// only if readline failed. Starlark errors are printed.
func rep(rl *liner.State, thread *starlark.Thread, globals starlark.StringDict) error {
eof := false
prompt := normalPrompt
readline := func() ([]byte, error) {
line, err := rl.Prompt(prompt)
if line == exitCommand {
eof = true
return nil, io.EOF
}
rl.AppendHistory(line)
prompt = extraPrompt
if err != nil {
if err == io.EOF {
eof = true
}
return nil, err
}
return []byte(line + "\n"), nil
}
// parse
f, err := syntax.ParseCompoundStmt("<stdin>", readline)
if err != nil {
if eof {
return io.EOF
}
printError(err)
return nil
}
if expr := soleExpr(f); expr != nil {
//TODO: check for 'exit'
// eval
v, err := starlark.EvalExpr(thread, expr, globals)
if err != nil {
printError(err)
return nil
}
// print
if v != starlark.None {
fmt.Println(v)
}
} else {
// compile
prog, err := starlark.FileProgram(f, globals.Has)
if err != nil {
printError(err)
return nil
}
// execute (but do not freeze)
res, err := prog.Init(thread, globals)
if err != nil {
printError(err)
}
// The global names from the previous call become
// the predeclared names of this call.
// If execution failed, some globals may be undefined.
for k, v := range res {
globals[k] = v
}
}
return nil
}
func soleExpr(f *syntax.File) syntax.Expr {
if len(f.Stmts) == 1 {
if stmt, ok := f.Stmts[0].(*syntax.ExprStmt); ok {
return stmt.X
}
}
return nil
}
// PrintError prints the error to stderr,
// or its backtrace if it is a Starlark evaluation error.
func printError(err error) {
if evalErr, ok := err.(*starlark.EvalError); ok {
fmt.Fprintln(os.Stderr, evalErr.Backtrace())
} else {
fmt.Fprintln(os.Stderr, err)
}
}
// MakeLoad returns a simple sequential implementation of module loading
// suitable for use in the REPL.
// Each function returned by MakeLoad accesses a distinct private cache.
func MakeLoad() func(thread *starlark.Thread, module string) (starlark.StringDict, error) {
type entry struct {
globals starlark.StringDict
err error
}
var cache = make(map[string]*entry)
return func(thread *starlark.Thread, module string) (starlark.StringDict, error) {
e, ok := cache[module]
if e == nil {
if ok {
// request for package whose loading is in progress
return nil, fmt.Errorf("cycle in load graph")
}
// Add a placeholder to indicate "load in progress".
cache[module] = nil
// Load it.
thread := &starlark.Thread{Name: "exec " + module, Load: thread.Load}
globals, err := starlark.ExecFile(thread, module, nil, nil)
e = &entry{globals, err}
// Update the cache.
cache[module] = e
}
return e.globals, e.err
}
}

297
pkg/terminal/starbind/starlark.go Normal file
View File

@ -0,0 +1,297 @@
package starbind
import (
"context"
"fmt"
"io/ioutil"
"runtime"
"strings"
"sync"
"go.starlark.net/resolve"
"go.starlark.net/starlark"
"github.com/go-delve/delve/service"
"github.com/go-delve/delve/service/api"
)
//go:generate go run ../../../scripts/gen-starlark-bindings.go go ./starlark_mapping.go
//go:generate go run ../../../scripts/gen-starlark-bindings.go doc ../../../Documentation/cli/starlark.md
const (
dlvCommandBuiltinName = "dlv_command"
readFileBuiltinName = "read_file"
writeFileBuiltinName = "write_file"
commandPrefix = "command_"
dlvContextName = "dlv_context"
curScopeBuiltinName = "cur_scope"
defaultLoadConfigBuiltinName = "default_load_config"
)
func init() {
resolve.AllowNestedDef = true
resolve.AllowLambda = true
resolve.AllowFloat = true
resolve.AllowSet = true
resolve.AllowBitwise = true
resolve.AllowRecursion = true
resolve.AllowGlobalReassign = true
}
// Context is the context in which starlark scripts are evaluated.
// It contains methods to call API functions, command line commands, etc.
type Context interface {
Client() service.Client
RegisterCommand(name, helpMsg string, cmdfn func(args string) error)
CallCommand(cmdstr string) error
Scope() api.EvalScope
LoadConfig() api.LoadConfig
}
// Env is the environment used to evaluate starlark scripts.
type Env struct {
env starlark.StringDict
contextMu sync.Mutex
cancelfn context.CancelFunc
ctx Context
}
// New creates a new starlark binding environment.
func New(ctx Context) *Env {
env := &Env{}
env.ctx = ctx
env.env = env.starlarkPredeclare()
env.env[dlvCommandBuiltinName] = starlark.NewBuiltin(dlvCommandBuiltinName, func(thread *starlark.Thread, _ *starlark.Builtin, args starlark.Tuple, kwargs []starlark.Tuple) (starlark.Value, error) {
if err := isCancelled(thread); err != nil {
return starlark.None, err
}
argstrs := make([]string, len(args))
for i := range args {
a, ok := args[i].(starlark.String)
if !ok {
return nil, fmt.Errorf("argument of dlv_command is not a string")
}
argstrs[i] = string(a)
}
err := env.ctx.CallCommand(strings.Join(argstrs, " "))
if err != nil && strings.Contains(err.Error(), " has exited with status ") {
return env.interfaceToStarlarkValue(err), nil
}
return starlark.None, decorateError(thread, err)
})
env.env[readFileBuiltinName] = starlark.NewBuiltin(readFileBuiltinName, func(thread *starlark.Thread, _ *starlark.Builtin, args starlark.Tuple, kwargs []starlark.Tuple) (starlark.Value, error) {
if len(args) != 1 {
return nil, decorateError(thread, fmt.Errorf("wrong number of arguments"))
}
path, ok := args[0].(starlark.String)
if !ok {
return nil, decorateError(thread, fmt.Errorf("argument of read_file was not a string"))
}
buf, err := ioutil.ReadFile(string(path))
if err != nil {
return nil, decorateError(thread, err)
}
return starlark.String(string(buf)), nil
})
env.env[writeFileBuiltinName] = starlark.NewBuiltin(writeFileBuiltinName, func(thread *starlark.Thread, _ *starlark.Builtin, args starlark.Tuple, kwargs []starlark.Tuple) (starlark.Value, error) {
if len(args) != 2 {
return nil, decorateError(thread, fmt.Errorf("wrong number of arguments"))
}
path, ok := args[0].(starlark.String)
if !ok {
return nil, decorateError(thread, fmt.Errorf("first argument of write_file was not a string"))
}
err := ioutil.WriteFile(string(path), []byte(args[1].String()), 0640)
return starlark.None, decorateError(thread, err)
})
env.env[curScopeBuiltinName] = starlark.NewBuiltin(curScopeBuiltinName, func(_ *starlark.Thread, _ *starlark.Builtin, args starlark.Tuple, kwargs []starlark.Tuple) (starlark.Value, error) {
return env.interfaceToStarlarkValue(env.ctx.Scope()), nil
})
env.env[defaultLoadConfigBuiltinName] = starlark.NewBuiltin(defaultLoadConfigBuiltinName, func(_ *starlark.Thread, _ *starlark.Builtin, args starlark.Tuple, kwargs []starlark.Tuple) (starlark.Value, error) {
return env.interfaceToStarlarkValue(env.ctx.LoadConfig()), nil
})
return env
}
// Execute executes a script. Path is the name of the file to execute and
// source is the source code to execute.
// Source can be either a []byte, a string or a io.Reader. If source is nil
// Execute will execute the file specified by 'path'.
// After the file is executed if a function named mainFnName exists it will be called, passing args to it.
func (env *Env) Execute(path string, source interface{}, mainFnName string, args []interface{}) (starlark.Value, error) {
defer func() {
err := recover()
if err == nil {
return
}
fmt.Printf("panic executing starlark script: %v\n", err)
for i := 0; ; i++ {
pc, file, line, ok := runtime.Caller(i)
if !ok {
break
}
fname := "<unknown>"
fn := runtime.FuncForPC(pc)
if fn != nil {
fname = fn.Name()
}
fmt.Printf("%s\n\tin %s:%d\n", fname, file, line)
}
}()
thread := env.newThread()
globals, err := starlark.ExecFile(thread, path, source, env.env)
if err != nil {
return starlark.None, err
}
err = env.exportGlobals(globals)
if err != nil {
return starlark.None, err
}
return env.callMain(thread, globals, mainFnName, args)
}
// exportGlobals saves globals with a name starting with a capital letter
// into the environment and creates commands from globals with a name
// starting with "command_"
func (env *Env) exportGlobals(globals starlark.StringDict) error {
for name, val := range globals {
switch {
case strings.HasPrefix(name, commandPrefix):
err := env.createCommand(name, val)
if err != nil {
return err
}
case name[0] >= 'A' && name[0] <= 'Z':
env.env[name] = val
}
}
return nil
}
// Cancel cancels the execution of a currently running script or function.
func (env *Env) Cancel() {
if env == nil {
return
}
env.contextMu.Lock()
if env.cancelfn != nil {
env.cancelfn()
env.cancelfn = nil
}
env.contextMu.Unlock()
}
func (env *Env) newThread() *starlark.Thread {
thread := &starlark.Thread{
Print: func(_ *starlark.Thread, msg string) { fmt.Println(msg) },
}
env.contextMu.Lock()
var ctx context.Context
ctx, env.cancelfn = context.WithCancel(context.Background())
env.contextMu.Unlock()
thread.SetLocal(dlvContextName, ctx)
return thread
}
func (env *Env) createCommand(name string, val starlark.Value) error {
fnval, ok := val.(*starlark.Function)
if !ok {
return nil
}
name = name[len(commandPrefix):]
helpMsg := fnval.Doc()
if helpMsg == "" {
helpMsg = "user defined"
}
if fnval.NumParams() == 1 {
if p0, _ := fnval.Param(0); p0 == "args" {
env.ctx.RegisterCommand(name, helpMsg, func(args string) error {
_, err := starlark.Call(env.newThread(), fnval, starlark.Tuple{starlark.String(args)}, nil)
return err
})
return nil
}
}
env.ctx.RegisterCommand(name, helpMsg, func(args string) error {
thread := env.newThread()
argval, err := starlark.Eval(thread, "<input>", "("+args+")", env.env)
if err != nil {
return err
}
argtuple, ok := argval.(starlark.Tuple)
if !ok {
argtuple = starlark.Tuple{argval}
}
_, err = starlark.Call(thread, fnval, argtuple, nil)
return err
})
return nil
}
// callMain calls the main function in globals, if one was defined.
func (env *Env) callMain(thread *starlark.Thread, globals starlark.StringDict, mainFnName string, args []interface{}) (starlark.Value, error) {
if mainFnName == "" {
return starlark.None, nil
}
mainval := globals[mainFnName]
if mainval == nil {
return starlark.None, nil
}
mainfn, ok := mainval.(*starlark.Function)
if !ok {
return starlark.None, fmt.Errorf("%s is not a function", mainFnName)
}
if mainfn.NumParams() != len(args) {
return starlark.None, fmt.Errorf("wrong number of arguments for %s", mainFnName)
}
argtuple := make(starlark.Tuple, len(args))
for i := range args {
argtuple[i] = env.interfaceToStarlarkValue(args[i])
}
return starlark.Call(thread, mainfn, argtuple, nil)
}
type argument struct {
name string
defaultValue defaultValue
}
type defaultValue uint8
const (
defaultNone = iota
defaultScope
defaultLoadConfig
)
func isCancelled(thread *starlark.Thread) error {
if ctx, ok := thread.Local(dlvContextName).(context.Context); ok {
select {
case <-ctx.Done():
return ctx.Err()
default:
}
}
return nil
}
func decorateError(thread *starlark.Thread, err error) error {
if err == nil {
return nil
}
pos := thread.Caller().Position()
if pos.Col > 0 {
return fmt.Errorf("%s:%d:%d: %v", pos.Filename(), pos.Line, pos.Col, err)
}
return fmt.Errorf("%s:%d: %v", pos.Filename(), pos.Line, err)
}

1164
pkg/terminal/starbind/starlark_mapping.go Normal file
View File

File diff suppressed because it is too large Load Diff

59
pkg/terminal/starlark.go Normal file
View File

@ -0,0 +1,59 @@
package terminal
import (
"github.com/go-delve/delve/pkg/terminal/starbind"
"github.com/go-delve/delve/service"
"github.com/go-delve/delve/service/api"
)
type starlarkContext struct {
term *Term
}
var _ starbind.Context = starlarkContext{}
func (ctx starlarkContext) Client() service.Client {
return ctx.term.client
}
func (ctx starlarkContext) RegisterCommand(name, helpMsg string, fn func(args string) error) {
cmdfn := func(t *Term, ctx callContext, args string) error {
return fn(args)
}
found := false
for i := range ctx.term.cmds.cmds {
cmd := &ctx.term.cmds.cmds[i]
for _, alias := range cmd.aliases {
if alias == name {
cmd.cmdFn = cmdfn
cmd.helpMsg = helpMsg
found = true
break
}
}
if found {
break
}
}
if !found {
newcmd := command{
aliases: []string{name},
helpMsg: helpMsg,
cmdFn: cmdfn,
}
ctx.term.cmds.cmds = append(ctx.term.cmds.cmds, newcmd)
}
}
func (ctx starlarkContext) CallCommand(cmdstr string) error {
return ctx.term.cmds.Call(cmdstr, ctx.term)
}
func (ctx starlarkContext) Scope() api.EvalScope {
return api.EvalScope{-1, ctx.term.cmds.frame, 0}
}
func (ctx starlarkContext) LoadConfig() api.LoadConfig {
return ctx.term.loadConfig()
}

157
pkg/terminal/starlark_test.go Normal file
View File

@ -0,0 +1,157 @@
package terminal
import (
"fmt"
"strings"
"testing"
)
func TestStarlarkExamples(t *testing.T) {
withTestTerminal("testvariables2", t, func(term *FakeTerminal) {
term.MustExec("continue")
t.Run("goroutine_start_line", func(t *testing.T) { testStarlarkExampleGoroutineStartLine(t, term) })
t.Run("create_breakpoint_main", func(t *testing.T) { testStarlarkExampleCreateBreakpointmain(t, term) })
t.Run("switch_to_main_goroutine", func(t *testing.T) { testStarlarkExampleSwitchToMainGoroutine(t, term) })
t.Run("linked_list", func(t *testing.T) { testStarlarkExampleLinkedList(t, term) })
t.Run("echo_expr", func(t *testing.T) { testStarlarkEchoExpr(t, term) })
t.Run("find_array", func(t *testing.T) { testStarlarkFindArray(t, term) })
t.Run("map_iteartion", func(t *testing.T) { testStarlarkMapIteration(t, term) })
})
}
func testStarlarkExampleGoroutineStartLine(t *testing.T, term *FakeTerminal) {
term.MustExec("source " + findStarFile("goroutine_start_line"))
out1 := term.MustExec("gsl")
t.Logf("gsl: %q", out1)
if !strings.Contains(out1, "func main() {") {
t.Fatalf("goroutine_start_line example failed")
}
}
func testStarlarkExampleCreateBreakpointmain(t *testing.T, term *FakeTerminal) {
out2p1 := term.MustExec("source " + findStarFile("create_breakpoint_main"))
t.Logf("create_breakpoint_main: %s", out2p1)
out2p2 := term.MustExec("breakpoints")
t.Logf("breakpoints: %q", out2p2)
if !strings.Contains(out2p2, "main.afunc1") {
t.Fatalf("create_breakpoint_runtime_func example failed")
}
}
func testStarlarkExampleSwitchToMainGoroutine(t *testing.T, term *FakeTerminal) {
gs, _, err := term.client.ListGoroutines(0, 0)
if err != nil {
t.Fatal(err)
}
for _, g := range gs {
if g.ID != 1 {
t.Logf("switching to goroutine %d\n", g.ID)
term.MustExec(fmt.Sprintf("goroutine %d", g.ID))
break
}
}
term.MustExec("source " + findStarFile("switch_to_main_goroutine"))
out3p1 := term.MustExec("switch_to_main_goroutine")
out3p2 := term.MustExec("goroutine")
t.Logf("%q\n", out3p1)
t.Logf("%q\n", out3p2)
if !strings.Contains(out3p2, "Goroutine 1:\n") {
t.Fatalf("switch_to_main_goroutine example failed")
}
}
func testStarlarkExampleLinkedList(t *testing.T, term *FakeTerminal) {
term.MustExec("source " + findStarFile("linked_list"))
out := term.MustExec("linked_list ll Next 3")
t.Logf("%q\n", out)
if n := len(strings.Split(strings.TrimRight(out, "\n"), "\n")); n != 3 {
t.Fatalf("wrong number of lines in output %d", n)
}
out = term.MustExec("linked_list ll Next 100")
t.Logf("%q\n", out)
lines := strings.Split(strings.TrimRight(out, "\n"), "\n")
for i, line := range lines {
if i == 5 {
if line != "5: *main.List nil" {
t.Errorf("mismatched line %d %q", i, line)
}
} else {
if !strings.HasPrefix(line, fmt.Sprintf("%d: *main.List {N: %d, Next: ", i, i)) {
t.Errorf("mismatched line %d %q", i, line)
}
}
}
if len(lines) != 6 {
t.Fatalf("wrong number of output lines %d", len(lines))
}
}
func testStarlarkEchoExpr(t *testing.T, term *FakeTerminal) {
term.MustExec("source " + findStarFile("echo_expr"))
out := term.MustExec("echo_expr 2+2, 1-1, 2*3")
t.Logf("echo_expr %q", out)
if out != "a 4 b 0 c 6\n" {
t.Error("output mismatch")
}
}
func testStarlarkFindArray(t *testing.T, term *FakeTerminal) {
term.MustExec("source " + findStarFile("find_array"))
out := term.MustExec(`find_array "s2", lambda x: x.A == 5`)
t.Logf("find_array (1) %q", out)
if out != "found 2\n" {
t.Error("output mismatch")
}
out = term.MustExec(`find_array "s2", lambda x: x.A == 20`)
t.Logf("find_array (2) %q", out)
if out != "not found\n" {
t.Error("output mismatch")
}
}
func testStarlarkMapIteration(t *testing.T, term *FakeTerminal) {
out := term.MustExec("source " + findStarFile("starlark_map_iteration"))
t.Logf("%s", out)
}
func TestStarlarkVariable(t *testing.T) {
withTestTerminal("testvariables2", t, func(term *FakeTerminal) {
term.MustExec("continue")
for _, tc := range []struct{ expr, tgt string }{
{`v = eval(None, "i1").Variable; print(v.Value)`, "1"},
{`v = eval(None, "f1").Variable; print(v.Value)`, "3"},
{`v = eval(None, "as1").Variable; print(v.Value.A)`, "1"},
{`v = eval(None, "as1").Variable; print(v.Value.B)`, "1"},
{`v = eval(None, "as1").Variable; print(v.Value["A"])`, "1"},
{`v = eval(None, "s1").Variable; print(v.Value[0])`, "one"},
{`v = eval(None, "nilstruct").Variable; print(v.Value)`, "*main.astruct nil"},
{`v = eval(None, "nilstruct").Variable; print(v.Value[0])`, "None"},
{`v = eval(None, 'm1["Malone"]').Variable; print(v.Value)`, "main.astruct {A: 2, B: 3}"},
{`v = eval(None, "m1").Variable; print(v.Value["Malone"])`, "main.astruct {A: 2, B: 3}"},
{`v = eval(None, "m2").Variable; print(v.Value[1])`, "*main.astruct {A: 10, B: 11}"},
{`v = eval(None, "c1.pb").Variable; print(v.Value)`, "*main.bstruct {a: main.astruct {A: 1, B: 2}}"},
{`v = eval(None, "c1.pb").Variable; print(v.Value[0])`, "main.bstruct {a: main.astruct {A: 1, B: 2}}"},
{`v = eval(None, "c1.pb").Variable; print(v.Value.a.B)`, "2"},
{`v = eval(None, "iface1").Variable; print(v.Value[0])`, "*main.astruct {A: 1, B: 2}"},
{`v = eval(None, "iface1").Variable; print(v.Value[0][0])`, "main.astruct {A: 1, B: 2}"},
{`v = eval(None, "iface1").Variable; print(v.Value.A)`, "1"},
{`v = eval(None, "as1", {"MaxStructFields": -1}).Variable; print(v.Value.A)`, "1"},
{`v = eval({"GoroutineID": -1}, "as1").Variable; print(v.Value.A)`, "1"},
{`v = eval(cur_scope(), "as1").Variable; print(v.Value.A)`, "1"},
{`v = eval(None, "as1", default_load_config()).Variable; print(v.Value.A)`, "1"},
// From starlark.md's examples
{`v = eval(None, "s2").Variable; print(v.Value[0])`, "main.astruct {A: 1, B: 2}"},
{`v = eval(None, "s2").Variable; print(v.Value[1].A)`, "3"},
{`v = eval(None, "s2").Variable; print(v.Value[1].A + 10)`, "13"},
{`v = eval(None, "s2").Variable; print(v.Value[1]["B"])`, "4"},
} {
out := strings.TrimSpace(term.MustExecStarlark(tc.expr))
if out != tc.tgt {
t.Errorf("for %q\nexpected %q\ngot %q", tc.expr, tc.tgt, out)
}
}
})
}

10
pkg/terminal/terminal.go
View File

@ -8,12 +8,12 @@ import (
"runtime"
"strings"
"sync"
"syscall"
"github.com/peterh/liner"
"github.com/go-delve/delve/pkg/config"
"github.com/go-delve/delve/pkg/terminal/starbind"
"github.com/go-delve/delve/service"
"github.com/go-delve/delve/service/api"
)
@ -54,6 +54,8 @@ type Term struct {
stdout io.Writer
InitFile string
starlarkEnv *starbind.Env
// quitContinue is set to true by exitCommand to signal that the process
// should be resumed before quitting.
quitContinue bool
@ -93,7 +95,7 @@ func New(client service.Client, conf *config.Config) *Term {
conf.SourceListLineColor = ansiBlue
}
return &Term{
t := &Term{
client: client,
conf: conf,
prompt: "(dlv) ",
@ -102,6 +104,9 @@ func New(client service.Client, conf *config.Config) *Term {
dumb: dumb,
stdout: w,
}
t.starlarkEnv = starbind.New(starlarkContext{t})
return t
}
// Close returns the terminal to its previous mode.
@ -111,6 +116,7 @@ func (t *Term) Close() {
func (t *Term) sigintGuard(ch <-chan os.Signal, multiClient bool) {
for range ch {
t.starlarkEnv.Cancel()
if multiClient {
answer, err := t.line.Prompt("Would you like to [s]top the target or [q]uit this client, leaving the target running [s/q]? ")
if err != nil {

335
scripts/gen-starlark-bindings.go Normal file
View File

@ -0,0 +1,335 @@
package main
import (
"bytes"
"fmt"
"go/format"
"go/token"
"go/types"
"io/ioutil"
"log"
"os"
"strings"
"unicode"
"golang.org/x/tools/go/packages"
)
// getSuitableMethods returns the list of methods of service/rpc2.RPCServer that are exported as API calls
func getSuitableMethods(pkg *types.Package, typename string) []*types.Func {
r := []*types.Func{}
mset := types.NewMethodSet(types.NewPointer(pkg.Scope().Lookup(typename).Type()))
for i := 0; i < mset.Len(); i++ {
fn := mset.At(i).Obj().(*types.Func)
if !fn.Exported() {
continue
}
if fn.Name() == "Command" {
r = append(r, fn)
continue
}
sig, ok := fn.Type().(*types.Signature)
if !ok {
continue
}
// arguments must be (args, *reply)
if sig.Params().Len() != 2 {
continue
}
if ntyp, isname := sig.Params().At(0).Type().(*types.Named); !isname {
continue
} else if _, isstr := ntyp.Underlying().(*types.Struct); !isstr {
continue
}
if _, isptr := sig.Params().At(1).Type().(*types.Pointer); !isptr {
continue
}
// return values must be (error)
if sig.Results().Len() != 1 {
continue
}
if sig.Results().At(0).Type().String() != "error" {
continue
}
r = append(r, fn)
}
return r
}
func fieldsOfStruct(typ types.Type) (fieldNames, fieldTypes []string) {
styp := typ.(*types.Named).Underlying().(*types.Struct)
for i := 0; i < styp.NumFields(); i++ {
fieldNames = append(fieldNames, styp.Field(i).Name())
fieldTypes = append(fieldTypes, styp.Field(i).Type().String())
}
return fieldNames, fieldTypes
}
func camelToDash(in string) string {
out := []rune{}
for i, ch := range in {
if ch < 'A' || ch > 'Z' {
out = append(out, ch)
continue
}
if i != 0 {
out = append(out, '_')
}
out = append(out, unicode.ToLower(ch))
}
return string(out)
}
type binding struct {
name string
fn *types.Func
argType, retType string
argNames []string
argTypes []string
}
func processServerMethods(serverMethods []*types.Func) []binding {
bindings := make([]binding, len(serverMethods))
for i, fn := range serverMethods {
sig, _ := fn.Type().(*types.Signature)
argNames, argTypes := fieldsOfStruct(sig.Params().At(0).Type())
name := camelToDash(fn.Name())
switch name {
case "set":
// avoid collision with builtin that already exists in starlark
name = "set_expr"
case "command":
name = "raw_command"
default:
// remove list_ prefix, it looks better
const listPrefix = "list_"
if strings.HasPrefix(name, listPrefix) {
name = name[len(listPrefix):]
}
}
retType := sig.Params().At(1).Type().String()
if fn.Name() == "Command" {
retType = "rpc2.CommandOut"
}
bindings[i] = binding{
name: name,
fn: fn,
argType: sig.Params().At(0).Type().String(),
retType: retType,
argNames: argNames,
argTypes: argTypes,
}
}
return bindings
}
func removePackagePath(typePath string) string {
lastSlash := strings.LastIndex(typePath, "/")
if lastSlash < 0 {
return typePath
}
return typePath[lastSlash+1:]
}
func genMapping(bindings []binding) []byte {
buf := bytes.NewBuffer([]byte{})
fmt.Fprintf(buf, "// DO NOT EDIT: auto-generated using scripts/gen-starlark-bindings.go\n\n")
fmt.Fprintf(buf, "package starbind\n\n")
fmt.Fprintf(buf, "import ( \"go.starlark.net/starlark\" \n \"github.com/go-delve/delve/service/api\" \n \"github.com/go-delve/delve/service/rpc2\" \n \"fmt\" )\n\n")
fmt.Fprintf(buf, "func (env *Env) starlarkPredeclare() starlark.StringDict {\n")
fmt.Fprintf(buf, "r := starlark.StringDict{}\n\n")
for _, binding := range bindings {
fmt.Fprintf(buf, "r[%q] = starlark.NewBuiltin(%q, func(thread *starlark.Thread, _ *starlark.Builtin, args starlark.Tuple, kwargs []starlark.Tuple) (starlark.Value, error) {", binding.name, binding.name)
fmt.Fprintf(buf, "if err := isCancelled(thread); err != nil { return starlark.None, decorateError(thread, err) }\n")
fmt.Fprintf(buf, "var rpcArgs %s\n", removePackagePath(binding.argType))
fmt.Fprintf(buf, "var rpcRet %s\n", removePackagePath(binding.retType))
// unmarshal normal unnamed arguments
for i := range binding.argNames {
fmt.Fprintf(buf, "if len(args) > %d && args[%d] != starlark.None { err := unmarshalStarlarkValue(args[%d], &rpcArgs.%s, %q); if err != nil { return starlark.None, decorateError(thread, err) } }", i, i, i, binding.argNames[i], binding.argNames[i])
switch binding.argTypes[i] {
case "*github.com/go-delve/delve/service/api.LoadConfig":
if binding.fn.Name() != "Stacktrace" {
fmt.Fprintf(buf, "else { cfg := env.ctx.LoadConfig(); rpcArgs.%s = &cfg }", binding.argNames[i])
}
case "github.com/go-delve/delve/service/api.LoadConfig":
fmt.Fprintf(buf, "else { rpcArgs.%s = env.ctx.LoadConfig() }", binding.argNames[i])
case "*github.com/go-delve/delve/service/api.EvalScope":
fmt.Fprintf(buf, "else { scope := env.ctx.Scope(); rpcArgs.%s = &scope }", binding.argNames[i])
case "github.com/go-delve/delve/service/api.EvalScope":
fmt.Fprintf(buf, "else { rpcArgs.%s = env.ctx.Scope() }", binding.argNames[i])
}
fmt.Fprintf(buf, "\n")
}
// unmarshal keyword arguments
if len(binding.argNames) > 0 {
fmt.Fprintf(buf, "for _, kv := range kwargs {\n")
fmt.Fprintf(buf, "var err error\n")
fmt.Fprintf(buf, "switch kv[0].(starlark.String) {\n")
for i := range binding.argNames {
fmt.Fprintf(buf, "case %q: ", binding.argNames[i])
fmt.Fprintf(buf, "err = unmarshalStarlarkValue(kv[1], &rpcArgs.%s, %q)\n", binding.argNames[i], binding.argNames[i])
}
fmt.Fprintf(buf, "default: err = fmt.Errorf(\"unknown argument %%q\", kv[0])")
fmt.Fprintf(buf, "}\n")
fmt.Fprintf(buf, "if err != nil { return starlark.None, decorateError(thread, err) }\n")
fmt.Fprintf(buf, "}\n")
}
fmt.Fprintf(buf, "err := env.ctx.Client().CallAPI(%q, &rpcArgs, &rpcRet)\n", binding.fn.Name())
fmt.Fprintf(buf, "if err != nil { return starlark.None, err }\n")
fmt.Fprintf(buf, "return env.interfaceToStarlarkValue(rpcRet), nil\n")
fmt.Fprintf(buf, "})\n")
}
fmt.Fprintf(buf, "return r\n")
fmt.Fprintf(buf, "}\n")
return buf.Bytes()
}
func genDocs(bindings []binding) []byte {
var buf bytes.Buffer
fmt.Fprintf(&buf, "Function | API Call\n")
fmt.Fprintf(&buf, "---------|---------\n")
for _, binding := range bindings {
argNames := strings.Join(binding.argNames, ", ")
fmt.Fprintf(&buf, "%s(%s) | Equivalent to API call [%s](https://godoc.org/github.com/go-delve/delve/service/rpc2#RPCServer.%s)\n", binding.name, argNames, binding.fn.Name(), binding.fn.Name())
}
fmt.Fprintf(&buf, "dlv_command(command) | Executes the specified command as if typed at the dlv_prompt\n")
fmt.Fprintf(&buf, "read_file(path) | Reads the file as a string\n")
fmt.Fprintf(&buf, "write_file(path, contents) | Writes string to a file\n")
fmt.Fprintf(&buf, "cur_scope() | Returns the current evaluation scope\n")
fmt.Fprintf(&buf, "default_load_config() | Returns the current default load configuration\n")
return buf.Bytes()
}
const (
startOfMappingTable = "<!-- BEGIN MAPPING TABLE -->"
endOfMappingTable = "<!-- END MAPPING TABLE -->"
)
func spliceDocs(docpath string, docs []byte, outpath string) {
docbuf, err := ioutil.ReadFile(docpath)
if err != nil {
log.Fatalf("could not read doc file: %v", err)
}
v := strings.Split(string(docbuf), startOfMappingTable)
if len(v) != 2 {
log.Fatal("could not find start of mapping table")
}
header := v[0]
v = strings.Split(v[1], endOfMappingTable)
if len(v) != 2 {
log.Fatal("could not find end of mapping table")
}
footer := v[1]
outbuf := make([]byte, 0, len(header)+len(docs)+len(footer)+len(startOfMappingTable)+len(endOfMappingTable)+1)
outbuf = append(outbuf, []byte(header)...)
outbuf = append(outbuf, []byte(startOfMappingTable)...)
outbuf = append(outbuf, '\n')
outbuf = append(outbuf, docs...)
outbuf = append(outbuf, []byte(endOfMappingTable)...)
outbuf = append(outbuf, []byte(footer)...)
if outpath != "-" {
err = ioutil.WriteFile(outpath, outbuf, 0664)
if err != nil {
log.Fatalf("could not write documentation file: %v", err)
}
} else {
os.Stdout.Write(outbuf)
}
}
func usage() {
fmt.Fprintf(os.Stderr, "gen-starlark-bindings [doc|doc/dummy|go] <destination file>\n\n")
fmt.Fprintf(os.Stderr, "Writes starlark documentation (doc) or mapping file (go) to <destination file>. Specify doc/dummy to generated documentation without overwriting the destination file.\n")
os.Exit(1)
}
func main() {
if len(os.Args) != 3 {
usage()
}
kind := os.Args[1]
path := os.Args[2]
fset := &token.FileSet{}
cfg := &packages.Config{
Mode: packages.LoadSyntax,
Fset: fset,
}
pkgs, err := packages.Load(cfg, "github.com/go-delve/delve/service/rpc2")
if err != nil {
log.Fatalf("could not load packages: %v", err)
}
var serverMethods []*types.Func
packages.Visit(pkgs, func(pkg *packages.Package) bool {
if pkg.PkgPath == "github.com/go-delve/delve/service/rpc2" {
serverMethods = getSuitableMethods(pkg.Types, "RPCServer")
}
return true
}, nil)
bindings := processServerMethods(serverMethods)
switch kind {
case "go":
mapping := genMapping(bindings)
outfh := os.Stdout
if path != "-" {
outfh, err = os.Create(path)
if err != nil {
log.Fatalf("could not create output file: %v", err)
}
defer outfh.Close()
}
src, err := format.Source(mapping)
if err != nil {
fmt.Fprintf(os.Stderr, "%s", string(mapping))
log.Fatal(err)
}
outfh.Write(src)
case "doc":
docs := genDocs(bindings)
spliceDocs(path, docs, path)
case "doc/dummy":
docs := genDocs(bindings)
spliceDocs(path, docs, "-")
default:
usage()
}
}

3
service/client.go
View File

@ -147,4 +147,7 @@ type Client interface {
// Disconnect closes the connection to the server without sending a Detach request first.
// If cont is true a continue command will be sent instead.
Disconnect(cont bool) error
// CallAPI allows calling an arbitrary rpc method (used by starlark bindings)
CallAPI(method string, args, reply interface{}) error
}

4
service/rpc2/client.go
View File

@ -410,3 +410,7 @@ func (c *RPCClient) ListDynamicLibraries() ([]api.Image, error) {
func (c *RPCClient) call(method string, args, reply interface{}) error {
return c.client.Call("RPCServer."+method, args, reply)
}
func (c *RPCClient) CallAPI(method string, args, reply interface{}) error {
return c.call(method, args, reply)
}

29
vendor/go.starlark.net/LICENSE generated vendored Normal file
View File

@ -0,0 +1,29 @@
Copyright (c) 2017 The Bazel Authors. All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the
distribution.
3. Neither the name of the copyright holder nor the names of its
contributors may be used to endorse or promote products derived
from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

1790
vendor/go.starlark.net/internal/compile/compile.go generated vendored Normal file
View File

File diff suppressed because it is too large Load Diff

373
vendor/go.starlark.net/internal/compile/serial.go generated vendored Normal file
View File

@ -0,0 +1,373 @@
package compile
// This file defines functions to read and write a compile.Program to a file.
//
// It is the client's responsibility to avoid version skew between the
// compiler used to produce a file and the interpreter that consumes it.
// The version number is provided as a constant.
// Incompatible protocol changes should also increment the version number.
//
// Encoding
//
// Program:
// "sky!" [4]byte # magic number
// str uint32le # offset of <strings> section
// version varint # must match Version
// filename string
// numloads varint
// loads []Ident
// numnames varint
// names []string
// numconsts varint
// consts []Constant
// numglobals varint
// globals []Ident
// toplevel Funcode
// numfuncs varint
// funcs []Funcode
// <strings> []byte # concatenation of all referenced strings
// EOF
//
// Funcode:
// id Ident
// code []byte
// pclinetablen varint
// pclinetab []varint
// numlocals varint
// locals []Ident
// numfreevars varint
// freevar []Ident
// maxstack varint
// numparams varint
// numkwonlyparams varint
// hasvarargs varint (0 or 1)
// haskwargs varint (0 or 1)
//
// Ident:
// filename string
// line, col varint
//
// Constant: # type data
// type varint # 0=string string
// data ... # 1=int varint
// # 2=float varint (bits as uint64)
// # 3=bigint string (decimal ASCII text)
//
// The encoding starts with a four-byte magic number.
// The next four bytes are a little-endian uint32
// that provides the offset of the string section
// at the end of the file, which contains the ordered
// concatenation of all strings referenced by the
// program. This design permits the decoder to read
// the first and second parts of the file into different
// memory allocations: the first (the encoded program)
// is transient, but the second (the strings) persists
// for the life of the Program.
//
// Within the encoded program, all strings are referred
// to by their length. As the encoder and decoder process
// the entire file sequentially, they are in lock step,
// so the start offset of each string is implicit.
//
// Program.Code is represented as a []byte slice to permit
// modification when breakpoints are set. All other strings
// are represented as strings. They all (unsafely) share the
// same backing byte slice.
//
// Aside from the str field, all integers are encoded as varints.
import (
"encoding/binary"
"fmt"
"math"
"math/big"
debugpkg "runtime/debug"
"unsafe"
"go.starlark.net/syntax"
)
const magic = "!sky"
// Encode encodes a compiled Starlark program.
func (prog *Program) Encode() []byte {
var e encoder
e.p = append(e.p, magic...)
e.p = append(e.p, "????"...) // string data offset; filled in later
e.int(Version)
e.string(prog.Toplevel.Pos.Filename())
e.idents(prog.Loads)
e.int(len(prog.Names))
for _, name := range prog.Names {
e.string(name)
}
e.int(len(prog.Constants))
for _, c := range prog.Constants {
switch c := c.(type) {
case string:
e.int(0)
e.string(c)
case int64:
e.int(1)
e.int64(c)
case float64:
e.int(2)
e.uint64(math.Float64bits(c))
case *big.Int:
e.int(3)
e.string(c.Text(10))
}
}
e.idents(prog.Globals)
e.function(prog.Toplevel)
e.int(len(prog.Functions))
for _, fn := range prog.Functions {
e.function(fn)
}
// Patch in the offset of the string data section.
binary.LittleEndian.PutUint32(e.p[4:8], uint32(len(e.p)))
return append(e.p, e.s...)
}
type encoder struct {
p []byte // encoded program
s []byte // strings
tmp [binary.MaxVarintLen64]byte
}
func (e *encoder) int(x int) {
e.int64(int64(x))
}
func (e *encoder) int64(x int64) {
n := binary.PutVarint(e.tmp[:], x)
e.p = append(e.p, e.tmp[:n]...)
}
func (e *encoder) uint64(x uint64) {
n := binary.PutUvarint(e.tmp[:], x)
e.p = append(e.p, e.tmp[:n]...)
}
func (e *encoder) string(s string) {
e.int(len(s))
e.s = append(e.s, s...)
}
func (e *encoder) bytes(b []byte) {
e.int(len(b))
e.s = append(e.s, b...)
}
func (e *encoder) ident(id Ident) {
e.string(id.Name)
e.int(int(id.Pos.Line))
e.int(int(id.Pos.Col))
}
func (e *encoder) idents(ids []Ident) {
e.int(len(ids))
for _, id := range ids {
e.ident(id)
}
}
func (e *encoder) function(fn *Funcode) {
e.ident(Ident{fn.Name, fn.Pos})
e.string(fn.Doc)
e.bytes(fn.Code)
e.int(len(fn.pclinetab))
for _, x := range fn.pclinetab {
e.int64(int64(x))
}
e.idents(fn.Locals)
e.idents(fn.Freevars)
e.int(fn.MaxStack)
e.int(fn.NumParams)
e.int(fn.NumKwonlyParams)
e.int(b2i(fn.HasVarargs))
e.int(b2i(fn.HasKwargs))
}
func b2i(b bool) int {
if b {
return 1
} else {
return 0
}
}
// DecodeProgram decodes a compiled Starlark program from data.
func DecodeProgram(data []byte) (_ *Program, err error) {
if len(data) < len(magic) {
return nil, fmt.Errorf("not a compiled module: no magic number")
}
if got := string(data[:4]); got != magic {
return nil, fmt.Errorf("not a compiled module: got magic number %q, want %q",
got, magic)
}
defer func() {
if x := recover(); x != nil {
debugpkg.PrintStack()
err = fmt.Errorf("internal error while decoding program: %v", x)
}
}()
offset := binary.LittleEndian.Uint32(data[4:8])
d := decoder{
p: data[8:offset],
s: append([]byte(nil), data[offset:]...), // allocate a copy, which will persist
}
if v := d.int(); v != Version {
return nil, fmt.Errorf("version mismatch: read %d, want %d", v, Version)
}
filename := d.string()
d.filename = &filename
loads := d.idents()
names := make([]string, d.int())
for i := range names {
names[i] = d.string()
}
// constants
constants := make([]interface{}, d.int())
for i := range constants {
var c interface{}
switch d.int() {
case 0:
c = d.string()
case 1:
c = d.int64()
case 2:
c = math.Float64frombits(d.uint64())
case 3:
c, _ = new(big.Int).SetString(d.string(), 10)
}
constants[i] = c
}
globals := d.idents()
toplevel := d.function()
funcs := make([]*Funcode, d.int())
for i := range funcs {
funcs[i] = d.function()
}
prog := &Program{
Loads: loads,
Names: names,
Constants: constants,
Globals: globals,
Functions: funcs,
Toplevel: toplevel,
}
toplevel.Prog = prog
for _, f := range funcs {
f.Prog = prog
}
if len(d.p)+len(d.s) > 0 {
return nil, fmt.Errorf("internal error: unconsumed data during decoding")
}
return prog, nil
}
type decoder struct {
p []byte // encoded program
s []byte // strings
filename *string // (indirect to avoid keeping decoder live)
}
func (d *decoder) int() int {
return int(d.int64())
}
func (d *decoder) int64() int64 {
x, len := binary.Varint(d.p[:])
d.p = d.p[len:]
return x
}
func (d *decoder) uint64() uint64 {
x, len := binary.Uvarint(d.p[:])
d.p = d.p[len:]
return x
}
func (d *decoder) string() (s string) {
if slice := d.bytes(); len(slice) > 0 {
// Avoid a memory allocation for each string
// by unsafely aliasing slice.
type string struct {
data *byte
len int
}
ptr := (*string)(unsafe.Pointer(&s))
ptr.data = &slice[0]
ptr.len = len(slice)
}
return s
}
func (d *decoder) bytes() []byte {
len := d.int()
r := d.s[:len:len]
d.s = d.s[len:]
return r
}
func (d *decoder) ident() Ident {
name := d.string()
line := int32(d.int())
col := int32(d.int())
return Ident{Name: name, Pos: syntax.MakePosition(d.filename, line, col)}
}
func (d *decoder) idents() []Ident {
idents := make([]Ident, d.int())
for i := range idents {
idents[i] = d.ident()
}
return idents
}
func (d *decoder) bool() bool { return d.int() != 0 }
func (d *decoder) function() *Funcode {
id := d.ident()
doc := d.string()
code := d.bytes()
pclinetab := make([]uint16, d.int())
for i := range pclinetab {
pclinetab[i] = uint16(d.int())
}
locals := d.idents()
freevars := d.idents()
maxStack := d.int()
numParams := d.int()
numKwonlyParams := d.int()
hasVarargs := d.int() != 0
hasKwargs := d.int() != 0
return &Funcode{
// Prog is filled in later.
Pos: id.Pos,
Name: id.Name,
Doc: doc,
Code: code,
pclinetab: pclinetab,
Locals: locals,
Freevars: freevars,
MaxStack: maxStack,
NumParams: numParams,
NumKwonlyParams: numKwonlyParams,
HasVarargs: hasVarargs,
HasKwargs: hasKwargs,
}
}

115
vendor/go.starlark.net/internal/spell/spell.go generated vendored Normal file
View File

@ -0,0 +1,115 @@
// Package spell file defines a simple spelling checker for use in attribute errors
// such as "no such field .foo; did you mean .food?".
package spell
import (
"strings"
"unicode"
)
// Nearest returns the element of candidates
// nearest to x using the Levenshtein metric,
// or "" if none were promising.
func Nearest(x string, candidates []string) string {
// Ignore underscores and case when matching.
fold := func(s string) string {
return strings.Map(func(r rune) rune {
if r == '_' {
return -1
}
return unicode.ToLower(r)
}, s)
}
x = fold(x)
var best string
bestD := (len(x) + 1) / 2 // allow up to 50% typos
for _, c := range candidates {
d := levenshtein(x, fold(c), bestD)
if d < bestD {
bestD = d
best = c
}
}
return best
}
// levenshtein returns the non-negative Levenshtein edit distance
// between the byte strings x and y.
//
// If the computed distance exceeds max,
// the function may return early with an approximate value > max.
func levenshtein(x, y string, max int) int {
// This implementation is derived from one by Laurent Le Brun in
// Bazel that uses the single-row space efficiency trick
// described at bitbucket.org/clearer/iosifovich.
// Let x be the shorter string.
if len(x) > len(y) {
x, y = y, x
}
// Remove common prefix.
for i := 0; i < len(x); i++ {
if x[i] != y[i] {
x = x[i:]
y = y[i:]
break
}
}
if x == "" {
return len(y)
}
if d := abs(len(x) - len(y)); d > max {
return d // excessive length divergence
}
row := make([]int, len(y)+1)
for i := range row {
row[i] = i
}
for i := 1; i <= len(x); i++ {
row[0] = i
best := i
prev := i - 1
for j := 1; j <= len(y); j++ {
a := prev + b2i(x[i-1] != y[j-1]) // substitution
b := 1 + row[j-1] // deletion
c := 1 + row[j] // insertion
k := min(a, min(b, c))
prev, row[j] = row[j], k
best = min(best, k)
}
if best > max {
return best
}
}
return row[len(y)]
}
func b2i(b bool) int {
if b {
return 1
} else {
return 0
}
}
func min(x, y int) int {
if x < y {
return x
} else {
return y
}
}
func abs(x int) int {
if x >= 0 {
return x
} else {
return -x
}
}

922
vendor/go.starlark.net/resolve/resolve.go generated vendored Normal file
View File

@ -0,0 +1,922 @@
// Copyright 2017 The Bazel Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Package resolve defines a name-resolution pass for Starlark abstract
// syntax trees.
//
// The resolver sets the Locals and FreeVars arrays of each DefStmt and
// the LocalIndex field of each syntax.Ident that refers to a local or
// free variable. It also sets the Locals array of a File for locals
// bound by comprehensions outside any function. Identifiers for global
// variables do not get an index.
package resolve // import "go.starlark.net/resolve"
// All references to names are statically resolved. Names may be
// predeclared, global, or local to a function or module-level comprehension.
// The resolver maps each global name to a small integer and each local
// name to a small integer; these integers enable a fast and compact
// representation of globals and locals in the evaluator.
//
// As an optimization, the resolver classifies each predeclared name as
// either universal (e.g. None, len) or per-module (e.g. glob in Bazel's
// build language), enabling the evaluator to share the representation
// of the universal environment across all modules.
//
// The lexical environment is a tree of blocks with the module block at
// its root. The module's child blocks may be of two kinds: functions
// and comprehensions, and these may have further children of either
// kind.
//
// Python-style resolution requires multiple passes because a name is
// determined to be local to a function only if the function contains a
// "binding" use of it; similarly, a name is determined be global (as
// opposed to predeclared) if the module contains a binding use at the
// top level. For both locals and globals, a non-binding use may
// lexically precede the binding to which it is resolved.
// In the first pass, we inspect each function, recording in
// 'uses' each identifier and the environment block in which it occurs.
// If a use of a name is binding, such as a function parameter or
// assignment, we add the name to the block's bindings mapping and add a
// local variable to the enclosing function.
//
// As we finish resolving each function, we inspect all the uses within
// that function and discard ones that were found to be local. The
// remaining ones must be either free (local to some lexically enclosing
// function), or nonlocal (global or predeclared), but we cannot tell
// which until we have finished inspecting the outermost enclosing
// function. At that point, we can distinguish local from global names
// (and this is when Python would compute free variables).
//
// However, Starlark additionally requires that all references to global
// names are satisfied by some declaration in the current module;
// Starlark permits a function to forward-reference a global that has not
// been declared yet so long as it is declared before the end of the
// module. So, instead of re-resolving the unresolved references after
// each top-level function, we defer this until the end of the module
// and ensure that all such references are satisfied by some definition.
//
// At the end of the module, we visit each of the nested function blocks
// in bottom-up order, doing a recursive lexical lookup for each
// unresolved name. If the name is found to be local to some enclosing
// function, we must create a DefStmt.FreeVar (capture) parameter for
// each intervening function. We enter these synthetic bindings into
// the bindings map so that we create at most one freevar per name. If
// the name was not local, we check that it was defined at module level.
//
// We resolve all uses of locals in the module (due to comprehensions)
// in a similar way and compute the set of its local variables.
//
// Starlark enforces that all global names are assigned at most once on
// all control flow paths by forbidding if/else statements and loops at
// top level. A global may be used before it is defined, leading to a
// dynamic error.
//
// TODO(adonovan): opt: reuse local slots once locals go out of scope.
import (
"fmt"
"log"
"sort"
"strings"
"go.starlark.net/internal/spell"
"go.starlark.net/syntax"
)
const debug = false
const doesnt = "this Starlark dialect does not "
// global options
// These features are either not standard Starlark (yet), or deprecated
// features of the BUILD language, so we put them behind flags.
var (
AllowNestedDef = false // allow def statements within function bodies
AllowLambda = false // allow lambda expressions
AllowFloat = false // allow floating point literals, the 'float' built-in, and x / y
AllowSet = false // allow the 'set' built-in
AllowGlobalReassign = false // allow reassignment to globals declared in same file (deprecated)
AllowRecursion = false // allow while statements and recursive functions
AllowBitwise = true // obsolete; bitwise operations (&, |, ^, ~, <<, and >>) are always enabled
)
// File resolves the specified file.
//
// The isPredeclared and isUniversal predicates report whether a name is
// a pre-declared identifier (visible in the current module) or a
// universal identifier (visible in every module).
// Clients should typically pass predeclared.Has for the first and
// starlark.Universe.Has for the second, where predeclared is the
// module's StringDict of predeclared names and starlark.Universe is the
// standard set of built-ins.
// The isUniverse predicate is supplied a parameter to avoid a cyclic
// dependency upon starlark.Universe, not because users should ever need
// to redefine it.
func File(file *syntax.File, isPredeclared, isUniversal func(name string) bool) error {
r := newResolver(isPredeclared, isUniversal)
r.stmts(file.Stmts)
r.env.resolveLocalUses()
// At the end of the module, resolve all non-local variable references,
// computing closures.
// Function bodies may contain forward references to later global declarations.
r.resolveNonLocalUses(r.env)
file.Locals = r.moduleLocals
file.Globals = r.moduleGlobals
if len(r.errors) > 0 {
return r.errors
}
return nil
}
// Expr resolves the specified expression.
// It returns the local variables bound within the expression.
//
// The isPredeclared and isUniversal predicates behave as for the File function.
func Expr(expr syntax.Expr, isPredeclared, isUniversal func(name string) bool) ([]*syntax.Ident, error) {
r := newResolver(isPredeclared, isUniversal)
r.expr(expr)
r.env.resolveLocalUses()
r.resolveNonLocalUses(r.env) // globals & universals
if len(r.errors) > 0 {
return nil, r.errors
}
return r.moduleLocals, nil
}
// An ErrorList is a non-empty list of resolver error messages.
type ErrorList []Error // len > 0
func (e ErrorList) Error() string { return e[0].Error() }
// An Error describes the nature and position of a resolver error.
type Error struct {
Pos syntax.Position
Msg string
}
func (e Error) Error() string { return e.Pos.String() + ": " + e.Msg }
// The Scope of a syntax.Ident indicates what kind of scope it has.
type Scope uint8
const (
Undefined Scope = iota // name is not defined
Local // name is local to its function
Free // name is local to some enclosing function
Global // name is global to module
Predeclared // name is predeclared for this module (e.g. glob)
Universal // name is universal (e.g. len)
)
var scopeNames = [...]string{
Undefined: "undefined",
Local: "local",
Free: "free",
Global: "global",
Predeclared: "predeclared",
Universal: "universal",
}
func (scope Scope) String() string { return scopeNames[scope] }
func newResolver(isPredeclared, isUniversal func(name string) bool) *resolver {
return &resolver{
env: new(block), // module block
isPredeclared: isPredeclared,
isUniversal: isUniversal,
globals: make(map[string]*syntax.Ident),
}
}
type resolver struct {
// env is the current local environment:
// a linked list of blocks, innermost first.
// The tail of the list is the module block.
env *block
// moduleLocals contains the local variables of the module
// (due to comprehensions outside any function).
// moduleGlobals contains the global variables of the module.
moduleLocals []*syntax.Ident
moduleGlobals []*syntax.Ident
// globals maps each global name in the module
// to its first binding occurrence.
globals map[string]*syntax.Ident
// These predicates report whether a name is
// pre-declared, either in this module or universally.
isPredeclared, isUniversal func(name string) bool
loops int // number of enclosing for loops
errors ErrorList
}
// container returns the innermost enclosing "container" block:
// a function (function != nil) or module (function == nil).
// Container blocks accumulate local variable bindings.
func (r *resolver) container() *block {
for b := r.env; ; b = b.parent {
if b.function != nil || b.isModule() {
return b
}
}
}
func (r *resolver) push(b *block) {
r.env.children = append(r.env.children, b)
b.parent = r.env
r.env = b
}
func (r *resolver) pop() { r.env = r.env.parent }
type block struct {
parent *block // nil for module block
// In the module (root) block, both these fields are nil.
function *syntax.Function // only for function blocks
comp *syntax.Comprehension // only for comprehension blocks
// bindings maps a name to its binding.
// A local binding has an index into its innermost enclosing container's locals array.
// A free binding has an index into its innermost enclosing function's freevars array.
bindings map[string]binding
// children records the child blocks of the current one.
children []*block
// uses records all identifiers seen in this container (function or module),
// and a reference to the environment in which they appear.
// As we leave each container block, we resolve them,
// so that only free and global ones remain.
// At the end of each top-level function we compute closures.
uses []use
}
type binding struct {
scope Scope
index int
}
func (b *block) isModule() bool { return b.parent == nil }
func (b *block) bind(name string, bind binding) {
if b.bindings == nil {
b.bindings = make(map[string]binding)
}
b.bindings[name] = bind
}
func (b *block) String() string {
if b.function != nil {
return "function block at " + fmt.Sprint(b.function.Span())
}
if b.comp != nil {
return "comprehension block at " + fmt.Sprint(b.comp.Span())
}
return "module block"
}
func (r *resolver) errorf(posn syntax.Position, format string, args ...interface{}) {
r.errors = append(r.errors, Error{posn, fmt.Sprintf(format, args...)})
}
// A use records an identifier and the environment in which it appears.
type use struct {
id *syntax.Ident
env *block
}
// bind creates a binding for id in the current block,
// if there is not one already, and reports an error if
// a global was re-bound.
// It returns whether a binding already existed.
func (r *resolver) bind(id *syntax.Ident) bool {
// Binding outside any local (comprehension/function) block?
if r.env.isModule() {
id.Scope = uint8(Global)
prev, ok := r.globals[id.Name]
if ok {
if !AllowGlobalReassign {
r.errorf(id.NamePos, "cannot reassign global %s declared at %s", id.Name, prev.NamePos)
}
id.Index = prev.Index
} else {
// first global binding of this name
r.globals[id.Name] = id
id.Index = len(r.moduleGlobals)
r.moduleGlobals = append(r.moduleGlobals, id)
}
return ok
}
// Mark this name as local to current block.
// Assign it a new local (positive) index in the current container.
_, ok := r.env.bindings[id.Name]
if !ok {
var locals *[]*syntax.Ident
if fn := r.container().function; fn != nil {
locals = &fn.Locals
} else {
locals = &r.moduleLocals
}
r.env.bind(id.Name, binding{Local, len(*locals)})
*locals = append(*locals, id)
}
r.use(id)
return ok
}
func (r *resolver) use(id *syntax.Ident) {
use := use{id, r.env}
// The spec says that if there is a global binding of a name
// then all references to that name in that block refer to the
// global, even if the use precedes the def---just as for locals.
// For example, in this code,
//
// print(len); len=1; print(len)
//
// both occurrences of len refer to the len=1 binding, which
// completely shadows the predeclared len function.
//
// The rationale for these semantics, which differ from Python,
// is that the static meaning of len (a reference to a global)
// does not change depending on where it appears in the file.
// Of course, its dynamic meaning does change, from an error
// into a valid reference, so it's not clear these semantics
// have any practical advantage.
//
// In any case, the Bazel implementation lags behind the spec
// and follows Python behavior, so the first use of len refers
// to the predeclared function. This typically used in a BUILD
// file that redefines a predeclared name half way through,
// for example:
//
// proto_library(...) # built-in rule
// load("myproto.bzl", "proto_library")
// proto_library(...) # user-defined rule
//
// We will piggyback support for the legacy semantics on the
// AllowGlobalReassign flag, which is loosely related and also
// required for Bazel.
if AllowGlobalReassign && r.env.isModule() {
r.useGlobal(use)
return
}
b := r.container()
b.uses = append(b.uses, use)
}
// useGlobals resolves use.id as a reference to a global.
// The use.env field captures the original environment for error reporting.
func (r *resolver) useGlobal(use use) binding {
id := use.id
var scope Scope
if prev, ok := r.globals[id.Name]; ok {
scope = Global // use of global declared by module
id.Index = prev.Index
} else if r.isPredeclared(id.Name) {
scope = Predeclared // use of pre-declared
} else if r.isUniversal(id.Name) {
scope = Universal // use of universal name
if !AllowFloat && id.Name == "float" {
r.errorf(id.NamePos, doesnt+"support floating point")
}
if !AllowSet && id.Name == "set" {
r.errorf(id.NamePos, doesnt+"support sets")
}
} else {
scope = Undefined
var hint string
if n := r.spellcheck(use); n != "" {
hint = fmt.Sprintf(" (did you mean %s?)", n)
}
r.errorf(id.NamePos, "undefined: %s%s", id.Name, hint)
}
id.Scope = uint8(scope)
return binding{scope, id.Index}
}
// spellcheck returns the most likely misspelling of
// the name use.id in the environment use.env.
func (r *resolver) spellcheck(use use) string {
var names []string
// locals
for b := use.env; b != nil; b = b.parent {
for name := range b.bindings {
names = append(names, name)
}
}
// globals
//
// We have no way to enumerate predeclared/universe,
// which includes prior names in the REPL session.
for _, id := range r.moduleGlobals {
names = append(names, id.Name)
}
sort.Strings(names)
return spell.Nearest(use.id.Name, names)
}
// resolveLocalUses is called when leaving a container (function/module)
// block. It resolves all uses of locals within that block.
func (b *block) resolveLocalUses() {
unresolved := b.uses[:0]
for _, use := range b.uses {
if bind := lookupLocal(use); bind.scope == Local {
use.id.Scope = uint8(bind.scope)
use.id.Index = bind.index
} else {
unresolved = append(unresolved, use)
}
}
b.uses = unresolved
}
func (r *resolver) stmts(stmts []syntax.Stmt) {
for _, stmt := range stmts {
r.stmt(stmt)
}
}
func (r *resolver) stmt(stmt syntax.Stmt) {
switch stmt := stmt.(type) {
case *syntax.ExprStmt:
r.expr(stmt.X)
case *syntax.BranchStmt:
if r.loops == 0 && (stmt.Token == syntax.BREAK || stmt.Token == syntax.CONTINUE) {
r.errorf(stmt.TokenPos, "%s not in a loop", stmt.Token)
}
case *syntax.IfStmt:
if !AllowGlobalReassign && r.container().function == nil {
r.errorf(stmt.If, "if statement not within a function")
}
r.expr(stmt.Cond)
r.stmts(stmt.True)
r.stmts(stmt.False)
case *syntax.AssignStmt:
r.expr(stmt.RHS)
isAugmented := stmt.Op != syntax.EQ
r.assign(stmt.LHS, isAugmented)
case *syntax.DefStmt:
if !AllowNestedDef && r.container().function != nil {
r.errorf(stmt.Def, doesnt+"support nested def")
}
r.bind(stmt.Name)
r.function(stmt.Def, stmt.Name.Name, &stmt.Function)
case *syntax.ForStmt:
if !AllowGlobalReassign && r.container().function == nil {
r.errorf(stmt.For, "for loop not within a function")
}
r.expr(stmt.X)
const isAugmented = false
r.assign(stmt.Vars, isAugmented)
r.loops++
r.stmts(stmt.Body)
r.loops--
case *syntax.WhileStmt:
if !AllowRecursion {
r.errorf(stmt.While, doesnt+"support while loops")
}
if !AllowGlobalReassign && r.container().function == nil {
r.errorf(stmt.While, "while loop not within a function")
}
r.expr(stmt.Cond)
r.loops++
r.stmts(stmt.Body)
r.loops--
case *syntax.ReturnStmt:
if r.container().function == nil {
r.errorf(stmt.Return, "return statement not within a function")
}
if stmt.Result != nil {
r.expr(stmt.Result)
}
case *syntax.LoadStmt:
if r.container().function != nil {
r.errorf(stmt.Load, "load statement within a function")
}
for i, from := range stmt.From {
if from.Name == "" {
r.errorf(from.NamePos, "load: empty identifier")
continue
}
if from.Name[0] == '_' {
r.errorf(from.NamePos, "load: names with leading underscores are not exported: %s", from.Name)
}
r.bind(stmt.To[i])
}
default:
log.Fatalf("unexpected stmt %T", stmt)
}
}
func (r *resolver) assign(lhs syntax.Expr, isAugmented bool) {
switch lhs := lhs.(type) {
case *syntax.Ident:
// x = ...
r.bind(lhs)
case *syntax.IndexExpr:
// x[i] = ...
r.expr(lhs.X)
r.expr(lhs.Y)
case *syntax.DotExpr:
// x.f = ...
r.expr(lhs.X)
case *syntax.TupleExpr:
// (x, y) = ...
if len(lhs.List) == 0 {
r.errorf(syntax.Start(lhs), "can't assign to ()")
}
if isAugmented {
r.errorf(syntax.Start(lhs), "can't use tuple expression in augmented assignment")
}
for _, elem := range lhs.List {
r.assign(elem, isAugmented)
}
case *syntax.ListExpr:
// [x, y, z] = ...
if len(lhs.List) == 0 {
r.errorf(syntax.Start(lhs), "can't assign to []")
}
if isAugmented {
r.errorf(syntax.Start(lhs), "can't use list expression in augmented assignment")
}
for _, elem := range lhs.List {
r.assign(elem, isAugmented)
}
case *syntax.ParenExpr:
r.assign(lhs.X, isAugmented)
default:
name := strings.ToLower(strings.TrimPrefix(fmt.Sprintf("%T", lhs), "*syntax."))
r.errorf(syntax.Start(lhs), "can't assign to %s", name)
}
}
func (r *resolver) expr(e syntax.Expr) {
switch e := e.(type) {
case *syntax.Ident:
r.use(e)
case *syntax.Literal:
if !AllowFloat && e.Token == syntax.FLOAT {
r.errorf(e.TokenPos, doesnt+"support floating point")
}
case *syntax.ListExpr:
for _, x := range e.List {
r.expr(x)
}
case *syntax.CondExpr:
r.expr(e.Cond)
r.expr(e.True)
r.expr(e.False)
case *syntax.IndexExpr:
r.expr(e.X)
r.expr(e.Y)
case *syntax.DictEntry:
r.expr(e.Key)
r.expr(e.Value)
case *syntax.SliceExpr:
r.expr(e.X)
if e.Lo != nil {
r.expr(e.Lo)
}
if e.Hi != nil {
r.expr(e.Hi)
}
if e.Step != nil {
r.expr(e.Step)
}
case *syntax.Comprehension:
// The 'in' operand of the first clause (always a ForClause)
// is resolved in the outer block; consider: [x for x in x].
clause := e.Clauses[0].(*syntax.ForClause)
r.expr(clause.X)
// A list/dict comprehension defines a new lexical block.
// Locals defined within the block will be allotted
// distinct slots in the locals array of the innermost
// enclosing container (function/module) block.
r.push(&block{comp: e})
const isAugmented = false
r.assign(clause.Vars, isAugmented)
for _, clause := range e.Clauses[1:] {
switch clause := clause.(type) {
case *syntax.IfClause:
r.expr(clause.Cond)
case *syntax.ForClause:
r.assign(clause.Vars, isAugmented)
r.expr(clause.X)
}
}
r.expr(e.Body) // body may be *DictEntry
r.pop()
case *syntax.TupleExpr:
for _, x := range e.List {
r.expr(x)
}
case *syntax.DictExpr:
for _, entry := range e.List {
entry := entry.(*syntax.DictEntry)
r.expr(entry.Key)
r.expr(entry.Value)
}
case *syntax.UnaryExpr:
r.expr(e.X)
case *syntax.BinaryExpr:
if !AllowFloat && e.Op == syntax.SLASH {
r.errorf(e.OpPos, doesnt+"support floating point (use //)")
}
r.expr(e.X)
r.expr(e.Y)
case *syntax.DotExpr:
r.expr(e.X)
// ignore e.Name
case *syntax.CallExpr:
r.expr(e.Fn)
var seenVarargs, seenKwargs bool
var seenName map[string]bool
var n, p int
for _, arg := range e.Args {
pos, _ := arg.Span()
if unop, ok := arg.(*syntax.UnaryExpr); ok && unop.Op == syntax.STARSTAR {
// **kwargs
if seenKwargs {
r.errorf(pos, "multiple **kwargs not allowed")
}
seenKwargs = true
r.expr(arg)
} else if ok && unop.Op == syntax.STAR {
// *args
if seenKwargs {
r.errorf(pos, "*args may not follow **kwargs")
} else if seenVarargs {
r.errorf(pos, "multiple *args not allowed")
}
seenVarargs = true
r.expr(arg)
} else if binop, ok := arg.(*syntax.BinaryExpr); ok && binop.Op == syntax.EQ {
// k=v
n++
if seenKwargs {
r.errorf(pos, "argument may not follow **kwargs")
}
x := binop.X.(*syntax.Ident)
if seenName[x.Name] {
r.errorf(x.NamePos, "keyword argument %s repeated", x.Name)
} else {
if seenName == nil {
seenName = make(map[string]bool)
}
seenName[x.Name] = true
}
r.expr(binop.Y)
} else {
// positional argument
p++
if seenVarargs {
r.errorf(pos, "argument may not follow *args")
} else if seenKwargs {
r.errorf(pos, "argument may not follow **kwargs")
} else if len(seenName) > 0 {
r.errorf(pos, "positional argument may not follow named")
}
r.expr(arg)
}
}
// Fail gracefully if compiler-imposed limit is exceeded.
if p >= 256 {
pos, _ := e.Span()
r.errorf(pos, "%v positional arguments in call, limit is 255", p)
}
if n >= 256 {
pos, _ := e.Span()
r.errorf(pos, "%v keyword arguments in call, limit is 255", n)
}
case *syntax.LambdaExpr:
if !AllowLambda {
r.errorf(e.Lambda, doesnt+"support lambda")
}
r.function(e.Lambda, "lambda", &e.Function)
case *syntax.ParenExpr:
r.expr(e.X)
default:
log.Fatalf("unexpected expr %T", e)
}
}
func (r *resolver) function(pos syntax.Position, name string, function *syntax.Function) {
// Resolve defaults in enclosing environment.
for _, param := range function.Params {
if binary, ok := param.(*syntax.BinaryExpr); ok {
r.expr(binary.Y)
}
}
// Enter function block.
b := &block{function: function}
r.push(b)
var seenOptional bool
var star *syntax.UnaryExpr // * or *args param
var starStar *syntax.Ident // **kwargs ident
var numKwonlyParams int
for _, param := range function.Params {
switch param := param.(type) {
case *syntax.Ident:
// e.g. x
if starStar != nil {
r.errorf(param.NamePos, "required parameter may not follow **%s", starStar.Name)
} else if star != nil {
numKwonlyParams++
} else if seenOptional {
r.errorf(param.NamePos, "required parameter may not follow optional")
}
if r.bind(param) {
r.errorf(param.NamePos, "duplicate parameter: %s", param.Name)
}
case *syntax.BinaryExpr:
// e.g. y=dflt
if starStar != nil {
r.errorf(param.OpPos, "optional parameter may not follow **%s", starStar.Name)
} else if star != nil {
numKwonlyParams++
}
if id := param.X.(*syntax.Ident); r.bind(id) {
r.errorf(param.OpPos, "duplicate parameter: %s", id.Name)
}
seenOptional = true
case *syntax.UnaryExpr:
// * or *args or **kwargs
if param.Op == syntax.STAR {
if starStar != nil {
r.errorf(param.OpPos, "* parameter may not follow **%s", starStar.Name)
} else if star != nil {
r.errorf(param.OpPos, "multiple * parameters not allowed")
} else {
star = param
}
} else {
if starStar != nil {
r.errorf(param.OpPos, "multiple ** parameters not allowed")
}
starStar = param.X.(*syntax.Ident)
}
}
}
// Bind the *args and **kwargs parameters at the end,
// so that regular parameters a/b/c are contiguous and
// there is no hole for the "*":
// def f(a, b, *args, c=0, **kwargs)
// def f(a, b, *, c=0, **kwargs)
if star != nil {
if id, _ := star.X.(*syntax.Ident); id != nil {
// *args
if r.bind(id) {
r.errorf(id.NamePos, "duplicate parameter: %s", id.Name)
}
function.HasVarargs = true
} else if numKwonlyParams == 0 {
r.errorf(star.OpPos, "bare * must be followed by keyword-only parameters")
}
}
if starStar != nil {
if r.bind(starStar) {
r.errorf(starStar.NamePos, "duplicate parameter: %s", starStar.Name)
}
function.HasKwargs = true
}
function.NumKwonlyParams = numKwonlyParams
r.stmts(function.Body)
// Resolve all uses of this function's local vars,
// and keep just the remaining uses of free/global vars.
b.resolveLocalUses()
// Leave function block.
r.pop()
// References within the function body to globals are not
// resolved until the end of the module.
}
func (r *resolver) resolveNonLocalUses(b *block) {
// First resolve inner blocks.
for _, child := range b.children {
r.resolveNonLocalUses(child)
}
for _, use := range b.uses {
bind := r.lookupLexical(use, use.env)
use.id.Scope = uint8(bind.scope)
use.id.Index = bind.index
}
}
// lookupLocal looks up an identifier within its immediately enclosing function.
func lookupLocal(use use) binding {
for env := use.env; env != nil; env = env.parent {
if bind, ok := env.bindings[use.id.Name]; ok {
if bind.scope == Free {
// shouldn't exist till later
log.Fatalf("%s: internal error: %s, %d", use.id.NamePos, use.id.Name, bind)
}
return bind // found
}
if env.function != nil {
break
}
}
return binding{} // not found in this function
}
// lookupLexical looks up an identifier use.id within its lexically enclosing environment.
// The use.env field captures the original environment for error reporting.
func (r *resolver) lookupLexical(use use, env *block) (bind binding) {
if debug {
fmt.Printf("lookupLexical %s in %s = ...\n", use.id.Name, env)
defer func() { fmt.Printf("= %d\n", bind) }()
}
// Is this the module block?
if env.isModule() {
return r.useGlobal(use) // global, predeclared, or not found
}
// Defined in this block?
bind, ok := env.bindings[use.id.Name]
if !ok {
// Defined in parent block?
bind = r.lookupLexical(use, env.parent)
if env.function != nil && (bind.scope == Local || bind.scope == Free) {
// Found in parent block, which belongs to enclosing function.
id := &syntax.Ident{
Name: use.id.Name,
Scope: uint8(bind.scope),
Index: bind.index,
}
bind.scope = Free
bind.index = len(env.function.FreeVars)
env.function.FreeVars = append(env.function.FreeVars, id)
if debug {
fmt.Printf("creating freevar %v in function at %s: %s\n",
len(env.function.FreeVars), fmt.Sprint(env.function.Span()), id.Name)
}
}
// Memoize, to avoid duplicate free vars
// and redundant global (failing) lookups.
env.bind(use.id.Name, bind)
}
return bind
}

18
vendor/go.starlark.net/starlark/debug.go generated vendored Normal file
View File

@ -0,0 +1,18 @@
package starlark
// This file defines an experimental API for the debugging tools.
// Some of these declarations expose details of internal packages.
// (The debugger makes liberal use of exported fields of unexported types.)
// Breaking changes may occur without notice.
// Local returns the value of the i'th local variable.
// It may be nil if not yet assigned.
//
// Local may be called only for frames whose Callable is a *Function (a
// function defined by Starlark source code), and only while the frame
// is active; it will panic otherwise.
//
// This function is provided only for debugging tools.
//
// THIS API IS EXPERIMENTAL AND MAY CHANGE WITHOUT NOTICE.
func (fr *Frame) Local(i int) Value { return fr.locals[i] }

3
vendor/go.starlark.net/starlark/empty.s generated vendored Normal file
View File

@ -0,0 +1,3 @@
// The presence of this file allows the package to use the
// "go:linkname" hack to call non-exported functions in the
// Go runtime, such as hardware-accelerated string hashing.

1438
vendor/go.starlark.net/starlark/eval.go generated vendored Normal file
View File

File diff suppressed because it is too large Load Diff

358
vendor/go.starlark.net/starlark/hashtable.go generated vendored Normal file
View File

@ -0,0 +1,358 @@
// Copyright 2017 The Bazel Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package starlark
import (
"fmt"
_ "unsafe" // for go:linkname hack
)
// hashtable is used to represent Starlark dict and set values.
// It is a hash table whose key/value entries form a doubly-linked list
// in the order the entries were inserted.
type hashtable struct {
table []bucket // len is zero or a power of two
bucket0 [1]bucket // inline allocation for small maps.
len uint32
itercount uint32 // number of active iterators (ignored if frozen)
head *entry // insertion order doubly-linked list; may be nil
tailLink **entry // address of nil link at end of list (perhaps &head)
frozen bool
}
const bucketSize = 8
type bucket struct {
entries [bucketSize]entry
next *bucket // linked list of buckets
}
type entry struct {
hash uint32 // nonzero => in use
key, value Value
next *entry // insertion order doubly-linked list; may be nil
prevLink **entry // address of link to this entry (perhaps &head)
}
func (ht *hashtable) freeze() {
if !ht.frozen {
ht.frozen = true
for i := range ht.table {
for p := &ht.table[i]; p != nil; p = p.next {
for i := range p.entries {
e := &p.entries[i]
if e.hash != 0 {
e.key.Freeze()
e.value.Freeze()
}
}
}
}
}
}
func (ht *hashtable) insert(k, v Value) error {
if ht.frozen {
return fmt.Errorf("cannot insert into frozen hash table")
}
if ht.itercount > 0 {
return fmt.Errorf("cannot insert into hash table during iteration")
}
if ht.table == nil {
ht.table = ht.bucket0[:1]
ht.tailLink = &ht.head
}
h, err := k.Hash()
if err != nil {
return err
}
if h == 0 {
h = 1 // zero is reserved
}
retry:
var insert *entry
// Inspect each bucket in the bucket list.
p := &ht.table[h&(uint32(len(ht.table)-1))]
for {
for i := range p.entries {
e := &p.entries[i]
if e.hash != h {
if e.hash == 0 {
// Found empty entry; make a note.
insert = e
}
continue
}
if eq, err := Equal(k, e.key); err != nil {
return err // e.g. excessively recursive tuple
} else if !eq {
continue
}
// Key already present; update value.
e.value = v
return nil
}
if p.next == nil {
break
}
p = p.next
}
// Key not found. p points to the last bucket.
// Does the number of elements exceed the buckets' load factor?
if overloaded(int(ht.len), len(ht.table)) {
ht.grow()
goto retry
}
if insert == nil {
// No space in existing buckets. Add a new one to the bucket list.
b := new(bucket)
p.next = b
insert = &b.entries[0]
}
// Insert key/value pair.
insert.hash = h
insert.key = k
insert.value = v
// Append entry to doubly-linked list.
insert.prevLink = ht.tailLink
*ht.tailLink = insert
ht.tailLink = &insert.next
ht.len++
return nil
}
func overloaded(elems, buckets int) bool {
const loadFactor = 6.5 // just a guess
return elems >= bucketSize && float64(elems) >= loadFactor*float64(buckets)
}
func (ht *hashtable) grow() {
// Double the number of buckets and rehash.
// TODO(adonovan): opt:
// - avoid reentrant calls to ht.insert, and specialize it.
// e.g. we know the calls to Equals will return false since
// there are no duplicates among the old keys.
// - saving the entire hash in the bucket would avoid the need to
// recompute the hash.
// - save the old buckets on a free list.
ht.table = make([]bucket, len(ht.table)<<1)
oldhead := ht.head
ht.head = nil
ht.tailLink = &ht.head
ht.len = 0
for e := oldhead; e != nil; e = e.next {
ht.insert(e.key, e.value)
}
ht.bucket0[0] = bucket{} // clear out unused initial bucket
}
func (ht *hashtable) lookup(k Value) (v Value, found bool, err error) {
h, err := k.Hash()
if err != nil {
return nil, false, err // unhashable
}
if h == 0 {
h = 1 // zero is reserved
}
if ht.table == nil {
return None, false, nil // empty
}
// Inspect each bucket in the bucket list.
for p := &ht.table[h&(uint32(len(ht.table)-1))]; p != nil; p = p.next {
for i := range p.entries {
e := &p.entries[i]
if e.hash == h {
if eq, err := Equal(k, e.key); err != nil {
return nil, false, err // e.g. excessively recursive tuple
} else if eq {
return e.value, true, nil // found
}
}
}
}
return None, false, nil // not found
}
// Items returns all the items in the map (as key/value pairs) in insertion order.
func (ht *hashtable) items() []Tuple {
items := make([]Tuple, 0, ht.len)
array := make([]Value, ht.len*2) // allocate a single backing array
for e := ht.head; e != nil; e = e.next {
pair := Tuple(array[:2:2])
array = array[2:]
pair[0] = e.key
pair[1] = e.value
items = append(items, pair)
}
return items
}
func (ht *hashtable) first() (Value, bool) {
if ht.head != nil {
return ht.head.key, true
}
return None, false
}
func (ht *hashtable) keys() []Value {
keys := make([]Value, 0, ht.len)
for e := ht.head; e != nil; e = e.next {
keys = append(keys, e.key)
}
return keys
}
func (ht *hashtable) delete(k Value) (v Value, found bool, err error) {
if ht.frozen {
return nil, false, fmt.Errorf("cannot delete from frozen hash table")
}
if ht.itercount > 0 {
return nil, false, fmt.Errorf("cannot delete from hash table during iteration")
}
if ht.table == nil {
return None, false, nil // empty
}
h, err := k.Hash()
if err != nil {
return nil, false, err // unhashable
}
if h == 0 {
h = 1 // zero is reserved
}
// Inspect each bucket in the bucket list.
for p := &ht.table[h&(uint32(len(ht.table)-1))]; p != nil; p = p.next {
for i := range p.entries {
e := &p.entries[i]
if e.hash == h {
if eq, err := Equal(k, e.key); err != nil {
return nil, false, err
} else if eq {
// Remove e from doubly-linked list.
*e.prevLink = e.next
if e.next == nil {
ht.tailLink = e.prevLink // deletion of last entry
} else {
e.next.prevLink = e.prevLink
}
v := e.value
*e = entry{}
ht.len--
return v, true, nil // found
}
}
}
}
// TODO(adonovan): opt: remove completely empty bucket from bucket list.
return None, false, nil // not found
}
func (ht *hashtable) clear() error {
if ht.frozen {
return fmt.Errorf("cannot clear frozen hash table")
}
if ht.itercount > 0 {
return fmt.Errorf("cannot clear hash table during iteration")
}
if ht.table != nil {
for i := range ht.table {
ht.table[i] = bucket{}
}
}
ht.head = nil
ht.tailLink = &ht.head
ht.len = 0
return nil
}
// dump is provided as an aid to debugging.
func (ht *hashtable) dump() {
fmt.Printf("hashtable %p len=%d head=%p tailLink=%p",
ht, ht.len, ht.head, ht.tailLink)
if ht.tailLink != nil {
fmt.Printf(" *tailLink=%p", *ht.tailLink)
}
fmt.Println()
for j := range ht.table {
fmt.Printf("bucket chain %d\n", j)
for p := &ht.table[j]; p != nil; p = p.next {
fmt.Printf("bucket %p\n", p)
for i := range p.entries {
e := &p.entries[i]
fmt.Printf("\tentry %d @ %p hash=%d key=%v value=%v\n",
i, e, e.hash, e.key, e.value)
fmt.Printf("\t\tnext=%p &next=%p prev=%p",
e.next, &e.next, e.prevLink)
if e.prevLink != nil {
fmt.Printf(" *prev=%p", *e.prevLink)
}
fmt.Println()
}
}
}
}
func (ht *hashtable) iterate() *keyIterator {
if !ht.frozen {
ht.itercount++
}
return &keyIterator{ht: ht, e: ht.head}
}
type keyIterator struct {
ht *hashtable
e *entry
}
func (it *keyIterator) Next(k *Value) bool {
if it.e != nil {
*k = it.e.key
it.e = it.e.next
return true
}
return false
}
func (it *keyIterator) Done() {
if !it.ht.frozen {
it.ht.itercount--
}
}
// hashString computes the hash of s.
func hashString(s string) uint32 {
if len(s) >= 12 {
// Call the Go runtime's optimized hash implementation,
// which uses the AESENC instruction on amd64 machines.
return uint32(goStringHash(s, 0))
}
return softHashString(s)
}
//go:linkname goStringHash runtime.stringHash
func goStringHash(s string, seed uintptr) uintptr
// softHashString computes the FNV hash of s in software.
func softHashString(s string) uint32 {
var h uint32
for i := 0; i < len(s); i++ {
h ^= uint32(s[i])
h *= 16777619
}
return h
}

350
vendor/go.starlark.net/starlark/int.go generated vendored Normal file
View File

@ -0,0 +1,350 @@
// Copyright 2017 The Bazel Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package starlark
import (
"fmt"
"math"
"math/big"
"strconv"
"go.starlark.net/syntax"
)
// Int is the type of a Starlark int.
type Int struct {
// We use only the signed 32 bit range of small to ensure
// that small+small and small*small do not overflow.
small int64 // minint32 <= small <= maxint32
big *big.Int // big != nil <=> value is not representable as int32
}
// newBig allocates a new big.Int.
func newBig(x int64) *big.Int {
if 0 <= x && int64(big.Word(x)) == x {
// x is guaranteed to fit into a single big.Word.
// Most starlark ints are small,
// but math/big assumes that since you've chosen to use math/big,
// your big.Ints will probably grow, so it over-allocates.
// Avoid that over-allocation by manually constructing a single-word slice.
// See https://golang.org/cl/150999, which will hopefully land in Go 1.13.
return new(big.Int).SetBits([]big.Word{big.Word(x)})
}
return big.NewInt(x)
}
// MakeInt returns a Starlark int for the specified signed integer.
func MakeInt(x int) Int { return MakeInt64(int64(x)) }
// MakeInt64 returns a Starlark int for the specified int64.
func MakeInt64(x int64) Int {
if math.MinInt32 <= x && x <= math.MaxInt32 {
return Int{small: x}
}
return Int{big: newBig(x)}
}
// MakeUint returns a Starlark int for the specified unsigned integer.
func MakeUint(x uint) Int { return MakeUint64(uint64(x)) }
// MakeUint64 returns a Starlark int for the specified uint64.
func MakeUint64(x uint64) Int {
if x <= math.MaxInt32 {
return Int{small: int64(x)}
}
if uint64(big.Word(x)) == x {
// See comment in newBig for an explanation of this optimization.
return Int{big: new(big.Int).SetBits([]big.Word{big.Word(x)})}
}
return Int{big: new(big.Int).SetUint64(x)}
}
// MakeBigInt returns a Starlark int for the specified big.Int.
// The caller must not subsequently modify x.
func MakeBigInt(x *big.Int) Int {
if n := x.BitLen(); n < 32 || n == 32 && x.Int64() == math.MinInt32 {
return Int{small: x.Int64()}
}
return Int{big: x}
}
var (
zero, one = Int{small: 0}, Int{small: 1}
oneBig = newBig(1)
_ HasUnary = Int{}
)
// Unary implements the operations +int, -int, and ~int.
func (i Int) Unary(op syntax.Token) (Value, error) {
switch op {
case syntax.MINUS:
return zero.Sub(i), nil
case syntax.PLUS:
return i, nil
case syntax.TILDE:
return i.Not(), nil
}
return nil, nil
}
// Int64 returns the value as an int64.
// If it is not exactly representable the result is undefined and ok is false.
func (i Int) Int64() (_ int64, ok bool) {
if i.big != nil {
x, acc := bigintToInt64(i.big)
if acc != big.Exact {
return // inexact
}
return x, true
}
return i.small, true
}
// BigInt returns the value as a big.Int.
// The returned variable must not be modified by the client.
func (i Int) BigInt() *big.Int {
if i.big != nil {
return i.big
}
return newBig(i.small)
}
// Uint64 returns the value as a uint64.
// If it is not exactly representable the result is undefined and ok is false.
func (i Int) Uint64() (_ uint64, ok bool) {
if i.big != nil {
x, acc := bigintToUint64(i.big)
if acc != big.Exact {
return // inexact
}
return x, true
}
if i.small < 0 {
return // inexact
}
return uint64(i.small), true
}
// The math/big API should provide this function.
func bigintToInt64(i *big.Int) (int64, big.Accuracy) {
sign := i.Sign()
if sign > 0 {
if i.Cmp(maxint64) > 0 {
return math.MaxInt64, big.Below
}
} else if sign < 0 {
if i.Cmp(minint64) < 0 {
return math.MinInt64, big.Above
}
}
return i.Int64(), big.Exact
}
// The math/big API should provide this function.
func bigintToUint64(i *big.Int) (uint64, big.Accuracy) {
sign := i.Sign()
if sign > 0 {
if i.BitLen() > 64 {
return math.MaxUint64, big.Below
}
} else if sign < 0 {
return 0, big.Above
}
return i.Uint64(), big.Exact
}
var (
minint64 = new(big.Int).SetInt64(math.MinInt64)
maxint64 = new(big.Int).SetInt64(math.MaxInt64)
)
func (i Int) Format(s fmt.State, ch rune) {
if i.big != nil {
i.big.Format(s, ch)
return
}
newBig(i.small).Format(s, ch)
}
func (i Int) String() string {
if i.big != nil {
return i.big.Text(10)
}
return strconv.FormatInt(i.small, 10)
}
func (i Int) Type() string { return "int" }
func (i Int) Freeze() {} // immutable
func (i Int) Truth() Bool { return i.Sign() != 0 }
func (i Int) Hash() (uint32, error) {
var lo big.Word
if i.big != nil {
lo = i.big.Bits()[0]
} else {
lo = big.Word(i.small)
}
return 12582917 * uint32(lo+3), nil
}
func (x Int) CompareSameType(op syntax.Token, v Value, depth int) (bool, error) {
y := v.(Int)
if x.big != nil || y.big != nil {
return threeway(op, x.BigInt().Cmp(y.BigInt())), nil
}
return threeway(op, signum64(x.small-y.small)), nil
}
// Float returns the float value nearest i.
func (i Int) Float() Float {
if i.big != nil {
f, _ := new(big.Float).SetInt(i.big).Float64()
return Float(f)
}
return Float(i.small)
}
func (x Int) Sign() int {
if x.big != nil {
return x.big.Sign()
}
return signum64(x.small)
}
func (x Int) Add(y Int) Int {
if x.big != nil || y.big != nil {
return MakeBigInt(new(big.Int).Add(x.BigInt(), y.BigInt()))
}
return MakeInt64(x.small + y.small)
}
func (x Int) Sub(y Int) Int {
if x.big != nil || y.big != nil {
return MakeBigInt(new(big.Int).Sub(x.BigInt(), y.BigInt()))
}
return MakeInt64(x.small - y.small)
}
func (x Int) Mul(y Int) Int {
if x.big != nil || y.big != nil {
return MakeBigInt(new(big.Int).Mul(x.BigInt(), y.BigInt()))
}
return MakeInt64(x.small * y.small)
}
func (x Int) Or(y Int) Int {
if x.big != nil || y.big != nil {
return Int{big: new(big.Int).Or(x.BigInt(), y.BigInt())}
}
return Int{small: x.small | y.small}
}
func (x Int) And(y Int) Int {
if x.big != nil || y.big != nil {
return MakeBigInt(new(big.Int).And(x.BigInt(), y.BigInt()))
}
return Int{small: x.small & y.small}
}
func (x Int) Xor(y Int) Int {
if x.big != nil || y.big != nil {
return MakeBigInt(new(big.Int).Xor(x.BigInt(), y.BigInt()))
}
return Int{small: x.small ^ y.small}
}
func (x Int) Not() Int {
if x.big != nil {
return MakeBigInt(new(big.Int).Not(x.big))
}
return Int{small: ^x.small}
}
func (x Int) Lsh(y uint) Int { return MakeBigInt(new(big.Int).Lsh(x.BigInt(), y)) }
func (x Int) Rsh(y uint) Int { return MakeBigInt(new(big.Int).Rsh(x.BigInt(), y)) }
// Precondition: y is nonzero.
func (x Int) Div(y Int) Int {
// http://python-history.blogspot.com/2010/08/why-pythons-integer-division-floors.html
if x.big != nil || y.big != nil {
xb, yb := x.BigInt(), y.BigInt()
var quo, rem big.Int
quo.QuoRem(xb, yb, &rem)
if (xb.Sign() < 0) != (yb.Sign() < 0) && rem.Sign() != 0 {
quo.Sub(&quo, oneBig)
}
return MakeBigInt(&quo)
}
quo := x.small / y.small
rem := x.small % y.small
if (x.small < 0) != (y.small < 0) && rem != 0 {
quo -= 1
}
return MakeInt64(quo)
}
// Precondition: y is nonzero.
func (x Int) Mod(y Int) Int {
if x.big != nil || y.big != nil {
xb, yb := x.BigInt(), y.BigInt()
var quo, rem big.Int
quo.QuoRem(xb, yb, &rem)
if (xb.Sign() < 0) != (yb.Sign() < 0) && rem.Sign() != 0 {
rem.Add(&rem, yb)
}
return MakeBigInt(&rem)
}
rem := x.small % y.small
if (x.small < 0) != (y.small < 0) && rem != 0 {
rem += y.small
}
return Int{small: rem}
}
func (i Int) rational() *big.Rat {
if i.big != nil {
return new(big.Rat).SetInt(i.big)
}
return new(big.Rat).SetInt64(i.small)
}
// AsInt32 returns the value of x if is representable as an int32.
func AsInt32(x Value) (int, error) {
i, ok := x.(Int)
if !ok {
return 0, fmt.Errorf("got %s, want int", x.Type())
}
if i.big != nil {
return 0, fmt.Errorf("%s out of range", i)
}
return int(i.small), nil
}
// NumberToInt converts a number x to an integer value.
// An int is returned unchanged, a float is truncated towards zero.
// NumberToInt reports an error for all other values.
func NumberToInt(x Value) (Int, error) {
switch x := x.(type) {
case Int:
return x, nil
case Float:
f := float64(x)
if math.IsInf(f, 0) {
return zero, fmt.Errorf("cannot convert float infinity to integer")
} else if math.IsNaN(f) {
return zero, fmt.Errorf("cannot convert float NaN to integer")
}
return finiteFloatToInt(x), nil
}
return zero, fmt.Errorf("cannot convert %s to int", x.Type())
}
// finiteFloatToInt converts f to an Int, truncating towards zero.
// f must be finite.
func finiteFloatToInt(f Float) Int {
if math.MinInt64 <= f && f <= math.MaxInt64 {
// small values
return MakeInt64(int64(f))
}
rat := f.rational()
if rat == nil {
panic(f) // non-finite
}
return MakeBigInt(new(big.Int).Div(rat.Num(), rat.Denom()))
}

575
vendor/go.starlark.net/starlark/interp.go generated vendored Normal file
View File

@ -0,0 +1,575 @@
package starlark
// This file defines the bytecode interpreter.
import (
"fmt"
"os"
"go.starlark.net/internal/compile"
"go.starlark.net/resolve"
"go.starlark.net/syntax"
)
const vmdebug = false // TODO(adonovan): use a bitfield of specific kinds of error.
// TODO(adonovan):
// - optimize position table.
// - opt: reduce allocations by preallocating a large stack, saving it
// in the thread, and slicing it.
// - opt: record MaxIterStack during compilation and preallocate the stack.
func (fn *Function) CallInternal(thread *Thread, args Tuple, kwargs []Tuple) (Value, error) {
if !resolve.AllowRecursion {
// detect recursion
for fr := thread.frame.parent; fr != nil; fr = fr.parent {
// We look for the same function code,
// not function value, otherwise the user could
// defeat the check by writing the Y combinator.
if frfn, ok := fr.Callable().(*Function); ok && frfn.funcode == fn.funcode {
return nil, fmt.Errorf("function %s called recursively", fn.Name())
}
}
}
f := fn.funcode
fr := thread.frame
nlocals := len(f.Locals)
stack := make([]Value, nlocals+f.MaxStack)
locals := stack[:nlocals:nlocals] // local variables, starting with parameters
stack = stack[nlocals:]
err := setArgs(locals, fn, args, kwargs)
if err != nil {
return nil, fr.errorf(fr.Position(), "%v", err)
}
fr.locals = locals // for debugger
if vmdebug {
fmt.Printf("Entering %s @ %s\n", f.Name, f.Position(0))
fmt.Printf("%d stack, %d locals\n", len(stack), len(locals))
defer fmt.Println("Leaving ", f.Name)
}
// TODO(adonovan): add static check that beneath this point
// - there is exactly one return statement
// - there is no redefinition of 'err'.
var iterstack []Iterator // stack of active iterators
sp := 0
var pc, savedpc uint32
var result Value
code := f.Code
loop:
for {
savedpc = pc
op := compile.Opcode(code[pc])
pc++
var arg uint32
if op >= compile.OpcodeArgMin {
// TODO(adonovan): opt: profile this.
// Perhaps compiling big endian would be less work to decode?
for s := uint(0); ; s += 7 {
b := code[pc]
pc++
arg |= uint32(b&0x7f) << s
if b < 0x80 {
break
}
}
}
if vmdebug {
fmt.Fprintln(os.Stderr, stack[:sp]) // very verbose!
compile.PrintOp(f, savedpc, op, arg)
}
switch op {
case compile.NOP:
// nop
case compile.DUP:
stack[sp] = stack[sp-1]
sp++
case compile.DUP2:
stack[sp] = stack[sp-2]
stack[sp+1] = stack[sp-1]
sp += 2
case compile.POP:
sp--
case compile.EXCH:
stack[sp-2], stack[sp-1] = stack[sp-1], stack[sp-2]
case compile.EQL, compile.NEQ, compile.GT, compile.LT, compile.LE, compile.GE:
op := syntax.Token(op-compile.EQL) + syntax.EQL
y := stack[sp-1]
x := stack[sp-2]
sp -= 2
ok, err2 := Compare(op, x, y)
if err2 != nil {
err = err2
break loop
}
stack[sp] = Bool(ok)
sp++
case compile.PLUS,
compile.MINUS,
compile.STAR,
compile.SLASH,
compile.SLASHSLASH,
compile.PERCENT,
compile.AMP,
compile.PIPE,
compile.CIRCUMFLEX,
compile.LTLT,
compile.GTGT,
compile.IN:
binop := syntax.Token(op-compile.PLUS) + syntax.PLUS
if op == compile.IN {
binop = syntax.IN // IN token is out of order
}
y := stack[sp-1]
x := stack[sp-2]
sp -= 2
z, err2 := Binary(binop, x, y)
if err2 != nil {
err = err2
break loop
}
stack[sp] = z
sp++
case compile.UPLUS, compile.UMINUS, compile.TILDE:
var unop syntax.Token
if op == compile.TILDE {
unop = syntax.TILDE
} else {
unop = syntax.Token(op-compile.UPLUS) + syntax.PLUS
}
x := stack[sp-1]
y, err2 := Unary(unop, x)
if err2 != nil {
err = err2
break loop
}
stack[sp-1] = y
case compile.INPLACE_ADD:
y := stack[sp-1]
x := stack[sp-2]
sp -= 2
// It's possible that y is not Iterable but
// nonetheless defines x+y, in which case we
// should fall back to the general case.
var z Value
if xlist, ok := x.(*List); ok {
if yiter, ok := y.(Iterable); ok {
if err = xlist.checkMutable("apply += to"); err != nil {
break loop
}
listExtend(xlist, yiter)
z = xlist
}
}
if z == nil {
z, err = Binary(syntax.PLUS, x, y)
if err != nil {
break loop
}
}
stack[sp] = z
sp++
case compile.NONE:
stack[sp] = None
sp++
case compile.TRUE:
stack[sp] = True
sp++
case compile.FALSE:
stack[sp] = False
sp++
case compile.MANDATORY:
stack[sp] = mandatory{}
sp++
case compile.JMP:
pc = arg
case compile.CALL, compile.CALL_VAR, compile.CALL_KW, compile.CALL_VAR_KW:
var kwargs Value
if op == compile.CALL_KW || op == compile.CALL_VAR_KW {
kwargs = stack[sp-1]
sp--
}
var args Value
if op == compile.CALL_VAR || op == compile.CALL_VAR_KW {
args = stack[sp-1]
sp--
}
// named args (pairs)
var kvpairs []Tuple
if nkvpairs := int(arg & 0xff); nkvpairs > 0 {
kvpairs = make([]Tuple, 0, nkvpairs)
kvpairsAlloc := make(Tuple, 2*nkvpairs) // allocate a single backing array
sp -= 2 * nkvpairs
for i := 0; i < nkvpairs; i++ {
pair := kvpairsAlloc[:2:2]
kvpairsAlloc = kvpairsAlloc[2:]
pair[0] = stack[sp+2*i] // name
pair[1] = stack[sp+2*i+1] // value
kvpairs = append(kvpairs, pair)
}
}
if kwargs != nil {
// Add key/value items from **kwargs dictionary.
dict, ok := kwargs.(IterableMapping)
if !ok {
err = fmt.Errorf("argument after ** must be a mapping, not %s", kwargs.Type())
break loop
}
items := dict.Items()
for _, item := range items {
if _, ok := item[0].(String); !ok {
err = fmt.Errorf("keywords must be strings, not %s", item[0].Type())
break loop
}
}
if len(kvpairs) == 0 {
kvpairs = items
} else {
kvpairs = append(kvpairs, items...)
}
}
// positional args
var positional Tuple
if npos := int(arg >> 8); npos > 0 {
positional = make(Tuple, npos)
sp -= npos
copy(positional, stack[sp:])
}
if args != nil {
// Add elements from *args sequence.
iter := Iterate(args)
if iter == nil {
err = fmt.Errorf("argument after * must be iterable, not %s", args.Type())
break loop
}
var elem Value
for iter.Next(&elem) {
positional = append(positional, elem)
}
iter.Done()
}
function := stack[sp-1]
if vmdebug {
fmt.Printf("VM call %s args=%s kwargs=%s @%s\n",
function, positional, kvpairs, f.Position(fr.callpc))
}
fr.callpc = savedpc
z, err2 := Call(thread, function, positional, kvpairs)
if err2 != nil {
err = err2
break loop
}
if vmdebug {
fmt.Printf("Resuming %s @ %s\n", f.Name, f.Position(0))
}
stack[sp-1] = z
case compile.ITERPUSH:
x := stack[sp-1]
sp--
iter := Iterate(x)
if iter == nil {
err = fmt.Errorf("%s value is not iterable", x.Type())
break loop
}
iterstack = append(iterstack, iter)
case compile.ITERJMP:
iter := iterstack[len(iterstack)-1]
if iter.Next(&stack[sp]) {
sp++
} else {
pc = arg
}
case compile.ITERPOP:
n := len(iterstack) - 1
iterstack[n].Done()
iterstack = iterstack[:n]
case compile.NOT:
stack[sp-1] = !stack[sp-1].Truth()
case compile.RETURN:
result = stack[sp-1]
break loop
case compile.SETINDEX:
z := stack[sp-1]
y := stack[sp-2]
x := stack[sp-3]
sp -= 3
err = setIndex(x, y, z)
if err != nil {
break loop
}
case compile.INDEX:
y := stack[sp-1]
x := stack[sp-2]
sp -= 2
z, err2 := getIndex(x, y)
if err2 != nil {
err = err2
break loop
}
stack[sp] = z
sp++
case compile.ATTR:
x := stack[sp-1]
name := f.Prog.Names[arg]
y, err2 := getAttr(x, name)
if err2 != nil {
err = err2
break loop
}
stack[sp-1] = y
case compile.SETFIELD:
y := stack[sp-1]
x := stack[sp-2]
sp -= 2
name := f.Prog.Names[arg]
if err2 := setField(x, name, y); err2 != nil {
err = err2
break loop
}
case compile.MAKEDICT:
stack[sp] = new(Dict)
sp++
case compile.SETDICT, compile.SETDICTUNIQ:
dict := stack[sp-3].(*Dict)
k := stack[sp-2]
v := stack[sp-1]
sp -= 3
oldlen := dict.Len()
if err2 := dict.SetKey(k, v); err2 != nil {
err = err2
break loop
}
if op == compile.SETDICTUNIQ && dict.Len() == oldlen {
err = fmt.Errorf("duplicate key: %v", k)
break loop
}
case compile.APPEND:
elem := stack[sp-1]
list := stack[sp-2].(*List)
sp -= 2
list.elems = append(list.elems, elem)
case compile.SLICE:
x := stack[sp-4]
lo := stack[sp-3]
hi := stack[sp-2]
step := stack[sp-1]
sp -= 4
res, err2 := slice(x, lo, hi, step)
if err2 != nil {
err = err2
break loop
}
stack[sp] = res
sp++
case compile.UNPACK:
n := int(arg)
iterable := stack[sp-1]
sp--
iter := Iterate(iterable)
if iter == nil {
err = fmt.Errorf("got %s in sequence assignment", iterable.Type())
break loop
}
i := 0
sp += n
for i < n && iter.Next(&stack[sp-1-i]) {
i++
}
var dummy Value
if iter.Next(&dummy) {
// NB: Len may return -1 here in obscure cases.
err = fmt.Errorf("too many values to unpack (got %d, want %d)", Len(iterable), n)
break loop
}
iter.Done()
if i < n {
err = fmt.Errorf("too few values to unpack (got %d, want %d)", i, n)
break loop
}
case compile.CJMP:
if stack[sp-1].Truth() {
pc = arg
}
sp--
case compile.CONSTANT:
stack[sp] = fn.constants[arg]
sp++
case compile.MAKETUPLE:
n := int(arg)
tuple := make(Tuple, n)
sp -= n
copy(tuple, stack[sp:])
stack[sp] = tuple
sp++
case compile.MAKELIST:
n := int(arg)
elems := make([]Value, n)
sp -= n
copy(elems, stack[sp:])
stack[sp] = NewList(elems)
sp++
case compile.MAKEFUNC:
funcode := f.Prog.Functions[arg]
freevars := stack[sp-1].(Tuple)
defaults := stack[sp-2].(Tuple)
sp -= 2
stack[sp] = &Function{
funcode: funcode,
predeclared: fn.predeclared,
globals: fn.globals,
constants: fn.constants,
defaults: defaults,
freevars: freevars,
}
sp++
case compile.LOAD:
n := int(arg)
module := string(stack[sp-1].(String))
sp--
if thread.Load == nil {
err = fmt.Errorf("load not implemented by this application")
break loop
}
dict, err2 := thread.Load(thread, module)
if err2 != nil {
err = fmt.Errorf("cannot load %s: %v", module, err2)
break loop
}
for i := 0; i < n; i++ {
from := string(stack[sp-1-i].(String))
v, ok := dict[from]
if !ok {
err = fmt.Errorf("load: name %s not found in module %s", from, module)
break loop
}
stack[sp-1-i] = v
}
case compile.SETLOCAL:
locals[arg] = stack[sp-1]
sp--
case compile.SETGLOBAL:
fn.globals[arg] = stack[sp-1]
sp--
case compile.LOCAL:
x := locals[arg]
if x == nil {
err = fmt.Errorf("local variable %s referenced before assignment", f.Locals[arg].Name)
break loop
}
stack[sp] = x
sp++
case compile.FREE:
stack[sp] = fn.freevars[arg]
sp++
case compile.GLOBAL:
x := fn.globals[arg]
if x == nil {
err = fmt.Errorf("global variable %s referenced before assignment", f.Prog.Globals[arg].Name)
break loop
}
stack[sp] = x
sp++
case compile.PREDECLARED:
name := f.Prog.Names[arg]
x := fn.predeclared[name]
if x == nil {
err = fmt.Errorf("internal error: predeclared variable %s is uninitialized", name)
break loop
}
stack[sp] = x
sp++
case compile.UNIVERSAL:
stack[sp] = Universe[f.Prog.Names[arg]]
sp++
default:
err = fmt.Errorf("unimplemented: %s", op)
break loop
}
}
// ITERPOP the rest of the iterator stack.
for _, iter := range iterstack {
iter.Done()
}
if err != nil {
if _, ok := err.(*EvalError); !ok {
err = fr.errorf(f.Position(savedpc), "%s", err.Error())
}
}
fr.locals = nil
return result, err
}
// mandatory is a sentinel value used in a function's defaults tuple
// to indicate that a (keyword-only) parameter is mandatory.
type mandatory struct{}
func (mandatory) String() string { return "mandatory" }
func (mandatory) Type() string { return "mandatory" }
func (mandatory) Freeze() {} // immutable
func (mandatory) Truth() Bool { return False }
func (mandatory) Hash() (uint32, error) { return 0, nil }

2250
vendor/go.starlark.net/starlark/library.go generated vendored Normal file
View File

File diff suppressed because it is too large Load Diff

1252
vendor/go.starlark.net/starlark/value.go generated vendored Normal file
View File

File diff suppressed because it is too large Load Diff

1048
vendor/go.starlark.net/syntax/parse.go generated vendored Normal file
View File

File diff suppressed because it is too large Load Diff

270
vendor/go.starlark.net/syntax/quote.go generated vendored Normal file
View File

@ -0,0 +1,270 @@
// Copyright 2017 The Bazel Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package syntax
// Starlark quoted string utilities.
import (
"bytes"
"fmt"
"strconv"
"strings"
)
// unesc maps single-letter chars following \ to their actual values.
var unesc = [256]byte{
'a': '\a',
'b': '\b',
'f': '\f',
'n': '\n',
'r': '\r',
't': '\t',
'v': '\v',
'\\': '\\',
'\'': '\'',
'"': '"',
}
// esc maps escape-worthy bytes to the char that should follow \.
var esc = [256]byte{
'\a': 'a',
'\b': 'b',
'\f': 'f',
'\n': 'n',
'\r': 'r',
'\t': 't',
'\v': 'v',
'\\': '\\',
'\'': '\'',
'"': '"',
}
// notEsc is a list of characters that can follow a \ in a string value
// without having to escape the \. That is, since ( is in this list, we
// quote the Go string "foo\\(bar" as the Python literal "foo\(bar".
// This really does happen in BUILD files, especially in strings
// being used as shell arguments containing regular expressions.
const notEsc = " !#$%&()*+,-./:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ{|}~"
// unquote unquotes the quoted string, returning the actual
// string value, whether the original was triple-quoted, and
// an error describing invalid input.
func unquote(quoted string) (s string, triple bool, err error) {
// Check for raw prefix: means don't interpret the inner \.
raw := false
if strings.HasPrefix(quoted, "r") {
raw = true
quoted = quoted[1:]
}
if len(quoted) < 2 {
err = fmt.Errorf("string literal too short")
return
}
if quoted[0] != '"' && quoted[0] != '\'' || quoted[0] != quoted[len(quoted)-1] {
err = fmt.Errorf("string literal has invalid quotes")
return
}
// Check for triple quoted string.
quote := quoted[0]
if len(quoted) >= 6 && quoted[1] == quote && quoted[2] == quote && quoted[:3] == quoted[len(quoted)-3:] {
triple = true
quoted = quoted[3 : len(quoted)-3]
} else {
quoted = quoted[1 : len(quoted)-1]
}
// Now quoted is the quoted data, but no quotes.
// If we're in raw mode or there are no escapes or
// carriage returns, we're done.
var unquoteChars string
if raw {
unquoteChars = "\r"
} else {
unquoteChars = "\\\r"
}
if !strings.ContainsAny(quoted, unquoteChars) {
s = quoted
return
}
// Otherwise process quoted string.
// Each iteration processes one escape sequence along with the
// plain text leading up to it.
var buf bytes.Buffer
for {
// Remove prefix before escape sequence.
i := strings.IndexAny(quoted, unquoteChars)
if i < 0 {
i = len(quoted)
}
buf.WriteString(quoted[:i])
quoted = quoted[i:]
if len(quoted) == 0 {
break
}
// Process carriage return.
if quoted[0] == '\r' {
buf.WriteByte('\n')
if len(quoted) > 1 && quoted[1] == '\n' {
quoted = quoted[2:]
} else {
quoted = quoted[1:]
}
continue
}
// Process escape sequence.
if len(quoted) == 1 {
err = fmt.Errorf(`truncated escape sequence \`)
return
}
switch quoted[1] {
default:
// In Python, if \z (for some byte z) is not a known escape sequence
// then it appears as literal text in the string.
buf.WriteString(quoted[:2])
quoted = quoted[2:]
case '\n':
// Ignore the escape and the line break.
quoted = quoted[2:]
case 'a', 'b', 'f', 'n', 'r', 't', 'v', '\\', '\'', '"':
// One-char escape
buf.WriteByte(unesc[quoted[1]])
quoted = quoted[2:]
case '0', '1', '2', '3', '4', '5', '6', '7':
// Octal escape, up to 3 digits.
n := int(quoted[1] - '0')
quoted = quoted[2:]
for i := 1; i < 3; i++ {
if len(quoted) == 0 || quoted[0] < '0' || '7' < quoted[0] {
break
}
n = n*8 + int(quoted[0]-'0')
quoted = quoted[1:]
}
if n >= 256 {
// NOTE: Python silently discards the high bit,
// so that '\541' == '\141' == 'a'.
// Let's see if we can avoid doing that in BUILD files.
err = fmt.Errorf(`invalid escape sequence \%03o`, n)
return
}
buf.WriteByte(byte(n))
case 'x':
// Hexadecimal escape, exactly 2 digits.
if len(quoted) < 4 {
err = fmt.Errorf(`truncated escape sequence %s`, quoted)
return
}
n, err1 := strconv.ParseUint(quoted[2:4], 16, 0)
if err1 != nil {
err = fmt.Errorf(`invalid escape sequence %s`, quoted[:4])
return
}
buf.WriteByte(byte(n))
quoted = quoted[4:]
}
}
s = buf.String()
return
}
// indexByte returns the index of the first instance of b in s, or else -1.
func indexByte(s string, b byte) int {
for i := 0; i < len(s); i++ {
if s[i] == b {
return i
}
}
return -1
}
// hex is a list of the hexadecimal digits, for use in quoting.
// We always print lower-case hexadecimal.
const hex = "0123456789abcdef"
// quote returns the quoted form of the string value "x".
// If triple is true, quote uses the triple-quoted form """x""".
func quote(unquoted string, triple bool) string {
q := `"`
if triple {
q = `"""`
}
var buf bytes.Buffer
buf.WriteString(q)
for i := 0; i < len(unquoted); i++ {
c := unquoted[i]
if c == '"' && triple && (i+1 < len(unquoted) && unquoted[i+1] != '"' || i+2 < len(unquoted) && unquoted[i+2] != '"') {
// Can pass up to two quotes through, because they are followed by a non-quote byte.
buf.WriteByte(c)
if i+1 < len(unquoted) && unquoted[i+1] == '"' {
buf.WriteByte(c)
i++
}
continue
}
if triple && c == '\n' {
// Can allow newline in triple-quoted string.
buf.WriteByte(c)
continue
}
if c == '\'' {
// Can allow ' since we always use ".
buf.WriteByte(c)
continue
}
if c == '\\' {
if i+1 < len(unquoted) && indexByte(notEsc, unquoted[i+1]) >= 0 {
// Can pass \ through when followed by a byte that
// known not to be a valid escape sequence and also
// that does not trigger an escape sequence of its own.
// Use this, because various BUILD files do.
buf.WriteByte('\\')
buf.WriteByte(unquoted[i+1])
i++
continue
}
}
if esc[c] != 0 {
buf.WriteByte('\\')
buf.WriteByte(esc[c])
continue
}
if c < 0x20 || c >= 0x80 {
// BUILD files are supposed to be Latin-1, so escape all control and high bytes.
// I'd prefer to use \x here, but Blaze does not implement
// \x in quoted strings (b/7272572).
buf.WriteByte('\\')
buf.WriteByte(hex[c>>6]) // actually octal but reusing hex digits 0-7.
buf.WriteByte(hex[(c>>3)&7])
buf.WriteByte(hex[c&7])
/*
buf.WriteByte('\\')
buf.WriteByte('x')
buf.WriteByte(hex[c>>4])
buf.WriteByte(hex[c&0xF])
*/
continue
}
buf.WriteByte(c)
continue
}
buf.WriteString(q)
return buf.String()
}

1092
vendor/go.starlark.net/syntax/scan.go generated vendored Normal file
View File

File diff suppressed because it is too large Load Diff

548
vendor/go.starlark.net/syntax/syntax.go generated vendored Normal file
View File

@ -0,0 +1,548 @@
// Copyright 2017 The Bazel Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Package syntax provides a Starlark parser and abstract syntax tree.
package syntax // import "go.starlark.net/syntax"
// A Node is a node in a Starlark syntax tree.
type Node interface {
// Span returns the start and end position of the expression.
Span() (start, end Position)
// Comments returns the comments associated with this node.
// It returns nil if RetainComments was not specified during parsing,
// or if AllocComments was not called.
Comments() *Comments
// AllocComments allocates a new Comments node if there was none.
// This makes possible to add new comments using Comments() method.
AllocComments()
}
// A Comment represents a single # comment.
type Comment struct {
Start Position
Text string // without trailing newline
}
// Comments collects the comments associated with an expression.
type Comments struct {
Before []Comment // whole-line comments before this expression
Suffix []Comment // end-of-line comments after this expression (up to 1)
// For top-level expressions only, After lists whole-line
// comments following the expression.
After []Comment
}
// A commentsRef is a possibly-nil reference to a set of comments.
// A commentsRef is embedded in each type of syntax node,
// and provides its Comments and AllocComments methods.
type commentsRef struct{ ref *Comments }
// Comments returns the comments associated with a syntax node,
// or nil if AllocComments has not yet been called.
func (cr commentsRef) Comments() *Comments { return cr.ref }
// AllocComments enables comments to be associated with a syntax node.
func (cr *commentsRef) AllocComments() {
if cr.ref == nil {
cr.ref = new(Comments)
}
}
// Start returns the start position of the expression.
func Start(n Node) Position {
start, _ := n.Span()
return start
}
// End returns the end position of the expression.
func End(n Node) Position {
_, end := n.Span()
return end
}
// A File represents a Starlark file.
type File struct {
commentsRef
Path string
Stmts []Stmt
// set by resolver:
Locals []*Ident // this file's (comprehension-)local variables
Globals []*Ident // this file's global variables
}
func (x *File) Span() (start, end Position) {
if len(x.Stmts) == 0 {
return
}
start, _ = x.Stmts[0].Span()
_, end = x.Stmts[len(x.Stmts)-1].Span()
return start, end
}
// A Stmt is a Starlark statement.
type Stmt interface {
Node
stmt()
}
func (*AssignStmt) stmt() {}
func (*BranchStmt) stmt() {}
func (*DefStmt) stmt() {}
func (*ExprStmt) stmt() {}
func (*ForStmt) stmt() {}
func (*WhileStmt) stmt() {}
func (*IfStmt) stmt() {}
func (*LoadStmt) stmt() {}
func (*ReturnStmt) stmt() {}
// An AssignStmt represents an assignment:
// x = 0
// x, y = y, x
// x += 1
type AssignStmt struct {
commentsRef
OpPos Position
Op Token // = EQ | {PLUS,MINUS,STAR,PERCENT}_EQ
LHS Expr
RHS Expr
}
func (x *AssignStmt) Span() (start, end Position) {
start, _ = x.LHS.Span()
_, end = x.RHS.Span()
return
}
// A Function represents the common parts of LambdaExpr and DefStmt.
type Function struct {
commentsRef
StartPos Position // position of DEF or LAMBDA token
Params []Expr // param = ident | ident=expr | * | *ident | **ident
Body []Stmt
// set by resolver:
HasVarargs bool // whether params includes *args (convenience)
HasKwargs bool // whether params includes **kwargs (convenience)
NumKwonlyParams int // number of keyword-only optional parameters
Locals []*Ident // this function's local variables, parameters first
FreeVars []*Ident // enclosing local variables to capture in closure
}
func (x *Function) Span() (start, end Position) {
_, end = x.Body[len(x.Body)-1].Span()
return x.StartPos, end
}
// A DefStmt represents a function definition.
type DefStmt struct {
commentsRef
Def Position
Name *Ident
Function
}
func (x *DefStmt) Span() (start, end Position) {
_, end = x.Function.Body[len(x.Body)-1].Span()
return x.Def, end
}
// An ExprStmt is an expression evaluated for side effects.
type ExprStmt struct {
commentsRef
X Expr
}
func (x *ExprStmt) Span() (start, end Position) {
return x.X.Span()
}
// An IfStmt is a conditional: If Cond: True; else: False.
// 'elseif' is desugared into a chain of IfStmts.
type IfStmt struct {
commentsRef
If Position // IF or ELIF
Cond Expr
True []Stmt
ElsePos Position // ELSE or ELIF
False []Stmt // optional
}
func (x *IfStmt) Span() (start, end Position) {
body := x.False
if body == nil {
body = x.True
}
_, end = body[len(body)-1].Span()
return x.If, end
}
// A LoadStmt loads another module and binds names from it:
// load(Module, "x", y="foo").
//
// The AST is slightly unfaithful to the concrete syntax here because
// Starlark's load statement, so that it can be implemented in Python,
// binds some names (like y above) with an identifier and some (like x)
// without. For consistency we create fake identifiers for all the
// strings.
type LoadStmt struct {
commentsRef
Load Position
Module *Literal // a string
From []*Ident // name defined in loading module
To []*Ident // name in loaded module
Rparen Position
}
func (x *LoadStmt) Span() (start, end Position) {
return x.Load, x.Rparen
}
// ModuleName returns the name of the module loaded by this statement.
func (x *LoadStmt) ModuleName() string { return x.Module.Value.(string) }
// A BranchStmt changes the flow of control: break, continue, pass.
type BranchStmt struct {
commentsRef
Token Token // = BREAK | CONTINUE | PASS
TokenPos Position
}
func (x *BranchStmt) Span() (start, end Position) {
return x.TokenPos, x.TokenPos.add(x.Token.String())
}
// A ReturnStmt returns from a function.
type ReturnStmt struct {
commentsRef
Return Position
Result Expr // may be nil
}
func (x *ReturnStmt) Span() (start, end Position) {
if x.Result == nil {
return x.Return, x.Return.add("return")
}
_, end = x.Result.Span()
return x.Return, end
}
// An Expr is a Starlark expression.
type Expr interface {
Node
expr()
}
func (*BinaryExpr) expr() {}
func (*CallExpr) expr() {}
func (*Comprehension) expr() {}
func (*CondExpr) expr() {}
func (*DictEntry) expr() {}
func (*DictExpr) expr() {}
func (*DotExpr) expr() {}
func (*Ident) expr() {}
func (*IndexExpr) expr() {}
func (*LambdaExpr) expr() {}
func (*ListExpr) expr() {}
func (*Literal) expr() {}
func (*ParenExpr) expr() {}
func (*SliceExpr) expr() {}
func (*TupleExpr) expr() {}
func (*UnaryExpr) expr() {}
// An Ident represents an identifier.
type Ident struct {
commentsRef
NamePos Position
Name string
// set by resolver:
Scope uint8 // see type resolve.Scope
Index int // index into enclosing {DefStmt,File}.Locals (if scope==Local) or DefStmt.FreeVars (if scope==Free) or File.Globals (if scope==Global)
}
func (x *Ident) Span() (start, end Position) {
return x.NamePos, x.NamePos.add(x.Name)
}
// A Literal represents a literal string or number.
type Literal struct {
commentsRef
Token Token // = STRING | INT
TokenPos Position
Raw string // uninterpreted text
Value interface{} // = string | int64 | *big.Int
}
func (x *Literal) Span() (start, end Position) {
return x.TokenPos, x.TokenPos.add(x.Raw)
}
// A ParenExpr represents a parenthesized expression: (X).
type ParenExpr struct {
commentsRef
Lparen Position
X Expr
Rparen Position
}
func (x *ParenExpr) Span() (start, end Position) {
return x.Lparen, x.Rparen.add(")")
}
// A CallExpr represents a function call expression: Fn(Args).
type CallExpr struct {
commentsRef
Fn Expr
Lparen Position
Args []Expr // arg = expr | ident=expr | *expr | **expr
Rparen Position
}
func (x *CallExpr) Span() (start, end Position) {
start, _ = x.Fn.Span()
return start, x.Rparen.add(")")
}
// A DotExpr represents a field or method selector: X.Name.
type DotExpr struct {
commentsRef
X Expr
Dot Position
NamePos Position
Name *Ident
}
func (x *DotExpr) Span() (start, end Position) {
start, _ = x.X.Span()
_, end = x.Name.Span()
return
}
// A Comprehension represents a list or dict comprehension:
// [Body for ... if ...] or {Body for ... if ...}
type Comprehension struct {
commentsRef
Curly bool // {x:y for ...} or {x for ...}, not [x for ...]
Lbrack Position
Body Expr
Clauses []Node // = *ForClause | *IfClause
Rbrack Position
}
func (x *Comprehension) Span() (start, end Position) {
return x.Lbrack, x.Rbrack.add("]")
}
// A ForStmt represents a loop: for Vars in X: Body.
type ForStmt struct {
commentsRef
For Position
Vars Expr // name, or tuple of names
X Expr
Body []Stmt
}
func (x *ForStmt) Span() (start, end Position) {
_, end = x.Body[len(x.Body)-1].Span()
return x.For, end
}
// A WhileStmt represents a while loop: while X: Body.
type WhileStmt struct {
commentsRef
While Position
Cond Expr
Body []Stmt
}
func (x *WhileStmt) Span() (start, end Position) {
_, end = x.Body[len(x.Body)-1].Span()
return x.While, end
}
// A ForClause represents a for clause in a list comprehension: for Vars in X.
type ForClause struct {
commentsRef
For Position
Vars Expr // name, or tuple of names
In Position
X Expr
}
func (x *ForClause) Span() (start, end Position) {
_, end = x.X.Span()
return x.For, end
}
// An IfClause represents an if clause in a list comprehension: if Cond.
type IfClause struct {
commentsRef
If Position
Cond Expr
}
func (x *IfClause) Span() (start, end Position) {
_, end = x.Cond.Span()
return x.If, end
}
// A DictExpr represents a dictionary literal: { List }.
type DictExpr struct {
commentsRef
Lbrace Position
List []Expr // all *DictEntrys
Rbrace Position
}
func (x *DictExpr) Span() (start, end Position) {
return x.Lbrace, x.Rbrace.add("}")
}
// A DictEntry represents a dictionary entry: Key: Value.
// Used only within a DictExpr.
type DictEntry struct {
commentsRef
Key Expr
Colon Position
Value Expr
}
func (x *DictEntry) Span() (start, end Position) {
start, _ = x.Key.Span()
_, end = x.Value.Span()
return start, end
}
// A LambdaExpr represents an inline function abstraction.
//
// Although they may be added in future, lambda expressions are not
// currently part of the Starlark spec, so their use is controlled by the
// resolver.AllowLambda flag.
type LambdaExpr struct {
commentsRef
Lambda Position
Function
}
func (x *LambdaExpr) Span() (start, end Position) {
_, end = x.Function.Body[len(x.Body)-1].Span()
return x.Lambda, end
}
// A ListExpr represents a list literal: [ List ].
type ListExpr struct {
commentsRef
Lbrack Position
List []Expr
Rbrack Position
}
func (x *ListExpr) Span() (start, end Position) {
return x.Lbrack, x.Rbrack.add("]")
}
// CondExpr represents the conditional: X if COND else ELSE.
type CondExpr struct {
commentsRef
If Position
Cond Expr
True Expr
ElsePos Position
False Expr
}
func (x *CondExpr) Span() (start, end Position) {
start, _ = x.True.Span()
_, end = x.False.Span()
return start, end
}
// A TupleExpr represents a tuple literal: (List).
type TupleExpr struct {
commentsRef
Lparen Position // optional (e.g. in x, y = 0, 1), but required if List is empty
List []Expr
Rparen Position
}
func (x *TupleExpr) Span() (start, end Position) {
if x.Lparen.IsValid() {
return x.Lparen, x.Rparen
} else {
return Start(x.List[0]), End(x.List[len(x.List)-1])
}
}
// A UnaryExpr represents a unary expression: Op X.
//
// As a special case, UnaryOp{Op:Star} may also represent
// the star parameter in def f(*args) or def f(*, x).
type UnaryExpr struct {
commentsRef
OpPos Position
Op Token
X Expr // may be nil if Op==STAR
}
func (x *UnaryExpr) Span() (start, end Position) {
if x.X != nil {
_, end = x.X.Span()
} else {
end = x.OpPos.add("*")
}
return x.OpPos, end
}
// A BinaryExpr represents a binary expression: X Op Y.
//
// As a special case, BinaryExpr{Op:EQ} may also
// represent a named argument in a call f(k=v)
// or a named parameter in a function declaration
// def f(param=default).
type BinaryExpr struct {
commentsRef
X Expr
OpPos Position
Op Token
Y Expr
}
func (x *BinaryExpr) Span() (start, end Position) {
start, _ = x.X.Span()
_, end = x.Y.Span()
return start, end
}
// A SliceExpr represents a slice or substring expression: X[Lo:Hi:Step].
type SliceExpr struct {
commentsRef
X Expr
Lbrack Position
Lo, Hi, Step Expr // all optional
Rbrack Position
}
func (x *SliceExpr) Span() (start, end Position) {
start, _ = x.X.Span()
return start, x.Rbrack
}
// An IndexExpr represents an index expression: X[Y].
type IndexExpr struct {
commentsRef
X Expr
Lbrack Position
Y Expr
Rbrack Position
}
func (x *IndexExpr) Span() (start, end Position) {
start, _ = x.X.Span()
return start, x.Rbrack
}

163
vendor/go.starlark.net/syntax/walk.go generated vendored Normal file
View File

@ -0,0 +1,163 @@
// Copyright 2017 The Bazel Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package syntax
// Walk traverses a syntax tree in depth-first order.
// It starts by calling f(n); n must not be nil.
// If f returns true, Walk calls itself
// recursively for each non-nil child of n.
// Walk then calls f(nil).
func Walk(n Node, f func(Node) bool) {
if n == nil {
panic("nil")
}
if !f(n) {
return
}
// TODO(adonovan): opt: order cases using profile data.
switch n := n.(type) {
case *File:
walkStmts(n.Stmts, f)
case *ExprStmt:
Walk(n.X, f)
case *BranchStmt:
// no-op
case *IfStmt:
Walk(n.Cond, f)
walkStmts(n.True, f)
walkStmts(n.False, f)
case *AssignStmt:
Walk(n.LHS, f)
Walk(n.RHS, f)
case *DefStmt:
Walk(n.Name, f)
for _, param := range n.Function.Params {
Walk(param, f)
}
walkStmts(n.Function.Body, f)
case *ForStmt:
Walk(n.Vars, f)
Walk(n.X, f)
walkStmts(n.Body, f)
case *ReturnStmt:
if n.Result != nil {
Walk(n.Result, f)
}
case *LoadStmt:
Walk(n.Module, f)
for _, from := range n.From {
Walk(from, f)
}
for _, to := range n.To {
Walk(to, f)
}
case *Ident, *Literal:
// no-op
case *ListExpr:
for _, x := range n.List {
Walk(x, f)
}
case *ParenExpr:
Walk(n.X, f)
case *CondExpr:
Walk(n.Cond, f)
Walk(n.True, f)
Walk(n.False, f)
case *IndexExpr:
Walk(n.X, f)
Walk(n.Y, f)
case *DictEntry:
Walk(n.Key, f)
Walk(n.Value, f)
case *SliceExpr:
Walk(n.X, f)
if n.Lo != nil {
Walk(n.Lo, f)
}
if n.Hi != nil {
Walk(n.Hi, f)
}
if n.Step != nil {
Walk(n.Step, f)
}
case *Comprehension:
Walk(n.Body, f)
for _, clause := range n.Clauses {
Walk(clause, f)
}
case *IfClause:
Walk(n.Cond, f)
case *ForClause:
Walk(n.Vars, f)
Walk(n.X, f)
case *TupleExpr:
for _, x := range n.List {
Walk(x, f)
}
case *DictExpr:
for _, entry := range n.List {
entry := entry.(*DictEntry)
Walk(entry.Key, f)
Walk(entry.Value, f)
}
case *UnaryExpr:
if n.X != nil {
Walk(n.X, f)
}
case *BinaryExpr:
Walk(n.X, f)
Walk(n.Y, f)
case *DotExpr:
Walk(n.X, f)
Walk(n.Name, f)
case *CallExpr:
Walk(n.Fn, f)
for _, arg := range n.Args {
Walk(arg, f)
}
case *LambdaExpr:
for _, param := range n.Function.Params {
Walk(param, f)
}
walkStmts(n.Function.Body, f)
default:
panic(n)
}
f(nil)
}
func walkStmts(stmts []Stmt, f func(Node) bool) {
for _, stmt := range stmts {
Walk(stmt, f)
}
}