diff --git a/_content/doc/articles/go_command.html b/_content/doc/articles/go_command.html new file mode 100644 index 00000000..5b6fd4d2 --- /dev/null +++ b/_content/doc/articles/go_command.html @@ -0,0 +1,254 @@ + + +

The Go distribution includes a command, named +"go", that +automates the downloading, building, installation, and testing of Go packages +and commands. This document talks about why we wrote a new command, what it +is, what it's not, and how to use it.

+ +

Motivation

+ +

You might have seen early Go talks in which Rob Pike jokes that the idea +for Go arose while waiting for a large Google server to compile. That +really was the motivation for Go: to build a language that worked well +for building the large software that Google writes and runs. It was +clear from the start that such a language must provide a way to +express dependencies between code libraries clearly, hence the package +grouping and the explicit import blocks. It was also clear from the +start that you might want arbitrary syntax for describing the code +being imported; this is why import paths are string literals.

+ +

An explicit goal for Go from the beginning was to be able to build Go +code using only the information found in the source itself, not +needing to write a makefile or one of the many modern replacements for +makefiles. If Go needed a configuration file to explain how to build +your program, then Go would have failed.

+ +

At first, there was no Go compiler, and the initial development +focused on building one and then building libraries for it. For +expedience, we postponed the automation of building Go code by using +make and writing makefiles. When compiling a single package involved +multiple invocations of the Go compiler, we even used a program to +write the makefiles for us. You can find it if you dig through the +repository history.

+ +

The purpose of the new go command is our return to this ideal, that Go +programs should compile without configuration or additional effort on +the part of the developer beyond writing the necessary import +statements.

+ +

Configuration versus convention

+ +

The way to achieve the simplicity of a configuration-free system is to +establish conventions. The system works only to the extent that those conventions +are followed. When we first launched Go, many people published packages that +had to be installed in certain places, under certain names, using certain build +tools, in order to be used. That's understandable: that's the way it works in +most other languages. Over the last few years we consistently reminded people +about the goinstall command +(now replaced by go get) +and its conventions: first, that the import path is derived in a known way from +the URL of the source code; second, that the place to store the sources in +the local file system is derived in a known way from the import path; third, +that each directory in a source tree corresponds to a single package; and +fourth, that the package is built using only information in the source code. +Today, the vast majority of packages follow these conventions. +The Go ecosystem is simpler and more powerful as a result.

+ +

We received many requests to allow a makefile in a package directory to +provide just a little extra configuration beyond what's in the source code. +But that would have introduced new rules. Because we did not accede to such +requests, we were able to write the go command and eliminate our use of make +or any other build system.

+ +

It is important to understand that the go command is not a general +build tool. It cannot be configured and it does not attempt to build +anything but Go packages. These are important simplifying +assumptions: they simplify not only the implementation but also, more +important, the use of the tool itself.

+ +

Go's conventions

+ +

The go command requires that code adheres to a few key, +well-established conventions.

+ +

First, the import path is derived in a known way from the URL of the +source code. For Bitbucket, GitHub, Google Code, and Launchpad, the +root directory of the repository is identified by the repository's +main URL, without the http:// prefix. Subdirectories are named by +adding to that path. +For example, the Go example programs are obtained by running

+ +
+git clone https://github.com/golang/example
+
+ +

and thus the import path for the root directory of that repository is +"github.com/golang/example". +The stringutil +package is stored in a subdirectory, so its import path is +"github.com/golang/example/stringutil".

+ +

These paths are on the long side, but in exchange we get an +automatically managed name space for import paths and the ability for +a tool like the go command to look at an unfamiliar import path and +deduce where to obtain the source code.

+ +

Second, the place to store sources in the local file system is derived +in a known way from the import path, specifically +$GOPATH/src/<import-path>. +If unset, $GOPATH defaults to a subdirectory +named go in the user's home directory. +If $GOPATH is set to a list of paths, the go command tries +<dir>/src/<import-path> for each of the directories in +that list. +

+ +

Each of those trees contains, by convention, a top-level directory named +"bin", for holding compiled executables, and a top-level directory +named "pkg", for holding compiled packages that can be imported, +and the "src" directory, for holding package source files. +Imposing this structure lets us keep each of these directory trees +self-contained: the compiled form and the sources are always near each +other.

+ +

These naming conventions also let us work in the reverse direction, +from a directory name to its import path. This mapping is important +for many of the go command's subcommands, as we'll see below.

+ +

Third, each directory in a source tree corresponds to a single +package. By restricting a directory to a single package, we don't have +to create hybrid import paths that specify first the directory and +then the package within that directory. Also, most file management +tools and UIs work on directories as fundamental units. Tying the +fundamental Go unit—the package—to file system structure means +that file system tools become Go package tools. Copying, moving, or +deleting a package corresponds to copying, moving, or deleting a +directory.

+ +

Fourth, each package is built using only the information present in +the source files. This makes it much more likely that the tool will +be able to adapt to changing build environments and conditions. For +example, if we allowed extra configuration such as compiler flags or +command line recipes, then that configuration would need to be updated +each time the build tools changed; it would also be inherently tied +to the use of a specific toolchain.

+ +

Getting started with the go command

+ +

Finally, a quick tour of how to use the go command. +As mentioned above, the default $GOPATH on Unix is $HOME/go. +We'll store our programs there. +To use a different location, you can set $GOPATH; +see How to Write Go Code for details. + +

We first add some source code. Suppose we want to use +the indexing library from the codesearch project along with a left-leaning +red-black tree. We can install both with the "go get" +subcommand:

+ +
+$ go get github.com/google/codesearch/index
+$ go get github.com/petar/GoLLRB/llrb
+$
+
+ +

Both of these projects are now downloaded and installed into $HOME/go, +which contains the two directories +src/github.com/google/codesearch/index/ and +src/github.com/petar/GoLLRB/llrb/, along with the compiled +packages (in pkg/) for those libraries and their dependencies.

+ +

Because we used version control systems (Mercurial and Git) to check +out the sources, the source tree also contains the other files in the +corresponding repositories, such as related packages. The "go list" +subcommand lists the import paths corresponding to its arguments, and +the pattern "./..." means start in the current directory +("./") and find all packages below that directory +("..."):

+ +
+$ cd $HOME/go/src
+$ go list ./...
+github.com/google/codesearch/cmd/cgrep
+github.com/google/codesearch/cmd/cindex
+github.com/google/codesearch/cmd/csearch
+github.com/google/codesearch/index
+github.com/google/codesearch/regexp
+github.com/google/codesearch/sparse
+github.com/petar/GoLLRB/example
+github.com/petar/GoLLRB/llrb
+$
+
+ +

We can also test those packages:

+ +
+$ go test ./...
+?   	github.com/google/codesearch/cmd/cgrep	[no test files]
+?   	github.com/google/codesearch/cmd/cindex	[no test files]
+?   	github.com/google/codesearch/cmd/csearch	[no test files]
+ok  	github.com/google/codesearch/index	0.203s
+ok  	github.com/google/codesearch/regexp	0.017s
+?   	github.com/google/codesearch/sparse	[no test files]
+?       github.com/petar/GoLLRB/example          [no test files]
+ok      github.com/petar/GoLLRB/llrb             0.231s
+$
+
+ +

If a go subcommand is invoked with no paths listed, it operates on the +current directory:

+ +
+$ cd github.com/google/codesearch/regexp
+$ go list
+github.com/google/codesearch/regexp
+$ go test -v
+=== RUN   TestNstateEnc
+--- PASS: TestNstateEnc (0.00s)
+=== RUN   TestMatch
+--- PASS: TestMatch (0.00s)
+=== RUN   TestGrep
+--- PASS: TestGrep (0.00s)
+PASS
+ok  	github.com/google/codesearch/regexp	0.018s
+$ go install
+$
+
+ +

That "go install" subcommand installs the latest copy of the +package into the pkg directory. Because the go command can analyze the +dependency graph, "go install" also installs any packages that +this package imports but that are out of date, recursively.

+ +

Notice that "go install" was able to determine the name of the +import path for the package in the current directory, because of the convention +for directory naming. It would be a little more convenient if we could pick +the name of the directory where we kept source code, and we probably wouldn't +pick such a long name, but that ability would require additional configuration +and complexity in the tool. Typing an extra directory name or two is a small +price to pay for the increased simplicity and power.

+ +

Limitations

+ +

As mentioned above, the go command is not a general-purpose build +tool. +In particular, it does not have any facility for generating Go +source files during a build, although it does provide +go +generate, +which can automate the creation of Go files before the build. +For more advanced build setups, you may need to write a +makefile (or a configuration file for the build tool of your choice) +to run whatever tool creates the Go files and then check those generated source files +into your repository. This is more work for you, the package author, +but it is significantly less work for your users, who can use +"go get" without needing to obtain and build +any additional tools.

+ +

More information

+ +

For more information, read How to Write Go Code +and see the go command documentation.

diff --git a/_content/doc/articles/index.html b/_content/doc/articles/index.html new file mode 100644 index 00000000..9ddd6697 --- /dev/null +++ b/_content/doc/articles/index.html @@ -0,0 +1,8 @@ + + +

+See the Documents page and the +Blog index for a complete list of Go articles. +

diff --git a/_content/doc/articles/race_detector.html b/_content/doc/articles/race_detector.html new file mode 100644 index 00000000..09188c15 --- /dev/null +++ b/_content/doc/articles/race_detector.html @@ -0,0 +1,440 @@ + + +

Introduction

+ +

+Data races are among the most common and hardest to debug types of bugs in concurrent systems. +A data race occurs when two goroutines access the same variable concurrently and at least one of the accesses is a write. +See the The Go Memory Model for details. +

+ +

+Here is an example of a data race that can lead to crashes and memory corruption: +

+ +
+func main() {
+	c := make(chan bool)
+	m := make(map[string]string)
+	go func() {
+		m["1"] = "a" // First conflicting access.
+		c <- true
+	}()
+	m["2"] = "b" // Second conflicting access.
+	<-c
+	for k, v := range m {
+		fmt.Println(k, v)
+	}
+}
+
+ +

Usage

+ +

+To help diagnose such bugs, Go includes a built-in data race detector. +To use it, add the -race flag to the go command: +

+ +
+$ go test -race mypkg    // to test the package
+$ go run -race mysrc.go  // to run the source file
+$ go build -race mycmd   // to build the command
+$ go install -race mypkg // to install the package
+
+ +

Report Format

+ +

+When the race detector finds a data race in the program, it prints a report. +The report contains stack traces for conflicting accesses, as well as stacks where the involved goroutines were created. +Here is an example: +

+ +
+WARNING: DATA RACE
+Read by goroutine 185:
+  net.(*pollServer).AddFD()
+      src/net/fd_unix.go:89 +0x398
+  net.(*pollServer).WaitWrite()
+      src/net/fd_unix.go:247 +0x45
+  net.(*netFD).Write()
+      src/net/fd_unix.go:540 +0x4d4
+  net.(*conn).Write()
+      src/net/net.go:129 +0x101
+  net.func·060()
+      src/net/timeout_test.go:603 +0xaf
+
+Previous write by goroutine 184:
+  net.setWriteDeadline()
+      src/net/sockopt_posix.go:135 +0xdf
+  net.setDeadline()
+      src/net/sockopt_posix.go:144 +0x9c
+  net.(*conn).SetDeadline()
+      src/net/net.go:161 +0xe3
+  net.func·061()
+      src/net/timeout_test.go:616 +0x3ed
+
+Goroutine 185 (running) created at:
+  net.func·061()
+      src/net/timeout_test.go:609 +0x288
+
+Goroutine 184 (running) created at:
+  net.TestProlongTimeout()
+      src/net/timeout_test.go:618 +0x298
+  testing.tRunner()
+      src/testing/testing.go:301 +0xe8
+
+ +

Options

+ +

+The GORACE environment variable sets race detector options. +The format is: +

+ +
+GORACE="option1=val1 option2=val2"
+
+ +

+The options are: +

+ + + +

+Example: +

+ +
+$ GORACE="log_path=/tmp/race/report strip_path_prefix=/my/go/sources/" go test -race
+
+ +

Excluding Tests

+ +

+When you build with -race flag, the go command defines additional +build tag race. +You can use the tag to exclude some code and tests when running the race detector. +Some examples: +

+ +
+// +build !race
+
+package foo
+
+// The test contains a data race. See issue 123.
+func TestFoo(t *testing.T) {
+	// ...
+}
+
+// The test fails under the race detector due to timeouts.
+func TestBar(t *testing.T) {
+	// ...
+}
+
+// The test takes too long under the race detector.
+func TestBaz(t *testing.T) {
+	// ...
+}
+
+ +

How To Use

+ +

+To start, run your tests using the race detector (go test -race). +The race detector only finds races that happen at runtime, so it can't find +races in code paths that are not executed. +If your tests have incomplete coverage, +you may find more races by running a binary built with -race under a realistic +workload. +

+ +

Typical Data Races

+ +

+Here are some typical data races. All of them can be detected with the race detector. +

+ +

Race on loop counter

+ +
+func main() {
+	var wg sync.WaitGroup
+	wg.Add(5)
+	for i := 0; i < 5; i++ {
+		go func() {
+			fmt.Println(i) // Not the 'i' you are looking for.
+			wg.Done()
+		}()
+	}
+	wg.Wait()
+}
+
+ +

+The variable i in the function literal is the same variable used by the loop, so +the read in the goroutine races with the loop increment. +(This program typically prints 55555, not 01234.) +The program can be fixed by making a copy of the variable: +

+ +
+func main() {
+	var wg sync.WaitGroup
+	wg.Add(5)
+	for i := 0; i < 5; i++ {
+		go func(j int) {
+			fmt.Println(j) // Good. Read local copy of the loop counter.
+			wg.Done()
+		}(i)
+	}
+	wg.Wait()
+}
+
+ +

Accidentally shared variable

+ +
+// ParallelWrite writes data to file1 and file2, returns the errors.
+func ParallelWrite(data []byte) chan error {
+	res := make(chan error, 2)
+	f1, err := os.Create("file1")
+	if err != nil {
+		res <- err
+	} else {
+		go func() {
+			// This err is shared with the main goroutine,
+			// so the write races with the write below.
+			_, err = f1.Write(data)
+			res <- err
+			f1.Close()
+		}()
+	}
+	f2, err := os.Create("file2") // The second conflicting write to err.
+	if err != nil {
+		res <- err
+	} else {
+		go func() {
+			_, err = f2.Write(data)
+			res <- err
+			f2.Close()
+		}()
+	}
+	return res
+}
+
+ +

+The fix is to introduce new variables in the goroutines (note the use of :=): +

+ +
+			...
+			_, err := f1.Write(data)
+			...
+			_, err := f2.Write(data)
+			...
+
+ +

Unprotected global variable

+ +

+If the following code is called from several goroutines, it leads to races on the service map. +Concurrent reads and writes of the same map are not safe: +

+ +
+var service map[string]net.Addr
+
+func RegisterService(name string, addr net.Addr) {
+	service[name] = addr
+}
+
+func LookupService(name string) net.Addr {
+	return service[name]
+}
+
+ +

+To make the code safe, protect the accesses with a mutex: +

+ +
+var (
+	service   map[string]net.Addr
+	serviceMu sync.Mutex
+)
+
+func RegisterService(name string, addr net.Addr) {
+	serviceMu.Lock()
+	defer serviceMu.Unlock()
+	service[name] = addr
+}
+
+func LookupService(name string) net.Addr {
+	serviceMu.Lock()
+	defer serviceMu.Unlock()
+	return service[name]
+}
+
+ +

Primitive unprotected variable

+ +

+Data races can happen on variables of primitive types as well (bool, int, int64, etc.), +as in this example: +

+ +
+type Watchdog struct{ last int64 }
+
+func (w *Watchdog) KeepAlive() {
+	w.last = time.Now().UnixNano() // First conflicting access.
+}
+
+func (w *Watchdog) Start() {
+	go func() {
+		for {
+			time.Sleep(time.Second)
+			// Second conflicting access.
+			if w.last < time.Now().Add(-10*time.Second).UnixNano() {
+				fmt.Println("No keepalives for 10 seconds. Dying.")
+				os.Exit(1)
+			}
+		}
+	}()
+}
+
+ +

+Even such "innocent" data races can lead to hard-to-debug problems caused by +non-atomicity of the memory accesses, +interference with compiler optimizations, +or reordering issues accessing processor memory . +

+ +

+A typical fix for this race is to use a channel or a mutex. +To preserve the lock-free behavior, one can also use the +sync/atomic package. +

+ +
+type Watchdog struct{ last int64 }
+
+func (w *Watchdog) KeepAlive() {
+	atomic.StoreInt64(&w.last, time.Now().UnixNano())
+}
+
+func (w *Watchdog) Start() {
+	go func() {
+		for {
+			time.Sleep(time.Second)
+			if atomic.LoadInt64(&w.last) < time.Now().Add(-10*time.Second).UnixNano() {
+				fmt.Println("No keepalives for 10 seconds. Dying.")
+				os.Exit(1)
+			}
+		}
+	}()
+}
+
+ +

Unsynchronized send and close operations

+ +

+As this example demonstrates, unsynchronized send and close operations +on the same channel can also be a race condition: +

+ +
+c := make(chan struct{}) // or buffered channel
+
+// The race detector cannot derive the happens before relation
+// for the following send and close operations. These two operations
+// are unsynchronized and happen concurrently.
+go func() { c <- struct{}{} }()
+close(c)
+
+ +

+According to the Go memory model, a send on a channel happens before +the corresponding receive from that channel completes. To synchronize +send and close operations, use a receive operation that guarantees +the send is done before the close: +

+ +
+c := make(chan struct{}) // or buffered channel
+
+go func() { c <- struct{}{} }()
+<-c
+close(c)
+
+ +

Supported Systems

+ +

+ The race detector runs on + linux/amd64, linux/ppc64le, + linux/arm64, freebsd/amd64, + netbsd/amd64, darwin/amd64, + darwin/arm64, and windows/amd64. +

+ +

Runtime Overhead

+ +

+The cost of race detection varies by program, but for a typical program, memory +usage may increase by 5-10x and execution time by 2-20x. +

+ +

+The race detector currently allocates an extra 8 bytes per defer +and recover statement. Those extra allocations are not recovered until the goroutine +exits. This means that if you have a long-running goroutine that is +periodically issuing defer and recover calls, +the program memory usage may grow without bound. These memory allocations +will not show up in the output of runtime.ReadMemStats or +runtime/pprof. +

diff --git a/_content/doc/articles/wiki/edit.html b/_content/doc/articles/wiki/edit.html new file mode 100644 index 00000000..044c3bed --- /dev/null +++ b/_content/doc/articles/wiki/edit.html @@ -0,0 +1,6 @@ +

Editing {{.Title}}

+ +
+
+
+
diff --git a/_content/doc/articles/wiki/final-noclosure.go b/_content/doc/articles/wiki/final-noclosure.go new file mode 100644 index 00000000..d894e7d3 --- /dev/null +++ b/_content/doc/articles/wiki/final-noclosure.go @@ -0,0 +1,105 @@ +// Copyright 2010 The Go Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + +// +build ignore + +package main + +import ( + "errors" + "html/template" + "io/ioutil" + "log" + "net/http" + "regexp" +) + +type Page struct { + Title string + Body []byte +} + +func (p *Page) save() error { + filename := p.Title + ".txt" + return ioutil.WriteFile(filename, p.Body, 0600) +} + +func loadPage(title string) (*Page, error) { + filename := title + ".txt" + body, err := ioutil.ReadFile(filename) + if err != nil { + return nil, err + } + return &Page{Title: title, Body: body}, nil +} + +func viewHandler(w http.ResponseWriter, r *http.Request) { + title, err := getTitle(w, r) + if err != nil { + return + } + p, err := loadPage(title) + if err != nil { + http.Redirect(w, r, "/edit/"+title, http.StatusFound) + return + } + renderTemplate(w, "view", p) +} + +func editHandler(w http.ResponseWriter, r *http.Request) { + title, err := getTitle(w, r) + if err != nil { + return + } + p, err := loadPage(title) + if err != nil { + p = &Page{Title: title} + } + renderTemplate(w, "edit", p) +} + +func saveHandler(w http.ResponseWriter, r *http.Request) { + title, err := getTitle(w, r) + if err != nil { + return + } + body := r.FormValue("body") + p := &Page{Title: title, Body: []byte(body)} + err = p.save() + if err != nil { + http.Error(w, err.Error(), http.StatusInternalServerError) + return + } + http.Redirect(w, r, "/view/"+title, http.StatusFound) +} + +func renderTemplate(w http.ResponseWriter, tmpl string, p *Page) { + t, err := template.ParseFiles(tmpl + ".html") + if err != nil { + http.Error(w, err.Error(), http.StatusInternalServerError) + return + } + err = t.Execute(w, p) + if err != nil { + http.Error(w, err.Error(), http.StatusInternalServerError) + } +} + +var validPath = regexp.MustCompile("^/(edit|save|view)/([a-zA-Z0-9]+)$") + +func getTitle(w http.ResponseWriter, r *http.Request) (string, error) { + m := validPath.FindStringSubmatch(r.URL.Path) + if m == nil { + http.NotFound(w, r) + return "", errors.New("invalid Page Title") + } + return m[2], nil // The title is the second subexpression. +} + +func main() { + http.HandleFunc("/view/", viewHandler) + http.HandleFunc("/edit/", editHandler) + http.HandleFunc("/save/", saveHandler) + log.Fatal(http.ListenAndServe(":8080", nil)) +} diff --git a/_content/doc/articles/wiki/final-noerror.go b/_content/doc/articles/wiki/final-noerror.go new file mode 100644 index 00000000..250236d4 --- /dev/null +++ b/_content/doc/articles/wiki/final-noerror.go @@ -0,0 +1,56 @@ +// Copyright 2010 The Go Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + +// +build ignore + +package main + +import ( + "html/template" + "io/ioutil" + "log" + "net/http" +) + +type Page struct { + Title string + Body []byte +} + +func (p *Page) save() error { + filename := p.Title + ".txt" + return ioutil.WriteFile(filename, p.Body, 0600) +} + +func loadPage(title string) (*Page, error) { + filename := title + ".txt" + body, err := ioutil.ReadFile(filename) + if err != nil { + return nil, err + } + return &Page{Title: title, Body: body}, nil +} + +func editHandler(w http.ResponseWriter, r *http.Request) { + title := r.URL.Path[len("/edit/"):] + p, err := loadPage(title) + if err != nil { + p = &Page{Title: title} + } + t, _ := template.ParseFiles("edit.html") + t.Execute(w, p) +} + +func viewHandler(w http.ResponseWriter, r *http.Request) { + title := r.URL.Path[len("/view/"):] + p, _ := loadPage(title) + t, _ := template.ParseFiles("view.html") + t.Execute(w, p) +} + +func main() { + http.HandleFunc("/view/", viewHandler) + http.HandleFunc("/edit/", editHandler) + log.Fatal(http.ListenAndServe(":8080", nil)) +} diff --git a/_content/doc/articles/wiki/final-parsetemplate.go b/_content/doc/articles/wiki/final-parsetemplate.go new file mode 100644 index 00000000..0b90cbd3 --- /dev/null +++ b/_content/doc/articles/wiki/final-parsetemplate.go @@ -0,0 +1,94 @@ +// Copyright 2010 The Go Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + +// +build ignore + +package main + +import ( + "html/template" + "io/ioutil" + "log" + "net/http" + "regexp" +) + +type Page struct { + Title string + Body []byte +} + +func (p *Page) save() error { + filename := p.Title + ".txt" + return ioutil.WriteFile(filename, p.Body, 0600) +} + +func loadPage(title string) (*Page, error) { + filename := title + ".txt" + body, err := ioutil.ReadFile(filename) + if err != nil { + return nil, err + } + return &Page{Title: title, Body: body}, nil +} + +func viewHandler(w http.ResponseWriter, r *http.Request, title string) { + p, err := loadPage(title) + if err != nil { + http.Redirect(w, r, "/edit/"+title, http.StatusFound) + return + } + renderTemplate(w, "view", p) +} + +func editHandler(w http.ResponseWriter, r *http.Request, title string) { + p, err := loadPage(title) + if err != nil { + p = &Page{Title: title} + } + renderTemplate(w, "edit", p) +} + +func saveHandler(w http.ResponseWriter, r *http.Request, title string) { + body := r.FormValue("body") + p := &Page{Title: title, Body: []byte(body)} + err := p.save() + if err != nil { + http.Error(w, err.Error(), http.StatusInternalServerError) + return + } + http.Redirect(w, r, "/view/"+title, http.StatusFound) +} + +func renderTemplate(w http.ResponseWriter, tmpl string, p *Page) { + t, err := template.ParseFiles(tmpl + ".html") + if err != nil { + http.Error(w, err.Error(), http.StatusInternalServerError) + return + } + err = t.Execute(w, p) + if err != nil { + http.Error(w, err.Error(), http.StatusInternalServerError) + } +} + +var validPath = regexp.MustCompile("^/(edit|save|view)/([a-zA-Z0-9]+)$") + +func makeHandler(fn func(http.ResponseWriter, *http.Request, string)) http.HandlerFunc { + return func(w http.ResponseWriter, r *http.Request) { + m := validPath.FindStringSubmatch(r.URL.Path) + if m == nil { + http.NotFound(w, r) + return + } + fn(w, r, m[2]) + } +} + +func main() { + http.HandleFunc("/view/", makeHandler(viewHandler)) + http.HandleFunc("/edit/", makeHandler(editHandler)) + http.HandleFunc("/save/", makeHandler(saveHandler)) + log.Fatal(http.ListenAndServe(":8080", nil)) +} diff --git a/_content/doc/articles/wiki/final-template.go b/_content/doc/articles/wiki/final-template.go new file mode 100644 index 00000000..5028664f --- /dev/null +++ b/_content/doc/articles/wiki/final-template.go @@ -0,0 +1,68 @@ +// Copyright 2010 The Go Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + +// +build ignore + +package main + +import ( + "html/template" + "io/ioutil" + "log" + "net/http" +) + +type Page struct { + Title string + Body []byte +} + +func (p *Page) save() error { + filename := p.Title + ".txt" + return ioutil.WriteFile(filename, p.Body, 0600) +} + +func loadPage(title string) (*Page, error) { + filename := title + ".txt" + body, err := ioutil.ReadFile(filename) + if err != nil { + return nil, err + } + return &Page{Title: title, Body: body}, nil +} + +func editHandler(w http.ResponseWriter, r *http.Request) { + title := r.URL.Path[len("/edit/"):] + p, err := loadPage(title) + if err != nil { + p = &Page{Title: title} + } + renderTemplate(w, "edit", p) +} + +func viewHandler(w http.ResponseWriter, r *http.Request) { + title := r.URL.Path[len("/view/"):] + p, _ := loadPage(title) + renderTemplate(w, "view", p) +} + +func saveHandler(w http.ResponseWriter, r *http.Request) { + title := r.URL.Path[len("/save/"):] + body := r.FormValue("body") + p := &Page{Title: title, Body: []byte(body)} + p.save() + http.Redirect(w, r, "/view/"+title, http.StatusFound) +} + +func renderTemplate(w http.ResponseWriter, tmpl string, p *Page) { + t, _ := template.ParseFiles(tmpl + ".html") + t.Execute(w, p) +} + +func main() { + http.HandleFunc("/view/", viewHandler) + http.HandleFunc("/edit/", editHandler) + http.HandleFunc("/save/", saveHandler) + log.Fatal(http.ListenAndServe(":8080", nil)) +} diff --git a/_content/doc/articles/wiki/final.go b/_content/doc/articles/wiki/final.go new file mode 100644 index 00000000..b1439b08 --- /dev/null +++ b/_content/doc/articles/wiki/final.go @@ -0,0 +1,92 @@ +// Copyright 2010 The Go Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + +// +build ignore + +package main + +import ( + "html/template" + "io/ioutil" + "log" + "net/http" + "regexp" +) + +type Page struct { + Title string + Body []byte +} + +func (p *Page) save() error { + filename := p.Title + ".txt" + return ioutil.WriteFile(filename, p.Body, 0600) +} + +func loadPage(title string) (*Page, error) { + filename := title + ".txt" + body, err := ioutil.ReadFile(filename) + if err != nil { + return nil, err + } + return &Page{Title: title, Body: body}, nil +} + +func viewHandler(w http.ResponseWriter, r *http.Request, title string) { + p, err := loadPage(title) + if err != nil { + http.Redirect(w, r, "/edit/"+title, http.StatusFound) + return + } + renderTemplate(w, "view", p) +} + +func editHandler(w http.ResponseWriter, r *http.Request, title string) { + p, err := loadPage(title) + if err != nil { + p = &Page{Title: title} + } + renderTemplate(w, "edit", p) +} + +func saveHandler(w http.ResponseWriter, r *http.Request, title string) { + body := r.FormValue("body") + p := &Page{Title: title, Body: []byte(body)} + err := p.save() + if err != nil { + http.Error(w, err.Error(), http.StatusInternalServerError) + return + } + http.Redirect(w, r, "/view/"+title, http.StatusFound) +} + +var templates = template.Must(template.ParseFiles("edit.html", "view.html")) + +func renderTemplate(w http.ResponseWriter, tmpl string, p *Page) { + err := templates.ExecuteTemplate(w, tmpl+".html", p) + if err != nil { + http.Error(w, err.Error(), http.StatusInternalServerError) + } +} + +var validPath = regexp.MustCompile("^/(edit|save|view)/([a-zA-Z0-9]+)$") + +func makeHandler(fn func(http.ResponseWriter, *http.Request, string)) http.HandlerFunc { + return func(w http.ResponseWriter, r *http.Request) { + m := validPath.FindStringSubmatch(r.URL.Path) + if m == nil { + http.NotFound(w, r) + return + } + fn(w, r, m[2]) + } +} + +func main() { + http.HandleFunc("/view/", makeHandler(viewHandler)) + http.HandleFunc("/edit/", makeHandler(editHandler)) + http.HandleFunc("/save/", makeHandler(saveHandler)) + + log.Fatal(http.ListenAndServe(":8080", nil)) +} diff --git a/_content/doc/articles/wiki/final_test.go b/_content/doc/articles/wiki/final_test.go new file mode 100644 index 00000000..76446997 --- /dev/null +++ b/_content/doc/articles/wiki/final_test.go @@ -0,0 +1,24 @@ +// Copyright 2019 The Go Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + +// +build ignore + +package main + +import ( + "fmt" + "log" + "net" + "net/http" +) + +func serve() error { + l, err := net.Listen("tcp", "127.0.0.1:0") + if err != nil { + log.Fatal(err) + } + fmt.Println(l.Addr().String()) + s := &http.Server{} + return s.Serve(l) +} diff --git a/_content/doc/articles/wiki/go.mod b/_content/doc/articles/wiki/go.mod new file mode 100644 index 00000000..38153ed7 --- /dev/null +++ b/_content/doc/articles/wiki/go.mod @@ -0,0 +1,3 @@ +module doc/articles/wiki + +go 1.14 diff --git a/_content/doc/articles/wiki/http-sample.go b/_content/doc/articles/wiki/http-sample.go new file mode 100644 index 00000000..803b88c4 --- /dev/null +++ b/_content/doc/articles/wiki/http-sample.go @@ -0,0 +1,18 @@ +// +build ignore + +package main + +import ( + "fmt" + "log" + "net/http" +) + +func handler(w http.ResponseWriter, r *http.Request) { + fmt.Fprintf(w, "Hi there, I love %s!", r.URL.Path[1:]) +} + +func main() { + http.HandleFunc("/", handler) + log.Fatal(http.ListenAndServe(":8080", nil)) +} diff --git a/_content/doc/articles/wiki/index.html b/_content/doc/articles/wiki/index.html new file mode 100644 index 00000000..a74a58e3 --- /dev/null +++ b/_content/doc/articles/wiki/index.html @@ -0,0 +1,741 @@ + + +

Introduction

+ +

+Covered in this tutorial: +

+ + +

+Assumed knowledge: +

+ + +

Getting Started

+ +

+At present, you need to have a FreeBSD, Linux, OS X, or Windows machine to run Go. +We will use $ to represent the command prompt. +

+ +

+Install Go (see the Installation Instructions). +

+ +

+Make a new directory for this tutorial inside your GOPATH and cd to it: +

+ +
+$ mkdir gowiki
+$ cd gowiki
+
+ +

+Create a file named wiki.go, open it in your favorite editor, and +add the following lines: +

+ +
+package main
+
+import (
+	"fmt"
+	"io/ioutil"
+)
+
+ +

+We import the fmt and ioutil packages from the Go +standard library. Later, as we implement additional functionality, we will +add more packages to this import declaration. +

+ +

Data Structures

+ +

+Let's start by defining the data structures. A wiki consists of a series of +interconnected pages, each of which has a title and a body (the page content). +Here, we define Page as a struct with two fields representing +the title and body. +

+ +{{code "doc/articles/wiki/part1.go" `/^type Page/` `/}/`}} + +

+The type []byte means "a byte slice". +(See Slices: usage and +internals for more on slices.) +The Body element is a []byte rather than +string because that is the type expected by the io +libraries we will use, as you'll see below. +

+ +

+The Page struct describes how page data will be stored in memory. +But what about persistent storage? We can address that by creating a +save method on Page: +

+ +{{code "doc/articles/wiki/part1.go" `/^func.*Page.*save/` `/}/`}} + +

+This method's signature reads: "This is a method named save that +takes as its receiver p, a pointer to Page . It takes +no parameters, and returns a value of type error." +

+ +

+This method will save the Page's Body to a text +file. For simplicity, we will use the Title as the file name. +

+ +

+The save method returns an error value because +that is the return type of WriteFile (a standard library function +that writes a byte slice to a file). The save method returns the +error value, to let the application handle it should anything go wrong while +writing the file. If all goes well, Page.save() will return +nil (the zero-value for pointers, interfaces, and some other +types). +

+ +

+The octal integer literal 0600, passed as the third parameter to +WriteFile, indicates that the file should be created with +read-write permissions for the current user only. (See the Unix man page +open(2) for details.) +

+ +

+In addition to saving pages, we will want to load pages, too: +

+ +{{code "doc/articles/wiki/part1-noerror.go" `/^func loadPage/` `/^}/`}} + +

+The function loadPage constructs the file name from the title +parameter, reads the file's contents into a new variable body, and +returns a pointer to a Page literal constructed with the proper +title and body values. +

+ +

+Functions can return multiple values. The standard library function +io.ReadFile returns []byte and error. +In loadPage, error isn't being handled yet; the "blank identifier" +represented by the underscore (_) symbol is used to throw away the +error return value (in essence, assigning the value to nothing). +

+ +

+But what happens if ReadFile encounters an error? For example, +the file might not exist. We should not ignore such errors. Let's modify the +function to return *Page and error. +

+ +{{code "doc/articles/wiki/part1.go" `/^func loadPage/` `/^}/`}} + +

+Callers of this function can now check the second parameter; if it is +nil then it has successfully loaded a Page. If not, it will be an +error that can be handled by the caller (see the +language specification for details). +

+ +

+At this point we have a simple data structure and the ability to save to and +load from a file. Let's write a main function to test what we've +written: +

+ +{{code "doc/articles/wiki/part1.go" `/^func main/` `/^}/`}} + +

+After compiling and executing this code, a file named TestPage.txt +would be created, containing the contents of p1. The file would +then be read into the struct p2, and its Body element +printed to the screen. +

+ +

+You can compile and run the program like this: +

+ +
+$ go build wiki.go
+$ ./wiki
+This is a sample Page.
+
+ +

+(If you're using Windows you must type "wiki" without the +"./" to run the program.) +

+ +

+Click here to view the code we've written so far. +

+ +

Introducing the net/http package (an interlude)

+ +

+Here's a full working example of a simple web server: +

+ +{{code "doc/articles/wiki/http-sample.go"}} + +

+The main function begins with a call to +http.HandleFunc, which tells the http package to +handle all requests to the web root ("/") with +handler. +

+ +

+It then calls http.ListenAndServe, specifying that it should +listen on port 8080 on any interface (":8080"). (Don't +worry about its second parameter, nil, for now.) +This function will block until the program is terminated. +

+ +

+ListenAndServe always returns an error, since it only returns when an +unexpected error occurs. +In order to log that error we wrap the function call with log.Fatal. +

+ +

+The function handler is of the type http.HandlerFunc. +It takes an http.ResponseWriter and an http.Request as +its arguments. +

+ +

+An http.ResponseWriter value assembles the HTTP server's response; by writing +to it, we send data to the HTTP client. +

+ +

+An http.Request is a data structure that represents the client +HTTP request. r.URL.Path is the path component +of the request URL. The trailing [1:] means +"create a sub-slice of Path from the 1st character to the end." +This drops the leading "/" from the path name. +

+ +

+If you run this program and access the URL: +

+
http://localhost:8080/monkeys
+

+the program would present a page containing: +

+
Hi there, I love monkeys!
+ +

Using net/http to serve wiki pages

+ +

+To use the net/http package, it must be imported: +

+ +
+import (
+	"fmt"
+	"io/ioutil"
+	"log"
+	"net/http"
+)
+
+ +

+Let's create a handler, viewHandler that will allow users to +view a wiki page. It will handle URLs prefixed with "/view/". +

+ +{{code "doc/articles/wiki/part2.go" `/^func viewHandler/` `/^}/`}} + +

+Again, note the use of _ to ignore the error +return value from loadPage. This is done here for simplicity +and generally considered bad practice. We will attend to this later. +

+ +

+First, this function extracts the page title from r.URL.Path, +the path component of the request URL. +The Path is re-sliced with [len("/view/"):] to drop +the leading "/view/" component of the request path. +This is because the path will invariably begin with "/view/", +which is not part of the page's title. +

+ +

+The function then loads the page data, formats the page with a string of simple +HTML, and writes it to w, the http.ResponseWriter. +

+ +

+To use this handler, we rewrite our main function to +initialize http using the viewHandler to handle +any requests under the path /view/. +

+ +{{code "doc/articles/wiki/part2.go" `/^func main/` `/^}/`}} + +

+Click here to view the code we've written so far. +

+ +

+Let's create some page data (as test.txt), compile our code, and +try serving a wiki page. +

+ +

+Open test.txt file in your editor, and save the string "Hello world" (without quotes) +in it. +

+ +
+$ go build wiki.go
+$ ./wiki
+
+ +

+(If you're using Windows you must type "wiki" without the +"./" to run the program.) +

+ +

+With this web server running, a visit to http://localhost:8080/view/test +should show a page titled "test" containing the words "Hello world". +

+ +

Editing Pages

+ +

+A wiki is not a wiki without the ability to edit pages. Let's create two new +handlers: one named editHandler to display an 'edit page' form, +and the other named saveHandler to save the data entered via the +form. +

+ +

+First, we add them to main(): +

+ +{{code "doc/articles/wiki/final-noclosure.go" `/^func main/` `/^}/`}} + +

+The function editHandler loads the page +(or, if it doesn't exist, create an empty Page struct), +and displays an HTML form. +

+ +{{code "doc/articles/wiki/notemplate.go" `/^func editHandler/` `/^}/`}} + +

+This function will work fine, but all that hard-coded HTML is ugly. +Of course, there is a better way. +

+ +

The html/template package

+ +

+The html/template package is part of the Go standard library. +We can use html/template to keep the HTML in a separate file, +allowing us to change the layout of our edit page without modifying the +underlying Go code. +

+ +

+First, we must add html/template to the list of imports. We +also won't be using fmt anymore, so we have to remove that. +

+ +
+import (
+	"html/template"
+	"io/ioutil"
+	"net/http"
+)
+
+ +

+Let's create a template file containing the HTML form. +Open a new file named edit.html, and add the following lines: +

+ +{{code "doc/articles/wiki/edit.html"}} + +

+Modify editHandler to use the template, instead of the hard-coded +HTML: +

+ +{{code "doc/articles/wiki/final-noerror.go" `/^func editHandler/` `/^}/`}} + +

+The function template.ParseFiles will read the contents of +edit.html and return a *template.Template. +

+ +

+The method t.Execute executes the template, writing the +generated HTML to the http.ResponseWriter. +The .Title and .Body dotted identifiers refer to +p.Title and p.Body. +

+ +

+Template directives are enclosed in double curly braces. +The printf "%s" .Body instruction is a function call +that outputs .Body as a string instead of a stream of bytes, +the same as a call to fmt.Printf. +The html/template package helps guarantee that only safe and +correct-looking HTML is generated by template actions. For instance, it +automatically escapes any greater than sign (>), replacing it +with &gt;, to make sure user data does not corrupt the form +HTML. +

+ +

+Since we're working with templates now, let's create a template for our +viewHandler called view.html: +

+ +{{code "doc/articles/wiki/view.html"}} + +

+Modify viewHandler accordingly: +

+ +{{code "doc/articles/wiki/final-noerror.go" `/^func viewHandler/` `/^}/`}} + +

+Notice that we've used almost exactly the same templating code in both +handlers. Let's remove this duplication by moving the templating code +to its own function: +

+ +{{code "doc/articles/wiki/final-template.go" `/^func renderTemplate/` `/^}/`}} + +

+And modify the handlers to use that function: +

+ +{{code "doc/articles/wiki/final-template.go" `/^func viewHandler/` `/^}/`}} +{{code "doc/articles/wiki/final-template.go" `/^func editHandler/` `/^}/`}} + +

+If we comment out the registration of our unimplemented save handler in +main, we can once again build and test our program. +Click here to view the code we've written so far. +

+ +

Handling non-existent pages

+ +

+What if you visit +/view/APageThatDoesntExist? You'll see a page containing +HTML. This is because it ignores the error return value from +loadPage and continues to try and fill out the template +with no data. Instead, if the requested Page doesn't exist, it should +redirect the client to the edit Page so the content may be created: +

+ +{{code "doc/articles/wiki/part3-errorhandling.go" `/^func viewHandler/` `/^}/`}} + +

+The http.Redirect function adds an HTTP status code of +http.StatusFound (302) and a Location +header to the HTTP response. +

+ +

Saving Pages

+ +

+The function saveHandler will handle the submission of forms +located on the edit pages. After uncommenting the related line in +main, let's implement the handler: +

+ +{{code "doc/articles/wiki/final-template.go" `/^func saveHandler/` `/^}/`}} + +

+The page title (provided in the URL) and the form's only field, +Body, are stored in a new Page. +The save() method is then called to write the data to a file, +and the client is redirected to the /view/ page. +

+ +

+The value returned by FormValue is of type string. +We must convert that value to []byte before it will fit into +the Page struct. We use []byte(body) to perform +the conversion. +

+ +

Error handling

+ +

+There are several places in our program where errors are being ignored. This +is bad practice, not least because when an error does occur the program will +have unintended behavior. A better solution is to handle the errors and return +an error message to the user. That way if something does go wrong, the server +will function exactly how we want and the user can be notified. +

+ +

+First, let's handle the errors in renderTemplate: +

+ +{{code "doc/articles/wiki/final-parsetemplate.go" `/^func renderTemplate/` `/^}/`}} + +

+The http.Error function sends a specified HTTP response code +(in this case "Internal Server Error") and error message. +Already the decision to put this in a separate function is paying off. +

+ +

+Now let's fix up saveHandler: +

+ +{{code "doc/articles/wiki/part3-errorhandling.go" `/^func saveHandler/` `/^}/`}} + +

+Any errors that occur during p.save() will be reported +to the user. +

+ +

Template caching

+ +

+There is an inefficiency in this code: renderTemplate calls +ParseFiles every time a page is rendered. +A better approach would be to call ParseFiles once at program +initialization, parsing all templates into a single *Template. +Then we can use the +ExecuteTemplate +method to render a specific template. +

+ +

+First we create a global variable named templates, and initialize +it with ParseFiles. +

+ +{{code "doc/articles/wiki/final.go" `/var templates/`}} + +

+The function template.Must is a convenience wrapper that panics +when passed a non-nil error value, and otherwise returns the +*Template unaltered. A panic is appropriate here; if the templates +can't be loaded the only sensible thing to do is exit the program. +

+ +

+The ParseFiles function takes any number of string arguments that +identify our template files, and parses those files into templates that are +named after the base file name. If we were to add more templates to our +program, we would add their names to the ParseFiles call's +arguments. +

+ +

+We then modify the renderTemplate function to call the +templates.ExecuteTemplate method with the name of the appropriate +template: +

+ +{{code "doc/articles/wiki/final.go" `/func renderTemplate/` `/^}/`}} + +

+Note that the template name is the template file name, so we must +append ".html" to the tmpl argument. +

+ +

Validation

+ +

+As you may have observed, this program has a serious security flaw: a user +can supply an arbitrary path to be read/written on the server. To mitigate +this, we can write a function to validate the title with a regular expression. +

+ +

+First, add "regexp" to the import list. +Then we can create a global variable to store our validation +expression: +

+ +{{code "doc/articles/wiki/final-noclosure.go" `/^var validPath/`}} + +

+The function regexp.MustCompile will parse and compile the +regular expression, and return a regexp.Regexp. +MustCompile is distinct from Compile in that it will +panic if the expression compilation fails, while Compile returns +an error as a second parameter. +

+ +

+Now, let's write a function that uses the validPath +expression to validate path and extract the page title: +

+ +{{code "doc/articles/wiki/final-noclosure.go" `/func getTitle/` `/^}/`}} + +

+If the title is valid, it will be returned along with a nil +error value. If the title is invalid, the function will write a +"404 Not Found" error to the HTTP connection, and return an error to the +handler. To create a new error, we have to import the errors +package. +

+ +

+Let's put a call to getTitle in each of the handlers: +

+ +{{code "doc/articles/wiki/final-noclosure.go" `/^func viewHandler/` `/^}/`}} +{{code "doc/articles/wiki/final-noclosure.go" `/^func editHandler/` `/^}/`}} +{{code "doc/articles/wiki/final-noclosure.go" `/^func saveHandler/` `/^}/`}} + +

Introducing Function Literals and Closures

+ +

+Catching the error condition in each handler introduces a lot of repeated code. +What if we could wrap each of the handlers in a function that does this +validation and error checking? Go's +function +literals provide a powerful means of abstracting functionality +that can help us here. +

+ +

+First, we re-write the function definition of each of the handlers to accept +a title string: +

+ +
+func viewHandler(w http.ResponseWriter, r *http.Request, title string)
+func editHandler(w http.ResponseWriter, r *http.Request, title string)
+func saveHandler(w http.ResponseWriter, r *http.Request, title string)
+
+ +

+Now let's define a wrapper function that takes a function of the above +type, and returns a function of type http.HandlerFunc +(suitable to be passed to the function http.HandleFunc): +

+ +
+func makeHandler(fn func (http.ResponseWriter, *http.Request, string)) http.HandlerFunc {
+	return func(w http.ResponseWriter, r *http.Request) {
+		// Here we will extract the page title from the Request,
+		// and call the provided handler 'fn'
+	}
+}
+
+ +

+The returned function is called a closure because it encloses values defined +outside of it. In this case, the variable fn (the single argument +to makeHandler) is enclosed by the closure. The variable +fn will be one of our save, edit, or view handlers. +

+ +

+Now we can take the code from getTitle and use it here +(with some minor modifications): +

+ +{{code "doc/articles/wiki/final.go" `/func makeHandler/` `/^}/`}} + +

+The closure returned by makeHandler is a function that takes +an http.ResponseWriter and http.Request (in other +words, an http.HandlerFunc). +The closure extracts the title from the request path, and +validates it with the validPath regexp. If the +title is invalid, an error will be written to the +ResponseWriter using the http.NotFound function. +If the title is valid, the enclosed handler function +fn will be called with the ResponseWriter, +Request, and title as arguments. +

+ +

+Now we can wrap the handler functions with makeHandler in +main, before they are registered with the http +package: +

+ +{{code "doc/articles/wiki/final.go" `/func main/` `/^}/`}} + +

+Finally we remove the calls to getTitle from the handler functions, +making them much simpler: +

+ +{{code "doc/articles/wiki/final.go" `/^func viewHandler/` `/^}/`}} +{{code "doc/articles/wiki/final.go" `/^func editHandler/` `/^}/`}} +{{code "doc/articles/wiki/final.go" `/^func saveHandler/` `/^}/`}} + +

Try it out!

+ +

+Click here to view the final code listing. +

+ +

+Recompile the code, and run the app: +

+ +
+$ go build wiki.go
+$ ./wiki
+
+ +

+Visiting http://localhost:8080/view/ANewPage +should present you with the page edit form. You should then be able to +enter some text, click 'Save', and be redirected to the newly created page. +

+ +

Other tasks

+ +

+Here are some simple tasks you might want to tackle on your own: +

+ + diff --git a/_content/doc/articles/wiki/notemplate.go b/_content/doc/articles/wiki/notemplate.go new file mode 100644 index 00000000..4b358f29 --- /dev/null +++ b/_content/doc/articles/wiki/notemplate.go @@ -0,0 +1,59 @@ +// Copyright 2010 The Go Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + +// +build ignore + +package main + +import ( + "fmt" + "io/ioutil" + "log" + "net/http" +) + +type Page struct { + Title string + Body []byte +} + +func (p *Page) save() error { + filename := p.Title + ".txt" + return ioutil.WriteFile(filename, p.Body, 0600) +} + +func loadPage(title string) (*Page, error) { + filename := title + ".txt" + body, err := ioutil.ReadFile(filename) + if err != nil { + return nil, err + } + return &Page{Title: title, Body: body}, nil +} + +func viewHandler(w http.ResponseWriter, r *http.Request) { + title := r.URL.Path[len("/view/"):] + p, _ := loadPage(title) + fmt.Fprintf(w, "

%s

%s
", p.Title, p.Body) +} + +func editHandler(w http.ResponseWriter, r *http.Request) { + title := r.URL.Path[len("/edit/"):] + p, err := loadPage(title) + if err != nil { + p = &Page{Title: title} + } + fmt.Fprintf(w, "

Editing %s

"+ + "
"+ + "
"+ + ""+ + "
", + p.Title, p.Title, p.Body) +} + +func main() { + http.HandleFunc("/view/", viewHandler) + http.HandleFunc("/edit/", editHandler) + log.Fatal(http.ListenAndServe(":8080", nil)) +} diff --git a/_content/doc/articles/wiki/part1-noerror.go b/_content/doc/articles/wiki/part1-noerror.go new file mode 100644 index 00000000..913c6dce --- /dev/null +++ b/_content/doc/articles/wiki/part1-noerror.go @@ -0,0 +1,35 @@ +// Copyright 2010 The Go Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + +// +build ignore + +package main + +import ( + "fmt" + "io/ioutil" +) + +type Page struct { + Title string + Body []byte +} + +func (p *Page) save() error { + filename := p.Title + ".txt" + return ioutil.WriteFile(filename, p.Body, 0600) +} + +func loadPage(title string) *Page { + filename := title + ".txt" + body, _ := ioutil.ReadFile(filename) + return &Page{Title: title, Body: body} +} + +func main() { + p1 := &Page{Title: "TestPage", Body: []byte("This is a sample page.")} + p1.save() + p2 := loadPage("TestPage") + fmt.Println(string(p2.Body)) +} diff --git a/_content/doc/articles/wiki/part1.go b/_content/doc/articles/wiki/part1.go new file mode 100644 index 00000000..2ff1abd2 --- /dev/null +++ b/_content/doc/articles/wiki/part1.go @@ -0,0 +1,38 @@ +// Copyright 2010 The Go Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + +// +build ignore + +package main + +import ( + "fmt" + "io/ioutil" +) + +type Page struct { + Title string + Body []byte +} + +func (p *Page) save() error { + filename := p.Title + ".txt" + return ioutil.WriteFile(filename, p.Body, 0600) +} + +func loadPage(title string) (*Page, error) { + filename := title + ".txt" + body, err := ioutil.ReadFile(filename) + if err != nil { + return nil, err + } + return &Page{Title: title, Body: body}, nil +} + +func main() { + p1 := &Page{Title: "TestPage", Body: []byte("This is a sample Page.")} + p1.save() + p2, _ := loadPage("TestPage") + fmt.Println(string(p2.Body)) +} diff --git a/_content/doc/articles/wiki/part2.go b/_content/doc/articles/wiki/part2.go new file mode 100644 index 00000000..db92f4c7 --- /dev/null +++ b/_content/doc/articles/wiki/part2.go @@ -0,0 +1,44 @@ +// Copyright 2010 The Go Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + +// +build ignore + +package main + +import ( + "fmt" + "io/ioutil" + "log" + "net/http" +) + +type Page struct { + Title string + Body []byte +} + +func (p *Page) save() error { + filename := p.Title + ".txt" + return ioutil.WriteFile(filename, p.Body, 0600) +} + +func loadPage(title string) (*Page, error) { + filename := title + ".txt" + body, err := ioutil.ReadFile(filename) + if err != nil { + return nil, err + } + return &Page{Title: title, Body: body}, nil +} + +func viewHandler(w http.ResponseWriter, r *http.Request) { + title := r.URL.Path[len("/view/"):] + p, _ := loadPage(title) + fmt.Fprintf(w, "

%s

%s
", p.Title, p.Body) +} + +func main() { + http.HandleFunc("/view/", viewHandler) + log.Fatal(http.ListenAndServe(":8080", nil)) +} diff --git a/_content/doc/articles/wiki/part3-errorhandling.go b/_content/doc/articles/wiki/part3-errorhandling.go new file mode 100644 index 00000000..2c8b42d0 --- /dev/null +++ b/_content/doc/articles/wiki/part3-errorhandling.go @@ -0,0 +1,76 @@ +// Copyright 2010 The Go Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + +// +build ignore + +package main + +import ( + "html/template" + "io/ioutil" + "log" + "net/http" +) + +type Page struct { + Title string + Body []byte +} + +func (p *Page) save() error { + filename := p.Title + ".txt" + return ioutil.WriteFile(filename, p.Body, 0600) +} + +func loadPage(title string) (*Page, error) { + filename := title + ".txt" + body, err := ioutil.ReadFile(filename) + if err != nil { + return nil, err + } + return &Page{Title: title, Body: body}, nil +} + +func renderTemplate(w http.ResponseWriter, tmpl string, p *Page) { + t, _ := template.ParseFiles(tmpl + ".html") + t.Execute(w, p) +} + +func viewHandler(w http.ResponseWriter, r *http.Request) { + title := r.URL.Path[len("/view/"):] + p, err := loadPage(title) + if err != nil { + http.Redirect(w, r, "/edit/"+title, http.StatusFound) + return + } + renderTemplate(w, "view", p) +} + +func editHandler(w http.ResponseWriter, r *http.Request) { + title := r.URL.Path[len("/edit/"):] + p, err := loadPage(title) + if err != nil { + p = &Page{Title: title} + } + renderTemplate(w, "edit", p) +} + +func saveHandler(w http.ResponseWriter, r *http.Request) { + title := r.URL.Path[len("/save/"):] + body := r.FormValue("body") + p := &Page{Title: title, Body: []byte(body)} + err := p.save() + if err != nil { + http.Error(w, err.Error(), http.StatusInternalServerError) + return + } + http.Redirect(w, r, "/view/"+title, http.StatusFound) +} + +func main() { + http.HandleFunc("/view/", viewHandler) + http.HandleFunc("/edit/", editHandler) + http.HandleFunc("/save/", saveHandler) + log.Fatal(http.ListenAndServe(":8080", nil)) +} diff --git a/_content/doc/articles/wiki/part3.go b/_content/doc/articles/wiki/part3.go new file mode 100644 index 00000000..437ea336 --- /dev/null +++ b/_content/doc/articles/wiki/part3.go @@ -0,0 +1,60 @@ +// Copyright 2010 The Go Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + +// +build ignore + +package main + +import ( + "html/template" + "io/ioutil" + "log" + "net/http" +) + +type Page struct { + Title string + Body []byte +} + +func (p *Page) save() error { + filename := p.Title + ".txt" + return ioutil.WriteFile(filename, p.Body, 0600) +} + +func loadPage(title string) (*Page, error) { + filename := title + ".txt" + body, err := ioutil.ReadFile(filename) + if err != nil { + return nil, err + } + return &Page{Title: title, Body: body}, nil +} + +func renderTemplate(w http.ResponseWriter, tmpl string, p *Page) { + t, _ := template.ParseFiles(tmpl + ".html") + t.Execute(w, p) +} + +func viewHandler(w http.ResponseWriter, r *http.Request) { + title := r.URL.Path[len("/view/"):] + p, _ := loadPage(title) + renderTemplate(w, "view", p) +} + +func editHandler(w http.ResponseWriter, r *http.Request) { + title := r.URL.Path[len("/edit/"):] + p, err := loadPage(title) + if err != nil { + p = &Page{Title: title} + } + renderTemplate(w, "edit", p) +} + +func main() { + http.HandleFunc("/view/", viewHandler) + http.HandleFunc("/edit/", editHandler) + //http.HandleFunc("/save/", saveHandler) + log.Fatal(http.ListenAndServe(":8080", nil)) +} diff --git a/_content/doc/articles/wiki/test_Test.txt.good b/_content/doc/articles/wiki/test_Test.txt.good new file mode 100644 index 00000000..f0eec86f --- /dev/null +++ b/_content/doc/articles/wiki/test_Test.txt.good @@ -0,0 +1 @@ +some content \ No newline at end of file diff --git a/_content/doc/articles/wiki/test_edit.good b/_content/doc/articles/wiki/test_edit.good new file mode 100644 index 00000000..36c6dbb7 --- /dev/null +++ b/_content/doc/articles/wiki/test_edit.good @@ -0,0 +1,6 @@ +

Editing Test

+ +
+
+
+
diff --git a/_content/doc/articles/wiki/test_view.good b/_content/doc/articles/wiki/test_view.good new file mode 100644 index 00000000..07e8edb2 --- /dev/null +++ b/_content/doc/articles/wiki/test_view.good @@ -0,0 +1,5 @@ +

Test

+ +

[edit]

+ +
some content
diff --git a/_content/doc/articles/wiki/view.html b/_content/doc/articles/wiki/view.html new file mode 100644 index 00000000..b1e87efe --- /dev/null +++ b/_content/doc/articles/wiki/view.html @@ -0,0 +1,5 @@ +

{{.Title}}

+ +

[edit]

+ +
{{printf "%s" .Body}}
diff --git a/_content/doc/articles/wiki/wiki_test.go b/_content/doc/articles/wiki/wiki_test.go new file mode 100644 index 00000000..1d976fd7 --- /dev/null +++ b/_content/doc/articles/wiki/wiki_test.go @@ -0,0 +1,165 @@ +// Copyright 2019 The Go Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + +package main_test + +import ( + "bytes" + "fmt" + "io/ioutil" + "net/http" + "os" + "os/exec" + "path/filepath" + "strings" + "testing" +) + +func TestSnippetsCompile(t *testing.T) { + if testing.Short() { + t.Skip("skipping slow builds in short mode") + } + + goFiles, err := filepath.Glob("*.go") + if err != nil { + t.Fatal(err) + } + + for _, f := range goFiles { + if strings.HasSuffix(f, "_test.go") { + continue + } + f := f + t.Run(f, func(t *testing.T) { + t.Parallel() + + cmd := exec.Command("go", "build", "-o", os.DevNull, f) + out, err := cmd.CombinedOutput() + if err != nil { + t.Errorf("%s: %v\n%s", strings.Join(cmd.Args, " "), err, out) + } + }) + } +} + +func TestWikiServer(t *testing.T) { + must := func(err error) { + if err != nil { + t.Helper() + t.Fatal(err) + } + } + + dir, err := ioutil.TempDir("", t.Name()) + must(err) + defer os.RemoveAll(dir) + + // We're testing a walkthrough example of how to write a server. + // + // That server hard-codes a port number to make the walkthrough simpler, but + // we can't assume that the hard-coded port is available on an arbitrary + // builder. So we'll patch out the hard-coded port, and replace it with a + // function that writes the server's address to stdout + // so that we can read it and know where to send the test requests. + + finalGo, err := ioutil.ReadFile("final.go") + must(err) + const patchOld = `log.Fatal(http.ListenAndServe(":8080", nil))` + patched := bytes.ReplaceAll(finalGo, []byte(patchOld), []byte(`log.Fatal(serve())`)) + if bytes.Equal(patched, finalGo) { + t.Fatalf("Can't patch final.go: %q not found.", patchOld) + } + must(ioutil.WriteFile(filepath.Join(dir, "final_patched.go"), patched, 0644)) + + // Build the server binary from the patched sources. + // The 'go' command requires that they all be in the same directory. + // final_test.go provides the implemtation for our serve function. + must(copyFile(filepath.Join(dir, "final_srv.go"), "final_test.go")) + cmd := exec.Command("go", "build", + "-o", filepath.Join(dir, "final.exe"), + filepath.Join(dir, "final_patched.go"), + filepath.Join(dir, "final_srv.go")) + out, err := cmd.CombinedOutput() + if err != nil { + t.Fatalf("%s: %v\n%s", strings.Join(cmd.Args, " "), err, out) + } + + // Run the server in our temporary directory so that it can + // write its content there. It also needs a couple of template files, + // and looks for them in the same directory. + must(copyFile(filepath.Join(dir, "edit.html"), "edit.html")) + must(copyFile(filepath.Join(dir, "view.html"), "view.html")) + cmd = exec.Command(filepath.Join(dir, "final.exe")) + cmd.Dir = dir + stderr := bytes.NewBuffer(nil) + cmd.Stderr = stderr + stdout, err := cmd.StdoutPipe() + must(err) + must(cmd.Start()) + + defer func() { + cmd.Process.Kill() + err := cmd.Wait() + if stderr.Len() > 0 { + t.Logf("%s: %v\n%s", strings.Join(cmd.Args, " "), err, stderr) + } + }() + + var addr string + if _, err := fmt.Fscanln(stdout, &addr); err != nil || addr == "" { + t.Fatalf("Failed to read server address: %v", err) + } + + // The server is up and has told us its address. + // Make sure that its HTTP API works as described in the article. + + r, err := http.Get(fmt.Sprintf("http://%s/edit/Test", addr)) + must(err) + responseMustMatchFile(t, r, "test_edit.good") + + r, err = http.Post(fmt.Sprintf("http://%s/save/Test", addr), + "application/x-www-form-urlencoded", + strings.NewReader("body=some%20content")) + must(err) + responseMustMatchFile(t, r, "test_view.good") + + gotTxt, err := ioutil.ReadFile(filepath.Join(dir, "Test.txt")) + must(err) + wantTxt, err := ioutil.ReadFile("test_Test.txt.good") + must(err) + if !bytes.Equal(wantTxt, gotTxt) { + t.Fatalf("Test.txt differs from expected after posting to /save.\ngot:\n%s\nwant:\n%s", gotTxt, wantTxt) + } + + r, err = http.Get(fmt.Sprintf("http://%s/view/Test", addr)) + must(err) + responseMustMatchFile(t, r, "test_view.good") +} + +func responseMustMatchFile(t *testing.T, r *http.Response, filename string) { + t.Helper() + + defer r.Body.Close() + body, err := ioutil.ReadAll(r.Body) + if err != nil { + t.Fatal(err) + } + + wantBody, err := ioutil.ReadFile(filename) + if err != nil { + t.Fatal(err) + } + + if !bytes.Equal(body, wantBody) { + t.Fatalf("%v: body does not match %s.\ngot:\n%s\nwant:\n%s", r.Request.URL, filename, body, wantBody) + } +} + +func copyFile(dst, src string) error { + buf, err := ioutil.ReadFile(src) + if err != nil { + return err + } + return ioutil.WriteFile(dst, buf, 0644) +} diff --git a/_content/doc/cmd.html b/_content/doc/cmd.html new file mode 100644 index 00000000..c3bd9181 --- /dev/null +++ b/_content/doc/cmd.html @@ -0,0 +1,100 @@ + + +

+There is a suite of programs to build and process Go source code. +Instead of being run directly, programs in the suite are usually invoked +by the go program. +

+ +

+The most common way to run these programs is as a subcommand of the go program, +for instance as go fmt. Run like this, the command operates on +complete packages of Go source code, with the go program invoking the +underlying binary with arguments appropriate to package-level processing. +

+ +

+The programs can also be run as stand-alone binaries, with unmodified arguments, +using the go tool subcommand, such as go tool cgo. +For most commands this is mainly useful for debugging. +Some of the commands, such as pprof, are accessible only through +the go tool subcommand. +

+ +

+Finally the fmt and godoc commands are installed +as regular binaries called gofmt and godoc because +they are so often referenced. +

+ +

+Click on the links for more documentation, invocation methods, and usage details. +

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Name    Synopsis
go     +The go program manages Go source code and runs the other +commands listed here. +See the command docs for usage +details. +
cgo    Cgo enables the creation of Go packages that call C code.
cover    Cover is a program for creating and analyzing the coverage profiles +generated by "go test -coverprofile".
fix    Fix finds Go programs that use old features of the language and libraries +and rewrites them to use newer ones.
fmt    Fmt formats Go packages, it is also available as an independent +gofmt command with more general options.
godoc    Godoc extracts and generates documentation for Go packages.
vet    Vet examines Go source code and reports suspicious constructs, such as Printf +calls whose arguments do not align with the format string.
+ +

+This is an abridged list. See the full command reference +for documentation of the compilers and more. +

diff --git a/_content/doc/codewalk/codewalk.css b/_content/doc/codewalk/codewalk.css new file mode 100644 index 00000000..a0814e4d --- /dev/null +++ b/_content/doc/codewalk/codewalk.css @@ -0,0 +1,234 @@ +/* + Copyright 2010 The Go Authors. All rights reserved. + Use of this source code is governed by a BSD-style + license that can be found in the LICENSE file. +*/ + +#codewalk-main { + text-align: left; + width: 100%; + overflow: auto; +} + +#code-display { + border: 0; + width: 100%; +} + +.setting { + font-size: 8pt; + color: #888888; + padding: 5px; +} + +.hotkey { + text-decoration: underline; +} + +/* Style for Comments (the left-hand column) */ + +#comment-column { + margin: 0pt; + width: 30%; +} + +#comment-column.right { + float: right; +} + +#comment-column.left { + float: left; +} + +#comment-area { + overflow-x: hidden; + overflow-y: auto; +} + +.comment { + cursor: pointer; + font-size: 16px; + border: 2px solid #ba9836; + margin-bottom: 10px; + margin-right: 10px; /* yes, for both .left and .right */ +} + +.comment:last-child { + margin-bottom: 0px; +} + +.right .comment { + margin-left: 10px; +} + +.right .comment.first { +} + +.right .comment.last { +} + +.left .comment.first { +} + +.left .comment.last { +} + +.comment.selected { + border-color: #99b2cb; +} + +.right .comment.selected { + border-left-width: 12px; + margin-left: 0px; +} + +.left .comment.selected { + border-right-width: 12px; + margin-right: 0px; +} + +.comment-link { + display: none; +} + +.comment-title { + font-size: small; + font-weight: bold; + background-color: #fffff0; + padding-right: 10px; + padding-left: 10px; + padding-top: 5px; + padding-bottom: 5px; +} + +.right .comment-title { +} + +.left .comment-title { +} + +.comment.selected .comment-title { + background-color: #f8f8ff; +} + +.comment-text { + overflow: auto; + padding-left: 10px; + padding-right: 10px; + padding-top: 10px; + padding-bottom: 5px; + font-size: small; + line-height: 1.3em; +} + +.comment-text p { + margin-top: 0em; + margin-bottom: 0.5em; +} + +.comment-text p:last-child { + margin-bottom: 0em; +} + +.file-name { + font-size: x-small; + padding-top: 0px; + padding-bottom: 5px; +} + +.hidden-filepaths .file-name { + display: none; +} + +.path-dir { + color: #555; +} + +.path-file { + color: #555; +} + + +/* Style for Code (the right-hand column) */ + +/* Wrapper for the code column to make widths get calculated correctly */ +#code-column { + display: block; + position: relative; + margin: 0pt; + width: 70%; +} + +#code-column.left { + float: left; +} + +#code-column.right { + float: right; +} + +#code-area { + background-color: #f8f8ff; + border: 2px solid #99b2cb; + padding: 5px; +} + +.left #code-area { + margin-right: -1px; +} + +.right #code-area { + margin-left: -1px; +} + +#code-header { + margin-bottom: 5px; +} + +#code { + background-color: white; +} + +code { + font-size: 100%; +} + +.codewalkhighlight { + font-weight: bold; + background-color: #f8f8ff; +} + +#code-display { + margin-top: 0px; + margin-bottom: 0px; +} + +#sizer { + position: absolute; + cursor: col-resize; + left: 0px; + top: 0px; + width: 8px; +} + +/* Style for options (bottom strip) */ + +#code-options { + display: none; +} + +#code-options > span { + padding-right: 20px; +} + +#code-options .selected { + border-bottom: 1px dotted; +} + +#comment-options { + text-align: center; +} + +div#content { + padding-bottom: 0em; +} diff --git a/_content/doc/codewalk/codewalk.js b/_content/doc/codewalk/codewalk.js new file mode 100644 index 00000000..4f59a8fc --- /dev/null +++ b/_content/doc/codewalk/codewalk.js @@ -0,0 +1,305 @@ +// Copyright 2010 The Go Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + +/** + * A class to hold information about the Codewalk Viewer. + * @param {jQuery} context The top element in whose context the viewer should + * operate. It will not touch any elements above this one. + * @constructor + */ + var CodewalkViewer = function(context) { + this.context = context; + + /** + * The div that contains all of the comments and their controls. + */ + this.commentColumn = this.context.find('#comment-column'); + + /** + * The div that contains the comments proper. + */ + this.commentArea = this.context.find('#comment-area'); + + /** + * The div that wraps the iframe with the code, as well as the drop down menu + * listing the different files. + * @type {jQuery} + */ + this.codeColumn = this.context.find('#code-column'); + + /** + * The div that contains the code but excludes the options strip. + * @type {jQuery} + */ + this.codeArea = this.context.find('#code-area'); + + /** + * The iframe that holds the code (from Sourcerer). + * @type {jQuery} + */ + this.codeDisplay = this.context.find('#code-display'); + + /** + * The overlaid div used as a grab handle for sizing the code/comment panes. + * @type {jQuery} + */ + this.sizer = this.context.find('#sizer'); + + /** + * The full-screen overlay that ensures we don't lose track of the mouse + * while dragging. + * @type {jQuery} + */ + this.overlay = this.context.find('#overlay'); + + /** + * The hidden input field that we use to hold the focus so that we can detect + * shortcut keypresses. + * @type {jQuery} + */ + this.shortcutInput = this.context.find('#shortcut-input'); + + /** + * The last comment that was selected. + * @type {jQuery} + */ + this.lastSelected = null; +}; + +/** + * Minimum width of the comments or code pane, in pixels. + * @type {number} + */ +CodewalkViewer.MIN_PANE_WIDTH = 200; + +/** + * Navigate the code iframe to the given url and update the code popout link. + * @param {string} url The target URL. + * @param {Object} opt_window Window dependency injection for testing only. + */ +CodewalkViewer.prototype.navigateToCode = function(url, opt_window) { + if (!opt_window) opt_window = window; + // Each iframe is represented by two distinct objects in the DOM: an iframe + // object and a window object. These do not expose the same capabilities. + // Here we need to get the window representation to get the location member, + // so we access it directly through window[] since jQuery returns the iframe + // representation. + // We replace location rather than set so as not to create a history for code + // navigation. + opt_window['code-display'].location.replace(url); + var k = url.indexOf('&'); + if (k != -1) url = url.slice(0, k); + k = url.indexOf('fileprint='); + if (k != -1) url = url.slice(k+10, url.length); + this.context.find('#code-popout-link').attr('href', url); +}; + +/** + * Selects the first comment from the list and forces a refresh of the code + * view. + */ +CodewalkViewer.prototype.selectFirstComment = function() { + // TODO(rsc): handle case where there are no comments + var firstSourcererLink = this.context.find('.comment:first'); + this.changeSelectedComment(firstSourcererLink); +}; + +/** + * Sets the target on all links nested inside comments to be _blank. + */ +CodewalkViewer.prototype.targetCommentLinksAtBlank = function() { + this.context.find('.comment a[href], #description a[href]').each(function() { + if (!this.target) this.target = '_blank'; + }); +}; + +/** + * Installs event handlers for all the events we care about. + */ +CodewalkViewer.prototype.installEventHandlers = function() { + var self = this; + + this.context.find('.comment') + .click(function(event) { + if (jQuery(event.target).is('a[href]')) return true; + self.changeSelectedComment(jQuery(this)); + return false; + }); + + this.context.find('#code-selector') + .change(function() {self.navigateToCode(jQuery(this).val());}); + + this.context.find('#description-table .quote-feet.setting') + .click(function() {self.toggleDescription(jQuery(this)); return false;}); + + this.sizer + .mousedown(function(ev) {self.startSizerDrag(ev); return false;}); + this.overlay + .mouseup(function(ev) {self.endSizerDrag(ev); return false;}) + .mousemove(function(ev) {self.handleSizerDrag(ev); return false;}); + + this.context.find('#prev-comment') + .click(function() { + self.changeSelectedComment(self.lastSelected.prev()); return false; + }); + + this.context.find('#next-comment') + .click(function() { + self.changeSelectedComment(self.lastSelected.next()); return false; + }); + + // Workaround for Firefox 2 and 3, which steal focus from the main document + // whenever the iframe content is (re)loaded. The input field is not shown, + // but is a way for us to bring focus back to a place where we can detect + // keypresses. + this.context.find('#code-display') + .load(function(ev) {self.shortcutInput.focus();}); + + jQuery(document).keypress(function(ev) { + switch(ev.which) { + case 110: // 'n' + self.changeSelectedComment(self.lastSelected.next()); + return false; + case 112: // 'p' + self.changeSelectedComment(self.lastSelected.prev()); + return false; + default: // ignore + } + }); + + window.onresize = function() {self.updateHeight();}; +}; + +/** + * Starts dragging the pane sizer. + * @param {Object} ev The mousedown event that started us dragging. + */ +CodewalkViewer.prototype.startSizerDrag = function(ev) { + this.initialCodeWidth = this.codeColumn.width(); + this.initialCommentsWidth = this.commentColumn.width(); + this.initialMouseX = ev.pageX; + this.overlay.show(); +}; + +/** + * Handles dragging the pane sizer. + * @param {Object} ev The mousemove event updating dragging position. + */ +CodewalkViewer.prototype.handleSizerDrag = function(ev) { + var delta = ev.pageX - this.initialMouseX; + if (this.codeColumn.is('.right')) delta = -delta; + var proposedCodeWidth = this.initialCodeWidth + delta; + var proposedCommentWidth = this.initialCommentsWidth - delta; + var mw = CodewalkViewer.MIN_PANE_WIDTH; + if (proposedCodeWidth < mw) delta = mw - this.initialCodeWidth; + if (proposedCommentWidth < mw) delta = this.initialCommentsWidth - mw; + proposedCodeWidth = this.initialCodeWidth + delta; + proposedCommentWidth = this.initialCommentsWidth - delta; + // If window is too small, don't even try to resize. + if (proposedCodeWidth < mw || proposedCommentWidth < mw) return; + this.codeColumn.width(proposedCodeWidth); + this.commentColumn.width(proposedCommentWidth); + this.options.codeWidth = parseInt( + this.codeColumn.width() / + (this.codeColumn.width() + this.commentColumn.width()) * 100); + this.context.find('#code-column-width').text(this.options.codeWidth + '%'); +}; + +/** + * Ends dragging the pane sizer. + * @param {Object} ev The mouseup event that caused us to stop dragging. + */ +CodewalkViewer.prototype.endSizerDrag = function(ev) { + this.overlay.hide(); + this.updateHeight(); +}; + +/** + * Toggles the Codewalk description between being shown and hidden. + * @param {jQuery} target The target that was clicked to trigger this function. + */ +CodewalkViewer.prototype.toggleDescription = function(target) { + var description = this.context.find('#description'); + description.toggle(); + target.find('span').text(description.is(':hidden') ? 'show' : 'hide'); + this.updateHeight(); +}; + +/** + * Changes the side of the window on which the code is shown and saves the + * setting in a cookie. + * @param {string?} codeSide The side on which the code should be, either + * 'left' or 'right'. + */ +CodewalkViewer.prototype.changeCodeSide = function(codeSide) { + var commentSide = codeSide == 'left' ? 'right' : 'left'; + this.context.find('#set-code-' + codeSide).addClass('selected'); + this.context.find('#set-code-' + commentSide).removeClass('selected'); + // Remove previous side class and add new one. + this.codeColumn.addClass(codeSide).removeClass(commentSide); + this.commentColumn.addClass(commentSide).removeClass(codeSide); + this.sizer.css(codeSide, 'auto').css(commentSide, 0); + this.options.codeSide = codeSide; +}; + +/** + * Adds selected class to newly selected comment, removes selected style from + * previously selected comment, changes drop down options so that the correct + * file is selected, and updates the code popout link. + * @param {jQuery} target The target that was clicked to trigger this function. + */ +CodewalkViewer.prototype.changeSelectedComment = function(target) { + var currentFile = target.find('.comment-link').attr('href'); + if (!currentFile) return; + + if (!(this.lastSelected && this.lastSelected.get(0) === target.get(0))) { + if (this.lastSelected) this.lastSelected.removeClass('selected'); + target.addClass('selected'); + this.lastSelected = target; + var targetTop = target.position().top; + var parentTop = target.parent().position().top; + if (targetTop + target.height() > parentTop + target.parent().height() || + targetTop < parentTop) { + var delta = targetTop - parentTop; + target.parent().animate( + {'scrollTop': target.parent().scrollTop() + delta}, + Math.max(delta / 2, 200), 'swing'); + } + var fname = currentFile.match(/(?:select=|fileprint=)\/[^&]+/)[0]; + fname = fname.slice(fname.indexOf('=')+2, fname.length); + this.context.find('#code-selector').val(fname); + this.context.find('#prev-comment').toggleClass( + 'disabled', !target.prev().length); + this.context.find('#next-comment').toggleClass( + 'disabled', !target.next().length); + } + + // Force original file even if user hasn't changed comments since they may + // have navigated away from it within the iframe without us knowing. + this.navigateToCode(currentFile); +}; + +/** + * Updates the viewer by changing the height of the comments and code so that + * they fit within the height of the window. The function is typically called + * after the user changes the window size. + */ +CodewalkViewer.prototype.updateHeight = function() { + var windowHeight = jQuery(window).height() - 5 // GOK + var areaHeight = windowHeight - this.codeArea.offset().top + var footerHeight = this.context.find('#footer').outerHeight(true) + this.commentArea.height(areaHeight - footerHeight - this.context.find('#comment-options').outerHeight(true)) + var codeHeight = areaHeight - footerHeight - 15 // GOK + this.codeArea.height(codeHeight) + this.codeDisplay.height(codeHeight - this.codeDisplay.offset().top + this.codeArea.offset().top); + this.sizer.height(codeHeight); +}; + +window.initFuncs.push(function() { + var viewer = new CodewalkViewer(jQuery('#codewalk-main')); + viewer.selectFirstComment(); + viewer.targetCommentLinksAtBlank(); + viewer.installEventHandlers(); + viewer.updateHeight(); +}); diff --git a/_content/doc/codewalk/codewalk.xml b/_content/doc/codewalk/codewalk.xml new file mode 100644 index 00000000..34e6e919 --- /dev/null +++ b/_content/doc/codewalk/codewalk.xml @@ -0,0 +1,124 @@ + + + + A codewalk is a guided tour through a piece of code. + It consists of a sequence of steps, each typically explaining + a highlighted section of code. +

+ + The godoc web server translates + an XML file like the one in the main window pane into the HTML + page that you're viewing now. +

+ + The codewalk with URL path /doc/codewalk/name + is loaded from the input file $GOROOT/doc/codewalk/name.xml. +

+ + This codewalk explains how to write a codewalk by examining + its own source code, + $GOROOT/doc/codewalk/codewalk.xml, + shown in the main window pane to the left. +
+ + + The codewalk input file is an XML file containing a single + <codewalk> element. + That element's title attribute gives the title + that is used both on the codewalk page and in the codewalk list. + + + + Each step in the codewalk is a <step> element + nested inside the main <codewalk>. + The step element's title attribute gives the step's title, + which is shown in a shaded bar above the main step text. + The element's src attribute specifies the source + code to show in the main window pane and, optionally, a range of + lines to highlight. +

+ + The first step in this codewalk does not highlight any lines: + its src is just a file name. +
+ + + The most complex part of the codewalk specification is + saying what lines to highlight. + Instead of ordinary line numbers, + the codewalk uses an address syntax that makes it possible + to describe the match by its content. + As the file gets edited, this descriptive address has a better + chance to continue to refer to the right section of the file. +

+ + To specify a source line, use a src attribute of the form + filename:address, + where address is an address in the syntax used by the text editors sam and acme. +

+ + The simplest address is a single regular expression. + The highlighted line in the main window pane shows that the + address for the “Title” step was /title=/, + which matches the first instance of that regular expression (title=) in the file. +
+ + + To highlight a range of source lines, the simplest address to use is + a pair of regular expressions + /regexp1/,/regexp2/. + The highlight begins with the line containing the first match for regexp1 + and ends with the line containing the first match for regexp2 + after the end of the match for regexp1. + Ignoring the HTML quoting, + The line containing the first match for regexp1 will be the first one highlighted, + and the line containing the first match for regexp2. +

+ + The address /<step/,/step>/ looks for the first instance of + <step in the file, and then starting after that point, + looks for the first instance of step>. + (Click on the “Steps” step above to see the highlight in action.) + Note that the < and > had to be written + using XML escapes in order to be valid XML. +
+ + + The /regexp/ + and /regexp1/,/regexp2/ + forms suffice for most highlighting. +

+ + The full address syntax is summarized in this table + (an excerpt of Table II from + The text editor sam): +

+ + + + + + + + + + + + + + + + + + + + +
Simple addresses
#nThe empty string after character n
nLine n
/regexp/The first following match of the regular expression
$The null string at the end of the file
Compound addresses
a1+a2The address a2 evaluated starting at the right of a1
a1-a2The address a2 evaluated in the reverse direction starting at the left of a1
a1,a2From the left of a1 to the right of a2 (default 0,$).
+
+ + + +
diff --git a/_content/doc/codewalk/codewalk_test.go b/_content/doc/codewalk/codewalk_test.go new file mode 100644 index 00000000..31f078ac --- /dev/null +++ b/_content/doc/codewalk/codewalk_test.go @@ -0,0 +1,52 @@ +// Copyright 2019 The Go Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + +package main_test + +import ( + "bytes" + "os" + "os/exec" + "strings" + "testing" +) + +// TestMarkov tests the code dependency of markov.xml. +func TestMarkov(t *testing.T) { + cmd := exec.Command("go", "run", "markov.go") + cmd.Stdin = strings.NewReader("foo") + cmd.Stderr = bytes.NewBuffer(nil) + out, err := cmd.Output() + if err != nil { + t.Fatalf("%s: %v\n%s", strings.Join(cmd.Args, " "), err, cmd.Stderr) + } + + if !bytes.Equal(out, []byte("foo\n")) { + t.Fatalf(`%s with input "foo" did not output "foo":\n%s`, strings.Join(cmd.Args, " "), out) + } +} + +// TestPig tests the code dependency of functions.xml. +func TestPig(t *testing.T) { + cmd := exec.Command("go", "run", "pig.go") + cmd.Stderr = bytes.NewBuffer(nil) + out, err := cmd.Output() + if err != nil { + t.Fatalf("%s: %v\n%s", strings.Join(cmd.Args, " "), err, cmd.Stderr) + } + + const want = "Wins, losses staying at k = 100: 210/990 (21.2%), 780/990 (78.8%)\n" + if !bytes.Contains(out, []byte(want)) { + t.Fatalf(`%s: unexpected output\ngot:\n%s\nwant output containing:\n%s`, strings.Join(cmd.Args, " "), out, want) + } +} + +// TestURLPoll tests the code dependency of sharemem.xml. +func TestURLPoll(t *testing.T) { + cmd := exec.Command("go", "build", "-o", os.DevNull, "urlpoll.go") + out, err := cmd.CombinedOutput() + if err != nil { + t.Fatalf("%s: %v\n%s", strings.Join(cmd.Args, " "), err, out) + } +} diff --git a/_content/doc/codewalk/functions.xml b/_content/doc/codewalk/functions.xml new file mode 100644 index 00000000..db518dcc --- /dev/null +++ b/_content/doc/codewalk/functions.xml @@ -0,0 +1,105 @@ + + + + Go supports first class functions, higher-order functions, user-defined + function types, function literals, closures, and multiple return values. +

+ + This rich feature set supports a functional programming style in a strongly + typed language. +

+ + In this codewalk we will look at a simple program that simulates a dice game + called Pig and evaluates + basic strategies. +
+ + + Pig is a two-player game played with a 6-sided die. Each turn, you may roll or stay. + + + The first person to reach 100 total points wins. +

+ + The score type stores the scores of the current and opposing + players, in addition to the points accumulated during the current turn. +
+ + + In Go, functions can be passed around just like any other value. A function's + type signature describes the types of its arguments and return values. +

+ + The action type is a function that takes a score + and returns the resulting score and whether the current turn is + over. +

+ + If the turn is over, the player and opponent fields + in the resulting score should be swapped, as it is now the other player's + turn. +
+ + + Go functions can return multiple values. +

+ + The functions roll and stay each return a pair of + values. They also match the action type signature. These + action functions define the rules of Pig. +
+ + + A function can use other functions as arguments and return values. +

+ + A strategy is a function that takes a score as input + and returns an action to perform.
+ (Remember, an action is itself a function.) +
+ + + Anonymous functions can be declared in Go, as in this example. Function + literals are closures: they inherit the scope of the function in which they + are declared. +

+ + One basic strategy in Pig is to continue rolling until you have accumulated at + least k points in a turn, and then stay. The argument k is + enclosed by this function literal, which matches the strategy type + signature. +
+ + + We simulate a game of Pig by calling an action to update the + score until one player reaches 100 points. Each + action is selected by calling the strategy function + associated with the current player. + + + + The roundRobin function simulates a tournament and tallies wins. + Each strategy plays each other strategy gamesPerSeries times. + + + + Variadic functions like ratioString take a variable number of + arguments. These arguments are available as a slice inside the function. + + + + The main function defines 100 basic strategies, simulates a round + robin tournament, and then prints the win/loss record of each strategy. +

+ + Among these strategies, staying at 25 is best, but the optimal strategy for + Pig is much more complex. +
+ +
diff --git a/_content/doc/codewalk/markov.go b/_content/doc/codewalk/markov.go new file mode 100644 index 00000000..5f62e051 --- /dev/null +++ b/_content/doc/codewalk/markov.go @@ -0,0 +1,130 @@ +// Copyright 2011 The Go Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + +/* +Generating random text: a Markov chain algorithm + +Based on the program presented in the "Design and Implementation" chapter +of The Practice of Programming (Kernighan and Pike, Addison-Wesley 1999). +See also Computer Recreations, Scientific American 260, 122 - 125 (1989). + +A Markov chain algorithm generates text by creating a statistical model of +potential textual suffixes for a given prefix. Consider this text: + + I am not a number! I am a free man! + +Our Markov chain algorithm would arrange this text into this set of prefixes +and suffixes, or "chain": (This table assumes a prefix length of two words.) + + Prefix Suffix + + "" "" I + "" I am + I am a + I am not + a free man! + am a free + am not a + a number! I + number! I am + not a number! + +To generate text using this table we select an initial prefix ("I am", for +example), choose one of the suffixes associated with that prefix at random +with probability determined by the input statistics ("a"), +and then create a new prefix by removing the first word from the prefix +and appending the suffix (making the new prefix is "am a"). Repeat this process +until we can't find any suffixes for the current prefix or we exceed the word +limit. (The word limit is necessary as the chain table may contain cycles.) + +Our version of this program reads text from standard input, parsing it into a +Markov chain, and writes generated text to standard output. +The prefix and output lengths can be specified using the -prefix and -words +flags on the command-line. +*/ +package main + +import ( + "bufio" + "flag" + "fmt" + "io" + "math/rand" + "os" + "strings" + "time" +) + +// Prefix is a Markov chain prefix of one or more words. +type Prefix []string + +// String returns the Prefix as a string (for use as a map key). +func (p Prefix) String() string { + return strings.Join(p, " ") +} + +// Shift removes the first word from the Prefix and appends the given word. +func (p Prefix) Shift(word string) { + copy(p, p[1:]) + p[len(p)-1] = word +} + +// Chain contains a map ("chain") of prefixes to a list of suffixes. +// A prefix is a string of prefixLen words joined with spaces. +// A suffix is a single word. A prefix can have multiple suffixes. +type Chain struct { + chain map[string][]string + prefixLen int +} + +// NewChain returns a new Chain with prefixes of prefixLen words. +func NewChain(prefixLen int) *Chain { + return &Chain{make(map[string][]string), prefixLen} +} + +// Build reads text from the provided Reader and +// parses it into prefixes and suffixes that are stored in Chain. +func (c *Chain) Build(r io.Reader) { + br := bufio.NewReader(r) + p := make(Prefix, c.prefixLen) + for { + var s string + if _, err := fmt.Fscan(br, &s); err != nil { + break + } + key := p.String() + c.chain[key] = append(c.chain[key], s) + p.Shift(s) + } +} + +// Generate returns a string of at most n words generated from Chain. +func (c *Chain) Generate(n int) string { + p := make(Prefix, c.prefixLen) + var words []string + for i := 0; i < n; i++ { + choices := c.chain[p.String()] + if len(choices) == 0 { + break + } + next := choices[rand.Intn(len(choices))] + words = append(words, next) + p.Shift(next) + } + return strings.Join(words, " ") +} + +func main() { + // Register command-line flags. + numWords := flag.Int("words", 100, "maximum number of words to print") + prefixLen := flag.Int("prefix", 2, "prefix length in words") + + flag.Parse() // Parse command-line flags. + rand.Seed(time.Now().UnixNano()) // Seed the random number generator. + + c := NewChain(*prefixLen) // Initialize a new Chain. + c.Build(os.Stdin) // Build chains from standard input. + text := c.Generate(*numWords) // Generate text. + fmt.Println(text) // Write text to standard output. +} diff --git a/_content/doc/codewalk/markov.xml b/_content/doc/codewalk/markov.xml new file mode 100644 index 00000000..7e44840d --- /dev/null +++ b/_content/doc/codewalk/markov.xml @@ -0,0 +1,307 @@ + + + + + + This codewalk describes a program that generates random text using + a Markov chain algorithm. The package comment describes the algorithm + and the operation of the program. Please read it before continuing. + + + + A chain consists of a prefix and a suffix. Each prefix is a set + number of words, while a suffix is a single word. + A prefix can have an arbitrary number of suffixes. + To model this data, we use a map[string][]string. + Each map key is a prefix (a string) and its values are + lists of suffixes (a slice of strings, []string). +

+ Here is the example table from the package comment + as modeled by this data structure: +
+map[string][]string{
+	" ":          {"I"},
+	" I":         {"am"},
+	"I am":       {"a", "not"},
+	"a free":     {"man!"},
+	"am a":       {"free"},
+	"am not":     {"a"},
+	"a number!":  {"I"},
+	"number! I":  {"am"},
+	"not a":      {"number!"},
+}
+ While each prefix consists of multiple words, we + store prefixes in the map as a single string. + It would seem more natural to store the prefix as a + []string, but we can't do this with a map because the + key type of a map must implement equality (and slices do not). +

+ Therefore, in most of our code we will model prefixes as a + []string and join the strings together with a space + to generate the map key: +
+Prefix               Map key
+
+[]string{"", ""}     " "
+[]string{"", "I"}    " I"
+[]string{"I", "am"}  "I am"
+
+
+ + + The complete state of the chain table consists of the table itself and + the word length of the prefixes. The Chain struct stores + this data. + + + + The Chain struct has two unexported fields (those that + do not begin with an upper case character), and so we write a + NewChain constructor function that initializes the + chain map with make and sets the + prefixLen field. +

+ This is constructor function is not strictly necessary as this entire + program is within a single package (main) and therefore + there is little practical difference between exported and unexported + fields. We could just as easily write out the contents of this function + when we want to construct a new Chain. + But using these unexported fields is good practice; it clearly denotes + that only methods of Chain and its constructor function should access + those fields. Also, structuring Chain like this means we + could easily move it into its own package at some later date. +
+ + + Since we'll be working with prefixes often, we define a + Prefix type with the concrete type []string. + Defining a named type clearly allows us to be explicit when we are + working with a prefix instead of just a []string. + Also, in Go we can define methods on any named type (not just structs), + so we can add methods that operate on Prefix if we need to. + + + + The first method we define on Prefix is + String. It returns a string representation + of a Prefix by joining the slice elements together with + spaces. We will use this method to generate keys when working with + the chain map. + + + + The Build method reads text from an io.Reader + and parses it into prefixes and suffixes that are stored in the + Chain. +

+ The io.Reader is an + interface type that is widely used by the standard library and + other Go code. Our code uses the + fmt.Fscan function, which + reads space-separated values from an io.Reader. +

+ The Build method returns once the Reader's + Read method returns io.EOF (end of file) + or some other read error occurs. +
+ + + This function does many small reads, which can be inefficient for some + Readers. For efficiency we wrap the provided + io.Reader with + bufio.NewReader to create a + new io.Reader that provides buffering. + + + + At the top of the function we make a Prefix slice + p using the Chain's prefixLen + field as its length. + We'll use this variable to hold the current prefix and mutate it with + each new word we encounter. + + + + In our loop we read words from the Reader into a + string variable s using + fmt.Fscan. Since Fscan uses space to + separate each input value, each call will yield just one word + (including punctuation), which is exactly what we need. +

+ Fscan returns an error if it encounters a read error + (io.EOF, for example) or if it can't scan the requested + value (in our case, a single string). In either case we just want to + stop scanning, so we break out of the loop. +
+ + + The word stored in s is a new suffix. We add the new + prefix/suffix combination to the chain map by computing + the map key with p.String and appending the suffix + to the slice stored under that key. +

+ The built-in append function appends elements to a slice + and allocates new storage when necessary. When the provided slice is + nil, append allocates a new slice. + This behavior conveniently ties in with the semantics of our map: + retrieving an unset key returns the zero value of the value type and + the zero value of []string is nil. + When our program encounters a new prefix (yielding a nil + value in the map) append will allocate a new slice. +

+ For more information about the append function and slices + in general see the + Slices: usage and internals article. +
+ + + Before reading the next word our algorithm requires us to drop the + first word from the prefix and push the current suffix onto the prefix. +

+ When in this state +
+p == Prefix{"I", "am"}
+s == "not" 
+ the new value for p would be +
+p == Prefix{"am", "not"}
+ This operation is also required during text generation so we put + the code to perform this mutation of the slice inside a method on + Prefix named Shift. +
+ + + The Shift method uses the built-in copy + function to copy the last len(p)-1 elements of p to + the start of the slice, effectively moving the elements + one index to the left (if you consider zero as the leftmost index). +
+p := Prefix{"I", "am"}
+copy(p, p[1:])
+// p == Prefix{"am", "am"}
+ We then assign the provided word to the last index + of the slice: +
+// suffix == "not"
+p[len(p)-1] = suffix
+// p == Prefix{"am", "not"}
+
+ + + The Generate method is similar to Build + except that instead of reading words from a Reader + and storing them in a map, it reads words from the map and + appends them to a slice (words). +

+ Generate uses a conditional for loop to generate + up to n words. +
+ + + At each iteration of the loop we retrieve a list of potential suffixes + for the current prefix. We access the chain map at key + p.String() and assign its contents to choices. +

+ If len(choices) is zero we break out of the loop as there + are no potential suffixes for that prefix. + This test also works if the key isn't present in the map at all: + in that case, choices will be nil and the + length of a nil slice is zero. +
+ + + To choose a suffix we use the + rand.Intn function. + It returns a random integer up to (but not including) the provided + value. Passing in len(choices) gives us a random index + into the full length of the list. +

+ We use that index to pick our new suffix, assign it to + next and append it to the words slice. +

+ Next, we Shift the new suffix onto the prefix just as + we did in the Build method. +
+ + + Before returning the generated text as a string, we use the + strings.Join function to join the elements of + the words slice together, separated by spaces. + + + + To make it easy to tweak the prefix and generated text lengths we + use the flag package to parse + command-line flags. +

+ These calls to flag.Int register new flags with the + flag package. The arguments to Int are the + flag name, its default value, and a description. The Int + function returns a pointer to an integer that will contain the + user-supplied value (or the default value if the flag was omitted on + the command-line). +
+ + + The main function begins by parsing the command-line + flags with flag.Parse and seeding the rand + package's random number generator with the current time. +

+ If the command-line flags provided by the user are invalid the + flag.Parse function will print an informative usage + message and terminate the program. +
+ + + To create the new Chain we call NewChain + with the value of the prefix flag. +

+ To build the chain we call Build with + os.Stdin (which implements io.Reader) so + that it will read its input from standard input. +
+ + + Finally, to generate text we call Generate with + the value of the words flag and assigning the result + to the variable text. +

+ Then we call fmt.Println to write the text to standard + output, followed by a carriage return. +
+ + + To use this program, first build it with the + go command: +
+$ go build markov.go
+ And then execute it while piping in some input text: +
+$ echo "a man a plan a canal panama" \
+	| ./markov -prefix=1
+a plan a man a plan a canal panama
+ Here's a transcript of generating some text using the Go distribution's + README file as source material: +
+$ ./markov -words=10 < $GOROOT/README
+This is the source code repository for the Go source
+$ ./markov -prefix=1 -words=10 < $GOROOT/README
+This is the go directory (the one containing this README).
+$ ./markov -prefix=1 -words=10 < $GOROOT/README
+This is the variable if you have just untarred a
+
+ + + The Generate function does a lot of allocations when it + builds the words slice. As an exercise, modify it to + take an io.Writer to which it incrementally writes the + generated text with Fprint. + Aside from being more efficient this makes Generate + more symmetrical to Build. + + +
diff --git a/_content/doc/codewalk/pig.go b/_content/doc/codewalk/pig.go new file mode 100644 index 00000000..941daaed --- /dev/null +++ b/_content/doc/codewalk/pig.go @@ -0,0 +1,121 @@ +// Copyright 2011 The Go Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + +package main + +import ( + "fmt" + "math/rand" +) + +const ( + win = 100 // The winning score in a game of Pig + gamesPerSeries = 10 // The number of games per series to simulate +) + +// A score includes scores accumulated in previous turns for each player, +// as well as the points scored by the current player in this turn. +type score struct { + player, opponent, thisTurn int +} + +// An action transitions stochastically to a resulting score. +type action func(current score) (result score, turnIsOver bool) + +// roll returns the (result, turnIsOver) outcome of simulating a die roll. +// If the roll value is 1, then thisTurn score is abandoned, and the players' +// roles swap. Otherwise, the roll value is added to thisTurn. +func roll(s score) (score, bool) { + outcome := rand.Intn(6) + 1 // A random int in [1, 6] + if outcome == 1 { + return score{s.opponent, s.player, 0}, true + } + return score{s.player, s.opponent, outcome + s.thisTurn}, false +} + +// stay returns the (result, turnIsOver) outcome of staying. +// thisTurn score is added to the player's score, and the players' roles swap. +func stay(s score) (score, bool) { + return score{s.opponent, s.player + s.thisTurn, 0}, true +} + +// A strategy chooses an action for any given score. +type strategy func(score) action + +// stayAtK returns a strategy that rolls until thisTurn is at least k, then stays. +func stayAtK(k int) strategy { + return func(s score) action { + if s.thisTurn >= k { + return stay + } + return roll + } +} + +// play simulates a Pig game and returns the winner (0 or 1). +func play(strategy0, strategy1 strategy) int { + strategies := []strategy{strategy0, strategy1} + var s score + var turnIsOver bool + currentPlayer := rand.Intn(2) // Randomly decide who plays first + for s.player+s.thisTurn < win { + action := strategies[currentPlayer](s) + s, turnIsOver = action(s) + if turnIsOver { + currentPlayer = (currentPlayer + 1) % 2 + } + } + return currentPlayer +} + +// roundRobin simulates a series of games between every pair of strategies. +func roundRobin(strategies []strategy) ([]int, int) { + wins := make([]int, len(strategies)) + for i := 0; i < len(strategies); i++ { + for j := i + 1; j < len(strategies); j++ { + for k := 0; k < gamesPerSeries; k++ { + winner := play(strategies[i], strategies[j]) + if winner == 0 { + wins[i]++ + } else { + wins[j]++ + } + } + } + } + gamesPerStrategy := gamesPerSeries * (len(strategies) - 1) // no self play + return wins, gamesPerStrategy +} + +// ratioString takes a list of integer values and returns a string that lists +// each value and its percentage of the sum of all values. +// e.g., ratios(1, 2, 3) = "1/6 (16.7%), 2/6 (33.3%), 3/6 (50.0%)" +func ratioString(vals ...int) string { + total := 0 + for _, val := range vals { + total += val + } + s := "" + for _, val := range vals { + if s != "" { + s += ", " + } + pct := 100 * float64(val) / float64(total) + s += fmt.Sprintf("%d/%d (%0.1f%%)", val, total, pct) + } + return s +} + +func main() { + strategies := make([]strategy, win) + for k := range strategies { + strategies[k] = stayAtK(k + 1) + } + wins, games := roundRobin(strategies) + + for k := range strategies { + fmt.Printf("Wins, losses staying at k =% 4d: %s\n", + k+1, ratioString(wins[k], games-wins[k])) + } +} diff --git a/_content/doc/codewalk/popout.png b/_content/doc/codewalk/popout.png new file mode 100644 index 00000000..9c0c2363 Binary files /dev/null and b/_content/doc/codewalk/popout.png differ diff --git a/_content/doc/codewalk/sharemem.xml b/_content/doc/codewalk/sharemem.xml new file mode 100644 index 00000000..8b47f12b --- /dev/null +++ b/_content/doc/codewalk/sharemem.xml @@ -0,0 +1,181 @@ + + + +Go's approach to concurrency differs from the traditional use of +threads and shared memory. Philosophically, it can be summarized: +

+Don't communicate by sharing memory; share memory by communicating. +

+Channels allow you to pass references to data structures between goroutines. +If you consider this as passing around ownership of the data (the ability to +read and write it), they become a powerful and expressive synchronization +mechanism. +

+In this codewalk we will look at a simple program that polls a list of +URLs, checking their HTTP response codes and periodically printing their state. +
+ + +The State type represents the state of a URL. +

+The Pollers send State values to the StateMonitor, +which maintains a map of the current state of each URL. +
+ + +A Resource represents the state of a URL to be polled: the URL itself +and the number of errors encountered since the last successful poll. +

+When the program starts, it allocates one Resource for each URL. +The main goroutine and the Poller goroutines send the Resources to +each other on channels. +
+ + +Each Poller receives Resource pointers from an input channel. +In this program, the convention is that sending a Resource pointer on +a channel passes ownership of the underlying data from the sender +to the receiver. Because of this convention, we know that +no two goroutines will access this Resource at the same time. +This means we don't have to worry about locking to prevent concurrent +access to these data structures. +

+The Poller processes the Resource by calling its Poll method. +

+It sends a State value to the status channel, to inform the StateMonitor +of the result of the Poll. +

+Finally, it sends the Resource pointer to the out channel. This can be +interpreted as the Poller saying "I'm done with this Resource" and +returning ownership of it to the main goroutine. +

+Several goroutines run Pollers, processing Resources in parallel. +
+ + +The Poll method (of the Resource type) performs an HTTP HEAD request +for the Resource's URL and returns the HTTP response's status code. +If an error occurs, Poll logs the message to standard error and returns the +error string instead. + + + +The main function starts the Poller and StateMonitor goroutines +and then loops passing completed Resources back to the pending +channel after appropriate delays. + + + +First, main makes two channels of *Resource, pending and complete. +

+Inside main, a new goroutine sends one Resource per URL to pending +and the main goroutine receives completed Resources from complete. +

+The pending and complete channels are passed to each of the Poller +goroutines, within which they are known as in and out. +
+ + +StateMonitor will initialize and launch a goroutine that stores the state +of each Resource. We will look at this function in detail later. +

+For now, the important thing to note is that it returns a channel of State, +which is saved as status and passed to the Poller goroutines. +
+ + +Now that it has the necessary channels, main launches a number of +Poller goroutines, passing the channels as arguments. +The channels provide the means of communication between the main, Poller, and +StateMonitor goroutines. + + + +To add the initial work to the system, main starts a new goroutine +that allocates and sends one Resource per URL to pending. +

+The new goroutine is necessary because unbuffered channel sends and +receives are synchronous. That means these channel sends will block until +the Pollers are ready to read from pending. +

+Were these sends performed in the main goroutine with fewer Pollers than +channel sends, the program would reach a deadlock situation, because +main would not yet be receiving from complete. +

+Exercise for the reader: modify this part of the program to read a list of +URLs from a file. (You may want to move this goroutine into its own +named function.) +
+ + +When a Poller is done with a Resource, it sends it on the complete channel. +This loop receives those Resource pointers from complete. +For each received Resource, it starts a new goroutine calling +the Resource's Sleep method. Using a new goroutine for each +ensures that the sleeps can happen in parallel. +

+Note that any single Resource pointer may only be sent on either pending or +complete at any one time. This ensures that a Resource is either being +handled by a Poller goroutine or sleeping, but never both simultaneously. +In this way, we share our Resource data by communicating. +
+ + +Sleep calls time.Sleep to pause before sending the Resource to done. +The pause will either be of a fixed length (pollInterval) plus an +additional delay proportional to the number of sequential errors (r.errCount). +

+This is an example of a typical Go idiom: a function intended to run inside +a goroutine takes a channel, upon which it sends its return value +(or other indication of completed state). +
+ + +The StateMonitor receives State values on a channel and periodically +outputs the state of all Resources being polled by the program. + + + +The variable updates is a channel of State, on which the Poller goroutines +send State values. +

+This channel is returned by the function. +
+ + +The variable urlStatus is a map of URLs to their most recent status. + + + +A time.Ticker is an object that repeatedly sends a value on a channel at a +specified interval. +

+In this case, ticker triggers the printing of the current state to +standard output every updateInterval nanoseconds. +
+ + +StateMonitor will loop forever, selecting on two channels: +ticker.C and update. The select statement blocks until one of its +communications is ready to proceed. +

+When StateMonitor receives a tick from ticker.C, it calls logState to +print the current state. When it receives a State update from updates, +it records the new status in the urlStatus map. +

+Notice that this goroutine owns the urlStatus data structure, +ensuring that it can only be accessed sequentially. +This prevents memory corruption issues that might arise from parallel reads +and/or writes to a shared map. +
+ + +In this codewalk we have explored a simple example of using Go's concurrency +primitives to share memory through communication. +

+This should provide a starting point from which to explore the ways in which +goroutines and channels can be used to write expressive and concise concurrent +programs. +
+ +
diff --git a/_content/doc/codewalk/urlpoll.go b/_content/doc/codewalk/urlpoll.go new file mode 100644 index 00000000..1fb99581 --- /dev/null +++ b/_content/doc/codewalk/urlpoll.go @@ -0,0 +1,116 @@ +// Copyright 2010 The Go Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + +package main + +import ( + "log" + "net/http" + "time" +) + +const ( + numPollers = 2 // number of Poller goroutines to launch + pollInterval = 60 * time.Second // how often to poll each URL + statusInterval = 10 * time.Second // how often to log status to stdout + errTimeout = 10 * time.Second // back-off timeout on error +) + +var urls = []string{ + "http://www.google.com/", + "http://golang.org/", + "http://blog.golang.org/", +} + +// State represents the last-known state of a URL. +type State struct { + url string + status string +} + +// StateMonitor maintains a map that stores the state of the URLs being +// polled, and prints the current state every updateInterval nanoseconds. +// It returns a chan State to which resource state should be sent. +func StateMonitor(updateInterval time.Duration) chan<- State { + updates := make(chan State) + urlStatus := make(map[string]string) + ticker := time.NewTicker(updateInterval) + go func() { + for { + select { + case <-ticker.C: + logState(urlStatus) + case s := <-updates: + urlStatus[s.url] = s.status + } + } + }() + return updates +} + +// logState prints a state map. +func logState(s map[string]string) { + log.Println("Current state:") + for k, v := range s { + log.Printf(" %s %s", k, v) + } +} + +// Resource represents an HTTP URL to be polled by this program. +type Resource struct { + url string + errCount int +} + +// Poll executes an HTTP HEAD request for url +// and returns the HTTP status string or an error string. +func (r *Resource) Poll() string { + resp, err := http.Head(r.url) + if err != nil { + log.Println("Error", r.url, err) + r.errCount++ + return err.Error() + } + r.errCount = 0 + return resp.Status +} + +// Sleep sleeps for an appropriate interval (dependent on error state) +// before sending the Resource to done. +func (r *Resource) Sleep(done chan<- *Resource) { + time.Sleep(pollInterval + errTimeout*time.Duration(r.errCount)) + done <- r +} + +func Poller(in <-chan *Resource, out chan<- *Resource, status chan<- State) { + for r := range in { + s := r.Poll() + status <- State{r.url, s} + out <- r + } +} + +func main() { + // Create our input and output channels. + pending, complete := make(chan *Resource), make(chan *Resource) + + // Launch the StateMonitor. + status := StateMonitor(statusInterval) + + // Launch some Poller goroutines. + for i := 0; i < numPollers; i++ { + go Poller(pending, complete, status) + } + + // Send some Resources to the pending queue. + go func() { + for _, url := range urls { + pending <- &Resource{url: url} + } + }() + + for r := range complete { + go r.Sleep(pending) + } +} diff --git a/_content/doc/contribute.html b/_content/doc/contribute.html new file mode 100644 index 00000000..66a47eb0 --- /dev/null +++ b/_content/doc/contribute.html @@ -0,0 +1,1294 @@ + + +

+The Go project welcomes all contributors. +

+ +

+This document is a guide to help you through the process +of contributing to the Go project, which is a little different +from that used by other open source projects. +We assume you have a basic understanding of Git and Go. +

+ +

+In addition to the information here, the Go community maintains a +CodeReview wiki page. +Feel free to contribute to the wiki as you learn the review process. +

+ +

+Note that the gccgo front end lives elsewhere; +see Contributing to gccgo. +

+ +

Becoming a contributor

+ +

Overview

+ +

+The first step is registering as a Go contributor and configuring your environment. +Here is a checklist of the required steps to follow: +

+ + + +

+If you prefer, there is an automated tool that walks through these steps. +Just run: +

+ +
+$ go get -u golang.org/x/tools/cmd/go-contrib-init
+$ cd /code/to/edit
+$ go-contrib-init
+
+ +

+The rest of this chapter elaborates on these instructions. +If you have completed the steps above (either manually or through the tool), jump to +Before contributing code. +

+ +

Step 0: Select a Google Account

+ +

+A contribution to Go is made through a Google account with a specific +e-mail address. +Make sure to use the same account throughout the process and +for all your subsequent contributions. +You may need to decide whether to use a personal address or a corporate address. +The choice will depend on who +will own the copyright for the code that you will be writing +and submitting. +You might want to discuss this topic with your employer before deciding which +account to use. +

+ +

+Google accounts can either be Gmail e-mail accounts, G Suite organization accounts, or +accounts associated with an external e-mail address. +For instance, if you need to use +an existing corporate e-mail that is not managed through G Suite, you can create +an account associated +with your existing +e-mail address. +

+ +

+You also need to make sure that your Git tool is configured to create commits +using your chosen e-mail address. +You can either configure Git globally +(as a default for all projects), or locally (for a single specific project). +You can check the current configuration with this command: +

+ +
+$ git config --global user.email  # check current global config
+$ git config user.email           # check current local config
+
+ +

+To change the configured address: +

+ +
+$ git config --global user.email name@example.com   # change global config
+$ git config user.email name@example.com            # change local config
+
+ + +

Step 1: Contributor License Agreement

+ +

+Before sending your first change to the Go project +you must have completed one of the following two CLAs. +Which CLA you should sign depends on who owns the copyright to your work. +

+ + + +

+You can check your currently signed agreements and sign new ones at +the Google Developers +Contributor License Agreements website. +If the copyright holder for your contribution has already completed the +agreement in connection with another Google open source project, +it does not need to be completed again. +

+ +

+If the copyright holder for the code you are submitting changes—for example, +if you start contributing code on behalf of a new company—please send mail +to the golang-dev +mailing list. +This will let us know the situation so we can make sure an appropriate agreement is +completed and update the AUTHORS file. +

+ + +

Step 2: Configure git authentication

+ +

+The main Go repository is located at +go.googlesource.com, +a Git server hosted by Google. +Authentication on the web server is made through your Google account, but +you also need to configure git on your computer to access it. +Follow these steps: +

+ +
    +
  1. +Visit go.googlesource.com +and click on "Generate Password" in the page's top right menu bar. +You will be redirected to accounts.google.com to sign in. +
  2. +
  3. +After signing in, you will be taken to a page with the title "Configure Git". +This page contains a personalized script that when run locally will configure Git +to hold your unique authentication key. +This key is paired with one that is generated and stored on the server, +analogous to how SSH keys work. +
  4. +
  5. +Copy and run this script locally in your terminal to store your secret +authentication token in a .gitcookies file. +If you are using a Windows computer and running cmd, +you should instead follow the instructions in the yellow box to run the command; +otherwise run the regular script. +
  6. +
+ +

Step 3: Create a Gerrit account

+ +

+Gerrit is an open-source tool used by Go maintainers to discuss and review +code submissions. +

+ +

+To register your account, visit +go-review.googlesource.com/login/ and sign in once using the same Google Account you used above. +

+ +

Step 4: Install the git-codereview command

+ +

+Changes to Go must be reviewed before they are accepted, no matter who makes the change. +A custom git command called git-codereview +simplifies sending changes to Gerrit. +

+ +

+Install the git-codereview command by running, +

+ +
+$ go get -u golang.org/x/review/git-codereview
+
+ +

+Make sure git-codereview is installed in your shell path, so that the +git command can find it. +Check that +

+ +
+$ git codereview help
+
+ +

+prints help text, not an error. If it prints an error, make sure that +$GOPATH/bin is in your $PATH. +

+ +

+On Windows, when using git-bash you must make sure that +git-codereview.exe is in your git exec-path. +Run git --exec-path to discover the right location then create a +symbolic link or just copy the executable from $GOPATH/bin to this +directory. +

+ + +

Before contributing code

+ +

+The project welcomes code patches, but to make sure things are well +coordinated you should discuss any significant change before starting +the work. +It's recommended that you signal your intention to contribute in the +issue tracker, either by filing +a new issue or by claiming +an existing one. +

+ +

Where to contribute

+ +

+The Go project consists of the main +go repository, which contains the +source code for the Go language, as well as many golang.org/x/... repostories. +These contain the various tools and infrastructure that support Go. For +example, golang.org/x/pkgsite +is for pkg.go.dev, +golang.org/x/playground +is for the Go playground, and +golang.org/x/tools contains +a variety of Go tools, including the Go language server, +gopls. You can see a +list of all the golang.org/x/... repositories on +go.googlesource.com. +

+ +

Check the issue tracker

+ +

+Whether you already know what contribution to make, or you are searching for +an idea, the issue tracker is +always the first place to go. +Issues are triaged to categorize them and manage the workflow. +

+ +

+The majority of the golang.org/x/... repos also use the main Go +issue tracker. However, a few of these repositories manage their issues +separately, so please be sure to check the right tracker for the repository to +which you would like to contribute. +

+ +

+Most issues will be marked with one of the following workflow labels: +

+ + + +

+You can use GitHub's search functionality to find issues to help out with. Examples: +

+ + + +

Open an issue for any new problem

+ +

+Excluding very trivial changes, all contributions should be connected +to an existing issue. +Feel free to open one and discuss your plans. +This process gives everyone a chance to validate the design, +helps prevent duplication of effort, +and ensures that the idea fits inside the goals for the language and tools. +It also checks that the design is sound before code is written; +the code review tool is not the place for high-level discussions. +

+ +

+When planning work, please note that the Go project follows a six-month development cycle +for the main Go repository. The latter half of each cycle is a three-month +feature freeze during which only bug fixes and documentation updates are +accepted. New contributions can be sent during a feature freeze, but they will +not be merged until the freeze is over. The freeze applies to the entire main +repository as well as to the code in golang.org/x/... repositories that is +needed to build the binaries included in the release. See the lists of packages +vendored into +the standard library +and the go command. +

+ +

+Significant changes to the language, libraries, or tools must go +through the +change proposal process +before they can be accepted. +

+ +

+Sensitive security-related issues (only!) should be reported to security@golang.org. +

+ +

Sending a change via GitHub

+ +

+First-time contributors that are already familiar with the +GitHub flow +are encouraged to use the same process for Go contributions. +Even though Go +maintainers use Gerrit for code review, a bot called Gopherbot has been created to sync +GitHub pull requests to Gerrit. +

+ +

+Open a pull request as you normally would. +Gopherbot will create a corresponding Gerrit change and post a link to +it on your GitHub pull request; updates to the pull request will also +get reflected in the Gerrit change. +When somebody comments on the change, their comment will be also +posted in your pull request, so you will get a notification. +

+ +

+Some things to keep in mind: +

+ + + +

Sending a change via Gerrit

+ +

+It is not possible to fully sync Gerrit and GitHub, at least at the moment, +so we recommend learning Gerrit. +It's different but powerful and familiarity with it will help you understand +the flow. +

+ +

Overview

+ +

+This is an overview of the overall process: +

+ + + +

+The rest of this section describes these steps in more detail. +

+ + +

Step 1: Clone the source code

+ +

+In addition to a recent Go installation, you need to have a local copy of the source +checked out from the correct repository. +You can check out the Go source repo onto your local file system anywhere +you want as long as it's outside your GOPATH. +Clone from go.googlesource.com (not GitHub): +

+ +

Main Go repository:

+
+$ git clone https://go.googlesource.com/go
+$ cd go
+
+ +

golang.org/x/... repository

+(golang.org/x/tools in this example): +
+$ git clone https://go.googlesource.com/tools
+$ cd tools
+
+ +

Step 2: Prepare changes in a new branch

+ +

+Each Go change must be made in a separate branch, created from the master branch. +You can use +the normal git commands to create a branch and add changes to the +staging area: +

+ +
+$ git checkout -b mybranch
+$ [edit files...]
+$ git add [files...]
+
+ +

+To commit changes, instead of git commit, use git codereview change. +

+ +
+$ git codereview change
+(open $EDITOR)
+
+ +

+You can edit the commit description in your favorite editor as usual. +The git codereview change command +will automatically add a unique Change-Id line near the bottom. +That line is used by Gerrit to match successive uploads of the same change. +Do not edit or delete it. +A Change-Id looks like this: +

+ +
+Change-Id: I2fbdbffb3aab626c4b6f56348861b7909e3e8990
+
+ +

+The tool also checks that you've +run go fmt over the source code, and that +the commit message follows the suggested format. +

+ +

+If you need to edit the files again, you can stage the new changes and +re-run git codereview change: each subsequent +run will amend the existing commit while preserving the Change-Id. +

+ +

+Make sure that you always keep a single commit in each branch. +If you add more +commits by mistake, you can use git rebase to +squash them together +into a single one. +

+ + +

Step 3: Test your changes

+ +

+You've written and tested your code, but +before sending code out for review, run all the tests for the whole +tree to make sure the changes don't break other packages or programs. +

+ +

In the main Go repository

+ +

This can be done by running all.bash:

+ +
+$ cd go/src
+$ ./all.bash
+
+ +

+(To build under Windows use all.bat) +

+ +

+After running for a while and printing a lot of testing output, the command should finish +by printing, +

+ +
+ALL TESTS PASSED
+
+ +

+You can use make.bash instead of all.bash +to just build the compiler and the standard library without running the test suite. +Once the go tool is built, it will be installed as bin/go +under the directory in which you cloned the Go repository, and you can +run it directly from there. +See also +the section on how to test your changes quickly. +

+ +

In the golang.org/x/... repositories

+ +

+Run the tests for the entire repository +(golang.org/x/tools, +in this example): +

+ +
+$ cd tools
+$ go test ./...
+
+ +

+If you're concerned about the build status, +you can check the Build Dashboard. +Test failures may also be caught by the TryBots in code review. +

+ +

+Some repositories, like +golang.org/x/vscode-go will +have different testing infrastructures, so always check the documentation +for the repository in which you are working. The README file in the root of the +repository will usually have this information. +

+ +

Step 4: Send changes for review

+ +

+Once the change is ready and tested over the whole tree, send it for review. +This is done with the mail sub-command which, despite its name, doesn't +directly mail anything; it just sends the change to Gerrit: +

+ +
+$ git codereview mail
+
+ +

+Gerrit assigns your change a number and URL, which git codereview mail will print, something like: +

+ +
+remote: New Changes:
+remote:   https://go-review.googlesource.com/99999 math: improved Sin, Cos and Tan precision for very large arguments
+
+ +

+If you get an error instead, check the +Troubleshooting mail errors section. +

+ +

+If your change relates to an open GitHub issue and you have followed the +suggested commit message format, the issue will be updated in a few minutes by a bot, +linking your Gerrit change to it in the comments. +

+ + +

Step 5: Revise changes after a review

+ +

+Go maintainers will review your code on Gerrit, and you will get notifications via e-mail. +You can see the review on Gerrit and comment on them there. +You can also reply +using e-mail +if you prefer. +

+ +

+If you need to revise your change after the review, edit the files in +the same branch you previously created, add them to the Git staging +area, and then amend the commit with +git codereview change: +

+ +
+$ git codereview change     # amend current commit
+(open $EDITOR)
+$ git codereview mail       # send new changes to Gerrit
+
+ +

+If you don't need to change the commit description, just save and exit from the editor. +Remember not to touch the special Change-Id line. +

+ +

+Again, make sure that you always keep a single commit in each branch. +If you add more +commits by mistake, you can use git rebase to +squash them together +into a single one. +

+ +

Good commit messages

+ +

+Commit messages in Go follow a specific set of conventions, +which we discuss in this section. +

+ +

+Here is an example of a good one: +

+ +
+math: improve Sin, Cos and Tan precision for very large arguments
+
+The existing implementation has poor numerical properties for
+large arguments, so use the McGillicutty algorithm to improve
+accuracy above 1e10.
+
+The algorithm is described at https://wikipedia.org/wiki/McGillicutty_Algorithm
+
+Fixes #159
+
+ +

First line

+ +

+The first line of the change description is conventionally a short one-line +summary of the change, prefixed by the primary affected package. +

+ +

+A rule of thumb is that it should be written so to complete the sentence +"This change modifies Go to _____." +That means it does not start with a capital letter, is not a complete sentence, +and actually summarizes the result of the change. +

+ +

+Follow the first line by a blank line. +

+ +

Main content

+ +

+The rest of the description elaborates and should provide context for the +change and explain what it does. +Write in complete sentences with correct punctuation, just like +for your comments in Go. +Don't use HTML, Markdown, or any other markup language. +

+ +

+Add any relevant information, such as benchmark data if the change +affects performance. +The benchstat +tool is conventionally used to format +benchmark data for change descriptions. +

+ +

Referencing issues

+ +

+The special notation "Fixes #12345" associates the change with issue 12345 in the +Go issue tracker. +When this change is eventually applied, the issue +tracker will automatically mark the issue as fixed. +

+ +

+If the change is a partial step towards the resolution of the issue, +write "Updates #12345" instead. +This will leave a comment in the issue linking back to the change in +Gerrit, but it will not close the issue when the change is applied. +

+ +

+If you are sending a change against a golang.org/x/... repository, you must use +the fully-qualified syntax supported by GitHub to make sure the change is +linked to the issue in the main repository, not the x/ repository. +Most issues are tracked in the main repository's issue tracker. +The correct form is "Fixes golang/go#159". +

+ + +

The review process

+ +

+This section explains the review process in detail and how to approach +reviews after a change has been mailed. +

+ + +

Common beginner mistakes

+ +

+When a change is sent to Gerrit, it is usually triaged within a few days. +A maintainer will have a look and provide some initial review that for first-time +contributors usually focuses on basic cosmetics and common mistakes. +These include things like: +

+ + + +

Trybots

+ +

+After an initial reading of your change, maintainers will trigger trybots, +a cluster of servers that will run the full test suite on several different +architectures. +Most trybots complete in a few minutes, at which point a link will +be posted in Gerrit where you can see the results. +

+ +

+If the trybot run fails, follow the link and check the full logs of the +platforms on which the tests failed. +Try to understand what broke, update your patch to fix it, and upload again. +Maintainers will trigger a new trybot run to see +if the problem was fixed. +

+ +

+Sometimes, the tree can be broken on some platforms for a few hours; if +the failure reported by the trybot doesn't seem related to your patch, go to the +Build Dashboard and check if the same +failure appears in other recent commits on the same platform. +In this case, +feel free to write a comment in Gerrit to mention that the failure is +unrelated to your change, to help maintainers understand the situation. +

+ +

Reviews

+ +

+The Go community values very thorough reviews. +Think of each review comment like a ticket: you are expected to somehow "close" it +by acting on it, either by implementing the suggestion or convincing the +reviewer otherwise. +

+ +

+After you update the change, go through the review comments and make sure +to reply to every one. +You can click the "Done" button to reply +indicating that you've implemented the reviewer's suggestion; otherwise, +click on "Reply" and explain why you have not, or what you have done instead. +

+ +

+It is perfectly normal for changes to go through several round of reviews, +with one or more reviewers making new comments every time +and then waiting for an updated change before reviewing again. +This cycle happens even for experienced contributors, so +don't be discouraged by it. +

+ +

Voting conventions

+ +

+As they near a decision, reviewers will make a "vote" on your change. +The Gerrit voting system involves an integer in the range -2 to +2: +

+ + + +

+At least two maintainers must approve of the change, and at least one +of those maintainers must +2 the change. +The second maintainer may cast a vote of Trust+1, meaning that the +change looks basically OK, but that the maintainer hasn't done the +detailed review required for a +2 vote. +

+ +

Submitting an approved change

+ +

+After the code has been +2'ed and Trust+1'ed, an approver will +apply it to the master branch using the Gerrit user interface. +This is called "submitting the change". +

+ +

+The two steps (approving and submitting) are separate because in some cases maintainers +may want to approve it but not to submit it right away (for instance, +the tree could be temporarily frozen). +

+ +

+Submitting a change checks it into the repository. +The change description will include a link to the code review, +which will be updated with a link to the change +in the repository. +Since the method used to integrate the changes is Git's "Cherry Pick", +the commit hashes in the repository will be changed by +the submit operation. +

+ +

+If your change has been approved for a few days without being +submitted, feel free to write a comment in Gerrit requesting +submission. +

+ + +

More information

+ +

+In addition to the information here, the Go community maintains a CodeReview wiki page. +Feel free to contribute to this page as you learn more about the review process. +

+ + + +

Miscellaneous topics

+ +

+This section collects a number of other comments that are +outside the issue/edit/code review/submit process itself. +

+ + + + +

+Files in the Go repository don't list author names, both to avoid clutter +and to avoid having to keep the lists up to date. +Instead, your name will appear in the +change log and in the CONTRIBUTORS file and perhaps the AUTHORS file. +These files are automatically generated from the commit logs periodically. +The AUTHORS file defines who “The Go +Authors”—the copyright holders—are. +

+ +

+New files that you contribute should use the standard copyright header: +

+ +
+// Copyright 2021 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+ +

+(Use the current year if you're reading this in 2022 or beyond.) +Files in the repository are copyrighted the year they are added. +Do not update the copyright year on files that you change. +

+ + + + +

Troubleshooting mail errors

+ +

+The most common way that the git codereview mail +command fails is because the e-mail address in the commit does not match the one +that you used during the registration process. + +
+If you see something like... +

+ +
+remote: Processing changes: refs: 1, done
+remote:
+remote: ERROR:  In commit ab13517fa29487dcf8b0d48916c51639426c5ee9
+remote: ERROR:  author email address XXXXXXXXXXXXXXXXXXX
+remote: ERROR:  does not match your user account.
+
+ +

+you need to configure Git for this repository to use the +e-mail address that you registered with. +To change the e-mail address to ensure this doesn't happen again, run: +

+ +
+$ git config user.email email@address.com
+
+ +

+Then change the commit to use this alternative e-mail address with this command: +

+ +
+$ git commit --amend --author="Author Name <email@address.com>"
+
+ +

+Then retry by running: +

+ +
+$ git codereview mail
+
+ + +

Quickly testing your changes

+ +

+Running all.bash for every single change to the code tree +is burdensome. +Even though it is strongly suggested to run it before +sending a change, during the normal development cycle you may want +to compile and test only the package you are developing. +

+ + + + +

Specifying a reviewer / CCing others

+ +

+Unless explicitly told otherwise, such as in the discussion leading +up to sending in the change, it's better not to specify a reviewer. +All changes are automatically CC'ed to the +golang-codereviews@googlegroups.com +mailing list. +If this is your first ever change, there may be a moderation +delay before it appears on the mailing list, to prevent spam. +

+ +

+You can specify a reviewer or CC interested parties +using the -r or -cc options. +Both accept a comma-separated list of e-mail addresses: +

+ +
+$ git codereview mail -r joe@golang.org -cc mabel@example.com,math-nuts@swtch.com
+
+ + +

Synchronize your client

+ +

+While you were working, others might have submitted changes to the repository. +To update your local branch, run +

+ +
+$ git codereview sync
+
+ +

+(Under the covers this runs +git pull -r.) +

+ + +

Reviewing code by others

+ +

+As part of the review process reviewers can propose changes directly (in the +GitHub workflow this would be someone else attaching commits to a pull request). + +You can import these changes proposed by someone else into your local Git repository. +On the Gerrit review page, click the "Download ▼" link in the upper right +corner, copy the "Checkout" command and run it from your local Git repo. +It will look something like this: +

+ +
+$ git fetch https://go.googlesource.com/review refs/changes/21/13245/1 && git checkout FETCH_HEAD
+
+ +

+To revert, change back to the branch you were working in. +

+ + +

Set up git aliases

+ +

+The git-codereview command can be run directly from the shell +by typing, for instance, +

+ +
+$ git codereview sync
+
+ +

+but it is more convenient to set up aliases for git-codereview's own +subcommands, so that the above becomes, +

+ +
+$ git sync
+
+ +

+The git-codereview subcommands have been chosen to be distinct from +Git's own, so it's safe to define these aliases. +To install them, copy this text into your +Git configuration file (usually .gitconfig in your home directory): +

+ +
+[alias]
+	change = codereview change
+	gofmt = codereview gofmt
+	mail = codereview mail
+	pending = codereview pending
+	submit = codereview submit
+	sync = codereview sync
+
+ + +

Sending multiple dependent changes

+ +

+Advanced users may want to stack up related commits in a single branch. +Gerrit allows for changes to be dependent on each other, forming such a dependency chain. +Each change will need to be approved and submitted separately but the dependency +will be visible to reviewers. +

+ +

+To send out a group of dependent changes, keep each change as a different commit under +the same branch, and then run: +

+ +
+$ git codereview mail HEAD
+
+ +

+Make sure to explicitly specify HEAD, which is usually not required when sending +single changes. More details can be found in the git-codereview documentation. +

diff --git a/_content/doc/debugging_with_gdb.html b/_content/doc/debugging_with_gdb.html new file mode 100644 index 00000000..e1fb292f --- /dev/null +++ b/_content/doc/debugging_with_gdb.html @@ -0,0 +1,554 @@ + + + + + +

+The following instructions apply to the standard toolchain +(the gc Go compiler and tools). +Gccgo has native gdb support. +

+

+Note that +Delve is a better +alternative to GDB when debugging Go programs built with the standard +toolchain. It understands the Go runtime, data structures, and +expressions better than GDB. Delve currently supports Linux, OSX, +and Windows on amd64. +For the most up-to-date list of supported platforms, please see + + the Delve documentation. +

+
+ +

+GDB does not understand Go programs well. +The stack management, threading, and runtime contain aspects that differ +enough from the execution model GDB expects that they can confuse +the debugger and cause incorrect results even when the program is +compiled with gccgo. +As a consequence, although GDB can be useful in some situations (e.g., +debugging Cgo code, or debugging the runtime itself), it is not +a reliable debugger for Go programs, particularly heavily concurrent +ones. Moreover, it is not a priority for the Go project to address +these issues, which are difficult. +

+ +

+In short, the instructions below should be taken only as a guide to how +to use GDB when it works, not as a guarantee of success. + +Besides this overview you might want to consult the +GDB manual. +

+ +

+

+ +

Introduction

+ +

+When you compile and link your Go programs with the gc toolchain +on Linux, macOS, FreeBSD or NetBSD, the resulting binaries contain DWARFv4 +debugging information that recent versions (≥7.5) of the GDB debugger can +use to inspect a live process or a core dump. +

+ +

+Pass the '-w' flag to the linker to omit the debug information +(for example, go build -ldflags=-w prog.go). +

+ +

+The code generated by the gc compiler includes inlining of +function invocations and registerization of variables. These optimizations +can sometimes make debugging with gdb harder. +If you find that you need to disable these optimizations, +build your program using go build -gcflags=all="-N -l". +

+ +

+If you want to use gdb to inspect a core dump, you can trigger a dump +on a program crash, on systems that permit it, by setting +GOTRACEBACK=crash in the environment (see the + runtime package +documentation for more info). +

+ +

Common Operations

+ + + + +

Go Extensions

+ +

+A recent extension mechanism to GDB allows it to load extension scripts for a +given binary. The toolchain uses this to extend GDB with a handful of +commands to inspect internals of the runtime code (such as goroutines) and to +pretty print the built-in map, slice and channel types. +

+ + + +

+If you'd like to see how this works, or want to extend it, take a look at src/runtime/runtime-gdb.py in +the Go source distribution. It depends on some special magic types +(hash<T,U>) and variables (runtime.m and +runtime.g) that the linker +(src/cmd/link/internal/ld/dwarf.go) ensures are described in +the DWARF code. +

+ +

+If you're interested in what the debugging information looks like, run +objdump -W a.out and browse through the .debug_* +sections. +

+ + +

Known Issues

+ +
    +
  1. String pretty printing only triggers for type string, not for types derived +from it.
  2. +
  3. Type information is missing for the C parts of the runtime library.
  4. +
  5. GDB does not understand Go’s name qualifications and treats +"fmt.Print" as an unstructured literal with a "." +that needs to be quoted. It objects even more strongly to method names of +the form pkg.(*MyType).Meth. +
  6. As of Go 1.11, debug information is compressed by default. +Older versions of gdb, such as the one available by default on MacOS, +do not understand the compression. +You can generate uncompressed debug information by using go +build -ldflags=-compressdwarf=false. +(For convenience you can put the -ldflags option in +the GOFLAGS +environment variable so that you don't have to specify it each time.) +
  7. +
+ +

Tutorial

+ +

+In this tutorial we will inspect the binary of the +regexp package's unit tests. To build the binary, +change to $GOROOT/src/regexp and run go test -c. +This should produce an executable file named regexp.test. +

+ + +

Getting Started

+ +

+Launch GDB, debugging regexp.test: +

+ +
+$ gdb regexp.test
+GNU gdb (GDB) 7.2-gg8
+Copyright (C) 2010 Free Software Foundation, Inc.
+License GPLv  3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
+Type "show copying" and "show warranty" for licensing/warranty details.
+This GDB was configured as "x86_64-linux".
+
+Reading symbols from  /home/user/go/src/regexp/regexp.test...
+done.
+Loading Go Runtime support.
+(gdb) 
+
+ +

+The message "Loading Go Runtime support" means that GDB loaded the +extension from $GOROOT/src/runtime/runtime-gdb.py. +

+ +

+To help GDB find the Go runtime sources and the accompanying support script, +pass your $GOROOT with the '-d' flag: +

+ +
+$ gdb regexp.test -d $GOROOT
+
+ +

+If for some reason GDB still can't find that directory or that script, you can load +it by hand by telling gdb (assuming you have the go sources in +~/go/): +

+ +
+(gdb) source ~/go/src/runtime/runtime-gdb.py
+Loading Go Runtime support.
+
+ +

Inspecting the source

+ +

+Use the "l" or "list" command to inspect source code. +

+ +
+(gdb) l
+
+ +

+List a specific part of the source parameterizing "list" with a +function name (it must be qualified with its package name). +

+ +
+(gdb) l main.main
+
+ +

+List a specific file and line number: +

+ +
+(gdb) l regexp.go:1
+(gdb) # Hit enter to repeat last command. Here, this lists next 10 lines.
+
+ + +

Naming

+ +

+Variable and function names must be qualified with the name of the packages +they belong to. The Compile function from the regexp +package is known to GDB as 'regexp.Compile'. +

+ +

+Methods must be qualified with the name of their receiver types. For example, +the *Regexp type’s String method is known as +'regexp.(*Regexp).String'. +

+ +

+Variables that shadow other variables are magically suffixed with a number in the debug info. +Variables referenced by closures will appear as pointers magically prefixed with '&'. +

+ +

Setting breakpoints

+ +

+Set a breakpoint at the TestFind function: +

+ +
+(gdb) b 'regexp.TestFind'
+Breakpoint 1 at 0x424908: file /home/user/go/src/regexp/find_test.go, line 148.
+
+ +

+Run the program: +

+ +
+(gdb) run
+Starting program: /home/user/go/src/regexp/regexp.test
+
+Breakpoint 1, regexp.TestFind (t=0xf8404a89c0) at /home/user/go/src/regexp/find_test.go:148
+148	func TestFind(t *testing.T) {
+
+ +

+Execution has paused at the breakpoint. +See which goroutines are running, and what they're doing: +

+ +
+(gdb) info goroutines
+  1  waiting runtime.gosched
+* 13  running runtime.goexit
+
+ +

+the one marked with the * is the current goroutine. +

+ +

Inspecting the stack

+ +

+Look at the stack trace for where we’ve paused the program: +

+ +
+(gdb) bt  # backtrace
+#0  regexp.TestFind (t=0xf8404a89c0) at /home/user/go/src/regexp/find_test.go:148
+#1  0x000000000042f60b in testing.tRunner (t=0xf8404a89c0, test=0x573720) at /home/user/go/src/testing/testing.go:156
+#2  0x000000000040df64 in runtime.initdone () at /home/user/go/src/runtime/proc.c:242
+#3  0x000000f8404a89c0 in ?? ()
+#4  0x0000000000573720 in ?? ()
+#5  0x0000000000000000 in ?? ()
+
+ +

+The other goroutine, number 1, is stuck in runtime.gosched, blocked on a channel receive: +

+ +
+(gdb) goroutine 1 bt
+#0  0x000000000040facb in runtime.gosched () at /home/user/go/src/runtime/proc.c:873
+#1  0x00000000004031c9 in runtime.chanrecv (c=void, ep=void, selected=void, received=void)
+ at  /home/user/go/src/runtime/chan.c:342
+#2  0x0000000000403299 in runtime.chanrecv1 (t=void, c=void) at/home/user/go/src/runtime/chan.c:423
+#3  0x000000000043075b in testing.RunTests (matchString={void (struct string, struct string, bool *, error *)}
+ 0x7ffff7f9ef60, tests=  []testing.InternalTest = {...}) at /home/user/go/src/testing/testing.go:201
+#4  0x00000000004302b1 in testing.Main (matchString={void (struct string, struct string, bool *, error *)} 
+ 0x7ffff7f9ef80, tests= []testing.InternalTest = {...}, benchmarks= []testing.InternalBenchmark = {...})
+at /home/user/go/src/testing/testing.go:168
+#5  0x0000000000400dc1 in main.main () at /home/user/go/src/regexp/_testmain.go:98
+#6  0x00000000004022e7 in runtime.mainstart () at /home/user/go/src/runtime/amd64/asm.s:78
+#7  0x000000000040ea6f in runtime.initdone () at /home/user/go/src/runtime/proc.c:243
+#8  0x0000000000000000 in ?? ()
+
+ +

+The stack frame shows we’re currently executing the regexp.TestFind function, as expected. +

+ +
+(gdb) info frame
+Stack level 0, frame at 0x7ffff7f9ff88:
+ rip = 0x425530 in regexp.TestFind (/home/user/go/src/regexp/find_test.go:148); 
+    saved rip 0x430233
+ called by frame at 0x7ffff7f9ffa8
+ source language minimal.
+ Arglist at 0x7ffff7f9ff78, args: t=0xf840688b60
+ Locals at 0x7ffff7f9ff78, Previous frame's sp is 0x7ffff7f9ff88
+ Saved registers:
+  rip at 0x7ffff7f9ff80
+
+ +

+The command info locals lists all variables local to the function and their values, but is a bit +dangerous to use, since it will also try to print uninitialized variables. Uninitialized slices may cause gdb to try +to print arbitrary large arrays. +

+ +

+The function’s arguments: +

+ +
+(gdb) info args
+t = 0xf840688b60
+
+ +

+When printing the argument, notice that it’s a pointer to a +Regexp value. Note that GDB has incorrectly put the * +on the right-hand side of the type name and made up a 'struct' keyword, in traditional C style. +

+ +
+(gdb) p re
+(gdb) p t
+$1 = (struct testing.T *) 0xf840688b60
+(gdb) p t
+$1 = (struct testing.T *) 0xf840688b60
+(gdb) p *t
+$2 = {errors = "", failed = false, ch = 0xf8406f5690}
+(gdb) p *t->ch
+$3 = struct hchan<*testing.T>
+
+ +

+That struct hchan<*testing.T> is the +runtime-internal representation of a channel. It is currently empty, +or gdb would have pretty-printed its contents. +

+ +

+Stepping forward: +

+ +
+(gdb) n  # execute next line
+149             for _, test := range findTests {
+(gdb)    # enter is repeat
+150                     re := MustCompile(test.pat)
+(gdb) p test.pat
+$4 = ""
+(gdb) p re
+$5 = (struct regexp.Regexp *) 0xf84068d070
+(gdb) p *re
+$6 = {expr = "", prog = 0xf840688b80, prefix = "", prefixBytes =  []uint8, prefixComplete = true, 
+  prefixRune = 0, cond = 0 '\000', numSubexp = 0, longest = false, mu = {state = 0, sema = 0}, 
+  machine =  []*regexp.machine}
+(gdb) p *re->prog
+$7 = {Inst =  []regexp/syntax.Inst = {{Op = 5 '\005', Out = 0, Arg = 0, Rune =  []int}, {Op = 
+    6 '\006', Out = 2, Arg = 0, Rune =  []int}, {Op = 4 '\004', Out = 0, Arg = 0, Rune =  []int}}, 
+  Start = 1, NumCap = 2}
+
+ + +

+We can step into the Stringfunction call with "s": +

+ +
+(gdb) s
+regexp.(*Regexp).String (re=0xf84068d070, noname=void) at /home/user/go/src/regexp/regexp.go:97
+97      func (re *Regexp) String() string {
+
+ +

+Get a stack trace to see where we are: +

+ +
+(gdb) bt
+#0  regexp.(*Regexp).String (re=0xf84068d070, noname=void)
+    at /home/user/go/src/regexp/regexp.go:97
+#1  0x0000000000425615 in regexp.TestFind (t=0xf840688b60)
+    at /home/user/go/src/regexp/find_test.go:151
+#2  0x0000000000430233 in testing.tRunner (t=0xf840688b60, test=0x5747b8)
+    at /home/user/go/src/testing/testing.go:156
+#3  0x000000000040ea6f in runtime.initdone () at /home/user/go/src/runtime/proc.c:243
+....
+
+ +

+Look at the source code: +

+ +
+(gdb) l
+92              mu      sync.Mutex
+93              machine []*machine
+94      }
+95
+96      // String returns the source text used to compile the regular expression.
+97      func (re *Regexp) String() string {
+98              return re.expr
+99      }
+100
+101     // Compile parses a regular expression and returns, if successful,
+
+ +

Pretty Printing

+ +

+GDB's pretty printing mechanism is triggered by regexp matches on type names. An example for slices: +

+ +
+(gdb) p utf
+$22 =  []uint8 = {0 '\000', 0 '\000', 0 '\000', 0 '\000'}
+
+ +

+Since slices, arrays and strings are not C pointers, GDB can't interpret the subscripting operation for you, but +you can look inside the runtime representation to do that (tab completion helps here): +

+
+
+(gdb) p slc
+$11 =  []int = {0, 0}
+(gdb) p slc-><TAB>
+array  slc    len    
+(gdb) p slc->array
+$12 = (int *) 0xf84057af00
+(gdb) p slc->array[1]
+$13 = 0
+ + + +

+The extension functions $len and $cap work on strings, arrays and slices: +

+ +
+(gdb) p $len(utf)
+$23 = 4
+(gdb) p $cap(utf)
+$24 = 4
+
+ +

+Channels and maps are 'reference' types, which gdb shows as pointers to C++-like types hash<int,string>*. Dereferencing will trigger prettyprinting +

+ +

+Interfaces are represented in the runtime as a pointer to a type descriptor and a pointer to a value. The Go GDB runtime extension decodes this and automatically triggers pretty printing for the runtime type. The extension function $dtype decodes the dynamic type for you (examples are taken from a breakpoint at regexp.go line 293.) +

+ +
+(gdb) p i
+$4 = {str = "cbb"}
+(gdb) whatis i
+type = regexp.input
+(gdb) p $dtype(i)
+$26 = (struct regexp.inputBytes *) 0xf8400b4930
+(gdb) iface i
+regexp.input: struct regexp.inputBytes *
+
diff --git a/_content/doc/diagnostics.html b/_content/doc/diagnostics.html new file mode 100644 index 00000000..438cdce4 --- /dev/null +++ b/_content/doc/diagnostics.html @@ -0,0 +1,472 @@ + + + + +

Introduction

+ +

+The Go ecosystem provides a large suite of APIs and tools to +diagnose logic and performance problems in Go programs. This page +summarizes the available tools and helps Go users pick the right one +for their specific problem. +

+ +

+Diagnostics solutions can be categorized into the following groups: +

+ + + +

+Note: Some diagnostics tools may interfere with each other. For example, precise +memory profiling skews CPU profiles and goroutine blocking profiling affects scheduler +trace. Use tools in isolation to get more precise info. +

+ +

Profiling

+ +

+Profiling is useful for identifying expensive or frequently called sections +of code. The Go runtime provides +profiling data in the format expected by the +pprof visualization tool. +The profiling data can be collected during testing +via go test or endpoints made available from the +net/http/pprof package. Users need to collect the profiling data and use pprof tools to filter +and visualize the top code paths. +

+ +

Predefined profiles provided by the runtime/pprof package:

+ + + + +

What other profilers can I use to profile Go programs?

+ +

+On Linux, perf tools +can be used for profiling Go programs. Perf can profile +and unwind cgo/SWIG code and kernel, so it can be useful to get insights into +native/kernel performance bottlenecks. On macOS, +Instruments +suite can be used profile Go programs. +

+ +

Can I profile my production services?

+ +

Yes. It is safe to profile programs in production, but enabling +some profiles (e.g. the CPU profile) adds cost. You should expect to +see performance downgrade. The performance penalty can be estimated +by measuring the overhead of the profiler before turning it on in +production. +

+ +

+You may want to periodically profile your production services. +Especially in a system with many replicas of a single process, selecting +a random replica periodically is a safe option. +Select a production process, profile it for +X seconds for every Y seconds and save the results for visualization and +analysis; then repeat periodically. Results may be manually and/or automatically +reviewed to find problems. +Collection of profiles can interfere with each other, +so it is recommended to collect only a single profile at a time. +

+ +

+What are the best ways to visualize the profiling data? +

+ +

+The Go tools provide text, graph, and callgrind +visualization of the profile data using +go tool pprof. +Read Profiling Go programs +to see them in action. +

+ +

+ +
+Listing of the most expensive calls as text. +

+ +

+ +
+Visualization of the most expensive calls as a graph. +

+ +

Weblist view displays the expensive parts of the source line by line in +an HTML page. In the following example, 530ms is spent in the +runtime.concatstrings and cost of each line is presented +in the listing.

+ +

+ +
+Visualization of the most expensive calls as weblist. +

+ +

+Another way to visualize profile data is a flame graph. +Flame graphs allow you to move in a specific ancestry path, so you can zoom +in/out of specific sections of code. +The upstream pprof +has support for flame graphs. +

+ +

+ +
+Flame graphs offers visualization to spot the most expensive code-paths. +

+ +

Am I restricted to the built-in profiles?

+ +

+Additionally to what is provided by the runtime, Go users can create +their custom profiles via pprof.Profile +and use the existing tools to examine them. +

+ +

Can I serve the profiler handlers (/debug/pprof/...) on a different path and port?

+ +

+Yes. The net/http/pprof package registers its handlers to the default +mux by default, but you can also register them yourself by using the handlers +exported from the package. +

+ +

+For example, the following example will serve the pprof.Profile +handler on :7777 at /custom_debug_path/profile: +

+ +

+

+package main
+
+import (
+	"log"
+	"net/http"
+	"net/http/pprof"
+)
+
+func main() {
+	mux := http.NewServeMux()
+	mux.HandleFunc("/custom_debug_path/profile", pprof.Profile)
+	log.Fatal(http.ListenAndServe(":7777", mux))
+}
+
+

+ +

Tracing

+ +

+Tracing is a way to instrument code to analyze latency throughout the +lifecycle of a chain of calls. Go provides +golang.org/x/net/trace +package as a minimal tracing backend per Go node and provides a minimal +instrumentation library with a simple dashboard. Go also provides +an execution tracer to trace the runtime events within an interval. +

+ +

Tracing enables us to:

+ + + +

+In monolithic systems, it's relatively easy to collect diagnostic data +from the building blocks of a program. All modules live within one +process and share common resources to report logs, errors, and other +diagnostic information. Once your system grows beyond a single process and +starts to become distributed, it becomes harder to follow a call starting +from the front-end web server to all of its back-ends until a response is +returned back to the user. This is where distributed tracing plays a big +role to instrument and analyze your production systems. +

+ +

+Distributed tracing is a way to instrument code to analyze latency throughout +the lifecycle of a user request. When a system is distributed and when +conventional profiling and debugging tools don’t scale, you might want +to use distributed tracing tools to analyze the performance of your user +requests and RPCs. +

+ +

Distributed tracing enables us to:

+ + + +

The Go ecosystem provides various distributed tracing libraries per tracing system +and backend-agnostic ones.

+ + +

Is there a way to automatically intercept each function call and create traces?

+ +

+Go doesn’t provide a way to automatically intercept every function call and create +trace spans. You need to manually instrument your code to create, end, and annotate spans. +

+ +

How should I propagate trace headers in Go libraries?

+ +

+You can propagate trace identifiers and tags in the +context.Context. +There is no canonical trace key or common representation of trace headers +in the industry yet. Each tracing provider is responsible for providing propagation +utilities in their Go libraries. +

+ +

+What other low-level events from the standard library or +runtime can be included in a trace? +

+ +

+The standard library and runtime are trying to expose several additional APIs +to notify on low level internal events. For example, +httptrace.ClientTrace +provides APIs to follow low-level events in the life cycle of an outgoing request. +There is an ongoing effort to retrieve low-level runtime events from +the runtime execution tracer and allow users to define and record their user events. +

+ +

Debugging

+ +

+Debugging is the process of identifying why a program misbehaves. +Debuggers allow us to understand a program’s execution flow and current state. +There are several styles of debugging; this section will only focus on attaching +a debugger to a program and core dump debugging. +

+ +

Go users mostly use the following debuggers:

+ + + +

How well do debuggers work with Go programs?

+ +

+The gc compiler performs optimizations such as +function inlining and variable registerization. These optimizations +sometimes make debugging with debuggers harder. There is an ongoing +effort to improve the quality of the DWARF information generated for +optimized binaries. Until those improvements are available, we recommend +disabling optimizations when building the code being debugged. The following +command builds a package with no compiler optimizations: + +

+

+$ go build -gcflags=all="-N -l"
+
+

+ +As part of the improvement effort, Go 1.10 introduced a new compiler +flag -dwarflocationlists. The flag causes the compiler to +add location lists that helps debuggers work with optimized binaries. +The following command builds a package with optimizations but with +the DWARF location lists: + +

+

+$ go build -gcflags="-dwarflocationlists=true"
+
+

+ +

What’s the recommended debugger user interface?

+ +

+Even though both delve and gdb provides CLIs, most editor integrations +and IDEs provides debugging-specific user interfaces. +

+ +

Is it possible to do postmortem debugging with Go programs?

+ +

+A core dump file is a file that contains the memory dump of a running +process and its process status. It is primarily used for post-mortem +debugging of a program and to understand its state +while it is still running. These two cases make debugging of core +dumps a good diagnostic aid to postmortem and analyze production +services. It is possible to obtain core files from Go programs and +use delve or gdb to debug, see the +core dump debugging +page for a step-by-step guide. +

+ +

Runtime statistics and events

+ +

+The runtime provides stats and reporting of internal events for +users to diagnose performance and utilization problems at the +runtime level. +

+ +

+Users can monitor these stats to better understand the overall +health and performance of Go programs. +Some frequently monitored stats and states: +

+ + + +

Execution tracer

+ +

Go comes with a runtime execution tracer to capture a wide range +of runtime events. Scheduling, syscall, garbage collections, +heap size, and other events are collected by runtime and available +for visualization by the go tool trace. Execution tracer is a tool +to detect latency and utilization problems. You can examine how well +the CPU is utilized, and when networking or syscalls are a cause of +preemption for the goroutines.

+ +

Tracer is useful to:

+ + +

However, it is not great for identifying hot spots such as +analyzing the cause of excessive memory or CPU usage. +Use profiling tools instead first to address them.

+ +

+ +

+ +

Above, the go tool trace visualization shows the execution started +fine, and then it became serialized. It suggests that there might +be lock contention for a shared resource that creates a bottleneck.

+ +

See go tool trace +to collect and analyze runtime traces. +

+ +

GODEBUG

+ +

Runtime also emits events and information if +GODEBUG +environmental variable is set accordingly.

+ + + +

The GODEBUG environmental variable can be used to disable use of +instruction set extensions in the standard library and runtime.

+ + diff --git a/_content/doc/editors.html b/_content/doc/editors.html new file mode 100644 index 00000000..e0d0c530 --- /dev/null +++ b/_content/doc/editors.html @@ -0,0 +1,33 @@ + + +

Introduction

+ +

+ This document lists commonly used editor plugins and IDEs from the Go ecosystem + that make Go development more productive and seamless. + A comprehensive list of editor support and IDEs for Go development is available at + the wiki. +

+ +

Options

+

+The Go ecosystem provides a variety of editor plugins and IDEs to enhance your day-to-day +editing, navigation, testing, and debugging experience. +

+ +