Restructure generated file structure (#167)

* Restructure templates

Update integration tests with new generated file structure

Update documentation for new file structure

Resolve #166

* Add DO NOT EDIT! to files that are overwritten on regeneration

* Update gitignore to exclude generated example services
This commit is contained in:
Adam Ryman 2017-03-29 15:06:04 -07:00 коммит произвёл GitHub
Родитель df5873b83f
Коммит b412131d40
30 изменённых файлов: 327 добавлений и 337 удалений

4
.gitignore поставляемый
Просмотреть файл

@ -5,6 +5,10 @@
# session
Session.vi
# generated example services
_example/*
!_example/echo.proto
# Prevents accidentally including a problematic folder within the `vendor/`
# folder.
vendor/github.com/Sirupsen/logrus/examples/

Просмотреть файл

@ -63,8 +63,6 @@ overwrite user code in the /NAME-service/handlers directory
Additional internal packages of note used by these programs are:
- `astmodifier`, located in `gengokit/astmodifier/`, used to
modify go files in place, and used by gengokit`
- `deftree`, located in `deftree/`, which makes sense of the protobuf file
passed to it by `protoc`, and is used by `gengokit` and
`gendoc`

Просмотреть файл

@ -46,7 +46,7 @@ add [http annotations](
https://github.com/googleapis/googleapis/blob/928a151b2f871b4239b7707e1bb59258df3fe10a/google/api/http.proto#L36)
for HTTP 1.1/JSON transport!
Then you open the `handlers/server/server_handler.go`,
Then you open the `handlers/handlers.go`,
add you business logic, and you're good to go.
Here is an example service definition: [Echo Service](./_example/echo.proto)

Просмотреть файл

@ -69,30 +69,32 @@ In your terminal, go to the folder containing echo.proto and run `truss *.proto`
```
.
├── echo-service
│ ├── docs
│ │   └── docs.md
│ ├── generated
│ │   └── ...
│ ├── handlers
│ │   └── server
│ │   └── server_handler.go
│ ├── middlewares
│ │   └── ...
│ ├── echo-cli-client
│ │   └── client_main.go
│ ├── echo-server
│ │   └── server_main.go
│ └── echo.pb.go
├── echo.proto
| ├── cmd
| │   ├── echo
| │   │   └── main.go
| │   └── echo-server
| │   └── main.go
| ├── docs
| │   └── docs.md
| ├── echo.pb.go
| ├── handlers
| │   ├── handlers.go
| │   └── hooks.go
| ├── middlewares
| │   ├── endpoints.go
| │   └── service.go
| └── svc
| └── ...
└── echo.proto
```
From the top down, within echo-service/:
- docs/ contains the generated documentation of the service API
- generated/ contains the wiring and encoding protocols necessary for service communication
- handlers/server/server_handler.go is populated with stubs where you will add the business logic
- middlewares/ is where you can put the middlewares (NOP by default)
- echo-cli-client/ contains the client side CLI (useful for testing)
- echo-server/ contains the service main, which you will build and run shortly
- echo.pb.go contains the RPC interface definitions and supporting structures that have been translated from echo.proto to golang
From the top down, within `echo-service/`:
- `docs/` contains the generated documentation of the service API
- `svc/` contains the wiring and encoding protocols necessary for service communication (generated code)
- `handlers/handlers.go` is populated with stubs where you will add the business logic
- `middlewares/` is where you can put the middlewares (NOP by default)
- `cmd/echo/` contains the client side CLI (useful for testing)
- `cmd/echo-server/` contains the service main, which you will build and run shortly
- `echo.pb.go` contains the RPC interface definitions and supporting structures that have been translated from `echo.proto` to golang
If you try to build and run your service now, it will respond with empty messages. There is no business logic yet! We shall add it in the next step.
@ -100,7 +102,7 @@ You can safely modify only the files in handlers/ and middlewares/. Changes to a
## Implement business logic
Open `handlers/server/server_handler.go` using your favorite editor. Find the Echo function stub. It should look like this:
Open `handlers/handlers.go` using your favorite editor. Find the Echo function stub. It should look like this:
```
func (s echoService) Echo(ctx context.Context, in *pb.EchoRequest) (*pb.EchoResponse, error) {
var resp pb.EchoResponse
@ -125,25 +127,25 @@ func (s echoService) Echo(ctx context.Context, in *pb.EchoRequest) (*pb.EchoResp
## Build/Run the client and server executables
From the directory containing echo.proto run
`go build echo-service/echo-cli-client/client_main.go` and
`go build echo-service/echo-server/server_main.go`
`go build echo-service/echo` and
`go build echo-service/echo`
Create another terminal window to run the server in, navigate to the same directory and launch the server:
`./server_main`
`./echo-server`
When server starts up, you should see something like this:
```
ts=2016-12-06T23:25:14Z caller=server_main.go:55 msg=hello
ts=2016-12-06T23:25:14Z caller=server_main.go:106 transport=HTTP addr=:5050
ts=2016-12-06T23:25:14Z caller=server_main.go:98 transport=debug addr=:5060
ts=2016-12-06T23:25:14Z caller=server_main.go:124 transport=gRPC addr=:5040
ts=2016-12-06T23:25:14Z caller=main.go:55 msg=hello
ts=2016-12-06T23:25:14Z caller=main.go:106 transport=HTTP addr=:5050
ts=2016-12-06T23:25:14Z caller=main.go:98 transport=debug addr=:5060
ts=2016-12-06T23:25:14Z caller=main.go:124 transport=gRPC addr=:5040
```
The server is now waiting for incoming messages.
At this point we can send a request to the server via networking tools (telnet, curl) and construct message directly, or we can use the client CLI.
Let's do the latter, in your first terminal. To learn how to launch client with proper parameters run `./client_main -help`. The printout will tell you what methods the service supports (-method flag options) and all the additional flags that must be set to call a certain method (flags of format -method_name.parameter_name).
Let's do the latter, in your first terminal. To learn how to launch client with proper parameters run `./echo -help`. The printout will tell you what methods the service supports and all the additional flags that must be set to call a certain method
Now run `./client_main -method echo -echo.in “hello microservices”`
Now run `./echo echo -in “hello microservices”`
The client terminal will display messages that were sent and received.
You can also specify the address to send messages to via -grpc.addr or -http.addr flags (e.g. `-grpc.addr localhost:5040`), should you want to change the port the server runs on, or test it out on separate machines.
@ -152,14 +154,14 @@ To shutdown the server, press Ctrl+C in the server terminal
## Implement more things!
The following is left as an exersise to the reader:
The following is left as an exercise to the reader:
- Implement logic for the Louder call
- code the logic inside the stub
- now separate this logic into an unexported helper function
- Define a new RPC call in echo.proto
- regenerate service with truss, check that your old logic remains
- implement the logic for your new call in a separate package, place it ouside of echo-service
- wire in the new logic by importing the package in the server_handler.go
- wire in the new logic by importing the package in the `handlers.go`
Suggestion: Save everything the service hears and echo all of it back. See repeated types (protobuf), package variables and init() function (golang).
- Remove an RPC call definition from echo.proto
- regenerate service with truss, verify that the call no longer exists
@ -173,16 +175,16 @@ The following is left as an exersise to the reader:
You can control the location of the output folders for your service by specifying the following flags when running truss
```
-svcout {go-style-package-path to where you want the {Name}-service folder to be}
-pbout {go-style-package-path to where you want the *.pb.go interface definitions to be}
--svcout {go-style-package-path to where you want the contents of {Name}-service folder to be}
--pbout {go-style-package-path to where you want the *.pb.go interface definitions to be}
```
Note: “go-style-package-path” means exactly the style you use in your golang import statements, relative to your $GOPATH. This is not your system file path, nor it is relative to location of the *.proto file; the start of the path must be accessible from your $GOPATH. Also no “/” at the end.
Note: “go-style-package-path” means exactly the style you use in your golang import statements, relative to your $GOPATH. This is not your system file path, nor it is relative to location of the *.proto file; the start of the path must be accessible from your $GOPATH.
For example:
```
truss -pbout truss-demo/interface-defs -svcout truss-demo/service echo.proto
truss --pbout truss-demo/interface-defs --svcout truss-demo/service echo.proto
```
Executing this command will place the *.pb.go files into `$GOPATH/truss-demo/interface-defs/`, and the entire echo-service directory (excepting the *.pb.go files) to `$GOPATH/truss-demo/service/`.
Executing this command will place the *.pb.go files into `$GOPATH/truss-demo/interface-defs/`, and the entire echo-service contents (excepting the *.pb.go files) to `$GOPATH/truss-demo/service/`.
## Middlewares

Просмотреть файл

@ -1,66 +1,11 @@
# Using truss
## File structure
Start with your service definition file, let's name it `svc.proto`.
For a more detailed example of a simple service definition see [echo.proto](./_example/echo.proto),
but for now we only care about the following structure:
```
// The package name determines the name of the directories that truss creates
package NAME;
// RPC interface definitions
...
```
The current directory should look like this:
```
.
└── svc.proto
```
Run truss on your service definition file: `truss svc.proto`.
Upon success, `NAME-service` folder will be created in your current directory.
(`NAME-service`, where NAME is the name of the package defined in your definition file.)
Your directory structure will look like this:
```
.
├── NAME-service
│ ├── docs
│ │   └── docs.md
│ ├── generated
│ │   └── ...
│ ├── handlers
│ │   └── server
│ │   └── server_handler.go
│ ├── middlewares
│ │   └── ...
│ ├── NAME-cli-client
│ │   └── client_main.go
│ ├── NAME-server
│ │   └── server_main.go
│ └── svc.pb.go
├── svc.proto
```
Now that you've generated your service, you can install the generated binaries
with `go install ./...` which will install `NAME-cli-client` and `NAME-server`,
where NAME is the name of the package defined in your definition file.
To add business logic, edit the `server_handler.go` file in `./NAME-service/handlers/server/`.
To add middlewares, edit ... (TODO)
## Our Contract
1. Modify ONLY the files in `handlers/` and `middlewares/`.
User logic can be imported and executed within the functions in the handlers. It can also be added as _unexported_ helper functions in the handler file.
User logic can be imported and executed within the functions in the handlers. It can also be added as _unexported_ helper functions in the handler file.
Truss will enforce that exported functions in `server_handler.go` conform to the rpc interface defined in the service *.proto files. All other exported functions will be removed upon next re-run of truss.
Truss will enforce that exported functions in `handlers/handlers.go` conform to the rpc interface defined in the service *.proto files. All other exported functions will be removed upon next re-run of truss.
2. DO NOT create files or directories in `NAME-service/`
All user logic must exist outside of `NAME-service/`, leaving organization of that logic up to the user.

Просмотреть файл

@ -29,18 +29,27 @@ func TestMain(m *testing.M) {
}
basePath = filepath.Join(wd, definitionDirectory)
// Create a standalone copy of the 'basic' service binary that persists
// through all the tests. This binary to be used to test things like flags
// and flag-groups.
exitCode := 1
defer func() {
if exitCode == 0 {
cleanTests(basePath)
}
os.Exit(exitCode)
}()
clean := flag.Bool("clean", false, "Remove all generated test files and do nothing else")
flag.Parse()
if *clean {
cleanTests(basePath)
os.Exit(0)
exitCode = 0
return
}
// Create a standalone copy of the 'basic' service binary that persists
// through all the tests. This binary to be used to test things like flags
// and flag-groups.
exitCode := 0
// Cleanup so that cp works as expected
cleanTests(basePath)
// Copy "1-basic" into a special "0-basic" which is removed
copy := exec.Command(
@ -54,7 +63,7 @@ func TestMain(m *testing.M) {
err = copy.Run()
if err != nil {
fmt.Printf("cannot copy '0-basic' service: %v", err)
os.Exit(1)
return
}
path := filepath.Join(basePath, "0-basic")
@ -62,20 +71,15 @@ func TestMain(m *testing.M) {
err = createTrussService(path)
if err != nil {
fmt.Printf("cannot create truss service: %v", err)
os.Exit(1)
return
}
err = buildTestService(filepath.Join(path, "test-service"))
if err != nil {
fmt.Printf("cannot build truss service: %v", err)
os.Exit(1)
return
}
defer func() {
os.RemoveAll(filepath.Join(basePath, "0-basic"))
os.Exit(exitCode)
}()
exitCode = m.Run()
}
@ -210,13 +214,9 @@ func testEndToEnd(defDir string, subcmd string, t *testing.T, trussOptions ...st
t.FailNow()
}
// If nothing failed, delete the generated files
removeTestFiles(path)
}
func createTrussService(path string, trussFlags ...string) error {
// Remove tests if they exists
removeTestFiles(path)
trussOut, err := truss(path, trussFlags...)
@ -275,14 +275,14 @@ func buildTestService(serviceDir string) (err error) {
return err
}
const serverPath = "/test-server"
const clientPath = "/test-cli-client"
const serverPath = "cmd/test-server"
const clientPath = "cmd/test"
// Build server and client
errChan := make(chan error)
go goBuild("test-server", binDir, filepath.Join(relDir, serverPath), errChan)
go goBuild("test-cli-client", binDir, filepath.Join(relDir, clientPath), errChan)
go goBuild("test", binDir, filepath.Join(relDir, clientPath), errChan)
err = <-errChan
if err != nil {
@ -399,7 +399,7 @@ func reapServer(server *exec.Cmd, errc chan error) error {
}
func runClient(path string, flags ...string) ([]byte, error) {
const relativeClientPath = "/bin/test-cli-client"
const relativeClientPath = "/bin/test"
client := exec.Command(
path+relativeClientPath,
@ -420,6 +420,8 @@ func fileExists(path string) bool {
// cleanTests removes all test files from all directories in servicesDir
func cleanTests(servicesDir string) {
// Remove the 0-basic used for non building tests
os.RemoveAll(filepath.Join(servicesDir, "0-basic"))
// Clean up the service directories in each test
dirs, _ := ioutil.ReadDir(servicesDir)
for _, d := range dirs {
@ -434,9 +436,16 @@ func cleanTests(servicesDir string) {
// removeTestFiles removes all files created by running truss and building the
// service from a single definition directory
func removeTestFiles(defDir string) {
// svcout dir
os.RemoveAll(filepath.Join(defDir, "tunelab"))
// service dir
os.RemoveAll(filepath.Join(defDir, "test-service"))
// where the binaries are compiled to
os.RemoveAll(filepath.Join(defDir, "bin"))
// test pbout
os.RemoveAll(filepath.Join(defDir, "pbout"))
// So that the directory exists for pbout
// TODO: Make pbout create the directory if it does not exist
os.MkdirAll(filepath.Join(defDir, "pbout"), 0777)
}

Просмотреть файл

@ -4,7 +4,7 @@ import (
"github.com/go-kit/kit/endpoint"
"golang.org/x/net/context"
svc "github.com/TuneLab/go-truss/cmd/_integration-tests/middlewares/middlewarestest-service/generated"
svc "github.com/TuneLab/go-truss/cmd/_integration-tests/middlewares/middlewarestest-service/svc"
)
// WrapEndpoints accepts the service's entire collection of endpoints, so that a

Просмотреть файл

@ -5,7 +5,7 @@ import (
"testing"
pb "github.com/TuneLab/go-truss/cmd/_integration-tests/middlewares/middlewarestest-service"
svc "github.com/TuneLab/go-truss/cmd/_integration-tests/middlewares/middlewarestest-service/generated"
svc "github.com/TuneLab/go-truss/cmd/_integration-tests/middlewares/middlewarestest-service/svc"
handler "github.com/TuneLab/go-truss/cmd/_integration-tests/middlewares/middlewarestest-service/handlers/server"
"github.com/TuneLab/go-truss/cmd/_integration-tests/middlewares/middlewarestest-service/middlewares"
)

Просмотреть файл

@ -9,7 +9,7 @@ import (
"google.golang.org/grpc"
pb "github.com/TuneLab/go-truss/cmd/_integration-tests/transport/transportpermutations-service"
grpcclient "github.com/TuneLab/go-truss/cmd/_integration-tests/transport/transportpermutations-service/generated/client/grpc"
grpcclient "github.com/TuneLab/go-truss/cmd/_integration-tests/transport/transportpermutations-service/svc/client/grpc"
)
var grpcAddr string

Просмотреть файл

@ -15,7 +15,7 @@ import (
"golang.org/x/net/context"
// This Service
pb "github.com/TuneLab/go-truss/cmd/_integration-tests/transport/transportpermutations-service"
httpclient "github.com/TuneLab/go-truss/cmd/_integration-tests/transport/transportpermutations-service/generated/client/http"
httpclient "github.com/TuneLab/go-truss/cmd/_integration-tests/transport/transportpermutations-service/svc/client/http"
"github.com/pkg/errors"
)

Просмотреть файл

@ -15,7 +15,7 @@ import (
"github.com/go-kit/kit/log"
pb "github.com/TuneLab/go-truss/cmd/_integration-tests/transport/transportpermutations-service"
svc "github.com/TuneLab/go-truss/cmd/_integration-tests/transport/transportpermutations-service/generated"
svc "github.com/TuneLab/go-truss/cmd/_integration-tests/transport/transportpermutations-service/svc"
handler "github.com/TuneLab/go-truss/cmd/_integration-tests/transport/transportpermutations-service/handlers/server"
)

Просмотреть файл

@ -87,7 +87,7 @@ func TestApplyTemplateFromPath(t *testing.T) {
t.Fatal(err)
}
end, err := applyTemplateFromPath("generated/endpoints.gotemplate", te)
end, err := applyTemplateFromPath("svc/endpoints.gotemplate", te)
if err != nil {
t.Fatal(err)
}

Просмотреть файл

@ -23,7 +23,7 @@ import (
const ignoredFunc = "NewService"
// ServerHadlerPath is the relative path to the server handler template file
const ServerHandlerPath = "handlers/server/server_handler.gotemplate"
const ServerHandlerPath = "handlers/handlers.gotemplate"
// New returns a truss.Renderable capable of updating server handlers.
// New should be passed the previous version of the server handler to parse.

Просмотреть файл

@ -134,7 +134,7 @@ func TestApplyServerTempl(t *testing.T) {
gen, err := applyServerTempl(te)
genBytes, err := ioutil.ReadAll(gen)
expected := `
package handler
package handlers
import (
"golang.org/x/net/context"
@ -304,7 +304,7 @@ func TestPruneDecls(t *testing.T) {
m := newMethodMap(sd.Service.Methods)
prev := `
package handler
package handlers
import (
"golang.org/x/net/context"

Просмотреть файл

@ -7,7 +7,7 @@ import (
"github.com/TuneLab/go-truss/gengokit"
)
const HookPath = "handlers/server/hooks.gotemplate"
const HookPath = "handlers/hooks.gotemplate"
// NewHook returns a new HookRender
func NewHook(prev io.Reader) gengokit.Renderable {

Просмотреть файл

@ -18,7 +18,7 @@ const serverMethsTempl = `
`
const serverTempl = `
package handler
package handlers
import (
"golang.org/x/net/context"
@ -49,7 +49,7 @@ type {{ToLower .Service.Name}}Service struct{}
{{- end}}
`
const hookTempl = `
package handler
package handlers
import (
"fmt"

Просмотреть файл

@ -181,6 +181,10 @@ func RemoveBraces(val string) string {
var HTTPAssistFuncs = PathParamsTemplate + BuildParamMapTemplate + RemoveBracesTemplate
var serverTemplate = `
// Code generated by truss.
// Rerunning truss will overwrite this file.
// DO NOT EDIT!
package svc
// This file provides server-side bindings for the HTTP transport.
@ -329,6 +333,10 @@ func headersToContext(ctx context.Context, r *http.Request) context.Context {
`
var clientTemplate = `
// Code generated by truss.
// Rerunning truss will overwrite this file.
// DO NOT EDIT!
// Package http provides an HTTP client for the {{.Service.Name}} service.
package http
@ -343,7 +351,7 @@ import (
"golang.org/x/net/context"
// This Service
svc "{{.ImportPath -}} /generated"
svc "{{.ImportPath -}} /svc"
pb "{{.PBImportPath -}}"
)

Просмотреть файл

@ -60,7 +60,7 @@ func TestNewEndpointMiddleware(t *testing.T) {
import (
_ "github.com/go-kit/kit/endpoint"
svc "github.com/TuneLab/go-truss/gengokit/general-service/generated"
svc "github.com/TuneLab/go-truss/gengokit/general-service/svc"
)
// WrapEndpoints accepts the service's entire collection of endpoints, so that a
@ -152,7 +152,7 @@ func TestRenderPrevEndpoints(t *testing.T) {
import (
"github.com/go-kit/kit/endpoint"
svc "github.com/TuneLab/go-truss/gengokit/general-service/generated"
svc "github.com/TuneLab/go-truss/gengokit/general-service/svc"
)
// WrapEndpoint will be called individually for all endpoints defined in

Просмотреть файл

@ -6,7 +6,7 @@ package middlewares
import (
_ "github.com/go-kit/kit/endpoint"
svc "{{.ImportPath -}} /generated"
svc "{{.ImportPath -}} /svc"
)
// WrapEndpoints accepts the service's entire collection of endpoints, so that a

Просмотреть файл

@ -1,3 +1,7 @@
// Code generated by truss.
// Rerunning truss will overwrite this file.
// DO NOT EDIT!
package main
import (
@ -16,8 +20,8 @@ import (
"github.com/go-kit/kit/log"
// This Service
handler "{{.ImportPath -}} /handlers/server"
svc "{{.ImportPath -}} /generated"
handler "{{.ImportPath -}} /handlers"
svc "{{.ImportPath -}} /svc"
pb "{{.PBImportPath -}}"
middlewares "{{.ImportPath -}} /middlewares"
)

Просмотреть файл

@ -1,3 +1,7 @@
// Code generated by truss.
// Rerunning truss will overwrite this file.
// DO NOT EDIT!
package main
import (
@ -15,9 +19,9 @@ import (
"github.com/pkg/errors"
// This Service
clientHandler "{{.ImportPath -}} /generated/cli/handlers"
grpcclient "{{.ImportPath -}} /generated/client/grpc"
httpclient "{{.ImportPath -}} /generated/client/http"
clientHandler "{{.ImportPath -}} /svc/cli/handlers"
grpcclient "{{.ImportPath -}} /svc/client/grpc"
httpclient "{{.ImportPath -}} /svc/client/http"
pb "{{.PBImportPath -}}"
)

Просмотреть файл

@ -1,3 +1,7 @@
// Code generated by truss.
// Rerunning truss will overwrite this file.
// DO NOT EDIT!
package clienthandler
import (

Просмотреть файл

@ -1,3 +1,7 @@
// Code generated by truss.
// Rerunning truss will overwrite this file.
// DO NOT EDIT!
// Package grpc provides a gRPC client for the {{.Service.Name}} service.
package grpc
@ -11,7 +15,7 @@ import (
grpctransport "github.com/go-kit/kit/transport/grpc"
// This Service
svc "{{.ImportPath -}} /generated"
svc "{{.ImportPath -}} /svc"
pb "{{.PBImportPath -}}"
)

Просмотреть файл

@ -1,3 +1,7 @@
// Code generated by truss.
// Rerunning truss will overwrite this file.
// DO NOT EDIT!
package svc
// This file contains methods to make individual endpoints from services,

Просмотреть файл

@ -1,3 +1,7 @@
// Code generated by truss.
// Rerunning truss will overwrite this file.
// DO NOT EDIT!
package svc
// This file provides server-side bindings for the gRPC transport.

Различия файлов скрыты, потому что одна или несколько строк слишком длинны