Azure
/
draft-classic
зеркало из https://github.com/Azure/draft-classic.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность

first draft of docs

This commit is contained in:
Matthew Fisher 2017-02-09 14:27:59 -08:00
Родитель 78718234f6
Коммит 522be81733
5 изменённых файлов: 405 добавлений и 330 удалений
Показать статистику Скачать Patch файл Скачать Diff файл Показать все файлы Свернуть все файлы

4
Makefile
Просмотреть файл

@ -85,6 +85,10 @@ test: test-unit
test-unit:
$(GO) test $(GOFLAGS) -run $(TESTS) $(PKG) $(TESTFLAGS)
.PHONY: test-e2e
test-e2e:
./tests/e2e.sh
HAS_GLIDE := $(shell command -v glide;)
HAS_GOX := $(shell command -v gox;)
HAS_GIT := $(shell command -v git;)

97
README.md
Просмотреть файл

@ -2,57 +2,39 @@
_NOTE: Prow is experimental and does not have a stable release yet._
Prow is a tool for developers to create cloud-native applications on [Kubernetes][].
Prow is a tool for developers to create and deploy cloud-native applications on [Kubernetes][]. It
can be used to
- Install any application onto Kubernetes; Prow is language-agnostic and customizable.
- Work with any Kubernetes cluster that supports [Helm][]
- Intelligently manage and upgrade Helm releases
- Create Helm Charts that can be packaged for delivery to your ops team
## Prow, the Elevator Pitch
Prow is a tool to install applications onto a Kubernetes cluster.
Prow has two parts: a client (prow) and a server (prowd). Prowd runs inside of your Kubernetes
cluster alongside [Tiller][helm] and packages a Docker image, uploads it to a Docker Registry and
installs a chart using that image. Prow runs on your laptop, CI/CD, or wherever you want it to run
and uploads your application to Prowd via `prow up`.
## Usage
_NOTE: These instructions are temporary until there is an official binary release._
Because Prow is currently experimental, there is no stable release out yet and users are expected
to build the project from source until we get some automation up in here. Please see
[this doc][hacking] to get started hacking on Prow.
Here's an example of installing Prow and creating an application:
```
$ go version
go version go1.7 linux/amd64
$ # install prowd
$ helm init
$ export IMAGE_PREFIX=bacongobbler
$ make info
Build tag: git-abc1234
Registry: quay.io
Immutable tag: quay.io/bacongobbler/prowd:git-abc1234
Mutable tag: quay.io/bacongobbler/prowd:canary
$ make bootstrap docker-build docker-push
$ cat chart/values.yaml | grep 'registry\|name'
registry: quay.io
name: deis/prowd
$ helm install ./chart --namespace prow --set image.name=${IMAGE_PREFIX}/prowd
$ make build && export PATH=$PWD/bin:$PATH
$ cd tests/testdata/example-dockerfile-http
$ prow up
--> Building Dockerfile
--> Pushing 127.0.0.1:5000/example-dockerfile-http:fc8c34ba4349ce3771e728b15ead2bb4c81cb9fd
--> Deploying to Kubernetes
Release "example-dockerfile-http" does not exist. Installing it now.
--> code:DEPLOYED
$ helm list
NAME REVISION UPDATED STATUS CHART
example-dockerfile-http 1 Tue Jan 24 15:04:27 2017 DEPLOYED example-dockerfile-http-1.0.0
$ kubectl get po
NAME READY STATUS RESTARTS AGE
example-dockerfile-http-3666132817-m2pkr 1/1 Running 0 30s
```
_NOTE: CLI usage will look like this when it is completed._
Start from a source code repository and let Prow create the Kubernetes packaging:
Now that's all done, start from a source code repository and let Prow create the Kubernetes
packaging:
```
$ cd my-app
$ ls
app.py
$ prow create --pack=python
--> Created ./charts/my-app
--> Created ./charts/my-app/Dockerfile
--> Created chart/
--> Created Dockerfile
--> Ready to sail
```
@ -61,8 +43,9 @@ Now start it up!
```
$ prow up
--> Building Dockerfile
--> Pushing my-app:latest
--> Pushing 127.0.0.1:5000/my-app:08db751fccc4fc01c8b2ee96111525ea43f064f2
--> Deploying to Kubernetes
Release "my-app" does not exist. Installing it now.
--> code:DEPLOYED
```
@ -72,37 +55,13 @@ Behind the scenes, Prow handles the heavy lifting:
- Builds a container image from application source code
- Pushes the image to a registry
- Creates a [Helm][] chart for the image
- Packages a [Helm][] chart from application source code
- Installs the chart to Kubernetes, deploying the application
After deploying, you can run `prow up` again to create new releases when
application source code has changed. Or you can let Prow continuously rebuild
your application.
application source code has changed.
## Hacking on Prow
The easiest way to hack on Prow is to deploy changes... Using Prow. Check it out!
```
$ helm list
NAME REVISION UPDATED STATUS CHART
warped-nightingale 1 Wed Feb 8 10:29:19 2017 DEPLOYED prowd-0.1.0
$ make build docker-binary
GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build -o ./rootfs/bin/prowd -a -installsuffix cgo -tags '' -ldflags ' -X github.com/deis/prow/pkg/version.Version= -X github.com/deis/prow/pkg/version.GitCommit=d46a18200576d6ba3007ed26efb647ffd86303ba -X github.com/deis/prow/pkg/version.GitTreeState=clean' github.com/deis/prow/cmd/prowd
$ ./bin/prow up --app warped-nightingale
--> Building Dockerfile
--> Pushing 127.0.0.1:5000/warped-nightingale:6f3b53003dcbf43821aea43208fc51455674d00e
--> Deploying to Kubernetes
--> code:DEPLOYED
```
## Features
- Prow is language-agnostic
- Prow works with any Kubernetes cluster that supports [Helm][]
- Charts that Prow creates can be edited to suit your needs
- Charts can be packaged for delivery to your ops team
[hacking]: docs/hacking.md
[Kubernetes]: https://kubernetes.io/
[Helm]: https://github.com/kubernetes/helm

261
design.md
Просмотреть файл

@ -1,261 +0,0 @@
# Prow: Cloud Native Development
Prow is a tool for building Kubernetes-aware applications with ease. You bring the source code and Prow will negotiate the Kubernetes waters for you.
**ROUGH DRAFT**
## TL;DR
Start from your source code repository and let Prow transform it for
Kubernetes:
```
$ cd my-app
$ ls
app.py
$ prow create my-app --pack=python
--> Created ./charts/my-app
--> Created ./charts/my-app/Dockerfile
--> Ready to sail
```
Now start it up!
```
$ prow up
--> Building Dockerfile
--> Pushing my-app:latest
--> Deploying to Kubernetes
--> Ready at 10.21.77.7:8080
```
That's it! You're now running your Python app in a Kubernetes cluster.
## What It Does
Prow is a tool for developing, organizing, packaging, deploying, and
managing applications in the Kubernetes ecosystem. It uses Helm for
orchestration. But it provides a toolbox for developers building
applications composed of one or more microservices.
## How To Use It
Prow has two main commands:
- `prow create` takes your existing code and creates a new Kubernetes app
- `prow up` deploys a development copy of your app into a Kubernetes
cluster. Optionally, you can have it watch your code for changes,
re-deploying automatically with each change.
## Start from a Dockerfile
Say you have an application that is already Dockerized, but not Helm-ified. Prow can start
from your existing Dockerfile and create a new app:
```
$ ls
Dockerfile
src/
$ prow create
--> Created ./chart
--> Ready to sail
$ ls
chart/
Dockerfile
src/
```
In the example above, `prow create` created a new application and
based configuration off of the Dockerfile. Prow constructed a new
Helm Chart for you, and stored it alongside your source code so
that you can add it to version control, and even manually edit it.
## Start from Scratch
If you want to start with an empty Prow app, you can simply run `prow
create` and it will scaffold a chart and a Dockerfile for you.
## Start from Code
The example in the _tl;dr_ section of this document showed starting from
existing source code. In this case, you must use a Prow _pack_ (a
preconfigured template for your chart) to tell Prow how to use your
source code. Prow provides some default packs, but it's also easy to add
custom ones.
## Start from an Existing Chart
If you've already created Kubernetes applications, you can start with an
existing chart, and simply begin using Prow. There are a few patterns
you may need to follow to meet the expectations for Prow, but this is a
matter of a few minutes of work; not a complete refactoring.
In this case, you don't even need `prow create`. You can just create the
directory structure and sail onward.
## Running Your Code
When you run `prow up`, Prow deploys your code to your Kubernetes
cluster for you. It does the following:
- Package your code using a Docker build.
- Send your code to the in-cluster Docker registry.
- Install (or upgrade) your chart using Helm
- (Optionally) watch your source code for changes, redeploying when
necessary.
And when You're done with development, Prow's "first class" objects
are all supported by the Kubernetes toolchain. Simply deploy the chart
to your production cluster in whatever way suits you.
## Prow Packs
Prow creates new charts based on two pieces of data: The information it
can learn from your project's directory structure, and a pre-build
"scaffold" called a _pack_.
A Prow pack contains one or two things:
- A Helm Chart scaffold
- A base Dockerfile (optional; used if a project does not contain a Dockerfile)
Prow ships with a `default` pack and a few basic alternative packs. The
default pack simply builds a chart with a service and a replica set.
From this, you can tailor packs to your specific needs.
- Build a language-specific pack that, for example, packages a Ruby
Rails app.
- Build a pack tailored to your team's DevOps needs or designs
- Build a pack that leverages custom Kubernetes ThirdPartyResource
types.
## How Prow Works
This is a look behind the curtain. Here's how Prow works:
- Prow uses several existing components:
- A Kubernetes cluster
- The Helm Tiller server
- An in-cluster Docker registry
- A repository of "packs" for specific templates
- While it's up to you to find the Kubernetes cluster and install Tiller,
Prow can install the rest for you should you so desire with `prow init`.
- `prow create` reads a scaffold out of the appropriate Pack, creates the
necessary file system objects, and writes some basic configuration
into your chart.
- `prow up` delivers your chart into the Kubernetes cluster (think `helm
upgrade --install my-app my-app`)
- `prow up` can be configured to either start your app and leave it
alone, or watch your local code for changes.
## Directory Structure
Imagine an app named `my-app`, which contains a Dockerfile and some
source:
```
myapp/
Dockerfile
src/
app.py
```
After running `prow create`, this directory would have a chart built for it:
```
myapp/
chart/
Chart.yaml
templates/
deployment.yaml
service.yaml
values.yaml
Dockerfile
src/
app.py
```
The `chart/` directory is a complete Helm chart, tailored to your
needs.
Inside of the `values.yaml` file, Prow configures images for your chart:
```
name: myapp
version: 0.1.0
registry: gcr.io
```
This information is then available to all of your Helm charts. (e.g. via
`{{.Values.name}}`)
The contents of the `templates/` directory are determined by the
particular Pack you've used. The `default` pack just creates a
replicated service for you.
### Questions and Answers
_Can I have multiple Prow charts in the same repo?_
At this time, no.
_Can I modify the chart, or do I have to accept whatever the pack gives
me?_
You can modify the contents of the `chart/` directory as you wish.
Consider them part of your source code.
_How do I add an existing chart to Prow?_
Just copy (`helm fetch`) it into the `chart/` directory. You need to
tweak the values file if you want Prow to regenerate Docker images for
you.
_How do I deploy applications to production?_
Prow is a developer tool. While you _could_ simply use `prow up` to do
this, we'd recommend using `helm` in conjuction with a CI/CD pipeline or
whatever tooling you're most comfortable with.
Remember: You can package a Prow-generated chart with `helm package chart/`
and load the results up to a chart repository, taking advantage of the
existing Helm ecosystem.
## Other Architectural Considerations
- instead of a prowd HTTP server, we could spawn a prow pod "job" (via `prow up`)
that runs only when `prow up` is called. In that case, the `prow` client
would be the main focal point for server-side configuration. This has the
advantage of requiring fewer resource demands server-side, but might make the
client implementation (and security story) significantly more difficult. Furthermore,
it might make two `prow up` operations between two clients differ
(the "Works on My Machine!" problem).
## User Personas and Stories
**Persona:** Inexperienced Kube Dev
This user wants to just work in their normal environment, but be able to
deploy to Kubernetes without any extra work.
**Persona:** Kubernetes App Developer
This user knows all about Kubernetes, but doesn't enjoy the hassle of
scaffolding out the same old stuff. This user wants a _starting point_.
- As a user, I want to create a new Kubernetes app...
- from scratch
- from existing code
- from a Dockerfile
- from a chart
- from some Kubernetes manifest files
- As a user, I want my app deployed quickly to a dev cluster
- As a user, I want to code, and have the deployed version auto-update
- As a user, I want to be as ignorant of the internals as possible, but
still be able to GSD.
- As a user, I want to be able to hand off code artifacts without having
to prepare them.

221
docs/design.md Normal file
Просмотреть файл

@ -0,0 +1,221 @@
# Prow Design
This document outlines the general design principles of Prow; what it does, how it works, what it
can (and cannot) do, as well as define its intended audience and place in the market.
## What It Does
Prow is a tool for developing, organizing, packaging, deploying, and managing applications in the
Kubernetes ecosystem. It uses Helm for orchestration, but it provides a toolbox for developers
building applications on top of Kubernetes.
## How To Use It
Prow has two main commands:
- `prow create` takes your existing code and creates a new Kubernetes app
- `prow up` deploys a development copy of your app into a Kubernetes cluster.
## Start from a Dockerfile
Say you have an application that is already Dockerized, but not Helm-ified. Prow can start from
your existing Dockerfile and create a new app:
```
$ ls
Dockerfile
src/
$ prow create
--> Created chart/
--> Dockerfile already exists, skipping
--> Ready to sail
$ ls
chart/
Dockerfile
src/
```
In the example above, `prow create` constructed a new Helm Chart for you, and stored it alongside
your source code so that you can add it to version control, and even manually edit it.
## Start from Scratch
If you want to start with an empty Prow app, you can simply run `prow create` and it will scaffold
a chart and a Dockerfile for you.
## Start from Code
The example in the _Start from Scratch_ section of this document showed starting from existing
source code. In this case, you must use a [Prow _pack_](packs.md) (a preconfigured template for
your chart) to tell Prow how to use your source code. Prow provides some default packs, but it's
also easy to add custom ones.
## Start from an Existing Chart
If you've already created Kubernetes applications, you can start with an existing chart, and simply
begin using Prow. There are a few patterns you may need to follow to meet the expectations for
Prow, but this is a matter of a few minutes of work; not a complete refactoring.
In this case, you don't even need `prow create`. You can just create the directory structure and
sail onward.
## Running Your Code
When you run `prow up`, Prow deploys your code to your Kubernetes cluster for you. It does the
following:
- Packages your code using a `docker build`.
- Sends your code to the in-cluster Docker Registry.
- Installs (or upgrades) your chart using Helm
And when you're done with development, Prow's "first class" objects are all supported by the
Kubernetes toolchain. Simply deploy the chart to your production cluster in whatever way suits you.
## Prow Packs
Prow creates new charts based on two pieces of data: The information it can learn from your
project's directory structure, and a pre-built "scaffold" called a _pack_.
A Prow pack contains one or two things:
- A Helm Chart scaffold
- A base Dockerfile
Prow ships with a `default` pack and a few basic alternative packs. The default pack simply builds
a default Helm Chart and a Dockerfile that points to `nginx:latest`.
From this, you can tailor packs to your specific needs.
- Build a language-specific or library-specific pack that, for example, packages a Ruby
Rails app
- Build a pack tailored to your team's DevOps needs or designs
- Build a pack that leverages custom Kubernetes ThirdPartyResource
types.
See [packs.md](packs.md) for more information.
## How Prow Works
This is a look behind the curtain. Here's how Prow works:
- Prow uses several existing components:
- A Kubernetes cluster
- The Helm Tiller server
- An in-cluster Docker Registry
- A directory full of "packs" for specific templates
- `prow create` reads a scaffold out of the appropriate pack, creates the necessary file system
objects, and writes some basic configuration into your chart.
- `prow up` delivers your chart into the Kubernetes cluster (think
`helm upgrade --install my-app my-app`)
## Directory Structure
Imagine an app named `my-app`, which contains a Dockerfile and some source:
```
myapp/
Dockerfile
src/
app.py
```
After running `prow create`, this directory would have a chart built for it:
```
myapp/
chart/
Chart.yaml
templates/
deployment.yaml
service.yaml
values.yaml
Dockerfile
src/
app.py
```
The `chart/` directory is a complete Helm chart, tailored to your
needs.
Inside of the `values.yaml` file, Prow configures images for your chart:
```
image:
registry: gcr.io
name: myapp
tag: 0.1.0
```
This information is then available to all of your Helm charts. (e.g. via `{{.Values.image.name}}`)
The contents of the `templates/` directory are determined by the particular Pack you've used.
### Questions and Answers
_Can I have multiple Prow charts in the same repo?_
At this time, no. You can however use a `requirements.yaml` in your chart to note what your chart
depends on.
_Can I modify the chart, or do I have to accept whatever the pack gives
me?_
You can modify the contents of the `chart/` directory as you wish.
Consider them part of your source code.
Keep in mind that there are three values injected from Prowd into the chart which you'll likely
want to use:
```
image:
registry: 10.0.0.131:5000 # the address of the in-cluster registry
name: myapp # the name of the image
tag: 08db751 # the release of the image in the registry
```
_How do I add an existing chart to Prow?_
Just copy (`helm fetch`) it into the `chart/` directory. You need to tweak the values file to
read from `image.regsitry`, `image.name` and `image.tag` if you want Prow to regenerate Docker
images for you. See above.
_How do I deploy applications to production?_
Prow is a developer tool. While you _could_ simply use `prow up` to do this, we'd recommend using
`helm package` in conjuction with a CI/CD pipeline.
Remember: You can always package a Prow-generated chart with `helm package chart/` and load the
results up to a chart repository, taking advantage of the existing Helm ecosystem.
## Other Architectural Considerations
Instead of a prowd HTTP server, we could spawn a prow pod "job" (via `prow up`) that runs only when
`prow up` is called. In that case, the `prow` client would be the main focal point for server-side
configuration. This has the advantage of requiring fewer resource demands server-side, but might
make the client implementation (and security story) significantly more difficult. Furthermore, it
might make two `prow up` operations between two clients differ (the "Works on My Machine!"
problem).
## User Personas and Stories
**Persona:** Inexperienced Kube Dev
This user wants to just work in their normal environment, but be able to deploy to Kubernetes
without any extra work.
**Persona:** Kubernetes App Developer
This user knows all about Kubernetes, but doesn't enjoy the hassle of scaffolding out the same old
stuff when testing changes. This user wants a _starting point_ to deploy in-progress changes to a
cluster.
- As a user, I want to create a new Kubernetes app...
- from scratch
- from existing code
- from a Dockerfile
- from a chart
- from some Kubernetes manifest files
- As a user, I want my app deployed quickly to a dev cluster
- As a user, I want to code, and have the deployed version auto-update
- As a user, I want to be as ignorant of the internals as possible, but still be able to GSD.
- As a user, I want to be able to hand off code artifacts without having to prepare them.

152
docs/hacking.md Normal file
Просмотреть файл

@ -0,0 +1,152 @@
# Hacking on Prow
This document is for developers who are interested in working directly on the Prow codebase. In
this guide, we walk you through the process of setting up a development environment that is
suitable for hacking on Prow.
## Prerequisites
In order to successfully compile and test Prow binaries and build Docker images of Prowd, the
following are required:
- [git](https://git-scm.com/)
- [Go](https://golang.org/) 1.7 or later, with support for compiling to `linux/amd64`
- [glide](https://github.com/Masterminds/glide)
- [docker](https://www.docker.com/)
- a Kubernetes cluster. We recommend [minikube](https://github.com/kubernetes/minikube)
- [helm](https://github.com/kubernetes/helm)
In most cases, you should simply install according to the instructions. We'll cover the special
cases below.
### Configuring Go
If your local workstation does not support the `linux/amd64` target environment, you will have to
install Go from source with cross-compile support for that environment. This is because some of the
components are built on your local machine and then injected into a Docker container.
Homebrew users can just install with cross compiling support:
```
$ brew install go --with-cc-common
```
It is also straightforward to build Go from source:
```
$ sudo su
$ curl -sSL https://storage.googleapis.com/golang/go1.7.5.src.tar.gz | tar -C /usr/local -xz
$ cd /usr/local/go/src
$ # compile Go for our default platform first, then add cross-compile support
$ ./make.bash --no-clean
$ GOOS=linux GOARCH=amd64 ./make.bash --no-clean
```
Once you can compile to `linux/amd64`, you should be able to compile Prowd for preparation into a
Docker image.
## Fork the Repository
Once the prerequisites have been met, we can begin to work with Prow.
Begin at Github by forking Prow, then clone that fork locally. Since Prow is written in Go, the
best place to put it is under `$GOPATH/src/github.com/deis/`.
```
$ mkdir -p $GOPATH/src/github.com/deis
$ cd $GOPATH/src/github.com/deis
$ git clone git@github.com:<username>/prow.git
$ cd prow
```
If you are going to be issuing pull requests to the upstream repository from which you forked, we
suggest configuring git such that you can easily rebase your code to the upstream repository's
master branch. There are various strategies for doing this, but the
[most common](https://help.github.com/articles/fork-a-repo/) is to add an `upstream` remote:
```
$ git remote add upstream https://github.com/deis/prow.git
```
## Make Your Changes
With your development environment set up and the code you wish to work on forked and cloned, you
can begin making your changes.
To do that, run the following to get your environment ready:
```
$ make bootstrap # runs `glide install`
$ make build # compiles `prow` and `prowd` inside bin/
```
## Test Your Changes
Prow includes a comprehensive suite of tests. To run these tests, run `make test`.
If you wish to run a more extensive set of tests, run `make test-e2e`.
## Deploying Your Changes
Although writing and executing tests are critical to ensuring code quality, you will likely want to
deploy changes to a live environment, whether to make use of those changes or to test them further.
To facilitate deploying Prowd images containing your changes to your Kubernetes cluster, you will
need to make use of a Docker Registry. This is a location to where you can push your custom-built
images and from where your Kubernetes cluster can retrieve those same images.
In most cases, a local registry will not be accessible to your Kubernetes nodes. A public registry
such as [DockerHub][dh] or [quay.io][quay] will suffice.
To use DockerHub for this purpose, for instance:
```
$ export DOCKER_REGISTRY="docker.io"
$ export IMAGE_PREFIX=<your DockerHub username>
```
To use quay.io:
```
$ export DEIS_REGISTRY="quay.io"
$ export IMAGE_PREFIX=<your quay.io username>
```
After your Docker Registry is set up, you can deploy your images using
```
$ make docker-build docker-push
```
To install Prowd, edit `chart/values.yaml` and change the `registry` and `name` values to the image
you just deployed:
```
$ $EDITOR charts/values.yaml
```
Then, install the chart:
```
$ helm install ./chart --name prow --namespace prow
```
You should see a new Helm release available in `helm list`.
### Re-deploying Your Changes
Because Prow deploys Kubernetes applications and Prow is a Kubernetes application itself, you can
use Prow to deploy Prow. How neat is that?!
To build your changes and upload it to Prowd, run
```
$ make build docker-binary
$ prow up
--> Building Dockerfile
--> Pushing 127.0.0.1:5000/prow:6f3b53003dcbf43821aea43208fc51455674d00e
--> Deploying to Kubernetes
--> code:DEPLOYED
```
You should see a new release of Prow available and deployed with `helm list`.