Граф коммитов

249 Коммитов

Автор SHA1 Сообщение Дата
Ahmet Alp Balkan f247f6e634
Add ISSUE_TEMPLATE for github issues 2016-07-27 15:25:27 -07:00
Ahmet Alp Balkan 887f16b298
wait-for-enable: add initial 1s delay
Inside the container when we invoke `nohup [...] &` it immediately
returns and `wait-for-enable` starts checking `ps aux` for extension
binary. However since travis environment is not fast, `wait-for-enable`
exited immediately once before the process started.

Adding a 1s delay while polling for exit of extension process should
solve this problem.
2016-07-27 15:17:43 -07:00
Ahmet Alp Balkan d560e6d43d Merge pull request #40 from ahmetalpbalkan/travis-bats
travis.yml: add integration tests
2016-07-27 15:17:05 -07:00
Ahmet Alp Balkan 792d491cbf
Makefile: change [[ to [
apparently travis uses /bin/sh to run tests; not /bin/bash
so [[ is missing. In this case we can safely change to [.
2016-07-27 15:01:43 -07:00
Ahmet Alp Balkan 9c26ac3e42 .travis.yml: add azure cli 2016-07-27 14:50:07 -07:00
Ahmet Alp Balkan e116b659da .travis.yml: add binary build step 2016-07-27 14:40:26 -07:00
Ahmet Alp Balkan 4eacb598ba travis.yml: add integration tests
- change sudo:true so that we can get a Docker engine
    - this will slow down the time to get a test box (a few mins)
      but it should be fine
- add bats in 'install'
- print docker info prior to testing
- execute bats
2016-07-27 14:36:16 -07:00
Ahmet Alp Balkan 69b60e35d0 Merge pull request #44 from ahmetalpbalkan/integ-shimfix
integration-test: fix for daemonized 'enable' cmd
2016-07-27 14:35:29 -07:00
Ahmet Alp Balkan a00233e966 Merge pull request #39 from ahmetalpbalkan/shim
Shim script for extension handler
2016-07-27 14:35:24 -07:00
Ahmet Alp Balkan 3e2b17215f Merge pull request #43 from ahmetalpbalkan/integ-storage-tests
integration-test: add storage tests for blobs as fileUris
2016-07-27 14:04:02 -07:00
Ahmet Alp Balkan cccd68f1e5
Address CR comments
- use "md5 -q {file}" instead of "cat {file} | md5 -q"
2016-07-27 13:58:41 -07:00
Ahmet Alp Balkan 7c8620f673
integration-test: fix for daemonized 'enable' cmd
Since #39 is adding a shim that daemonizes 'enable' command and
exits and shim is the entrypoint of the container command, Docker
will kill the container when that entrypoint exits (will not wait
background process to finish). This adds a `wait-for-enable` command
that we use in integration tests. It runs inside the test container
and loops indefinitely until the background process exits.
2016-07-27 13:51:43 -07:00
Ahmet Alp Balkan 7f028f2698
integration-test: add storage tests
When AZURE_STORAGE_ACCOUNT and AZURE_STORAGE_ACCESS_KEY passed,
we now have a scenario that runs by uploading two 512k files to
Azure Blob Storage and configures extension to download it.

We verify by checking test container fs diff to see files are
downloaded and we verify checksum on one of the files.
2016-07-27 13:44:00 -07:00
Ahmet Alp Balkan 7659a941fd Merge pull request #41 from ahmetalpbalkan/vendor-dockerext-4
Vendor pkg/vmextension/status again
2016-07-27 12:30:09 -07:00
Ahmet Alp Balkan 2a19e60698 Merge pull request #42 from ahmetalpbalkan/plumb-logger
download: plumb logger for logging retries
2016-07-26 15:51:05 -07:00
Ahmet Alp Balkan b1742f20e4
download: plumb logger for logging retries
Plumbing logger reference into the download package so that
users can get more visibility into the download failures
during the retries by looking at the logs.
2016-07-26 15:35:55 -07:00
Ahmet Alp Balkan 307e4287a7
Vendor pkg/vmextension/status again
Addresses status file write atomicity issue.
Closes #37.
2016-07-26 15:16:18 -07:00
Ahmet Alp Balkan c5044e0ced Merge pull request #32 from ahmetalpbalkan/integration-tests
integration-test: add test scenarios
2016-07-26 15:14:15 -07:00
Ahmet Alp Balkan 29306d1abb Merge pull request #24 from ahmetalpbalkan/integration-test-env
Containerized integration test environment
2016-07-26 15:14:08 -07:00
Ahmet Alp Balkan 07ce77953d
Shim script for extension handler
Wrote a bash shim that forwards logs of the handler process
reliably and daemonizes the 'enable' command via `nohup .. &`.

Adopted from https://github.com/Azure/azure-docker-extension/blob/master/scripts/run-in-background.sh
and extended to be used for all commands (not just 'enable')

Fixes #35.
2016-07-26 14:31:40 -07:00
Ahmet Alp Balkan 9b7043a120
Remove placeholder dirs, Dockerfile update
- removed placeholder dirs (create them in dockerfile)
- change handlermanifest to use the shim
2016-07-26 14:28:04 -07:00
Ahmet Alp Balkan 7396c150bd
integration-test: add test scenarios
The integation tests use `bats` to run the scenarios as bash scripts.

To test the extension handler functionality, we simply:

- build a Docker image using test.Dockerfile
    - copy some files from ` make it look like a `/var/lib/waagent` directory
    - copy extension binary into the container
- remove the `test` container if it exists
- create a Docker container (name: `test`) from image
    - specify which handler subcommand will be invoked (e.g. `fake-waagent
      install`)
- push .settings file and certificate/private key (.crt, .prv)
    - do other things on the container that we need to craft the environment
- start the container
- collect the output from the command execution
- validate using the following:
    - check status code
    - validate output of the command
    - `docker diff test` to validate file changes in the container
    - copy files out of container and validate their contents

Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
2016-07-26 13:44:59 -07:00
Ahmet Alp Balkan 10c9068bd9 Merge pull request #31 from ahmetalpbalkan/main-pr
Implement main binary
2016-07-26 12:24:37 -07:00
Ahmet Alp Balkan 56d6328adb
cmds: fix checkAndSaveSeqNum logic based on #33 2016-07-26 12:21:03 -07:00
Ahmet Alp Balkan 3cf0367488 cmds: split enable() into several methods, add tests
extracted logic in enable() to smaller and testable
methods.
2016-07-26 12:16:08 -07:00
Ahmet Alp Balkan ad61709398 main: move status methods to status*.go 2016-07-26 12:16:08 -07:00
Ahmet Alp Balkan f7588d0938 Implement main binary
This change actually gives us a functioning extension handler.

- We store data at /var/lib/azure/custom-script (created upon 'install')
    - This is different than what old extension does.
- This data directory removed upon 'uninstall'.
- Opereations 'update' and 'disable' is no-op (same as before)
- Processed sequence number file (formerly known as 'mrseq') now resides
    at this data directory. This way agent does not need to move the
    'mrseq' file around.
- Operation 'enable' is where the magic happens and it is clear to follow.
    - parses handler settings
    - creates download/output directory
    - download the files
    - runs the command
- There is no 'daemon' subcommand yet as I am planning to invoke 'enable'
  via a shim that does double-fork and log forwarding to a custom log path
  (e.g. /var/log/azure/custom-script.log)
- Moved the check for 'commandToExecute' not empty from json-schema validation
  to logical validation.

Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
2016-07-26 12:16:08 -07:00
Ahmet Alp Balkan f0f35e6d7c Merge pull request #33 from ahmetalpbalkan/seqnum-again
seqnum: convert <= logic to <
2016-07-26 12:15:57 -07:00
Ahmet Alp Balkan 0081a7d679
seqnum: convert <= logic to <
Changing the seqnum helper method to give the following truth table:

(stored,given) => outcome
(none, 0) => true
(0, 0) => false
(0, 1) => true
(1, 0) => false
(1, 1) => false
(1, 2) => true
2016-07-26 12:04:19 -07:00
Ahmet Alp Balkan 194d278d2c Merge pull request #30 from ahmetalpbalkan/vendor-dockerext-3
Vendor: github.com/Azure/azure-docker-extension
2016-07-25 22:27:25 -07:00
Ahmet Alp Balkan 4648e3e7e6
Vendor: github.com/Azure/azure-docker-extension
Incorporate latest changes (PRs 96, 97, 98, 99) from azure-docker-extension repo
where we grab extension handler authoring utility pkgs for Go.
2016-07-25 22:23:17 -07:00
Ahmet Alp Balkan e1a3653389 Addressed CR comments
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
2016-07-25 14:26:53 -07:00
Ahmet Alp Balkan 16dc05ac18 Containerized integration test environment
Built a containerized environment that looks like an actual VM
and wrote a fake agent which you can invoke the extension binary
through.

Refer to the integration-test/env/README.md.

This helps us test:
- agent binary can handle the directory structure presented by waagent
- agent binary can handle the HandlerEnvironment, log dirs etc
- agent binary decrypts/parses the settings correctly
- agent binary can write status to the right place

By containerizing there will be no residue left on the dev machine
while testing the changes.

Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
2016-07-25 14:26:52 -07:00
Ahmet Alp Balkan d61c6c59cf Merge pull request #27 from ahmetalpbalkan/seqnum-updates
seqnum: update IsSmallerThan→IsSmallerOrEqualThan
2016-07-25 14:25:08 -07:00
Ahmet Alp Balkan 2ca1d1d7fa
Address CR feedback
Fixed a nit in godoc.
2016-07-25 14:24:44 -07:00
Ahmet Alp Balkan 57ead79b61 Merge pull request #26 from ahmetalpbalkan/vendor-logpkgs
Vendor logging dependencies
2016-07-25 14:23:31 -07:00
Ahmet Alp Balkan dc79b7d1a6 Merge pull request #29 from ahmetalpbalkan/Makefile
Makefile and binary versioning
2016-07-25 14:22:59 -07:00
Ahmet Alp Balkan 8274ac9b83 Merge pull request #28 from ahmetalpbalkan/gitignore-minor
.gitignore minor update
2016-07-25 14:22:47 -07:00
Ahmet Alp Balkan 806bd36ddc
Makefile and binary versioning
Added a Makefile with binary build step and used Go linker
options to parse the version number, git commit, build date
and the state of build tree as version string to the built
binary.

Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
2016-07-22 09:53:20 -07:00
Ahmet Alp Balkan 15e527a000 .gitignore minor update
This way we only ignore "bin" at the root level and not in
nested directories.

Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
2016-07-21 23:09:59 -07:00
Ahmet Alp Balkan 1a7351c2c3
Vendor logging dependencies
- github.com/go-kit/kit/log
    - github.com/go-logfmt/logfmt
        - github.com/kr/logfmt
    - github.com/go-stack/stack

Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
2016-07-21 22:42:25 -07:00
Ahmet Alp Balkan 397c3e8d0a seqnum: update IsSmallerThan→IsSmallerOrEqualThan
This package was designed to be used to make sure we do not
execute same or lower sequence numbers more than once. Changing
the method accordingly to be more useful in that regard.

Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
2016-07-21 18:06:37 -07:00
Ahmet Alp Balkan 51c8f588ff Merge pull request #23 from ahmetalpbalkan/main-0
main.go: starting to write main.go
2016-07-20 14:59:04 -07:00
Ahmet Alp Balkan 0dc361873b
addressed CR comments 2016-07-20 14:58:38 -07:00
Ahmet Alp Balkan 0d31797f8d Merge pull request #22 from ahmetalpbalkan/download-cov
files_test.go: increase coverage
2016-07-20 14:58:11 -07:00
Ahmet Alp Balkan 3ac0eedbd9
main.go: starting to write main.go 2016-07-20 14:28:11 -07:00
Ahmet Alp Balkan 9329be9dad Merge pull request #20 from ahmetalpbalkan/nit-0
nit: better errors for chmod comparisons in tests
2016-07-20 13:48:28 -07:00
Ahmet Alp Balkan 074fcf2e9e
files_test.go: increase coverage 2016-07-20 13:48:05 -07:00
Ahmet Alp Balkan 99f7996d0d Merge pull request #18 from ahmetalpbalkan/seqnum
seqnum: sequence number tracking via file
2016-07-20 13:44:09 -07:00
Ahmet Alp Balkan a8110c83e5 Merge pull request #19 from ahmetalpbalkan/retrycount
download/retry: reduce retry duration
2016-07-20 13:41:44 -07:00