Граф коммитов

126 Коммитов

Автор SHA1 Сообщение Дата
Paul McLanahan f3c79c3531 Switch deployment to use dockerized envcat 2017-05-12 10:35:24 -04:00
Paul McLanahan 914d8007f4 Restructure jenkins deployment. Demos build in main pipeline.
* Fix demo deploy and update docs.
* Make it work with deis1 and deis2
* Enforce max 63 char app names
* Fix SSL redirect for Deis v2 clusters. Allow settings per cluster.
* Improve env file handling and combination for demo deploys
* Use new envcat utility
* Deploy our dev, stage, and prod apps to virginia and tokyo clusters
2017-05-10 17:01:06 -04:00
Paul McLanahan 01088d4549 Switch to UglifyJS and CleanCSS compressors 2017-04-03 15:55:58 -04:00
Paul McLanahan 7bab6434d2 Downgrade Requests to fix LinkChecker
Turns out LinkChecker checks for a good version of python-requests
in a dumb way, so it sees version "2.13.0" as being lower than
"2.2.0". Also the project seems to have been abandoned, so there
isn't a newer release, though it seems this issue has been fixed
in the master branch.

Easiest fix is to just downgrade requests to the latest version
that is < 2.10.0, which is what I've done.
2017-03-15 10:10:56 -04:00
Paul McLanahan 252d599454 Add removal of containers to docker run commands
Also switch to using short git hash from build number for container names.
2017-03-02 13:35:17 -05:00
Paul McLanahan b5e40444e4 Add branch name to deploy image
This means that we can push dev l10n to www-dev and prod
l10n to stage and prod. Currently they all get prod l10n
at deployment then dev updates when cron runs.
2017-03-02 13:20:44 -05:00
Paul McLanahan 4a16758fd7 Add bash utility for setting git environment variables 2017-03-02 13:20:44 -05:00
Paul McLanahan 74c7c63c5d Add more options to jenkins.yml for deployments
* Add ability to toggle smoke tests
* Add ability to toggle deployment
* Add ability to toggle integration tests
* Add ability to toggle pushing docker images to public registry
2017-03-02 13:20:44 -05:00
alexgibson 3647ea0d23 [fix bug 1342036] Remove Firefox OS feed links 2017-02-28 09:43:56 -05:00
Paul McLanahan 8364aa313f Move docker scripts, envfiles, and groovy files 2017-02-22 16:41:19 -05:00
Paul McLanahan e8d6ca674b Cleanup bin dir and move run scripts to bin 2017-02-22 15:53:31 -05:00
Paul McLanahan 7a4ef8a0bd Move integration test properties into the run script
Removes the need for separate properties files and makes bulk
editing of properties easier.
2017-02-22 15:36:26 -05:00
Paul McLanahan a2cc3e9b2d Change how we do docker
* More simple and streamlined base image build file
* Use a builder image to decrease the size and security footprint of running containers
* Make demos more prod-like
* Add l10n for prod builds via docker commit
* Specifically include only those files required for the running site
* Tests in circleci now use the same docker build and test scripts
* Add lock for docker image building per git hash
* Add environment variable support to jenkins.yml
* Add headless and firefox integration tests to prod deploys
* Run integration smoke tests before deployment
2017-02-21 16:54:00 -05:00
Paul McLanahan 63a9dbedf3 Use demo app alias URLs in IRC notifications
e.g. www-demo1.allizom.org
2017-02-07 22:32:48 -05:00
Paul McLanahan 488454f5a4 Fix jenkins demo app name function 2017-02-07 13:45:03 -05:00
Paul McLanahan 120d514fb2 Use the full app URL with suffix for integration tests 2017-02-07 13:10:36 -05:00
Paul McLanahan 9c87525d55 Only push requested apps to private registry 2017-02-07 11:51:09 -05:00
Josh Mize 2b50eb7005 Fix DEIS_APPS in pushPrivateReg 2017-02-07 09:31:54 -06:00
Paul McLanahan 160d6e55d8 Build and deploy via Jenkinsfile
* Add jenkins.yml for config and other groovy files for functions.
* Make the ircNotification function more flexible
* Add locking for deploy/test steps
* Allow a branch config to require a commit be tagged for deployment
* Deploy all apps in a region before moving to next region
* Update deployment pipeline docs
* Add Jenkinsfile milestones
* Optimize steps for BlueOcean
2017-02-06 14:46:52 -05:00
Paul McLanahan 509b635e24 Move jenkins IRC notification to bash script
Add color and bold support to IRC messages
2017-01-04 15:06:24 -05:00
Paul McLanahan 6c59f305b8 Deploy demos with Jenkins
* Disable circle demo deploy
* Add IRC notification to Jenkinsfile deployments
* Fix the deployment docs for new pipeline
2017-01-03 14:46:18 -05:00
Paul McLanahan ab985fb81e Switch to using a Jenkins Multibranch Pipeline for deploy
The new Jenkins Multibranch Pipeline will now start the bedrock
build jobs instead of having CircleCI do it. This wil mean
having to push to a "prod" branch to trigger a prod deployment,
but it should be much better and faster than waiting for Circle.
2016-12-23 14:39:18 -05:00
Paul McLanahan 841f1a5361 Fix bug 1315082: Store git repo state in the db
This reverts commit d962ef4b77
which itself reverted an earlier attempt at this but which
caused some tests to fail.
2016-12-16 15:07:08 -05:00
Josh Mize d962ef4b77 Revert "Fix bug 1315082: Reset git repos on failed db updates" 2016-12-15 16:32:24 -05:00
Paul McLanahan 6c91d7b5ab Fix bug 1315082: Retry git repos on failed db updates
Store the last successfully updated git hash in the
database. As long as we only update said hash upon
successful completion of an import it should try again
next time. Also switch product-details and security-
advisory import commands to simply update all files
when a git update is detected.
2016-12-15 13:43:45 -05:00
Paul McLanahan 8e9726a446 Bug 1296646: Convert update_product_details_files command to use git repo
* Kept the data file validation just in case. Can be removed later.
* Add tests for update_product_details_files command
* Switch sync_all to use new product-details command
2016-10-13 09:58:03 -04:00
Josh Mize 7bd03a4a82 Validate squashed tag exists before using 2016-10-10 12:39:19 -05:00
Paul McLanahan 103cd86a0d Run all python tests in docker in circleci 2016-10-05 11:16:55 -04:00
Josh Mize ee81fd9ca7 Fix ownership of locale dir 2016-10-03 15:39:10 -05:00
Paul McLanahan eb1a5b6369 Fix supervisor running in dev
This was done before using a different process type in
Procfile, but only "web" and "cmd" types are served as
HTTP traffic, so that wouldn't work. This new direction
should work for any instance for which we'd like to use
supervisord.

Also reunified the cron.py file and used arguments to
determine which jobs are scheduled.
2016-09-30 15:16:21 -04:00
Paul McLanahan 2888eee4de Give ability to run supervisor for dev
This will allow us to run supervisord in www-dev
that will run the site and a process to keep l10n
updated. It allows us to run this without also running
the other cron tasks that update the DB in every container,
since we use a separate clock container for that. Demo instances
will also run the DB update process via supervisor.
2016-09-29 13:11:50 -04:00
Josh Mize 3ac5acbb0d Update docker-squash command in build_image.sh
Switch to https://github.com/goldmann/docker-squash/
2016-09-27 10:24:18 -05:00
Josh Mize cb86a67042 Remove deprecated flag from docker tag commands 2016-09-27 09:34:59 -05:00
Paul McLanahan 9b493574ee Fix bug 1300090: Exempt css files from softlinkstatic.py
The content of the hashed CSS files is different from the
non-hashed versions. In the former the references to other
files (e.g. images) have been changed to their hashed versions.
We want to keep these versions since they are served with far-future
cache headers and thus reduce our server loads.
2016-09-14 17:15:06 -04:00
Josh Mize d8a08d2d81 Merge pull request #4271 from pmac/upgrade-node-libs-again
Bug 1276208: Upgrade nodejs in dockerfiles
2016-09-02 00:37:49 -05:00
Josh Mize 6239e13c6a Stop using ghprbActualCommit 2016-09-01 10:03:26 -05:00
Paul McLanahan 44128d6631 Bug 1276208: Upgrade nodejs in dockerfiles
- Change the base image to quay.io/mozmar/base
- Upgrade prod node dependencies
- Switch to yuglify for js and css minification

(cherry picked from commit 0ab71c8ddf)
2016-09-01 09:03:57 -04:00
Paul McLanahan f73755d092 Upgrade prod-details lib. Fix issue with database name. 2016-08-18 15:45:19 -04:00
Paul McLanahan 684111c9a2 Remove unused run-dev and run-common scripts 2016-08-17 09:33:31 -04:00
Paul McLanahan 8c520c4b9f Fix 1295691: Bedrock process starts without database
* Alter config to start wsgi process even without database server
* Change some templates to avoid relying on db data
* No longer run migrations on container start
2016-08-16 21:48:15 -04:00
Josh Mize 996fd7ae21 Revert "Bug 1276208: Upgrade nodejs in dockerfiles and node dependencies" 2016-08-01 11:19:34 -05:00
Josh Mize f0621463ed Merge pull request #4221 from pmac/upgrade-node-libs-node6
Bug 1276208: Upgrade nodejs in dockerfiles and node dependencies
2016-08-01 10:39:18 -05:00
Josh Mize e30223952e Remove .git from locale/.dockerignore 2016-07-25 13:45:31 -05:00
Josh Mize 240b73cdee pip install requirements/test.txt in bedrock_base 2016-07-22 17:19:29 -05:00
Paul McLanahan d9f67e9554 Switch from django-nose to py.test
Remove most of nose libs. Nose itself is still required
for nose.tools functions.
2016-07-22 16:05:25 -04:00
Paul McLanahan 0ab71c8ddf Bug 1276208: Upgrade nodejs in dockerfiles
Change the base image to quay.io/mozmar/ubuntu-slim-python
2016-07-15 13:47:46 -04:00
Josh Mize 50d5ca118b Merge pull request #4200 from alexgibson/exclude-headless-in-sanity-tests
Don't generate headless tests when running IE SauceLabs jobs
2016-07-07 11:51:38 -05:00
Josh Mize 4642be28b3 Add ENV LANG=C.UTF-8 to base Dockerfile 2016-07-06 13:00:12 -05:00
alexgibson 81304688ef Don't generate headless tests when running IE SauceLabs jobs 2016-07-01 12:20:55 +01:00
Giorgos Logiotatidis 07a67e0a3d Remove running selenium containers after local tests only. 2016-05-16 16:02:11 +03:00