mozilla
/
MozDef
зеркало из https://github.com/mozilla/MozDef.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность
5 773 коммитов 4 Ветки 53 Теги 68 MiB
Граф коммитов

338 Коммитов

Автор SHA1 Сообщение Дата
Andrew Krug b22b0dbc74
fix accidental addition of template mappings 2019-05-25 18:44:31 -07:00
Andrew Krug 691ee439b5
fix es subnet mappings 2019-05-25 18:42:40 -07:00
Andrew Krug 364cfcdb2c
fixup subnet list passed to es 2019-05-24 15:40:21 -07:00
Gene Wood ad1322a0f1
Switch SQS worker from SNS+SQS to just SQS 2019-05-24 15:34:21 -07:00
Andrew Krug 6b04f21ab0
remove username 2019-05-24 15:17:26 -07:00
Andrew Krug 6a6a1c98d6
minor syntax fixes 2019-05-24 15:08:08 -07:00
Andrew Krug 9ee42eaf4a
fix template generation 2019-05-24 13:47:50 -07:00
Gene Wood 8a9f64ea5c
Error on deploy if OIDC secret missing 2019-05-24 10:05:56 -07:00
Gene Wood 679d160d81
Limit VPC Flow logs to accepted SSH connections 2019-05-24 09:36:09 -07:00
Gene Wood 310fb8c4ee
Correctly parse flow log message as a space delimited string 
Cast int field values as ints
Don't send empty SQS batches
 2019-05-23 21:39:11 -07:00
Gene Wood 4e79fc91e0
Initial capture of the CodeBuild configuration and role in CloudFormation 
This likely still requires the one time binding action to link the CodeBuild project with the GitHub project and webhook. But this may get us a large part of the way there before that manual step.
 2019-05-23 20:32:02 -07:00
Gene Wood 9fd89765ca
Fix malformed LogGroup ARN 2019-05-23 17:54:43 -07:00
Andrew J Krug d983262b44
Merge branch 'reinforce2019' into eis_996-2 2019-05-23 16:48:31 -07:00
Andrew Krug e14f3441d0
fix tabs 2019-05-23 16:43:16 -07:00
Andrew Krug 84ce9ad987
add support for injecting map to reinforce template 2019-05-23 16:32:46 -07:00
Andrew Krug 4c6b830412
initial attempt to break apart reinforce features 2019-05-23 16:32:46 -07:00
Gene Wood 1bbce41b4b
Add VPC flow log transformer 
Also change to using the SQS queue created by the parent instead of creating our own
 2019-05-23 16:32:14 -07:00
Gene Wood 4d4d9b8dc4
Emit SQS URL 2019-05-23 16:32:14 -07:00
Gene Wood 65f125c83a
Initial VPC Flow log template 2019-05-23 16:32:14 -07:00
Andrew Krug 4774ca3e44
add cloudformation to create alert writers environment 2019-05-23 16:32:14 -07:00
Andrew Krug 17345d5492
make ci build on reinforce branch as well temporarily 2019-05-23 16:32:13 -07:00
Andrew Krug e169205b22
update ignores 2019-05-23 16:32:13 -07:00
Andrew Krug 3b59924f0b
update scaffold for alert write env 2019-05-23 16:31:23 -07:00
Andrew Krug d656ee6e2b
stub out layers dir 2019-05-23 16:27:56 -07:00
Andrew J Krug 05d96ac50d
Merge pull request #1275 from mozilla/eis_975 
Package the alert writers environment as cloudformation
 2019-05-23 15:50:32 -07:00
Gene Wood 151c7fc09f
Add VPC flow log transformer 
Also change to using the SQS queue created by the parent instead of creating our own
 2019-05-21 10:41:53 -07:00
Gene Wood bd67cd08f2
Emit SQS URL 2019-05-21 10:39:53 -07:00
Gene Wood 9207a18dca
Initial VPC Flow log template 2019-05-20 17:02:16 -07:00
Andrew Krug f891007922
add cloudformation to create alert writers environment 2019-05-20 11:15:57 -07:00
Andrew Krug 3c52992250
make ci build on reinforce branch as well temporarily 2019-05-20 10:17:52 -07:00
Andrew Krug 5433129ef8
update ignores 2019-05-19 11:05:14 -07:00
Andrew Krug 16f4c5a132
update scaffold for alert write env 2019-05-19 11:04:07 -07:00
Andrew Krug 414b168d1f
remove files that should be gitignored 2019-05-19 11:03:51 -07:00
Andrew Krug d822a3cce0
update gitignore 2019-05-19 11:02:58 -07:00
Andrew Krug 14ead82961
stub out layers dir 2019-05-19 10:05:07 -07:00
Gene Wood 5f42a2435e
Remove reference to OIDCDiscoveryURL 2019-05-19 10:05:07 -07:00
Gene Wood 5417830513
Remove reference to OIDCDiscoveryURL 2019-05-17 13:17:11 -07:00
Brandon Myers 4dc9a02cbf
Change kibana url to http for cloudy mozdef temporarily 2019-05-15 12:02:36 -05:00
Brandon Myers 73afb84a27
Readd OPTIONS_METEOR_KIBANAURL 2019-05-14 15:08:03 -05:00
Brandon Myers 48ec5f2657
Merge remote-tracking branch 'origin/master' into testing_kibanaurl 2019-05-14 15:04:20 -05:00
Andrew J Krug 4d554b28ea
Merge branch 'master' into alert_sqs_support 2019-05-14 11:56:54 -07:00
Brandon Myers 5f9ccef547
Remove full kibanaurl from rest api 2019-05-13 18:01:35 -05:00
Brandon Myers 2c7181ccc8
Fixup kibanaurl for cloudy mozdef setup 2019-05-13 16:43:01 -05:00
Brandon Myers 9dda92daed
Disable certain web ui features for cloudy mozdef 2019-05-13 13:24:45 -05:00
Brandon Myers c15e545a52
Add watchlist and enable cloudtrail alerts in cloudy mozdef 2019-05-13 13:21:25 -05:00
Andrew Krug ca340dcb57
a few fixes for region transport options 2019-05-12 10:19:22 -07:00
Andrew Krug 6f6fc8b6b6
fixup default queue, fix IAM policy, fix entrypoint for resty 2019-05-10 12:09:33 -07:00
Andrew Krug 6d05bc6650
multiple fixes 2019-05-10 08:55:02 -07:00
Andrew Krug 9e945ea870
add additional sqs queue and opts to alert taskexchange to no longer be rabbitmq 2019-05-08 11:24:30 -07:00
Andrew Krug c41b729bfd
WIP on adding lambda layer 2019-05-08 08:43:37 -07:00
Andrew Krug d8f4b270cf
add log upper bound to containers 2019-05-08 08:43:37 -07:00
Andrew Krug cf7764f87f
fix outputs of parent stack to return user DNS name 2019-05-08 08:43:37 -07:00
Andrew Krug b15981fcbf
add cloudformation conditions to support basic_auth, basic auth with ssl, and oidc with ssl 2019-05-08 08:43:37 -07:00
Andrew Krug 18004717cd
multiple fixes for basic auth and non-ssl 2019-05-08 08:43:36 -07:00
Andrew Krug ea157c8f2f
add new container to tag and push 2019-05-08 08:43:36 -07:00
Andrew Krug 1261232367
begin removing OIDC as a requirement. Make SSL optional. 2019-05-08 08:43:36 -07:00
Andrew Krug ff97c84fb8
modify rabbit user add to take password from file and allow ingress to es on port 80 to support cron jobs. 2019-05-01 15:07:00 -07:00
Andrew Krug fafc00ac6e
fix es to bind to sec group created for it 2019-05-01 10:58:57 -07:00
Gene Wood 7519a94186
Use systemd to launch docker containers 
This changes from launching docker containers with docker compose
via a make target to a systemd service.

This should ensure graceful shutdown of containers upon instance shutdown.
Previously containers were not shutting down gracefully causing a leftover
lock file for mongodb in the EFS filesystem
 2019-04-19 12:58:31 +01:00
Gene Wood af1e08976e
Avoid installing packer in CodeBuild unless we're going to build 
Also add some additional logging
 2019-04-12 15:57:56 -07:00
Gene Wood 7fe0541dcd
Move to model where we build once in CodeBuild and fetch images in packer 2019-04-09 15:30:56 -07:00
Gene Wood e7cf6e4695
Fix incorrect docker tag syntax and add logging 2019-04-08 13:21:39 -07:00
Gene Wood e228a2c87e
Fix regex in deploy to match multi digit semver versions 2019-04-01 16:43:37 -07:00
Gene Wood 78473a4fac
Clarify the log messages to reflect the fact that we don't run the tests in CodeBuild 2019-03-29 22:34:33 -07:00
Gene Wood 8c79459308
Fix deploy so it triggers on tags 
The tag regex was missing the "v" prefix for the tag name
(e.g. "v1.2.3")
Also removing the march_swarm case as it's no longer needed
 2019-03-29 22:34:06 -07:00
Gene Wood 43b29c4876
Add set -e to fail if any step fails 
This will prevent the build from continuing if one of the steps fails,
ensuring that the output from CodeBuild is accurate about success or
failure
 2019-03-29 22:32:23 -07:00
Gene Wood f20e70264f
Fix docker push and tag 
* Only tag an image as latest if it came from master branch
* Upload images to dockerhub tagged as latest in addition to those tagged as the branch
 2019-03-29 15:41:00 -07:00
Gene Wood 8c65a04605
Fix missing and grouped Makefile phony lines 2019-03-25 08:05:12 -07:00
Gene Wood 4bd78fee32
Fix missing env files for pull 2019-03-22 22:13:31 -07:00
Gene Wood 326bfd4b2c
Fix error when pulling docker images 
This addresses error `Couldn't find env file: /opt/mozdef/docker/compose/cloudy_mozdef.env`
 2019-03-22 18:41:37 -07:00
Gene Wood 1f6520249a
Output the git hash that's being built in the logs to help in troubleshooting 2019-03-22 18:40:36 -07:00
Gene Wood 380f87bd8d
Remove wget progress output during CodeBuild build 2019-03-22 16:53:42 -07:00
Gene Wood eee914e15a
Move hub-login to before docker-push-tagged 
This is for clarify since the second target depends on the first and hub-login
isn't needed before this point
 2019-03-22 14:35:05 -07:00
Gene Wood 0c01fd6eac
Remove CodeBuild call to set-version-and-fetch-docker-container make target 
This is because
* We've already built the containers in CodeBuild and don't need to do it again
* We don't need to tag the docker images because `docker-push-tagged` will trigger that
* We don't need to modify docker/compose/docker-compose-cloudy-mozdef.yml because we don't use it in CodeBuild
 2019-03-22 14:30:43 -07:00
Gene Wood 9aa3d8246d
Add docker-compose pull to packer.json to fetch locally packages other than those we build 
See #1163
 2019-03-22 14:28:41 -07:00
Gene Wood 4d2674540a
Fix bug where deploy is publishing templates from branches and tags 
overwriting the /master branch directory in S3 instead of writing
to their respective branch and tag specific directories
 2019-03-22 11:45:14 -07:00
Gene Wood 3e5ef4e0ab
Add log output and comments to publish_versioned_templates 2019-03-22 11:44:08 -07:00
Gene Wood 43eae2e9b3
Fix update-dev-stack, removing S3TemplateLocation 
Similar to fc72de6554 in #1154
 2019-03-22 11:43:17 -07:00
Gene Wood 84c3c7d8b9
Revert 64fcfdc3a1 
We found that the increased instance size didn't improve build time significantly
 2019-03-21 13:16:11 -07:00
Gene Wood aa55ca3c06
Revert #1079 as the ArnEquals Conditions is how to constrain access to the queue, not with Principal 
https://docs.aws.amazon.com/sns/latest/dg/sns-sqs-as-subscriber.html#SendMessageToSQS.sqs.permissions
 2019-03-21 10:41:19 -07:00
Gene Wood 64fcfdc3a1
Increase packer builder instance size to speed up builds 2019-03-21 09:02:13 -07:00
Gene Wood fc72de6554
Fix create-dev-stack, removing S3TemplateLocation 2019-03-21 09:00:23 -07:00
Gene Wood c82d6b5c00
Increase disk size as MozDef is 11GB (larger than 8GB) 2019-03-21 08:57:52 -07:00
Gene Wood ddbc827877
Fix Rabbit MQ password variable escaping 2019-03-21 08:57:30 -07:00
Gene Wood 3ba1aedf5c
Update example parameters to match new parent parameter list 
Related to #1143
 2019-03-20 21:05:38 -07:00
Gene Wood 4e196b515a
Fix missing path to rabbitmq env file 
Fix bug in #1139
 2019-03-20 21:03:57 -07:00
Gene Wood 20a85721f0
Fix incorrect env file name 
Fix bug in #971
 2019-03-20 21:02:54 -07:00
Gene Wood 2a912e74b9
Fix variable collision between AWS CloudFormation Fn::Sub and bash 
Fixes bug in #1139
 2019-03-20 20:18:08 -07:00
Gene Wood 3038499218
Fix invalid policy principle structure from #1079 2019-03-20 19:59:49 -07:00
Gene Wood b8a04393d0
Fix invalid ARN syntax from 2869df7ddd 2019-03-20 19:41:27 -07:00
Gene Wood 56e83218b6
Fix malformed YAML from 2869df7ddd 2019-03-20 19:25:51 -07:00
Gene Wood 4b68bb85ba
Fix bad make BRANCH value 2019-03-20 17:31:47 -07:00
Gene Wood dc2585fb63
Merge pull request #1143 from gene1wood/add-version-to-stacks 
Add support for versioned templates
 2019-03-20 16:30:26 -07:00
Andrew Krug a59638650b
fix make target 2019-03-20 16:21:55 -07:00
Gene Wood eee06b82fa
Add support for versioned templates 2019-03-20 16:20:37 -07:00
Andrew Krug 10dcd163b3
simplify deploy script and reduce code dup! 2019-03-20 15:43:44 -07:00
Andrew Krug ca7864c7d6
add additional comments to cloud-init 2019-03-20 15:32:10 -07:00
Andrew Krug 78c567226c
add params to cloudinit to set rabbitmq password. 2019-03-20 14:40:55 -07:00
Gene Wood 9e60297a58
Add comment explaining need for RegionMap 2019-03-20 11:52:17 -07:00
Gene Wood f760992379
Change AMI parameter to static region map of AMI IDs 
This is a requirement for admission to the AWS Marketplace
 2019-03-20 11:43:40 -07:00