mozilla
/
MozDef
зеркало из https://github.com/mozilla/MozDef.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность
5 995 коммитов 4 Ветки 53 Теги 68 MiB
Граф коммитов

615 Коммитов

Автор SHA1 Сообщение Дата
Michal Purzynski bc64101e00 remove automated black formating 2019-10-03 13:55:42 -07:00
Michal Purzynski 8b108cc7d1 Ingestion layer for Stackdriver over PubSub 2019-10-03 13:35:53 -07:00
Phrozyn c89604e616
adding zoom_fixup.py to remove uneccessary fields 2019-10-02 15:10:41 -05:00
Michal Purzynski 75f63afaac cleanups 2019-09-30 14:00:44 -07:00
Michal Purzynski 05fe8c4880 Fixups for ES exception handling plus error messages when we drop events 2019-09-25 12:50:48 -07:00
Brandon Myers 15b48cd48e
Add describehostrequest to cloudtrail mq plugin 2019-09-24 14:18:52 -05:00
Brandon Myers e1b6f03246
Fixup uptycs cron script and add to lower keys plugin 2019-09-05 16:18:34 -05:00
Brandon Myers dbad0bb8af
Add customizable sleep option for mq workers 2019-08-26 13:06:27 -05:00
Brandon Myers 29b6f99581
Remove unused config option 2019-08-26 13:01:14 -05:00
Brandon Myers 89374b3fe6
Remove sys.path where possible from mq 2019-08-07 16:40:27 -05:00
Brandon Myers e79c290094
Remove sys.path from mq plugin tests 2019-08-07 16:32:56 -05:00
Jan Andre Ikenmeyer 51822417a3
Update MPL license to https 2019-08-02 01:41:37 +02:00
Brandon Myers 211ab423e7
Update regex strings 2019-07-31 18:01:26 -05:00
Brandon Myers cb723a114c
Merge pull request #1398 from mpurzynski/dhcpfixups 
Dhcpfixups
 2019-07-31 17:24:51 -05:00
Michal Purzynski 2c5f5fd736 MAC address is 6 bytes, not 8 2019-07-31 15:09:22 -07:00
Michal Purzynski d17e38da00 fixups for fixups part 2 2019-07-31 13:22:58 -07:00
Michal Purzynski 0e6fb423ba fixups for fixups 2019-07-31 13:17:46 -07:00
Phrozyn 64efeaf9cd
adding missing comma 2019-07-30 15:24:49 -05:00
Phrozyn 35037392dc
adding new requestparam callerreference 2019-07-30 15:22:08 -05:00
Michal Purzynski 02c35da3b8 EIS-879 2019-07-26 13:21:42 -07:00
Michal Purzynski 4ff8e2a599 EIS-879 2019-07-26 13:14:02 -07:00
Brandon Myers 7ce8e6147e
Merge pull request #1385 from mozilla/remove_unused_cloudtrailconfig 
Remove unused eventexchange options
 2019-07-25 10:29:11 -04:00
Phrozyn 3422c5dd0c
Adds additional fields to the cloudtrail plugin to prevent field conflicts 2019-07-22 17:10:42 -05:00
Phrozyn 1566a45378
Adds additional fields to the cloudtrail plugin to prevent field conflicts 2019-07-22 17:08:50 -05:00
Brandon Myers fa7875f5c4
Fix region parameter in connect_sqs (#1383) 
* Fix region parameter in connect_sqs

* Modify parameter names to get_aws_credentials

* Clean up connect_sqs function to call get_aws_credentials

* Cleanup workers to use connect_sqs call

* Fix local import in sqs file

* Fix parameter name in cloudtrail worker

* Assert AWS region of SQS queues

Since the two SQS queues are provisioned by CloudTrail in the same region as the CloudTrail
stack, let's assert to MozDef that the region for those queus is indeed the same region.

* Update region name parameter in cloudtrail worker
 2019-07-19 17:50:58 -04:00
Brandon Myers aa03a4da57
Remove unused eventexchange options 2019-07-19 13:47:12 -04:00
Brandon Myers ee13b19fa1
Allow config option to tune cloudtrail sleep time 2019-07-09 13:54:34 -05:00
Brandon Myers ac3b27977c
Fixup lower_keys plugin 2019-07-08 12:31:50 -05:00
Brandon Myers 721675cfbf
Update fluentd plugin 2019-07-08 10:56:38 -05:00
Brandon Myers 6e0ee8d0a0
Merge remote-tracking branch 'origin/master' into python_3_upgrade 2019-07-08 10:48:19 -05:00
Brandon Myers 17ab5bbb30
Fixup cloudtrail worker to no longer use old boto version 2019-07-03 17:10:29 -05:00
Brandon Myers 8a8562fce8
Convert sqs boto use to boto3 2019-07-03 12:58:35 -05:00
Michal Purzynski 5f5cd58d39 Merge branch 'master' of https://github.com/mozilla/mozdef into dhcpzeek 2019-07-03 10:24:16 -07:00
Brandon Myers 63c6cbf857
Update mq workers to remove need to use RawMessage 2019-07-02 19:14:33 -05:00
Brandon Myers 51f03e9f21
Merge pull request #1339 from mozilla/fixup_printstatements 
Ensure parenthesis for print statements
 2019-07-01 18:07:08 -05:00
Brandon Myers 81eebb18b7
Fixup cloudtrail worker to use BytesIO 2019-06-30 17:17:37 -05:00
Brandon Myers 1cb8709681
Fix local includes for mq and alerts 2019-06-30 16:52:32 -05:00
Brandon Myers e3543a86a6
Fix relative imports for mq lib 2019-06-30 16:05:21 -05:00
Brandon Myers ed1d4aa8cf
Fixup remaining python3 leftovers 2019-06-29 15:51:00 -05:00
Brandon Myers 9a075dcbe0
Remove unicode-u keyword 2019-06-29 15:11:00 -05:00
Brandon Myers 8506c4eb1a
Update syntax in cloudtrail worker 2019-06-28 18:59:06 -05:00
Brandon Myers d421dbb33f
Fixup mozdef_util query_model tests 2019-06-28 18:52:41 -05:00
Brandon Myers 390a3feef2
Rename unicode type to str 2019-06-28 18:21:48 -05:00
Brandon Myers 9e736c3b80
Fixup mq tests 2019-06-28 18:12:04 -05:00
Brandon Myers e30f3f1d69
Remove call to encode ascii on strings 2019-06-28 17:26:58 -05:00
Brandon Myers 827e99ffa6
Convert keys function return to list 2019-06-28 17:10:27 -05:00
Michal Purzynski 0b2f738662 Upgrade the DHCP message format to Zeek's format 2019-06-28 14:53:24 -07:00
Brandon Myers bd4c48db9a
Rename iteritems to items for dictionaries 2019-06-28 16:49:30 -05:00
Brandon Myers b2ca33ed7a
Fixup reload function namespace 2019-06-28 16:43:52 -05:00
Brandon Myers 3a37b42bca
Fix exception message function no longer available 2019-06-28 16:40:38 -05:00
Brandon Myers f1c4287fa5
Ensure parenthesis for print statements 2019-06-28 16:28:14 -05:00
Brandon Myers 49798f15f5
Ensure parenthesis for print statements 2019-06-28 13:13:28 -05:00
Brandon Myers a0f8e3fa10
Add geo_points for sourceip and destinationip 2019-06-28 11:14:56 -05:00
Brandon Myers 9510ba4556
Update cloudtrail plugin to support details.responseelements.credentials 2019-06-20 12:09:21 -05:00
Brandon Myers a89cc567b0
Update cloudtrail plugin for describeflowlogsrequest 2019-06-18 00:21:35 -05:00
Brandon Myers 7714970cac
Merge pull request #1308 from mozilla/deprecate_vidyo 
Move vidyo cron script and dependencies into mozdef-deprecated
 2019-06-03 17:52:32 -05:00
Brandon Myers 4d28c6d273
Update cloudtrail mapping 2019-06-03 17:37:30 -05:00
Brandon Myers 3bd574571c
Move vidyo cron script and dependencies into mozdef-deprecated 2019-06-03 13:17:07 -05:00
Brandon Myers c9e9e4ec62
Remove unused comments from mq plugins 2019-05-02 13:58:06 -05:00
Brandon Myers dc86d98c24
Fix if statement in vulnerability plugin 2019-04-11 14:00:22 -05:00
Brandon Myers 9ed9b3f866
Add check to see if type exists on message vulnerability plugin 2019-04-11 13:53:40 -05:00
Brandon Myers 12b2e85b2b
Remove unnecessary whitespace from file 2019-04-11 12:42:43 -05:00
Brandon Myers 0331731328
Merge remote-tracking branch 'origin/master' into doc_type_removal 2019-04-11 12:13:49 -05:00
Phrozyn c472f963db
Adding type as a static entry regardless if already set. 2019-04-10 13:03:39 -05:00
Phrozyn ff20881548
Modifying some of the type references. 2019-04-09 11:09:49 -05:00
A Smith 5845bd17eb
Merge pull request #1206 from mozilla/Fix_invalid_literal_proxy_fixup 
Fixing invalid literal in squidFixup.py
 2019-04-04 14:08:38 -05:00
Phrozyn 246b50d200
Fixing invalid literal in squidFixup.py 2019-04-04 14:07:14 -05:00
Phrozyn 7da9ba2044
Updating squidFixup to include a summary. 2019-04-04 13:49:27 -05:00
A Smith fb898a2da9
Merge pull request #1192 from mozilla/guardduty_fix_null_date 
Adding check for None type object in date fields.
 2019-04-03 13:47:19 -05:00
Phrozyn 12b9e9ef0e
adding tags assertions to tests. 2019-04-02 12:50:19 -05:00
Phrozyn a43c7ddc1f
lowercase TAGS in squidFixup.py 2019-04-01 15:10:59 -05:00
Phrozyn 87b23c19d6
fixing conditional syntax 2019-03-28 19:20:04 -05:00
Phrozyn 5a82201040
Adding check for None type object in date fields. 2019-03-28 18:09:03 -05:00
Phrozyn 4ea91f7ac0
Fixing flake8 erros 2019-03-28 11:21:23 -05:00
Phrozyn 9eafc93c01
Update to remove doc_type in favor of type, edited comments to reflect accuracy. 2019-03-25 13:15:49 -05:00
Phrozyn 6a9cdc3c9f
Minor tweaks to mq workers. 2019-03-25 13:14:42 -05:00
Phrozyn dcc3f68623
Updating sns_sqs worker to remove doc_type and add type. 2019-03-25 13:14:36 -05:00
Phrozyn 884ebbc98d
Removing doc_type parameters from papertrail worker, this will be handled by elasticsearch client. 2019-03-25 13:14:30 -05:00
Phrozyn fbe6b83f4c
Removing doc_type parameters from eventtask worker, this will be handled by elasticsearch client. 2019-03-25 13:14:25 -05:00
Phrozyn af076675da
removing doc_type to be handled via elasticsearch client, adding new type to handle subcategory filters 2019-03-25 13:14:18 -05:00
Brandon Myers e25d16ba21
Merge pull request #1132 from mozilla/fix_cloudtrail_parsing 
updating cloudtrail plugin to add details.requestparameters.tagging.
 2019-03-21 16:49:19 -05:00
A Smith 9c10b7c745
Merge pull request #1118 from mozilla/parse_sqs_sshd_events_properly 
Resolving issues with sshd events not parsing correctly.
 2019-03-20 14:14:27 -05:00
Phrozyn 5149b8cbf7
updating cloudtrail plugin to add details.requestparameters.tagging. 2019-03-19 18:44:30 -05:00
Brandon Myers 435553cf1f
Fix pyyaml warning messages to use safe loader 2019-03-14 14:51:01 -05:00
Phrozyn 5d47bf2f37
Resolving issues with sshd events not parsing correctly. 2019-03-05 15:21:27 -06:00
Brandon Myers e9566f614a
Merge pull request #1064 from mozilla/replace_timer_with_threads 
Replace timer with threads
 2019-02-28 12:31:49 -05:00
Brandon Myers 4190c8d5c5
Merge pull request #1105 from mozilla/fixup_keys_references 
Remove .keys() call during key exists comparison
 2019-02-27 18:03:44 -05:00
Brandon Myers e16ec577bf
Remove .keys() call during key exists comparison 2019-02-15 12:11:15 -06:00
Brandon Myers 1d38a41369
Exclude auth_success field if not present on message bro ssh logs 2019-02-13 12:11:08 -06:00
Brandon Myers b875dcd627
Project plugins key from mq plugins removing it 2019-02-08 11:29:15 -06:00
Brandon Myers 14652f6511
Update mq plugins to create key correctly 2019-01-31 19:15:43 -06:00
Brandon Myers b3be820e15
Merge pull request #1083 from mozilla/reorder_plugins_key 
Update plugins key on event ordering
 2019-01-31 18:36:15 -06:00
Brandon Myers 3a51bc6583
Update plugins key on event ordering 2019-01-31 18:26:20 -06:00
Michal Purzynski 01c1339d38
Merge branch 'master' into squid_parsing 2019-01-30 18:19:13 -08:00
Brandon Myers 1ca517b3f3
Merge remote-tracking branch 'origin/master' into replace_timer_with_threads 2019-01-30 13:24:03 -06:00
Brandon Myers ea53957621
Merge remote-tracking branch 'origin/master' into replace_timer_with_threads 2019-01-30 13:22:52 -06:00
Brandon Myers 3f87b3e14a
Merge pull request #1070 from mozilla/lowercase_matching_key_mq_plugins 
Lowercase potential matching keys in mq plugins
 2019-01-30 13:21:19 -06:00
Brandon Myers 25488a483b
Merge pull request #1071 from mozilla/add_plugins_field_events 
Add plugins field to events and populate with mq plugins ran
 2019-01-30 13:20:43 -06:00
Michal Purzynski fc422b4327 Remove debugging leftovers 2019-01-29 20:40:42 -08:00
Michal Purzynski 6f18480102 PEP8 changes 2019-01-24 15:52:25 -08:00
Brandon Myers 2db449ec5c
Add plugins field to events and populate with mq plugins ran 2019-01-24 15:36:06 -06:00
Brandon Myers 92edd1d0c1
Lowercase potential matching keys in mq plugins 2019-01-24 15:30:24 -06:00
Brandon Myers 57c5dad652
Replace timer with threads 2019-01-23 11:59:31 -06:00
Brandon Myers 6c5ea5083e
Replace timer with thread for reauth in cloudtrail 2019-01-23 11:05:37 -06:00
Brandon Myers 0522b3ce6c
Remove duplicate code from cloudtrail worker 2019-01-22 12:39:47 -06:00
Brandon Myers 08749db287
Modify import for get_aws_credentials 2019-01-22 12:39:35 -06:00
Brandon Myers 7e7c10fdbb
Rename common file to lib/aws 2019-01-22 12:37:46 -06:00
Brandon Myers 7576a55ed7
Merge pull request #990 from ryandeivert/ryandeivert-dry-get-creds 
deduplicating get_aws_credentials function
 2019-01-22 12:35:23 -06:00
Michal Purzynski 529dfa45e4 Changed the data model, added heuristics to figure the destination in case of denies 2019-01-22 10:21:46 -08:00
Michal Purzynski 40d6c12ca3 A new plugin - parse Squid access log messages, coming from syslog-ng via AMQP. Replaces the squid2mozdef script 2019-01-18 16:51:44 -08:00
Brandon Myers 0f014f152f
Fixup filterlog mq plugin 2019-01-14 12:12:43 -06:00
Brandon Myers d8d88a5d35
Merge pull request #1020 from mozilla/lower_keys_fixes 
lowercasing tags for fxa
 2018-12-27 13:22:26 -05:00
Michal Purzynski 319532aed7 Remove the netaddr import 2018-12-26 14:50:32 -08:00
Michal Purzynski d93b2cbb29 Work around the lower_case plugin changes 2018-12-26 14:43:29 -08:00
Phrozyn 15b174743c
lowercasing tags for fxa, this fixes nothing. 2018-12-26 16:03:55 -06:00
Phrozyn 2963b703c9
moving this to run after lower_keys.py 2018-12-19 14:52:15 -06:00
Phrozyn 5da575f246
Correcting registration for fxa events, and removing replacement code. 2018-12-19 14:49:42 -06:00
Phrozyn 6e4d12c717
Resolving areas where keys are manipulated after lower_keys is run. 2018-12-19 11:27:00 -06:00
A Smith 9abad28a43
Merge pull request #1004 from mozilla/key_update_for_pulseguardian 
updating key fields for pulseguardian events to move source_ip to sou…
 2018-12-18 17:41:47 -06:00
A Smith 7215580095
Merge pull request #964 from mozilla/lower_keys 
Lower keys
 2018-12-18 17:41:27 -06:00
Brandon Myers 97409a248c
Merge pull request #995 from mozilla/add_port_details_root 
Move source port and destination port to details root
 2018-12-18 12:48:56 -06:00
Phrozyn 365c565023
updating key fields for pulseguardian events to move source_ip to sourceipaddress. 2018-12-17 10:58:39 -06:00
Brandon Myers 46be867d2f
Fixup unused variables check 2018-12-14 14:06:21 -06:00
Brandon Myers df84a1942d
Fixup block comments not having a space after hash 2018-12-14 13:40:07 -06:00
Brandon Myers be7788089d
Fixup missing whitespace around arithmetic operator 2018-12-14 12:49:25 -06:00
Brandon Myers 09989706a0
Fixup closing bracket indentation not matching original 2018-12-14 12:39:23 -06:00
Brandon Myers d04485c850
Fixup pep8 undefined library 2018-12-14 12:27:57 -06:00
Brandon Myers fc771bd531
Remove unused import statements 2018-12-14 11:34:42 -06:00
Brandon Myers e77b791c8a
Merge pull request #934 from mpurzynski/githubevent_pr 
A MozDef plugin that parses GitHub's Webhook events to create meaning…
 2018-12-13 15:52:41 -05:00
Michal Purzynski 9693dfa58e Address nits from the review - use mozdef_util instead of changing the path, remove unnecessary config file 2018-12-12 12:47:12 -08:00
Brandon Myers 4e28602162
Move source port and destination port to details root 2018-12-10 01:55:54 -05:00
Jeff Bryner 410eb27e1b explicitly accept/map 'source' field 2018-12-03 15:38:24 -05:00
Michal Purzynski 43f1fa2f53 Dynamically resolve path to the config file 2018-11-29 18:06:36 -08:00
Ryan Deivert 42032a99a7 deduplicating get_aws_credentials function 2018-11-29 15:37:45 -08:00
Michal Purzynski ebfacbe147 Move the mapping configuration to a plugin directory 2018-11-29 13:53:43 -08:00
Michal Purzynski 2548178183 Merge remote-tracking branch 'upstream/master' into githubevent_pr 2018-11-29 13:44:16 -08:00
A Smith 03dabc7524
Merge branch 'master' into lower_keys 2018-11-29 10:44:50 -06:00
Phrozyn 307d65165d
lowering keys that the lower_keys plugin will affect, and removing unused details.Random field. 2018-11-26 18:38:51 -06:00
Jeff Bryner 839d545dd6 pull ip from an occasionally present list 2018-11-23 09:26:45 -08:00
andrewkrug 440d50478d
fix flake 8 error 2018-11-21 07:43:37 -08:00
andrewkrug 5845d59dbb
ensure mozdef always polls the SQS queue we create 2018-11-21 06:55:46 -08:00
andrewkrug a14f51fd0e
standardize es_worker credential handling 2018-11-21 06:13:48 -08:00
Michal Purzynski fd5ffafbca Move the configuration file where it can be found 2018-11-20 15:37:22 -08:00
Brandon Myers 21aacc57a0
Add Principal key to cloudtrail plugin 2018-11-14 13:51:55 -06:00
Brandon Myers 006b708693
Sort cloudtrail keys in mq plugin 2018-11-14 13:51:17 -06:00
Phrozyn f9af2dc8f0
Updated code that works on subkeys. 2018-11-14 09:57:47 -06:00
Phrozyn 33e21788bf
initial commit 2018-11-13 16:10:09 -06:00
Brandon Myers 4d07a1e470
Merge pull request #933 from mpurzynski/large_strings_github 
Truncate, if present, the GitHub Webhook's pr_body field
 2018-11-05 15:35:47 -06:00
Michal Purzynski 90b746e5c6 remove newline at the end of the file 2018-11-05 12:11:58 -08:00
Brandon Myers acc00029fe
Merge pull request #932 from mpurzynski/fixup_fxafixup 
Make sure the key eventsource exists before referencing it
 2018-11-05 14:09:33 -06:00