mozilla
/
MozDef
зеркало из https://github.com/mozilla/MozDef.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность
5 424 коммитов 4 Ветки 53 Теги 68 MiB
Граф коммитов

509 Коммитов

Автор SHA1 Сообщение Дата
Phrozyn 44a81da8d6
hostname field normalization phase I 2018-09-13 14:04:22 -05:00
Michal Purzynski ba05341f19 Initial version of the plugin that parses Suricata eve-log alerts and matches field names to Bro 2018-09-12 19:25:05 -07:00
Brandon Myers 1150857fd9
Add callerReference to cloudtrail plugin handler 2018-09-06 13:59:27 -05:00
Phrozyn 638a2220bc
changing modification of sourceip to eventsourceipaddress instead of sourceipaddress. 2018-08-19 19:36:39 -05:00
A Smith 371158e5db
Merge pull request #733 from mozilla/properly_kill_bulk_queue 
Modify workers to stop bulk queue on errors
 2018-08-08 14:06:10 -05:00
Brandon Myers 80e3cc78b9
Removed unused sys exit in sqs worker 2018-08-06 13:11:39 -05:00
Brandon Myers a4980a249f
Modify workers to stop bulk queue on errors 2018-08-06 13:09:58 -05:00
Brandon Myers 67cc8be0fe
Add more keys to cloudtrail plugin 2018-08-06 11:14:03 -05:00
Brandon Myers dec8c1ec51
Add parsing for request source in cloudtrail plugin 2018-08-02 12:39:20 -05:00
Jeff Bryner 0e1ef26a90
Add details.requestparameters.instanceType 
Log errors say details.requestparameters.instanceType is sometimes an object:
"instanceType": {"value": "t2.medium"}}
 2018-07-28 12:09:06 -07:00
Brandon Myers 9e05f32acc
Add responseelements lastModified in cloudtrail plugin 2018-07-20 12:08:26 -05:00
Brandon Myers b77e38f8b0
Modify bro plugin to properly handle unicode for smtp 2018-07-19 10:50:45 -05:00
Jeff Bryner 2fe84fad0a
rename details.service to details.finding 
As per: https://github.com/mozilla/guardDuty2MozDef/pull/1/files
 2018-07-11 09:22:59 -07:00
Brandon Myers e4c096a680
Merge pull request #712 from mozilla/GuardDuty-Plugin 
Guard duty plugin
 2018-06-07 18:21:50 -05:00
Jeff Bryner ee14fb2c76
Pull in required fields 
If the sqs message contains, source, summary or processname use them.
 2018-06-07 10:32:59 -07:00
Jeff Bryner edd2f40db5 dot dict import 2018-06-06 12:25:54 -07:00
Jeff Bryner daf5a7db83 guard duty fixup for dates and ip addresses 2018-06-06 12:24:49 -07:00
Brandon Myers 07ed39a39d
Convert value to string for cloudtrail plugin 2018-05-08 18:12:34 -05:00
Brandon Myers 7634112ac6
Lower severity of few logger statements papertrail 2018-05-08 16:10:55 -05:00
A Smith a987b32893
Merge pull request #680 from mozilla/retry_papertrail_error 
Add retry error handling to papertrail worker
 2018-05-08 09:49:10 -07:00
A Smith 6e9d49bd81
Merge pull request #682 from mozilla/add_long_message_plugin 
Add plugin to cut off long message fields
 2018-05-08 09:48:15 -07:00
Brandon Myers 46d6bd1420
Add few more keys to cloudtrail plugin 2018-05-07 21:33:47 -05:00
Brandon Myers 98302918e0
Convert cloudtrail over to dynamic string mapping modification 2018-05-07 21:27:45 -05:00
Brandon Myers 382cd8b50c
Add plugin to cut off long message fields 2018-05-07 16:49:26 -05:00
Brandon Myers 9294d97e3e
Change severity of log line in papertrail worker 2018-05-07 15:44:00 -05:00
Brandon Myers 6dc3944886
Add retry error handling to papertrail worker 2018-05-07 11:48:38 -05:00
A Smith 85c6fdf12b
Merge pull request #677 from mozilla/fixup_sso_feedback 
Fixup worker and alert for sso feedback events
 2018-04-30 15:34:10 -05:00
Brandon Myers 26701ffa15
Fixup alert and worker for SSO feedback events 2018-04-30 12:43:59 -05:00
Brandon Myers d4514e943b
Update ini files to use new virtualenv path 2018-04-20 13:23:36 -05:00
Michal Purzynski 9a85cadd2e Always truncate the details.uri to prevent scanners from crashing us 2018-04-10 14:22:21 -07:00
A Smith 2651326fb2
Merge pull request #646 from mozilla/add_error_handling_papertrail 
Add error handling to papertrail worker
 2018-04-04 10:21:36 -05:00
Michal Purzynski 4699a05b68 Fixup BroFixup by moving the software's log fields away so they don't conflict with a details.version 2018-04-02 10:41:06 -07:00
Brandon Myers e3cd22c585
Add error handling to papertrail worker 2018-03-22 12:39:51 -05:00
Brandon Myers 3445ebdae3
Add handling of securitygroups in cloudtrail plugin 2018-03-15 12:52:07 -05:00
Brandon Myers a98b7136a1
Merge pull request #593 from mpurzynski/master 
A new and better version of brofixup for syslog-ng plus some tiny cle…
 2018-03-08 16:43:56 -06:00
Phrozyn e9a46b5aff
reverting processname edits but leaving regex changes 2018-03-08 10:29:28 -06:00
Tristan Weir a7ce5126c5 Added additional logic to check for field before analysing 2018-03-03 07:51:24 -08:00
A Smith 67da3b7ad2
Merge pull request #622 from mozilla/add_ebs_cloudtrail_mapping 
Add mapping exception for ebsoptimized in cloudtrail plugin
 2018-03-02 14:10:14 -06:00
Phrozyn 75ebf49cdc
removing session_closed regex since it's handled by session_open regex. 2018-03-02 13:41:29 -06:00
Phrozyn e86347b5f3
correcting if statement for details.program. 2018-03-02 12:47:24 -06:00
Brandon Myers c75349e0a3
Add mapping exception for ebsoptimized in cloudtrail plugin 2018-02-26 14:09:58 -06:00
Michal Purzynski 6ca16ae21d Contributors 2018-02-26 10:55:59 -08:00
Michal Purzynski 648d088731 Changes as requested 2018-02-23 16:56:59 -08:00
Phrozyn 1a87bd7764
Updates to parse_sshd.py to account for other fingerprint types. 2018-02-23 18:26:12 -06:00
Brandon Myers 27f928daba
Modify cloudtrail plugin to convert objects to string 2018-02-15 14:45:20 -06:00
Brandon Myers b5e118c0c0
Modify cloudtrail plugin to handle details.responseelements.endpoint 2018-02-15 13:32:35 -06:00
Michal Purzynski 951fcf61c0 A completely new version of the brofixup code with unit tests 2018-02-14 21:01:34 -08:00
Michal Purzynski ea6e080504 Merge remote-tracking branch 'upstream/master' 2018-02-14 21:00:40 -08:00
Brandon Myers 79fd605d3d
Add rule and subnets to cloudtrail plugin 2018-02-14 11:07:40 -06:00
Brandon Myers 55b9f2e840
Improve cloudtrail plugin parsing of string fields 2018-02-13 14:53:43 -06:00
Brandon Myers d16ac47ab8
Update cloudtrail plugin to handle description field type error 2018-02-07 11:43:58 -06:00
Brandon Myers a7058333f3
Add additional safe checks to cloudtrail mq plugin 2018-02-01 13:13:10 -06:00
Brandon Myers 49dc451097
Modify cloudtrail plugin to match on source 2018-02-01 13:02:30 -06:00
Brandon Myers 3cd95c22fe
Change key names to raw_value for details string in messages 2018-01-31 18:10:53 -06:00
Brandon Myers c160030a1b
Convert object type handling for cloudtrail into plugin 2018-01-31 18:07:59 -06:00
Michal Purzynski 927e4d9436 A new and better version of brofixup for syslog-ng plus some tiny cleanups 2018-01-29 14:47:45 -08:00
Brandon Myers eb7ec7ad6a
Modify workers to handle details key as non dict 2018-01-25 12:33:55 -06:00
Brandon Myers 4e4699eb95
Reapply cloudtrail worker improvements 2018-01-18 12:41:41 -06:00
Brandon Myers ec7efb70c3
Add logic to drop event in sns sqs worker 2018-01-12 15:48:16 -06:00
Brandon Myers c18875f65b
Add try except to on_message in cloudtrail worker 2018-01-12 15:05:00 -06:00
Brandon Myers 08762af4b7
Remove unnecessary new line in logger statement 2018-01-12 15:04:34 -06:00
Brandon Myers e5be0a0a3f
Convert sns sqs worker to use logger 2018-01-12 14:51:03 -06:00
Brandon Myers 7833800975
Modify sqs worker to use logger 2018-01-12 14:50:45 -06:00
Brandon Myers 4b248bde1c
Convert papertrail worker to using logger 2018-01-12 14:45:14 -06:00
Brandon Myers df4c12dafd
Convert cloudtrail esworker to using logger 2018-01-12 14:44:55 -06:00
Brandon Myers 38ddb2ee1a
Add logger to mq plugins 2018-01-12 14:44:31 -06:00
Brandon Myers 5835665e55
Log malformed event in eventtask worker 2018-01-11 17:02:33 -06:00
Brandon Myers 7c602afdf9
Switch workers to use lib functions 2018-01-11 16:07:12 -06:00
Brandon Myers c60c7b8c36
Remove extra line after copywrite date 2018-01-04 17:15:35 -06:00
Yash Mehrotra 90d7e3b6d3
Remove free-form 'Contributor:' text from code. Fixes #407 2017-12-23 02:14:53 +05:30
Brandon Myers 6ff09b9de6
Provide temporary patch for cloudtrail worker 2017-12-19 14:14:08 -06:00
Brandon Myers 59d95ff178
Revert "Merge pull request #554 from mozilla/improve_cloudtrail_worker" 
This reverts commit 501819cfb5, reversing
changes made to b09c700cb9.
 2017-12-08 16:09:57 -06:00
Brandon Myers f73cc3364d
Revert "Merge pull request #560 from mozilla/fix_cloudtrail_mapping" 
This reverts commit 804757f242, reversing
changes made to 501819cfb5.
 2017-12-08 16:09:43 -06:00
Brandon Myers ed49aee5ab
Fix missing import statements 2017-11-28 12:54:57 -06:00
Brandon Myers b006036528
Uppercase cloudtrail verb by default 2017-11-28 12:53:31 -06:00
Brandon Myers 4190ef43d6
Remove debugger line in mq worker 2017-11-15 17:25:14 -06:00
Brandon Myers 7c474d72ce
Update cloudtrail esworker fields 2017-11-15 17:16:49 -06:00
Brandon Myers 4278ffa39f
Update description of mq plugin 2017-11-13 22:25:30 -06:00
Brandon Myers f97b0f0c70
Add filterlog firewall mq plugin 2017-11-13 22:21:40 -06:00
Brandon Myers 58fa07d7cf
Add support to eventtask worker for syslog messages 2017-10-30 13:14:45 -05:00
Michal Purzynski d9ff430b21 Use the Bro's src field as sourceipaddress if present 2017-10-26 15:14:14 -07:00
Michal Purzynski aa7097156d Change the type field name to source - ES has problems if there is _type and type 2017-10-14 16:53:42 -07:00
A Smith f7834f79d2 Merge pull request #490 from mpurzynski/normalization_auth 
Normalization auth
 2017-10-12 11:00:17 -05:00
Brandon Myers 8ef7c4fd71
Merge remote-tracking branch 'origin' into add_events_class 2017-10-10 13:15:51 -05:00
Phrozyn 0f6cbd5fde
Merge branch 'naming_convention_changes' of https://github.com/Phrozyn/MozDef into naming_convention_changes 2017-10-10 10:59:42 -05:00
Phrozyn 7cf87ac628
Merge branch 'master' of https://github.com/mozilla/MozDef into naming_convention_changes 2017-10-10 10:59:27 -05:00
Phrozyn b6d5d1b57c
Fixing merge conflict 2017-10-10 10:55:13 -05:00
Phrozyn 1fd7335355
Naming Convention and Logging Changes. 2017-10-04 15:59:49 -05:00
Brandon Myers c4134f1764
Modify mq workers to use save_event method from es client 2017-09-28 14:57:18 -05:00
Brandon Myers badd86a44f Merge pull request #456 from mpurzynski/brofixup 
A first take on the new brofixup plugin.
 2017-09-28 12:02:20 -05:00
Michal Purzynski 435a267922 Last minute changes 2017-09-27 15:48:14 -07:00
Michal Purzynski 8a465bf29a More small fixes, correct unicode handling in SMTP summary 2017-09-27 13:33:08 -07:00
Michal Purzynski a8016907eb Even more refactoring and small changes 2017-09-26 10:25:34 -07:00
Michal Purzynski 991d94308a More unit tests 2017-09-25 17:42:58 -07:00
Michal Purzynski 2e18a286dd Testing never ends 2017-09-22 17:14:29 -04:00
Michal Purzynski c234e19b3f Small fixups 2017-09-21 16:46:25 -04:00
Brandon Myers 6db687cfb5
Modify esworker sns sqs to cast processid to str 2017-09-21 14:57:15 -05:00
Michal Purzynski ede31aad62 Small fixups here and there 2017-09-20 18:02:11 -04:00
Phrozyn bc3b56d151
Corrected some typos and added syslog change to syslog filter 2017-09-05 11:58:05 -05:00
Phrozyn 1a1a892dac
Merge branch 'master' of https://github.com/Phrozyn/MozDef into replace_dots_with_underscores_in_filenames 2017-09-05 10:18:09 -05:00
Gene Wood 6cd241a329 Extract action verb and add it along with readonly to the event 2017-09-01 13:11:28 -07:00
Michal Purzynski fa67e3d5d7 Even more cleanups 2017-08-31 16:40:28 -07:00
Michal Purzynski ccc7aae3c8 Initial commit for the data normalization initiative 2017-08-30 15:55:33 -07:00
Michal Purzynski 74dd2c2374 A first take on the new brofixup plugin. 2017-08-29 15:58:09 -07:00
Phrozyn 6199701f61
updated papertrail with changes from repo. 2017-08-25 13:34:45 -05:00
Phrozyn 4f1007a134
Updated code to reflect naming convention changes. 2017-08-25 12:17:53 -05:00
Phrozyn 2c415b673b
updated dots to underscores 2017-08-25 11:58:31 -05:00
Brandon Myers e396e5f230
Remove unused functions from esworker 2017-08-23 15:33:49 -04:00
Brandon Myers a7934e6f9b
Remove unused functions from mq 2017-08-23 15:22:48 -04:00
Brandon Myers 40fb30172f
Change default mq creds in conf 2017-08-17 18:21:07 -05:00
Brandon Myers 81fa3819cc
Update bot and mq plugin to use GeoIP class 2017-08-08 12:46:54 -05:00
Brandon Myers 4b665d8771
Convert registration term to lowercase fxa plugin 2017-07-17 13:18:48 -05:00
Brandon Myers caaf662ab7
Update fxa mq plugin to use new category 2017-07-17 13:01:44 -05:00
Brandon Myers ad64804e32
Add travisci to project and stabalize tests 2017-07-05 16:37:41 -05:00
Brandon Myers 63b3cf2194
Remove old leftover files 2017-06-15 15:13:03 -05:00
Brandon Myers fe96636655
Improve cloudtrail mq worker 2017-06-15 15:07:46 -05:00
Brandon Myers c632ed8250
Fix mozillaLocation mq plugin 2017-06-15 15:07:46 -05:00
Brandon Myers c6aaa8add8
Remove mozilla mq worker sample conf files 2017-06-15 15:07:45 -05:00
Brandon Myers cd25328625
Remove mozilla specific workers 2017-06-15 15:07:45 -05:00
Brandon Myers e59d2097ed
Remove default rabbitmq config 2017-06-15 15:07:44 -05:00
Brandon Myers b52c506810
Add defaults for sns sqs worker 2017-06-15 15:07:44 -05:00
Brandon Myers 29e3dec9ed
Add alerts to use config files 2017-06-15 15:07:42 -05:00
Brandon Myers bac6c7450a
Remove unncessary parsys file 2017-06-15 15:07:40 -05:00
Brandon Myers 43a722c65d
Fix typo in parsys ini file 2017-06-15 15:07:40 -05:00
Brandon Myers 1c4fc1071c
Remove unused mq workers 2017-06-15 15:07:38 -05:00
Brandon Myers 496311a364
Add parsys mq worker 2017-06-15 15:07:30 -05:00
Brandon Myers 9e734175e7
Add SNS SQS mq worker 2017-06-15 15:07:30 -05:00
Phrozyn ab3714d22a
Adding log drain back into uwsgi ini files. 2017-06-15 15:07:28 -05:00
Phrozyn 06899804fb
Adding contegix-auditd service and dummy conf and ini. 2017-06-15 15:07:25 -05:00
Phrozyn 1b4716ad2c
Moving uwsgi logging to syslog. 2017-06-15 15:07:22 -05:00
Phrozyn 24c2df918f
New contegix worker 2017-06-15 15:07:21 -05:00
Brandon Myers 5180f496e9
Add files for SSO sqs worker 2017-06-15 15:07:19 -05:00
Brandon Myers 7873cc38ea
Add thread to reauth every 30 minutes cloudtrail 2017-06-15 15:07:18 -05:00
Brandon Myers dbb78759ed
Add prefetch option to get_messages 2017-06-15 15:07:18 -05:00
Brandon Myers 1e300f7915
Add exception handling to cloudtrail worker 2017-06-15 15:07:18 -05:00
Brandon Myers 48e008346e
Add bulk to cloudtrail worker 2017-06-15 15:07:18 -05:00
Brandon Myers aa497395a7
Switch cloudtrail from cron to mq worker 2017-06-15 15:07:17 -05:00
Phrozyn 028505cd3b
Adding new mqworkers for fluentd2mozdef from aws infosec services. 2017-06-15 15:06:21 -05:00
Aaron Meihm 39ab8738ea
add configuration to drain mig sqs log queue 2017-06-15 15:06:02 -05:00
Brandon Myers f87c94a088
Unencrypt config files 
Signed-off-by: Brandon Myers <bmyers@mozilla.com>
 2017-06-15 15:05:55 -05:00
Phrozyn 97b0d685c6
Fixing mule issue in fxa with moar mules. 2017-06-15 15:05:53 -05:00
Brandon Myers d7a38c83f5
Remove creds from mq directory 
Signed-off-by: Brandon Myers <bmyers@mozilla.com>
 2017-06-15 15:05:43 -05:00
Brandon Myers 5fb9fbea7d
Move papertrail disabled to ini script 
Signed-off-by: Brandon Myers <bmyers@mozilla.com>
 2017-06-15 15:05:42 -05:00
Brandon Myers fb8806814b
Remove prod versions of esworker conf 
Signed-off-by: Brandon Myers <bmyers@mozilla.com>
 2017-06-15 15:05:42 -05:00
Phrozyn 7b903a1b81
Changing filenames to reflect a better workflow from dev to stage to prod. 2017-06-15 15:05:42 -05:00
Phrozyn 8da9fb5c34
Correcting mqwFxa.ini to remove stage from pid path 2017-06-15 15:05:41 -05:00
Brandon Myers c7b1e934b4
Update location of geolitecity data file 
Signed-off-by: Brandon Myers <bmyers@mozilla.com>
 2017-06-15 15:05:40 -05:00
Brandon Myers 5d03bc03d7
Remove mules from papertrail 
Signed-off-by: Brandon Myers <bmyers@mozilla.com>
 2017-06-15 15:05:33 -05:00
Brandon Myers 577c5cecfa
Fix missing import in fluentdSqsFixup 
Signed-off-by: Brandon Myers <bmyers@mozilla.com>
 2017-06-15 15:05:32 -05:00
Brandon Myers 13aa806b1b
Move unittest from mq plugin to own file 
Signed-off-by: Brandon Myers <bmyers@mozilla.com>
 2017-06-15 15:05:32 -05:00
Brandon Myers 1fb67e49fb
Remove unittest from fluentdSqsFixup 
Signed-off-by: Brandon Myers <bmyers@mozilla.com>
 2017-06-15 15:05:31 -05:00
Phrozyn cf55546506
Omitting the FxaOauthWebserver eventsource. 2017-06-15 15:05:19 -05:00
Brandon Myers d2ea5c3334
Add missing esworker releng conf 
Signed-off-by: Brandon Myers <bmyers@mozilla.com>
 2017-06-15 15:05:07 -05:00
Brandon Myers 79b2ee84ca
Add more workers to mqwSyslog 
Signed-off-by: Brandon Myers <bmyers@mozilla.com>
 2017-06-15 15:05:07 -05:00
Brandon Myers c37c2fb7d1
Update mq creds in mq alertWorker 
Signed-off-by: Brandon Myers <bmyers@mozilla.com>
 2017-06-15 15:05:05 -05:00
Phrozyn 66e963fc51
Updating mq/esworker.conf mq creds for mozdef user (was using qa2) 2017-06-15 15:05:01 -05:00
Phrozyn b4ff2e575d
Updating packaged config to include mozdef4. 2017-06-15 15:05:00 -05:00
Brandon Myers ec5d1ad5b7
Keep in sync with qa1 #70 
Signed-off-by: Brandon Myers <bmyers@mozilla.com>
 2017-06-15 15:04:54 -05:00
Phrozyn dbfc18190f
Fixed mozdefmqwFxaStage to reflect correct pid path. 2017-06-15 15:04:54 -05:00
Phrozyn 9c6d9364de
Fixed mozdefmqwAutoland to reflect correct pid path. 2017-06-15 15:04:53 -05:00
Phrozyn 2089dc225f
Added all prod service files and mq workers. 2017-06-15 15:04:53 -05:00
Phrozyn b86413db27
Updated pid path for all uwsgi instances to run from /var/run/ 2017-06-15 15:04:53 -05:00
Brandon Myers 16abe5adcc
Remove cloudtrail fixup mapping 
Signed-off-by: Brandon Myers <bmyers@mozilla.com>
 2017-06-15 15:04:52 -05:00
Phrozyn 3e02f27d14
modified esservers to new cluster. 2017-06-15 15:04:45 -05:00
Brandon Myers ee07fe18a3
Modify esservers from localhost to cluster 
Signed-off-by: Brandon Myers <bmyers@mozilla.com>
 2017-06-15 15:03:45 -05:00
Brandon Myers 28080dd980
Fix remaining qa references in prod 
Signed-off-by: Brandon Myers <bmyers@mozilla.com>
 2017-06-15 15:03:45 -05:00
Brandon Myers 75bb6542ee
Merge prod mq ini files with qa 
Signed-off-by: Brandon Myers <bmyers@mozilla.com>
 2017-06-15 15:03:44 -05:00
Brandon Myers ef6e483c7e
First import of existing files from prod 
Signed-off-by: Brandon Myers <bmyers@mozilla.com>
 2017-06-15 15:03:44 -05:00
Brandon Myers e9a4a67e5a
Modify .py scripts to use /opt dir 
Signed-off-by: Brandon Myers <bmyers@mozilla.com>
 2017-06-15 15:03:41 -05:00
Brandon Myers 007cf86c35
Modify .ini.disabled scripts to use /opt dir 
Signed-off-by: Brandon Myers <bmyers@mozilla.com>
 2017-06-15 15:03:41 -05:00
Brandon Myers 50a7cb772a
Modify .ini scripts to use /opt dir 
Signed-off-by: Brandon Myers <bmyers@mozilla.com>
 2017-06-15 15:03:40 -05:00
Brandon Myers 81a07bc2d5
Rename mozdefqa1 to localhost in configs 
Signed-off-by: Brandon Myers <bmyers@mozilla.com>
 2017-06-15 15:03:40 -05:00
Aaron Meihm a6f7c78597
update vulnerability plugin to handle version 2 messages 2017-06-15 15:03:39 -05:00
Brandon Myers 71692067cc
Add error support plus tests to bulk import 
Signed-off-by: Brandon Myers <bmyers@mozilla.com>
 2017-06-15 15:03:38 -05:00
Brandon Myers ea17b5883c
Fix toUTC isoformat problem 
Signed-off-by: Brandon Myers <bmyers@mozilla.com>
 2017-06-15 15:03:33 -05:00
Brandon Myers bfba1d3c4c
Add apiVersion mapping fix for cloudtrail 
Signed-off-by: Brandon Myers <bmyers@mozilla.com>
 2017-06-15 15:03:23 -05:00
Brandon Myers 6774599a37
Add exception in fxaFixup for fxa-auth-server 
Signed-off-by: Brandon Myers <bmyers@mozilla.com>
 2017-06-15 15:03:23 -05:00
Brandon Myers 6caaad320d
Remove duplicate definitions of toUTC 
Signed-off-by: Brandon Myers <bmyers@mozilla.com>
 2017-06-15 15:02:46 -05:00
Brandon Myers e832b313ee
Fix flush_bulk for pyes only 
Signed-off-by: Brandon Myers <bmyers@mozilla.com>
 2017-06-15 15:02:01 -05:00
Brandon Myers 76174add7d
Update mq directory with search class 
Signed-off-by: Brandon Myers <bmyers@mozilla.com>
 2017-06-15 15:02:01 -05:00
Brandon Myers 5082d87f68
Update alertWorker config 
Signed-off-by: Brandon Myers <bmyers@mozilla.com>
 2017-06-15 15:02:00 -05:00
Brandon Myers 49a042107e
Remove mq/safe directory and files 
Signed-off-by: Brandon Myers <bmyers@mozilla.com>
 2017-06-15 15:01:58 -05:00
Brandon Myers 67b38ae579
Remove mq/mq files and directory 
Signed-off-by: Brandon Myers <bmyers@mozilla.com>
 2017-06-15 15:01:42 -05:00
Phrozyn e7cef0564f
Adding additional mq ini changes. 2017-06-15 15:00:49 -05:00
Phrozyn edcc26f84e
Modifying thread/Process values to be in alignment with mozdefqa1's resources. Disabled unused workers. 2017-06-15 15:00:49 -05:00
Brandon Myers 375b0290de
Update conf files to use US/Pacific 
Signed-off-by: Brandon Myers <bmyers@mozilla.com>
 2017-06-15 15:00:48 -05:00
Brandon Myers e5e98c1304
Switch mq directory to US/Pacific 
Signed-off-by: Brandon Myers <bmyers@mozilla.com>
 2017-06-15 15:00:46 -05:00
Phrozyn dbc7e43e41
unencrypting ini files 2017-06-15 15:00:46 -05:00
Phrozyn ac9925be6d
adding unencrypted mqESmules.ini 
adding unencrypted mqESmulesAWS.ini
 2017-06-15 15:00:45 -05:00
Phrozyn 5c990d90ef
Unencrypting ini files. 2017-06-15 15:00:45 -05:00
Phrozyn 700f0abf5f
Releng Papertrail ini for esworker. 2017-06-15 15:00:44 -05:00
Brandon Myers 99fa7ca655
Remove rra files 
Signed-off-by: Brandon Myers <bmyers@mozilla.com>
 2017-06-15 15:00:42 -05:00
Gene Wood d9911b4a77
adding mozdefmq support for infosec sqs non prod queue 2017-06-15 15:00:42 -05:00
Brandon Myers 1d8c59b93f
Setup codebase for merge of two repos 2017-06-15 14:56:47 -05:00
Brandon Myers 9a2388c398 Update GeoLiteCity.dat location in mq plugin 
Signed-off-by: Brandon Myers <bmyers@mozilla.com>
 2016-10-19 16:35:46 -05:00
Jeff Bryner d9afcb288b Merge pull request #350 from Phrozyn/master 
corrected typo in mq/plugins/fluentdSqsFixup.py
 2016-06-28 20:51:09 -07:00
Phrozyn 58a31fdc3c corrected typo in mq/plugins/fluentdSqsFixup.py 2016-06-28 19:17:37 -05:00
Jeff Bryner a0580d1848 Merge pull request #345 from pwnbus/remove_time_fluentdSqsFixup 
Remove details.time from fluentdSqsFixup
 2016-06-08 17:07:01 -07:00
Brandon Myers f84c3ca4e1 Remove details.time from fluentdSqsFixup 
Signed-off-by: Brandon Myers <bmyers@mozilla.com>
 2016-06-08 18:01:04 -05:00
kang 950b0868eb Sync with rra2json message format 
Add support for RRA versionning
 2016-06-03 11:35:22 -07:00
Jeff Bryner 7fd56b8d93 update geoip cache file location 2016-03-23 14:13:59 -07:00
Jeff Bryner d87569d486 add common/handy options 2016-03-23 12:57:46 -07:00
Aaron Meihm a3d9668888 adds an esworker for processing data from papertrail 2016-03-01 14:57:33 -06:00
Guillaume Destuynder 09f7a038b3 Use details.program as standard field for processname instead of fluentd 2015-10-22 10:54:42 -07:00
Guillaume Destuynder 231c3415b3 Add mq plugin: normalizer for fluentd-SQS messages (AWS). Ensure registration matches your SQS queue tag. 2015-10-22 10:54:15 -07:00
Guillaume Destuynder 334f5466a4 Fix reading of SQS JSON msgs - this works regardless of messages being raw JSON or base64-encoded JSON. 
Since Boto does base64 encode messages while writing to the queue this can happen (also since we use Boto, we were
previously expecting all messages to be base64 encoded, which wouldn't work if your writer wasn't Boto)
 2015-10-20 12:44:03 -07:00
Jeff Bryner f2524fb132 Merge pull request #302 from gdestuynder/master 
Support more validation filters to accomodate different RRA fields.
 2015-10-18 12:29:34 -07:00
Guillaume Destuynder 996a566813 Support more validation filters to accomodate different RRA fields. 
This enhance the validation accuracy ;-)
 2015-10-14 17:21:49 -07:00
Jeff Bryner f259564a78 add sqs-specific worker, closes #294 2015-10-12 14:00:05 -07:00
Jeff Bryner af526d6e4e revert sqs changes due to kombu issues 2015-10-12 13:59:32 -07:00
Guillaume Destuynder ec334de898 Merge branch 'master' of https://github.com/jeffbryner/MozDef 2015-10-09 18:45:30 -07:00
Guillaume Destuynder 80df3b0e44 Update to support new data classification 2015-10-09 18:44:39 -07:00
Jeff Bryner e0ff817332 fix dict2list to support embedded list of dicts, closes #297 2015-10-08 13:21:59 -07:00
Jeff Bryner f43d574b94 initial support for SQS in esworker, #294 2015-10-08 13:14:05 -07:00
Jeff Bryner eae8bdf1f4 add hostname to the message metadata, closes #289 2015-09-27 18:57:25 -07:00
Guillaume Destuynder f87c675d9c Also warn on missing service names for debugging 2015-06-17 14:21:35 -07:00
Guillaume Destuynder 1ad2d8c37d Fix validation check (entered CIA but not RPF) 
Added more verbose warning on validation check
 2015-06-16 17:05:30 -07:00
Guillaume Destuynder f4aafb5945 Plugin support for RRA index/events 2015-06-15 16:28:52 -07:00
Jeff Bryner 63bcbf4373 rm old ini file for old alertWorker 2015-03-22 20:16:28 -07:00
Jeff Bryner ad69a216f8 add alert plug in system, closes #162 2015-03-22 20:15:17 -07:00
Julien Vehent 8929794486 Remove doctype requirement on complianceitems plugin 2015-03-13 17:17:47 -04:00
Julien Vehent e7cb5760f7 Make complianceitem plugin extract item data from event message 2015-03-13 16:28:17 -04:00
Jeff Bryner fb1cbe0458 smarter IP finding 2015-02-13 09:31:13 -08:00
Aaron Meihm 6fb0ea4c13 also copy tags during compliance item event cleanup 2015-02-10 11:40:15 -06:00
Aaron Meihm 67d7d84bcf sourcename in vuln event docid to add isolation between different writers 2015-02-02 14:19:08 -06:00
Jeff Bryner c0218c08e2 vulnerability->vulnerabilities for consistent index naming 2015-01-30 12:24:35 -08:00
Aaron Meihm 9a4efd1e12 add MozDef vulnerability processing plugin 2015-01-30 11:36:49 -06:00
Jeff Bryner c104efd126 Merge pull request #216 from jvehent/master 
complianceitems plugin, take 2
 2014-12-16 17:02:00 -08:00
Julien Vehent 25f5ec69d6 complianceitems plugin, take 2 2014-12-16 19:03:59 -05:00
Jeff Bryner 1777c70781 Merge pull request #215 from jvehent/master 
complianceitems mozdef plugin, take 1
 2014-12-16 13:18:17 -08:00
Julien Vehent 2d57f88380 complianceitems mozdef plugin, take 1 2014-12-16 16:13:49 -05:00
Michal Purzynski bf0c21eb36 Add X-Cluster-Client-IP generated from NSM as yet another possible source of the real client IP 2014-12-16 21:25:28 +01:00
Jeff Bryner f35743b2c3 update esworker to accept utctimestamp as a field, closes #208 2014-12-01 10:21:42 -08:00
Jeff Bryner 981678eaa9 observium parsing plugin 2014-10-08 10:38:53 -07:00
Jeff Bryner ff4544de2f sourcehostname==hostname for consistency 2014-09-26 11:17:09 -07:00
Jeff Bryner 9c919996ca rework netflow plugin to match netflow to rabbit MQ input source 2014-09-15 13:07:34 -07:00
jeffbryner eeb62ea246 Merge pull request #185 from netantho/averez-netflow 
netflow v5
 2014-07-31 11:21:31 -07:00
Anthony Verez 13ac6341da averez-netflow: add netflow esworker plugin 2014-07-31 11:20:03 -07:00
Anthony Verez c3899f7ad1 averez-observium: Observium plugin by @XioNoX 2014-07-31 10:54:25 -07:00
Jeff Bryner c7975a3fbd improve logic and ipv4 finding 2014-07-03 08:47:51 -07:00
Jeff Bryner bee13b0066 bugfix: use sane version of found IP 2014-07-02 18:53:11 -07:00
Jeff Bryner 5128e29ac8 works for fail2ban also 2014-07-02 16:47:11 -07:00
Jeff Bryner a76fc32f55 fixup IP finding for edge cases with quoted strings 2014-07-02 15:03:57 -07:00
Jeff Bryner a8609e6348 account for netaddr seeing 1,0,etc as valid ipv4 addresses 2014-06-30 12:35:46 -07:00
Jeff Bryner 7cb8dc105b add support for nxlog windows event log parsing 2014-06-27 11:31:54 -07:00
Jeff Bryner 8d8c82a7f2 sshd event plugin to find ips in the message string 2014-06-25 12:57:54 -07:00
Jeff Bryner 8bbbf387c5 standardize the field names 2014-06-24 09:13:18 -07:00
Jeff Bryner 23ddf455fb internz mixing tabs and spaces 2014-06-24 08:59:13 -07:00
jeffbryner 506b035b46 Merge pull request #118 from netantho/averez-snmptt-plugin 
snmptt plugin
 2014-06-24 08:54:51 -07:00
jeffbryner 6f5e8ca23b Merge pull request #117 from netantho/averez-rtflow-plugin 
RT_FLOW plugin
 2014-06-24 08:54:16 -07:00