зеркало из https://github.com/mozilla/MozDef.git
Merge remote-tracking branch 'origin/master' into add_feedback_event
This commit is contained in:
Brandon Myers
Коммит
f465ccf120
27
Makefile
27
Makefile
|
|
@ -3,11 +3,6 @@
|
|||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# 2xyo <yohann@lepage.info>
|
||||
# Yohann Lepage yohann@lepage.info
|
||||
# Anthony Verez averez@mozilla.com
|
||||
# Brandon Myers bmyers@mozilla.com
|
||||
|
||||
# usage:
|
||||
# make single-build - build new single image from Dockerfile
|
||||
|
|
@ -17,12 +12,18 @@
|
|||
# make single-stop - stop a single instance of MozDef
|
||||
# make single-rebuild - build, stop and run a new single instance of MozDef
|
||||
# make multiple-build - build new mozdef environment in multiple containers
|
||||
# make multiple-build-tests - build new mozdef environment for tests in multiple containers
|
||||
# make multiple-build-no-cache - build new mozdef environment in multiple containers from scratch
|
||||
# make multiple-run - run new mozdef environment in multiple containers
|
||||
# make multiple-run-tests - run new mozdef environment for tests in multiple containers
|
||||
# make multiple-stop - stop new mozdef environment in multiple containers
|
||||
# make multiple-stop-tests - stop new mozdef environment for tests in multiple containers
|
||||
# make multiple-rm - stop new mozdef environment in multiple containers and deattach volumes
|
||||
# make multiple-rm-tests - stop new mozdef tests environment in multiple containers and deattach volumes
|
||||
# make multiple-rebuild - build, stop and run new mozdef environment in multiple containers
|
||||
# make multiple-rebuild-new - build, stop/rm and run new mozdef environment in multiple containers
|
||||
# make multiple-rebuild-tests - build, stop/rm and run new mozdef environment for tests in multiple containers
|
||||
# make multiple-rebuild-tests-new - build, stop/rm and run new mozdef environment for tests in multiple containers
|
||||
|
||||
NAME=mozdef
|
||||
VERSION=0.1
|
||||
|
|
@ -76,20 +77,36 @@ single-rebuild: single-build single-stop single-run
|
|||
multiple-run:
|
||||
docker-compose -f docker/compose/docker-compose.yml -p $(NAME) up -d
|
||||
|
||||
multiple-run-tests:
|
||||
docker-compose -f docker/compose/docker-compose-tests.yml -p $(NAME) up -d --remove-orphans
|
||||
|
||||
multiple-build:
|
||||
docker-compose -f docker/compose/docker-compose.yml -p $(NAME) build
|
||||
|
||||
multiple-build-tests:
|
||||
docker-compose -f docker/compose/docker-compose-tests.yml -p $(NAME) build
|
||||
|
||||
multiple-build-no-cache:
|
||||
docker-compose -f docker/compose/docker-compose.yml -p $(NAME) build --no-cache
|
||||
|
||||
multiple-stop:
|
||||
-docker-compose -f docker/compose/docker-compose.yml -p $(NAME) stop
|
||||
|
||||
multiple-stop-tests:
|
||||
-docker-compose -f docker/compose/docker-compose-tests.yml -p $(NAME) stop
|
||||
|
||||
multiple-rm:
|
||||
-docker-compose -f docker/compose/docker-compose.yml -p $(NAME) down -v --remove-orphans
|
||||
|
||||
multiple-rm-tests:
|
||||
-docker-compose -f docker/compose/docker-compose-tests.yml -p $(NAME) down -v --remove-orphans
|
||||
|
||||
multiple-rebuild: multiple-build multiple-stop multiple-run
|
||||
|
||||
multiple-rebuild-new: multiple-build multiple-rm multiple-run
|
||||
|
||||
multiple-rebuild-tests: multiple-build-tests multiple-stop-tests multiple-run-tests
|
||||
|
||||
multiple-rebuild-tests-new: multiple-build-tests multiple-rm-tests multiple-run-tests
|
||||
|
||||
.PHONY: multiple-build multiple-run multiple-stop multiple-rebuild
|
||||
|
|
|
|||
17
README.md
17
README.md
|
|
@ -1,18 +1,15 @@
|
|||
[](https://travis-ci.org/mozilla/MozDef)
|
||||
[](http://mozdef.readthedocs.io/en/latest/?badge=latest)
|
||||
|
||||
MozDef: The Mozilla Defense Platform
|
||||
=====================================
|
||||
# MozDef: The Mozilla Defense Platform
|
||||
|
||||
Why?
|
||||
----
|
||||
## Why?
|
||||
|
||||
The inspiration for MozDef comes from the large arsenal of tools available to attackers. Suites like metasploit, armitage, lair, dradis and others are readily available to help attackers coordinate, share intelligence and finely tune their attacks in real time. Defenders are usually limited to wikis, ticketing systems and manual tracking databases attached to the end of a Security Information Event Management (SIEM) system.
|
||||
|
||||
The Mozilla Defense Platform (MozDef) seeks to automate the security incident handling process and facilitate the real-time activities of incident handlers.
|
||||
|
||||
Goals:
|
||||
------
|
||||
## Goals:
|
||||
|
||||
* Provide a platform for use by defenders to rapidly discover and respond to security incidents.
|
||||
* Automate interfaces to other systems like bunker, banhammer, mig
|
||||
|
|
@ -21,10 +18,10 @@ Goals:
|
|||
* Facilitate repeatable, predictable processes for incident handling
|
||||
* Go beyond traditional SIEM systems in automating incident handling, information sharing, workflow, metrics and response automation
|
||||
|
||||
Status:
|
||||
--------
|
||||
## Status:
|
||||
|
||||
MozDef is in production at Mozilla where we are using it to process over 300 million events per day.
|
||||
|
||||
DOCS:
|
||||
-----
|
||||
## DOCS:
|
||||
|
||||
http://mozdef.readthedocs.org/en/latest/
|
||||
|
|
|
|||
|
|
@ -5,10 +5,6 @@
|
|||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Jeff Bryner jbryner@mozilla.com
|
||||
# Brandon Myers bmyers@mozilla.com
|
||||
#
|
||||
# Alert Worker to listen for alerts and call python plugins
|
||||
# for user-controlled reaction to alerts.
|
||||
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Anthony Verez averez@mozilla.com
|
||||
|
||||
from lib.alerttask import AlertTask
|
||||
from query_models import SearchQuery, TermMatch, ExistsMatch, PhraseMatch
|
||||
|
|
|
|||
|
|
@ -4,13 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Anthony Verez averez@mozilla.com
|
||||
# Jeff Bryner jbryner@mozilla.com
|
||||
# Aaron Meihm ameihm@mozilla.com
|
||||
# Michal Purzynski <mpurzynski@mozilla.com>
|
||||
# Alicia Smith <asmith@mozilla.com>
|
||||
|
||||
from lib.alerttask import AlertTask
|
||||
from query_models import SearchQuery, TermMatch, PhraseMatch
|
||||
|
|
|
|||
|
|
@ -4,11 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2017 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Anthony Verez averez@mozilla.com
|
||||
# Jeff Bryner jbryner@mozilla.com
|
||||
# Brandon Myers bmyers@mozilla.com
|
||||
|
||||
from lib.alerttask import AlertTask
|
||||
from query_models import SearchQuery, TermMatch, PhraseMatch, TermsMatch
|
||||
|
|
|
|||
|
|
@ -4,10 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2017 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Michal Purzynski michal@mozilla.com
|
||||
# Brandon Myers bmyers@mozilla.com
|
||||
|
||||
from lib.alerttask import AlertTask
|
||||
from query_models import SearchQuery, TermMatch, ExistsMatch, PhraseMatch
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2017 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Brandon Myers bmyers@mozilla.com
|
||||
|
||||
|
||||
from lib.alerttask import AlertTask
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2017 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Brandon Myers bmyers@mozilla.com
|
||||
|
||||
from lib.alerttask import AlertTask
|
||||
from query_models import SearchQuery, TermMatch
|
||||
|
|
@ -18,10 +15,10 @@ class AlertCloudtrailLoggingDisabled(AlertTask):
|
|||
|
||||
search_query.add_must([
|
||||
TermMatch('_type', 'cloudtrail'),
|
||||
TermMatch('details.eventname', 'StopLogging'),
|
||||
TermMatch('eventName', 'StopLogging'),
|
||||
])
|
||||
|
||||
search_query.add_must_not(TermMatch('details.errorcode', 'AccessDenied'))
|
||||
search_query.add_must_not(TermMatch('errorCode', 'AccessDenied'))
|
||||
|
||||
self.filtersManual(search_query)
|
||||
self.searchEventsSimple()
|
||||
|
|
@ -32,6 +29,6 @@ class AlertCloudtrailLoggingDisabled(AlertTask):
|
|||
tags = ['cloudtrail', 'aws', 'cloudtrailpagerduty']
|
||||
severity = 'CRITICAL'
|
||||
|
||||
summary = 'Cloudtrail Logging Disabled: ' + event['_source']['details']['requestparameters']['name']
|
||||
summary = 'Cloudtrail Logging Disabled: ' + event['_source']['requestParameters']['name']
|
||||
|
||||
return self.createAlertDict(summary, category, tags, [event], severity)
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Jonathan Claudius jclaudius@mozilla.com
|
||||
|
||||
from lib.alerttask import AlertTask
|
||||
from query_models import SearchQuery, TermMatch, QueryStringMatch
|
||||
|
|
|
|||
|
|
@ -4,10 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2017 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Michal Purzynski michal@mozilla.com
|
||||
# Brandon Myers bmyers@mozilla.com
|
||||
|
||||
from lib.alerttask import AlertTask
|
||||
from query_models import SearchQuery, TermMatch, ExistsMatch, PhraseMatch
|
||||
|
|
|
|||
|
|
@ -5,10 +5,6 @@
|
|||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2017 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Jeff Bryner jbryner@mozilla.com
|
||||
# Brandon Myers bmyers@mozilla.com
|
||||
#
|
||||
# a collection of alerts looking for the lack of events
|
||||
# to alert on a dead input source.
|
||||
|
||||
|
|
|
|||
|
|
@ -2,14 +2,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2017 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Anthony Verez averez@mozilla.com
|
||||
# Jeff Bryner jbryner@mozilla.com
|
||||
# Aaron Meihm ameihm@mozilla.com
|
||||
# Michal Purzynski <mpurzynski@mozilla.com>
|
||||
# Alicia Smith <asmith@mozilla.com>
|
||||
# Brandon Myers bmyers@mozilla.com
|
||||
|
||||
from lib.alerttask import AlertTask
|
||||
from query_models import SearchQuery, TermMatch, ExistsMatch, PhraseMatch
|
||||
|
|
|
|||
|
|
@ -5,9 +5,6 @@
|
|||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# kang@mozilla.com
|
||||
#
|
||||
# This script alerts when openvpn's duo security failed to contact the duo server and let the user in.
|
||||
# This is a very serious warning that must be acted upon as it means MFA failed and only one factor was validated (in
|
||||
# this case a VPN certificate)
|
||||
|
|
|
|||
|
|
@ -4,10 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2017 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Jeff Bryner jbryner@mozilla.com
|
||||
# Brandon Myers bmyers@mozilla.com
|
||||
|
||||
from lib.alerttask import AlertTask
|
||||
from query_models import SearchQuery, TermMatch, PhraseMatch, WildcardMatch
|
||||
|
|
|
|||
|
|
@ -4,10 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2017 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# kang@mozilla.com
|
||||
# bmyers@mozilla.com
|
||||
|
||||
# TODO: Dont use query_models, nicer fixes for AlertTask
|
||||
|
||||
|
|
|
|||
|
|
@ -4,10 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2015 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Aaron Meihm <ameihm@mozilla.com>
|
||||
# Brandon Myers <bmyers@mozilla.com>
|
||||
|
||||
from lib.alerttask import AlertTask
|
||||
from query_models import SearchQuery, TermMatch
|
||||
|
|
|
|||
|
|
@ -4,10 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Anthony Verez averez@mozilla.com
|
||||
# Jeff Bryner jbryner@mozilla.com
|
||||
|
||||
from lib.alerttask import AlertTask
|
||||
from query_models import SearchQuery, TermMatch, ExistsMatch, PhraseMatch
|
||||
|
|
|
|||
|
|
@ -4,10 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2017 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Michal Purzynski michal@mozilla.com
|
||||
# Brandon Myers bmyers@mozilla.com
|
||||
|
||||
from lib.alerttask import AlertTask
|
||||
from query_models import SearchQuery, TermMatch, ExistsMatch, PhraseMatch
|
||||
|
|
|
|||
|
|
@ -4,10 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2017 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Michal Purzynski michal@mozilla.com
|
||||
# Brandon Myers bmyers@mozilla.com
|
||||
|
||||
from lib.alerttask import AlertTask
|
||||
from query_models import SearchQuery, TermMatch, ExistsMatch, PhraseMatch
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Jeff Bryner jbryner@mozilla.com
|
||||
|
||||
from lib.alerttask import AlertTask
|
||||
from query_models import SearchQuery, TermMatch
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Jeff Bryner jbryner@mozilla.com
|
||||
|
||||
from lib.alerttask import AlertTask
|
||||
from query_models import SearchQuery, TermMatch
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Jeff Bryner jbryner@mozilla.com
|
||||
|
||||
from lib.alerttask import AlertTask
|
||||
from query_models import SearchQuery, TermMatch, PhraseMatch
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Jeff Bryner jbryner@mozilla.com
|
||||
|
||||
from lib.alerttask import AlertTask
|
||||
from query_models import SearchQuery, TermMatch, PhraseMatch
|
||||
|
|
|
|||
|
|
@ -10,7 +10,7 @@ class AlertPluginSet(PluginSet):
|
|||
|
||||
def send_message_to_plugin(self, plugin_class, message, metadata=None):
|
||||
if 'utctimestamp' in message and 'summary' in message:
|
||||
message_log_str = '{0} received message: ({1}) {2}'.format(plugin_class.__module__, message['utctimestamp'], message['summary'])
|
||||
message_log_str = u'{0} received message: ({1}) {2}'.format(plugin_class.__module__, message['utctimestamp'], message['summary'])
|
||||
logger.info(message_log_str)
|
||||
|
||||
return plugin_class.onMessage(message), metadata
|
||||
|
|
|
|||
|
|
@ -4,11 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2017 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Anthony Verez averez@mozilla.com
|
||||
# Jeff Bryner jbryner@mozilla.com
|
||||
# Brandon Myers bmyers@mozilla.com
|
||||
|
||||
import collections
|
||||
import json
|
||||
|
|
@ -26,7 +21,7 @@ from config import RABBITMQ, ES
|
|||
sys.path.append(os.path.join(os.path.dirname(__file__), "../../lib"))
|
||||
from utilities.toUTC import toUTC
|
||||
from elasticsearch_client import ElasticsearchClient
|
||||
from query_models import TermMatch
|
||||
from query_models import TermMatch, ExistsMatch
|
||||
|
||||
|
||||
# utility functions used by AlertTask.mostCommon
|
||||
|
|
@ -258,6 +253,13 @@ class AlertTask(Task):
|
|||
relative to the _source that's returned from elastic search.
|
||||
ex: details.sourceipaddress
|
||||
"""
|
||||
|
||||
# We automatically add the key that we're matching on
|
||||
# for aggregation, as a query requirement
|
||||
aggreg_key_exists = ExistsMatch(aggregationPath)
|
||||
if aggreg_key_exists not in self.main_query.must:
|
||||
self.main_query.add_must(aggreg_key_exists)
|
||||
|
||||
try:
|
||||
esresults = self.main_query.execute(self.es, indices=self.event_indices)
|
||||
results = esresults['hits']
|
||||
|
|
|
|||
|
|
@ -4,10 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Anthony Verez averez@mozilla.com
|
||||
# Jeff Bryner jbryner@mozilla.com
|
||||
|
||||
from celery.schedules import crontab, timedelta
|
||||
import time
|
||||
|
|
|
|||
|
|
@ -4,11 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Anthony Verez averez@mozilla.com
|
||||
# Jeff Bryner jbryner@mozilla.com
|
||||
# Michal Purzynski <mpurzynski@mozilla.com>
|
||||
|
||||
from lib.alerttask import AlertTask
|
||||
from query_models import SearchQuery, TermMatch, ExistsMatch, TermsMatch
|
||||
|
|
|
|||
|
|
@ -5,9 +5,6 @@
|
|||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2017 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Brandon Myers bmyers@mozilla.com
|
||||
#
|
||||
# Looks for events that have an old timestamp
|
||||
# which could mean theres something wrong in the event pipeline
|
||||
|
||||
|
|
|
|||
|
|
@ -4,10 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2017 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Jonathan Claudius jclaudius@mozilla.com
|
||||
# Brandon Myers bmyers@mozilla.com
|
||||
|
||||
from lib.alerttask import AlertTask
|
||||
from query_models import SearchQuery, TermMatch, PhraseMatch
|
||||
|
|
|
|||
|
|
@ -2,9 +2,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Brandon Myers bmyers@mozilla.com
|
||||
|
||||
import hjson
|
||||
import os
|
||||
|
|
|
|||
|
|
@ -2,9 +2,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Jeff Bryner jbryner@mozilla.com
|
||||
|
||||
import requests
|
||||
import json
|
||||
|
|
|
|||
|
|
@ -5,9 +5,6 @@
|
|||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Michal Purzynski mpurzynski@mozilla.com
|
||||
#
|
||||
# This code alerts on every successfully opened session on any of the host from a given list
|
||||
|
||||
from lib.alerttask import AlertTask
|
||||
|
|
|
|||
|
|
@ -5,9 +5,6 @@
|
|||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Michal Purzynski mpurzynski@mozilla.com
|
||||
#
|
||||
# This code alerts on every successfully opened session on any of the host from a given list
|
||||
|
||||
from lib.alerttask import AlertTask
|
||||
|
|
|
|||
|
|
@ -4,11 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Jonathan Claudius jclaudius@mozilla.com
|
||||
# Brandon Myers bmyers@mozilla.com
|
||||
# Alicia Smith asmith@mozilla.com
|
||||
|
||||
|
||||
from lib.alerttask import AlertTask
|
||||
|
|
|
|||
|
|
@ -5,9 +5,6 @@
|
|||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Michal Purzynski mpurzynski@mozilla.com
|
||||
#
|
||||
# This code alerts on every successfully opened session for any user in the list
|
||||
|
||||
import datetime
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2017 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# bmyers@mozilla.com
|
||||
|
||||
from lib.alerttask import AlertTask
|
||||
from query_models import SearchQuery, TermMatch
|
||||
|
|
|
|||
|
|
@ -4,10 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2017 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Aaron Meihm <ameihm@mozilla.com>
|
||||
# Brandon Myers bmyers@mozilla.com
|
||||
|
||||
from lib.alerttask import AlertTask
|
||||
from query_models import SearchQuery, TermMatch, PhraseMatch, QueryStringMatch
|
||||
|
|
|
|||
|
|
@ -4,10 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2017 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Michal Purzynski michal@mozilla.com
|
||||
# Brandon Myers bmyers@mozilla.com
|
||||
|
||||
from lib.alerttask import AlertTask
|
||||
from query_models import SearchQuery, TermMatch, ExistsMatch, PhraseMatch
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2015 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Aaron Meihm <ameihm@mozilla.com>
|
||||
|
||||
from lib.alerttask import AlertTask
|
||||
from query_models import SearchQuery, TermMatch
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2017 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Aaron Meihm <ameihm@mozilla.com>
|
||||
|
||||
from lib.alerttask import AlertTask
|
||||
from query_models import SearchQuery, TermMatch
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2017 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Aaron Meihm <ameihm@mozilla.com>
|
||||
|
||||
from lib.alerttask import AlertTask
|
||||
from query_models import SearchQuery, TermMatch, QueryStringMatch, PhraseMatch
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Michal Purzynski michal@mozilla.com
|
||||
|
||||
from lib.alerttask import AlertTask
|
||||
from query_models import SearchQuery, TermMatch, ExistsMatch
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2015 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Aaron Meihm <ameihm@mozilla.com>
|
||||
|
||||
from lib.alerttask import AlertTask
|
||||
from query_models import SearchQuery, TermMatch, QueryStringMatch, PhraseMatch
|
||||
|
|
|
|||
|
|
@ -4,10 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Anthony Verez averez@mozilla.com
|
||||
# Jeff Bryner jbryner@mozilla.com
|
||||
|
||||
from lib.alerttask import AlertTask
|
||||
from query_models import SearchQuery, TermMatch, PhraseMatch
|
||||
|
|
|
|||
|
|
@ -2,9 +2,6 @@
|
|||
// License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
// file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
// Copyright (c) 2014 Mozilla Corporation
|
||||
//
|
||||
// Contributors:
|
||||
// Anthony Verez averez@mozilla.com
|
||||
|
||||
// Usage: node ./insert_bulk.js <processes> <insertsPerQuery> <totalInserts> <host1> [host2] [host3] [...]
|
||||
|
||||
|
|
|
|||
|
|
@ -2,9 +2,6 @@
|
|||
// License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
// file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
// Copyright (c) 2014 Mozilla Corporation
|
||||
//
|
||||
// Contributors:
|
||||
// Anthony Verez averez@mozilla.com
|
||||
|
||||
// Usage: node ./insert_simple.js <processes> <totalInserts> <host1> [host2] [host3] [...]
|
||||
|
||||
|
|
|
|||
|
|
@ -2,9 +2,6 @@
|
|||
// License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
// file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
// Copyright (c) 2014 Mozilla Corporation
|
||||
//
|
||||
// Contributors:
|
||||
// Anthony Verez averez@mozilla.com
|
||||
|
||||
// Usage: node ./search_all_fulltext.js <processes> <totalSearches> <host1> [host2] [host3] [...]
|
||||
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Jeff Bryner jbryner@mozilla.com
|
||||
|
||||
import os
|
||||
import sys
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Jeff Bryner jbryner@mozilla.com
|
||||
#
|
||||
# Copy of https://github.com/gdestuynder/Stupid-python-bot/blob/master/modules/roulette.py
|
||||
# ported to kitnirc
|
||||
|
||||
|
|
@ -56,4 +53,4 @@ class Roulette(Module):
|
|||
return True
|
||||
|
||||
# Let KitnIRC know what module class it should be loading.
|
||||
module = Roulette
|
||||
module = Roulette
|
||||
|
|
|
|||
|
|
@ -5,10 +5,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# kang@mozilla.com
|
||||
#
|
||||
|
||||
import logging
|
||||
from kitnirc.client import Channel
|
||||
|
|
@ -66,15 +62,17 @@ class Zilla(Module):
|
|||
except AttributeError:
|
||||
_log.warning("zilla could not load search terms")
|
||||
return
|
||||
try:
|
||||
res = self._bugzilla.search_bugs(terms)
|
||||
except Exception as e:
|
||||
_log.error('Error querying bugzilla' + str(e))
|
||||
return
|
||||
for bug in res['bugs']:
|
||||
bugsummary = bug['summary'].encode('utf-8', 'replace')
|
||||
self.controller.client.msg(self.channel, "\x037\x02WARNING\x03\x02 \x032\x02NEW\x03\x02 bug: {url}{bugid} {summary}".format(summary=bugsummary,
|
||||
url=self.url, bugid=bug['id']))
|
||||
|
||||
for search_group in terms:
|
||||
try:
|
||||
res = self._bugzilla.search_bugs(search_group)
|
||||
except Exception as e:
|
||||
_log.error('Error querying bugzilla' + str(e))
|
||||
return
|
||||
for bug in res['bugs']:
|
||||
bugsummary = bug['summary'].encode('utf-8', 'replace')
|
||||
self.controller.client.msg(self.channel, "\x037\x02WARNING\x03\x02 \x032\x02NEW\x03\x02 bug: {url}{bugid} {summary}".format(summary=bugsummary,
|
||||
url=self.url, bugid=bug['id']))
|
||||
|
||||
def start(self, *args, **kwargs):
|
||||
super(Zilla, self).start(*args, **kwargs)
|
||||
|
|
|
|||
|
|
@ -19,5 +19,4 @@ url = https://bugzilla.mozilla.org/
|
|||
api_key = <add_api_key>
|
||||
interval= 300
|
||||
channel = #somechannel
|
||||
search_terms = [{"product": "<add_product_name>"}, {"status": "NEW"}, {"status": "UNCONFIRMED"}]
|
||||
|
||||
search_terms = [[{"product": "<add_product_name>"}, {"status": "NEW"}, {"status": "UNCONFIRMED"}]]
|
||||
|
|
|
|||
|
|
@ -3,9 +3,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Jeff Bryner jbryner@mozilla.com
|
||||
|
||||
"""mozdef bot using KitnIRC."""
|
||||
import json
|
||||
|
|
|
|||
|
|
@ -3,9 +3,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Brandon Myers bmyers@mozilla.com
|
||||
|
||||
""" mozdef bot using slack
|
||||
to install - 'pip install slackclient'
|
||||
|
|
|
|||
|
|
@ -141,7 +141,7 @@
|
|||
"index" : "not_analyzed",
|
||||
"type" : "long"
|
||||
},
|
||||
"apiVersion" : {
|
||||
"apiversion" : {
|
||||
"type" : "string",
|
||||
"index" : "not_analyzed",
|
||||
"doc_values" : true
|
||||
|
|
|
|||
|
|
@ -4,10 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Anthony Verez averez@mozilla.com
|
||||
# Jeff Bryner jbryner@mozilla.com
|
||||
|
||||
# Snapshot configured backups
|
||||
# Meant to be run once/day
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Anthony Verez averez@mozilla.com
|
||||
|
||||
source /opt/mozdef/envs/mozdef/bin/activate
|
||||
/opt/mozdef/envs/mozdef/cron/backupSnapshot.py -c /opt/mozdef/envs/mozdef/cron/backup.conf
|
||||
|
|
|
|||
|
|
@ -4,10 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Jeff Bryner jbryner@mozilla.com
|
||||
# Brandon Myers bmyers@mozilla.com
|
||||
|
||||
import collections
|
||||
import json
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Jeff Bryner jbryner@mozilla.com
|
||||
|
||||
source /opt/mozdef/envs/mozdef/bin/activate
|
||||
/opt/mozdef/envs/mozdef/cron/collectAttackers.py -c /opt/mozdef/envs/mozdef/cron/collectAttackers.conf
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Jeff Bryner jbryner@mozilla.com
|
||||
|
||||
import logging
|
||||
import random
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Jeff Bryner jbryner@mozilla.com
|
||||
|
||||
source /opt/mozdef/envs/mozdef/bin/activate
|
||||
/opt/mozdef/envs/mozdef/cron/collectSSHFingerprints.py -c /opt/mozdef/envs/mozdef/cron/collectSSHFingerprints.conf
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Jeff Bryner jbryner@mozilla.com
|
||||
|
||||
import json
|
||||
import logging
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Jeff Bryner jbryner@mozilla.com
|
||||
|
||||
source /opt/mozdef/envs/mozdef/bin/activate
|
||||
/opt/mozdef/envs/mozdef/cron/correlateUserMacAddress.py -c /opt/mozdef/envs/mozdef/cron/correlateUserMacAddress.conf
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Jeff Bryner jbryner@mozilla.com
|
||||
|
||||
import boto
|
||||
import boto.s3
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Jeff Bryner jbryner@mozilla.com
|
||||
|
||||
source /opt/mozdef/envs/mozdef/bin/activate
|
||||
/opt/mozdef/envs/mozdef/cron/createIPBlockList.py -c /opt/mozdef/envs/mozdef/cron/createIPBlockList.conf
|
||||
|
|
|
|||
|
|
@ -3,9 +3,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
#
|
||||
# Contributors:
|
||||
# Guillaume Destuynder kang@mozilla.com
|
||||
# Brandon Myers bmyers@mozilla.com
|
||||
|
||||
import sys
|
||||
import os
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Jeff Bryner jbryner@mozilla.com
|
||||
|
||||
source /opt/mozdef/envs/mozdef/bin/activate
|
||||
/opt/mozdef/envs/mozdef/cron/rotateIndexes.py -c /opt/mozdef/envs/mozdef/cron/backup.conf
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Jeff Bryner jbryner@mozilla.com
|
||||
|
||||
import json
|
||||
import logging
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Brandon Myers bmyers@mozilla.com
|
||||
|
||||
source /opt/mozdef/envs/mozdef/bin/activate
|
||||
/opt/mozdef/envs/mozdef/cron/eventStats.py -c /opt/mozdef/envs/mozdef/cron/eventStats.conf
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Jeff Bryner jbryner@mozilla.com
|
||||
|
||||
import os
|
||||
import sys
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Jeff Bryner jbryner@mozilla.com
|
||||
|
||||
source /opt/mozdef/envs/mozdef/bin/activate
|
||||
/opt/mozdef/envs/mozdef/cron/google2mozdef.py -c /opt/mozdef/envs/mozdef/cron/google2mozdef.conf
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Brandon Myers bmyers@mozilla.com
|
||||
|
||||
source /opt/mozdef/envs/mozdef/bin/activate
|
||||
/opt/mozdef/envs/mozdef/cron/healthAndStatus.py -c /opt/mozdef/envs/mozdef/cron/healthAndStatus.fxa.conf
|
||||
|
|
|
|||
|
|
@ -4,10 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Jeff Bryner jbryner@mozilla.com
|
||||
# Anthony Verez averez@mozilla.com
|
||||
|
||||
import json
|
||||
import logging
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Jeff Bryner jbryner@mozilla.com
|
||||
|
||||
source /opt/mozdef/envs/mozdef/bin/activate
|
||||
/opt/mozdef/envs/mozdef/cron/healthAndStatus.py -c /opt/mozdef/envs/mozdef/cron/healthAndStatus.conf
|
||||
|
|
|
|||
|
|
@ -4,9 +4,7 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Anthony Verez averez@mozilla.com
|
||||
|
||||
|
||||
import logging
|
||||
import requests
|
||||
|
|
@ -79,7 +77,7 @@ def getEsNodesStats():
|
|||
jsonobj = r.json()
|
||||
results = []
|
||||
for nodeid in jsonobj['nodes']:
|
||||
# Skip non masters and data nodes since it won't have full stats
|
||||
# Skip non masters and non data nodes since it won't have full stats
|
||||
if ('attributes' in jsonobj['nodes'][nodeid] and
|
||||
jsonobj['nodes'][nodeid]['attributes']['master'] == 'false' and
|
||||
jsonobj['nodes'][nodeid]['attributes']['data'] == 'false'):
|
||||
|
|
@ -90,6 +88,7 @@ def getEsNodesStats():
|
|||
'disk_free': jsonobj['nodes'][nodeid]['fs']['total']['free_in_bytes'] / (1024 * 1024 * 1024),
|
||||
'disk_total': jsonobj['nodes'][nodeid]['fs']['total']['total_in_bytes'] / (1024 * 1024 * 1024),
|
||||
'mem_heap_per': jsonobj['nodes'][nodeid]['jvm']['mem']['heap_used_percent'],
|
||||
'gc_old': jsonobj['nodes'][nodeid]['jvm']['gc']['collectors']['old']['collection_time_in_millis'] / 1000,
|
||||
'cpu_usage': jsonobj['nodes'][nodeid]['os']['cpu_percent'],
|
||||
'load': jsonobj['nodes'][nodeid]['os']['load_average']
|
||||
})
|
||||
|
|
@ -162,4 +161,3 @@ if __name__ == '__main__':
|
|||
initConfig()
|
||||
initLogger()
|
||||
main()
|
||||
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Brandon Myers bmyers@mozilla.com
|
||||
|
||||
source /opt/mozdef/envs/mozdef/bin/activate
|
||||
/opt/mozdef/envs/mozdef/cron/healthToMongo.py -c /opt/mozdef/envs/mozdef/cron/healthToMongo.conf
|
||||
/opt/mozdef/envs/mozdef/cron/healthToMongo.py -c /opt/mozdef/envs/mozdef/cron/healthToMongo.conf
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2017 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Brandon Myers bmyers@mozilla.com
|
||||
|
||||
import os
|
||||
import sys
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2017 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Brandon Myers bmyers@mozilla.com
|
||||
|
||||
source /opt/mozdef/envs/mozdef/bin/activate
|
||||
/opt/mozdef/envs/mozdef/cron/import_threat_exchange.py -c /opt/mozdef/envs/mozdef/cron/import_threat_exchange.conf
|
||||
/opt/mozdef/envs/mozdef/cron/import_threat_exchange.py -c /opt/mozdef/envs/mozdef/cron/import_threat_exchange.conf
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Jeff Bryner jbryner@mozilla.com
|
||||
|
||||
import os
|
||||
import sys
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Jeff Bryner jbryner@mozilla.com
|
||||
|
||||
source /opt/mozdef/envs/mozdef/bin/activate
|
||||
/opt/mozdef/envs/mozdef/cron/okta2mozdef.py -c /opt/mozdef/envs/mozdef/cron/okta2mozdef.conf
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Jeff Bryner jbryner@mozilla.com
|
||||
|
||||
source /opt/mozdef/envs/mozdef/bin/activate
|
||||
/opt/mozdef/envs/mozdef/cron/pruneIndexes.py -c /opt/mozdef/envs/mozdef/cron/backup.conf
|
||||
|
|
|
|||
|
|
@ -4,10 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Jeff Bryner jbryner@mozilla.com
|
||||
# Anthony Verez averez@mozilla.com
|
||||
|
||||
# set this to run as a cronjob (after backup has completed)
|
||||
# to regularly remove indexes
|
||||
|
|
@ -60,7 +56,7 @@ def esPruneIndexes():
|
|||
index_to_prune += '-%s' % idate
|
||||
|
||||
if index_to_prune in indices:
|
||||
logger.info('Deleting index: %s' % index_to_prune)
|
||||
logger.debug('Deleting index: %s' % index_to_prune)
|
||||
es.delete_index(index_to_prune, True)
|
||||
else:
|
||||
logger.error('Error deleting index %s, index missing' % index_to_prune)
|
||||
|
|
|
|||
|
|
@ -4,10 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Jeff Bryner jbryner@mozilla.com
|
||||
# Anthony Verez averez@mozilla.com
|
||||
|
||||
# set this to run as a cronjob at 00:00 UTC to create the indexes
|
||||
# necessary for mozdef
|
||||
|
|
@ -76,8 +72,18 @@ def esRotateIndexes():
|
|||
logger.debug('do not rotate %s index, month has not changed yet' % index)
|
||||
continue
|
||||
if newindex not in indices:
|
||||
logger.debug('Creating %s index' % newindex)
|
||||
es.create_index(newindex)
|
||||
if 'alerts' in newindex:
|
||||
logger.debug('Creating %s index with single shard' % newindex)
|
||||
index_config = {
|
||||
"settings": {
|
||||
"number_of_shards": 1
|
||||
}
|
||||
}
|
||||
logger.debug('Creating %s index' % newindex)
|
||||
es.create_index(newindex, index_config)
|
||||
else:
|
||||
logger.debug('Creating %s index' % newindex)
|
||||
es.create_index(newindex)
|
||||
# set aliases: events to events-YYYYMMDD
|
||||
# and events-previous to events-YYYYMMDD-1
|
||||
logger.debug('Setting {0} alias to index: {1}'.format(index, newindex))
|
||||
|
|
|
|||
|
|
@ -4,10 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Jeff Bryner jbryner@mozilla.com
|
||||
# Anthony Verez averez@mozilla.com
|
||||
|
||||
# Use this to setup the index templates for mozdef
|
||||
# You only need to run it once, it will setup the templates
|
||||
|
|
@ -48,4 +44,4 @@ if __name__ == '__main__':
|
|||
initConfig()
|
||||
es = es_module.Elasticsearch(options.esservers[0])
|
||||
for templatename, templatefile in zip(options.templatenames, options.templatefiles):
|
||||
es.setupIndexTemplate(templatename, templatefile)
|
||||
es.setupIndexTemplate(templatename, templatefile)
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Jeff Bryner jbryner@mozilla.com
|
||||
|
||||
import calendar
|
||||
import logging
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Jeff Bryner jbryner@mozilla.com
|
||||
|
||||
source /opt/mozdef/envs/mozdef/bin/activate
|
||||
/opt/mozdef/envs/mozdef/cron/syncAlertsToMongo.py -c /opt/mozdef/envs/mozdef/cron/syncAlertsToMongo.conf
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2017 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Brandon Myers bmyers@mozilla.com
|
||||
|
||||
from git import Repo, cmd
|
||||
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2017 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Brandon Myers bmyers@mozilla.com
|
||||
|
||||
source /opt/mozdef/envs/mozdef/bin/activate
|
||||
/opt/mozdef/envs/mozdef/cron/update_generic_alerts.py -c /opt/mozdef/envs/mozdef/cron/update_generic_alerts.conf
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2017 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Brandon Myers bmyers@mozilla.com
|
||||
|
||||
import sys
|
||||
import os
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2017 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Brandon Myers bmyers@mozilla.com
|
||||
|
||||
source /opt/mozdef/envs/mozdef/bin/activate
|
||||
/opt/mozdef/envs/mozdef/cron/update_geolite_db.py -c /opt/mozdef/envs/mozdef/cron/update_geolite_db.conf
|
||||
|
|
|
|||
|
|
@ -5,3 +5,4 @@ aws_bucket_name = <add_aws_bucket_name>
|
|||
aws_document_key_name = allv4networks.txt
|
||||
local_ip_list_path=/opt/mozdef/envs/mozdef/static/iplist.txt
|
||||
ips_list_threshold = 20
|
||||
manual_additions = 1.2.3.0/24,4.5.6.7/32
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2017 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Brandon Myers bmyers@mozilla.com
|
||||
|
||||
import sys
|
||||
import os
|
||||
|
|
@ -42,6 +39,13 @@ def main():
|
|||
logger.debug('Starting')
|
||||
logger.debug(options)
|
||||
ips = fetch_ip_list(options.aws_access_key_id, options.aws_secret_access_key, options.aws_bucket_name, options.aws_document_key_name)
|
||||
|
||||
for manual_addition in options.manual_additions:
|
||||
if manual_addition == '':
|
||||
continue
|
||||
logger.debug("Adding manual addition: " + manual_addition)
|
||||
ips.append(manual_addition)
|
||||
|
||||
if len(ips) < options.ips_list_threshold:
|
||||
raise LookupError('IP List contains less than ' + str(options.ips_list_threshold) + ' entries...something is probably up here.')
|
||||
save_ip_list(options.local_ip_list_path, ips)
|
||||
|
|
@ -59,7 +63,7 @@ def initConfig():
|
|||
|
||||
options.local_ip_list_path = getConfig('local_ip_list_path', '', options.configfile)
|
||||
options.ips_list_threshold = getConfig('ips_list_threshold', 20, options.configfile)
|
||||
|
||||
options.manual_additions = getConfig('manual_additions', '', options.configfile).split(',')
|
||||
|
||||
if __name__ == '__main__':
|
||||
parser = OptionParser()
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2017 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Brandon Myers bmyers@mozilla.com
|
||||
|
||||
source /opt/mozdef/envs/mozdef/bin/activate
|
||||
/opt/mozdef/envs/mozdef/cron/update_ip_list.py -c /opt/mozdef/envs/mozdef/cron/update_ip_list.conf
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2017 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Brandon Myers bmyers@mozilla.com
|
||||
|
||||
import sys
|
||||
import os
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2017 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Brandon Myers bmyers@mozilla.com
|
||||
|
||||
source /opt/mozdef/envs/mozdef/bin/activate
|
||||
/opt/mozdef/envs/mozdef/cron/verify_event_fields.py -c /opt/mozdef/envs/mozdef/cron/verify_event_fields.conf
|
||||
|
|
|
|||
|
|
@ -5,9 +5,6 @@
|
|||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
|
||||
# Contributor: gdestuynder@mozilla.com
|
||||
# Contributor: jbryner@mozilla.com
|
||||
|
||||
import copy
|
||||
import os
|
||||
import sys
|
||||
|
|
|
|||
|
|
@ -4,9 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Jeff Bryner jbryner@mozilla.com
|
||||
|
||||
source /opt/mozdef/envs/mozdef/bin/activate
|
||||
/opt/mozdef/envs/mozdef/cron/vidyo2MozDef.py -c /opt/mozdef/envs/mozdef/cron/vidyo2MozDef.conf
|
||||
|
|
|
|||
|
|
@ -3,12 +3,6 @@
|
|||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Jeff Bryner jbryner@mozilla.com
|
||||
# Yohann Lepage yohann@lepage.info
|
||||
# Anthony Verez averez@mozilla.com
|
||||
# Charlie Lewis clewis@iqt.org
|
||||
# Brandon Myers bmyers@mozilla.com
|
||||
|
||||
FROM centos:7
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,26 @@
|
|||
---
|
||||
version: '2.2'
|
||||
services:
|
||||
elasticsearch:
|
||||
build:
|
||||
context: ../../
|
||||
dockerfile: docker/compose/elasticsearch/Dockerfile
|
||||
restart: always
|
||||
command: bin/elasticsearch -Des.insecure.allow.root=true -Dnetwork.host=0.0.0.0
|
||||
ports:
|
||||
- 9200:9200
|
||||
networks:
|
||||
- default
|
||||
rabbitmq:
|
||||
build:
|
||||
context: ../../
|
||||
dockerfile: docker/compose/rabbitmq/Dockerfile
|
||||
restart: always
|
||||
command: rabbitmq-server
|
||||
ports:
|
||||
- 5672:5672
|
||||
networks:
|
||||
- default
|
||||
|
||||
networks:
|
||||
default:
|
||||
|
|
@ -4,10 +4,6 @@
|
|||
# License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
||||
# Copyright (c) 2014 Mozilla Corporation
|
||||
#
|
||||
# Contributors:
|
||||
# Anthony Verez averez@mozilla.com
|
||||
# Brandon Myers bmyers@mozilla.com
|
||||
|
||||
from celery.schedules import crontab, timedelta
|
||||
import time
|
||||
|
|
|
|||
Некоторые файлы не были показаны из-за слишком большого количества измененных файлов Показать больше
Загрузка…
Ссылка в новой задаче