Distributed & real time digital forensics at the speed of the cloud
Перейти к файлу
Julien Vehent 4e9b29f816
[minor] fix container build in travisci
2017-03-08 12:30:25 -05:00
actions [minor] remove upgrade module and additional references to module 2016-12-21 15:08:49 -06:00
client [minor] fswatch and examplepersist modules are optional 2017-02-27 14:57:36 -06:00
conf [minor] fswatch and examplepersist modules are optional 2017-02-27 14:57:36 -06:00
database [minor] add a new loadername column to agents table 2017-02-07 16:18:35 -06:00
doc doc: remove html generated docs 2017-03-06 08:54:32 -05:00
mig-agent [minor] fswatch and examplepersist modules are optional 2017-02-27 14:57:36 -06:00
mig-api [minor] Create an API endpoint for agents to retrieve PGP public keys and update the documentation for api endpoints fixes #240 2016-09-12 23:39:44 +05:30
mig-loader [minor] remove symlinks from repo and replace with copies 2016-11-08 13:43:12 -06:00
mig-runner [medium] move compression apply into client package 2016-01-22 14:14:17 -06:00
mig-scheduler [minor] add keys to action log values 2017-02-07 16:18:35 -06:00
modules [minor] add default fswatch profiles for darwin/windows, currently empty 2017-02-27 17:00:58 -06:00
pgp [minor] update pgp getPINNaïve so stty -echo is successful 2017-02-03 10:52:30 -06:00
runner-plugins [medium] unify hashes under sha2/sha3, fixes #155 2016-01-14 16:42:06 +05:30
testutil [doc] add newline after license header to ignore it in godoc 2015-08-27 10:41:13 -04:00
tools [minor] move integration test to standalone install 2017-03-07 13:51:55 -05:00
vendor [minor] vendor fsnotify and associated deps 2017-02-27 14:58:37 -06:00
workers [minor] Remove old worker code and update documentation 2015-12-29 17:25:09 -06:00
.gitignore [minor] add vagrant dev box 2016-10-14 07:15:02 -04:00
.travis.yml [minor] fix container build in travisci 2017-03-08 12:30:25 -05:00
AUTHORS [doc] add Rob Murtha to AUTHORS file 2016-12-22 15:52:26 -06:00
CONTRIBUTING.md [doc] update Contributing.md 2016-10-07 21:21:22 -04:00
Dockerfile [minor] build standalone instance into Docker container 2017-03-07 11:43:06 -05:00
LICENSE [medium] Makefile support 2014-02-03 10:42:36 -05:00
Makefile [doc] add git tag makefile target 2017-03-08 12:00:44 -05:00
README.md [doc] Removed gap in README.md 2016-11-04 16:31:42 +01:00
acl.go [medium/bug] Prevent one investigator from signing multiple times 2016-09-24 21:42:24 -04:00
action.go [minor] Add ONLYVERIFYPUBKEY global to bypass verification of ACLs if set to true. 2017-02-06 18:22:07 -05:00
agent.go [minor] scheduler option to log summary of each agent actions/commands 2017-02-07 16:18:35 -06:00
command.go [doc] add newline after license header to ignore it in godoc 2015-08-27 10:41:13 -04:00
constants.go [medium/bug] terminate scheduler when heartbeat to relays fails, fixes #146 2015-11-05 08:14:00 -05:00
investigator.go [minor] improve investigator permission descriptive output 2016-08-16 12:28:24 -05:00
loader.go [medium] add environment validation to loader authorization 2016-08-11 15:28:57 -05:00
logging_posix.go [minor] add log file rotation for file output mode 2016-04-18 21:59:28 -05:00
logging_windows.go [minor/bug] fix to build windows agent 2016-04-22 17:43:43 +02:00
manifest.go [minor/bug] cache signers on validation and reject duplicates 2016-08-18 16:26:32 -05:00
runner.go [minor] initial commit of mig-runner 2015-09-15 14:40:26 -05:00
version.go [doc] update version.go 2016-04-29 11:46:55 -04:00

README.md

MIG: Mozilla InvestiGator

Build Status

Build one-liner:

$ go get mig.ninja/mig && cd $GOPATH/src/mig.ninja/mig && make

MIG is OpSec's platform for investigative surgery of remote endpoints.

MIG is composed of agents installed on all systems of an infrastructure that are be queried in real-time to investigate the file-systems, network state, memory or configuration of endpoints.

Capability Linux MacOS Windows
file inspection check check check
network inspection check check (partial)
memory inspection check check check
vuln management check (planned) (planned)
log analysis (planned) (planned) (planned)
system auditing (planned) (planned) (planned)

Imagine it is 7am on a saturday morning, and someone just released a critical vulnerability for your favorite PHP application. The vuln is already exploited and security groups are releasing indicators of compromise (IOCs). Your weekend isn't starting great, and the thought of manually inspecting thousands of systems isn't making it any better.

MIG can help. The signature of the vulnerable PHP app (the md5 of a file, a regex, or just a filename) can be searched for across all your systems using the file module. Similarly, IOCs such as specific log entries, backdoor files with md5 and sha1/2/3 hashes, IP addresses from botnets or byte strings in processes memories can be investigated using MIG. Suddenly, your weekend is looking a lot better. And with just a few commands, thousands of systems will be remotely investigated to verify that you're not at risk.

MIG command line demo

MIG agents are designed to be lightweight, secure, and easy to deploy so you can ask your favorite sysadmins to add it to a base deployment without fear of breaking the entire production network. All parameters are built into the agent at compile time, including the list and ACLs of authorized investigators. Security is enforced using PGP keys, and even if MIG's servers are compromised, as long as our keys are safe on your investigator's laptop, no one will break into the agents.

MIG is designed to be fast, and asynchronous. It uses AMQP to distribute actions to endpoints, and relies on Go channels to prevent components from blocking. Running actions and commands are stored in a Postgresql database and on disk cache, such that the reliability of the platform doesn't depend on long-running processes.

Speed is a strong requirement. Most actions will only take a few hundreds milliseconds to run on agents. Larger ones, for example when looking for a hash in a big directory, should run in less than a minute or two. All in all, an investigation usually completes in between 10 and 300 seconds.

Privacy and security are paramount. Agents never send raw data back to the platform, but only reply to questions instead. All actions are signed by GPG keys that are not stored in the platform, thus preventing a compromise from taking over the entire infrastructure.

Technology

MIG is built in Go and uses a REST API that receives signed JSON messages distributed to agents via RabbitMQ and stored in a Postgres database.

It is:

  • Massively Distributed means Fast.
  • Simple to deploy and Cross-Platform.
  • Secured using OpenPGP.
  • Respectful of privacy by never retrieving raw data from endpoints.

Check out this 10 minutes video for a more general presentation and a demo of the console interface.

MIG youtube video

MIG was recently presented at the SANS DFIR Summit in Austin, Tx. You can watch the recording below:

MIG @ DFIR Summit 2015

Discussion

Join #mig on irc.mozilla.org (use a web client such as mibbit).

We also have a public mailing list at list@mig.ninja.

Documentation

All documentation is available in the 'doc' directory and on http://mig.mozilla.org .

Testing

Assuming you have a dedicated Ubuntu system (like a VM), you can use the standalone installation script to deploy a test environment rapidly.

$ sudo apt-get install golang git

# must be >= 1.5
$ go version
go version go1.6.1 linux/amd64

$ export GOPATH=$HOME/go

$ mkdir $GOPATH

$ go get mig.ninja/mig

$ cd $GOPATH/src/mig.ninja/mig

$ bash tools/standalone_install.sh

This script will install all of the components MIG needs for a localhost only installation. Follow instructions at the end of the script to convert it to a real infrastructure, or read Installation & Configuration.