MozDef: You've collected your security logs, now what?

Anthony Verez

Slides: anthony-verez.fr/mozdef

Questions: averez on #airmozilla and #security

• Operations Security team: protects the networks, systems, services and data at Mozilla
• How to use logs for security monitoring?
• MozDef: The Mozilla Defense Platform
• Better overview of our security events

• I want to extract averez from

  Aug  4 15:01:25 averez.local sudo[19300]:   averez :
  TTY=ttys004 ; PWD=/Users/averez ; USER=root ;
  COMMAND=/usr/bin/su

• $ grep "COMMAND=/usr/bin/su" system.log | sed -n -e
  "s/.*sudo\[[0-9]*\]:   \(.*\) :.*/\1/p"

• Wait...

  root@averez /v/log# ls system.log*
  system.log system.log.0.gz system.log.1.gz
  system.log.2.gz system.log.3.gz system.log.4.gz

• $ zgrep "COMMAND=/usr/bin/su" system.log* | sed -n
  -e "s/.*sudo\[[0-9]*\]:   \(.*\) :.*/\1/p"

• Cool, now I have to use pssh, ansible, or whatnot to execute this on 1000+ systems...

• All our logs in one place
• Scalable solution
• FAST search
• Send logs
• Parsing
• Powerful visualization
• Alerts
• Useful notifications
• Easy to use

• 5,000 logs/second
• Categories: Network, systems, services, audit logs
• Often syslog to central hosts

• Format: JSON
• Transport: HTTP(S), AMQP(S) (RabbitMQ)

curl -XPOST http://localhost:8080/events -d '{
"category": "demo",
"hostname": "localhost",
"severity": "INFO",
"summary": "This is a test"
}'

• Python plugin system
• Modify the message before it's saved
• Example: rename fields

• Thanks to Jeff Bryner, Joe Stevensen, OpSec, IT and Mozilla
•  
• MozDef: https://github.com/jeffbryner/MozDef
•  
• Questions: averez on #security and #airmozilla
• @netantho

• Frontend log processing (uwsgi + python and RabbitMQ) and DB (Elasticsearch)
• Vertical scaling: More ressources on a machine (cores for us)
• Horizontal scaling: More nodes in a cluster

• Elasticsearch distributes a search on all the nodes of the cluster

• Simple match on a field
• Aggregation of source ip addresses