MozDef: You've collected your security logs, now what?

MozDef: You've collected your security logs, now what?

Anthony Verez


Questions: averez on #airmozilla and #security

What is this presentation about?

  • Operations Security team: protects the networks, systems, services and data at Mozilla
  • How to use logs for security monitoring?
  • MozDef: The Mozilla Defense Platform
  • Better overview of our security events

Security Analysis can be painful

  • I want to extract averez from
    Aug  4 15:01:25 averez.local sudo[19300]:   averez :
    TTY=ttys004 ; PWD=/Users/averez ; USER=root ;
  • $ grep "COMMAND=/usr/bin/su" system.log | sed -n -e
    "s/.*sudo\[[0-9]*\]:   \(.*\) :.*/\1/p"

Security Analysis can be painful 2

  • Wait...
    root@averez /v/log# ls system.log*
    system.log system.log.0.gz system.log.1.gz
    system.log.2.gz system.log.3.gz system.log.4.gz
  • $ zgrep "COMMAND=/usr/bin/su" system.log* | sed -n
    -e "s/.*sudo\[[0-9]*\]:   \(.*\) :.*/\1/p"
  • Cool, now I have to use pssh, ansible, or whatnot to execute this on 1000+ systems...

Security Analysis can be painful 3

That sucks!

So, what do we want?

  • All our logs in one place
  • Scalable solution
  • FAST search
  • Send logs
  • Parsing
  • Powerful visualization
  • Alerts
  • Useful notifications
  • Easy to use

All our logs in one place

  • 5,000 logs/second
  • Categories: Network, systems, services, audit logs
  • Often syslog to central hosts

Send logs

  • Format: JSON
  • Transport: HTTP(S), AMQP(S) (RabbitMQ)
curl -XPOST http://localhost:8080/events -d '{
"category": "demo",
"hostname": "localhost",
"severity": "INFO",
"summary": "This is a test"

Parsing and log manipulation

Visualization - Kibana

Visualization - Kibana


Ratio of successful/failed LDAP auth


Visualization - Attackers == Ogres

Where are the attackers?


Red panda (Firefox) Photo by Yortw


  • Frontend log processing (uwsgi + python and RabbitMQ) and DB (Elasticsearch)
  • Vertical scaling: More ressources on a machine (cores for us)
  • Horizontal scaling: More nodes in a cluster
More data

Fast search

  • Elasticsearch distributes a search on all the nodes of the cluster