Test Driven Security in Mozilla’s infrastructures

Test Driven Security (TDS) is the process of verifying the compliance of systems, networks and services against security policies.

We use four standardized levels: low, medium, high and maximum. Each level maps to specific requirements at the system, network and service levels. General policies, such as “must monitor root user activity on servers”, are derived into specific requirements that guide the technical choices, like “on linux servers, the audit daemon must monitor system calls from the root user for processes creations”.

TDS provides a way to control security compliance in near real time, and assist operational teams in the implementation and maintenance of secure infrastructures.

Compliance checks

OpSec built two systems to implement TDS: MIG and MozDef (it is worth noting that TDS is only one of the goals that these systems are built for).

• MIG (Mozilla InvestiGator) is a live forensic platform that queries endpoints in real time for security investigations. MIG runs an agent on every system that processes requests from investigators, and stores the results in a database.
• MozDef is Mozilla Defense Platform, a SIEM built to facilitate incident response. MozDef collects and analyzes millions of events every day, from sources across the Mozilla network.

MIG runs tests on target systems, retrieves the results and converts them into “compliance item” format. MozDef downloads compliance items from MIG at regular intervals, and stores them into its database. Users visualize the compliance items in a MozDef dashboard (powered by Kibana, the dashboard interface developed by Elasticsearch).

Security tests, also called compliance checks, verify that the configuration of a system matches the requirements of a security assurance level. A simple test would be to verify that SSH root login is disabled, which can be expressed into a MIG Action using the following JSON:

{
    "name": "compliance check for openssh",
    "target": "linux",
    "threat": {
        "level": "medium",
        "family": "compliance",
        "type": "system",
        "ref": "sysmediumssh1"
    },
    "operations": [
        {
            "module": "filechecker",
            "parameters": {
                "/etc/ssh/sshd_config": {
                    "regex": {
                        "check root login is off": [
                            "(?i)^permitrootlogin no$"
                        ]
                    }
                }
            }
        }
    ],
    "syntaxversion": 1
}

Compliance checks are generic. The specific example above targets all linux systems. Compliance items, however, represent the result of one compliance check on one target system. The compliance item below shows that target server1.example.net successfully passed compliance of the SSH Root Login check.

{
    "target": "server1.example.net",
    "utctimestamp": "2014-08-07T14:04:23.96561Z",
    "compliance": true,
    "link": "https://link.to.mig.api",
    "policy": {
        "url": "https://link.to.security.policy",
        "name": "system",
        "level": "medium"
    },
    "check": {
        "test": {
            "type": "regex",
            "value": "(?i)^permitrootlogin no$"
        },
        "location": "/etc/ssh/sshd_config",
        "ref": "sysmediumssh1",
        "description": "compliance check for openssh",
        "name": "check root login is off"
    }
}

At the moment, most tests are regex tests ran against configuration files. In the future, we plan to support checks for the network, and for configurations that aren’t stored as files (such as the strength of a SSL connection, or the fact that a server is protected by a bastion host). The logic will remain the same: a test looks for a specific security requirement and returns a boolean that represents compliance.

Below is a screenshot of the MozDef dashboard for an example target. We run all compliance checks daily and on demand. Operational teams have real time visibility over the security of their systems.