We’re excited to introduce our new open-source tool for facilitating continuous vulnerability scanning in the cloud: NessAWS. If you’ve read other posts on this blog, you’ll notice that we won’t be talking about the dark web, the latest PII dump, or anything related to threat intelligence or incident response. The infrastructure team at Terbium is responsible for maintaining the security posture for the company. It’s our job to focus on all of information security, including asset inventory, secure configuration, auditing, implementing least-privilege, etc.
While developing our information security strategy, we thought a lot about continuous monitoring – what it means to us, how to implement it in our environment, and how we would use the data to support risk-based decisions. This post will describe our dynamic infrastructure environment, our approach to continuous vulnerability monitoring, justification for the need of a tool, and a brief overview of the design and implementation of NessAWS.
THE CHALLENGES OF DYNAMIC INFRASTRUCTURE
NessAWS, as the name implies, is meant for orchestrating vulnerability scans against multiple Amazon Web Services (AWS) accounts/regions. Almost all our company’s infrastructure runs on AWS, including load balancers, API servers, internal tools, databases, etc. We have multiple AWS accounts for development, staging, and production environments. Did I also mention that Terbium operates a massive dark web crawler? We use EC2 Spot Fleets, which are discounted compute resources (from now on I’ll refer to these as “instances”) that fluctuate in price based on availability. This infrastructure type is perfect for our distributed microservices deployment, since losing an instance in the middle of processing will not interrupt our pipeline. Spot fleets also allow us to define a max price that we are willing to pay for instances so we don’t break the bank. For systems that require high-availability, we use EC2 Auto Scaling Groups, which can automatically scale up or down instances based on metrics and thresholds that we define.
While these services provide considerable benefit from an availability and cost perspective, they present challenges from a security perspective. The number of instances we have fluctuates drastically based on the amount of work in our pipeline and the market availability of instances. Thus, we have instances constantly spinning up and being terminated into which we want visibility. We regularly fluctuate through at least 2,000 unique instances per day, which isn’t a lot compared to big companies like Netflix, but enough to require automation.
SELECTING A VULNERABILITY SCANNER
As the first half of the name implies, NessAWS is integrated with Tenable’s Nessus Vulnerability Scanner. The reasons for choosing to utilize Nessus are varied and biased: we had experience working with this scanner, we had a leftover Professional license from a previous project, and we knew there was a well-documented API that could be used to automate scans. We tried other solutions such as AWS Inspector, but we were unhappy with the results.
DESIGN AND DEVELOPMENT
With our decision to use Nessus Professional instead of an agent-based scanning approach, we knew there were a few constraints to consider. For example, you must submit a penetration testing request and wait for AWS to approve it before launching scans. AWS provides a web form that you must fill out while signed into your root account. The form requires you to detail the instance ID’s you wish to scan, IP addresses of the source of scans and destination instances, the timeframe you will conduct the scan, and other miscellaneous information. This was an additional process we wanted to automate, but knew from experience that requests can take 24-72 hours or sometimes require a second submittal to be approved. Thus, we needed a flexible way to submit the penetration test request and another process to perform the scans.
We also needed a method for explicitly defining which instances to scan. AWS supports associating arbitrary tags to instances. Tags allow you to categorize your resources in a key-value manner. Commonly used tags are name, description, purpose, or environment. We concluded that a tag for NessAWS would need to contain the name of the Nessus scan to execute on this instance and an additional value that could be specified to scan “classes” of instances. These classes could be subgroups of instances that we want to scan, such as webservers, databases, API servers, etc. In our implementation, we took this a step further by defining the risk level associated with each instance. For our high-risk instances (publicly-exposed, high value) we tag these as “daily” indicating that we want them to be scanned accordingly. We also have tags “weekly” and “monthly” for moderate and low risk instances. AWS makes it easy to propagate these tags to ephemeral instances using built-in services like Launch Configurations. There are also other free & open-source tools to tag instances, so you should only need