Like many organizations, the Dow Jones financial media company has faced challenges in folding automated security checks into its continuous integration and delivery pipeline, a practice known as “DevSecOps.” So the company has developed a number of open source tools for automation, as well as some management practices, to check the security of new code without disrupting developer flow.
“Security tooling has been notoriously non-CI/CD friendly,” said Jay Kelath, Dow Jones’ director of product security, speaking at the All Things Open conference, held this week in Raleigh, North Carolina.
The goal of the two-member product security team is to make Dow Jones’ 800 developers as productive as possible, namely by automating as much of the low-level grunt security work that comes with software development, said Pranavkumar Patel, security architect from the Dow Jones security team.
To this end, they have developed two applications that automate a number of different security tests, so they can be folded into the company’s continuous integration and deployment pipelines, reducing the time it takes to execute the time and effort it took to run these checks by hand.
The security team decided to build the software themselves after reviewing the offerings of DevSecOp tool vendors and finding them wanting, in terms of either coverage and/or automation. So they built, and open sourced, their own.
As the publisher of such titles as The Wall Street Journal, Barron’s, MarketWatch, and the Dow Jones News Service, Dow Jones has relied on open source at least since 2012. That was when the company started using Node.js. It has since expanded into using React and Express.js for web development, and Terraform for managing cloud infrastructure, said Vishaal Patil, Dow Jones’ director of engineering. The company recently has become an open source contributor as well.
The first tool, Hammer, came about as a result of a publicized security breach a year ago in which millions of user accounts were compromised due to misconfigured permissions settings on an Amazon Web Services S3 bucket. It was a stupid mistake, the team now admits, and so it led for software, called Hammer, that checked, quite literally, for such mistakes, such as simple configuration errors. It can even “auto-fix” some errors, and offers easily-actionable alerts for the others.
The software itself is a number of AWS Lambda functions that scan Dow Jones code and data in AWS S3 buckets for patterns of misconfiguration of public Docker instances such as open ports, incorrect permission settings, incorrect IP numbers, exposed keys. A vulnerability can come from something simple as a typo. “These are real issues that can create a lot of incidents across the industry,” Patel said.
Tentatively called Braavos, the second tool is used to automate security checks within the deployment pipeline, Kelath noted. The idea with this package is to minimize the disruption that security checks take in the developer workflow.
Initially, the Dow Jones development pipeline was populated with a comprehensive array of security checking tools that are all triggered at various places along the pipeline, such as during the build phase. Developers complained about them, however, because they set off false positives and otherwise just brought the dev pipeline to a halt with additional demands. So instead, the security team looked for a way to quietly slip these tools into the pipeline, “Trojan Horse-style,” Kelath said.
As an alternative, the team bundled a set of the most important tools and rule-sets into a single container. In this way, “the developer doesn’t really interact with the security tools. They’re not testers,” Kelath said. The team wanted to reduce the time it would take to run all the security checks to 10 minutes or less. So they reduced the number of checking tools to those things that “we just know we are bad at,” such as cross-side scripting, SQL-injection issues, he noted. Other issues can be addressed outside the pipeline.
“Don’t try to boil the ocean. Just look at two or three big issues you really want to drive down in your organization,” Kelath said. Currently, the check process for the production pipeline takes about 53 minutes to complete, with the security portion taking about five minutes to complete.
For Dow Jones, this group of tools included CheckMarx (static code analysis), Git-Secret (a private data repository), NPM-Audit (security audits), Burp (dynamic web vulnerability scanner), and SonarCube quality management software.
In particular, false positives can be a particular challenge for CI/CD pipelines, though there are a number of things that can be done to lower them, Kelath noted. A number of tools are designed simply for reducing false positives, such as DefectDoJo, from The Open Web Application Security Project.
The team had a number of additional advisements for others trying to insert security into their pipelines. One is to alert developers of a bug while they are working in that stage of development. It is more difficult, and generates less interest on the part of developers, to fix a bug after the code has moved on. Also, the security checks should be as fully integrated into the developer’s own work environment as possible.
“As soon as you tell developers to go into a security tool and review the results, you have completely lost them,” Kelath said.
All Things Open assisted in travel costs to cover this event.
Feature image (left to right): Dow Jones’ Vishaal Patil, Pranavkumar Patel, Jay Kelath.
The post How Dow Jones Folded Security into its Automated Dev Pipeline appeared first on The New Stack.