So…What’s DevSecOps All About?!?

Disclaimer: This is a personal weblog. The opinions expressed here represent my own and not those of my employer.

Recently I have been involved with customer discussions around the topic of…

“How are organizations achieving secure software deployments in production while limiting AWS Management Console Access?”

A great opportunity to share my personal thoughts based on research and real-world experience on the topic. I think we can all agree that DevOps has been all the buzz lately and ANY successful implementation takes time and buy in from many levels of Management – in reality it’s much less about specific pieces of technology but more about specific policies, procedures and culture put in place. I did a presentation at VMworld 2018 on this very topic (too bad it wasn’t recorded).

The short answer to the question at hand: This is the panacea and most organizations are never able to fully achieve DevSecOps completely.

What is DevSecOps?

DevSecOps is the combination of cultural philosophies, practices, and tools that exploits the advances made in IT automation to achieve a state of production immutability, frequent delivery of business value, and automated enforcement of security policies. DevSecOps is achieved by integrating and automating the enforcement of preventive, detective, and responsive security controls into the CI/CD pipeline.

There are many Security Best practices in which organizations should be following. Provision AWS accounts in accordance with team-based organization. Separate environments by their access levels. Whitelist the environments, API’s, and services the pipeline is allowed to interact with. Utilize IAM roles combined with IAM policies (and CloudTrail!) to your advantage. Ask questions! What is the organizations stance on CI/CD pipeline security? Create good architectures for pipelines, for example dedicated pipeline vs. shared vs. team? The main goal at the end of the day is that you should limit the blast radius if something were to go awry.

3 Main Components of a Successful DevSecOps Strategy

• Security of the pipeline
• Security in the pipeline
• Enforcement of the pipeline

A CI/CD pipeline is simply a workload whose purpose is to deploy other workloads and the pipeline needs to be protected so that we can trust the pipeline. Where do we start to secure the pipeline? AWS Professional Services created the Cloud Adoption Framework (CAF) which focuses on several different epics within the business and technical perspectives. The Security epic within CAF focuses on answering some very important questions.

Security Epic in CAF

• Identity and Access Management (IAM) helps you integrate AWS into your identity management life cycle, and sources of authentication and authorization.
  • How are users authenticated to the pipeline and its components?  How are users authorized to the pipeline and its various components?
    • Users
    • Permissions
    • Groups
    • Conditions
    • Auditing?
    • Passwords
    • Rotation?
    • MFA?
    • Sharing?!!?
    • Roles
    • Root (NEVER USE ROOT!)

• Detective Control provides guidance to help identify potential security incidents within your AWS environment.
  • What logs are produced by the pipeline and its components?  Where do they go?  How do they get there?  What is interesting in those logs?  At what threshold are they interesting?  What do we do once we meet those thresholds? Who has console login access (break glass only?)
    • Coordinate time (clock) sources
    • Capture all logs
    • Determine audit-able events and audit record content
    • Protect the confidentiality and integrity of audit logs (Write Once Read Many – WORM)
    • Detect audit processing failures
    • Determine thresholds for warnings and alerts
    • Respond to warnings and alerts

• Infrastructure Security helps you implement control methodologies necessary to comply with best practices as well as meet industry or regulatory obligations.
  • What are the components of the pipeline?  How are those protected?  What does the network architecture look like?
    • The pipeline is a workload and needs to be treated with the same rigor as other critical infrastructure.
    • Build a pipeline factory to build pipelines from known good configurations.
    • Deploy workloads into known good environments.

• Data Protection helps you to implement appropriate safeguards that protect data in transit and at rest.
  • Where are the critical pieces of data in the pipeline (e.g. repos)?  How do we protect the confidentiality, integrity, and availability of that data?
    • Control access and permissions to the code repository
    • Trigger builds automatically (time-based or event-based)
    • Use tokenization or dummy data in non-production environments (NO PRODUCTION DATA IN RESEARCH / TEST)
    • Categorize data and enforce restrictions through pipeline
    • A pipeline configured to build Dev environment is not allowed to pull Production data from repository

• Incident Response helps you define and execute a response to security incidents.
  • What are the common incident scenarios with the pipeline?  How would we respond to them? What if a console login occurs, how is this handled?
    • Develop an Incident Response Plan for pipelines.
    • Compile a list of abuse/misuse cases.
    • Build playbooks and runbooks.
    • Actively test the ability to detect, respond, and recover from abuse/misuse cases in pipelines.

Infrastructure as Code

Utilizing Infrastructure as Code and Security as Code to perform static analysis within the pipeline as well as proper code testing (unit, integration and system). For example CloudFormation and Terraform are really good pieces of technology to take care of this! After all, the main goal is to trust the pipeline which means we must have the proper preventive, detective, and responsive security controls in place. To enforce the pipeline, humans should have increasingly fewer rights as code is promoted from Research -> Test -> Production and only the pipeline should have permission to make changes.