Enterprise Cloud Security Guide: Secure Application Development

This article is part of the Enterprise Cloud Security Series with Part I - Introduction which introduces the space and how it differs from on-premise security. Part II - Foundation covers the security consideration for building the cloud foundation.

This article focuses on building security in the application development process.

The security of application can, at a high level, be classified into the following:

Application is being developed with security in mind and is secure
Application development infrastructure is secure
Application is secure while running
Application runtime infrastructure is secure.

Developing a secure application requires that the application is designed and developed to be secure on a secure development platform (Reflections on Trusting Trust).

Application security in DevOps

CNCF has done a great job putting together a security white paper based on the NIST Application Security Container Guide, Center for Internet Security (CIS), NIST Security Strategies for Micro-service, and OpenSCAP. It covers the typical lifecycle of development, distribution, deployment, and runtime of applications. Due to its focus on cloud-native development, it is a bit light on some aspects like unique considerations for securing SaaS and PaaS services.

The overall approach of DevOps is to ensure that security is achieved as the following types of artifacts flow through the process.

Application — represents a collection of all the artifacts that are part of the runtime.
Configuration — consists of SaaS, PaaS, IaaS, and application configuration that drives the provisioning and runtime behavior of the application.
Code — primarily deployed to IaaS (Infrastructure as a Service), CaaS (container as a service), FaaS (Function as a Service), and PaaS (Platform as a Service) as part of the application.
Image — provides the platform on which the code runs. This typically includes VM and container images.

As part of the DevOps, the following stages form the part of the process

Develop— involves the developers writing code, configuration, and sometimes putting together the image on which the solution is going to run. Developers are also responsible for testing and may also be either responsible for or involved in threat modeling, given their understanding of the domain and technology involved.
Distribute — focuses on packaging the outcome of the development process to either use it within or share it with other teams, business units, and organizations for use. This typically involves more comprehensive testing to ensure that artifacts are “production-ready” for release. The packaged artifacts are typically pushed to the registry for staging purposes.
Registry staging — involves storing the artifacts for future use by downstream systems and users.
Deployment — focuses on setting up the application in a landing zone designated for the application.
Runtime — consists of an application and associated infrastructure that provides the service that the application was designed for.

Across these stages, the various aspects of security must be monitored and addressed to ensure that the application is “secure-by-default”. This “shift-left” approach to application security can reduce the time, effort, and cost of fixing these security issues during runtime. The following aspects of security are typically addressed as part of DevOps processes.

Application

Threat modeling — focuses on identifying, communicating, and understanding threats and mitigations within the context of protecting something of value, i.e., data and/or service. This is an important part of the development process to provide appropriate guidance for users to deploy applications securely. During the deployment phase, threat modeling is typically performed to identify all the runtime threats based on the deployment architecture.
Code review — the changes to code, configuration, and images should be reviewed to ensure that it aligns with functional and non-functional requirements and enterprise guidelines.
Testing — ensure that the code, configuration, and images are aligned with functional and non-functional (including security) requirements. Depending on the requirements, the testing would typically involve unit testing, system testing, integration testing, performance testing, and others during the entire development lifecycle. With regards to security, penetration testing is one of the typical testing performed to ensure that implementation is secure by design.
Third-party artifact analysis — looks at all the third-party artifacts like VM and container images, libraries that are pulled from external sources like maven repositories, docker container repository, and cloud marketplace and performs vulnerability analysis, signing, and publishing to enterprise registry for use by application teams.
Security policy validation — involves checking all the components that form part of the runtime environment, including code, configuration, image, network, against pre-defined enterprise policies to ensure that the application is secure after deployment.
Log Flow validation — ensures that all the application and infrastructure logs are set up correctly to flow into the SIEM and/or performance/ telemetry platform after deployment.
Resiliency validation — focuses on verifying whether the deployed application is deployed such that it is aligned with the resiliency (failover, disaster recovery) requirement identified.
Forensic validation — ensures that deployed infrastructure is set up appropriately (e.g., installation of any agent, access to the environment from forensic automation tools) to quickly perform evidence collection for forensic investigations in case of any incident.

Image

Docker file vulnerability scan — focuses on identifying the various artifacts that form part of each layer of container images and validating the vulnerability associated with the artifact.
Kubernetes manifest scan — looks for security and configuration issues with Kubernetes workload.
VM Image vulnerability scan — looks for potential points of exploits in the VM image.
Image hardening — strips unnecessary software and changes configuration to limit potential vulnerabilities that can be exploited by attackers.
Image signing and validation— adds digital fingerprint to the image, which can be tested later to verify trust.
Public registry access — needs to be managed to ensure that unapproved or insecure images or image layers are not pulled in as part of any of the phases.
Marketplace restriction — ensures that images being developed are approved or aligned with enterprise policies.
Image encryption — ensures that any intellectual property stored on the image is protected.

Code

Static Analysis (SAST) — focuses on white box testing by analyzing source code before it is compiled to find vulnerabilities that may be exploitable at runtime depending on the deployment architecture and other factors.
Dependency License Scan — reviews all the dependencies associated with code to ensure that corresponding licenses align with enterprise policies and limit exposure.
Artifact and package signing — adds digital fingerprint to packages and artifacts created as part of compilation and packaging processes.
Dependency vulnerability scan — identifies any existing exploit associated with dependencies that are being used by code.
External library access — ensures that only authorized libraries are used.
Public Repo access scan — looks for references to public and unauthorized repositories in the code, configuration, and build process.

Configuration

Infrastructure as code scan— IaaC (Infrastructure as a Code) focuses on defining (and provisioning) infrastructure using a configuration or code. The configuration (e.g., cloud formation, terraform, azure ARM, GCP deployment manager) can be scanned using predefined and custom policies to ensure that infrastructure is secure (e.g., the disk is encrypted, logs are enabled, access is configured, etc.) and configured correctly.
Runtime policy validation — is the runtime equivalent of IaaC, where the same policies are validated against the runtime to ensure that the deployed infrastructure aligns with security and business policies.
Shared secret scanning — looks for shared secrets, keys, private certificates in the registry.

DevOps platform security

Security of the DevOps platform consists of securing all the technical components, enterprise processes with the idea that resistance to developers’ flow increases as the application moves from development to production. Each enterprise has to find the right balance of the developer’s freedom with security and operational controls needed to ensure business objectives. A sample CI/CD flow is shown below:

DevOps consists of different components that need to be secured. Apply the V.L.A.D.R. criteria to structure the approach below.

Code repository/Source Control Management

Depending on the delivery model (IaaS, PaaS, or SaaS) and applicable shared responsibility, the enterprise may be responsible for managing the basic platform security aspects like technology vulnerabilities, logging, and resiliency. Code repository poses a unique set of security challenges like process vulnerabilities, access, and data which should be considered appropriately.

The code repository is a type of database since it contains business and trade secrets, algorithms that can result in a loss if exposed. Keeping that in mind, code repositories should be handled similarly to data within the enterprise. This would typically involve separating the code repository based on sensitivity. Organization policies to secure data at rest by encryption with organization-managed keys may mean that SaaS code repository can not be used for sensitive code.

Access to the code repository forms another important part of the process. Strong controls regarding user’s access should be adequately be supplemented by control over identities (e.g., API keys) and access of scripts and other automation processes (e.g., automatic key rotation, service identity for authentication). It is important to ensure that API keys and similar tokens have the same level of access control as user/service accounts (e.g., SAML authentication may enforce device authentication, which API keys may miss). All the access should be based on the principle of least privilege with, where possible, the use of UEBA (User & Entity Behavior Analysis) to continuously fine-tune the access and with periodic access reviews. Additional network access control may be required to be put in place for sensitive repositories to reduce the attack surface.

Code repository in most of the cases also stores other aspects of the DevOps (e.g., CI/CD pipeline definition, application configuration, SAST/DAST configuration, vulnerability exceptions), which can become a source of process vulnerabilities. It is important to ensure that appropriate pre-commit hooks; scanning processes are in place to detect changes to CI/CD pipeline that circumvent enterprise processes, shared secrets/API keys/private certificates in the code base, ignore false-positive vulnerability without exception, and so on. The enterprise should review the developers’ toolsets and use-cases to identify these process vulnerabilities and deploy controls to address them.

Registry

Besides basic platform security aspects like technology vulnerabilities, logging, and resiliency that enterprise may be responsible for depending on the delivery model and corresponding shared responsibility, registry-specific process vulnerabilities and access model along with data continue to be the focus.

Depending on the degree of control in place, applications may not have access to a public repository during the development and distribution phase. The repository may operate as a proxy for external dependencies like libraries, VM Images, container images, and tool plugins (e.g., approved terraform providers). Moreover, additional controls may be in place to scan the external dependencies and generated artifacts for vulnerabilities, sensitive information.

Access to the registry is another aspect that plays an important role. This access can be of significant importance when the repository operates as a proxy to public repositories. The ability to add or remove the external repository allowed can have a significant impact.

Most registries also support the ability to sign and encrypt the artifacts, which provide data-level security.

CI/CD Automation

CI/CD automation platform provides a way to execute the build process and deploy the artifacts. This platform needs to ensure that the process being executed is aligned with enterprise policies (e.g., the application should not have critical severity vulnerability before deployment in production) and deployment is being with the least privilege access. In addition to that, the build and packaging tools should be from trusted sources and publishers (including extensions like terraform provider plugins) to ensure that artifacts being generated are not compromised during the build process. Due to the involvement of secrets/keys, it is important that automation is executed on the platform with limited accessibility and with an adequate level of encryption of data at rest and in motion.

Furthermore, adequate environment hygiene (e.g., delete all the files after build, encrypt and store state files using BYOK on an alternate storage system) should be maintained during and after execution of the build and deployment process to ensure that details about the application, including secrets/service account passwords are not present on the file system after the CI/CD pipeline has been executed.

Adding security to development and operational processes in any enterprise can be a significant daunting task further exacerbated by the changes in culture within most enterprises. Maintaining this balance between developer’s choice and enterprise security posture can be a significant challenge with the rollout of DevOps within organizations.

Also published at Medium's subdomain.