Cloud-deployed containers and container orchestration systems are becoming mainstream. Vulnerabilities and exploits associated with containers are gathering pace; however, the best practice of securing containers continues to be overlooked.

Containers allow for packaging and deployment of single applications. Decoupling applications or services from various parts of a target environment can provide a variety of operational and security benefits. Containers can reduce the complexity of running multiple concurrent services on a single system, especially where overlapping or conflicting dependencies exist. Due to their abstraction, containers can also help mitigate the effects of system-wide failure, ensuring uptime and helping to prevent the compromise of applications or mission-critical services.

Containers run on an abstracted layer above the host operating system. The abstraction provides a level of separation and the opportunity to apply a layered defense model. In addition to their native abstraction, containers can be configured to run in only trusted and isolated environments. This configuration provides an additional barrier of separation, enhancing security even further. However, containers are not impervious and are susceptible to various vulnerabilities and compromise mechanisms.

For example, Kubernetes is an open-source container orchestration management system that provides automation for container deployment and monitoring. In the first week of December 2018, Kubernetes released versions v1.10.11, v1.11.5, and v1.12.3 to address CVE-2018-1002105, a non-complex critical privilege escalation vulnerability that permits cluster-admin level access without requiring user interaction or special privileges. This access level allows the creation of arbitrary service instances. Threat actors could steal sensitive data, inject malicious code, or disable Kubernetes production applications and services. Unauthorized requests do not appear in audit or server logs.

In another example, car manufacturer Tesla’s Kubernetes consoles were infiltrated because they were not password protected. Within one Kubernetes pod, system access credentials were exposed to Tesla’s Amazon Web Services (AWS) environment, which contained an Amazon S3 (Amazon Simple Storage Service) bucket that held sensitive data such as vehicle telemetry. Security researchers also found threat actors cryptomining from within one of Tesla’s Kubernetes pods.

Secureworks® Counter Threat Unit™ (CTU) researchers expect many other container technology-based exploits in the future.

Container vulnerabilities

As with all software, container vulnerabilities need to be carefully managed. In July 2018, IBM addressed two vulnerabilities (CVE-2018-11756 and CVE-2018-11757) in the Apache OpenWhisk application, a component IBM uses to provide container-based services. These vulnerabilities allowed an attacker to overwrite back-end functions in the platform with arbitrary code. Risks of this type of vulnerability range from simple denial of service (DoS) to data modification, deletion, and exfiltration.

Patching containers and specific application libraries inside containers can sometimes be problematic due to the number of containers running. Containers can encourage widespread diversity in an environment.

For containers in the cloud, offerings from Microsoft Azure, Amazon Web Services, and the Google Cloud Platform can provide nightly automatic deployment of security patches. However, administrators must still reboot nodes and schedule downtime for patches to be applied.

Compromised container images

It is especially important to validate security risks if downloading and using third-party container images. Organizations should scan external container images for vulnerabilities, particularly if they are intending to run sensitive applications in the container. Standard hardening policies should be tested and validated on containers that have been externally sourced and on the host operating system running the containers. Digitally signing images to detect if image contents have changed can help with these checks, especially signing images with private keys to provide cryptographic reassurance.

In 2017, security researchers found approximately 19 malicious Docker container images stored on Docker Hub. Malicious container images can employ a compromised base system configured to perform actions ranging from simple cryptocurrency mining to attempted lateral movement and exploitation of both internal and external endpoints.

Leveraging the power of automation, container orchestration platforms can act as force multipliers for malicious actions, making compromise of containers more widespread and swift. Take for example Kubernetes, an open-source container orchestration management system that provides automation for container deployment, updates, and monitoring. Security researchers found preconfigured Kubernetes deployments running on honeypot servers originally ‘poisoned’ with malicious Docker containers and configured to mine Monero cryptocurrency.

Configuration mismanagement

Containers can be exposed due to simple configuration mismanagement. If mistakenly configured to run as a privileged elevated user on a host operating system, a compromised container could provide privileged user access back into the host or into other containers. To help limit access back into the host or other containers if a peer service or application becomes compromised, organizations should configure each container using a least privilege context. One method is to create dedicated limited user accounts and configure each container to run as a specified limited user. In addition, specific audit policies can be established for these accounts to monitor for anomalous, unexpected, or malicious actions.

Organizations should also be careful when mapping host volumes to containers. If not done carefully, mappings can provide another method of direct access to the host from a compromised container. For example, mistakenly mapping the “/dev” folder could provide easy direct access from within the container to all devices on the host.

Use of secrets

‘Secrets’ are blobs of data containing sensitive information that may need to be passed between the host and a container. Common examples of secrets are passwords, SSH private keys, SSL/TLS certificates, connection strings, and other data that should not be transmitted via clear text nor stored unencrypted.

Microsoft Windows Server 2016, including the Docker container platform running on the Microsoft Azure cloud, supports the delivery of secrets to containers at runtime. Amazon Web Services provides a Parameter Store feature for the managing secrets within containerized applications running on Amazon ECS (Elastic Container Service). The Parameter Store allows for the rotation and revocation of secrets and supports encryption at rest to further protect sensitive data. The Google Cloud Platform also supports secrets via its Cloud Kubernetes management system.

Regardless of the service employed, organizations should ensure that secrets are properly secured and centrally managed. Secrets should be transmitted only to containers that specifically need them, encrypted both in transit and at rest, accessible only to container services and applications that have been granted explicit access, and ideally provided only while those specific service tasks are running.

Conclusion

In light of the ever-growing use and complexity of containers both on-premises and within the cloud, container and cloud vendors are continually developing more robust container security solutions to better manage configuration and automation. As of this publication, IBM is focusing development on its Nabla containers, Google is focusing on improving its gVisor product, Microsoft is focusing on Azure Kubernetes and Windows Server 2019, and Amazon is focusing on its Elastic Container Service (Amazon ECS). The OpenStack Foundation also announced its open-source Kata Containers project, which will “combine the security advantages of virtual machines with the deployment and management advantages of software-based containers.” Similarly, VMWare plans security improvements to its NSX network virtualization software by combining it with the Pivotal Container Service offering.

Despite improvements from third-party vendors, securing internal, cloud-based, or hybrid containers may require organizations to augment existing security monitoring and threat detection programs. Some organizations may even need to establish new security programs for these platforms. Visibility into container security by existing controls may be lacking, especially where security controls focus on inspecting network traffic running between endpoints as opposed to inspecting applications and services running in containers. Containers deployed by orchestration allow for vulnerabilities and attack surfaces to change rapidly, thus requiring careful segmentation and instrumentation for continuous visibility and protection.

Republished from https://www.secureworks.com/blog/cloud-container-security-best-practices