Google Kubernetes Engine – GKE Security

~ jayendrapatil
Table of Contents hide
GKE Security
Authentication and Authorization
Control Plane Security
Node Security
Network Security
Giving Pods Access to Google Cloud resources
Binary Authorization
GCP Certification Exam Practice Questions

GKE Security

Google Kubernetes Engine – GKE Security provides multiple layers of security to secure workloads including the contents of the container image, the container runtime, the cluster network, and access to the cluster API server.

Authentication and Authorization

  • Kubernetes supports two types of authentication:
    • User accounts are accounts that are known to Kubernetes but are not managed by Kubernetes
    • Service accounts are accounts that are created and managed by Kubernetes but can only be used by Kubernetes-created entities, such as pods.
  • In a GKE cluster, Kubernetes user accounts are managed by Google Cloud and can be of the following type
    • Google Account
    • Google Cloud service account
  • Once authenticated, these identities need to be authorized to create, read, update or delete Kubernetes resources.
  • Kubernetes service accounts and Google Cloud service accounts are different entities.
    • Kubernetes service accounts are part of the cluster in which they are defined and are typically used within that cluster.
    • Google Cloud service accounts are part of a Google Cloud project, and can easily be granted permissions both within clusters and to Google Cloud project clusters themselves, as well as to any Google Cloud resource using IAM.

Control Plane Security

  • In GKE, the Kubernetes control plane components are managed and maintained by Google.
  • Control plane components host the software that runs the Kubernetes control plane, including the API server, scheduler, controller manager, and the etcd database where the Kubernetes configuration is persisted.
  • By default, the control plane components use a public IP address.
  • Kubernetes API server can be protected by using authorized networks, and private clusters, which allow assigning a private IP address to the control plane and disable access on the public IP address.
  • Control plane can also be secured by doing credential rotation on a regular basis. When credential rotation is initiated, the SSL certificates and cluster certificate authority are rotated. This process is automated by GKE and also ensures that your control plane IP address rotates.

Node Security

Container-Optimized OS

  • GKE nodes, by default, use Google’s Container-Optimized OS as the operating system on which to run Kubernetes and its components.
  • Container-Optimized OS  features include
    • Locked-down firewall
    • Read-only filesystem where possible
    • Limited user accounts and disabled root login

Node upgrades

  • GKE recommends upgrading nodes on regular basis to patch the OS for security issues in the container runtime, Kubernetes itself, or the node operating system
  • GKE also allows automatic as well as manual upgrades

Protecting nodes from untrusted workloads

  • GKE Sandbox can be enabled on the cluster to isolate untrusted workloads in sandboxes on the node if the clusters run unknown or untrusted workloads.
  • GKE Sandbox is built using gVisor, an open-source project.

Securing instance metadata

  • GKE nodes run as CE instances and have access to instance metadata by default, which a Pod running on the node does not necessarily need.
  • Sensitive instance metadata paths can be locked down by disabling legacy APIs and by using metadata concealment.
  • Metadata concealment ensures that Pods running in a cluster are not able to access sensitive data by filtering requests to fields such as the kube-env

Network Security

  • Network Policies help cluster administrators and users to lock down the ingress and egress connections created to and from the Pods in a namespace
  • Network policies allow you to use tags to define the traffic flowing through the Pods.
  • MTLS for Pod to Pod communication can be enabled using the Istio service mesh

Giving Pods Access to Google Cloud resources

Workload Identity (recommended)

  • Simplest and most secure way to authorize Pods to access Google Cloud resources is with Workload Identity.
  • Workload identity allows a Kubernetes service account to run as a Google Cloud service account.
  • Pods that run as the Kubernetes service account have the permissions of the Google Cloud service account.

Node Service Account

  • Pods can authenticate to Google Cloud using the Kubernetes clusters’ service account credentials from metadata.
  • Node Service Account credentials can be reached by any Pod running in the cluster if Workload Identity is not enabled.
  • It is recommended to create and configure a custom service account that has the minimum IAM roles required by all the Pods running in the cluster.

Service Account JSON key

  • Applications can access Google Cloud resources by using the service account’s key.
  • This approach is NOT recommended because of the difficulty of securely managing account keys.
  • Each service account is assigned only the IAM roles that are needed for its paired application to operate successfully. Keeping the service account application-specific makes it easier to revoke its access in the case of a compromise without affecting other applications.
  • A JSON service account key can be created and then mounted into the Pod using a Kubernetes Secret.

Binary Authorization

  • Binary Authorization works with images deployed to GKE from Container Registry or another container image registry.
  • Binary Authorization helps ensure that internal processes that safeguard the quality and integrity of the software have successfully completed before an application is deployed to the production environment.
  • Binary Authorization provides:
    • A policy model that lets you describe the constraints under which images can be deployed
    • An attestation model that lets you define trusted authorities who can attest or verify that required processes in your environment have completed before deployment
    • A deploy-time enforcer that prevents images that violate the policy from being deployed

GCP Certification Exam Practice Questions

  • Questions are collected from Internet and the answers are marked as per my knowledge and understanding (which might differ with yours).
  • GCP services are updated everyday and both the answers and questions might be outdated soon, so research accordingly.
  • GCP exam questions are not updated to keep up the pace with GCP updates, so even if the underlying feature has changed the question might not be updated
  • Open to further feedback, discussion and correction.
  1. You are building a product on top of Google Kubernetes Engine (GKE). You have a single GKE cluster. For each of your customers, a Pod is running in that cluster, and your customers can run arbitrary code inside their Pod. You want to maximize the isolation between your customers’ Pods. What should you do?
    1. Use Binary Authorization and whitelist only the container images used by your customers’ Pods.
    2. Use the Container Analysis API to detect vulnerabilities in the containers used by your customers’ Pods.
    3. Create a GKE node pool with a sandbox type configured to gVisor. Add the parameter runtimeClassName: gvisor to the specification of your customers’ Pods.
    4. Use the cos_containerd image for your GKE nodes. Add a nodeSelector with the value cloud.google.com/gke-os-distribution: cos_containerd to the specification of your customers’ Pods.

References

Google Kubernetes Engine – Security Overview