Managed Service for Kubernetes security
- Responsibility
- Sensitive data
- Resource model
- Network security Managed Service for Kubernetes
- Secure Managed Service for Kubernetes configuration
- Protection against malicious code in Managed Service for Kubernetes
- Vulnerability management Managed Service for Kubernetes
- Security updates
- Backup and recovery
- Security policies in Kubernetes
- Practices for securely creating and using Docker images
- Runtime protection
- Load sharing between nodes
- Collecting, monitoring, and analyzing audit logs Managed Service for Kubernetes
The section includes recommendations for Nebius AI users on security settings in Managed Service for Kubernetes.
Responsibility
The user is responsible for all actions made inside the Kubernetes node. The user is responsible for the security of the nodes and their proper setup in accordance with security standards.
Nebius AI is responsible for the Kubernetes API security.
The user is responsible for correctly choosing security settings in Managed Service for Kubernetes, including selecting the channel and the update schedule.
Sensitive data
When using Managed Service for Kubernetes to comply with security standards, it is forbidden to:
- Use sensitive data in names and descriptions of clusters, node groups, namespaces, services, and pods.
- Use sensitive data in Kubernetes node labels.
- Use sensitive data in pod manifests.
- Use sensitive data in etcd in clear text.
- Write sensitive data to Managed Service for Kubernetes logs.
Resource model
Wherever possible, ensure maximum isolation between resources:
- Use a separate organization for each "large-scale" project.
- Use a separate cloud for each development team.
- Use a separate Kubernetes cluster located in a separate folder for each service.
- Use a separate namespace for each microservice.
Your clouds must have no shared resources. Cloud members must have access only to their clouds.
Less strong isolation models are also possible, for example:
- Projects are split between different clouds.
- Development teams are assigned independent folders.
- Services have separate Kubernetes clusters.
- Microservices have independent namespaces.
Network security Managed Service for Kubernetes
Setting up incoming network access
For online endpoints, we recommend that you allocate an independent Kubernetes cluster or independent node groups (using Taints and Tolerations
To enable incoming network access to your workloads via HTTP/HTTPS, use the Ingress
Secure Managed Service for Kubernetes configuration
Secure configuration
In Managed Service for Kubernetes, the user is fully in control of all node group settings, but only partially in control of the master settings. These settings are part of the user's overall cluster security responsibility.
The CIS Kubernetes Benchmark
In Nebius AI, the Kubernetes node groups are deployed by default with the configuration that complies with CIS Kubernetes Benchmark.
The kube-bench
Here
In addition, kube-bench supports integration with Starboard Operator
Starboard Operator is a free tool that helps you automate scanning of images for vulnerabilities and checking that the configuration complies with CIS Kubernetes Benchmark.
Integrity control (FIM — File integrity monitoring)
You must control two levels of file integrity in node groups:
- OS files of the node - for example, configuration files.
- Container files - for example, critical files that the user application writes to the volume.
OS files of the node
You can use, for example, Osquery
Container files
One of the methods to solve this task:
- Use readOnlyRootFilesystem
in pods. - Make sure to mount folders to write the data to as separate volumes: as emptydir or individual disks.
If you mount folders as emptydir, files are stored on the node in the folder /var/lib/kubelet/pods/PODUID/volumes/kubernetes.ioempty-dir/VOLUMENAME
. To ensure data integrity, you can monitor this folder by Osquery as OS node files.
In the case of separate disks (not emptydir), you can mount volumes in read mode to the above-mentioned DaemonSet running Osquery.
To control file integrity on the Kubernetes nodes, you can also use the tools listed in Integrity control.
There exist dedicated free solutions for Kubernetes nodes from Google or Argus, including file-integrity-operator
Encryption in transit
For in-transit encryption, use TLS interaction between pods. If you can't use TLS interaction, use service mesh solutions:
Protection against malicious code in Managed Service for Kubernetes
To protect the containerization host levels, you can use a variety of paid and free solutions from the "Runtime security" and "Antivirus engine" classes. Examples of free solutions:
- Kubernetes ClamAV
- Sysdig Falco
(it can also function as an Intrusion Detection System)
Be sure to also use the Kubernetes built-in support for AppArmor
Vulnerability management Managed Service for Kubernetes
Nebius AI within Managed Service for Kubernetes is in charge of vulnerability management and security updates on the master. The user must independently control vulnerabilities on the Kubernetes worker nodes.
Scanning for vulnerabilities
You can break vulnerability scanning into the following levels:
- Image-level vulnerability scanning.
- Vulnerability scanning of the OS nodes in Kubernetes.
Vulnerability scanning at the image level is detailed in Protection against malicious code in Managed Service for Kubernetes.
Examples of free universal solutions for vulnerability scanning of the OS nodes in Kubernetes are given in Scanning for vulnerabilities.
There also exist both paid and free solutions for scanning the OS nodes in Kubernetes and Kubernetes hosts for vulnerabilities: for example, free tools such as kube-hunter and trivi (scan filesystem).
Security updates
Managed Service for Kubernetes issues updates in a regular manner. To meet the Information Security standards:
- Select a relevant update channel and enable either automatic installation of updates, or manual installation immediately after publication in the selected channel.
- Double-check that the ad settings meet the Information Security standards.
- Use one of the three latest Kubernetes versions, because updates (including security updates) are only released for these versions.
Backup and recovery
Set up backups in Managed Service for Kubernetes by following the guide. When storing your backups in Object Storage, follow recommendations from the Secure configuration for Object Storage.
Security policies in Kubernetes
Requirements listed in Pod Security Standards
To implement the requirements, you can either use the Kubernetes built-in Pod Security Admission Controller
Examples using Kyverno:
To control compliance with Pod Security Standards, you can also use the following tools within CI/CD:
Or a separate Kubesec
Practices for securely creating and using Docker images
Use these check lists to meet requirements for secure creation of images:
You can control Dockerfile in your CI/CD pipeline using the Conftest
Runtime protection
When using minimal images or distroless images without a shell, use ephemeral containers
Load sharing between nodes
Data loads with different security contexts (such as different severities of data processed) must be processed on different Kubernetes nodes. To enable load sharing within a cluster, use different node groups with different settings for node labels
node taints
Collecting, monitoring, and analyzing audit logs Managed Service for Kubernetes
Events available to the user in the Managed Service for Kubernetes service can be classified as levels:
- Kubernetes node events
- Kubernetes pod events
- Kubernetes metrics
- Kubernetes flow logs
Kubernetes node level
Kubernetes node level events are collected and exported similarly to collecting OS audit logs.
Kubernetes pod level
Different options for collecting and exporting pod-level events in Kubernetes is described in the Kubernetes official documentation
Managed Service for Kubernetes role model audit
In the Managed Service for Kubernetes console, you can audit the current role model used in the service. For this, go to the Access management tab in the service.
You can also use: