Relationships between resources in Managed Service for Kubernetes
Kubernetes is an environment for managing containerized applications. Kubernetes provides mechanisms for interacting with clusters that can automate tasks like deploying, scaling, and managing applications in containers.
The main entity in the service is the Kubernetes cluster.
Kubernetes clusters consist of a master and one or more node groups. The master is responsible for managing the Kubernetes cluster. Containerized user applications are run on nodes.
The service fully controls the master and monitors the status and health of node groups. Users can manage nodes directly and configure Kubernetes clusters using the Nebius AI management console and the Managed Service for Kubernetes CLI and API.
Groups of Kubernetes nodes require internet access for downloading images and components.
Internet access can be provided in the following ways:
Kubernetes clusters in the Nebius AI infrastructure use the following resources:
|Kubernetes reserves ranges of IP addresses to use for pods and services.
|The number N includes:
* One public IP address for the NAT instance.
* A public IP address assigned to each node in the group if you use one-to-one NAT technology.
Masters are components that manage Kubernetes clusters.
They run Kubernetes control processes that include the Kubernetes API server, scheduler, and main resource controllers. The master lifecycle is managed by the service when creating or deleting a Kubernetes cluster. The master is responsible for global solutions that are run on all Kubernetes cluster nodes. These include scheduling workloads (such as containerized applications), managing the lifecycle of workloads, and scaling.
There are two types of masters that differ by their location:
- Single: A master created in one subnet.
- Highly available: A master created and distributed in three subnets. If a physical server or a server rack becomes unavailable, the highly available master remains functional.
A node group is a group of VMs in a Kubernetes cluster that have the same configuration and run the user's containers.
When creating a group of nodes, you can configure the following VM parameters:
Type and number of cores (vCPU).
Amount of memory (RAM) and disk space.
- Safe kernel parameters are isolated between pods.
- Unsafe parameters affect the operation of the pods and the node as a whole. In Managed Service for Kubernetes, you can't change unsafe kernel parameters unless their names have been explicitly specified when creating a node group.
For more information about kernel parameters, see the Kubernetes documentation.
You can create groups with different configurations in a Kubernetes cluster.
For Managed Service for Kubernetes, the
containerd platform is used as a container runtime environment.
Connecting to group nodes
You can connect to nodes in a group via SSH. Learn more in Connecting to a node via SSH.
Taints and tolerations policies
Taints are special policies applied to nodes in the group. Using taint policies, you can prohibit some pods from running on certain nodes. For example, you can specify that rendering pods can only be run on nodes with GPUs.
Advantages of taint policies:
- The policies are preserved when a node is restarted or replaced.
- The policies are applied automatically to nodes added to the group.
- The policies are automatically applied to new nodes during node group scaling.
You can only apply a taint policy to a node group when creating it.
Do not confuse Kubernetes node labels (
node_labels) managed by Managed Service for Kubernetes and taint policies.
Every taint policy consists of three parts:
<key> = <value>:<effect>
A list of available taint effects:
NO_SCHEDULE: Prohibit running new pods on group nodes (doesn't affect running pods).
PREFER_NO_SCHEDULE: Avoid running pods on group nodes if there are available resources for this in other groups.
NO_EXECUTE: Evict pods from nodes in this group to other groups, and prohibit running new pods.
Tolerations are exceptions from taint policies. Tolerations allow you to permit certain pods to run on nodes even if a taint policy prevents it.
For example, if the
key1=value1:NoSchedule taint policy is set for group nodes, you can place pods on this node using tolerations:
- key: "key1"
For system pods, tolerations permitting them to run on all available nodes are automatically set.
For more information about taints and tolerations, see the Kubernetes documentation.
A pod is a request to run one or more containers on a group node. In a Kubernetes cluster, each pod has a unique IP address so that applications do not conflict when using ports.
Containers are described in pods via JSON or YAML objects.
IP masquerade for pods
If a pod needs access to resources outside the cluster, its IP address will be replaced by the IP address of the node the pod is running on. For this purpose, the cluster uses IP masquerading.
By default, IP masquerade is enabled for the entire range of pod IP addresses.
To implement IP masquerading, the
ip-masq-agent pod is deployed on each cluster node. The settings for this pod are stored in a ConfigMap object called
ip-masq-agent. If you need to disable pod IP masquerading, for example, to access the pods over a VPN, specify the desired IP ranges in the
- <CIDR of pod IP addresses to skip masquerading>
A service is an abstraction that provides network load balancing functions. Traffic rules are configured for a group of pods united by a set of labels.
A namespace is an abstraction that logically isolates Kubernetes cluster resources and distributes quotas to them. This is useful for isolating resources of different teams and projects in a single Kubernetes cluster.
Managed Service for Kubernetes clusters use two types of service accounts:
Cloud service accounts
These accounts exist at the level of an individual folder in the cloud and can be used both by Managed Service for Kubernetes and other services.
For more information, see Access management in Managed Service for Kubernetes and Service accounts.
Kubernetes Service accounts
These accounts exist and run only at a level of an individual Managed Service for Kubernetes cluster. Kubernetes uses them for:
- To authenticate cluster API calls from applications deployed in the cluster.
- To configure access for these applications.
A number of Kubernetes service accounts are automatically created in the
kube-systemnamespace when deploying a Managed Service for Kubernetes cluster.
Kubernetes creates a token for each of these accounts. This token is used for authentication within the Kubernetes cluster that the account belongs to.
For more information, see the Kubernetes documentation.
Don't confuse cloud service accounts and Kubernetes service accounts.
In the service documentation, service account refers to a regular cloud service account unless otherwise specified.
node_labels are a mechanism for grouping nodes together in Kubernetes. You can use node labels to manage pod distribution across the nodes of a cluster. For more information, see the Kubernetes documentation.
Node labels can only be set when creating a node group. Each object can be assigned a set of node labels in the form of a
key:value pair. Each key must be unique to an object.
Node label keys can consist of two parts: an optional prefix and a name separated by a
A prefix is an optional part of a key. The prefix requirements are as follows:
- It must be a DNS subdomain, i.e., a series of DNS tags separated by periods (
- It may be up to 253 characters long.
- The last character must be followed by a
A name is a required part of a key. The naming requirements are as follows:
- It may be up to 63 characters long.
- It may contain lowercase Latin letters, numbers, hyphens, underscores, and periods.
- The first and the last characters must be a letter or number.
For managing node labels, see Managing Kubernetes cluster node labels.