Kubernetes volumes provide a way for containers in a Pod to access and share data via the filesystem. There are different kinds of volume that you can use for different purposes, such as:
Data sharing can be between different local processes within a container, or between different containers, or between Pods.
Data persistence: On-disk files in a container are ephemeral, which presents some problems for non-trivial applications when running in containers. One problem occurs when a container crashes or is stopped; the container state is not saved, so all of the files that were created or modified during the lifetime of the container are lost. After a crash, kubelet restarts the container with a clean state.
Shared storage: Another problem occurs when multiple containers are running in a Pod and
need to share files. It can be challenging to set up
and access a shared filesystem across all of the containers.
The Kubernetes volume abstraction can help you to solve both of these problems.
Before you learn about volumes, PersistentVolumes, and PersistentVolumeClaim, you should read up about Pods and make sure that you understand how Kubernetes uses Pods to run containers.
Kubernetes supports many types of volumes. A Pod can use any number of volume types simultaneously. Ephemeral volume types have a lifetime linked to a specific Pod, but persistent volumes exist beyond the lifetime of any individual Pod. When a Pod ceases to exist, Kubernetes destroys ephemeral volumes; however, Kubernetes does not destroy persistent volumes. For any kind of volume in a given Pod, data is preserved across container restarts.
At its core, a volume is a directory, possibly with some data in it, which is accessible to the containers in a pod. How that directory comes to be, the medium that backs it, and the contents of it are determined by the particular volume type used.
To use a volume, specify the volumes to provide for the Pod in .spec.volumes
and declare where to mount those volumes into containers in .spec.containers[*].volumeMounts.
When a Pod is launched, a process in the container sees a filesystem view composed from the initial contents of the container image, plus volumes (if defined) mounted inside the container. The process sees a root filesystem that initially matches the contents of the container image. Any writes to within that filesystem hierarchy, if allowed, affect what that process views when it performs a subsequent filesystem access. Volumes are mounted at specified paths within the container filesystem. For each container defined within a Pod, you must independently specify where to mount each volume that the container uses.
Volumes cannot mount within other volumes (but see Using subPath for a related mechanism). Also, a volume cannot contain a hard link to anything in a different volume.
Kubernetes supports the following types of volumes.
See Volume Types for details of every volume type, including deprecated and removed types.
Kubernetes creates these volumes when the Pod starts on a node, then deletes them when the Pod is removed.
You can only use them in a Pod.
configMap: mounts the
data from a ConfigMap in a
read-only directory.downwardAPI: exposes
information about the Pod, such as its
labels, through the
downward API, in a
read-only directory.emptyDir: empty scratch
space that is initialized when the Pod is assigned to a
node, and is deleted when the
Pod is removed from that node.image: the files from a
container image or artifact,
mounted read-only.projected: combines
several sources, such as a Secret and a ConfigMap, into one directory.secret: mounts the data from
a Secret in a read-only
directory.These volumes connect to storage that exists outside the Pod. Their data persist after the Pod is removed.
You can use hostPath, nfs, iscsi, and fc in a Pod or in a PersistentVolume.
fc: a Fibre Channel
block storage volume.hostPath: a file or
directory from the node's filesystem.iscsi: an iSCSI
volume.local: a disk, partition or
directory on the node's filesystem.nfs: an NFS share.local in a PersistentVolume.These volumes request storage from a PersistentVolumeClaim.
You can only use these types in a Pod.
persistentVolumeClaim:
durable storage that you request through a PersistentVolumeClaim.ephemeral:
a PersistentVolumeClaim that Kubernetes creates and deletes with the Pod.The PersistentVolumeClaim binds to a PersistentVolume, which provides the underlying storage. A PersistentVolume can use these non-legacy types.
csi is the recommended type. It supports provisioning, snapshots, and cloning.
csi: storage from an external driver.fc: Fibre Channel block storage.hostPath: path local to node. Use local instead.iscsi: iSCSI (SCSI over IP) storage.local: device mounted locally to node.nfs: NFS share.These types use out-of-tree volume plugins. These are plugins developed by storage vendors outside of the Kubernetes code base.
You can use these types in a Pod or in a PersistentVolume.
For details, see Out-of-tree volume plugins.
csi: a driver that implements the
Container Storage Interface
(CSI).flexVolume (deprecated): an older, exec-based plugin
interface.csi only works for drivers that support ephemeral volumes.Sometimes, it is useful to share one volume for multiple uses in a single Pod.
The volumeMounts[*].subPath property specifies a sub-path inside the referenced volume
instead of its root.
The following example shows how to configure a Pod with a LAMP stack (Linux, Apache, MySQL, PHP)
using a single, shared volume. This sample subPath configuration is not recommended
for production use.
The PHP application's code and assets map to the volume's html folder and
the MySQL database is stored in the volume's mysql folder. For example:
apiVersion: v1
kind: Pod
metadata:
name: my-lamp-site
spec:
containers:
- name: mysql
image: mysql
env:
- name: MYSQL_ROOT_PASSWORD
value: "rootpasswd"
volumeMounts:
- mountPath: /var/lib/mysql
name: site-data
subPath: mysql
- name: php
image: php:7.0-apache
volumeMounts:
- mountPath: /var/www/html
name: site-data
subPath: html
volumes:
- name: site-data
persistentVolumeClaim:
claimName: my-lamp-site-data
Use the subPathExpr field to construct subPath directory names from
downward API environment variables.
The subPath and subPathExpr properties are mutually exclusive.
In this example, a Pod uses subPathExpr to create a directory pod1 within
the hostPath volume /var/log/pods.
The hostPath volume takes the Pod name from the downwardAPI.
The host directory /var/log/pods/pod1 is mounted at /logs in the container.
apiVersion: v1
kind: Pod
metadata:
name: pod1
spec:
containers:
- name: container1
env:
- name: POD_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.name
image: busybox:1.28
command: [ "sh", "-c", "while [ true ]; do echo 'Hello'; sleep 10; done | tee -a /logs/hello.txt" ]
volumeMounts:
- name: workdir1
mountPath: /logs
# The variable expansion uses round brackets (not curly brackets).
subPathExpr: $(POD_NAME)
restartPolicy: Never
volumes:
- name: workdir1
hostPath:
path: /var/log/pods
The storage medium (such as Disk or SSD) of an emptyDir volume is determined by the
medium of the filesystem holding the kubelet root dir (typically
/var/lib/kubelet). There is no limit on how much space an emptyDir or
hostPath volume can consume, and no isolation between containers or
Pods.
To learn about requesting space using a resource specification, see how to manage resources.
The out-of-tree volume plugins include Container Storage Interface (CSI), and also FlexVolume (which is deprecated). These plugins enable storage vendors to create custom storage plugins without adding their plugin source code to the Kubernetes repository.
Previously, all volume plugins were "in-tree". The "in-tree" plugins were built, linked, compiled, and shipped with the core Kubernetes binaries. This meant that adding a new storage system to Kubernetes (a volume plugin) required checking code into the core Kubernetes code repository.
Both CSI and FlexVolume allow volume plugins to be developed independently of the Kubernetes code base, and deployed (installed) on Kubernetes clusters as extensions.
For storage vendors looking to create an out-of-tree volume plugin, please refer to the volume plugin FAQ.
Container Storage Interface (CSI) defines a standard interface for container orchestration systems (like Kubernetes) to expose arbitrary storage systems to their container workloads.
Please read the CSI design proposal for more information.
Once a CSI-compatible volume driver is deployed on a Kubernetes cluster, users
may use the csi volume type to attach or mount the volumes exposed by the
CSI driver.
A csi volume can be used in a Pod in three different ways:
The following fields are available to storage administrators to configure a CSI persistent volume:
driver: A string value that specifies the name of the volume driver to use.
This value must correspond to the value returned in the GetPluginInfoResponse
by the CSI driver as defined in the
CSI spec.
It is used by Kubernetes to identify which CSI driver to call out to, and by
CSI driver components to identify which PV objects belong to the CSI driver.volumeHandle: A string value that uniquely identifies the volume. This value
must correspond to the value returned in the volume.id field of the
CreateVolumeResponse by the CSI driver as defined in the
CSI spec.
The value is passed as volume_id in all calls to the CSI volume driver when
referencing the volume.readOnly: An optional boolean value indicating whether the volume is to be
"ControllerPublished" (attached) as read-only. Default is false. This value is passed
to the CSI driver via the readonly field in the ControllerPublishVolumeRequest.fsType: If the PV's VolumeMode is Filesystem, then this field may be used
to specify the filesystem that should be used to mount the volume. If the
volume has not been formatted and formatting is supported, this value will be
used to format the volume.
This value is passed to the CSI driver via the VolumeCapability field of
ControllerPublishVolumeRequest, NodeStageVolumeRequest, and
NodePublishVolumeRequest.volumeAttributes: A map of string to string that specifies static properties
of a volume. This map must correspond to the map returned in the
volume.attributes field of the CreateVolumeResponse by the CSI driver as
defined in the CSI spec.
The map is passed to the CSI driver via the volume_context field in the
ControllerPublishVolumeRequest, NodeStageVolumeRequest, and
NodePublishVolumeRequest.controllerPublishSecretRef: A reference to the secret object containing
sensitive information to pass to the CSI driver to complete the CSI
ControllerPublishVolume and ControllerUnpublishVolume calls. This field is
optional, and may be empty if no secret is required. If the Secret
contains more than one secret, all secrets are passed.nodeExpandSecretRef: A reference to the secret containing sensitive
information to pass to the CSI driver to complete the CSI
NodeExpandVolume call. This field is optional and may be empty if no
secret is required. If the object contains more than one secret, all
secrets are passed. When you have configured secret data for node-initiated
volume expansion, the kubelet passes that data via the NodeExpandVolume()
call to the CSI driver. All supported versions of Kubernetes offer the
nodeExpandSecretRef field, and have it available by default. Kubernetes releases
prior to v1.25 did not include this support.CSINodeExpandSecret for each kube-apiserver and for the kubelet on every
node. Since Kubernetes version 1.27, this feature has been enabled by default
and no explicit enablement of the feature gate is required.
You must also be using a CSI driver that supports or requires secret data during
node-initiated storage resize operations.nodePublishSecretRef: A reference to the secret object containing
sensitive information to pass to the CSI driver to complete the CSI
NodePublishVolume call. This field is optional and may be empty if no
secret is required. If the secret object contains more than one secret, all
secrets are passed.nodeStageSecretRef: A reference to the secret object containing
sensitive information to pass to the CSI driver to complete the CSI
NodeStageVolume call. This field is optional and may be empty if no secret
is required. If the Secret contains more than one secret, all secrets
are passed.Vendors with external CSI drivers can implement raw block volume support in Kubernetes workloads.
You can set up your PersistentVolume/PersistentVolumeClaim with raw block volume support as usual, without any CSI-specific changes.
You can directly configure CSI volumes within the Pod specification. Volumes specified in this way are ephemeral and do not persist across Pod restarts. See Ephemeral Volumes for more information.
For more information on how to develop a CSI driver, refer to the kubernetes-csi documentation
CSI node plugins need to perform various privileged operations like scanning of disk devices and mounting of file systems. These operations differ for each host operating system. For Linux worker nodes, containerized CSI node plugins are typically deployed as privileged containers. For Windows worker nodes, privileged operations for containerized CSI node plugins are supported using csi-proxy, a community-managed, stand-alone binary that needs to be pre-installed on each Windows node.
For more details, refer to the deployment guide of the CSI plugin you wish to deploy.
The CSIMigration feature directs operations against existing in-tree
plugins to corresponding CSI plugins (which are expected to be installed and configured).
As a result, operators do not have to make any
configuration changes to existing Storage Classes, PersistentVolumes, or PersistentVolumeClaims
(referring to in-tree plugins) when transitioning to a CSI driver that supersedes an in-tree plugin.
Existing PVs created by an in-tree volume plugin can still be used in the future without any configuration changes, even after the migration to CSI is completed for that volume type, and even after you upgrade to a version of Kubernetes that doesn't have compiled-in support for that kind of storage.
As part of that migration, you - or another cluster administrator - must have installed and configured the appropriate CSI driver for that storage. The core of Kubernetes does not install that software for you.
After that migration, you can also define new PVCs and PVs that refer to the legacy, built-in storage integrations. Provided you have the appropriate CSI driver installed and configured, the PV creation continues to work, even for brand-new volumes. The actual storage management now happens through the CSI driver.
The operations and features that are supported include: provisioning/delete, attach/detach, mount/unmount, and resizing of volumes.
In-tree plugins that support CSIMigration and have a corresponding CSI driver implemented
are listed in Removed volume types.
FlexVolume is an out-of-tree plugin interface that uses an exec-based model to interface with storage drivers. The FlexVolume driver binaries must be installed in a pre-defined volume plugin path on each node, and in some cases, the control plane nodes as well.
Pods interact with FlexVolume drivers through the flexVolume in-tree volume plugin.
The following FlexVolume plugins, deployed as PowerShell scripts on the host, support Windows nodes:
FlexVolume is deprecated. Using an out-of-tree CSI driver is the recommended way to integrate external storage with Kubernetes.
Maintainers of the FlexVolume driver should implement a CSI Driver and help migrate users of FlexVolume drivers to CSI. Users of FlexVolume should move their workloads to use the equivalent CSI Driver.
hostPath
or memory-backed emptyDir volumes. See
Kubernetes issue #95049
for more context.Mount propagation allows for sharing volumes mounted by a container to other containers in the same Pod, or even to other Pods on the same node.
Mount propagation of a volume is controlled by the mountPropagation field
in containers[*].volumeMounts. Its values are:
None - This volume mount will not receive any subsequent mounts
that are mounted to this volume or any of its subdirectories by the host.
In a similar fashion, no mounts created by the container will be visible on
the host. This is the default mode.
This mode is equal to rprivate mount propagation as described in
mount(8)
However, the CRI runtime may choose rslave mount propagation (i.e.,
HostToContainer) when rprivate propagation is not applicable.
cri-dockerd (Docker) is known to choose rslave mount propagation when the
mount source contains the Docker daemon's root directory (/var/lib/docker).
HostToContainer - This volume mount will receive all subsequent mounts
that are mounted to this volume or any of its subdirectories.
In other words, if the host mounts anything inside the volume mount, the container will see it mounted there.
Similarly, if any Pod with Bidirectional mount propagation to the same
volume mounts anything there, the container with HostToContainer mount
propagation will see it.
This mode is equal to rslave mount propagation as described in the
mount(8)
Bidirectional - This volume mount behaves the same as the HostToContainer mount.
In addition, all volume mounts created by the container will be propagated
back to the host and to all containers of all Pods that use the same volume.
A typical use case for this mode is a Pod with a FlexVolume or CSI driver, or
a Pod that needs to mount something on the host using a hostPath volume.
This mode is equal to rshared mount propagation as described in the
mount(8)
Bidirectional mount propagation can be dangerous. It can damage
the host operating system, and therefore, it is allowed only in privileged
containers. Familiarity with Linux kernel behavior is strongly recommended.
In addition, any volume mounts created by containers in Pods must be destroyed
(unmounted) by the containers on termination.A mount can be made read-only by setting the .spec.containers[*].volumeMounts[*].readOnly
field to true.
This does not make the volume itself read-only, but that specific container will
not be able to write to it.
Other containers in the Pod may mount the same volume as read-write.
On Linux, read-only mounts are not recursively read-only by default.
For example, consider a Pod that mounts the hosts /mnt as a hostPath volume. If
there is another filesystem mounted read-write on /mnt/<SUBMOUNT> (such as tmpfs,
NFS, or USB storage), the volume mounted into the container(s) will also have a writeable
/mnt/<SUBMOUNT>, even if the mount itself was specified as read-only.
This is a stable feature in Kubernetes, and has been since version v1.33. It was first available in the v1.30 release. You can no longer disable or opt out of this feature or behavior (it is locked); if you explicitly set a value for the associated feature gate RecursiveReadOnlyMounts, Kubernetes ignores it but does not report any error.
Recursive read-only mounts can be enabled by setting the
RecursiveReadOnlyMounts feature gate
for kubelet and kube-apiserver, and setting the .spec.containers[*].volumeMounts[*].recursiveReadOnly
field for a Pod.
The allowed values are:
Disabled (default): no effect.
Enabled: makes the mount recursively read-only.
Needs all the following requirements to be satisfied:
readOnly is set to truemountPropagation is unset, or set to NoneIt will fail if any of these is not true.
IfPossible: attempts to apply Enabled, and falls back to Disabled
if the feature is not supported by the kernel or the runtime class.
Example:
apiVersion: v1
kind: Pod
metadata:
name: rro
spec:
volumes:
- name: mnt
hostPath:
# tmpfs is mounted on /mnt/tmpfs
path: /mnt
containers:
- name: busybox
image: busybox
args: ["sleep", "infinity"]
volumeMounts:
# /mnt-rro/tmpfs is not writable
- name: mnt
mountPath: /mnt-rro
readOnly: true
mountPropagation: None
recursiveReadOnly: Enabled
# /mnt-ro/tmpfs is writable
- name: mnt
mountPath: /mnt-ro
readOnly: true
# /mnt-rw/tmpfs is writable
- name: mnt
mountPath: /mnt-rw
When this property is recognized by kubelet and kube-apiserver,
the .status.containerStatuses[*].volumeMounts[*].recursiveReadOnly field is set to either
Enabled or Disabled.
Volume file group ownership (GID) is controlled by the pod's spec.securityContext.fsGroup.
For detailed configuration steps, refer to Configure a Security Context for a Pod or Container.
To use this feature, you (or a cluster administrator) will need to enable the AtomicWriteVolumeUserFields feature gate for all relevant components in your cluster.
See Enable Or Disable Feature Gates for more information.
Setting the AtomicWriteVolumeUserFields feature gate
enables file user ownership (UID) fields of configMap, secret, downwardAPI and projected volumes.
When defaultUser is specified at the volume level, it sets the owner UID for all its data files at creation time.
At the item level, the user field controls owner UID of an individual file and takes precedence over defaultUser.
Example:
apiVersion: v1
kind: Pod
metadata:
name: volume-user-fields-example
spec:
containers:
- name: test
image: busybox:1.28
command: ['sh', '-c', 'echo "The app is running!" && tail -f /dev/null']
volumeMounts:
- name: volA
mountPath: /mnt/volA
- name: volB
mountPath: /mnt/volB
- name: volC
mountPath: /mnt/volC
volumes:
- name: volA
configMap:
defaultUser: 1000
name: cm1
items:
- key: foo # Owner=defaultUser
path: foo
- key: bar # Owner=user
path: bar
user: 1001
- name: volB
secret: # Owner=defaultUser
defaultUser: 1000
secretName: secret1
- name: volC
projected:
sources:
- secret:
name: secret2
items:
- key: moo # Owner=root
path: moo
- key: baa # Owner=user
path: baa
user: 1000
The following container runtimes are known to support recursive read-only mounts.
CRI-level:
OCI-level:
To use this feature, you (or a cluster administrator) will need to enable the VolumeBindMountOptions feature gate for all relevant components in your cluster.
See Enable Or Disable Feature Gates for more information.
The .spec.containers[*].volumeMounts[*].bindMountOptions field lets you apply security-related
Linux bind mount flags to any volume mount. The allowed values are:
noexec - prevents execution of binaries on the mounted volumenodev - ignores device special files on the mounted volumenosuid - ignores set-user-identifier or set-group-identifier bits on the mounted volumeThese options apply per container, so different containers in the same Pod can mount the same volume with different bind mount options. The field is not supported with image volumes.
mount_options
field in the CRI Mount message. If the runtime does not advertise support, the kubelet
rejects Pods that use bindMountOptions. This field has no effect on Windows nodes.Follow an example of deploying WordPress and MySQL with Persistent Volumes.
Items on this page refer to third party products or projects that provide functionality required by Kubernetes. The Kubernetes project authors aren't responsible for those third-party products or projects. See the CNCF website guidelines for more details.
You should read the content guide before proposing a change that adds an extra third-party link.