Kubernetes Objects - Volume, Secret, ConfigMap

Overview

Kubernetes provides comprehensive storage solutions for managing data persistence, sensitive information, and configuration management. This document covers the essential components for data management within Kubernetes clusters.

Volume Management

Purpose

Container files are ephemeral and exist only while the container is running. When containers are deleted, these files are also removed. Each time a new container is created, container-specific files are recreated from scratch. (This represents the stateless concept)

In certain scenarios, these files need to persist beyond container lifecycle. This is where Ephemeral (Temporary) Volume concepts become relevant. (This represents the stateful concept) For example, to prevent data loss in database containers, volume structures should be implemented.

Ephemeral Volumes can be mounted to all containers within the same Pod simultaneously. Another purpose is to create shared storage areas that multiple containers in the same Pod can utilize collaboratively.

Important Note: In Ephemeral Volumes, if the Pod is deleted, all data is lost. However, if only the container is deleted and recreated, data persists as long as the Pod remains intact.

Volume Types

There are two types of Ephemeral Volumes:

emptyDir Volume

Sample YAML file to create an emptyDir volume:

emptydir.yaml

apiVersion: v1
kind: Pod
metadata:
  name: emptydir
spec:
  containers:
    - name: frontend
      image: ozguryazilimci/k8s:blue
      ports:
        - containerPort: 80
      livenessProbe:
        httpGet:
          path: /healthcheck
          port: 80
        initialDelaySeconds: 5
        periodSeconds: 5
      volumeMounts:
        - name: cache-vol # Provides connection with volume
          mountPath: /cache # Mount point within the container
    - name: sidecar
      image: busybox
      command: [ "/bin/sh" ]
      args: [ "-c", "sleep 3600" ]
      volumeMounts:
        - name: cache-vol
          mountPath: /tmp/log # Mount point within the container
  volumes:
    - name: cache-vol # First we create the volume, then we mount it to containers
      emptyDir: { }

hostPath Volume

Usage Warning: This volume type is rarely used and requires careful consideration when implemented.

While emptyDir creates volume folders within the Pod, hostPath creates volume folders on the worker node itself. Three different types are supported:

• Directory → Used for folders that already exist on the worker node
• DirectoryOrCreate → Used for existing folders or to create the folder if it doesn't exist
• FileOrCreate → Not a folder! Used for single files. If the file doesn't exist, it's created

Sample YAML file to create a hostPath volume:

hostpath.yaml

apiVersion: v1
kind: Pod
metadata:
  name: hostpath
spec:
  containers:
    - name: hostpathcontainer
      image: ozguryazilimci/k8s:blue
      ports:
        - containerPort: 80
      livenessProbe:
        httpGet:
          path: /healthcheck
          port: 80
        initialDelaySeconds: 5
        periodSeconds: 5
      volumeMounts:
        - name: directory-vol
          mountPath: /dir1
        - name: dircreate-vol
          mountPath: /cache
        - name: file-vol
          mountPath: /cache/config.json
  volumes:
    - name: directory-vol
      hostPath:
        path: /tmp
        type: Directory
    - name: dircreate-vol
      hostPath:
        path: /cache
        type: DirectoryOrCreate
    - name: file-vol
      hostPath:
        path: /cache/config.json
        type: FileOrCreate

Secret Management

Purpose

Although sensitive information (database credentials, etc.) can be stored in Environment Variables, this approach may not be ideal for security and management purposes.

The Secret object allows organizations to separate sensitive information from application object definitions in YAML files and manage them independently. It's always safer and more flexible to store sensitive data such as tokens, usernames, and passwords in Secret objects.

Creation Methods

Declarative Creation

Important: The Secret and the Pods it will be assigned to must be in the same namespace.

Eight different types of secrets can be created. Opaque is a generic type that can store almost all sensitive data.

Example secret.yaml file:

apiVersion: v1
kind: Secret
metadata:
  name: mysecret
type: Opaque
stringData: # Sensitive data is written under stringData
  db_server: db.example.com
  db_username: admin
  db_password: P@ssw0rd!

To view the data in the secret:

kubectl describe secrets <secretName>

Imperative Creation

kubectl create secret generic <secretName> --from-literal=db_server=db.example.com --from-literal=db_username=admin

⚠️ Note: generic here corresponds to Opaque type specified in YAML.

Alternative Approach: If organizations prefer not to enter sensitive data via CLI, they can store each piece of data in separate .txt files and use the --from-file=db_server=server.txt command. They can also use .json files instead of .txt files by specifying --from-file=config.json.

JSON example:

{
  "apiKey": "9bxa108d4b2212f2c30c71dfa279e1f77cc5c3b1"
}

Integration with Pods

There are two methods to transfer created Secrets to Pods:

Method 1: Volume Mount and Method 2: Environment Variables

Both methods are demonstrated in the YAML file below:

secretpodvolume.yaml

apiVersion: v1
kind: Pod
metadata:
  name: secretpodvolume
spec:
  containers:
    - name: secretcontainer
      image: ozguryazilimci/k8s:blue
      volumeMounts: # 2) We include the created secret volume in the pod
        - name: secret-vol
          mountPath: /secret # 3) The /secret folder in the application is mounted to the volume
          # Now we can access this file from within the application and read the values
  volumes: # 1) First we create the volume and include the secret in the volume
    - name: secret-vol
      secret:
        secretName: mysecret3
        # When we enter this pod with exec, we will see a secret folder under root. The file names here are "KEYs", the values inside are "VALUEs"
---
apiVersion: v1
kind: Pod
metadata:
  name: secretpodenv
spec:
  containers:
    - name: secretcontainer
      image: ozguryazilimci/k8s:blue
      env: # We can define all secrets as environment variables in the pod
        # In this method, we defined all secrets and their values individually
        - name: username
          valueFrom:
            secretKeyRef:
              name: mysecret3 # Take the value with "db_username" key from the secret named mysecret3
              key: db_username
        - name: password
          valueFrom:
            secretKeyRef:
              name: mysecret3
              key: db_password
        - name: server
          valueFrom:
            secretKeyRef:
              name: mysecret3
              key: db_server
---
apiVersion: v1
kind: Pod
metadata:
  name: secretpodenvall
spec:
  containers:
    - name: secretcontainer
      image: ozguryazilimci/k8s:blue
      envFrom: # Same as method 2, only difference is we define all secrets at once
        - secretRef:
            name: mysecret3

Viewing All Environment Variables in Pods

kubectl exec <podName> -- printenv

ConfigMap Management

Purpose

ConfigMaps operate using exactly the same logic as Secret objects. The key difference is that Secrets are stored encrypted in etcd using base64 encoding, while ConfigMaps are not encrypted and therefore should not contain sensitive data.

ConfigMaps can be defined as Volumes or Environment Variables in Pods. Since creation methods are identical to Secrets, the commands above remain valid.

Configuration

configmap.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: myconfigmap
data: # Should be entered in Key-Value format
  db_server: "db.example.com"
  database: "mydatabase"
  site.settings: | # "|" is used for multi-line writing
    color=blue
    padding:25px
---
apiVersion: v1
kind: Pod
metadata:
  name: configmappod
spec:
  containers:
    - name: configmapcontainer
      image: ozguryazilimci/k8s:blue
      env:
        - name: DB_SERVER
          valueFrom:
            configMapKeyRef:
              name: myconfigmap
              key: db_server
        - name: DATABASE
          valueFrom:
            configMapKeyRef:
              name: myconfigmap
              key: database
      volumeMounts:
        - name: config-vol
          mountPath: "/config"
          readOnly: true
  volumes:
    - name: config-vol
      configMap:
        name: myconfigmap

File-based Creation

Consider scenarios where organizations have config.qa.yaml or config.prod.json files for different environments (QA, SIT, and PROD) to be used in applications. How can ConfigMaps be created from the appropriate configuration file based on these environments in CI/CD?

config.json

{
  "name": "TestName",
  "surName": "TestSurname",
  "email": "test@testmail.com",
  "apiKey": "9bxa108d4b2212f2c30c71dfa279e1f77cc5c3b1",
  "text": [
    "test",
    "example",
    "one",
    "two",
    3,
    true
  ]
}

Creation Process

Creating a ConfigMap from config.json file via Kubectl:

If running in CI/CD and want to track logs:

# --dry-run is normally deprecated, but still used in older versions
kubectl create configmap xyzconfig --from-file ${configFile} -o yaml --dry-run | kubectl apply -f -

# New version --dry-run="client"
kubectl create configmap testconfig --from-file config.json -o yaml --dry-run="client" | kubectl apply -f -

# The "-" (dash) at the end takes the output from the first part of the pipe

When running the command above, kubectl takes the config.json file, creates ConfigMap YAML content that can be used with the kubectl apply command, and prints this content to the screen as output due to the --dry-run option.

Organizations capture the incoming output (using bash "pipe | ") and send it to their cluster with the kubectl apply command to create the ConfigMap.

If organizations want to create a ConfigMap directly from the config.json file without reading logs:

# Organizations can use many format files instead of config.json: e.g., yaml
kubectl create configmap <configName> --from-file config.json

1. Pod Integration: When creating the Pod, organizations need to transfer the values in ConfigMaps to files in folders inside Pods and store them as files using "volume" logic.

2. Define volumes and configMaps in the "volumes" section

3. In the "volumeMounts" section inside the Pod, introduce this volume to the pod
   • In the mountPath section, specify under which folder the file in the configMap will be copied inside the Pod and what its new name will be
   • With subPath, provide the name of the file in the configMap (e.g., config.json). Thus, when the Pod is created, specify "This file's name will change"

configmappod4.yaml

apiVersion: v1
kind: Pod
metadata:
  name: configmappod4
spec:
  containers:
    - name: configmapcontainer
      image: ozguryazilimci/k8s:blue
      volumeMounts:
        - name: config-vol
          mountPath: "/config/newconfig.json"
          subPath: "config.json"
          readOnly: true
  volumes:
    - name: config-vol
      configMap:
        name: test-config # This ConfigMap contains the config.json file

Alternative Volume Definition Syntax

Organizations can also write the volumes section in a different way:

configmappod.yaml

apiVersion: v1
kind: Pod
metadata:
  name: configmappod
spec:
  containers:
    - name: configmapcontainer
      image: ozguryazilimci/k8s:blue
      volumeMounts:
        - name: config-vol
          mountPath: "/config/newconfig.json"
          subPath: "config.json"
          readOnly: true
  volumes:
    - name: config-vol
      projected:
        sources:
          - configMap:
              name: test-config
              items:
                - key: config.json
                  path: config.json

References

• Kubernetes Official Volume Overview
• Kubernetes Official Secrets Overview
• Kubernetes Official Configmaps Overview
• GitHub - Aytitech K8sFundamentals - Volume
• GitHub - Aytitech K8sFundamentals - Secret/Configmap