Mounting a Kubernetes Secret as a single file inside a Pod

Recently I needed to mount an SSH private key used for one app to connect to another app into a running Pod, but to make sure it was done securely, we put the SSH key into a Kubernetes Secret, and then mounted the Secret into a file inside the Pod spec for a Deployment.

I wanted to document the process here because (a) I know I'm going to have to do it again and this will save me a few minutes' research, and (b) it's very slightly unintuitive (at least to me).

First I defined a secret in a namespace:

apiVersion: v1
kind: Secret
  name: ssh-key
  namespace: acme
  id_rsa: {{ secret_value_base64_encoded }}

Note the key of id_rsa for the secret data—I used this because when you mount a secret into a volume, the mount point will be a directory, and each file in that directory corresponds to a key in the Secret's data. So in this case, if I set a mount path of /var/my-app, then Kubernetes would place a file in there named id_rsa, with the value from the Secret. (Note that I'm using Ansible to template and apply manifests, so I'm actually using a value like {{ ansible_vault_encrypted_string | b64encode }}, which uses Ansible Vault to decrypt an encrypted private key in a playbook variable—though that's besides the point here).

To get that file to mount in the path /var/my-app/id_rsa, I add the volume like so in my Deployment spec:

      - image: "my-image:latest"
        name: my-app
          - mountPath: "/var/my-app"
            name: ssh-key
            readOnly: true
        - name: ssh-key
            secretName: ssh-key

Note that you can control the secrets files permissions using defaultMode in the volumes definition, or even individually per file (if there are multiple keys in the Secret's data), but that exercise is left up to the reader. See the Secrets documentation for more on that (specifically, the section on Secret files permissions).

Mounting a secret to a single file in an existing directory

One thing that is not supported, unfortunately, is mounting a single secret to a single file in a directory which already exists inside the container. This means secrets can't be mounted as files in the same way you'd do a file-as-volume-mount in Docker or mount a ConfigMap item into an existing directory. When you mount a secret to a directory (like /var/my-app in the above example), Kubernetes will mount the entire directory /var/my-app with only the contents of your secret / secretName items.

To overcome this issue, you can mount the secret somewhere else (e.g. /var/my-app-secrets), then use a postStart lifecycle hook to copy it into place:

                - /bin/sh
                - -c
                - cp /var/my-app-secrets/id_rsa /var/my-app/id_rsa

That way, existing contents of the /var/my-app directory are be preserved.


Hi, thank you for the informative tutorial.

Could you provide more information on how you are using Ansible to populate the values inside the secrets file?

This was useful, I stop by here to remind myself, because it's the simplest guide out there. Thanks!

In Linux I expect a mount to make anything in the directory be mounted onto to become unavailable. Here it looks like /var/my-app is not getting wiped out, but added to. Is that correct?

Be careful when moving secrets in containers, secret volumes are tmpfs by default, you may write them on non volatil device.
You'd better use a symlink in postStart.

You can use subPath to mount a configMap or secret key to a specific subdirectory without overwriting it's contents.

          - mountPath: "/var/my-app/id_rsa"
              subPath: id_rsa
            name: ssh-key
            readOnly: true
        - name: ssh-key
            secretName: ssh-key
              - key: id_rsa
                path: id_rsa

You can use a link instead of copying the file, so you can get your file contact updated when the secret content is modified.

<br />
        lifecycle:<br />
          postStart:<br />
            exec:<br />
              command:<br />
                - /bin/sh<br />
                - -c<br />
                - ln -sf /var/my-app-secrets/settings.json /var/my-app/settings.json<br />

What happens to id_rsa on mount? Does it decoded automatically? This my issue. It doesn't work for me. Is decoding manually done in separate command? k8s doc says it decodes on mount.

I don't think it will always work due to the fact that your entrypoint script and poststart are running async.

I just want to confirm. Does the data in the `data` field get decoded to plain text when the Secret get mounted to the pod by file or by environment variables? It seems like so in my testing. Is there a doc explaining this? Or is this a *Secret*?

Please don't mind the previous comment I made. The answer is yes. I think I read too much and become stupid. It was in the doc as well.