Configuring X509 and Azure AD authentication in the Kubernetes cluster
I am continuing my quest to configure my homelab’s Kubernetes cluster. As for now I’ve done:
Today, I want to configure authentication so that I can login to the cluster from my computer and not from one of the masters directly.
There are plenty of authentication mechanisms in Kubernetes, but I want 2 focus on 2 techniques that are discussed in the documentation: x509 client certificates and OpenId Connect. For the OpenId Connect provider, I will use Azure Active Directory.
Authentication using X509 client certificates
The documentation describes pretty well how to create a certificate for a normal user.
First, I need to generate a private/public key pair
|
1 2 |
openssl genrsa -out domk8s.key 2048 openssl req -new -key domk8s.key -out domk8s.csr |
It is important to set CN and O attribute of the CSR. CN is the name of the user and O is the group that this user will belong to.
It my case, I set the CN to dominique and the O to clusterUsers.
I then convert my public key to base64
|
1 |
cat domk8s.csr | base64 | tr -d "\n" |
Copy the output value. On a server/computer that has kubectl access, I create a CSR request using the API
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
username="dom" certInBase64="<VALUE>" cat <<EOF | kubectl apply -f - apiVersion: certificates.k8s.io/v1 kind: CertificateSigningRequest metadata: name: $username spec: groups: - system:authenticated request: $certInBase64 usages: - client auth EOF |
I can list all the CSR requests using kubectl get csr and approve my request using kubectl certificate approve dom.
Once approved, I can get my issued certificate, by querying the csr, and extract the base64 encoded certificate, under the status.certificate node.
|
1 |
kubectl get csr/dom -o json | jq -r '.status.certificate' | base64 -d > domk8s.crt |
I copy the domk8s.crt to my machine. Make sure to copy the cert < 1 hour, because there’s a cleanup process, that runs every hour, that cleans up approved, denied and pending requests so that requests don’t stay and pollute the cluster.
The certificate is valid for 365 days (1 year) as shown with openssl x509 -in domk8s.crt -text -noout.
You can also generate a certificate using the CA CSR and key located on any of the master nodes in the /etc/kubernetes/pki folder, respectively the ca.crt file and ca.key.
|
1 2 3 4 5 6 |
sudo openssl x509 -req -in domk8s.csr \ -CA /etc/kubernetes/pki/ca.crt \ -CAkey /etc/kubernetes/pki/ca.key \ -CAcreateserial \ -out domk8s.crt \ -days 36500 |
Note that I am making my certificate valid for 100 years. If you don’t set the -days parameters, the default is 30 days. Do not do this in production. You should have a way to regenerate certificates.
Setting up kubeconfig
We need to setup our kubeconfig (the file located usually in ~/.kube/config to be used with kubectl).
Add new credentials
|
1 |
kubectl config set-credentials domhomelabx509 --client-key=~/.keys/domk8s.key --client-certificate=~/.keys/domk8s.crt |
Then configure the cluster and the context
|
1 2 |
kubectl config set-cluster homelab --server=https://192.168.1.21:6443 --insecure-skip-tls-verify kubectl config set-context homelabx509 --cluster=homelab --user=domhomelabx509 |
and switch to your context using kubectl config use-context homelabx509
Creating the clusterrolebinding
A rolebinding is a mapping between a user and a role. In my case I create a clusterrolebinding so that it can be applied cluster wise. I add my user to the cluster-admin role
|
1 |
kubectl create clusterrolebinding homelab-cluster-admin --clusterrole=cluster-admin --user=dom |
Once done, I verify that I can get all the pods in all my cluster by running kubectl get pods --all-namespaces.
Authenticating using Azure Active Directory
To authenticate with Azure AD, we will use a plugin extension called kubelogin.
Creating an application for authentication
The first thing I need, to be able to authenticate using Azure AD, is an application. In the portal, navigate to the Azure Active Directory blade. In that blade, click on App Registrations. Click on the New Registration button. In the new Registration blade, I enter a name, Homelab Kubernetes, and for the redirect URI, I put http://localhost:8000. The valid redirect URIs for the kubelogin plugin are:
http://localhost:8000http://localhost:18000(used if the port 8000 is already in use)
Once in your application, Set the Application ID URI to http://homelabkubernetes. Now, time to set the secret for authentication. Go to the Certificates & secrets blade and create a new secret with expiry never. Record the password.
If you are a PowerShell user, you can do the same with the New-AzADApplication cmdlet
|
1 2 3 |
$SecureStringPassword = ConvertTo-SecureString -String "password" -AsPlainText -Force $EndDate = Get-Date -Date "2099-12-31 23:59:59Z" New-AzADApplication -DisplayName "Homelab Kubernetes" -IdentifierUris "http://homelabkubernetes" -ReplyUrls "http://localhost:8000" -Password $SecureStringPassword -EndDate $EndDate |
Install kubelogin
To install the kubelogin plugin, I will use Krew. Follow the install procedure to install Krew.
Using admin privileges, run kubectl krew install oidc-login . Once installed, use kubectl to proceed with the setup
|
1 2 3 4 |
kubectl oidc-login setup \ --oidc-issuer-url=ISSUER_URL \ --oidc-client-id=APPLICATION_ID \ --oidc-client-secret=SECRET |
where:
| Key | Description |
|---|---|
| SECRET | The secret you created early |
| APPLICATION_ID | The application ID that can be found in the application on the portal |
| ISSUER_URL | You can find the issuer URL by clicking on the endpoints button, in the overview blade, and opening the well-known url in a browser. Copy the issuer field value. |
After running the command, you will get plenty of valuable information:
- In the token claims, the sub is your user
- You can create a clusterrolebinding to give yourself cluster-admin:
1kubectl create clusterrolebinding homelab-cluster-admin --clusterrole=cluster-admin --user='ISSUER_URL#SUB'
If you already have, like me, setup the x509 authentication and created a clusterrolebinding, add the user to the role:- Get the current clusterrolebinding:
1kubectl get clusterrolebinding homelab-cluster-admin -o yaml > homelab-cluster-admin-clusterrolebding.yaml - Modify the yaml to add the user in the format ISSUER_URL#SUB
- Update the clusterrolebinding:
1cat homelab-cluster-admin-clusterrolebding.yaml | kubectl apply -f -
- Get the current clusterrolebinding:
We also need to add flags to our kube-apiserver
On each master, edit the file /etc/kubernetes/manifests/kube-apiserver.yaml and add:
|
1 2 |
- --oidc-issuer-url=ISSUER_URL - --oidc-client-id=APPLICATION_ID |
kubelet is watching this directory and will restart any kube-apiserver pods if it sees that the file has changed. You can see that by checking the pods in the kube-system named kube-apiserver-* and see that the age is not old.
I can now setup my kubeconfig
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
# create a new cluster if not there already kubectl config set-cluster homelab --server=https://192.168.1.21:6443 --insecure-skip-tls-verify # create a new context for aad kubectl config set-context homelabaad --cluster=homelab --user=domhomelabaad # create the credentials kubectl config set-credentials domhomelabaad \ --exec-api-version=client.authentication.k8s.io/v1beta1 \ --exec-command=kubectl \ --exec-arg=oidc-login \ --exec-arg=get-token \ --exec-arg=--oidc-issuer-url=ISSUER_URL \ --exec-arg=--oidc-client-id=APPLICATION_ID \ --exec-arg=--oidc-client-secret=SECRET |
Then switch your context to use your newly added Azure AD OIDC provider
kubectl config use-context homelabaad
If you’ve properly configured everything, executing
kubectl get pods --all-namespaces should return you all the pods in all namespaces.
