Running Azure Pipeline Agents on Gigahatch Managed Kubernetes

In this guide we will show you how you can run self-hosted Azure Pipeline agents on the cheap and with minimal maintenance using Gigahatch Managed Kubernetes, starting at just 4.87€ per month.

Table of Contents

• Prerequisites
• Creating the cluster
• Setting up kubectl
• Using kubectl
• Preparing Azure DevOps
• Creating the agents
• Docker
• Deploying the agents
• Configuring the pipeline
• Conclusion

Prerequisites

To follow along, you will need the following:

• A Gigahatch Managed Kubernetes account.
• The kubectl cli.
• An Azure DevOps account with permissions to manage agent pools.

Creating the cluster

To start with, we need to create a cluster to run our agents. If you already have one, you can skip this step.

1. Go to your Gigahatch Managed Kubernetes organisation.
2. Click Create cluster.
3. Give the cluster a name, like AZP Agents and choose a node size. For this example we will be using one replica of the smallest SharedI1 with 2 CPUs and 4GB RAM. If you need more power for your agents, or you would like to run multiple of them, you can use larger node sizes.
4. Click Create Cluster and wait a few minutes for your cluster to be created.

Setting up kubectl

After your cluster has finished creating, click the Get Kubeconfig button. This will download a yaml file that you need to connect to your cluster. Save it somewhere secure and open your terminal. Now you need to tell your kubectl cli where to find this file. The easiest way is to set it as the KUBECONFIG environment variable.

In bash this looks like so:

export KUBECONFIG='PATH_TO_YAML'

In powershell this looks like so:

$env:KUBECONFIG = "PATH_TO_YAML"
# Or if its in the current directory:
$env:KUBECONFIG = "$(Get-Location)\<NAME_OF_YAML>.yaml"

If you would rather not use the environment variable, you can pass it as the --kubeconfig flag to the kubectl cli. For example:

kubectl --kubeconfig 'PATH_TO_YAML' get nodes

Using kubectl

To check if you can access the cluster, run:

kubectl get nodes

You should now get a list of nodes like so:

NAME                                                       STATUS   ROLES                       AGE     VERSION
41488847-47e9-4f03-90df-7fb5f7f8e15f-l2ljs-qx8mf           Ready    <none>                      37s     v1.31.0+k3s1
9b43501b-f5d8-4cce-bdab-b5fdcb777325-control-plane-n67gv   Ready    control-plane,etcd,master   3m56s   v1.31.0+k3s1

If this doesn’t work, try using the cli flag.

Preparing Azure DevOps

We will now create the agent pool and PAT (Personal Access Token) that the agents will use to run.

1. Go to your Azure DevOps project and click the gear icon in the bottom left.
2. Click on Agent pools in the left sidebar. You should now see a list of Agent pools.
3. Click Add pool in the top left.
4. Select Pool to link: New and Pool type: self-hosted.
5. Enter a name for your agent pool. You will need this name when setting up the agents and to configure your pipelines. I chose Kubernetes.
6. Optionally enter a description and click Create.

Now we need to create the Personal Access Token (PAT) that the agents will use to authenticate with your Azure DevOps.

1. In the top right click on User settings (the icon with a person and a gear) and click Personal access tokens.
2. Click New Token and enter a name for your token so you remember what it is for. I chose Kubernetes Agents.
3. Select an expiration date (I would recommend you choose the max of one year so you don’t have to refresh it that often)
4. Choose Custom defined for the Scopes.
5. At the bottom there is a small link Show all scopes that we have to click to see the scope we need.
6. Select Agent Pools -> Read & manage.
7. Click create and make sure you copy the token first before you close the sidebar. I would recommend not closing the sidebar before completing the next steps.

Creating the agents

Now we can create the agents in the cluster. First of all, we need to save the PAT in a kubernetes secret. To do this, run the following command:

kubectl create secret generic azp --from-literal=azp-token='<PAT_FROM_AZURE_DEVOPS_HERE>'

Make sure to pass the --kubeconfig flag if not using the environment variable. You can now close the sidebar in Azure DevOps.

Then we can create the kubernetes deployment that will actually run the agent. Create a deployment.yaml file with the following content:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: azp-agent
spec:
  selector:
    matchLabels:
      app: azp-agent
  replicas: 1 # Number of agents to create
  strategy:
    rollingUpdate:
      maxSurge: 0 # No more instances than replicas will be created at a time
      maxUnavailable: 1 # Only one agent will be updated at a time
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: azp-agent
    spec:
      containers:
        - name: azp-agent
          image: ghcr.io/gigahatch/azp-agent:ubuntu-24.04
          resources:
            limits:
              memory: '1536Mi' # Depending on the node size, you might need to increase this
              cpu: '1500m' # Depending on the node size, you might need to increase this
          env:
            - name: AZP_URL
              value: https://dev.azure.com/<YOUR_ORGANISATION> # <- Add your organisation URL here
            - name: AZP_POOL
              value: Kubernetes # <- Add the name of the agent pool here
            - name: AZP_TOKEN
              valueFrom:
                secretKeyRef:
                  name: azp
                  key: azp-token

Make sure that:

1. That the URL of your organisation and the name of the agent pool in the environment variables of the deployment is correct.
2. If you chose a bigger node size or more replicas, you can also change the number of agents using the replicas property in the file. We will leave it at 1. You can also change the memory and cpu limits. 1500m means 1.5 CPUs and 1536Mi means 1.5GB RAM.
3. The update strategy is set to RollingUpdate with a max of 0 surge and 1 unavailable. This means that only one agent will be updated at a time.

Docker

If you want to run docker commands in the container, you will have to disable some security features. Add

securityContext:
  privileged: true

to your container definition. Your yaml should now look like this:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: azp-agent
spec:
  selector:
    matchLabels:
      app: azp-agent
  replicas: 1 # Number of agents to create
  strategy:
    rollingUpdate:
      maxSurge: 0 # No more instances than replicas will be created at a time
      maxUnavailable: 1 # Only one agent will be updated at a time
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: azp-agent
    spec:
      containers:
        - name: azp-agent
          image: ghcr.io/gigahatch/azp-agent:ubuntu-24.04
          resources:
            limits:
              memory: '1536Mi'
              cpu: '1500m'
          env:
            - name: AZP_URL
              value: https://dev.azure.com/<YOUR_ORGANISATION>
            - name: AZP_POOL
              value: Kubernetes
            - name: AZP_TOKEN
              valueFrom:
                secretKeyRef:
                  name: azp
                  key: azp-token
          securityContext:
            privileged: true # <- Add this

This will basically make the agent run like a normal process on the system, with no sandboxing. It is probably best if you only use this in a cluster solely dedicated to your build agents.

Deploying the agents

Now run the following command to create the deployment in the cluster:

kubectl apply -f deployment.yaml

It should print out deployment.apps/azp-agent created. Wait a bit and run this to check if the agent was created successfully.

kubectl get pods

It should print out something like this:

NAME                         READY   STATUS    RESTARTS   AGE
azp-agent-557fdcc78f-kr6kn   1/1     Running   0          4m38s

Now go back to your Azure DevOps agent pools, click on the agent pool you created previously and click on the Agents tab. You should now see an agent. Now we can run jobs on our cluster.

Configuring the pipeline

To run the pipeline on the new agent pool, just change the pool property in the pipeline yaml to the following:

pool: Kubernetes # Or whatever name you gave to the Agent Pool

Now try running your pipeline. The first time it runs, you will probably have to give it access to the agent pool, so just refresh the page on the pipeline run a few times until the banner appears to give it access.

Conclusion

You have now successfully setup your Azure Pipeline Agents on Gigahatch Managed Kubernetes. If you have any questions or are stuck somewhere, please write us at info@gigahatch.ch. We look forward to hearing from you.