How to Configure Self-hosted Azure DevOps Agents

In this article, I will show you the configuration process of a Self-hosted agent on macOS and Docker. Azure provides Microsoft-hosted and Self-hosted agents to execute CI/CD pipelines.

The advantage of using a Self-hosted agent include:

• Price

  • 1 Microsoft-hosted job with 1,800 minutes per month for CI/CD and 1 self-hosted job with unlimited minutes per month
  • $40 per extra Microsoft-hosted CI/CD parallel job and $15 per additional self-hosted CI/CD similar job with unlimited minutes

• It allows you to run the agent on any supported operating system (Linux, Windows, MacOS).

Prerequisites

• Azure user.
• Azure DevOps organizations user.

Generate access token.

Step 1: Log in to “Azure DevOps organization”, select “user settings”, and choose “Personal access tokens”

Step 2: Create an Access token.

On the next page select “New Token“ and provide “Name”, “Organization“, and “Expiration date“, and select all necessary permissions (In this demo we will use Full access). After filling out the form push the “Create“ button.

Install an agent on macOS.

Step 1: Download the agent.

For this execute the following command.

wget https://vstsagentpackage.azureedge.net/agent/3.220.5/vsts-agent-osx-x64-3.220.5.tar.gz

Step 2: Extract the downloaded archive file.

For this execute the following command. Where we will create a new folder and extract our archive file to this folder.

mkdir myagent && cd myagent
tar zxvf ../vsts-agent-osx-x64-3.220.5.tar.gz

After extracting we will have these files and folders.

Step 3: Configure the agent.

To configure the agent, we need to run the “config.sh“ file, which will be asked us to enter your server URL, authentication type, and personal access token (from previous steps).

./config.sh

Step 4: Run the agent.

For this run the following command.

./run.sh 

Step 5: Check the result.

For this return back to Azure DevOps and select “Project settings“ at the bottom left corner of the page. Then choose “Agent pools“ → “Default“.

Running agent in Docker.

Another option to run the Azure agent, it is running as a container. In this article, we will run an agent on the “ubuntu:20.04” image.

Prerequisites

• Docker.

Step 1. Create Dockerfile.

First of all, we need to create an empty folder and then create Dockerfile.

mkdir azureagent && cd azureagent
nano Dockerfile

Then paste the following content and save the file.

FROM ubuntu:20.04
RUN DEBIAN_FRONTEND=noninteractive apt-get update
RUN DEBIAN_FRONTEND=noninteractive apt-get upgrade -y

RUN DEBIAN_FRONTEND=noninteractive apt-get install -y -qq --no-install-recommends \
    apt-transport-https \
    apt-utils \
    ca-certificates \
    curl \
    git \
    iputils-ping \
    jq \
    lsb-release \
    software-properties-common

RUN curl -sL https://aka.ms/InstallAzureCLIDeb | bash

# Can be 'linux-x64', 'linux-arm64', 'linux-arm', 'rhel.6-x64'.
ENV TARGETARCH=linux-x64

WORKDIR /azp

COPY ./start.sh .
RUN chmod +x start.sh

ENTRYPOINT [ "./start.sh" ]

Step 2. Create the “start.sh“ file.

Create a new file in the same directory where we created the Dockerfile with the name “start.sh“

nano start.sh

Paste the following content to the file and save it.

#!/bin/bash
set -e

if [ -z "$AZP_URL" ]; then
  echo 1>&2 "error: missing AZP_URL environment variable"
  exit 1
fi

if [ -z "$AZP_TOKEN_FILE" ]; then
  if [ -z "$AZP_TOKEN" ]; then
    echo 1>&2 "error: missing AZP_TOKEN environment variable"
    exit 1
  fi

  AZP_TOKEN_FILE=/azp/.token
  echo -n $AZP_TOKEN > "$AZP_TOKEN_FILE"
fi

unset AZP_TOKEN

if [ -n "$AZP_WORK" ]; then
  mkdir -p "$AZP_WORK"
fi

export AGENT_ALLOW_RUNASROOT="1"

cleanup() {
  if [ -e config.sh ]; then
    print_header "Cleanup. Removing Azure Pipelines agent..."

    # If the agent has some running jobs, the configuration removal process will fail.
    # So, give it some time to finish the job.
    while true; do
      ./config.sh remove --unattended --auth PAT --token $(cat "$AZP_TOKEN_FILE") && break

      echo "Retrying in 30 seconds..."
      sleep 30
    done
  fi
}

print_header() {
  lightcyan='\033[1;36m'
  nocolor='\033[0m'
  echo -e "${lightcyan}$1${nocolor}"
}

# Let the agent ignore the token env variables
export VSO_AGENT_IGNORE=AZP_TOKEN,AZP_TOKEN_FILE

print_header "1. Determining matching Azure Pipelines agent..."

AZP_AGENT_PACKAGES=$(curl -LsS \
    -u user:$(cat "$AZP_TOKEN_FILE") \
    -H 'Accept:application/json;' \
    "$AZP_URL/_apis/distributedtask/packages/agent?platform=$TARGETARCH&top=1")

AZP_AGENT_PACKAGE_LATEST_URL=$(echo "$AZP_AGENT_PACKAGES" | jq -r '.value[0].downloadUrl')

if [ -z "$AZP_AGENT_PACKAGE_LATEST_URL" -o "$AZP_AGENT_PACKAGE_LATEST_URL" == "null" ]; then
  echo 1>&2 "error: could not determine a matching Azure Pipelines agent"
  echo 1>&2 "check that account '$AZP_URL' is correct and the token is valid for that account"
  exit 1
fi

print_header "2. Downloading and extracting Azure Pipelines agent..."

curl -LsS $AZP_AGENT_PACKAGE_LATEST_URL | tar -xz & wait $!

source ./env.sh

trap 'cleanup; exit 0' EXIT
trap 'cleanup; exit 130' INT
trap 'cleanup; exit 143' TERM

print_header "3. Configuring Azure Pipelines agent..."

./config.sh --unattended \
  --agent "${AZP_AGENT_NAME:-$(hostname)}" \
  --url "$AZP_URL" \
  --auth PAT \
  --token $(cat "$AZP_TOKEN_FILE") \
  --pool "${AZP_POOL:-Default}" \
  --work "${AZP_WORK:-_work}" \
  --replace \
  --acceptTeeEula & wait $!

print_header "4. Running Azure Pipelines agent..."

trap 'cleanup; exit 0' EXIT
trap 'cleanup; exit 130' INT
trap 'cleanup; exit 143' TERM

chmod +x ./run.sh

# To be aware of TERM and INT signals call run.sh
# Running it with the --once flag at the end will shut down the agent after the build is executed
./run.sh "[email protected]" & wait $!

Step 3. Build the Docker image.

For this run the following command inside folder where we created previous files.

docker build -t dockeragent:latest .

Step 4. Run a Docker container.

After successfully creating the Docker Image we can start our container.

For this execute the following command.

docker run -e AZP_URL=https://dev.azure.com/vbilay -e AZP_TOKEN=143324h453425ljdfshg45 -e AZP_AGENT_NAME=mydockeragent dockeragent:latest

Where:

AZP_URL - your server URL.

AZP_TOKEN - your personal access token (from previous steps).

Step5. Check the result.

For this return back to Azure DevOps and select “Project settings“ at the bottom left corner of the page. Then choose “Agent pools“ → “Default“.

That’s it, we successfully configured Azure DevOps agents on macOS and Docker.