Azure Pipeline and Azure Artefacts harmony

Photo by chuttersnap on Unsplash

Azure DevOps has been flourishing for some time now improving engineering experience sprint by sprint.

Recently, I have been observing how dependency management can be taken care of and how to make Azure Pipelines and Azure Artefacts services work together effectively. Although the examples will be specific to Java and Gradle, the principle remains the same no matter your language stack is.

Problem

The ACME organisation you work for develops a system comprising a series of Java projects using Gradle as a dependency manager. It uses several artefact repositories across the world which have different availability guarantees including those not having any guarantees at all. The project architect has asked you to integrate your existing pipelines with Azure Artefacts to mitigate the problem. You need to take care of:

• ensuring Azure Artefacts is the only upstream repository for all your Java projects
• making sure all developers will have a unique “access key” while connecting to the Azure artefacts
• ensuring unified experience for both local and remote (Azure Pipelines) builds

Solution

First step is to update build.gradle of all your Java projects with the following snippet:

repositories {
    maven {
        url 'https://pkgs.dev.azure.com/{your_organisation}/_packaging/{your_repository_name}/maven/v1'
        credentials {
            username "AZURE_ARTIFACTS"
            password System.getenv("ACCESS_TOKEN")
        }
    }
}

This will guarantee all your dependencies will be resolved only through Azure Artefacts.

ACCESS_TOKEN environment variable is expected to be set at developer machines. For example, if your OS is MacOS you could keep its definition in ~/.bashrc, if you use BASH.

But what value should be put there? Azure DevOps to generate an unique token called PATs (Personal Access Token) for every member of your Azure DevOps project. PATs are always valid within limited period of time (1 year max) and scope. The members of your project should be instructed on how to generate PATs in a safe and secure manner i.e. only for 3 months and limited scope. PATs must not be shared and treated as secrets.

Now that developers are happy, the only bit left is to enable associated pipelines to follow the same mechanism. You could potentially generate a PAT as an administrator, make it a secure variable within Variable Group and then set it as a ACCESS_TOKEN environment variable within all pipelines. This opens up a can of worms really as:

1. The PAT will expire sooner or later and you need to remember to extend its validity period or generate a new one. Although Azure DevOps will send you a notification about closing in on expiry date, it can lead your project grinding to a halt unexpectedly.
2. The PAT is assigned to you. What if you are assigned to a different project or leave? You cannot transfer the PAT to someone else.
3. Additional overhead of creating Variable Groups just for the sake of keeping PAT does not seem like a great idea.

Fortunately, there is a right solution for that. Every time your pipeline starts, it populates Environment Variables of the agent with System.AccessToken variable. It holds a PAT with short expiry date (2 days if I remember well). You can use it as follows:

steps:
 - task: Gradle@2
 displayName: Build Java Project X
 env:
    ACCESS_TOKEN: $(System.AccessToken)
 inputs:
   workingDirectory: 'ArtefactsDemo'
   gradleWrapperFile: 'ArtefactsDemo/gradlew'
   gradleOptions: '-Xmx3072m'
   javaHomeOption: 'JDKVersion'
   jdkVersionOption: '1.11'
   tasks: 'clean build'

With all the steps above you end up with a clean and secure integration between your Azure Pipelines and Azure Artefacts. The process of refreshing developer’s PAT is not too burdensome as it happens only once a certain period.

That’s all folks. Cheers.