Private build agents have many advantages over the hosted agents Microsoft provide, however they also require managing. My preference is to use an Azure scale set with a custom image but, this doesn’t meet every scenario. Sometimes you have jobs that are short running and don’t require much compute resources, so it makes more sense to have multiple agents on one server. The problem with this is the disk space on your agent can quickly be consumed by the jobs the multiple agents run.

Adding a PowerShell task to clean up the files once the job has finished (A glorified get-childitem -recurse | remove-item -force) combined with setting your job retention to only 1/2 days, is normally enough to keep your disk space usage in check.

PSA: I found that the out of box task to clean the build directories, does not work if you use YAML pipelines

However, if you use npm in your builds, you could have another culprit gobbling up your disk space. It caches the packages it uses on your local machine. These are normally stored in your user profile, but if you use npm in your build pipeline, the cache folder is created in the networkService profile.

C:\Windows\ServiceProfiles\NetworkService\AppData\Roaming\npm-cache

If you don’t keep on top of this folder, it will just keep growing, the largest I had seen was 70GB 😬. I know what you’re thinking, just delete the folder right? Remember in this scenario it’s a permanent build agent. If you cause an issue with npm on the server, you can’t just deploy another scale set instance in 5 mins. You will have to deploy and configure another build agent. Yes, you could install and configure software using tools like Azure DSC, but that will still take time. Also I haven’t seen many people automating the DevOps agent install via DSC (If you have, please let me know).

I did some internet searches to try and find in black and white, both the consequences of deleting this cache and also examples on how to manage it. Strangely I couldn’t really find anything. I also tried the Azure DevOps Bat signal #LoECDA on twitter, which brought a swift reply from DevOps royalty Donovan Brown! 🤩. Honestly, if you have any Azure DevOps issues or you just need some advice, use this hash tag. The responses are amazing.

To cover all my bases I also posted on reddit. I don’t use reddit much, but thought it would be worth a punt. Two people replied ( vlkmr and phoxtricks ), they both had a similar requirement and suggested redirecting the npm cache to another drive.

I use Azure VMs for my build agents and they have a D drive which gets cleared when the server is deallocated. Normally they just hold the page file and some other temporary files and Microsoft is quite clear that you shouldn’t store any data there, but its perfect for the cache as it’s not important data that I need to retain. If it gets deleted, the next build will download it again.

One of the suggestions in the reddit post was to use an environment variable to set the cache location. I needed to test this setting first, before I committed to the new cache location, so I decided to use the npm cli to set the location instead. This only seems to apply to the session and takes about 20 seconds. I also added some code to check for the folder that I am redirecting to and create it if it doesn’t exist.

- task: PowerShell@2
  displayName: "Set NPM Cache location"
  inputs:
    targetType: 'inline'
    pwsh: true
    failOnStderr: true
    showWarnings: true
    script: |
      $cachePath  = "C:\npmCache"
      if (!(test-path  -path $cachePath  -ErrorAction SilentlyContinue)) {
          new-item -type  Directory -Path $cachePath  -force
      }

      npm config set cache $cachePath --global      

I put this task before I run the npm install command, this makes sure that is puts any downloaded files into the new location. The build agents have been running with this config for a few weeks now and so far the cache hasn’t grown too much. I’m going to keep any eye on it, but it looks great so far. Now I just need to sort the Nuget cache 🤦‍♂️