Last week Microsoft announced “Pipeline as code (YAML)” giving us the ability to store our builds in source control. This allows us to take advantage of Configuration as Code as well as other benefits not available through the Visual Designer builds:
- The YAML files follow the same branching structure as your code. This gives us multiple versions of the build, preventing breaking changes to the build process.
- Validate changes through code reviews in pull requests and branch build policies.
- If the build breaks, changes can be easily identified in version control.
- Working with code is preferable to using the Web interface.
- Fan build steps in/out across build agents.
I’ll let you read the getting started guide which walks you through the basics. In this post I wanted to go through the process I used to convert some of my builds and some gotchas I found on the way.
Where is the export button?
Edit: This is no longer necessary. You can now click “View YAML”.
Unfortunately Microsoft hasn’t created an easy way to export an existing build definition. This means we’ll need to use the API to get the JSON from a build definition, convert it to YAML, tweak the syntax, and update the tasks that are referenced.
- Use the API to get a definition:
https://{instance}/DefaultCollection/{project}/_apis/build/definitions/{definitionId}?api-version={version}[&revision={int}] - Copy/paste the JSON into a JSON to YAML converter. Example: https://www.json2yaml.com/
- Copy the YAML to a text editor.
- Use the guide to tweak the syntax.
- Replace the refName key values with task names and version. Built in tasks can be found here (look in task.json file) https://github.com/Microsoft/vsts-tasks/tree/master/Tasks All the tasks in your account can be found here:
https://{instance}/DefaultCollection/{project}/_apis/distributedtask/tasks - Replace all tabs with spaces in case you accidentally added them (YAML syntax).
Phases
The YAML builds provide the ability to separate steps by phases. The phases can be run in parallel and even fanned in and out across agents.
Fan out example:
phases: - phase: InitialPhase steps: - script: echo hello from initial phase - phase: SubsequentA dependsOn: InitialPhase steps: - script: echo hello from subsequent A - phase: SubsequentB dependsOn: InitialPhase steps: - script: echo hello from subsequent B
Take advantage of this anywhere that steps can run in parallel. Keep in mind that when you fan out across agents, each phase will have it’s own working, staging, and artifact directories.
Gotchas
– Task Groups cannot be referenced as tasks. Same process above can be used to convert these using the API.
– When adding service endpoints you will need to re-save the build definition in order to authenticate.
Example
Here is an example YAML file I created that has 5 phases. The first 4 fan out and run in parallel and the last phase waits for all of them to complete before beginning. I removed endpoints and all company specific data.
Just wanted to say I love reading your blog and look forward to
all your posts! Keep up the superb work!
Thanks for this. Is it possible to capture the exit code from one task and use this as an input to another task?
David, yes it is! Here is an example: https://roadtoalm.com/2016/08/11/set-output-variable-in-a-powershell-vsts-build-task/
It’s a pity you don’t have a donate button! I’d definitely donate to this brilⅼiant Ьlog! I suppose for noᴡ i’ll settleе for bookmarking and adding your RSS feed to my Google account.
Hi Marcus,
I am trying to build my project on Azure DevOps via Yaml file configuration, I am new to it. I am receiving an error “#[error]Unhandled: Input required: TargetFolder” , could you please advice where am I going wrong.
below is my Yaml file:
# Build app using Azure Pipelines
# https://docs.microsoft.com/azure/devops/pipelines/languages/javascript?view=vsts
pool:
vmImage: ‘vs2017-win2016’
steps:
– script: echo hello world
– task: NodeTool@0
inputs:
versionSpec: ‘8.x’
– task: CopyFiles@1
displayName: ‘Copy Files to: $(Build.ArtifactStagingDirectory)’
inputs:
SourceFolder: ‘$(build.sourcesdirectory)’
Contents:
\C:\VSCodeGit\ColleagueModCAutoGenFiles\ColleagueModuleArchitectureC\ColleagueModuleArchitectureC.Web\Web.config\
TartgetFolder: ‘$(Build.ArtifactStagingDirectory)’
condition: succeededOrFailed()
– task: PublishBuildArtifacts@1
displayName: ‘Publish Artifact: drop’
inputs:
PathtoPublish: ‘$(Build.ArtifactStagingDirectory)’
condition: succeededOrFailed()
Thanks,
A
A,
I highly recommend using the Azure Pipelines Visual Studio Code extension. It has syntax highlighting and autocompletion for Azure Pipelines YAML. You can use the Azure Pipelines Web UI to create the steps, click “View YAML” in the upper right, then copy paste that into VS Code.
Here is an example of what I would use for your snippet: https://gist.github.com/MarcusFelling/0f0b49001354206de6d2be7f6915b3f5
I would suggest using predefined build variables in directories for sections like “Content:”. The working directory for build agents could change at any time.
Cheers,
Marcus
The funny thing about this, is how history repeats itself. I was storing my build definitions in VSS 15 years ago. And then came xaml which was mess. Then came vNext, with a web interface (what a mess, I export my defs to json, edit them, and reimport them).
Funny, but we are going back in time from my perspective.
It’s like I always said, if you manage the data, the process doesn’t really matter.
Ha! So true. Not that I experienced that pain, but it sounds like Microsoft finally figured things out with their new focus on YAML pipelines. They actually make it a touch harder to even find the visual designer option when creating new definitions. A step in the right direction…