Hi all, Gosuke Miyashita here at ATL.
We’ve released Walter v2.0.0.beta1.
Please read on for the details.
What is Walter?
Walter is a simple command line tool for defining and executing build pipelines in YAML.
Although many tools already exist for handling build pipelines and work flow, they tend to be cumbersome.
Walter has been designed as a thoroughly dependable, simple alternative.
Why v2?
v2 isn’t a mere revision of v1. Rather, Walter has been redeveloped entirely from the ground up to provide the following:
- Improved ease-of-use
- (Sacrificed backward-compatibility)
- Simplification of the codebase
- Easier extension
- Better facilitate the participation of other developers
Major Changes in v2
Here are the major changes in v2.
Pipeline Definition Format
In v1, executable pipelines were defined as follows:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
pipeline: - name: setup build type: command command: echo "setting up ..." - name: run build type: command command: echo "building ..." cleanup: - name: cleanup build type: command command: echo "cleanup build ..." |
In v2, the same content would be defined as follows:
|
1 2 3 4 5 6 7 8 9 10 |
build: tasks: - name: setup build command: echo "setting up ..." - name: run build command: echo "building ..." cleanup: - name: cleanup build command: echo "cleanup build ..." |
Build Phase & Deploy Phase
In v2, it’s now possible to partition pipelines into separate build and deploy phases:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
build: tasks: - name: setup build command: echo "setting up ..." - name: run build command: echo "building ..." cleanup: - name: cleanup build command: echo "cleanup build ..." deploy: tasks: - name: run deploy command: echo "deploying ..." cleanup: - name: cleanup command: echo "cleanup deploy ..." |
You can now optionally specify whether to execute the build alone, deployment alone, or both build and deployment:
|
1 2 3 4 5 6 7 8 9 |
// build だけ実行 $ walter -build // deploy だけ実行 $ walter -deploy // build と deploy 両方実行 $ walter -build -deploy |
External Definition File Format & Calling
In both v1 and v2, it’s possible to partition pipeline definition files externally and call them from the main file. However, file format and call method have been modified in v2.
In v1, pipelines were defined in an external file formatted differently from ordinary tasks:
task1.yml
|
1 2 3 4 5 6 7 |
namespace: task1 stages: - def: name: task command: echo task1 |
task2.yml
|
1 2 3 4 5 6 7 |
namespace: task2 stages: - def: name: task command: echo task2 |
External files “task1.yml” and “task2.yml” were called per the following:
pipeline.yml
|
1 2 3 4 5 6 7 |
require: - task1.yml - task2.yml pipeline: - call: task1::task - call: task2::task |
For external files in v2, the build and deploy phase portion is omitted, only requiring task declaration. This eliminates namespace from v1 and makes it possible to declare tasks in the same manner as the main file.
task1.yml
|
1 2 3 |
- name: task1 command: echo task1 |
task2.yml
|
1 2 3 |
- name: task2 command: echo task2 |
External files “task1.yml” and “task2.yml” would be called per the following:
pipeline.yml
|
1 2 3 4 5 |
build: tasks: - include: task1.yml - include: task2.yml |
Furthermore, pipelines are now executable using an individual external file. This makes it easier to execute a portion of the pipeline for testing.
|
1 2 |
$ walter -build -config task2.yml |
wait_for Format
Formatting of the wait_for function (which waits for a condition such as port availability or file creation before executing a task) has been modified in v2.
In v1, parameters were declared on one line:
|
1 2 3 4 5 6 7 |
pipeline: - name: launch solr command: bin/solr start - name: post data to solr index command: bin/post -d ~/tmp/foobar.js wait_for: host=localhost port=8983 state=ready |
In v2, parameters YAML mapping is used to declare parameters:
|
1 2 3 4 5 6 7 8 9 10 11 |
build: tasks: - name: launch solr command: bin/solr start - name: post data to solr index command: bin/post -d ~/tmp/foobar.js wait_for: host: localhost port: 8983 state: ready |
Notification Configuration
The process of stipulating recipients of execution result notification has been modified in v2.
In v1, notification recipients were stipulated as follows:
|
1 2 3 4 5 6 7 |
messenger: type: slack channel: serverspec url: $SLACK_WEBHOOK_URL icon_url: http://example.jp/walter.jpg username: walter |
In v2, messenger has been changed to notify, making it possible to send notifications to multiple destinations:
|
1 2 3 4 5 6 7 |
notify: - type: slack channel: serverspec url: $SLACK_WEBHOOK_URL icon_url: http://example.jp/walter.jpg username: walter |
Special Variables
Special variables such as __OUT, __ERR, __COMBINED, and __RESULT have been eliminated. This class of special variables facilitates the reuse of a given task’s standard output in another task, such as with __OUT[“task name”]. However, in the interest of pipeline simplicity, this has been modified in v2 to only allow piping of standard output from the immediately preceding task. Accordingly, these special variables have been eliminated.
In the following defined pipeline, standard output from the 1st task is implicitly passed to the standard input of the 2nd task.
|
1 2 3 4 5 6 7 |
build: tasks: - name: setup build command: echo "setting up" - name: run build command: cat |
GitHub Integration
GitHub code acquisition and pull request functionality have been removed in v2. These and other non-core functions will be reallocated to plugins and walter-agent (addressed below).
Questions, Comments, Pull Requests
If you have any questions, comments, or pull requests regarding v2, please visit the Walter page on the GitHub repository and open an Issue/Pull Request. (Inquiries in Japanese are also welcome.)
As v2 is still in the beta phase, development is taking place on the v2 branch, rather than the master branch. Please direct v2 pull requests to the v2 branch.
Future Plans
As with v1, we plan to implement walter-server and walter-agent for v2. Additionally, we hope to provide notification and other non-core functionality as plugins. However, in the meantime, this will wait until the plugins are performing stably in Golang.
As there’s been demand for a Windows version, that’s also on the docket.