Subject: [PATCH 0/9] Offer to run CI/PR builds in Visual Studio Team Services

From: "Johannes Schindelin via GitGitGadget" <[email-masked]>
Date: Mon, 03 Sep 2018 14:10:28 -0700 (PDT)

For a long time already, we have Git's source code continuously tested via
Travis CI, see e.g. https://travis-ci.org/git/git/builds/421738884. It has
served us well, and more and more developers actually pay attention and
benefit from the testing this gives us.

It is also an invaluable tool for contributors who can validate their code
contributions via PRs on GitHub, e.g. to verify that their tests do actually
run on macOS (i.e. with the BSD family of Unix tools instead of the GNU
one).

The one sad part about this is the Windows support. Travis lacks it, and we
work around that by using Visual Studio Team Services (VSTS) indirectly: one
phase in Travis would trigger a build, wait for its log, and then paste that
log.

As Git's Windows builds (and tests!) take quite a bit of time, Travis often
timed out, or somehow the trigger did not work, and for security reasons
(the Windows builds are performed in a private pool of containers), the
Windows builds are completely disabled for Pull Requests on GitHub.

One might ask why we did not use Visual Studio Team Services directly. There
were a couple of reasons for that:

 * most notably, VSTS's build logs could not be viewed anonymously,
 * while VSTS had Linux and Windows agents, it lacked macOS agents,
 * etc

The main two reasons no longer apply: macOS agents are available now
[https://docs.microsoft.com/en-us/vsts/release-notes/2018/jul-10-vsts], and
there is a limited preview of "public projects"
[https://blogs.msdn.microsoft.com/devops/2018/04/27/vsts-public-projects-limited-preview/]
, i.e. it is possible to configure a VSTS project so that anybody can view
the logs.

I had secured such a public project for Git for Windows already, and I
recently also got one for Git. For now, the latter is hooked up with my
personal git.git fork on GitHub, but it is my hope that I convince y'all
that these VSTS builds are a good idea, and then hook it up with 
https://github.com/git/git.

As a special treat, this patch series adds the ability to present the
outcome of Git's test suite as JUnit-style .xml files. This allows the VSTS
build to present fun diagrams, trends, and makes it a lot easier to drill
down to test failures than before. See for example 
https://git.visualstudio.com/git/_build/results?buildId=113&view=ms.vss-test-web.test-result-details
[https://git.visualstudio.com/git/_build/results?buildId=113&view=ms.vss-test-web.test-result-details] 
(you can click on the label of the failed test, and then see the detailed
output in the right pane).

This patch series took way more time than I had originally planned, but I
think that in particular the advanced display of the test results was worth
it. Please let me know what you think about this.

Johannes Schindelin (9):
  ci: rename the library of common functions
  ci/lib.sh: encapsulate Travis-specific things
  test-date: add a subcommand to measure times in shell scripts
  tests: optionally write results as JUnit-style .xml
  ci/lib.sh: add support for VSTS CI
  Add a build definition for VSTS
  tests: include detailed trace logs with --write-junit-xml upon failure
  tests: record more stderr with --write-junit-xml in case of failure
  README: add a build badge (status of the VSTS build)

 .vsts-ci.yml                   | 296 +++++++++++++++++++++++++++++++++
 README.md                      |   2 +
 ci/install-dependencies.sh     |   5 +-
 ci/{lib-travisci.sh => lib.sh} |  67 ++++++--
 ci/mount-fileshare.sh          |  26 +++
 ci/print-test-failures.sh      |   4 +-
 ci/run-build-and-tests.sh      |   2 +-
 ci/run-linux32-docker.sh       |   2 +-
 ci/run-static-analysis.sh      |   2 +-
 ci/run-windows-build.sh        |   2 +-
 ci/test-documentation.sh       |   3 +-
 t/.gitignore                   |   1 +
 t/helper/test-date.c           |  12 ++
 t/test-lib.sh                  | 142 ++++++++++++++++
 14 files changed, 544 insertions(+), 22 deletions(-)
 create mode 100644 .vsts-ci.yml
 rename ci/{lib-travisci.sh => lib.sh} (61%)
 create mode 100755 ci/mount-fileshare.sh

base-commit: 2f743933341f276111103550fbf383a34dfcfd38
Published-As: https://github.com/gitgitgadget/git/releases/tags/pr-31%2Fdscho%2Fvsts-ci-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-31/dscho/vsts-ci-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/31
-- 
gitgitgadget