Subject: [PATCH v2 0/4] Travis + Azure jobs for linux with musl libc

From: Đoàn Trần Công Danh <[email-masked]>
Date: Sun, 29 Mar 2020 17:12:28 +0700

Recently, we've un-broken git for Linux with musl libc,
and we have a serie to fix false negative with busybox shell utils.

Add a CI job on Travis and Azure to make sure we won't break it again.

There is a sample travis build for this serie applied on top of:
jt/rebase-allow-duplicate
https://travis-ci.org/github/sgn/git/builds/668300819

And, after merging with junio's pu to fix busybox false negative:
https://travis-ci.org/github/sgn/git/builds/668300919
https://dev.azure.com/git/git/_build/results?buildId=1910&view=results

Change from v1:
- fix spelling
- run-docker.sh: use "jobname" environment variable instead of passing argument
- add linux-musl job on Azure
- Add 4th patch for jt/rebase-allow-duplicate (feel free to squash into
 jt/rebase-allow-duplicate)

The first 3 patches could be applied on top of master,
but the last patch needs to be applied on top of jt/rebase-allow-duplicate

Đoàn Trần Công Danh (4):
  ci: libify logic for usage and checking CI_USER
  ci: refactor docker runner script
  travis: build and test on Linux with musl libc and busybox
  t3402: use POSIX compliant regex(7)

 .travis.yml                                 | 10 +++++-
 azure-pipelines.yml                         | 39 +++++++++++++++++++--
 ci/lib-docker.sh                            | 37 +++++++++++++++++++
 ci/run-alpine-build.sh                      | 31 ++++++++++++++++
 ci/{run-linux32-docker.sh => run-docker.sh} | 26 ++++++++++----
 ci/run-linux32-build.sh                     | 35 +-----------------
 t/t3402-rebase-merge.sh                     |  8 ++---
 7 files changed, 139 insertions(+), 47 deletions(-)
 create mode 100644 ci/lib-docker.sh
 create mode 100755 ci/run-alpine-build.sh
 rename ci/{run-linux32-docker.sh => run-docker.sh} (46%)

Range-diff against v1:
1:  f23f2a563a = 1:  1ec7c2024d ci: libify logic for usage and checking CI_USER
2:  6fd1370678 ! 2:  140e0ef390 ci: refactor docker runner script
    @@ Metadata
      ## Commit message ##
         ci: refactor docker runner script
     
    -    We will support alpine check in docker later in this serie.
    +    We will support alpine check in docker later in this series.
     
         While we're at it, tell people to run as root in podman.
     
    @@ .travis.yml: matrix:
              - docker
            before_install:
     -      script: ci/run-linux32-docker.sh
    -+      script: ci/run-docker.sh linux32
    ++      script: ci/run-docker.sh
          - env: jobname=StaticAnalysis
            os: linux
            compiler:
    @@ azure-pipelines.yml: jobs:
      
             res=0
     -       sudo AGENT_OS="$AGENT_OS" BUILD_BUILDNUMBER="$BUILD_BUILDNUMBER" BUILD_REPOSITORY_URI="$BUILD_REPOSITORY_URI" BUILD_SOURCEBRANCH="$BUILD_SOURCEBRANCH" BUILD_SOURCEVERSION="$BUILD_SOURCEVERSION" SYSTEM_PHASENAME="$SYSTEM_PHASENAME" SYSTEM_TASKDEFINITIONSURI="$SYSTEM_TASKDEFINITIONSURI" SYSTEM_TEAMPROJECT="$SYSTEM_TEAMPROJECT" CC=$CC MAKEFLAGS="$MAKEFLAGS" bash -lxc ci/run-linux32-docker.sh || res=1
    -+       sudo AGENT_OS="$AGENT_OS" BUILD_BUILDNUMBER="$BUILD_BUILDNUMBER" BUILD_REPOSITORY_URI="$BUILD_REPOSITORY_URI" BUILD_SOURCEBRANCH="$BUILD_SOURCEBRANCH" BUILD_SOURCEVERSION="$BUILD_SOURCEVERSION" SYSTEM_PHASENAME="$SYSTEM_PHASENAME" SYSTEM_TASKDEFINITIONSURI="$SYSTEM_TASKDEFINITIONSURI" SYSTEM_TEAMPROJECT="$SYSTEM_TEAMPROJECT" CC=$CC MAKEFLAGS="$MAKEFLAGS" bash -lxc ci/run-docker.sh linux32 || res=1
    ++       sudo AGENT_OS="$AGENT_OS" BUILD_BUILDNUMBER="$BUILD_BUILDNUMBER" BUILD_REPOSITORY_URI="$BUILD_REPOSITORY_URI" BUILD_SOURCEBRANCH="$BUILD_SOURCEBRANCH" BUILD_SOURCEVERSION="$BUILD_SOURCEVERSION" SYSTEM_PHASENAME="$SYSTEM_PHASENAME" SYSTEM_TASKDEFINITIONSURI="$SYSTEM_TASKDEFINITIONSURI" SYSTEM_TEAMPROJECT="$SYSTEM_TEAMPROJECT" CC=$CC MAKEFLAGS="$MAKEFLAGS" jobname=Linux32 bash -lxc ci/run-docker.sh || res=1
      
             sudo chmod a+r t/out/TEST-*.xml
             test ! -d t/failed-test-artifacts || sudo chmod a+r t/failed-test-artifacts
    @@ azure-pipelines.yml: jobs:
             test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || res=1
             exit $res
     -    displayName: 'ci/run-linux32-docker.sh'
    -+    displayName: 'ci/run-docker.sh linux32'
    ++    displayName: 'jobname=Linux32 ci/run-docker.sh'
          env:
            GITFILESHAREPWD: $(gitfileshare.pwd)
        - task: PublishTestResults@2
    @@ ci/run-docker.sh (new)
     @@
     +#!/bin/sh
     +#
    -+# Download and run Docker image to build and test git
    ++# Download and run Docker image to build and test Git
     +#
     +
     +. ${0%/*}/lib.sh
     +
    -+CI_TARGET=${1:-linux32}
    -+case "$CI_TARGET" in
    -+linux32) CI_CONTAINER="daald/ubuntu32:xenial" ;;
    -+*)       exit 1 ;;
    ++case "$jobname" in
    ++Linux32)
    ++	CI_TARGET=linux32
    ++	CI_CONTAINER="daald/ubuntu32:xenial"
    ++	;;
    ++*)
    ++	exit 1 ;;
     +esac
     +
     +docker pull "$CI_CONTAINER"
3:  2f68e65fb7 ! 3:  6cf6400f2e travis: build and test on Linux with musl libc and busybox
    @@ .travis.yml
     @@ .travis.yml: matrix:
              - docker
            before_install:
    -       script: ci/run-docker.sh linux32
    -+    - env: jobname=linux-musl-busybox
    +       script: ci/run-docker.sh
    ++    - env: jobname=linux-musl
     +      os: linux
     +      compiler:
     +      addons:
     +      services:
     +        - docker
     +      before_install:
    -+      script: ci/run-docker.sh alpine
    ++      script: ci/run-docker.sh
          - env: jobname=StaticAnalysis
            os: linux
            compiler:
     
    + ## azure-pipelines.yml ##
    +@@ azure-pipelines.yml: jobs:
    +       PathtoPublish: t/failed-test-artifacts
    +       ArtifactName: failed-test-artifacts
    + 
    ++- job: linux_musl
    ++  displayName: linux-musl
    ++  condition: succeeded()
    ++  pool:
    ++    vmImage: ubuntu-latest
    ++  steps:
    ++  - bash: |
    ++       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
    ++
    ++       res=0
    ++       sudo AGENT_OS="$AGENT_OS" BUILD_BUILDNUMBER="$BUILD_BUILDNUMBER" BUILD_REPOSITORY_URI="$BUILD_REPOSITORY_URI" BUILD_SOURCEBRANCH="$BUILD_SOURCEBRANCH" BUILD_SOURCEVERSION="$BUILD_SOURCEVERSION" SYSTEM_PHASENAME="$SYSTEM_PHASENAME" SYSTEM_TASKDEFINITIONSURI="$SYSTEM_TASKDEFINITIONSURI" SYSTEM_TEAMPROJECT="$SYSTEM_TEAMPROJECT" CC=$CC MAKEFLAGS="$MAKEFLAGS" jobname=linux-musl bash -lxc ci/run-docker.sh || res=1
    ++
    ++       sudo chmod a+r t/out/TEST-*.xml
    ++       test ! -d t/failed-test-artifacts || sudo chmod a+r t/failed-test-artifacts
    ++
    ++       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || res=1
    ++       exit $res
    ++    displayName: 'jobname=linux-musl ci/run-docker.sh'
    ++    env:
    ++      GITFILESHAREPWD: $(gitfileshare.pwd)
    ++  - task: PublishTestResults@2
    ++    displayName: 'Publish Test Results **/TEST-*.xml'
    ++    inputs:
    ++      mergeTestResults: true
    ++      testRunTitle: 'musl'
    ++      platform: Linux
    ++      publishRunAttachments: false
    ++    condition: succeededOrFailed()
    ++  - task: PublishBuildArtifacts@1
    ++    displayName: 'Publish trash directories of failed tests'
    ++    condition: failed()
    ++    inputs:
    ++      PathtoPublish: t/failed-test-artifacts
    ++      ArtifactName: failed-test-artifacts
    ++
    + - job: static_analysis
    +   displayName: StaticAnalysis
    +   condition: succeeded()
    +
      ## ci/run-alpine-build.sh (new) ##
     @@
     +#!/bin/sh
    @@ ci/run-alpine-build.sh (new)
     +'
     
      ## ci/run-docker.sh ##
    -@@
    - CI_TARGET=${1:-linux32}
    - case "$CI_TARGET" in
    - linux32) CI_CONTAINER="daald/ubuntu32:xenial" ;;
    -+alpine)  CI_CONTAINER="alpine" ;;
    - *)       exit 1 ;;
    +@@ ci/run-docker.sh: Linux32)
    + 	CI_TARGET=linux32
    + 	CI_CONTAINER="daald/ubuntu32:xenial"
    + 	;;
    ++linux-musl)
    ++	CI_TARGET=alpine
    ++	CI_CONTAINER=alpine
    ++	;;
    + *)
    + 	exit 1 ;;
      esac
    - 
-:  ---------- > 4:  a4eacb4362 t3402: use POSIX compliant regex(7)
-- 
2.26.0.302.g234993491e