Skip to content

DVPS-351: fail RAML build jobs when nothing is generated

Dmitrii Tolstoi requested to merge dvps-351-raml-ci-fail-on-empty-build into master

Problem

Build jobs discarded generator failures via || true plus a blanket allow_failure: true, and publish jobs finished as success on a zero-iteration while read nuspec loop. A broken contract therefore shipped as a fully green pipeline that published nothing — 8 consecutive runs on thor/schemas/r1.raml (master and before-contract-id-compatibility-branch) reported success while producing no packages at all.

Job colour also depended on how early the generator died rather than on whether anything was built: an early abort left no ./out and failed on mv, while a late failure left empty folders and passed.

What changes

Partial success stays non-blocking — one broken contract must not hold back the several hundred others. What changes is that three outcomes become distinguishable:

Outcome Before After
Nothing queued for build success success
Some contracts failed success exit 2 — warning, non-blocking
Nothing produced at all success exit 1 — red
  • Generator exit code is captured instead of discarded, and the result is judged by output: a contract counts as built when at least one nuspec carries its title as prefix. One contract yields Server/Client/Client.Contract, and the generator upper-cases the version part of the title (R1.Foo.v1R1.Foo.V1.Server), so matching is case-insensitive on the prefix.
  • allow_failure narrowed from true to exit_codes: [2].
  • The mkdir out_x && mv ./out/* pair no longer doubles as the failure detector; the move is guarded and reports nothing by itself.
  • Each build job prints a summary: contracts queued / built / nuspec produced / which contracts failed.
  • raml-build-report-<447|801|705>.txt lets publish tell a legitimate empty build list apart from a build that fell over. It is deliberately kept outside the out_* folders, because raml-browser-ui-ci.yml hands out_raml2 to publish-801-npm as SOURCE_DIR — that folder must stay free of files the downstream image does not expect. The build number in the file name keeps the three build jobs of a single browser-ui pipeline from overwriting each other.
  • Publish jobs fail when the build queued contracts but no nuspec arrived, stay green when zero were queued, and warn-and-proceed when no report is present so an in-flight pipeline started before this change still works.
  • Artifacts are kept on failure (when: always) — otherwise a partial build would drop the packages it did produce and starve the publish stage — and now carry errors.log / build.log, which the generator writes but CI discarded.

Case folding uses tr rather than ${var,,} because the three generator images are not guaranteed to ship bash 4.

Verified on real pipelines

The template branch was exercised through include: ref: from throwaway branches of thor/schemas/r1.raml:

Scenario Pipeline Result
Empty build list 966331 build green (There is noting to build); publish reports Nothing was queued for build, nothing to publish and never calls push
Nothing generated (contract deliberately broken) 966340 build exit 1, pipeline red, publish skipped, errors.log + build.log uploaded on failure
Partial: 1 of 2 contracts built 966342 build exit 2 (warning), pipeline not blocked, 31 artefact entries, publish available

Note that GitLab CI Lint via the API silently ignores ref: on include and lints the default branch instead, so it reports valid: true against master — these pipeline runs are the actual verification.

Consumers

Both thor/schemas/r1.raml and thor/schemas/r1.browserui.raml (the latter through Jobs/raml-browser-ui-ci.yml, which includes this template).

.build-447-raml-base / .publish-447-nuget-base are not dead code — there is no extends inside this file, but raml-browser-ui-ci.yml extends them, so they are covered by this change too.

Reviewer notes

  • browser-ui build jobs will start reporting warning instead of success. Its 447 line already fails 6 of 13 contracts with Could not find a part of the path '.../NuGetFallbackFolder/system.memory/4.5.1/ref/netstandard2.0/System.Memory.dll'. Publication is unaffected — the packages that do build still publish. Root cause is that .build-447-raml-base lacks the workaround that raml-multi-target-before-cid-ci.yml has; tracked separately, not fixed here to keep this diff about CI signalling.
  • expire_in raised from 1 hour to 1 day — a deliberate behaviour change. With a one-hour window a failure is no longer diagnosable the next morning; we hit exactly that while investigating DVPS-351.

Refs: DVPS-351

Merge request reports