Coverage for .github / scripts / check_bash_coverage.py: 100.00%
361 statements
« prev ^ index » next coverage.py v7.13.5, created at 2026-09-14 22:07 +0000
1"""Enforce the shell-script coverage floor from a bashcov/SimpleCov report.
3The ``unit:bats:shell`` job runs the BATS suite under ``bashcov``, which traces
4Bash through ``BASH_XTRACEFD`` and writes a SimpleCov resultset describing every
5*relevant* line of every shell file it saw and how many times each ran. This
6script turns that resultset into a pass/fail gate.
8Deciding which lines of a shell script are even executable is the hard part of
9Bash coverage — here-documents, ``case`` arms, line continuations and function
10headers all have to be classified — so that judgement is deliberately left to
11bashcov's lexer rather than re-implemented here, with two corrections that
12come from measuring what ``set -x`` actually prints:
14**Lines Bash never traces.** The lexer works from the text alone and marks two
15shapes executable that the tracer never reports, so no test could ever cover
16them: a compound-command terminator that carries only redirections (``done <<<
17"$rows"``, ``} > "$report"`` — the redirection belongs to the loop or group,
18and ``set -x`` prints simple commands, not the loop) and a ``case`` arm with no
19body (``*/*) ;;``). ``untraceable_lines()`` recognises exactly those two shapes
20and ``evaluate()`` leaves them out of the count, the way SimpleCov leaves out a
21comment. Anything that carries a command — a pipe into ``sed`` after ``}``, a
22``:`` in the arm, a process substitution — is still measured.
24**Statements that span lines.** Bash reports one line per statement, and which
25physical line it picks depends on the shape: the first line of ``python3 -c
26"..."`` with a multi-line string, the *second* line of a plain backslash chain
27(``kill_it \\ / one \\ / two`` reports line 2) or of a backgrounded one, and the
28*last* line of ``VAR=$(...)``, of ``VAR="multi\\nline"``, and of the ``cat
29<<'EOF' ... EOF )`` heredoc-in-substitution the client examples build their
30payloads with. The lexer propagates the first line's count across some of these
31shapes and not others (a ``\\"`` inside the string or a ``||`` on the last line
32of a chain defeats its patterns), which left dozens of lines permanently at
33zero. ``statement_spans()`` scans each script for statements that continue
34across lines — a trailing backslash, an unclosed ``(``/``$(``, an open quote,
35a here-document body — and ``evaluate()`` folds every span onto its first line
36with the highest count seen on any of its lines. A statement is covered when
37Bash reported it, wherever it reported it. A chain is split where a list
38operator (``||``, ``&&``, ``|``) starts a new command at the top level, because
39Bash does report those elements on their own lines; the fallback in
40``aws ... 2>&1 || echo "may already exist"`` therefore stays a separately
41measured statement.
43What this script owns beyond that is everything SimpleCov cannot know about
44*this* repository:
46**Path shape.** bashcov reports absolute paths (``/home/runner/work/.../demo/
47lib_demo.sh``), while the inventory and every error message use
48repository-relative ones. Reported paths are mapped back onto the tracked
49script they refer to by longest path suffix, falling back to a unique basename,
50and hits from every path that maps to the same script are merged — so a script
51exercised by several suites is credited with all of them.
53That merging also covers copies of a script, which matters because a BATS
54suite may ``cp`` the script under test into ``$BATS_TEST_TMPDIR`` and run it
55from an isolated fake repository. It does not rescue such a suite on its own,
56though: SimpleCov reads each file when it renders the report, and by then BATS
57has deleted its temporary directories, so the copies are dropped before this
58script ever sees them. A suite has to run the tracked file in place for its
59hits to count (the recorders take a repository-root override for exactly
60this). The merging is what keeps a *surviving* copy, or the same script seen
61under two different absolute prefixes, from being counted as two half-covered
62files.
64**The floor.** Every tracked script must be fully covered. The climb to 100%
65was staged through a shrink-only list of not-yet-covered scripts
66(``[tool.bash-coverage] ratchet`` in ``pyproject.toml``); it emptied and was
67deleted, and this script reads no exclusion list of any kind — a new script is
68covered, not listed. ``tests/test_check_bash_coverage.py`` keeps the list from
69coming back.
71The gate fails closed: a tracked script absent from the report entirely is an
72error, not a pass, because that is what a silently mis-scoped bashcov run or a
73suite that never executes its subject looks like.
75**The published report.** SimpleCov's own HTML renders the raw line hits, so
76it shows the two corrected shapes as misses and a lower number than the gate.
77``--report DIR`` writes a statement-level HTML report and a ``summary.json``
78from the corrected data instead; ``unit:bats:shell`` ships it in the
79``bash-coverage-report`` artifact and ``pages.yml`` serves it at
80``/bash-coverage/`` and renders the README badge from the summary, so the
81badge, the report and the gate describe one measurement.
83Usage::
85 python3 .github/scripts/check_bash_coverage.py coverage/
86 python3 .github/scripts/check_bash_coverage.py coverage/ --report coverage/report
87 python3 .github/scripts/check_bash_coverage.py coverage/.resultset.json
89Exit codes::
91 0 every tracked script is fully covered
92 1 at least one tracked script has uncovered lines or is missing
93 2 the report could not be found or parsed
95The module is importable from the test suite — ``evaluate()`` holds the whole
96decision and takes plain data, so it can be exercised without Ruby or bats.
97"""
99from __future__ import annotations
101import argparse
102import json
103import re
104import subprocess # nosec B404 # fixed argv, no shell: `git ls-files` only
105import sys
106from dataclasses import dataclass, field
107from pathlib import Path
109REPO_ROOT = Path(__file__).resolve().parent.parent.parent
110RESULTSET_NAME = ".resultset.json"
112# A redirection and its target word: an optional descriptor, one of the
113# operators Bash has (including here-strings and ``>|``/``>&``), then a single
114# quoted or bare word. Deliberately not a process substitution (``< <(cmd)``):
115# the command inside one is traced on this line, so the line is measurable.
116_REDIRECTION = r"""\d*(?:<<<|>>|<>|>\||[<>]&?|<)\s*(?:"(?:[^"\\]|\\.)*"|'[^']*'|[^\s()<>|&;]+)"""
118# ``done``, ``fi``, ``esac`` or ``}`` followed by nothing but redirections.
119_TERMINATOR_WITH_REDIRECTIONS = re.compile(
120 rf"^\s*(?:done|fi|esac|\}})(?:\s+{_REDIRECTION})+\s*(?:#.*)?$"
121)
123# A ``case`` arm whose body is empty: ``pattern) ;;``. A ``:`` or any other
124# command in the arm is a traced statement and keeps the line measurable.
125_EMPTY_CASE_ARM = re.compile(r"^\s*[^)#\s][^)#]*\)\s*;;\s*(?:#.*)?$")
128def untraceable_lines(source: str) -> set[int]:
129 """Return the 1-based lines of ``source`` that Bash's tracer never reports.
131 See the module docstring: compound-command terminators carrying only
132 redirections, and empty ``case`` arms. Both are marked executable by
133 bashcov's lexer, so without this they read as permanently uncovered.
134 """
135 return {
136 number
137 for number, line in enumerate(source.splitlines(), start=1)
138 if _TERMINATOR_WITH_REDIRECTIONS.match(line) or _EMPTY_CASE_ARM.match(line)
139 }
142_HEREDOC = re.compile(r"<<-?\s*(?P<quote>['\"]?)(?P<tag>\w+)(?P=quote)")
143_LIST_OPERATOR = re.compile(r"\|\||&&|\|(?!\|)")
146@dataclass(frozen=True)
147class _Scan:
148 """Lexer state carried from one physical line to the next.
150 ``parens`` is the stack of unclosed parentheses; ``True`` marks one that
151 keeps a statement open across lines — ``$(``, an array's ``=(``, a process
152 substitution's ``<(``/``>(``, an arithmetic ``((`` — while ``False`` marks a
153 subshell ``(``, whose inner commands Bash reports on their own lines and
154 which must therefore not be folded.
155 """
157 parens: tuple[bool, ...] = ()
158 quote: str | None = None # "'" or '"' while inside a quoted string
159 heredoc: str | None = None # terminator of the here-document being read
160 heredoc_strip: bool = False # <<- : leading tabs are stripped before comparing
161 continued: bool = False # the line ended with an escaping backslash
163 @property
164 def depth(self) -> int:
165 return sum(1 for spanning in self.parens if spanning)
167 def open(self) -> bool:
168 """Whether the statement is still open at the end of a line."""
169 return (
170 self.depth > 0 or self.quote is not None or self.heredoc is not None or self.continued
171 )
174def _scan_line(line: str, state: _Scan) -> tuple[_Scan, bool]:
175 """Advance ``state`` over one physical line.
177 Returns the new state and whether a list operator (``||``, ``&&``, ``|``)
178 appeared at the top level of this line — i.e. outside quotes and outside
179 any parenthesis — which is where Bash starts a new, separately reported
180 command inside a backslash chain.
181 """
182 parens = list(state.parens)
183 quote, heredoc, heredoc_strip = state.quote, state.heredoc, state.heredoc_strip
184 if heredoc is not None:
185 candidate = line.lstrip("\t") if heredoc_strip else line
186 if candidate == heredoc:
187 heredoc = None
188 return _Scan(tuple(parens), quote, heredoc, heredoc_strip, False), False
190 pending_heredoc: tuple[str, bool] | None = None
191 list_operator = False
192 continued = False
193 index = 0
194 length = len(line)
195 while index < length:
196 char = line[index]
197 if quote == "'":
198 if char == "'":
199 quote = None
200 index += 1
201 continue
202 if quote == '"':
203 if char == "\\":
204 if index == length - 1:
205 continued = True # a backslash-newline inside "..." continues the string
206 index += 2
207 continue
208 if char == '"':
209 quote = None
210 index += 1
211 continue
212 # Unquoted.
213 if char == "\\":
214 if index == length - 1:
215 continued = True
216 index += 2
217 continue
218 if char == "#" and (index == 0 or line[index - 1] in " \t;("):
219 break # comment to end of line
220 if char in "'\"":
221 quote = char
222 index += 1
223 continue
224 if char == "(":
225 parens.append(index > 0 and line[index - 1] in "$=<>(")
226 elif char == ")":
227 if parens:
228 parens.pop() # a `pattern)` case arm has no opener: nothing to pop
229 elif char == "<" and pending_heredoc is None:
230 match = _HEREDOC.match(line, index)
231 if match:
232 pending_heredoc = (match.group("tag"), line[index : index + 3] == "<<-")
233 index = match.end()
234 continue
235 elif char in "|&" and not parens:
236 match = _LIST_OPERATOR.match(line, index)
237 if match:
238 list_operator = True
239 index = match.end()
240 continue
241 index += 1
243 if pending_heredoc is not None:
244 heredoc, heredoc_strip = pending_heredoc
245 return _Scan(tuple(parens), quote, heredoc, heredoc_strip, continued), list_operator
248def statement_spans(source: str) -> list[tuple[int, int]]:
249 """Return ``(first, last)`` line pairs for statements spanning several lines.
251 A statement continues onto the next line while a parenthesis, quote or
252 here-document is open or the line ends with a backslash. Inside a
253 backslash chain a line that starts a new top-level list element (``||``,
254 ``&&``, ``|``) begins a new span, since Bash reports that command on its
255 own line. Single-line statements are not returned.
256 """
257 spans: list[tuple[int, int]] = []
258 state = _Scan()
259 start: int | None = None
260 for number, line in enumerate(source.splitlines(), start=1):
261 was_open = state.open()
262 chain_only = was_open and state.depth == 0 and state.quote is None and state.heredoc is None
263 state, list_operator = _scan_line(line, state)
264 if chain_only and list_operator and start is not None:
265 # `cmd \` / ` arg \` / ` arg || fallback`: the fallback is its own statement.
266 if number - 1 > start:
267 spans.append((start, number - 1))
268 start = number
269 elif not was_open:
270 start = number
271 if not state.open():
272 if start is not None and number > start:
273 spans.append((start, number))
274 start = None
275 if start is not None and state.open():
276 spans.append((start, len(source.splitlines())))
277 return spans
280class ReportError(Exception):
281 """The bashcov report is missing, unreadable or not a SimpleCov resultset."""
284@dataclass
285class ScriptCoverage:
286 """Merged coverage for one tracked script, across all of its copies."""
288 path: str
289 hits: dict[int, int] = field(default_factory=dict)
290 sources: set[str] = field(default_factory=set)
292 @property
293 def measured(self) -> bool:
294 """Whether any reported path mapped onto this script.
296 False means no BATS suite executed it under a traced Bash, which is not
297 the same as "fully covered" — the coverage is simply unknown.
298 """
299 return bool(self.sources)
301 @property
302 def total_lines(self) -> int:
303 return len(self.hits)
305 @property
306 def missed_lines(self) -> list[int]:
307 return sorted(line for line, count in self.hits.items() if count == 0)
309 @property
310 def percent(self) -> float:
311 if not self.hits:
312 return 100.0
313 covered = self.total_lines - len(self.missed_lines)
314 return 100.0 * covered / self.total_lines
317@dataclass
318class Result:
319 """Outcome of a gate evaluation: one record per tracked script."""
321 scripts: list[ScriptCoverage]
322 failures: list[str]
323 unmapped: list[str]
325 @property
326 def ok(self) -> bool:
327 return not self.failures
330def find_report(target: Path) -> Path:
331 """Return the SimpleCov resultset inside ``target``.
333 ``target`` may be the resultset itself or the directory bashcov wrote
334 (``coverage/`` by default).
335 """
336 if target.is_file():
337 return target
338 if not target.is_dir():
339 raise ReportError(f"bashcov output path does not exist: {target}")
340 direct = target / RESULTSET_NAME
341 if direct.is_file():
342 return direct
343 nested = sorted(target.glob(f"**/{RESULTSET_NAME}"))
344 if not nested:
345 raise ReportError(f"no {RESULTSET_NAME} found under {target}")
346 return nested[0]
349def parse_report(report: Path) -> dict[str, dict[int, int]]:
350 """Return ``{reported_path: {line_number: hits}}`` from a SimpleCov resultset.
352 SimpleCov stores one entry per test command, each mapping an absolute file
353 path to a list indexed by line number minus one, where ``null`` marks a
354 line that is not executable and an integer is a hit count. Commands are
355 merged by taking the highest count seen for each line, so a script executed
356 by several suites is credited with all of them.
357 """
358 try:
359 raw = json.loads(report.read_text(encoding="utf-8"))
360 except (OSError, json.JSONDecodeError) as exc:
361 raise ReportError(f"could not parse {report}: {exc}") from exc
362 if not isinstance(raw, dict):
363 raise ReportError(f"{report} is not a SimpleCov resultset (expected an object)")
365 parsed: dict[str, dict[int, int]] = {}
366 for command in raw.values():
367 if not isinstance(command, dict):
368 continue
369 coverage = command.get("coverage")
370 if not isinstance(coverage, dict):
371 continue
372 for filename, entry in coverage.items():
373 if not filename.endswith(".sh"):
374 continue
375 # SimpleCov 1.x nests the array under "lines"; older payloads store
376 # the bare array. Accept both so a gem bump cannot silently zero
377 # the gate.
378 lines = entry.get("lines") if isinstance(entry, dict) else entry
379 if not isinstance(lines, list):
380 continue
381 merged = parsed.setdefault(filename, {})
382 for index, hits in enumerate(lines):
383 if not isinstance(hits, int):
384 continue # null => line is not executable
385 line_number = index + 1
386 merged[line_number] = max(merged.get(line_number, 0), hits)
387 return parsed
390def map_to_tracked(reported_path: str, inventory: list[str]) -> str | None:
391 """Map a path from the report onto the tracked script it is a copy of.
393 Prefers the longest matching path suffix (``/tmp/x/y/demo/lib_demo.sh``
394 maps to ``demo/lib_demo.sh``). Falls back to matching on basename alone,
395 which is safe only while script basenames are unique — a property
396 ``tests/test_check_bash_coverage.py`` asserts against the real inventory.
397 Returns ``None`` when nothing matches.
398 """
399 normalised = reported_path.replace("\\", "/")
400 suffix_matches = [
401 tracked
402 for tracked in inventory
403 if normalised == tracked or normalised.endswith("/" + tracked)
404 ]
405 if suffix_matches:
406 return max(suffix_matches, key=lambda tracked: (tracked.count("/"), len(tracked)))
408 basename = normalised.rsplit("/", 1)[-1]
409 basename_matches = [tracked for tracked in inventory if tracked.rsplit("/", 1)[-1] == basename]
410 if len(basename_matches) == 1:
411 return basename_matches[0]
412 return None
415def fold_spans(hits: dict[int, int], spans: list[tuple[int, int]]) -> dict[int, int]:
416 """Collapse each multi-line statement onto its first line.
418 Every line of a span that the report lists is replaced by the span's first
419 line carrying the highest count seen anywhere in the span, so a statement
420 Bash reported on its second or last line counts once, as covered. A span
421 none of whose lines the report mentions stays absent (bashcov judged it
422 non-executable, e.g. a multi-line comment block would never be a span).
423 """
424 folded = dict(hits)
425 for first, last in spans:
426 members = [line for line in range(first, last + 1) if line in folded]
427 if not members:
428 continue
429 best = max(folded[line] for line in members)
430 for line in members:
431 del folded[line]
432 folded[first] = best
433 return folded
436def evaluate(
437 reported: dict[str, dict[int, int]],
438 inventory: list[str],
439 untraceable: dict[str, set[int]] | None = None,
440 spans: dict[str, list[tuple[int, int]]] | None = None,
441) -> Result:
442 """Merge reported coverage onto the inventory and apply the floor to all of it.
444 ``untraceable`` maps a tracked script to the line numbers that
445 :func:`untraceable_lines` found in it; those lines are dropped from the
446 script's count whatever the report says about them. ``spans`` maps a
447 tracked script to its :func:`statement_spans`, each folded onto its first
448 line by :func:`fold_spans`.
449 """
450 merged: dict[str, ScriptCoverage] = {path: ScriptCoverage(path=path) for path in inventory}
451 unmapped: list[str] = []
452 untraceable = untraceable or {}
453 spans = spans or {}
455 for reported_path, lines in sorted(reported.items()):
456 tracked = map_to_tracked(reported_path, inventory)
457 if tracked is None:
458 unmapped.append(reported_path)
459 continue
460 record = merged[tracked]
461 record.sources.add(reported_path)
462 skipped = untraceable.get(tracked, set())
463 for line_number, hits in lines.items():
464 if line_number in skipped:
465 continue
466 record.hits[line_number] = max(record.hits.get(line_number, 0), hits)
468 for path, record in merged.items():
469 if record.hits and path in spans:
470 record.hits = fold_spans(record.hits, spans[path])
472 scripts = [record for _, record in sorted(merged.items())]
474 failures: list[str] = []
475 for record in scripts:
476 if not record.measured:
477 failures.append(
478 f"{record.path}: absent from the bashcov report — no BATS suite executed it, "
479 "so its coverage is unknown"
480 )
481 continue
482 missed = record.missed_lines
483 if missed:
484 shown = ", ".join(str(line) for line in missed[:20])
485 more = "" if len(missed) <= 20 else f" (+{len(missed) - 20} more)"
486 failures.append(
487 f"{record.path}: {len(missed)}/{record.total_lines} lines uncovered "
488 f"({record.percent:.2f}%): {shown}{more}"
489 )
490 return Result(scripts=scripts, failures=failures, unmapped=unmapped)
493def tracked_shell_scripts(root: Path) -> list[str]:
494 """Return every tracked ``*.sh`` path, excluding the test-suite's own.
496 Uses ``git ls-files`` so generated and untracked scripts never enter the
497 gate, matching how ``lint:shellcheck:shell`` builds its inventory.
498 """
499 try:
500 completed = subprocess.run( # nosec B603 # fixed argv, no shell
501 ["git", "-C", str(root), "ls-files", "*.sh"],
502 capture_output=True,
503 text=True,
504 check=True,
505 )
506 except (OSError, subprocess.CalledProcessError) as exc:
507 raise ReportError(f"could not list tracked shell scripts: {exc}") from exc
508 paths = [line.strip() for line in completed.stdout.splitlines() if line.strip()]
509 return sorted(path for path in paths if not path.startswith("tests/"))
512def classify_scripts(
513 root: Path, inventory: list[str]
514) -> tuple[dict[str, set[int]], dict[str, list[tuple[int, int]]]]:
515 """Run :func:`untraceable_lines` and :func:`statement_spans` over the inventory.
517 Returns the two per-script maps :func:`evaluate` takes, each holding only
518 the scripts that have something to report.
519 """
520 untraceable: dict[str, set[int]] = {}
521 spans: dict[str, list[tuple[int, int]]] = {}
522 for path in inventory:
523 try:
524 source = (root / path).read_text(encoding="utf-8")
525 except OSError as exc:
526 raise ReportError(f"could not read tracked script {path}: {exc}") from exc
527 lines = untraceable_lines(source)
528 if lines:
529 untraceable[path] = lines
530 found = statement_spans(source)
531 if found:
532 spans[path] = found
533 return untraceable, spans
536def format_report(result: Result) -> str:
537 """Render the human-facing summary printed by ``main``."""
538 lines: list[str] = []
539 failing = [record for record in result.scripts if record.missed_lines or not record.measured]
540 covered = len(result.scripts) - len(failing)
541 lines.append(f"bash coverage: {covered}/{len(result.scripts)} tracked scripts at 100%")
542 if result.unmapped:
543 lines.append(
544 f"note: {len(result.unmapped)} reported path(s) matched no tracked script "
545 "and were ignored:"
546 )
547 lines.extend(f" {path}" for path in result.unmapped[:10])
548 if result.failures:
549 lines.append("")
550 lines.append("ERROR: shell scripts are not fully covered:")
551 lines.extend(f" {failure}" for failure in result.failures)
552 lines.append("")
553 lines.append(
554 "Add BATS coverage for the lines above. Every tracked *.sh file is held "
555 "to 100%: a new script ships with a suite that executes it, and a script "
556 "absent from the report needs its suite to run the tracked file in place "
557 "rather than a copy."
558 )
559 return "\n".join(lines)
562# --------------------------------------------------------------------------
563# The published report
564#
565# SimpleCov's own HTML report renders the raw line hits, so it shows the
566# untraceable lines and the later lines of multi-line statements as misses
567# and reports a lower number than the gate does. The report published to
568# GitHub Pages (by pages.yml, from the artifact) is rendered here instead,
569# from the same corrected data the verdict comes from, so the badge, the
570# report and the gate cannot tell three different stories.
571# --------------------------------------------------------------------------
573SUMMARY_NAME = "summary.json"
575_REPORT_CSS = """
576body { font: 14px/1.4 -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif; margin: 2em auto; max-width: 72em; padding: 0 1em; color: #222; }
577h1 { font-size: 1.5em; } h1 code { font-size: 0.9em; }
578table { border-collapse: collapse; width: 100%; }
579th, td { text-align: left; padding: 0.25em 0.6em; border-bottom: 1px solid #ddd; }
580th { background: #f3f3f3; } td.num { text-align: right; font-variant-numeric: tabular-nums; }
581.ok { color: #1a7f37; } .bad { color: #b42318; } .muted { color: #666; }
582table.source { font: 12px/1.45 ui-monospace, SFMono-Regular, Menlo, monospace; }
583table.source td { border: 0; padding: 0 0.6em; white-space: pre; }
584table.source td.n { text-align: right; color: #888; user-select: none; width: 3em; }
585table.source td.c { text-align: right; color: #666; width: 3em; }
586tr.covered td.s { background: #dafbe1; } tr.missed td.s { background: #ffebe9; }
587tr.continued td.n::after { content: " \\2026"; }
588tr.untraceable td.s { background: #f3f3f3; color: #666; }
589.legend span { display: inline-block; padding: 0 0.5em; margin-right: 0.6em; }
590.legend .covered { background: #dafbe1; } .legend .missed { background: #ffebe9; } .legend .untraceable { background: #f3f3f3; }
591"""
594def _escape(text: str) -> str:
595 return text.replace("&", "&").replace("<", "<").replace(">", ">")
598def _page_name(path: str) -> str:
599 """The file page for a script: the path with ``/`` doubled into ``__``.
601 A leading dot is dropped so ``.github/scripts/x.sh`` does not become a
602 hidden file on the site. Basenames are unique across the inventory (a test
603 asserts it), so no two scripts share a page.
604 """
605 return path.replace("/", "__").lstrip(".") + ".html"
608def line_states(
609 source: str,
610 hits: dict[int, int],
611 untraceable: set[int],
612 spans: list[tuple[int, int]],
613) -> list[tuple[str, int | None, str]]:
614 """Classify every physical line of a script for its file page.
616 Returns one ``(kind, count, text)`` per line. ``kind`` is ``covered`` or
617 ``missed`` for a measured statement, the same with ``continued`` added for
618 the later physical lines of a multi-line statement (they take their
619 statement's fate — ``hits`` is the folded map, so only the first line
620 carries the count), ``untraceable`` for a line Bash never traces, and
621 ``none`` for a line that is not executable.
622 """
623 continued: dict[int, int] = {}
624 for first, last in spans:
625 if first in hits: # folded onto its first line, so the statement is measured
626 for line in range(first + 1, last + 1):
627 continued[line] = first
628 states: list[tuple[str, int | None, str]] = []
629 for number, text in enumerate(source.splitlines(), start=1):
630 if number in untraceable:
631 states.append(("untraceable", None, text))
632 elif number in hits:
633 states.append(("covered" if hits[number] else "missed", hits[number], text))
634 elif number in continued:
635 count = hits[continued[number]]
636 states.append((("covered" if count else "missed") + " continued", None, text))
637 else:
638 states.append(("none", None, text))
639 return states
642def _percent(covered: int, total: int) -> float:
643 return round(100.0 * covered / total, 2) if total else 0.0
646def summarize(result: Result) -> dict[str, object]:
647 """The machine-readable summary the badge is rendered from."""
648 files = []
649 statements = covered = 0
650 for record in result.scripts:
651 missed = record.missed_lines
652 statements += record.total_lines
653 covered += record.total_lines - len(missed)
654 files.append(
655 {
656 "path": record.path,
657 "measured": record.measured,
658 "statements": record.total_lines,
659 "covered": record.total_lines - len(missed),
660 "missed": missed,
661 "percent": _percent(record.total_lines - len(missed), record.total_lines),
662 }
663 )
664 at_floor = [record for record in result.scripts if record.measured and not record.missed_lines]
665 return {
666 "ok": result.ok,
667 "scripts": len(result.scripts),
668 "scripts_at_100": len(at_floor),
669 "unmeasured": [record.path for record in result.scripts if not record.measured],
670 "statements": statements,
671 "covered": covered,
672 "missed": statements - covered,
673 "percent": _percent(covered, statements),
674 "files": files,
675 }
678def _file_page(record: ScriptCoverage, states: list[tuple[str, int | None, str]]) -> str:
679 stats: str
680 if not record.measured:
681 stats = '<p class="bad">Not executed by any suite, so its coverage is unknown.</p>'
682 else:
683 missed = len(record.missed_lines)
684 klass = "ok" if not missed else "bad"
685 stats = (
686 f'<p class="{klass}">{record.total_lines - missed} of {record.total_lines} '
687 f"statements covered ({_percent(record.total_lines - missed, record.total_lines):.2f}%)"
688 f"{'' if not missed else f', {missed} missed'}.</p>"
689 )
690 rows = []
691 for number, (kind, count, text) in enumerate(states, start=1):
692 shown = "" if count is None else str(count)
693 rows.append(
694 f'<tr class="{kind}"><td class="n">{number}</td><td class="c">{shown}</td>'
695 f'<td class="s">{_escape(text) or " "}</td></tr>'
696 )
697 legend = (
698 '<p class="legend"><span class="covered">covered</span>'
699 '<span class="missed">missed</span>'
700 '<span class="untraceable">never traced by Bash (not counted)</span>'
701 "A line ending in \u2026 continues the statement above it and shares its fate.</p>"
702 )
703 return (
704 '<!DOCTYPE html>\n<html lang="en"><head><meta charset="utf-8">'
705 f"<title>{_escape(record.path)} \u2014 shell coverage</title>"
706 f"<style>{_REPORT_CSS}</style></head><body>\n"
707 f'<p><a href="../index.html">\u2190 all scripts</a></p>\n'
708 f"<h1><code>{_escape(record.path)}</code></h1>\n{stats}\n{legend}\n"
709 '<table class="source">\n' + "\n".join(rows) + "\n</table>\n</body></html>\n"
710 )
713def _index_page(summary: dict[str, object]) -> str:
714 files = summary["files"]
715 assert isinstance(files, list)
716 rows = []
717 for entry in files:
718 link = f'<a href="files/{_page_name(entry["path"])}">{_escape(entry["path"])}</a>'
719 if not entry["measured"]:
720 rows.append(
721 f"<tr><td>{link}</td>"
722 '<td class="bad" colspan="4">not executed by any suite</td></tr>'
723 )
724 continue
725 klass = "ok" if not entry["missed"] else "bad"
726 rows.append(
727 f"<tr><td>{link}</td>"
728 f'<td class="num">{entry["statements"]}</td>'
729 f'<td class="num">{entry["covered"]}</td>'
730 f'<td class="num">{len(entry["missed"])}</td>'
731 f'<td class="num {klass}">{entry["percent"]:.2f}%</td></tr>'
732 )
733 klass = "ok" if summary["ok"] else "bad"
734 verdict = (
735 f'<p class="{klass}"><b>{summary["scripts_at_100"]}/{summary["scripts"]} tracked scripts '
736 f"at 100%</b> \u2014 {summary['covered']} of {summary['statements']} statements covered "
737 f"({summary['percent']:.2f}%).</p>"
738 )
739 return (
740 '<!DOCTYPE html>\n<html lang="en"><head><meta charset="utf-8">'
741 "<title>Shell coverage</title>"
742 f"<style>{_REPORT_CSS}</style></head><body>\n"
743 "<h1>Shell coverage</h1>\n"
744 f"{verdict}\n"
745 '<p class="muted">Every tracked shell script, measured in statements by the '
746 "<code>unit:bats:shell</code> job: BATS runs each script under bashcov, and "
747 "<code>check_bash_coverage.py</code> corrects the raw line hits for the lines Bash "
748 "never traces and for statements that span several lines. This page is that "
749 "corrected view, and the same numbers are what the gate enforces.</p>\n"
750 "<table><thead><tr><th>Script</th><th>Statements</th><th>Covered</th>"
751 "<th>Missed</th><th>Coverage</th></tr></thead>\n<tbody>\n"
752 + "\n".join(rows)
753 + "\n</tbody></table>\n</body></html>\n"
754 )
757def write_report(
758 result: Result,
759 root: Path,
760 untraceable: dict[str, set[int]],
761 spans: dict[str, list[tuple[int, int]]],
762 out_dir: Path,
763) -> dict[str, object]:
764 """Write ``index.html``, one page per script and ``summary.json`` to ``out_dir``.
766 Rendered from the evaluated result, so the numbers agree with the verdict
767 ``main`` prints. Returns the summary that was written.
768 """
769 summary = summarize(result)
770 files_dir = out_dir / "files"
771 files_dir.mkdir(parents=True, exist_ok=True)
772 for record in result.scripts:
773 source = (root / record.path).read_text(encoding="utf-8")
774 states = line_states(
775 source, record.hits, untraceable.get(record.path, set()), spans.get(record.path, [])
776 )
777 (files_dir / _page_name(record.path)).write_text(
778 _file_page(record, states), encoding="utf-8"
779 )
780 (out_dir / "index.html").write_text(_index_page(summary), encoding="utf-8")
781 (out_dir / SUMMARY_NAME).write_text(json.dumps(summary, indent=2) + "\n", encoding="utf-8")
782 return summary
785def main(argv: list[str] | None = None) -> int:
786 parser = argparse.ArgumentParser(description=__doc__.split("\n", 1)[0])
787 parser.add_argument(
788 "bashcov_output",
789 type=Path,
790 help="bashcov output directory (coverage/), or a .resultset.json directly.",
791 )
792 parser.add_argument(
793 "--root",
794 type=Path,
795 default=REPO_ROOT,
796 help="Repository root used to build the tracked-script inventory.",
797 )
798 parser.add_argument(
799 "--report",
800 type=Path,
801 default=None,
802 metavar="DIR",
803 help=(
804 "Also write the statement-level HTML report (index.html, files/*.html) and "
805 f"{SUMMARY_NAME} to DIR — the view pages.yml publishes. Written whatever the "
806 "verdict, so a failing run can be inspected."
807 ),
808 )
809 args = parser.parse_args(argv)
811 try:
812 report = find_report(args.bashcov_output)
813 reported = parse_report(report)
814 inventory = tracked_shell_scripts(args.root)
815 untraceable, spans = classify_scripts(args.root, inventory)
816 except ReportError as exc:
817 print(f"ERROR: {exc}", file=sys.stderr)
818 return 2
820 result = evaluate(reported, inventory, untraceable, spans)
821 if args.report is not None:
822 write_report(result, args.root, untraceable, spans, args.report)
823 print(format_report(result))
824 return 0 if result.ok else 1
827if __name__ == "__main__":
828 sys.exit(main())