My Commit-Message Script Has 8 Assertions in --selftest. None of Them Touch the Code That Can Actually Fail.
A personal code review reveals that a commit message generation script powered by Claude AI has complete selftest blocks, but none of them actually exercise the real failure paths.
Stock photo for illustration only, not from the actual event
- Discovered test coverage gaps in git_commit.py after reviewing code practices inspired by a trending post.
- Sibling files publish_devto.py and server.py properly stub and test failure branches.
- git_commit.py runs as a top-level script, preventing easy function calls with mocked dependencies.
- The proper fix requires refactoring logic into a function and mocking subprocess.check_output like the other files.
There are three files in this repository that shell out over the network or call a subprocess, carrying the potential to fail in interesting ways: publish_devto.py, server.py, and git_commit.py. Two of these files contain --selftest blocks that stub out risky network calls and actively exercise the actual failure branches.
One file lacks this treatment, a fact the author only noticed after seeking reasons to be suspicious of their own test coverage following a trending post about counting assertions in test suites and not liking what they found.
The script git_commit.py reads a staged diff and calls claude -p to turn it into a commit message. It features five distinct exit paths, all guarding real failure modes previously encountered in the project.
Comparing this to the --selftest blocks in the other two files written during the same week reveals a different standard was applied. For instance, publish_devto.py stubs urllib.request.urlopen directly to drive its failure paths:
def _fake_url_error(req, timeout=30): raise urllib.error.URLError("timed out")
Stock photo for illustration only, not from the actual event
Writing self-tests for utility scripts is frequently neglected by developers who view them as temporary helper tools. However, scripts interacting with external CLI tools and APIs are inherently fragile. Implementing stubs and mocks to simulate real errors effectively prevents unexpected bugs, as demonstrated in the publish_devto.py and server.py modules.
The core issue with git_commit.py is its architectural structure. It consists of roughly twenty lines of top-level script executing everything sequentially—reading diffs, calling claude -p, and printing results at module scope—guarded only by an early check for if "--selftest" in sys.argv: ... raise SystemExit(0) at the top. This leaves no function available to call with a stubbed subprocess.check_output without refactoring.
The honest fix is a two-part process: wrapping the diff-read-and-generate logic inside a callable function matching its sibling files, and applying the same stub-and-assert treatment utilized by already_published(), _gh(), and _dev().
Source: Dev.to
Found something wrong in this article? Report an issue with this article
Comments
Leave a Comment