Building MindMap Debugger: Fixing Bugs in 3 Days
A deep dive into building MindMap Debugger for AWS First Commit under #WeMakeDevs, fixing reasoning model leaks and consensus extraction challenges.
Stock photo for illustration only, not from the actual event
- MindMap Debugger extracts propositions to detect contradictions and loops.
- The developer utilized the gpt-oss-120b model via Groq's API.
- Addressed model reasoning output leaks using the extra_body parameter.
- Overcame non-deterministic outputs by implementing consensus extraction.
Careful reading does not scale well in modern fast-paced environments where meeting transcripts often contain conflicting timelines that go unnoticed because individuals focus only on their specific domains. MindMap Debugger was created to address this exact friction by ingesting pasted text or transcripts, extracting core claims and relational ties, and executing detection logic to highlight contradictions and circular arguments.
For the model backend, the developer relied on Groq due to its free tier requiring no credit card, bypassing AWS account restrictions with UPI. To satisfy the AWS-native tooling requirement while using Groq outside Bedrock, the integration wrapped Groq calls inside the AWS Strands Agents SDK utilizing an OpenAI-compatible endpoint.
Stock photo for illustration only, not from the actual event
On the first day, an immediate bug emerged: the reasoning model gpt-oss-120b leaked its internal chain-of-thought straight into the output instead of clean JSON. This resulted in a wall of internal monologue hiding the actual structured response deep underneath.
"the deadline is fixed" vs "the deadline cannot move"
An example of paraphrased duplicates handled by the merge logic
The resolution required passing reasoning_format=hidden and reasoning_effort=high via the extra_body parameter in Strands, as standard response_format parameters failed silently. By the end of Day 1, the pipeline successfully extracted claims and surfaced basic contradictions in a rough user interface.
Developing natural language processing tools frequently exposes the non-deterministic nature of large language models. Implementing consensus extraction serves as a standard engineering pattern to mitigate variance, though it introduces secondary challenges regarding paraphrased duplication and edge-case merging.
Day 2 introduced reliability testing by running identical inputs five consecutive times, revealing fluctuating findings between 2 and 6 outputs due to Groq's non-deterministic serving. To counter this, a consensus extraction method ran the same extraction three times before merging. However, merging noisy outputs spawned additional bugs regarding unmerged paraphrased duplicates, blind spots in cycle detection, and duplicate findings.
The most critical flaw was discovered only after reviewing raw console logs from pipeline.py during edge-case testing. Witness-testimony test cases revealed that propositions sharing common English scaffolding were improperly collapsed into single claims because the old similarity formula divided overlap by the smaller set's size. Switching to Jaccard similarity resolved this by dividing overlap by the union of both sets.
Source: Dev.to
Found something wrong in this article? Report an issue with this article
Comments
Leave a Comment