Building an LLM Council: Stress-Test Startup Ideas

A few weeks ago, a waste management startup team in Ghana was completely stuck after spending weeks circling the same concepts, including gamified recycling rewards and route optimization, without reaching any consensus. While the team was full of enthusiasm, they lacked a reliable way to critically test their own thinking before committing another month to building the wrong product.

This represents a common failure point for founders and engineers: opening ChatGPT, asking whether an idea is good, and immediately receiving three paragraphs of validation. This phenomenon happens because single large language models (LLMs) are trained to provide the exact responses users want to hear, finding something positive to say about almost everything and rarely pointing out fundamental flaws.

The deadlock broke when the team ran their disagreements through a structured multi-model critique workflow known as an "LLM Council," modeled on an approach popularized by Andrej Karpathy, former Director of AI at Tesla and OpenAI founding member. Each model adopted a skeptical persona—such as an investor, a security engineer, a frustrated customer, or a competitor—to expose crucial trade-offs that are easily missed by a single agent.