Fly Language Model (FLM): Fruit Fly Connectome Meets LLM
Developers wire the full MaleCNS fruit fly connectome into a frozen 1.2B language model, with controls showing the biological wiring provides no specific gain.
Stock photo for illustration only, not from the actual event
- Fly Language Model (FLM) integrates the full MaleCNS fruit fly connectome into a 1.2B parameter LLM.
- The system utilizes reservoir computing with only 278,528 readout parameters trained.
- Control experiments reveal that the biological neural wiring does not outperform random configurations.
Artificial intelligence researchers have introduced an intriguing open-source project named the Fly Language Model, or FLM, which wires a complete biological connectome into a frozen language model. Deployable locally via the nftechie/flm repository on GitHub under an MIT license, the system operates on Python 3.12 across macOS or Linux using MPS, CUDA, or CPU hardware without requiring any API key.
The system architecture functions as a reservoir computer bolted directly onto a language model. All 166,700 retained nodes and 25,582,938 directed edges from the MaleCNS fruit fly graph participate in the setup. The graph backbone, along with random input and output projections, remains entirely fixed. Only a 278,528-parameter readout interface is trained, accounting for roughly 0.0238% of the total 1,170,340,608 backbone parameters.
Stock photo for illustration only, not from the actual event
At each token generation step, a fixed Gaussian projection compresses the 2,048-dimensional token embedding down to 128 channels. Each reservoir node receives a single channel paired with a random sign. The entire graph subsequently updates its states through mathematical functions, passing through bias-free matrices and projecting through a frozen vocabulary head as a bounded residual capped at an RMS of 0.25.
Evaluated across a freshly frozen set of 32 SmolTalk everyday-conversation dialogues consisting of 1,236 target tokens across three fit seeds, the fly readout improved upon the baseline backbone by 0.0222 nats per token, shifting perplexity from 3.98 to 3.90. However, a direct-input control experiment—which routed the identical 128-channel projection straight into an identical readout without any graph—outperformed the biological wiring across all three seeds by 0.000488 nats per token.
"The paired bootstrap interval does not support a fly-specific gain from the biological connectome wiring."
Fly Language Model Research Report
Additional control tests yielded critical insights into the system's mechanics. Setting the weight matrix to zero precisely removed the residual, confirming active graph participation. Conversely, shuffling node identities without retraining returned a negative log-likelihood near baseline levels, indicating that the readout depends on its learned interface alignment rather than superior fly topology.
Combining biological neural networks with modern artificial intelligence architectures, often explored within computational neuroscience and Neuro-AI, aims to investigate whether millions of years of evolutionary wiring can enhance artificial language capabilities. Although FLM demonstrates that a raw connectome does not inherently surpass random baselines in perplexity metrics, it establishes a valuable open-source framework for exploring hybrid architectures with rigorous controls.
The research report further notes that recurrence contracts initial-state differences quickly, proving that piling in hundreds of thousands of cells does not inherently purchase long-term memory, which remains managed by the backbone. Building upon earlier prototypes like ngxson/fly-hf, FLM's primary distinctions lie in its full-scale graph implementation and frozen-backbone design.
Source: MarkTechPost
Found something wrong in this article? Report an issue with this article
Comments
Leave a Comment