Perplexity Introduces Photon Rust Retrieval Engine
Perplexity launches Photon, a Rust-based retrieval engine that slashes p99 latency from 800 ms to 65 ms, available as a hosted API.
Stock photo for illustration only, not from the actual event
- Perplexity has introduced Photon, a new Rust-based retrieval engine
- Cuts p99 search latency dramatically from 800 ms down to 65 ms
- Available as a hosted API priced at $1 per 1,000 requests
Perplexity has rolled out Photon, a brand new retrieval engine built from scratch using the Rust programming language designed to overcome previous scaling bottlenecks. The development team concluded that building a fresh engine was simpler and far more cost-effective than continuing to maintain their existing fork as the index grew larger over time.
Photon is deployed specifically as a hosted API. Developers can utilize the engine by configuring search_type: "fast" within a POST /search request, with pricing set at $1 per 1,000 requests. It is worth noting that Photon is proprietary software and not open source, meaning self-hosting the engine is not an option for users.
Under the hood, a load balancer directs each incoming query to a Photon broker. The broker fans out the workload to a shard group while actively monitoring for any timeouts. Each shard processes retrieval, initial ranking, and secondary ranking stages, after which the broker merges the candidates and fetches key document fields. Consequently, a complete web index can now be built within single-digit hours.
Stock photo for illustration only, not from the actual event
Additionally, Perplexity paired Photon with a lighter ranking mechanism tailored for agentic workflows under a feature called Fast Search. Testing was conducted across 6 distinct benchmarks:
- WideSearch
- BrowseComp
- DSQA
- FRAMES
- SEAL-0
- SEAL-Hard
Across 3,554 evaluated tasks, Fast Search achieved a 64.3% score at an estimated model-plus-search cost of $59.73. In comparison, the default preset scored 64.0% at $187.60, making the Fast configuration roughly 68% cheaper. However, this speed comes with trade-offs in broader search quality; internal long-tail relevance scores (DCG) dropped from 2.45 to 2.21, and answer availability decreased by 2.9 percentage points from 0.596 to 0.567.
The adoption of Rust for Photon highlights a growing industry preference for systems programming languages that offer memory safety without garbage collection overhead. Sparing milliseconds in retrieval latency is critical for agentic AI loops where autonomous systems must execute dozens of iterative search queries in rapid succession.
Source: MarkTechPost
Found something wrong in this article? Report an issue with this article
Comments
Leave a Comment