Pawscript: Turn Your Dog's Photos into Letters to You
Developed for the DEV Weekend Challenge, Pawscript uses Gemini multimodal AI and optional audio to write personalized letters from a dog's perspective.
Stock photo for illustration only, not from the actual event
- Pawscript generates personalized letters from your dog based on uploaded photos.
- Powered by Gemini multimodal AI to analyze both visual details and optional audio.
- Enforces structured JSON output so results are grounded in the specific photo.
- Prioritizes user privacy by not storing any uploaded photos or audio files.
A software developer has introduced Pawscript as a submission for the DEV Weekend Challenge: Dog Days Edition. The application allows pet owners to upload a photo of their dog, optionally accompanied by a bark, whine, or sigh, prompting the system to generate a warm, short letter written directly in the dog's own voice.
The primary goal behind this project was to bypass the typical pattern seen in most pet applications that simply default to uploading a photo and receiving a random label. Instead, Pawscript is designed to be genuinely grounded in the specific photo provided, meaning the exact same dog scanned twice will produce genuinely different letters rather than a recycled, copy-paste result.
Stock photo for illustration only, not from the actual event
The development journey began with a preferred user interface featuring a five-paw scanning progress meter equipped with a breathing and pulse animation. The underlying logic was then rebuilt to function in reality rather than serving as a mere decorative placeholder, tying the scanning progress animation directly to the actual fetch lifecycle.
Leveraging multimodal AI capabilities such as Gemini to fuse visual and acoustic data represents an innovative approach in pet-focused software engineering. By allowing models to interpret acoustic tones alongside visual cues, developers can create deeply immersive applications that bridge the gap between human interpretation and artificial intelligence in consumer tech.
At the core of the application is a single multimodal call sent to Gemini, containing the uploaded image and inline base64 audio. System instructions require the model to reference specific visible details—such as coat, posture, setting, and expression—preventing generic responses. Furthermore, the system enforces a strict structured JSON output comprising letter, mood_tag, observations, and used_audio fields to streamline UI parsing.
The trickiest aspect of the build involved handling the optional audio path, instructing Gemini to fuse acoustic tones like a bright yip or low grumble into the letter while strictly avoiding invented sound characteristics when no audio is provided. Additionally, privacy remains a core focus: nothing uploaded is stored, as photos and audio go straight to Gemini for analysis and are never retained.
Source: Dev.to
Found something wrong in this article? Report an issue with this article
Comments
Leave a Comment