appTutorial: On-Screen AI Windows Tutor for Apps
Developer releases appTutorial, a local Windows AI tool powered by Qwen2.5-VL to guide users through complex software tasks step by step.
Stock photo for illustration only, not from the actual event
- appTutorial is a local Windows AI assistant powered by the Qwen2.5-VL open-weight vision model without cloud reliance.
- Users press a hotkey on any confusing app to receive step-by-step on-screen guidance and live verification.
- This project was created as a submission for Week 2 of the Hacktoberfest Open-Source AI Challenge.
Navigating complicated applications or unfamiliar software often presents a steep learning curve for many users. While searching the web for solutions remains an option, it is typically inconvenient and lacks the real-time interactivity needed during active workflows.
To address this challenge, a developer has introduced appTutorial, an on-screen AI software tutor designed for Windows. Running entirely on a local open-weight vision model via Ollama, the application operates without cloud AI dependencies or API keys.
When a user encounters a confusing interface—such as Excel, an Integrated Development Environment, or a web application—pressing a hotkey triggers the system to capture the active window. The tool then identifies the software and the user's intent, walking them through tasks step by step while verifying progress against the live screen and highlighting focal points.
Running AI models locally instead of relying on external cloud servers significantly enhances data privacy and security, making it highly advantageous when handling sensitive documents or proprietary software workflows.
For setup and installation, developers and users can follow these steps:
- Install Ollama from ollama.com
- Pull the model using the command:
ollama pull qwen2.5vl - Clone the repository via
git clone https://github.com/codename-ryan/appTutorial.gitand change directory tocd appTutorial - Set up a virtual environment with
python -m venv .venvand activate it using.venv\Scripts\Activate.ps1 - Install dependencies with
pip install -e . - Run the application using
python -m apptutorial.app
Regarding its current development status, version 1 has been successfully built, with Microsoft Excel serving as the most thoroughly tested software. Full technical specifications are documented in the project's SPEC.md file.
The project utilizes the Qwen2.5-VL open-weight model to identify applications, plan tutorials, and recognize user actions. Ollama hosts the model, exposes an endpoint, and applies JSON-schema-constrained decoding to generate structured outputs.
Furthermore, the creator consulted close friends and acquaintances to test the application. Their feedback yielded overwhelmingly positive reviews and commitments to utilize the tool for streamlining their daily work routines.
Source: Dev.to
Found something wrong in this article? Report an issue with this article
Comments
Leave a Comment