ByteDance has introduced Utars 1.5, an advanced vision-language agent capable of perceiving and interacting with graphical user interfaces (GUIs) across various platforms like Windows, Android, and web browsers. Unlike previous models that relied on external tools or complex prompting, Utars 1.5 processes the entire screen as an image and uses a single neural network for perception, planning, and low-level actions such as clicking, typing, and dragging. The agent was trained on extensive datasets including screenshots, GUI tutorials, and recorded action traces, developing both rapid, intuitive System One thinking and more deliberate, analytical System Two reasoning. Benchmarks show Utars 1.5 outperforming earlier agents like OpenAI's Operator and Claude on diverse tasks, demonstrating particular strength in complex GUI navigation and grounding. A key aspect is ByteDance's release of a 7B parameter model under an Apache 2.0 licence, making this powerful technology accessible for research and commercial use, facilitating adaptation to specific or custom interfaces.
Source :
AI Revolution
YouTube Channel