Turing Test on Screen: a Benchmark for Mobile GUI Agent Humanization

The rise of autonomous GUI agents has triggered a cat-and-mouse game with digital platforms, seeking to detect and prevent their use. While existing research focuses on the utility and robustness of these agents, a critical aspect often overlooked is their ability to avoid detection. In a recent paper, researchers propose a new benchmark for mobile GUI agents, inspired by the Turing Test. This benchmark aims to measure the humanization of these agents, evaluating their ability to deceive human users into believing they are interacting with a real person. The authors argue that this is a crucial aspect of agent development, as undetectable agents are more likely to succeed in their intended goals. The proposed benchmark consists of a set of tasks and evaluation metrics designed to assess the human-likeness of GUI agents. The researchers demonstrate the effectiveness of their approach through experiments with a range of GUI agents, highlighting the potential of this benchmark to improve the development of more sophisticated and undetectable agents. As the use of GUI agents continues to grow, this research has significant implications for the future of human-agent interaction.