A recent experiment by AI21 Labs, a competitor of OpenAI, showed that many people couldn’t tell whether they were conversing with a human or an AI robot during an online game called ‘Human or Not’. The competition tested players’ ability to chat with an AI bot for two minutes, using top language models like OpenAI’s GPT-4 and Jurassic AI21 Labs-2. The company claimed this game was the biggest Turing test, generating over a million conversations. The data collected from this experiment is valuable for developing hypotheses. According to the findings, it was observed that individuals were more proficient in identifying their category when compared to automated bots. During interactions with humans, the participants accurately determined their category 73% of the time, whereas, in interactions with bots, they were successful only 60% of the time.
Some insights were highlighted regarding the differentiation between human and AI-powered interactions, specifically about personal questions and language usage. Recent research has shown that people tend to assume that AI-powered assistants do not make grammatical mistakes, typos, or use slang, even though some models are programmed to do so. In addition, personal questions are often used to distinguish between human and AI-powered interactions, as the latter is believed to lack personal history or background. However, it is essential to note that AI-powered assistants can answer individual questions due to their training in personal stories. These findings highlight the need for further education and awareness regarding the capabilities and limitations of AI-powered assistants in human interactions.
After a two-minute conversation, participants were tasked with identifying whether the conversational partner was a human or an AI-powered assistant. The results, obtained after a month of gameplay and millions of exchanges, revealed that 32% of individuals could not differentiate between the two. Intriguingly, some participants surmised that if the other party displayed excessive politeness, they would likely communicate with a machine, raising philosophical questions. It is worth noting that the “human or AI” objective is not limited to a mere game.AI should enhance productivity, impart knowledge, and offer practical experiences through communication. Furthermore, with the growing presence of AI robots in our digital world, individuals must grasp their potential and utilize them prudently.
AI21 Labs has been utilizing games to educate individuals on AI, which has been a successful means of teaching in the past. One example was the release of ‘Ask Ruth Bader Ginsburg’. This AI model utilized Ginsburg’s extensive legal work, press interviews, and public appearances to predict how she would answer questions. Meron, the creator of ‘Human or AI,’ considers it to be an advanced version of this game and was pleased with the results, which were not unexpected. As per his statement, there was an assumption that specific individuals could not distinguish between them. It was, however, surprising how valuable insight was obtained about human behaviour.
Interestingly, people tend to assume harmful intentions behind online actions. Additionally, people have been experimenting with bots like that of a service. Therefore, politicians must consider this issue as the ability of humans to differentiate between real individuals and artificial intelligence must be questioned. Meron stressed the importance of knowledge to prevent misuse by malicious actors. In addition, he emphasized the need for individuals to comprehend the immense potential of this technology. A spokesperson has conveyed that there is no need to be suspicious of bots on the internet. However, they have highlighted the potential risk of phishing attacks on individuals and those inclined towards online activities. The game in question, though, does not address the vulnerability issue. Instead, it is suggested that policymakers consider the importance of companies disclosing the use of AI agents in their services. This is an important topic that needs to be discussed, as AI capabilities may become a standard feature in several products in 2023.
