According to a study, artificial intelligence would be unable to summarize the news
One might think that artificial intelligence has already reached a sufficient level for certain tasks to be a formality. Among them, the ability to summarize the news: this function is even considered as an argument for the sale of many models. For example, an Openai spokesperson was able to declare, about this function: “We support publishers and creators by helping 300 million weekly chatgpt users to discover quality content thanks to summaries, quotes, clear links and attributions“However, the BBC has looked more closely at it, by studying several models of artificial intelligence.
One summary in two per AI would be false factually
In this study, the BBC asked Chatgpt, Copilot, Gemini and Perplexity to summarize 100 news articles and evaluated each response. Various journalists in the field then assessed the quality of the responses of artificial intelligence assistants. Thus, according to this study, 51% of the answers of AI to questions on the news presented significant problems in one form or another. In parallel, 19% of summaries even contained factual errors (dates, figures or erroneous declarations).
As examples of errors, the BBC indicates that Chatgpt And Copilot said Rishi Sunak and Nicola Sturgeon were still in office when they had already left their duties, that Gemini wrongly said that the NHS did not recommend the electronic cigarette as a help to quit smoking. In general, Microsoft and Gemini from Google Copilot have encountered more problems than Openai and Perplexity Chatgpt. The main problem revealed by this study is that chatbots found it difficult to distinguish opinion and made, made editorial interpretations and often omitted from the essential context, in addition to containing factual errors. Note also that, if 51% of summaries had errors, this implies that 49% offered a satisfactory response: in a few years, this figure should largely increase. In the meantime: it remains essential to remain vigilant, the AI not being the perfect product without any margin of error that certain companies seem to want to present.
