The purpose of this thread is specifically to provide a place to discuss the many ways in which AI
™ is harmful to humans and human societies. Please refrain from tangental discussions such as the merits or validity of AI
™.
My own experience interacting with those who reference or rely on AI
™ generated text strings as if they had well constrained information content, or truth value, has been nearly all negative, sometimes frightening.
Today I found
this article which very much reflects conclusions drawn from personal interactions (including some discussions here on LQ) but is backed by measurble data from
this study. From the article:
Quote:
Researchers found AI advice suppressed judgment suspension from 44% to 3%, accuracy from 27% to 9%, while confidence rose from 30% to 76%. People trusted wrong AI answers.
Researchers from three French and Italian universities found that access to AI advice collapsed people’s willingness to say “I don’t know” from 44% to 3%. Accuracy dropped from 27% to 9%. Confidence, meanwhile, rose from 30% to 76%. “People became much worse, the accuracy was only one third, but they were twice as confident,” said Valerio Capraro, associate professor at the University of Milano-Bicocca.
|
The article provided a name for a phenomena I have witnessed repeatedly, "Cognitive Surrender":
Quote:
|
...researchers coined the term “cognitive surrender” earlier this year to describe the same phenomenon: people accepting incorrect AI answers 80% of the time while reporting higher confidence than those working without AI. The new study adds a sharper data point. It is not just that people trust wrong AI answers. It is that the mere availability of AI suppresses the cognitive habit of recognising what you do not know.
|
The whole basis of human societies consists of our interactions, communications and exchange of knowledge and concepts.
Any widely used source (i.e.
authority) or tool which simultaneously reduces accuracy of our shared knowledge by 2/3 while effectively removing our willingness to judge for ourselves and say we don't know, instilling unfounded confidence in the incorrect knowledge unavoidably does major harm to the societies within which they operate, in proportion to their acceptance.
