In the last one or two years, companies have begun to use so-called «AI agents» as normal «employees», including them in their organizational charts.
Emma Wiles, a professor at Boston University who studies how artificial intelligence affects workers, accidentally discovered this phenomenon in October, at a conference where two human resources executives stated that treating artificial intelligence agents as real employees was a way to increase productivity and to place their companies at the cutting edge of technology.
But when Dr. Wiles and her three Boston Consulting Group associates investigated further, they discovered a trap, according to The New York Times.
In an experiment involving dozens of companies with «employees» artificial intelligence, researchers found that directors tended to check documents with less attention when they were told that they had been created by a «official» artificial intelligence.
The directors did not detect mistakes other directors had detected when they were told they controlled a man’s work.
Dr. Wiles assumed that directors did not consider that detecting errors made by artificial intelligence employees was their responsibility.
If something went wrong, they could attribute it as a mistake by the tech team or executives who wanted artificial intelligence employees in the first place.
«But it's not your problem.», he said, expressing the directors' mindset about their own roles.
A new technology with many flaws
In the years since artificial intelligence appeared dynamically, many companies have realized the flaws created by technology and, at times, have taken steps to compensate for them.
They know that artificial intelligence models can be biased against certain groups of people, such as non-whites.
They know that chatbots can provide sure but incorrect answers to questions.
They know that bots sometimes reveal information that should remain private.
However, as companies rush to integrate artificial intelligence into their daily functions, researchers discover more subtle imperfections.
In principle, these imperfections could also be corrected. For example, companies could make directors directly responsible for the mistakes of existing artificial intelligence.
In practice, however, most corporate users seem to completely ignore these issues, which increases the possibility of undermining the promise of artificial intelligence for increased productivity and huge cost savings.
Shows AI «racism» To the people?
Even researchers who study artificial intelligence may only know a small part of the problems that technology creates. «There's a whole series of unknown strangers.», said Dr. Wiles.
A well documented but underestimated defect of artificial intelligence models is that they tend to favor the work produced by artificial intelligence.
A 2025 study in the Proceedings of the National Academy of Sciences found that several large language models had a low opinion of texts written by people, creating a «potentially a serious form of indirect ‘anti-human’ prejudice».
However, many companies seemed to ignore this problem or, at least, cannot imagine how it could cause chaos in their operations.
When a group of researchers analysed it in detail in a later study, finding that the artificial intelligence models used by companies to evaluate biographicals tend to favor those drawn up with the help of artificial intelligence against those who have been compiled entirely by people, this drew attention to some responsible hirings in companies.
Jane Yi Jiang, professor of business administration at Ohio University and co-author of this study, said that she and her co-authors were willing to help when hiring companies asked «how to improve their procedures».
However, they pointed out that this almost certainly was not the only problem that companies inadvertently introduced in their rush to adopt artificial intelligence. «People move very quickly on the use of L.L.M. without thinking too much about the effects and prejudices»He said.
For example, some companies now use artificial intelligence to answer questions such as how much to charge for a product or where to open a new store. Technology dependence for such purposes, however, can quickly lead to the wrong direction.
When left to judgment, people often cooperate and seek results that benefit both sides.
But when artificial intelligence models evaluate a situation, they tend to adopt the coldest computing, «rational» mindset derived from basic game theory.
They may, for example, lead a company to aggressively underestimate a competitor, even if this involves the risk of a disastrous price war.
The problem of AI rationalism
«Most of the L.L.M. models we test believe people are more rational than they actually are.», stated Giannan Xu, a PhD candidate at the University of Maryland and associate of Dr Jiang.
«But the most rational reaction leads to a bad situation for all» In many cases.
In principle, developers and users of artificial intelligence can correct these prejudices. Dr Jiang and Mr Xu, for example, found that they could reduce anti-human prejudice simply by instructing models to focus on the quality of the written material they evaluate and avoid taking into account the author.
The «unknown»
However, artificial intelligence researchers cannot correct prejudices of which they are not aware, and several scholars said the impact of these undetectable prejudices may increase.
If, for example, future models are trained with data produced by current models without sufficient attention, a kind of self-enhancing cycle will inevitably be created.
In that case, «the trend towards consolidation of existing prospects and behaviour seems likely», said Shayne Longpre, an artificial intelligence researcher and founder of Data Provence Initiative, a group that monitors the infrastructure of artificial intelligence.
And then there are the «blind spots» which arise not so much from artificial intelligence itself as from the way people use it.
Scholars turning towards artificial intelligence at each stage the research process — Asking artificial intelligence what questions are worth studying, asking her advice on how to answer these questions, using it for data analysis, based on it to compile findings — may inadvertently limit the scope of their work.
«We do not necessarily observe it at individual level», stated Cecilie Steenbuch Traberg, a psychologist at Copenhagen Business School and author of a recent study on the subject.
«If you exchange views with a chatbot, which helps you come up with ideas, it might look great to you. But on a collective level, it looks quite similar. Everyone sounds the same.».
Dr. Wiles stated that weaknesses were not necessarily inherent in technology, but arose when people adopted it without paying particular attention to what could go wrong.
In recent centuries, business scholars and leaders have developed a reliable set of practices for human management.
However, the psychology of managing artificial intelligence is completely different, and «We're going out there blind».
