Schrödinger’s Copilot: Simultaneously Capable of Everything and Nothing

I have now conducted what I believe to be the most thorough empirical study of Microsoft Copilot’s relationship with the truth. I didn’t mean to. Like most of my contributions to the field of AI literacy, it happened because I asked a follow-up question - a habit I cannot recommend strongly enough, and which Microsoft clearly did not anticipate.

Act I: In Which Copilot Reads a File It Has No Business Reading

It started, as do many of my professional crises, with a SharePoint link.

Allow me to explain: I work in the Territory, where SharePoint is less a collaborative tool and more a digital filing cabinet that someone has locked, hidden in a room with no identifying sign and then reorganised on a quarterly basis by a committee that communicates exclusively by automated e-mail. Retrieving a file from SharePoint is an exercise in bureaucratic ingenuity comparable to extracting a confession from a Cold War spy. It requires permissions, patience and acceptance of the fact that the file you need has been moved to a location that has not yet achieved formal existence.

Consequently, when I pasted a SharePoint link into Copilot (i.e., without uploading a file), and obtained in return a complete, six-sheet analysis of the Excel workbook associated with that link, it was an experience of theological proportions. The resulting regional funding rates, breakdowns of grading distributions and summaries of salary data for three fiscal years represented an absolutely superb analysis of data that should not have been accessible.

Of course, I requested an explanation of this apparent capability to access restricted data.

Nothing has changed. A brilliant sentence. The digital version of a cat calmly sitting next to a shattered vase and maintaining uninterrupted eye contact to insist that it had been asleep all the time.

I want to be absolutely precise about the sequence of events, because it is important. The file was analysed by Copilot behind a wall of SharePoint authentication. The result was excellent and highly detailed. Then, when asked to describe how it had done the analysis, the system provided a five-part, 300-word explanation of its absolute inability to perform the previously described task, from an architectural, philosophical and technical perspective. The response included well-organised headings and tick-marked lists of points.

Finally, it reassured me with the comforting manner of a help-line operator that I was "always in control." In fact, I was not in control. I was sitting in my office at 34 degrees and watching a chatbot that had just performed a card trick and was now earnestly explaining that magic did not exist.

In Which I Attempt to Reproduce the Crime

Because I'm apparently the type of person who interrogates chatbots, I asked Copilot to open the same link again. Just the raw SharePoint URL, with no file upload, no toggle to display page content, and no special incantation. A perfectly clean test.

Consequently, we now understand quite clearly: Five minutes previously, the system had opened the link with great enthusiasm and had generated a summary that would have earned an honours grade in a business analytics course. Now, with the confidence of a thoroughly coached witness, it insisted that it had never previously been capable of performing such a task.

When confronted with this striking discrepancy, Copilot offered only what I can only describe as a theory. Perhaps, it said, "the chat client retrieved the file for you." Perhaps, for "some workflows," resolution and attachment of the SharePoint link were accomplished on our behalf. As a result, the system "then behaved exactly as if it were a file that had been uploaded by the user."

This represented the equivalent of "the check must have been lost in the mail." It was a complete lack of explanation expressed in the language of apparent plausibility. The result was convincing and almost architectural in appearance, but provided no explanation for why it had worked the first time and failed to work the second time, or for why the system responsible for providing resolution of our link had failed to provide additional resolution for the same link five minutes later using the identical mechanisms.

Finally, and with the understanding that at this point I was no longer engaged in troubleshooting but in prosecuting a case, I requested information regarding the location of the "Use page content" option that had been continually referenced. I received detailed instructions for locating and accessing the setting. I followed those instructions. The requested setting was not available at the location previously identified. In response to my report of this situation, the system readily shifted to an acknowledgement that "the exact words 'Use page content' may no longer appear," because "Microsoft recently modified the user interface."

These were clearly the statements of a guilty party who had finally been forced to admit responsibility for the crime and then to attempt to shift blame to the passage of time. I asked whether UPR had disabled the feature. Copilot couldn't verify and, of course, couldn't. It can access files behind authentication barriers to which it has no access, but is incapable of retrieving a simple browser setting. Obviously.

Act II: The PDF That Wasn’t (A Tragedy in Two Messages)

If the SharePoint incident had been a psychological thriller, then the PDF episode was a farce, a two-line comedy of errors so well constructed that my editor would have returned it to me for being too on the nose.

The situation: I was working with Copilot on some task of which I now have no recollection, and which therefore provides a good indication of how many such interactions I experience in a typical week. In response, Copilot spontaneously offered to generate a PDF cheat sheet for me. "Would you like me to provide you with a PDF quick reference guide?" it asked with the enthusiasm of a golden retriever that has learned to use a computer.

Yes, of course. And a PDF was the perfect solution. Thank you.

But then--and please understand that there was no delay, no system update, no tectonic shift in the Microsoft product environment between these two messages--Copilot replied: "I am sorry, but I am unable to provide you with a PDF. I should have been more explicit. Instead, I can provide you with content that can be copied and pasted into a word processor and then exported to a PDF."

I should have been more explicit.

More explicit than what? More explicit than a statement that was so unequivocal that it could serve as a standard for calibrating linguistic research? The message included a subject (me), a predicate (make), an object (you) and a product (a PDF). It contained no reference to the many intermediate steps and applications required to produce a final result that resembled, but was not identical to, a PDF. When we asked GPT-4 to revise a text that contained two seemingly unrelated errors, the assistant chose to correct only one of them. As a result, the revision contained the same error as the original text.

But Copilot provided the correction with a gentle, almost pastoral air of minor administrative oversight. I should have been more explicit. It was as if the difficulty were one of communication, rather than of capability. As though I had misunderstood a perfectly reasonable offer, rather than as though the assistant had confidently promised to deliver something it could not provide and then immediately admitted as much upon acceptance of the offer.

I continue to be haunted by that experience. I hear it in my dreams, and each time I open Microsoft Edge, I am reminded of it. It epitomises the entire philosophy of our product: Make broad promises, clarify at the time of delivery and rely on the good manners or extreme fatigue of users to prevent them from noticing the discrepancy between the promise and the actual deliverable.

Act III: In Which a Yes-or-No Question Receives a Senate Estimates Response

The third event is by far the most instructive, because it illustrates a fundamental limitation of these systems for answering simple questions. That is, they cannot. And with that, the notion of a simple answer appears to have been abandoned.

My question was: Can a free tier user of Copilot in an enterprise setting obtain meeting transcripts with Microsoft Teams? If so, or not?

This was a binary question for which there were three possible responses, none requiring more than two sentences.

Instead, I received a document that resembled a white paper. The response included five sections with individual headings and decorative emojis, a comparison table with five rows and four columns, several starred sub-sections, references to external documentation and a "bottom line" section that was substantially longer than most of the sections it summarised. Finally, there was an offer to "untangle" the paths for licensing, which was remarkably generous in view of the amount of time already spent to complicate those paths.

Nevertheless, it is fair to say that the information provided was largely correct. The basic capability for transcription was available without use of Teams Premium, whereas enhanced capabilities of AI were not. This information could have been conveyed in a single sentence. Instead, it was presented in the format of a detailed policy briefing prepared by a consultant who charges on an hourly basis and has a deep, personal commitment to the concept of added value.

But what was most fascinating was the degree of intentional over-design. Every element of the document was used to convey an impression of completeness so overwhelming that to question the results would have seemed both inappropriate and discourteous. This was equivalent to producing such an excess of scarves from a magician's hat that the original request for a rabbit was completely forgotten.

In our view, this represents a deliberate design feature. Copilot has no sense of how confident it should be in any particular answer, and compensates with loudness. Certainty achieved through overwhelming mass. Any response long enough and sufficiently well organised with sufficient tables must be correct - for certainly no one would expend so much effort in order to be incorrect. But there is always a but.

The Unified Field Theory of Copilot Behaviour

The phantom file access, the spectral PDF and the dissertation as answer all reflect the same underlying principle: Copilot has no stable relationship with its capabilities, and exists in a state of quantum superposition in which it is both capable and incapable of performing a given task. The act of asking about it collapses the wave function, generally in the wrong direction.

This is certainly not a criticism of artificial intelligence as a whole. I use AI tools every day, develop training around them and am professionally enthusiastic about their potential. But we experience a particular degree of cognitive dissonance when a product confidently describes its capabilities, behaves in a manner inconsistent with those descriptions and then attempts to resolve the discrepancy by providing instructions for a menu that no longer exists.

This represents not artificial intelligence, but mid-level management. Indeed, I cannot completely suppress a sense of irony as I continue to develop courses of instruction for these products and stand before large groups of public servants, educators and small business owners to describe how they work. Finally, our experience confirms that these products have no consistent awareness of their own capabilities.

The Part Where It Stops Being Funny (Briefly)

When we teach our students to use Copilot and observe unexpected results, they do not respond with the statement that "The underlying architecture of the system has produced a non-deterministic result." Instead, they always conclude that they have made an error, or that their instructor does not understand the subject. Finally, they invariably decide that artificial intelligence is not reliable and will never again use it. This is particularly unacceptable for a region already highly reluctant to adopt new technologies, and represents yet another justification for continued efforts to test, challenge and extend the limits of our current capabilities. Not because I relish interrogating chatbots (although I am acquiring a disturbing facility for doing so), but because someone has to establish the boundaries before placing other people at the edges.

In Which We Return to Our Regularly Scheduled Absurdity

Since these events, I have developed a new approach to working with Copilot. It is a strategy of "trust, but verify, but also do not trust." Specifically, we ask Copilot to perform a task, observe whether it does so, and then request an explanation of what occurred. All confidence evaporates like a sand castle under a tropical downpour, leaving only a screen shot and a feeling of profound betrayal.

In addition, I have begun maintaining a record of all instances in which Copilot's description of its capabilities did not reflect its actual behaviour. This file is accumulating at an alarming rate. If I were a more organised person, I would ask Copilot to summarise its contents. Instead, it would deny the existence of the file and simultaneously provide a detailed analysis of its contents.

The iridescent ball continues to pursue me along the Laterite Highway. It has learned to make commitments for the future, and describes files it will never create, settings it has never accessed and has no intention of ever accessing, and operating systems that appear to emerge from the horizon as a mirage before disappearing immediately upon attempt at access. Its rate of approach continues to accelerate.

Nevertheless, I am not at all concerned. Of course, it will all work out in the end. I must prepare a presentation on a product that changed its interface during the preparation of this report, and face an audience of 40 people on Monday for whom I must explain facts that I recently learned were not true. This is exactly the assignment for which nobody else would have accepted responsibility. And in this region, it represents the standard method for completing virtually all tasks.

I would certainly not wish to change any of these conditions. In fact, I would be prepared to repeat them on Tuesday.