On each planet that exhibits axial parallelism and tilt, each summer will inevitably be followed by a winter. Each hemisphere will take turns facing the system’s star, cycling through summers and winters. This will hold true until the energy resources of the star are depleted, assuming the planet will conserve its axial tilt and stay in orbit until then. Interestingly enough, this seemingly specific astronomical principle is applicable to almost any concept imaginable; this is nature’s very own boom-bust cycle.

Since the mid-1950s, it’s been possible to observe the same cyclical pattern on the topic of artificial intelligence. It started out quite positively, especially with the introduction of perceptron, the first neural network. Neural networks are still the most prominent AI models. It wasn’t until 1973, the year the famous Lighthill report of Sir James Lighthill, Artificial Intelligence: A General Survey, was published in which he heavily criticized the state of the AI research for a multitude of reasons. Though tendentious, the paper has sparked a controversy amongst British government officials and has resulted in spending cuts to many universities for AI research. Obviously the effects of this were not just contained in Great Britain; the same year, in the United States, DARPA also stopped supporting AI research. These developments marked the start of the first of the two major AI winters, which was present in the field until about 1980.

The second AI boom in the 1980s was with the adoption of Lisp based expert systems. These stormed the industry; many businesses were acquiring expert systems to improve their day to day workflow. The popularity of these systems helped bloom a market around them. Soon enough, in 1987, the market of the expert systems collapsed with the introduction of multiple Lisp alternatives. It was predicted by Marvin Minsky and Roger Schank in 1984 that a similar AI winter is imminent, due to the unrealistic hype present for technically unattainable goals and dreams.

This time it took about 13 years for the attention around the topic of AI to brew up. Since the year 2000, we’ve been seeing the focus on AI steadily increasing, notably with 2012’s machine learning hype. Finally, in 2022, the release of GPT-3.5 marked the biggest AI boom in history. This time it was different; it was all public, it was all easily marketable, it was all promised to change the daily lives of everyone. To some extent it really did change the workflows of the many. But now, at the time I’m writing this, I feel (not fear) that there is an impending winter again, and the cracks have already started to show.

To understand why, we first need to understand what the current hype surrounds. In recent years, almost all R&D under AI have been towards large language models. Many businesses want to incorporate LLM based AI software both in their workflows and their product offerings. Individuals, too, tend to use LLMs for almost anything. Many have already substituted their regular ways of research for using LLMs despite their inclination towards hallucinating and spewing outright false information. The Web has been rapidly littered with AI generated content, namely AI slops, and dodging them is getting harder each day.

The main problem is that people, even businesses, do not understand the limits or the capabilities of LLMs at all. These models don’t have any logic or reasoning capabilities despite being treated as such, unfortunately. There are many people viewing the models as if they are cognitively capable, when in reality they are pretty advanced pattern finding, tokenized text completion models. Recently, we got one of the first Lighthill reports of our era from Apple, titled The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity, in which they recognize the nature and limitations of modern language models. Nothing is actually novel in the paper; me and the people that already knew how natural language processing works have been saying basically the same thing since the current AI craze began in 2022. I’ll not be delving into the details of the paper, but it is going to be definitely shaping the expectations from these models, at least in the corporate world in the near future.

The misunderstanding of the capabilities and the technical details of language models have also led to the expectations from them to be sky high, much higher than both physically and theoretically possible. Attempts at multimodality seem to further establish this fallacy surrounding AI, where this multimodal inference and generation help reinforce the anthropic (pun intended) image of AI in the minds of the masses. But this multimodality does not improve the performance of models; they are still severely limited by their non-reasoning and non-logical nature. This leads me to a realization; in the Lighthill report of 1973, in which Sir James Lighthill analyzed the state of the field in three categories of which one of them focused on the ability of the AI to solve real life problems that induce combinatorial explosion with the increased number of variables unsupervised, his conclusion was that the AI had utterly failed in every respect. Interestingly, Apple’s aforementioned paper also incorporated the game of Tower of Hanoi as one of the test problems, which is a classic example of combinatorial explosion. To no one’s surprise, it failed again, long before even reaching the limits of compute.

It is apparent that these systems are an essential part of many people’s workflow. I don’t believe that is going to change any time soon, or rather at all. But what is going to change, and must change is the people’s vision, and expectations from them. It seems that while we are making huge progress in specialized machine learning applications such as chess engines, self-driving cars, medical binary classifiers, and similar areas, we are rapidly reaching the upper limits of the current LLM boom. While I’ve always been astounded at the progress rate of these specialized tools, the rate of progress/potential/hype in the usage and application of large language models is daunting. This is why I feel that another AI winter, though maybe not as severe as the previous ones, is imminent. We are once again entering a period in which the term “AI” will take on negative connotations. Hopefully, the next summer would be based on more realistic solutions with more attainable goals.