Apple's Glowtime Event and the Race for Edge AI
On September 9th, Apple held its latest product launch event, “Its Glowtime,” where it introduced the iPhone 16 series. The theme hinted at the newly enhanced Siri voice assistant but also alluded to the newest addition to the iPhone color palette—“desert titanium,” a gold hue designed to add a touch of luxury to the iPhone 16 Pro. Despite the excitement surrounding the event, however, it lacked the kind of "zing" Apple fans have come to expect from these releases. Tim Cook, Apple’s CEO, emphasized the promise of generative artificial intelligence (AI) features, dubbed “Apple Intelligence,” first teased in June. Yet, while these features are powered by the new A18 chips in the latest devices, users will have to wait until at least October to access them—and even then, only in a limited beta version. The demonstrations, too, failed to impress.
Apple’s efforts to integrate generative AI into its devices are part of a broader trend among tech companies trying to move AI from the cloud to smaller, local devices, a practice referred to as “edge AI.” Apple’s rival Samsung has already made strides in this area with its Galaxy S24, and Microsoft has introduced AI-powered PCs with their Copilot+ system. Although many companies are racing to dominate the AI market, none have yet established clear leadership, and the challenges of implementing AI on smaller devices are considerable.
Challenges and Solutions in Implementing Edge AI*
One major hurdle is the enormous computational power needed to run large language models (LLMs), the technology behind AI systems like OpenAI’s ChatGPT. LLMs are typically trained using high-performance graphics processing units (GPUs), which consume large amounts of energy and require vast data sets. The cost of operating these systems is enormous; for instance, it reportedly costs OpenAI 36 cents every time a user makes a request through ChatGPT. By contrast, edge AI relies on smaller, more efficient models distilled from these larger systems. This approach is designed to deliver faster, cheaper responses with lower latency, giving users the experience of interacting with AI in real-time.
However, the performance of edge AI is still lacking in some areas. While smaller models are effective for simpler tasks like identifying menu items at a restaurant or answering basic questions, more complex tasks such as planning a vacation still require the raw power of cloud-based LLMs. The challenge of balancing computational demands with device battery life is another barrier, as running even smaller AI models can quickly drain a phone or tablet.
Companies like Apple are exploring hybrid solutions to overcome these obstacles. Apple Intelligence will first attempt to answer user queries directly on the device and will only refer more complex requests to its cloud-based systems or third-party LLMs like ChatGPT, with user consent. Yet, this approach may raise privacy concerns since smartphones are privy to sensitive personal data such as contacts, location, and purchasing habits. Some users may prefer that such data remains confined to their devices.
The Growing Role of NPUs and the Open Race for Edge AI
Beyond Apple, other tech firms are also searching for ways to make AI on edge devices more efficient. One promising development is the use of neural processing units (NPUs), which are optimized for AI tasks and consume far less energy than traditional GPUs. Qualcomm, a key player in this space, is working on maximizing the "performance per watt" of NPUs, making them not only more efficient but also cheaper. For companies like Apple, using NPUs is crucial as they develop AI-powered devices that need to perform well without inflating prices to match the high costs associated with GPUs.
Despite these advancements, the race to dominate edge AI is wide open. While Nvidia has cornered the market for GPUs used in cloud-based AI, there is no clear leader in edge AI technology. This creates opportunities for multiple players to compete, and according to Neil Shah of the research firm Counterpoint, those who succeed could trigger a new supercycle in device sales and open up new markets for apps and digital advertising.
The Hype Cycle and the Uncertain Future of AI
But AI’s path to ubiquity may not be as smooth as many had hoped. In the past few weeks, a growing number of investors have begun to express doubts about the profitability of AI, and the stock prices of leading AI firms have dropped by 10% since their recent peaks. Despite billions of dollars in investment and high expectations, adoption of AI by businesses has been slow. According to recent Census Bureau data, only 5.1% of American companies currently use AI to produce goods and services, a slight decline from earlier this year.
These developments have led some to wonder whether AI is following a pattern familiar to those in Silicon Valley: the "hype cycle." Popularized by the research firm Gartner, the hype cycle describes a period of excessive enthusiasm for a new technology followed by a "trough of disillusionment" when the technology fails to deliver immediate results. Eventually, the technology often makes a comeback as it becomes more refined and widely adopted. Past technologies like railroads in the 19th century and the internet in the late 1990s followed this pattern. In both cases, early investment created the infrastructure necessary for these technologies to revolutionize society, despite initial setbacks.
Whether AI is destined to follow a similar trajectory remains uncertain. AI has already experienced several cycles of hype and disappointment, with periods of intense interest followed by "AI winters," where investment and enthusiasm waned. While generative AI has sparked renewed excitement, it is unclear whether this latest wave will lead to lasting, widespread adoption.
Some influential technologies, such as cloud computing and solar power, bypassed the hype cycle entirely, moving smoothly from innovation to mass adoption. On the other hand, many hyped technologies—like Web3, 3D printers, and carbon nanotubes—failed to make a significant impact after their initial bursts of excitement. Studies have shown that only about 20% of emerging technologies follow the full arc of the hype cycle, with most either becoming widely used without a period of disillusionment or fading into obscurity after their initial hype.
Revolution or Reassessment?
While generative AI holds immense potential, it remains to be seen whether it will revolutionize the real economy or merely become another flash in the pan. The current slowdown in AI investment and adoption suggests that the technology may be entering a period of reassessment. For companies like Apple, which are betting heavily on AI-powered devices, success will depend on overcoming technical challenges and convincing consumers and businesses alike that AI is worth the investment. For now, edge AI is still in its early stages—perhaps not quite ready for prime time or “Glowtime.”
Comments