Even as tech giants scramble to lead the AI arms race, Apple has been slow. However, this is not the case, Apple has been working on AI for a long time. Since the brand is synonymous with secrecy, not much has been known about its grand schemes. However, recently, Apple has introduced a generative AI model named OpenELM which has reportedly outperformed several other language models that have been trained on public data.
OpenELM is a family of small open-source language models designed to run efficiently on devices such as iPhones and Macs. Apple claims that OpenELM is a state-of-the-art language model that uses a layer-wise scaling strategy to efficiently allocate parameters within each layer of the transformer model resulting in enhanced accuracy. Reportedly, OpenELM consists of eight models with four different parameter sizes – 270M, 450M, 1.1B, and 3B – all of which are trained on public datasets.
What is OpenELM?
Reportedly, the model family is optimised for on-device use, allowing for AI-powered tasks to be handled without relying on cloud servers. OpenELM has reportedly outperformed similar open-source models like OLMo despite it requiring 2x less training data. OpenELM has reportedly been trained on CoreNet, an open-source library, along with other models that enable “efficient inference and fine-tuning on Apple devices.”
“Diverging from prior practices that only provide model weights and inference code, and pre-train on private datasets, our release includes the complete framework for training and evaluation of the language model on publicly available datasets, including training logs, multiple checkpoints, and pre-training configurations. We also release code to convert models to MLX library for inference and fine-tuning on Apple devices. This comprehensive release aims to empower and strengthen the open research community, paving the way for future open research endeavours,” read the research paper shared by Apple.
The release comes weeks ahead of WWDC in June where Apple is likely to debut its iOS 18. The latest iteration of the mobile operating system from Apple is expected to feature a collection of new AI features. However, the release of OpenELM gives a glimpse of what is going on behind the scenes.
From Microsoft with Phi-3 models to Apple’s OpenELM, it seems tech giants are getting on the small model bandwagon. The latest release from Apple shows how the tech giant may use on-device AI going forward. Also, another noteworthy facet is that Apple has made this an open-source release, a distinct turn from Apple’s previously restrictive and secretive ways.