Cornell University and Apple researchers have been working an open-source multimodal LLM titled ‘Ferret’

The open source LLM has been added in Github in October and was largely unnoticed without any fanfare or announcement from both parties. ‘Ferret’ code was revealed along with Ferret-Bench and checkpoint releases were added in December. Sometime later, ‘Ferret’ was discovered by AI researchers and the word spread. AI-in-medicine non-profit operator Bart De Witte posted on X, saying that it was a ‘missed’ release and speaks about Apple’s ‘testament of the company’s commitment to impactful AI research.’

The open-source release is done under a non-commercial license, which means commercialization is currently not allowed. There’s still a possibility that it will be integrated into Apple services or products in the future. Zhe Gan, Apple AI/ML researcher says that use cases for Ferret include the ability to ‘refer and ground anything, anywhere, at any granularity.’