Groq, an ambitious startup developing cutting-edge chips to accelerate generative AI models, announced on Monday that it has raised $640 million in a new funding round led by Blackrock. The round saw participation from notable investors, including Neuberger Berman, Type One Ventures, Cisco, KDDI, and Samsung Catalyst Fund. This significant funding brings Groq’s total raised to over $1 billion and values the company at $2.8 billion, a considerable leap from its previous valuation of $1 billion in April 2021.
Groq’s CEO, Jonathan Ross, known for his role in inventing Google’s tensor processing unit (TPU), co-founded Groq in 2016 alongside Douglas Wightman, a former engineer at Alphabet’s X moonshot lab. The startup is making waves with its innovative language processing unit (LPU) inference engine, which claims to run AI models like OpenAI’s ChatGPT and GPT-4 at ten times the speed and one-tenth the energy of traditional processors.
Adding to the company’s prestige, Meta’s chief AI scientist Yann LeCun has joined Groq as a technical advisor, while Stuart Pann, former head of Intel’s foundry business and ex-CIO at HP, has taken on the role of chief operating officer. LeCun’s involvement is particularly noteworthy, given Meta’s own investments in AI chips, highlighting Groq’s growing influence in the sector.
GroqCloud, the company’s LPU-powered developer platform, supports a range of open AI models, including Meta’s Llama 3.1, Google’s Gemma, OpenAI’s Whisper, and Mistral’s Mixtral. The platform, which also offers an API for cloud instances and a playground for AI chatbots called GroqChat, has attracted over 356,000 developers as of July. A portion of the new funding will be used to expand GroqCloud’s capacity and add new models and features.
Stuart Pann emphasized Groq’s broad enterprise appeal, stating, “By our estimates, over 75% of the Fortune 100 are represented” among GroqCloud’s developers. This widespread adoption underscores Groq’s potential to disrupt the AI chip market, which is currently dominated by Nvidia.
Nvidia, which controls an estimated 70% to 95% of the AI chip market, has been aggressively innovating to maintain its lead. The company plans to release a new AI chip architecture annually and is establishing a business unit to design custom chips for cloud computing and AI hardware.
Groq’s competition extends beyond Nvidia, facing formidable rivals like Amazon, Google, and Microsoft, all developing custom AI chips. Amazon offers Trainium, Inferentia, and Graviton processors through AWS, Google Cloud uses TPUs and plans to introduce the Axion chip, and Microsoft has previewed Azure instances with its Cobalt 100 CPU, with Maia 100 AI Accelerator instances forthcoming.
The competitive landscape also includes Arm, Intel, AMD, and several startups like D-Matrix and Etched, each innovating within the AI chip market expected to reach $400 billion in annual sales within five years.
Groq’s strategy includes significant enterprise and government outreach. In March, Groq acquired Definitive Intelligence to form Groq Systems, targeting organizations, including U.S. government agencies, for AI chip integration. Collaborations with government IT contractor Carahsoft and European firm Earth Wind & Power further bolster its public sector footprint.
Moreover, Groq is partnering with Saudi Arabian consulting firm Aramco Digital to install LPUs in future Middle Eastern data centers. The company is also progressing towards its next-gen chip, contracting with Samsung’s foundry business to produce 4nm LPUs, promising enhanced performance and efficiency over its initial 13nm chips.
With plans to deploy over 108,000 LPUs by the end of Q1 2025, Groq is positioned to challenge the giants in the AI chip industry, leveraging its innovative technology and substantial new funding to carve out a significant niche in this rapidly evolving market.
