MOUNTAIN VIEW, Calif.--(BUSINESS WIRE)--Dec 23, 2024--
H2O.ai, the leader in open-source Generative AI and the most accurate Predictive AI platforms, today announced that h2oGPTe Agent has secured the #1 position on the GAIA (General AI Assistants) benchmark leaderboard with an unprecedented score of 65% — outperforming Google’s Langfun Agent (49%), Microsoft Research (38%), and Hugging Face (33%) leading entries. This remarkable achievement underscores H2O.ai's dominance in the emerging domain of general-purpose AI agents, setting a new gold standard for the industry.
This press release features multimedia. View the full release here: https://www.businesswire.com/news/home/20241223840924/en/
Why GAIA Matters
The GAIA benchmark measures how useful AI systems are in solving real-world tasks that require a lot of time, thought and effort for skilled humans. It consists of hundreds of challenges that require laborious research, data analysis, document handling and reasoning. Degree-holding human respondents achieve a score of 92% and require several human-days to solve all 300 test set problems.
h2oGPTe Agent outpaced competitors by delivering consistent robustness, accuracy and efficiency, highlighting its readiness for enterprise use cases that depend heavily on skilled human assistants.
Enterprise h2oGPTe Agent: A Landmark Achievement
This achievement solidifies H2O.ai’s leadership in the global race to build intelligent, adaptable AI assistants capable of transforming businesses.
Sri Ambati, Founder and CEO of H2O.ai, shared his enthusiasm:
“Today we are announcing that AI is only 30% away from matching human-level general intelligence on the GAIA benchmark. Open-ended questions in GAIA are a better measure of intelligence than MMLU, which relies on multiple choice. To share how exciting this is: the entire Gen AI ecosystem was barely able to pass a tenth in accuracy on one of the toughest AGI benchmarks merely a year ago.
“Makers at H2O.ai built h2oGPTe Agentic AI wielding the best models in the world for reasoning, multi-modal image, video, language understanding, code generation and execution to ace the GAIA benchmark with a stunning 15% accuracy leap over the previous record set by researchers from Google Deepmind using the same Claude-3.5-Sonnet. h2oGPTe Agent also beat Microsoft Research’s agent Magentic-1 that used OpenAI’s o1 model by 27%.
“Agentic AI is eating SaaS and with h2oGPTe Agentic AI now being generally available, all our enterprise customers can solve a wide range of sophisticated business and research problems.”
H2O.ai's success on GAIA underscores its philosophy of simplicity and adaptability:
H2O.ai’s win reaffirms its leadership in AI innovation, particularly in agentic systems poised to reshape business workflows.
Enterprise h2oGPTe 1.6 includes the Agent feature and is available on all public clouds, virtual private clouds and for on-premise deployments: https://h2o.ai/platform/enterprise-h2ogpte/
Read technical blog https://h2o.ai/blog/2024/h2o-ai-tops-gaia-leaderboard/
About H2O.ai
Founded in 2012, H2O.ai is at the forefront of the AI movement to democratize Generative AI. H2O.ai’s open-source Generative AI and Enterprise h2oGPTe, combined with Document AI and the award-winning autoML Driverless AI, have transformed more than 20,000 global organizations, and over half of the Fortune 500, including AT&T, Commonwealth Bank of Australia, Chipotle, Singtel, Workday, Progressive Insurance, and AES.
H2O.ai partners include Dell, Deloitte, Ernst & Young (EY), PricewaterhouseCoopers (PwC), NVIDIA, Snowflake, AWS, Google Cloud Platform (GCP) and Microsoft Azure. H2O.ai’s AI for Good program supports nonprofit groups, foundations, and communities in advancing education, healthcare, and environmental conservation. With a vibrant community of 2 million data scientists worldwide, H2O.ai aims to co-create valuable AI applications for all users.
H2O.ai has raised $256 million from investors, including Commonwealth Bank, Nvidia, Goldman Sachs, Wells Fargo, Capital One, Nexus Ventures and New York Life.
Example GAIA challenge solved by the h2oGPTe Agent (Graphic: Business Wire)
Example GAIA challenge solved by the h2oGPTe Agent (Graphic: Business Wire)
h2oGPTe Agent Tops GAIA Benchmark Test Results Dec 2024 (Graphic: Business Wire)
Oklahoma City leads the Western Conference and has a MVP candidate in Shai Gilgeous-Alexander. Milwaukee has the NBA's leading scorer in Giannis Antetokounmpo. They were the teams that made their way to the NBA Cup final.
By any measure, they're both very good teams.
And neither will play on Christmas Day this year. Bah, humbug.
In its defense, the NBA faces the same challenge every summer, figuring out which 10 teams will get the honor — and it is an honor — of playing on Christmas Day.
The Knicks and Lakers almost always find their way onto the slate, which makes sense given the size of the New York and Los Angeles media markets. The teams coming off runs to the NBA Finals typically get a Christmas invite, so hello, Boston and Dallas. The rest is a mix of stars, ratings potential, storylines and, hopefully, good teams.
Victor Wembanyama — the French star who already is a massive draw both in the U.S. and Europe — makes his Christmas debut when San Antonio goes to New York. Minnesota plays Dallas in a West finals rematch. Boston plays Philadelphia, a forever rivalry. The Lakers play Golden State, LeBron James vs. Stephen Curry. And Denver plays Phoenix in the nightcap. Nobody could have said those games were Christmas mistakes when the schedule came out.
It's also a sign of the parity in the NBA right now — there are clearly more than 10 teams worthy of Christmas consideration. That said, the Thunder and Bucks certainly have the right to feel snubbed.
“We should’ve had a Christmas day game, I believe, but the NBA felt different,” Bucks guard Khris Middleton said on media day, back in September. “That’s how they feel. I said my opinion on it. And sometimes it happens, sometimes it doesn’t.”
The Thunder not getting picked is puzzling, especially after being the No. 1 seed in the West last season.
This is the 17th consecutive season of the NBA having five games on Christmas Day. More often than not — 75% of the time — the No. 1 seeds for the East and West playoffs have been picked for a Christmas game the following season. And entering this year, each of the previous 11 No. 1 seeds in the West playoffs wound up having a game on Dec. 25 that same year.
But not the Thunder.
“Disappointed, for sure," Gilgeous-Alexander said. “I'd love to play on Christmas Day. And I think we're that caliber of team. The NBA makes their decisions. Can't slight them for it. Ball's in our court to prove to them why we deserve to be in that game.”
There have been reasons for some of the eight No. 1 seeds who got snubbed in that span to get snubbed. Some examples:
— Cleveland was the East's No. 1 seed in 2010. LeBron James left Cleveland that summer and went to Miami, and the Cavaliers became a far-less-marketable team overnight so putting them on the Christmas schedule wouldn't have made much sense.
— Indiana was the East's No. 1 seed in 2014. Paul George got hurt badly while playing for USA Basketball that summer and the Pacers weren't the same the following the season, which the schedule-makers probably realized could happen.
— Toronto was the East's No. 1 seed in 2018. Toronto didn't make the Christmas list a few months later for the simple reason that the Raptors are from Canada, and Canadian viewers don't count in American television ratings. (The then-reigning NBA champion Raptors made the Christmas lineup in 2019.)
None of those situations would apply to the Thunder.
Oklahoma City is a small market, sure. Only Memphis and New Orleans are smaller among NBA cities in the U.S., according to Nielsen. It's easy to deduce that national ratings play a part in the decision of who plays on Christmas and who doesn't, but it's also clear that those who just want to see good ball on Dec. 25 probably would have tuned in for a Thunder game.
“They make the schedule. We play it,” Thunder coach Mark Daigneault said. “Our players, I know, would have liked to play on Christmas because that's such a staple day in the NBA season. But we can't control that. All we can control is playing the schedule we're given, playing the hand we're dealt.”
Around The NBA analyzes the biggest topics in the NBA during the season.
AP NBA: https://apnews.com/NBA
Milwaukee Bucks forward Giannis Antetokounmpo (34) bites his NBA Cup medal after his team's victory in the championship game of the NBA Cup basketball tournament against the Oklahoma City Thunder Tuesday, Dec. 17, 2024, in Las Vegas. (AP Photo/Ian Maule)
Miami Heat forward Haywood Highsmith (24) defends Oklahoma City Thunder guard Shai Gilgeous-Alexander (2) during the second half of an NBA basketball game, Friday, Dec. 20, 2024, in Miami. (AP Photo/Marta Lavandier)
