AWS Trainium2 Instances Now Generally Available

LAS VEGAS--(BUSINESS WIRE)--Dec 3, 2024--

At AWS re:Invent, Amazon Web Services, Inc. (AWS), an Amazon.com, Inc. company (NASDAQ: AMZN), today announced the general availability of AWS Trainium2-powered Amazon Elastic Compute Cloud (Amazon EC2) instances, introduced new Trn2 UltraServers, enabling customers to train and deploy today’s latest AI models as well as future large language models (LLM) and foundation models (FM) with exceptional levels of performance and cost efficiency, and unveiled next-generation Trainium3 chips.

This press release features multimedia. View the full release here: https://www.businesswire.com/news/home/20241203165432/en/

“Trainium2 is purpose built to support the largest, most cutting-edge generative AI workloads, for both training and inference, and to deliver the best price performance on AWS,” said David Brown, vice president of Compute and Networking at AWS. “With models approaching trillions of parameters, we understand customers also need a novel approach to train and run these massive workloads. New Trn2 UltraServers offer the fastest training and inference performance on AWS and help organizations of all sizes to train and deploy the world’s largest models faster and at a lower cost.”

As models grow in size, they are pushing the limits of compute and networking infrastructure as customers seek to reduce training times and inference latency—the time between when an AI system receives an input and generates the corresponding output. AWS already offers the broadest and deepest selection of accelerated EC2 instances for AI/ML, including those powered by GPUs and ML chips. But even with the fastest accelerated instances available today, customers want more performance and scale to train these increasingly sophisticated models faster, at a lower cost. As model complexity and data volumes grow, simply increasing cluster size fails to yield faster training time due to parallelization constraints. Simultaneously, the demands of real-time inference push single-instance architectures beyond their capabilities.

Trn2 is the highest performing Amazon EC2 instance for deep learning and generative AI

Trn2 offers 30-40% better price performance than the current generation of GPU-based EC2 instances. A single Trn2 instance combines 16 Trainium2 chips interconnected with ultra-fast NeuronLink high-bandwidth, low-latency chip-to-chip interconnect to provide 20.8 peak petaflops of compute, ideal for training and deploying models that are billions of parameters in size.

Trn2 UltraServers meet increasingly demanding AI compute needs of the world’s largest models

For the largest models that require even more compute, Trn2 UltraServers allow customers to scale training beyond the limits of a single Trn2 instance, reducing training time, accelerating time to market, and enabling rapid iteration to improve model accuracy. Trn2 UltraServers are a completely new EC2 offering that use ultra-fast NeuronLink interconnect to connect four Trn2 servers together into one giant server. With new Trn2 UltraServers, customers can scale up their generative AI workloads across 64 Trainium2 chips. For inference workloads, customers can use Trn2 UltraServers to improve real-time inference performance for trillion-parameter models in production. Together with Anthropic, AWS is building an EC2 UltraCluster of Trn2 UltraServers, named Project Rainier, which will scale out distributed model training across hundreds of thousands of Trainium2 chips interconnected with third-generation, low-latency petabit scale EFA networking—more than 5x the number of exaflops that Anthropic used to train their current generation of leading AI models. When completed, it is expected to be the world’s largest AI compute cluster reported to date available for Anthropic to build and deploy their future models on.

Anthropic is an AI safety and research company that creates reliable, interpretable, and steerable AI systems. Anthropic’s flagship product is Claude, an LLM trusted by millions of users worldwide. As part of Anthropic’s expanded collaboration with AWS, they’ve begun optimizing Claude models to run on Trainium2, Amazon’s most advanced AI hardware to date. Anthropic will be using hundreds of thousands of Trainium2 chips—over five times the size of their previous cluster—to deliver exceptional performance for customers using Claude in Amazon Bedrock.

Databricks’ Mosaic AI enables organizations to build and deploy quality agent systems. It is built natively on top of the data lakehouse, enabling customers to easily and securely customize their models with enterprise data and deliver more accurate and domain-specific outputs. Thanks to Trainium's high performance and cost-effectiveness, customers can scale model training on Mosaic AI at a low cost. Trainium2’s availability will be a major benefit to Databricks and its customers as demand for Mosaic AI continues to scale across all customer segments and around the world. Databricks, one of the largest data and AI companies in the world, plans to use Trn2 to deliver better results and lower TCO by up to 30% for its customers.

Hugging Face is the leading open platform for AI builders, with more than 2 million models, datasets, and AI applications shared by a community of more than 5 million researchers, data scientists, machine learning engineers, and software developers. Hugging Face has collaborated with AWS over the last couple of years, making it easier for developers to experience the performance and cost benefits of AWS Inferentia and Trainium through the Optimum Neuron open-source library, integrated in Hugging Face Inference Endpoints and now optimized within the new HUGS self-deployment service, available on the AWS Marketplace. With the launch of Trainium2, Hugging Face users will have access to even higher performance to develop and deploy models faster.

poolside is set to build a world where AI will drive the majority of economically valuable work and scientific progress. poolside believes that software development will be the first major capability in neural networks that reaches human-level intelligence. To enable that, they're building FMs, an API, and an assistant to bring the power of generative AI to developers' hands. A key to enable this technology is the infrastructure they’re using to build and run their products. With AWS Trainium2, poolside’s customers will be able to scale their usage of poolside at a price performance ratio unlike other AI accelerators. In addition, poolside plans to train future models with Trainium2 UltraServers, with expected savings of 40% compared to EC2 P5 instances.

Trainium3 chips—designed for the high-performance needs of the next frontier of generative AI workloads

AWS unveiled Trainium3, its next-generation AI training chip. Trainium3 will be the first AWS chip made with a 3-nanometer process node, setting a new standard for performance, power efficiency, and density. Trainium3-powered UltraServers are expected to be 4x more performant than Trn2 UltraServers, allowing customers to iterate even faster when building models and deliver superior real-time performance when deploying them. The first Trainium3-based instances are expected to be available in late 2025.

Enabling customers to unlock the performance of Trainium2 with AWS Neuron software

The Neuron SDK includes compiler, runtime libraries, and tools to help developers optimize their models to run on Trainium. It provides developers with the ability to optimize models for optimal performance on Trainium chips. Neuron is natively integrated with popular frameworks like JAX and PyTorch so customers can continue using their existing code and workflows on Trainium with fewer code changes. Neuron also supports over 100,000 models on the Hugging Face model hub. With the Neuron Kernel Interface (NKI), developers get access to bare metal Trainium chips, enabling them to write compute kernels that maximize performance for demanding workloads.

Neuron software is designed to make it easy to use popular frameworks like JAX to train and deploy models on Trainium2 while minimizing code changes and tie-in to vendor-specific solutions. Google is supporting AWS's efforts to enable customers to use JAX for large-scale training and inference through its native OpenXLA integration, providing users an easy and portable coding path to get started with Trn2 instances quickly. With industry wide open-source collaboration and the availability of Trainium2, Google expects to see increased adoption of JAX across the ML community—a significant milestone for the entire ML ecosystem.

Trn2 instances are generally available today in the US East (Ohio) AWS Region, with availability in additional regions coming soon. Trn2 UltraServers are available in preview.

To learn more, visit:

About Amazon Web Services

Since 2006, Amazon Web Services has been the world’s most comprehensive and broadly adopted cloud. AWS has been continually expanding its services to support virtually any workload, and it now has more than 240 fully featured services for compute, storage, databases, networking, analytics, machine learning and artificial intelligence (AI), Internet of Things (IoT), mobile, security, hybrid, media, and application development, deployment, and management from 108 Availability Zones within 34 geographic regions, with announced plans for 18 more Availability Zones and six more AWS Regions in Mexico, New Zealand, the Kingdom of Saudi Arabia, Taiwan, Thailand, and the AWS European Sovereign Cloud. Millions of customers—including the fastest-growing startups, largest enterprises, and leading government agencies—trust AWS to power their infrastructure, become more agile, and lower costs. To learn more about AWS, visit aws.amazon.com.

About Amazon

Amazon is guided by four principles: customer obsession rather than competitor focus, passion for invention, commitment to operational excellence, and long-term thinking. Amazon strives to be Earth's Most Customer-Centric Company, Earth's Best Employer, and Earth's Safest Place to Work. Customer reviews, 1-Click shopping, personalized recommendations, Prime, Fulfillment by Amazon, AWS, Kindle Direct Publishing, Kindle, Career Choice, Fire tablets, Fire TV, Amazon Echo, Alexa, Just Walk Out technology, Amazon Studios, and The Climate Pledge are some of the things pioneered by Amazon. For more information, visit amazon.com/about and follow @AmazonNews.

AWS Trainium2 chip (Photo: Business Wire)

Amazon EC2 Trn2 UltraServers (Photo: Business Wire)

SALCEDO, Ecuador (AP) — Ice-cream production in Salcedo, a quaint town in Ecuador's central highlands, began in the mid-20th century, born from the ingenuity of Franciscan nuns.

Locals say the sisters would drink fruit shakes made with milk from the region’s dairy farms until one of them began collecting the leftovers, turning them into creamy popsicles that became an overnight sensation.

The nuns sold the popsicles in town to gather funds for the poor. But the people of Salcedo saw a business opportunity and began experimenting with new flavors and techniques, establishing a thriving popsicle industry that has made their small town famous among ice-cream lovers.

However, a recent wave of power outages, triggered by a prolonged dry spell, is threatening the future of Salcedo’s ice-cream industry and melting away its dreams of a more prosperous future.

The daily power outages that began earlier this year and intensified in September can last up to 14 hours. They have come about due to drier than usual weather in Ecuador, which relies heavily on hydroelectric plants. During most months of the year, hydroelectric plants produce between 70% to 90% Ecuador’s electricity. The nation of 17 million people invested heavily in dams over the past two decades, but it is now struggling to come up with alternatives.

“We are living through the worst third-world conditions," said Gabriel Pumasunta, owner of the Polar Bear ice-cream factory. “We're plunged into darkness.”

Pumasunta, who runs the small company with his two brothers, said that if the power outages continue this month, they will have no choice but to shut down the plant.

“We will have to work elsewhere,” he said.

Pumasunta said that power outages have halted production and storage, causing much of his product to melt.

Before outages intensified in September, Polar Bear was producing 60,000 popsicles per month, Salcedo said. Now, the small company makes 10,000 popsicles per month, and it has been forced to let go of eight of its 10 employees.

To stay afloat, Pumasunta has dipped into the company’s savings and those of his family. He now works on the plant’s machines himself, taking care of deliveries to bring down costs, while his parents have also stepped up to help.

President Daniel Noboa, who was elected last year in a special election that followed the resignation of Ecuador’s previous president, has not been able to solve the electricity crisis. The chamber of commerce in Guayaquil, Ecuador’s largest city, estimates that power outages are generating weekly losses of $700 million for Ecuadorian businesses.

Germán Soria, president of the association of artisanal ice-cream makers, said that before the power crisis, Salcedo had 80 small- and medium-sized ice-cream plants and three large factories. Now, 30 of these smaller plants have been forced to close.

Soria’s own ice-cream plant, housed in a shed behind his house, now sits empty. He said he stopped production three weeks ago and has been forced to sell some equipment, as well as a plot of land to cover his debts.

He said that the outages, which occur at unexpected times, make it hard to plan production. “Our clients are also less keen to buy ice-cream, because they’re not sure if they can keep the popsicles from melting."

According to local officials, the power crisis has cost 300 jobs in Salcedo, a town renowned for its fruit-flavored ice-cream, including flavors such as blackberries, and taxo, an aromatic and somewhat acidic fruit that resembles passion fruit.

President Noboa has promised that power shortages in Ecuador will end this month, but analysts doubt his claim, saying the country lacks the capacity to make up for a 1,900-megawatt deficit.

The Ecuadorian government has bought and rented dozens of large electric generators that run on fossil fuels and are expected to arrive in the following weeks. But as Marco Acuña, the president of Ecuador's national guild of engineers, notes: "They do not have the capacity to make up for the current deficit.”

Meanwhile in Salcedo, the collapse of the local ice-cream industry has rippled through the local economy, impacting dairy farms, fruit growers, transporters and mom-and-pop stores that specialized in selling the famous popsicles.

“Before (the power cuts) we would buy up to 150 popsicles a day,” said Maria Juliette López, who owns a small shop on the Panamerican highway where tourists and truck drivers stop to buy snacks. “Now we can only take up to 40 popsicles a day, because any product that is left over will melt” when her freezer stops working, she said.

While the power outages appear to be hitting small- and medium-sized producers and vendors the most, Salcedo’s three largest ice-cream factories say they have also been impacted.

One of those is Corp Ice-Cream, whose factory used to buzz with activity at noon, with its 35 employees working hard to produce up to 20,000 popsicles per day. But during a recent visit to the plant, there was only one employee sweeping in the dark, due to the lack of power.

Paco Hinojosa, the company's manager, said Corp Ice Cream managed to stay afloat thanks to its customers in the United States, which order 120,000 popsicles per month during the summer. But the winter months bring a halt to U.S. exports, which won't resume until around March.

Corp Ice Cream’s sales in Ecuador have gone down by 40% since the power outages began in mid September and that is taking a toll on its suppliers. “We used to buy 2,500 liters of milk and 160 liters of cream each day,” Hinojosa said. “Now we have cut down our orders by half — and the same goes for fruit.”

Next to his desk, Hinojosa has a small altar to St. Michael the Archangel, the city’s patron saint, whom he has been asking for health — and more rain.

“It brings tears to your eyes to see this factory paralyzed due to the lack of electricity,” he said.

Follow AP’s coverage of Latin America and the Caribbean at https://apnews.com/hub/latin-america

A vendor waves a popsicle-shaped sign to attract prospective customers in Salcedo, Ecuador, Thursday, Nov. 28, 2024, amid a wave of power outages, triggered by a prolonged dry spell. (AP Photo/Dolores Ochoa)

Signs advertise Salcedo's creamy popsicles along the Pan-American highway, in Latacunga, Ecuador, Thursday, Nov. 28, 2024, amid a wave of power outages, triggered by a prolonged dry spell. (AP Photo/Dolores Ochoa)

Workers prepare popsicles at an artisanal factory in Salcedo, Ecuador, Thursday, Nov. 28, 2024, amid a wave of power outages, triggered by a prolonged dry spell. (AP Photo/Dolores Ochoa)

A worker packages popsicles at an artisanal factory in Salcedo, Ecuador, Thursday, Nov. 28, 2024, amid a wave of power outages, triggered by a prolonged dry spell. (AP Photo/Dolores Ochoa)

A cutout of a popsicle adorns a shelf next to a statue of patron saint Michael the Archangel in Salcedo, Ecuador, Thursday, Nov. 28, 2024, amid a wave of power outages, triggered by a prolonged dry spell. (AP Photo/Dolores Ochoa)

A worker wipes equipment at an artisanal popsicle factory in Salcedo, Ecuador, Thursday, Nov. 28, 2024, amid a wave of power outages, triggered by a prolonged dry spell. (AP Photo/Dolores Ochoa)