A：The next wave of AI is expected to evolve beyond just thinking to taking action, with AI moving from answers to action by pushing inference compute to a tremendous scale.

A：AMD's mission is to push the boundaries of high-performance AI computing to help solve the world's most important challenges.

A：AI is helping researchers identify new drug candidates faster, solving previously unsolvable scientific problems, and changing the way work is done across every industry, with more capabilities being developed as AI progresses.

A：In just two years, the demand for AI compute power has grown nearly 160 times, with more than 35 quadrillion tokens consumed each month.

A：It is expected that for the first time, the world will use more AI compute to run models than to train them, with roughly 60% of global AI compute capacity being used for inference this year.

A：'Agent-centric AI' refers to the next big step for AI where agents can answer full questions and solve problems until completion, leading to a significant increase in compute demand as these agents require extensive reasoning and data access.

A：The AI accelerator market is expected to grow significantly, with a forecast from 500 billion by 2028 to 1.4 trillion by 2030, closely approaching the size of the entire semiconductor market.

A：The server CPU market is expected to grow by over 50%, starting from a $25 billion market to over $200 billion, driven by the rapid adoption of AI and the need for more CPU infrastructure.

A：AI at the edge is a key part of AMD's strategy, with a focus on bringing intelligence to devices that sense and act in real time, alongside the cloud, to fulfill the need for AI to be infused everywhere.

A：AMD is positioned for future success with the broadest product portfolio, the strongest roadmaps, and deep partnerships with companies building the future. This allows for a multi-year strategy aligned with compute leadership, open platforms, and AI integration across products.

A：The new AI accelerator built with TSMC's leading 2nm and 3nm process technology features 320 billion transistors and combines 12 compute and IO chiplets with 432 GB of memory, all connected by industry-leading 3D chiplets technology. It is the highest performance AI accelerator in the industry.

A：The CPU board in the Helios compute tray houses a high-speed 96th EPYC processor, DDR 5 memory, and all the necessary IO to feed the GPUs, along with the Sela DPU, a critical part of the networking infrastructure for front-end and scale-out bandwidth delivery.

A：The volcano AI node allows for up to 6 volcano nicks on a board, using the open alternative Ethernet standard. Each Helios compute tray includes two volcano boards for scale-out and scale-up networking. Additionally, there are 6 dedicated networking bricks handling scale-up networking, connecting 75 GPUs together with UA link over Ethernet with silicon from partners, demonstrating an open ecosystem approach.

A：Helios delivers 15% more compute, 50% more Hpm form memory capacity and memory bandwidth, and 50% more scale out bandwidth compared to the competition. This means Helios can deliver more performance for large models, more capacity for longer context, and the bandwidth to scale across thousands of GPUs.

A：The production hardware weighs more than 160 pounds and stands less than 2 inches tall. It comprises more than 1,000 CDNA 5 GPU compute units, over 4,600 Ze 6 CPU cores, and 31 TB of HBM4 memory, all within a single rack. This amount of technology is required to run genetic AI at scale.

A：Helios is expected to start shipping at the end of the third quarter and to ramp up into the fourth quarter and the second half of the year. Customer demand for Helios is extremely strong, and it is being adopted by leading AI labs.

A：Anthropic's compute strategy is to ensure they have the best chips for the best workloads as the industry scale grows. They have been working to optimize their use of different chips and are excited about the performance and ease of use of Helios, which they plan to deploy up to 2 GW of.

A：The deployment of up to 2 GW of Helios is significant because Helios is an amazing machine that is performing exceptionally well, as evidenced by an engineer being able to set it up and see performance improvements in their leading model over the weekend. The deployment also reflects a strong belief in the capabilities of AMD's architecture to service the market.

A：AMD and Anthropic have a strong collaboration where they are working together to familiarize leading foundational models with AMD architecture, with a belief that this will improve services to the overall market. They are particularly excited about working on highly engineering-specific workloads to further advance AI technology.

A：Key areas of excitement and potential collaboration between AMD and Anthropic include working together for scale-up to deliver better compute results and models that add value to downstream users. Another area is security, where both companies can work to ensure safety at the chip layer, server level, and throughout the network, benefiting not just themselves but the entire industry.

A：Helios provides significant performance benefits, with up to 34 times more throughput at high concurrency compared to the prior generation. It also reduces the cost per token by up to 18 times compared to previous generations. In terms of system performance and cost, Helios offers up to 10% to 15% more performance than the competition while also providing up to 30% more tokens per dollar, resulting in a strong overall value proposition.

A：The collaboration with AMD started on Mi 300 and expanded to 355. The team optimized the software stack, networking, and deployed models on AMD infrastructure. Recently, they started working with Helios racks, with engineers from both teams working side by side to optimize the software stack and run GPT-class workloads. They anticipate deploying Helios at a massive scale starting towards the end of the year and accelerating throughout 2027.

A：AI is expected to recursively improve the systems it runs on and eventually do research itself, reducing the time expert engineers spend on optimizing and enabling more efficient model deployment. AMD is involved by working with researchers to develop AI solutions that can automate significant portions of the process of programming to an AMD GPU, thus speeding up the transition from new model development to production deployment.

A：The rise of AI is creating a new class of infrastructure that places greater importance on computing power, as reflected in the development of Venice, which is designed for this emergent need. It involves different types of server computing with varying workloads - GPU servers, AI genic servers, and traditional general-purpose servers. Having the right CPU for each workload is essential, and Venice is positioned to lead across all three classes of workloads.

A：The 6th generation of Epic CPUs, including the Z 6 core with higher IPC and frequency, delivers up to 1.8 times more performance than the previous generation. These CPUs are built on TSMC's newest 2nm process and can support up to 512 threads per socket, offering the highest compute density while doubling memory and IO bandwidth. The chiplet architecture of Venice allows for an entire family of chips that cater to different needs, from high-frequency cores for AI host nodes to more general-purpose designs. This versatility and performance make the 6th generation of Epic CPUs the best in the data center.

A：Having a consistent CPU architecture enables the development of a range of chips that can cater to different types of workloads, as seen in the Epic CPU family. This approach allows for optimization across different workloads like AI, general-purpose servers, and high-performance computing. The versatility and the ability to scale performance and efficiency make this architecture unique in the industry, and it underpins the leadership of the 6th generation Epic CPUs.

A：Optimizing data centers and racks for genetic AI is significant because it allows for more agents per rack, thereby providing customers with the choice to pick the right cooling, space requirements, and overall system design that best suits their data center needs.

A：Venice is in full production and is experiencing incredible customer demand, which is the strongest for any new EPIC generation.

A：Every major server OEM and every major cloud provider are on track to begin rolling out Venice in the fourth quarter.

A：There is a need to consider data centers as integrated systems because the demand for inference training and recommendation systems is increasing exponentially, and there is a vision to deliver personal superintelligence to billions of people. Coordinating the entire system, including servers, hardware, networking, cooling, and power, is crucial as none of these components are optional.

A：The shift in workload and demand presents opportunities for collaboration because there is a need for flexibility and heterogeneity in the industry. No single company or partner can address all needs, so collaboration is key, especially with partners like AMD, to co-create and design systems.

A：The partnership between Meta and AMD contributes to innovation in CPU infrastructure through long-term collaboration, deployment of multiple generations of CPUs and GPUs, and development of Rescale OCP systems together. The partnership spans multiple products, including CPUs and GPUs, and is characterized by ongoing co-design and co-creation efforts.

A：The characteristics of different inference workloads include varying needs for throughput, latency, and compute power. Some workloads require maximum throughput or low cost, others require balanced throughput, and a new class demands very fast results. Disaggregated inference addresses these workloads by allowing independent tuning of compute and memory bandwidth requirements and delivering both great compute and great memory bandwidth.

A：Cerebras provides ultra low latency inference solutions through the development of the world's largest and fastest chip, which is then packaged into systems, racks, and clusters. This solution is designed to meet the needs of customers who require rapid results and operates in conjunction with AMD CPUs and Cerebras' Helios and Wafer Scale Engine to create a disaggregated solution.

A：The new solution combines the industry leaders in performance and memory capacity (Helios rack) with the highest SRAM and memory bandwidth capabilities, which allows for unmatched performance and capacity in the industry.

A：The new solution will be first available in the cloud with the Cerebros platform and will be later released at a store near the customer.

A：The new solution gives customers the choice to build the solution that fits their needs, offering a huge step up in capabilities for the ultra low latency segment.

A：The collaboration between the speaker's company and Cerebrals has led to a solution that offers five times the throughput while maintaining extraordinary speed, which is very compelling to customers.

A：AI is transforming software development for GPUs by reducing the time to bring up workloads, making optimization automated, and removing barriers to broad adoption. It is anticipated that AI will significantly reduce the time it takes to program GPUs and make the process more efficient.

A：Racom AI is an AI-assisted GPU programming platform that aims to simplify GPU programming for developers by allowing them to use AI coding agents to optimize their workloads, thereby improving performance and reducing complexity.

A：The current software stack developed by the speaker's company, which includes features like Hyperloop, is significantly more advanced than previous versions. It is able to analyze workloads, tune configurations, select and tune kernels, adjust parallelism strategies, and iterate towards performance goals.

A：The new AI-assisted tools, Racom AI, have resulted in performance gains of up to 38% for specific models like MiniMax M with BLM. The tools also provide up to a 2.4 times speed improvement in training performance, with innovations such as fused flash attention, more efficient checkpointing, and advanced parallelism strategies.

A：The OpenAI team has been instrumental in pushing the limits of AI infrastructure. They have developed solutions like Triton and Gluon, which have evolved to meet the needs of different workflows and have been optimized for platforms like AMD. Their collaboration with the speaker's company has been pivotal in advancing software development and the frontier of AI.

A：The highlighted collaboration involves teams working closely to use AI to program AMD GPUs.

A：AI has significantly improved in generating high-quality GPU kernels over the past six months.

A：The two main points are the rapid improvement in AI agents' capabilities and the benefits of AMD's open software approach to advance AI.

A：AMD's open source compiler stack enables AI agents to perform very low level code generation, including instruction scheduling in assembly, which has led to very significant performance gains.

A：The speaker describes AI as picking up on the open code produced by AMD, leading to a big value proposition, particularly in creating powerful capabilities through the combination of better abstractions with AI-assisted development and hardware-software co-design.

A：The Mi 430x is built with AMD's modular chiplet architecture, providing native FP64 hardware, which is crucial for scientific computing, and offers leadership performance across both AI and HPC workloads.

A：The industry is experiencing an inflection point in the ability of AI to program complex hardware systems, which is making it dramatically easier to program AI systems.

A：AMD's strategy is to deliver the right compute tangent to the right workload, continuously drive optimality across their portfolio, and support AI in data centers, enterprise, personal, and physical AI forms.

A：Different computing environments have unique requirements: enterprises are often constrained by power and cooling, personal AI requires limited compute and memory footprint, and physical AI must perform reliably in real-world environments. AMD addresses these with a complete portfolio spanning all development models.

A：The Epic portfolio spans from 8 to 256 cores with various power, frequency, and IO capabilities, providing flexibility for customers to choose the right solution for their workload and optimize costs.

A：Enterprises are concerned about predictable infrastructure costs, data security, and integrating AI into their existing infrastructure. They require a distributed deployment model encompassing a range of AI deployment environments.

A：The Instinct Mi 350p is designed to fit within the power and cooling constraints of enterprise servers without requiring upgrades, supporting up to 260 billion parameters, and delivering superior performance and economics compared to competitors.

A：The Instinct Mi 350p supports up to 260 billion parameters, offers up to four times the tokens per dollar compared to the competition, and provides higher productivity with up to 2 to 5 times faster tokens per second.

A：AMD's IT team has tested AI in autonomous threat detection and a personalized AI assistant, reducing token costs by 43% while delivering up to 3x faster response times for local workloads.

A：AMD supports customers with a complete ecosystem of software and solutions, including numerous AI ISVs, zero-day support for leading open models, robust frameworks, and trusted platform software and OEM partnerships.

A：The overarching goal of AMD's strategy is to help customers deploy AI where it creates the most value, whether in the cloud, on-premise, or at the edge.

A：The primary applications of AI within the enterprise include fraud detection, customer care, and optimizing the placement of cell towers or Ran that is the most effective use of it inside of a network.

A：The transcription of the over 300,000 calls received daily contributes to an enormous workload, from which insights are driven back into the business.

A：The work being done with AI spans across various business functions like HR, finance, cash forecasting, and employee onboarding, which signifies the company's broader use of AI in rebuilding entire workflows.

A：Having world-class teams and people who think in workflows is important because the process involves a lot of moving from one stage to another within a workflow, which requires well-coordinated efforts.

A：The partnership with AMD benefits the company by leveraging open source models, different sets of chip sets, and building and training models. The company has been able to drive down token costs by post-training a model on AMD and achieving performance and accuracy comparable to more expensive chip sets.

A：Being the first telecom company to train a model on AMD signifies a pioneering effort in using open source models and a commitment to leveraging the capabilities of AMD in the telecom industry.

A：The purpose of Oel 1.0 and Oel 2.0 is to provide a set of open models for the industry. Oel 1.0 was announced over a year ago with over 18 million downloads, and Oel 2.0, which is more advanced and trained on AMD, is being made available via open source to showcase the significance of training models and how they work within an enterprise.

A：The company is extending its leadership in enterprise computing to AI by listening to customers and delivering solutions for their workloads, and by focusing on helping customers accelerate their AI journey.

A：Personal AI plays a role in reshaping the computing landscape by extending the reach of AI beyond data centers. It involves agents that can enhance productivity and understanding in the physical world, requiring a compute ecosystem to scale together.

A：Personal AI impacts data centers by shifting some workloads from the cloud to PCs, reducing infrastructure costs and allowing data centers to focus on larger and more demanding tasks. It also brings advantages such as data staying with the user, applications remaining responsive, and systems working even without a network.

A：The new generation of smaller AI models affects the computing landscape by dramatically reducing the compute required to deliver advanced intelligence, making it possible to run workloads once reserved for the cloud on personal computing engines, thus enhancing the efficiency and optimization of AI development.

A：The speaker reveals that the partnership with Hugging Face will bring the OpenAI ecosystem directly onto Radeon AI Halo, providing developers with optimized performance for open models and GPT flows. This partnership benefits developers by offering them access to the right models, optimization for the platform, and the ability to turn them into real applications quickly.

A：The next phase for AI involves deploying AI at scale, with a fundamental shift in patterns of inferencing moving from a human-led, spiky pattern to a more persistent demand signal for infrastructure. The challenges enterprises face include security concerns, cost management, and the need to operate a large number of AI systems with the appropriate security, governance, and control.

A：The critical factors to consider when deploying AI in enterprises include ensuring an adequate amount of network bandwidth to support the needs of local agents, managing token costs, monitoring agent behavior for safety and security, and having a governance and management system in place.

A：The full stack for AI deployment includes an isolated secure agent sandbox, intelligent routing, an MCP (likely referring to a Core Set of Integrations), security policy enforcement, resiliency in the infrastructure stack, observability of the apparatus and infrastructure, monitoring of agent behavior and tokenomics, and a unified management plane for control.

A：Enterprises can manage their AI deployments at scale by utilizing a management plane like Cisco Cloud Control to oversee the entire estate of AI devices, whether they are on desktops, data centers, or in the cloud. This allows for full visibility and control over the entire deployment.

A：The new AMD products for AI and robotics, such as the AMD Creo AI system on module and the c AI robots developer platform, provide faster reactions, greater capacity for complex work, and ensure safety and control. They bring together CPU, GPU, NPU, and unified memory for simultaneous and real-time processing, and offer a complete development path from concept to real-world application.

A：AMD's future roadmap for AI and computing products includes the introduction of new CPU lines such as Florence and Ravenna, as well as continued development of their Instinct GPUs with new generations every year, maintaining a cadence of performance improvement. AMD also plans to deliver new Helios systems every year to support larger models and better total cost of ownership.