Point of View: IBM Follows New Recipe to Deliver Performance Gains with POWER9 Servers

Point of View: IBM Follows New Recipe to Deliver Performance Gains with POWER9 Servers

By Blake Basom | Sr. Systems Engineer (The ATS Group)

IBM has been producing servers based on their line of Power processors for close to 30 years. These servers continue to improve with each new generation, and POWER9 is no exception. The difference this time derives from the ways in which IBM achieved those performance gains.

With POWER9, IBM continues to separate themselves from the competition, offering a host of improvements over their POWER8 predecessors. In fact, IBM claims that POWER9 servers offer the following benefits over their competitors:

  • 5x max I/O bandwidth vs. x86
  • 2x high performance cores vs. x86
  • 6x more RAM supported vs. x86
  • 8x more memory bandwidth vs. x86

In this document, I will delve into some of the more significant features and changes of the POWER9 servers and offer my opinion on the benefits. This document is intended to be an overview of POWER9’s new and enhanced features, and while it will be fairly detailed, it is not meant to be a deep dive in any specific technology. The features that we look at will be grouped into categories of Processor, Memory, and I/O, but first an overview of the new server line.

POWER9 Server Line

IBM began their launch of POWER9 servers with the AC922 server in late 2017. This server is designed specifically for compute-heavy Artificial Intelligence (AI) and cognitive workloads, rather than for general computing. This system was the first to embed PCIe 4.0, Nvidia NVLink, and OpenCAPI technologies. As a result, IBM claims that the AC922 enables data to move 9.5 times faster than on PCIe 3.0 based x86 systems.

Servers designed for a more general workload began to be launched in early 2018.  They began with the “Scale Out” versions of their servers – one and two-socket rack-mountable servers in sizes from 1U – 4U.  The S914, S922, and S924 PowerVM based servers are in the traditional mold, supporting AIX, IBM i, and Linux workloads. The L922 server is a Linux-only model, while the H922 and H924 servers have been optimized for SAP HANA. Also offered are LC921 and LC922 models, which are processor and storage dense servers designed for Linux Clusters.

Finalizing the POWER9 server line, IBM announced the larger and more powerful “Scale Up” models in August, 2018. These enterprise servers offer increased computing capability, along with enhanced security and availability, and simplified cloud management. The 4-socket E950 offers up to 48 processor cores and up to 16TB of memory in a 4U package. Last, but far from least, the E980 represents the top of the POWER9 server line, offering up to 192 processor cores and 64TB of memory.


The most obvious place to start when looking at the POWER9 servers is the POWER9 processor itself. In years past, performance improvements were achieved in part through improving the fabrication process to reduce the transistor sizes, allowing the clock to run faster. While the fabrication process continues to improve and transistor sizes continue to shrink, server manufacturers are not greatly increasing clock speeds, leaving performance improvements to be achieved through other means. In the case of the POWER9 processor, it is primarily due to improving processor pipeline efficiency, increasing the data flow between components, and allowing for faster access from external sources.

The POWER9 processor was fabricated via a highly advanced 14nm finFET Silicon-On-Insulator lithography process (using a 17-layer metal stack), an improvement from the 22nm process that was used for POWER8. This allowed IBM to jam a total of 8 billion transistors in each chip, compared to 4.2 billion in POWER8. Clock speeds run up to 4 GHz, which is similar to POWER8.

The POWER9 chip is a more modular design, and performance was improved by shortening the pipeline, improving fixed-point and floating-point operations, and improving instruction management. These changes allow more instructions to be completed per clock cycle, leading to performance improvements without raising the clock speed. Increasing the amount of on-chip memory (particularly L3 cache) helps as well, and on-chip switching bandwidth of over 7 TB/s allows data to move in and out of the processor cores at 256 GB/s in the SMT8 model.

You may be thinking, “SMT8 model? Aren’t all POWER9 chips SMT8, as the POWER8 chips were?” Actually, no. IBM is producing two main variants of the POWER9 processor – the PowerVM based general purpose servers will use a full SMT8 processor (which allows up to 8 threads per core), while certain non-PowerVM based Linux models will use SMT4 versions of the POWER9 processors (which only allow up to 4 threads per core). This may sound like a step backward, but it is a result of IBM listening to its customers and partners. Basically, IBM learned that a segment of the Linux market desired the reduced SMT version, which allowed more cores to be packaged in a single chip.  In fact, the SMT4 versions will allow up to 24 cores per die, while the SMT8 models only offer up to 12 cores.

Speaking of SMT4 vs. SMT8, what is the best multi-threading mode in which to run the new processors? When IBM introduced SMT8 with POWER8 processors, there were some performance problems initially. The problems weren’t necessarily severe, but running in SMT8 mode didn’t necessarily equate to much improvement in processing power, and in some cases IBM was recommending running POWER8 servers in SMT4 mode. This issue has seemingly been fixed in POWER9, with SMT8 being the preferred mode for most applications, offering a distinct performance boost over running in SMT4 mode (under most circumstances).

IBM also introduced Workload Optimized Frequency with POWER9, where the processor can dynamically change clock speeds based on the running workload, to allow for enery savings when the workload is low, with the ability to quickly ramp up when needed. This feature can be controlled through processor mode settings and can be changed without a reboot.

All of that sounds nice, but what does it really mean? How much faster are the POWER9 processors? Well, of course it varies by server model and workload, but in general you can expect a 30-50% improvement over comparable POWER8 models, along with 20-30% improvement in price/performance ratio (more bang for your buck).

Note that when migrating workload from POWER8 to POWER9, you will likely want to reduce the number of virtual CPUs, which may improve performance, while reducing software licensing costs. Each case will be unique, so testing a specific workload with different numbers of VCPUs will reveal the optimal allocation. Likewise, running tests in both SMT4 mode and SMT8 mode will show which threading mode is best.


The POWER9 servers use top of the line DDR4 memory (some of the later POWER8 models used this as well). The SMT4 processor models allow for direct attached memory DIMMs, while the SMT8 versions allow more memory to be attached, via buffers. The SMT4 models offer up to 120 GB/s of sustained memory bandwidth, while the SMT8 models offer up to 230 GB/s of sustained bandwidth with theoretical peaks of 340 GB/s. Memory capacity varies by model, up to 4 TB for Scale Out models, and up 64TB for Scale Up models.


Most modern servers are not self-contained, meaning they are connected to external devices for storage, networking, and increasingly for hardware acceleration devices. With the blazing speeds of current processors and memory, the computing bottleneck has shifted to Input/Output devices. IBM has spent a lot of effort in this area with the POWER9 servers, offering a number of options to improve the speed and bandwidth to attached devices.

The latest edition of PCIe (Gen4) is available in POWER9 servers, offering up to twice the bandwidth of PCIe Gen3 (note that Gen3 adapters will work in Gen4 slots, albeit with the Gen3 bandwidth). 48 lanes of PCIe Gen4 adds up to 192 GB/s duplex bandwidth to attached devices.  In addition to traditional PCI adapters for network and SAN connectivity, some PCIe Gen4 slots are enabled for CAPI 2.0 devices such as ASICs and FPGAs. CAPI 2.0 using PCIe Gen4 offers 4x the bandwidth of CAPI 1.0 on POWER8.

Additional connectivity is provided by a 25 Gb/s Common Link – 48 lanes provides up to 300 GB/s bandwidth for devices attached via NVLink 2.0 or OpenCAPI 3.0 (not available on PowerVM based servers). NVLink can be used for high speed GPU attachment, while OpenCAPI is an upcoming open hardware standard that is supported by a consortium of industry heavyweights, which will be used to connect components like high-speed network and SAN adapters, as well as additional memory and GPU accelerators.

POWER9 provides support for the next generation of SR-IOV Ethernet adapters – with increased port speeds of 10Gb, 25Gb, 40Gb, and 100Gb. Additional enhancements allow more VFs per port (target 60VFs per port / 120 VFs per adapter for 100Gb adapters), as well as vNIC and vNIC failover support for Linux.

Server I/O performance is also improved by the on-chip acceleration capabilities of the POWER9 processors themselves, which speed up the common but intensive tasks of compression/decompression and encryption/decryption.

Some POWER9 servers also support internal Non-Volatile Memory (NVMe) devices. These bootable disks are meant primarily for operating systems, offering high-speed access with low latency, but in a read-mostly format.


When you put it all together, it is clear that IBM put an emphasis on overall server performance with their line of POWER9 servers, rather than just trying to crank out the fastest processor that they could. By focusing on I/O enhancements, and partnering with great companies across the industry, they have achieved some impressive results.  But they didn’t forget about the processor either – the POWER9 processor improved upon an already industry leading standard.  From general purpose Scale Out servers, all the way up to the enterprise class Scale Up servers, IBM has provided a robust line of servers to meet the UNIX computing needs of users around the globe. And with certain models customized for specific technologies, users can expect optimized performance for their specific needs. As a longtime user, administrator, and consultant for IBM Power servers, I think that POWER9 represents another impressive step forward for IBM, offering endless possibilities for world class computing.

Did this content resonate with you and your organization? Download the full version of the Point of View: IBM Follows New Recipe to Deliver Performance Gains with POWER9 Servers document to share with peers.

We’re headed to Hollywood | Join Galileo & the ATS Group at the Upcoming IBM Systems Technical University

See What's Happening in Hollywood at the IBM Systems Technical University

IBM events offer a variety of ways to learn, grow, and connect. As an IBM Gold Business Partner, you can always find the teams from Galileo Performance Explorer and the ATS Group at IBM Systems Technical Universities (#IBMTechU) events, and this October is no different! Thousands of IT professionals gather to experience unparalleled technical education at an IBM Tech U events that feature technical sessions about Galileo-supported technologies, such as IBM Storage, IBM Power Systems and IBM z.

Learn how to plan, architect, install, configure, migrate, operate and tune your IBM infrastructure for today and tomorrow at the IBM Systems Technical University coming up in October in Hollywood, FL. Packed with a punch, this event offers focused, in-depth training sessions, hands-on labs and demos delivered by IBM Distinguished Engineers, developers or product experts.

IBM Systems Technical University
Featuring IBM Power Systems and IBM Storage
October 15 – 19  | Hollywood, Florida USA

Join Galileo’s VIP Reception on Day 1

We’d love to tell you how Galileo can support your upcoming IBM infrastructure projects. With tagging and trending across all IT assets, Galileo provides a unique, comprehensive view that streamlines problem determination and empowers successful strategic initiatives such as cloud migration, capacity planning and server consolidation. Want to learn more? Our team is hosting a fun VIP Welcome Reception at Portico to kick off the week at IBM Systems Technical University. Please RSVP to join us and feel free share with others in your organization that will be in Hollywood!

Galileo VIP Reception at IBM Systems Technical University
RSVP: Monday, October 15th from 7-10PM
Portico at The Diplomat Beach Resort
3460 South Ocean Drive | Hollywood, FL

Join the team from Galileo Performance Explorer for a Welcome Reception at Portico!

Join the team from Galileo for a Welcome Reception at Portico during the IBM Systems Tech U in Hollywood!

Lunch & Learn with Galileo on Day 2

The reliability of your infrastructure takes on a new level of importance with today’s data-intensive workloads. Through Galileo, organizations can streamline support and develop realistic roadmaps for growth and transformation through data visualization, trending and tagging capabilities. Take a break from the hustle of the conference and join us for an intimate Lunch and Learn at Point Royal at The Diplomat Beach Resort.

Lunch & Learn with Galileo at IBM Systems Technical University
RSVP: Tuesday, October 16th from 12:00 to 1:30PM
Point Royal at The Diplomat Beach Resort
3555 South Ocean Drive | Hollywood, FL

Join Galileo for an intimate lunch and learn on October 16th at Point Royal during IBM Tech U in Hollywood.

What’s new in Hollywood?

IBM is always looking to further the skills and knowledge of their partners and customers, and empower the IT community with the tools necessary to adapt and optimize their infrastructures for tomorrow. At the IBM Systems Technical University in Hollywood, there are a couple of new training sessions to take advantage of:

Professional and Leadership Development Training
Whether you are already a leader or aspiring to be one, you can find content to support your professional and leadership growth.

  1. Stay up to date on hot topics like IBM Design Thinking and Digital Transformation.
  2. Give your left brain a break during TechU to work on your soft skills. You can grow your career with a focus on things like presentation techniques and communication tips, and even how to make your IT project successful.
  3. Check out the current list of topics at your preferred TechU.

Cognitive Systems Track
Cognitive solutions are set to transform the world in dramatic ways. At TechU, you can leverage the experts to go deep on AI on IBM Power Systems. IBM experts will share their deep technical expertise in applied AI, model tuning, GPUs and distributed systems, workload provisioning and management. You’ll find concepts of AI, machine learning and deep learning along with industry use cases, best practices, development frameworks and industry tools. TechU offers focused, in-depth technical training sessions and labs. They will share how to :

  1. Leverage the capabilities of POWER9 for AI
  2. Deploy a fully optimized AI platform with IBM PowerAI
  3. Collaborate on AI projects with the IBM Data Science Experience
  4. Integrate and manage your AI, Spark and Hadoop big data workloads
  5. Use medical image classification with Machine Learning technology
  6. Leverage a high performance file system with IBM Spectrum Scale
  7. Optimize distributed learning models with IBM Deep Impact

Galileo is Ready for IBM Storage

Galileo Performance Explorer recently announced a partnership with the Ready for IBM Storage Program to empower IBM clients to proactively monitor, manage and optimize their IT environments. Galileo is now available as a validated IBM PartnerWorld solution, enabling Galileo to support customers, VARs, MSPs and enterprise partners worldwide. The Ready for IBM Storage program integrates and validates business partner offerings with IBM technology to enable end-to-end use cases, simplify deployment and reduce risk for clients.

Galileo support an array of IBM servers, storage, systems and cloud, ensuring users have the most insight into the capacity and performance of their most crucial systems. Galileo provides monitoring for IBM server, storage and cloud systems including:

  1. IBM AIX
  2. IBM i
  3. IBM Spectrum Scale
  4. IBM DS3000, DS4000 and DS5000
  5. IBM DS8000
  6. IBM FlashSystem
  8. IBM Spectrum Virtualize
  9. IBM V7000 Unified
  10. IBM VIX
  11. IBM Power
  12. IBM z
  13. IBM Cloud

Other solutions have gaps in monitoring, especially when it comes to cloud infrastructure. Galileo monitors your entire infrastructure, including on-site and cloud-based systems. To find out more about what Galileo can do for your unique IT systems, join us at IBM Systems Technical University in Hollywood or connect with us to schedule a demo or start a free trial.

Stay in the know!

Tech U features skill-building sessions that are in demand and appeal to attendees who share a technical vision and curiosity. Your event experience can be enhanced by following and discussing the topics you want to know about related to IBM Systems. Join the IBM Systems Technical Universities (TechU) LinkedIn Group to view event highlights, connect with attendees and tech experts and stay on top of the latest technology trends.