Tag: ibm

Getting to Know IBM FlashSystem 9100 | The Performance of Flash and NVMe & the Reliability of FlashCore

Getting to Know IBM FlashSystem 9100 | The Performance of Flash and NVMe & the Reliability of FlashCore

The new FlashSystem 9100 combines the high performance of NVMe or IBM FlashCore drives with the advanced capabilities of the Spectrum Virtualize family software stack, in just a 2U footprint. ATS Group had the opportunity to participate in the beta program for the FlashSystem 9100, prior to its release. FlashSystem 9100 is available in two models: FlashSystem 9110 and FlashSystem 9150. The system we used during the beta program is a 9110, with dual 8-core CPUs and 128 GiB cache per node canister. A FlashSystem 9100 enclosure can be configured with up to 24 standard NVMe drives or IBM FlashCore drives; our beta system was configured with six 1.92 TB NVMe drives.

Specifications for the FlashSystem 9100 are as follows:

FlashSystem 9110


FlashSystem 9150


CPU (per node canister) Dual 8-core Intel Skylake 1.7 GHz Dual 14-core Intel Skylake 2.2 GHz
Internal M.2 boot drives (per node canister) 1 2 (mirrored)
Compression assist cards (per node canister) 1 (40Gbit) 1 (100Gbit)
Cache (per node canister) 64 GiB – 768 GiB
FlashCore drives available 4.8 TB, 9.6 TB, 19.2 TB
Industry standard NVMe drives available 1.92 TB, 3.84 TB, 7.68 TB, 15.36 TB
Rack footprint 2U
Maximum usable DRAID-6 capacity 379 TB (758 TB with 2:1 data reduction)
Number of I/O slots (per node canister) 3
Maximum number of I/O ports (per node canister) Up to 12 x 16 Gbit/s Fiber Channel

4 x 10 Gbit/s iSCSI Ethernet

Up to 6 x 25 Gbit/s iWARP or RoCE Ethernet

Up to 2 x 12 Gbit/s SAS (for expansion enclosures only)

Maximum number of control enclosures in a cluster 4 (8 node canisters)


FlashSystem 9100’s GUI presents a clean new system overview, with both front and rear views available simultaneously:

As in previous FlashSystem and Spectrum Virtualize GUIs, selecting an individual component provides more detail on that component. The FlashSystem 9100 GUI takes that one step further, however, offering the ability to select all components of a given type. For example, to highlight/select all fiber channel ports, just click on the “Fibre Channel Port” button:

The GUI is tightly coupled with IBM Storage Insights, offering to register the system in the main FlashSystem 9100 GUI:

Once the system is registered and a data collector agent installed, properties and performance of the FlashSystem 9100 can be viewed from anywhere via the Storage Insights web interface:

Considering that the FlashSystem 9100 combines flash storage with advanced Spectrum Virtualize capabilities, we performed a brief comparison of our FlashSystem 9100 and a system comprised of two SAN Volume Controller (SVC) nodes and a FlashSystem 900. As our system is a FlashSystem 9110, we would not expect it to equal the SVC/FS900’s performance. The more powerful FlashSystem 9150 should compare more favorably to the SVC/FS900. However, the following results show that even the 9110 can get close to the SVC/FS900 performance on these workloads. Also note that neither our FlashSystem 9110 nor our FlashSystem 900 are fully-populated with drives, and our FlashSystem 9110 was running beta code, so these brief comparisons should not be considered full benchmark tests.

Four 50 GiB fully-allocated volumes are presented to the test host from a two-node 2145-SV1 cluster, backed by a FlashSystem 900. Data reduction is not enabled on the SVC pool.

Likewise, four 50 GiB fully-allocated volumes are presented to the host from the FlashSystem 9100, backed by internal NVMe drives. Data reduction is not enabled on the FlashSystem 9100 pool. Our FlashSystem 9100 is equipped with standard NVMe drives, so the inline hardware-based compression of the IBM FlashCore modules was not available on our system.

The host, the SVC/FS900, and the FlashSystem 9100 are all connected to a pair of Brocade 16 Gbit/s fiber channel switches, however, the host is connected at 8 Gbit/s. Storage ports are connected at 16 Gbit/s

Using IOmeter on the host, we started an IOps-intensive read workload:

  • 100% random reads, 4 KiB transfer size, aligned on 4 KiB boundaries
  • 55 outstanding requests per LUN.

The FlashSystem 9100 volumes slightly outperformed the SVC/FS900 for read transfers/sec:

The FlashSystem 9100 volumes also exhibited slightly better read service times with this workload:

We next reconfigured the IOmeter workload to approximate that of one of our customers:

  • 80% random reads at 4 KiB, aligned on 4 KiB boundaries
  • 20% sequential writes at 32 KiB, aligned on 4 KiB boundaries.
  • 55 outstanding requests per volume.

The FlashSystem 9100 performed only slightly slower than the SVC/FS900, as expected. On the IOPs-intensive random reads, the FlashSystem 9100 volumes performed within about 12% of the SVC/FS900 volumes, with the range from about 15,500 to 17,500 transfers/sec per volume.

Read service times for all volumes were in the 0.4–0.6 ms range:

For the throughput-intensive sequential write component, the FlashSystem 9100 volumes performed within about 10% of the SVC/FS900 volumes, with write throughput ranging from 125 MiB/s to 140 MiB/s per volume.

We next configured IOmeter for a 70/30 workload:

  • 70% random 8 KiB reads, aligned on 4 KiB boundaries.
  • 30% random 8 KiB writes, aligned on 4 KiB boundaries.
  • 55 outstanding requests per volume.

On this workload, the FlashSystem 9110 performed on par with the SVC/FS900 system with regard to random read transfers per second. The volumes ranged from 13,500 to 15,000 transfers/sec per volume.

Read service times for that workload were very similar between the two systems, ranging from 0.40 to 0.55 ms:

The random write component of the workload produced similar results, with the FlashSystem 9110 slightly slower than the SVC/FS900. Write transfers/sec ranged from 5,900 to 6,400 transfers/sec per volume.

For write service times, the SVC/FS00 appears to have an advantage, posting lower write service times than the 9110. As previously noted, if we had tested a FlashSystem 9150 instead of the 9110, its results may be closer to the SVC/FS900 results.

With all three workloads, the FlashSystem 9110, which consumes 2U of rack space, performed similarly or just slightly below the SVC/FS900 system, which consumes 6U of rack space. The FlashSystem 9150 would likely perform better than the 9110.

The combination of high performance plus the advanced Spectrum Virtualize features makes the 9100 a compelling package, especially considering its small rack footprint. It is quite expandable, as well. Assuming a reasonable data reduction ratio of 2:1, the 9100 can be outfitted to store 758 TB in a single 2U enclosure. Expansion enclosures containing SAS drives can be added to the system, and Easy Tier can migrate data between the NVMe or FlashCore drives and the lower-tier SAS drives. All of these attributes combine to make the FlashSystem 9100 an attractive new offering from IBM.

About ATS

As new tech emerges offering business advantages, enterprises need support and expertise that will enable them to reap the benefits. Based near Philadelphia, the ATS Group offers agile services aligned with modern IT innovations, providing a critical competitive edge. For almost 20 years, our consultants have worked together to provide independent and objective technical advice, creative infrastructure consulting and managed support services for organizations of all sizes. Our specialists help clients store, protect, and manage their data, while optimizing performance and efficiency. The ATS Group specializes in server and storage system integration, containerized workloads, high performance computing (HPC), software defined infrastructure, DevOps, data protection and storage management, cloud consulting, infrastructure performance management and real-time monitoring for cloud, on-premises and hybrid solutions. The ATS Group supports solutions from today’s top IT vendors including IBM, VMware, Oracle, AWS, Microsoft, Cisco, Lenovo, Pure Storage and Red Hat.

Did this content resonate with you and your organization? Download the full version of the Getting to Know IBM FlashSystem 9100 | The Performance of Flash and NVMe & the Reliability of FlashCore document to share with peers.


Point of View: IBM Follows New Recipe to Deliver Performance Gains with POWER9 Servers

Point of View: IBM Follows New Recipe to Deliver Performance Gains with POWER9 Servers

By Blake Basom | Sr. Systems Engineer (The ATS Group)

IBM has been producing servers based on their line of Power processors for close to 30 years. These servers continue to improve with each new generation, and POWER9 is no exception. The difference this time derives from the ways in which IBM achieved those performance gains.

With POWER9, IBM continues to separate themselves from the competition, offering a host of improvements over their POWER8 predecessors. In fact, IBM claims that POWER9 servers offer the following benefits over their competitors:

  • 5x max I/O bandwidth vs. x86
  • 2x high performance cores vs. x86
  • 6x more RAM supported vs. x86
  • 8x more memory bandwidth vs. x86

In this document, I will delve into some of the more significant features and changes of the POWER9 servers and offer my opinion on the benefits. This document is intended to be an overview of POWER9’s new and enhanced features, and while it will be fairly detailed, it is not meant to be a deep dive in any specific technology. The features that we look at will be grouped into categories of Processor, Memory, and I/O, but first an overview of the new server line.

POWER9 Server Line

IBM began their launch of POWER9 servers with the AC922 server in late 2017. This server is designed specifically for compute-heavy Artificial Intelligence (AI) and cognitive workloads, rather than for general computing. This system was the first to embed PCIe 4.0, Nvidia NVLink, and OpenCAPI technologies. As a result, IBM claims that the AC922 enables data to move 9.5 times faster than on PCIe 3.0 based x86 systems.

Servers designed for a more general workload began to be launched in early 2018.  They began with the “Scale Out” versions of their servers – one and two-socket rack-mountable servers in sizes from 1U – 4U.  The S914, S922, and S924 PowerVM based servers are in the traditional mold, supporting AIX, IBM i, and Linux workloads. The L922 server is a Linux-only model, while the H922 and H924 servers have been optimized for SAP HANA. Also offered are LC921 and LC922 models, which are processor and storage dense servers designed for Linux Clusters.

Finalizing the POWER9 server line, IBM announced the larger and more powerful “Scale Up” models in August, 2018. These enterprise servers offer increased computing capability, along with enhanced security and availability, and simplified cloud management. The 4-socket E950 offers up to 48 processor cores and up to 16TB of memory in a 4U package. Last, but far from least, the E980 represents the top of the POWER9 server line, offering up to 192 processor cores and 64TB of memory.


The most obvious place to start when looking at the POWER9 servers is the POWER9 processor itself. In years past, performance improvements were achieved in part through improving the fabrication process to reduce the transistor sizes, allowing the clock to run faster. While the fabrication process continues to improve and transistor sizes continue to shrink, server manufacturers are not greatly increasing clock speeds, leaving performance improvements to be achieved through other means. In the case of the POWER9 processor, it is primarily due to improving processor pipeline efficiency, increasing the data flow between components, and allowing for faster access from external sources.

The POWER9 processor was fabricated via a highly advanced 14nm finFET Silicon-On-Insulator lithography process (using a 17-layer metal stack), an improvement from the 22nm process that was used for POWER8. This allowed IBM to jam a total of 8 billion transistors in each chip, compared to 4.2 billion in POWER8. Clock speeds run up to 4 GHz, which is similar to POWER8.

The POWER9 chip is a more modular design, and performance was improved by shortening the pipeline, improving fixed-point and floating-point operations, and improving instruction management. These changes allow more instructions to be completed per clock cycle, leading to performance improvements without raising the clock speed. Increasing the amount of on-chip memory (particularly L3 cache) helps as well, and on-chip switching bandwidth of over 7 TB/s allows data to move in and out of the processor cores at 256 GB/s in the SMT8 model.

You may be thinking, “SMT8 model? Aren’t all POWER9 chips SMT8, as the POWER8 chips were?” Actually, no. IBM is producing two main variants of the POWER9 processor – the PowerVM based general purpose servers will use a full SMT8 processor (which allows up to 8 threads per core), while certain non-PowerVM based Linux models will use SMT4 versions of the POWER9 processors (which only allow up to 4 threads per core). This may sound like a step backward, but it is a result of IBM listening to its customers and partners. Basically, IBM learned that a segment of the Linux market desired the reduced SMT version, which allowed more cores to be packaged in a single chip.  In fact, the SMT4 versions will allow up to 24 cores per die, while the SMT8 models only offer up to 12 cores.

Speaking of SMT4 vs. SMT8, what is the best multi-threading mode in which to run the new processors? When IBM introduced SMT8 with POWER8 processors, there were some performance problems initially. The problems weren’t necessarily severe, but running in SMT8 mode didn’t necessarily equate to much improvement in processing power, and in some cases IBM was recommending running POWER8 servers in SMT4 mode. This issue has seemingly been fixed in POWER9, with SMT8 being the preferred mode for most applications, offering a distinct performance boost over running in SMT4 mode (under most circumstances).

IBM also introduced Workload Optimized Frequency with POWER9, where the processor can dynamically change clock speeds based on the running workload, to allow for enery savings when the workload is low, with the ability to quickly ramp up when needed. This feature can be controlled through processor mode settings and can be changed without a reboot.

All of that sounds nice, but what does it really mean? How much faster are the POWER9 processors? Well, of course it varies by server model and workload, but in general you can expect a 30-50% improvement over comparable POWER8 models, along with 20-30% improvement in price/performance ratio (more bang for your buck).

Note that when migrating workload from POWER8 to POWER9, you will likely want to reduce the number of virtual CPUs, which may improve performance, while reducing software licensing costs. Each case will be unique, so testing a specific workload with different numbers of VCPUs will reveal the optimal allocation. Likewise, running tests in both SMT4 mode and SMT8 mode will show which threading mode is best.


The POWER9 servers use top of the line DDR4 memory (some of the later POWER8 models used this as well). The SMT4 processor models allow for direct attached memory DIMMs, while the SMT8 versions allow more memory to be attached, via buffers. The SMT4 models offer up to 120 GB/s of sustained memory bandwidth, while the SMT8 models offer up to 230 GB/s of sustained bandwidth with theoretical peaks of 340 GB/s. Memory capacity varies by model, up to 4 TB for Scale Out models, and up 64TB for Scale Up models.


Most modern servers are not self-contained, meaning they are connected to external devices for storage, networking, and increasingly for hardware acceleration devices. With the blazing speeds of current processors and memory, the computing bottleneck has shifted to Input/Output devices. IBM has spent a lot of effort in this area with the POWER9 servers, offering a number of options to improve the speed and bandwidth to attached devices.

The latest edition of PCIe (Gen4) is available in POWER9 servers, offering up to twice the bandwidth of PCIe Gen3 (note that Gen3 adapters will work in Gen4 slots, albeit with the Gen3 bandwidth). 48 lanes of PCIe Gen4 adds up to 192 GB/s duplex bandwidth to attached devices.  In addition to traditional PCI adapters for network and SAN connectivity, some PCIe Gen4 slots are enabled for CAPI 2.0 devices such as ASICs and FPGAs. CAPI 2.0 using PCIe Gen4 offers 4x the bandwidth of CAPI 1.0 on POWER8.

Additional connectivity is provided by a 25 Gb/s Common Link – 48 lanes provides up to 300 GB/s bandwidth for devices attached via NVLink 2.0 or OpenCAPI 3.0 (not available on PowerVM based servers). NVLink can be used for high speed GPU attachment, while OpenCAPI is an upcoming open hardware standard that is supported by a consortium of industry heavyweights, which will be used to connect components like high-speed network and SAN adapters, as well as additional memory and GPU accelerators.

POWER9 provides support for the next generation of SR-IOV Ethernet adapters – with increased port speeds of 10Gb, 25Gb, 40Gb, and 100Gb. Additional enhancements allow more VFs per port (target 60VFs per port / 120 VFs per adapter for 100Gb adapters), as well as vNIC and vNIC failover support for Linux.

Server I/O performance is also improved by the on-chip acceleration capabilities of the POWER9 processors themselves, which speed up the common but intensive tasks of compression/decompression and encryption/decryption.

Some POWER9 servers also support internal Non-Volatile Memory (NVMe) devices. These bootable disks are meant primarily for operating systems, offering high-speed access with low latency, but in a read-mostly format.


When you put it all together, it is clear that IBM put an emphasis on overall server performance with their line of POWER9 servers, rather than just trying to crank out the fastest processor that they could. By focusing on I/O enhancements, and partnering with great companies across the industry, they have achieved some impressive results.  But they didn’t forget about the processor either – the POWER9 processor improved upon an already industry leading standard.  From general purpose Scale Out servers, all the way up to the enterprise class Scale Up servers, IBM has provided a robust line of servers to meet the UNIX computing needs of users around the globe. And with certain models customized for specific technologies, users can expect optimized performance for their specific needs. As a longtime user, administrator, and consultant for IBM Power servers, I think that POWER9 represents another impressive step forward for IBM, offering endless possibilities for world class computing.

Did this content resonate with you and your organization? Download the full version of the Point of View: IBM Follows New Recipe to Deliver Performance Gains with POWER9 Servers document to share with peers.

We’re headed to Hollywood | Join Galileo & the ATS Group at the Upcoming IBM Systems Technical University

See What's Happening in Hollywood at the IBM Systems Technical University

IBM events offer a variety of ways to learn, grow, and connect. As an IBM Gold Business Partner, you can always find the teams from Galileo Performance Explorer and the ATS Group at IBM Systems Technical Universities (#IBMTechU) events, and this October is no different! Thousands of IT professionals gather to experience unparalleled technical education at an IBM Tech U events that feature technical sessions about Galileo-supported technologies, such as IBM Storage, IBM Power Systems and IBM z.

Learn how to plan, architect, install, configure, migrate, operate and tune your IBM infrastructure for today and tomorrow at the IBM Systems Technical University coming up in October in Hollywood, FL. Packed with a punch, this event offers focused, in-depth training sessions, hands-on labs and demos delivered by IBM Distinguished Engineers, developers or product experts.

IBM Systems Technical University
Featuring IBM Power Systems and IBM Storage
October 15 – 19  | Hollywood, Florida USA

Join Galileo’s VIP Reception on Day 1

We’d love to tell you how Galileo can support your upcoming IBM infrastructure projects. With tagging and trending across all IT assets, Galileo provides a unique, comprehensive view that streamlines problem determination and empowers successful strategic initiatives such as cloud migration, capacity planning and server consolidation. Want to learn more? Our team is hosting a fun VIP Welcome Reception at Portico to kick off the week at IBM Systems Technical University. Please RSVP to join us and feel free share with others in your organization that will be in Hollywood!

Galileo VIP Reception at IBM Systems Technical University
RSVP: Monday, October 15th from 7-10PM
Portico at The Diplomat Beach Resort
3460 South Ocean Drive | Hollywood, FL

Join the team from Galileo Performance Explorer for a Welcome Reception at Portico!

Join the team from Galileo for a Welcome Reception at Portico during the IBM Systems Tech U in Hollywood!

Lunch & Learn with Galileo on Day 2

The reliability of your infrastructure takes on a new level of importance with today’s data-intensive workloads. Through Galileo, organizations can streamline support and develop realistic roadmaps for growth and transformation through data visualization, trending and tagging capabilities. Take a break from the hustle of the conference and join us for an intimate Lunch and Learn at Point Royal at The Diplomat Beach Resort.

Lunch & Learn with Galileo at IBM Systems Technical University
RSVP: Tuesday, October 16th from 12:00 to 1:30PM
Point Royal at The Diplomat Beach Resort
3555 South Ocean Drive | Hollywood, FL

Join Galileo for an intimate lunch and learn on October 16th at Point Royal during IBM Tech U in Hollywood.

What’s new in Hollywood?

IBM is always looking to further the skills and knowledge of their partners and customers, and empower the IT community with the tools necessary to adapt and optimize their infrastructures for tomorrow. At the IBM Systems Technical University in Hollywood, there are a couple of new training sessions to take advantage of:

Professional and Leadership Development Training
Whether you are already a leader or aspiring to be one, you can find content to support your professional and leadership growth.

  1. Stay up to date on hot topics like IBM Design Thinking and Digital Transformation.
  2. Give your left brain a break during TechU to work on your soft skills. You can grow your career with a focus on things like presentation techniques and communication tips, and even how to make your IT project successful.
  3. Check out the current list of topics at your preferred TechU.

Cognitive Systems Track
Cognitive solutions are set to transform the world in dramatic ways. At TechU, you can leverage the experts to go deep on AI on IBM Power Systems. IBM experts will share their deep technical expertise in applied AI, model tuning, GPUs and distributed systems, workload provisioning and management. You’ll find concepts of AI, machine learning and deep learning along with industry use cases, best practices, development frameworks and industry tools. TechU offers focused, in-depth technical training sessions and labs. They will share how to :

  1. Leverage the capabilities of POWER9 for AI
  2. Deploy a fully optimized AI platform with IBM PowerAI
  3. Collaborate on AI projects with the IBM Data Science Experience
  4. Integrate and manage your AI, Spark and Hadoop big data workloads
  5. Use medical image classification with Machine Learning technology
  6. Leverage a high performance file system with IBM Spectrum Scale
  7. Optimize distributed learning models with IBM Deep Impact

Galileo is Ready for IBM Storage

Galileo Performance Explorer recently announced a partnership with the Ready for IBM Storage Program to empower IBM clients to proactively monitor, manage and optimize their IT environments. Galileo is now available as a validated IBM PartnerWorld solution, enabling Galileo to support customers, VARs, MSPs and enterprise partners worldwide. The Ready for IBM Storage program integrates and validates business partner offerings with IBM technology to enable end-to-end use cases, simplify deployment and reduce risk for clients.

Galileo support an array of IBM servers, storage, systems and cloud, ensuring users have the most insight into the capacity and performance of their most crucial systems. Galileo provides monitoring for IBM server, storage and cloud systems including:

  1. IBM AIX
  2. IBM i
  3. IBM Spectrum Scale
  4. IBM DS3000, DS4000 and DS5000
  5. IBM DS8000
  6. IBM FlashSystem
  8. IBM Spectrum Virtualize
  9. IBM V7000 Unified
  10. IBM VIX
  11. IBM Power
  12. IBM z
  13. IBM Cloud

Other solutions have gaps in monitoring, especially when it comes to cloud infrastructure. Galileo monitors your entire infrastructure, including on-site and cloud-based systems. To find out more about what Galileo can do for your unique IT systems, join us at IBM Systems Technical University in Hollywood or connect with us to schedule a demo or start a free trial.

Stay in the know!

Tech U features skill-building sessions that are in demand and appeal to attendees who share a technical vision and curiosity. Your event experience can be enhanced by following and discussing the topics you want to know about related to IBM Systems. Join the IBM Systems Technical Universities (TechU) LinkedIn Group to view event highlights, connect with attendees and tech experts and stay on top of the latest technology trends.

IBM Spectrum Protect and Suse Enterprise Storage 5

A Brief History…

The ATS Group has long provided services around the IBM Spectrum Protect product. It has been a mature solution that we continue to standby. We also continuously explore new technology everyday. Roughly 2 years ago I remember heavily investing time in reviewing the Open Source project Ceph. The product felt like it had all of the pieces, however, the complexity seemed rather unnecessary. Most notably performance and data consistency concerns plagued the project. Consequently, as a result, I followed the project until some major players began to take interest.

 Fast forwarding…

The Ceph project seems to have finally become mainstream. All previous issues have been largely addressed. Many large enterprises have adopted the project for a wide variety of workloads. The Suse Enterprise Storage product built on top of Ceph has since become a leader as a solution for this project. Check out our findings below in how we believe Suse Enterprise Storage 5 can complement our customers existing Spectrum Protect solutions.

18 - ATS Group - IBM Spectrum Protect - WP - Jan - V2