Day: June 18, 2018

File Operations Monitoring With IBM Spectrum Scale File Audit Logging

File Operations Monitoring With IBM Spectrum Scale File Audit Logging

With the release of Spectrum Scale 5.0, IBM is now offering File Audit Logging capability. Spectrum Scale File Audit Logging takes locally generated file system events and puts them on a multi-node message queue from which they are consumed and written to a retention enabled fileset. These events, called lightweight events, occur at the file system level. They are protocol agnostic, which means that they capture all access to a monitored file system from protocol exports to even root access that occurs directly on nodes. Spectrum Scale File Audit Logging is integrated into the system health infrastructure. Alerts are generated for elements of the message queue and the processes that consume the events and create the audit logs.

Key features of Spectrum Scale File Audit Logging

  • Creates an audit trail of every file access in a Spectrum Scale filesystem.
  • Allows monitoring of file access by every user including super user.
  • Monitors all file access via Native GPFS (directly on the Spectrum Scale node) or SMB and NFS from Protocol Nodes.
  • Enables companies to implement strict Corporate security policies with a wide range of Spectrum Scale ACLs and File Audit Logging events.

Use Cases

Data Security and Monitoring Data Access is increasingly becoming vital for companies to protect business critical data from insider threats and security breaches.

While the IBM Spectrum Scale Encryption process helps to secure the data at  rest, the new File Audit Logging feature helps customers monitor business critical data access at a more granular level.

As an IBM Business Partner, ATS Group provides an end-to-end Spectrum Scale cluster solutions architecture, implementation and managed support services to our Financial and Genomic Research customers.

One of our customers uses Spectrum Scale – SAS Grid cluster to store data for their economic, financial and strategic consulting services. They are currently using native Redhat Linux auditing process to monitor the data access from a Spectrum Scale filesystem. They intend to switch from native Redhat Linux Auditing process to Spectrum Scale File Audit Logging in order to monitory their business critical data access. Their business consultants access the data using many Analytical applications and also from Windows desktops via SMB. They asked us to evaluate the new Spectrum Scale File Audit Logging features in depth and come up with a detailed report about its capabilities, requirements, and performance.

Majority of clustered filesystems, which are POSIX compliant and support NFS/SMB ACLs have some kind of file access auditing process where as other filesystems which are not POSIX compliant like HDFS usually does not provide file access auditing functionality.  File operations on Windows based filesystems like NTFS can be monitored by locally or by enabling Windows file auditing policy on the Active Directory server. Lustre which is an open source clustered filesystem, also offers file auditing capability. Filesystem OneFS, used by EMC Isilon, also has built-in File Auditing capability. Filesystem CEPH which is primarily used as an object store does not provide File access auditing. Though few of the clustered filesystem have some kind of auditing functionality, Spectrum Scale File Audit Logging offers more flexibility and scalability. File operations across the entire cluster can be monitored by configuring Spectrum Scale File Audit logging on only three quorum nodes in the cluster. Spectrum Scale File Audit Logging captures all file operations in the cluster, when multiple applications and users access files on any node within the cluster. There are a wide variety of commercial software packages available to analyze, report and generate alerting events from the Spectrum Scale File Audit Logging information.

Spectrum Scale File Audit Logging – Requirements

  • Spectrum Scale Advanced or Data Management Edition
  • x86 or Power-8 Little Endian (Not supported on Big Endian)
  • RHEL 7.x or Ubuntu 16.04 and above.
  • 3 x Spectrum Scale Quorum Nodes running RHEL 7.x or Ubuntu that act as Apache – Zookeeper Nodes
  • 3 x Spectrum Scale Nodes running RHEL 7.x or Ubuntu with a minimum of 5 GB local disk space that act as Apache – Kafka Message Queue Servers (Brokers)
  • Spectrum Scale Nodes that act as Zookeeper and Kafka message queue servers need ports 2181, 9092 and 9093 along with port range 2888:3888 open for communication.

Spectrum Scale File Audit Logging Proof of Concept

This document describes a complete Proof of Concept of Spectrum Scale File Audit Logging completed on IBM Power-8 servers at ATS Group Innovation Center.

PoC – Environment
  1. 2x Power-8 S822 Servers
  2. PowerVM
  3. 500 GB of IBM Flash Storage
  4. 3x PowerVM LPARs running CentOS 7.4 Power-8 Little Endian Kernel
  5. IBM Spectrum Scale 5.0.0.2 Advanced Edition
PoC – Objectives
  1. Create a 3 -node Spectrum Scale cluster: 2x NSD Servers which also serve as Protocol Nodes, 1x NSD Client, All 3 nodes act as File Audit Logging Zookeeper Nodes and Kafka Message brokers.
  2. Create a 250 GB Spectrum Scale filesystem on 5 x 50 GB NSDs from IBM flash storage.
  3. Install and Configure and enable File Audit Logging on the Spectrum Scale Filesystem.
  4. Access the Spectrum Scale filesystem on Windows Server via SMB.
  5. Verify the entries and events generated by FAL File Audit Logging in the audit log.
PoC – Spectrum Scale Cluster Install and File Audit Logging Configuration
  1. Download IBM Spectrum Scale 5.0.0.2 Advanced Edition package SpectrumScaleProto_ADV500PWRleLNX.tar.gz from IBM passport website.
  2. Extract the RPMs and installation script from the package.
  3. Enable File Audit Logging
  4. Define Cluster name, Cluster Nodes, NSD servers, Protocol Nodes and configuration using the spectrumscale command.
  5. Define Spectrum Scale filesystems.
  6. Run Spectrum Scale Installation pre-requisite check:
  7. Proceed with Spectrum Scale RPMs install.
  8. Define filesystem for Audit Logging, Fileset for Audit logs and retention time.
  9. Run Spectrum Scale deploy pre-requisite check
  10. Deploy Spectrum Cluster and File Audit Logging which will install Apache-Zookeeper and Kafka RPMs.
  11. Verify Audit Logging is enabled on the filesystem.
  12. Verify Audit Logging Message Queues.
  13. Verify the date-wise Audit Logs in Audit Log Fileset.

Spectrum Scale File Audit Logging Functionality Testing – Native GPFS File Access

At this point we are ready to test some basic File Audit logging functionality.

  1. Let’s create a test file, read it , change its permissions, rename and delete it as shown.
  2. Verify the entries in the Audit Log.

Spectrum Scale File Audit Logging  Functionality Testing – SMB Access on Windows

  1. Create a test GPFS fileset and link the fileset.
  2. Create a SMB Share:
  3. Map the SMB Share on to a Windows Desktop and create a test folder.
  4. Create few test files using Windows fsutil or any other methods.
  5. Verify the entries in the Audit file log.
  6. Test a ACL Change on the file from Windows.
  7. Verify the Windows ACL Change in the Audit log.

Spectrum Scale File Audit Logging – Limitations

  • Supported for filesystems created on Spectrum Scale 5.0.0 or later. Filesystems created on older versions of Spectrum Scale need to be upgraded to 5.0.0 or latest after upgrading all cluster nodes.
  • Requires at least three Linux Quorum cluster nodes with x86/Power 8 Little Endian.
  • Protocol Nodes or NSD servers can function as File Audit Logging nodes, but might add additional load on the nodes.
  • AIX and Power 8 Big-Endian are not currently supported for File Audit Logging. So, if your cluster consists of these nodes, then data access from these nodes is not monitored.
  • Available with Spectrum Scale Advanced and Data-management editions and not with Standard edition.
  • Needs additional data for storing audit logs within the Spectrum Scale filesystem. These logs can occupy large space depending on filesystem activity, retention time, number of users/applications etc.
  • Needs additional software for analyzing the audit logs and creating reports.
  • Events generated by non-Linux and SLES nodes are not audited currently.
  • Enabling File Audit Logging can impact Filesystem I/O performance.

Summary and Conclusion

IBM is continuously enhancing Spectrum Scale by adding more and more features and functionality that truly make it a Enterprise class clustered filesystem. Spectrum Scale File Audit Logging adds additional capability to Spectrum Scale meet the data security standards required that help companies protect and monitor their business critical data from a wide variety of security threats. Spectrum Scale File Audit Logging can be configured quickly on new Spectrum Scale 5.0.0 clusters and can replace native OS audit logging which do not has the capability of monitoring events generated from via SMB or NFS protocols. File Audit logging monitors all filesystem data access just from three nodes and obsoletes the need to configure native OS Audit logging on every node in the cluster.

ATS Group, Galileo & IBM: Empowering Optimal IT Infrastructures Together

ATS Group, Galileo & IBM: Empowering Optimal IT Infrastructures Together

For years, IBM has been a leading choice for technology solutions within the enterprise industry and beyond. This staple technology giant is responsible for some of today’s most advanced innovations, and here at Galileo, we’re incredibly fortunate to have robust ties to Big Blue.

ATS Group & Galileo: Born from IBM

The story begins with Galileo Performance Explorer’s parent company, The ATS Group, which actually has strong connections to IBM itself. Before establishing The ATS Group, and later developing Galileo Performance Explorer, founders Tim Conley and Chris Churchey both served as IBM Systems architects and engineers.

In addition, among The ATS Group’s leadership staff is Senior Account Manager and Senior Systems Engineer Bill Maloney, who is also an IBM Certified Specialist. Further strengthening ties between The ATS Group, Galileo and IBM is the recognition of the ATS Group’s Systems Engineer, Josh Kwedar, as one of 2017’s Fresh Faces of IBM AIX by IBM Systems Magazine.

A Gold Partner at Top IBM Events

The ATS Group is an IBM Gold Partner, and their team, along with Galileo Performance Explorer team members, are mainstays at leading annual IBM conferences, including their newest conglomerate, IBM Think. Our Galileo Performance Explorer team particularly enjoyed our time at this year’s conference, which covered concepts in cloud, technology infrastructure, security, artificial intelligence, blockchain and more.

Galileo consistently takes part in the IBM Systems Technical Universities (#IBMTechU) – the most recent of which was held in May and included talks on a wide variety of topics. Attendees who visited our Galileo booth enjoyed a relaxed environment where they could receive tips and best practices on maintaining, migrating and transforming their critical IBM systems.

Galileo and IBM: An Ideal Match

Speaking of Galileo-specific benefits, our ties with IBM don’t end at the tech giant’s annual universities and conferences. These events provide an ideal opportunity for us to showcase how well Galileo’s infrastructure performance monitoring capabilities can optimize IBM infrastructure solutions and initiatives.

Galileo Performance Explorer is proud to support an array of IBM products, ensuring users have the most insight into the capacity and performance of their most crucial systems. Galileo provides monitoring for IBM server, storage and cloud systems including:

  1. IBM AIX
  2. IBM i
  3. IBM Spectrum Scale
  4. IBM DS3000, DS4000 and DS5000
  5. IBM DS8000
  6. IBM FlashSystem
  7. IBM SONAS
  8. IBM Spectrum Virtualize
  9. IBM V7000 Unified
  10. IBM VIX
  11. IBM Cloud

We’re also expanding our IBM technology agents for Galileo Performance Explorer all the time – we’ll also soon support IBM Power HMC.

Galileo: A Validated Technology

We’re also pleased to be a validated technology as part of IBM’s Ready for Program with a designation as Ready for IBM Storage. As a validated IBM PartnerWorld solution, Galileo users are empowered through our intelligent and user-friendly dashboards to monitor, manage and enhance their essential IBM infrastructure systems.

“Our inclusion in the Ready for IBM Storage program further validates our vision and commitment to comprehensively support IBM clients in their infrastructure optimization initiatives from basic capacity planning needs to extensive IT transformations,” said Galileo’s Vice President of Marketing, Kelly Nuckolls.

The ATS Group and Galileo Performance Explorer understand the critical importance of IBM systems within enterprise infrastructures across every industry sector, and we’re pleased to provide solutions that seamlessly integrate and empower IBM users to glean the most value from their technology.

To find out more about the advantages of leveraging Galileo Performance Explorer alongside your company’s key IBM systems, connect with our experts today.

See What’s Happening in Hollywood at the IBM Systems Technical University

See What's Happening in Hollywood at the IBM Systems Technical University

IBM events offer a variety of ways to learn, grow, and connect. As an IBM Gold Business Partner, you can always find the teams from Galileo Performance Explorer and the ATS Group at IBM Systems Technical Universities (#IBMTechU) events, and this October is no different! Thousands of IT professionals gather to experience unparalleled technical education at an IBM Tech U events that feature technical sessions about Galileo-supported technologies, such as IBM Storage, IBM Power Systems and IBM z.

Learn how to plan, architect, install, configure, migrate, operate and tune your IBM infrastructure for today and tomorrow at the IBM Systems Technical University coming up in October in Hollywood, FL. Packed with a punch, this event offers focused, in-depth training sessions, hands-on labs and demos delivered by IBM Distinguished Engineers, developers or product experts.

IBM Systems Technical University
Featuring IBM Power Systems and IBM Storage
October 15 – 19  | Hollywood, Florida USA

Join our VIP Reception on Day 1

We’d love to tell you how the ATS Group and Galileo can support your upcoming IBM infrastructure projects. With tagging and trending across all IT assets, Galileo provides a unique, comprehensive view that streamlines problem determination and empowers successful strategic initiatives such as cloud migration, capacity planning and server consolidation. Want to learn more? Our team is hosting a fun VIP Welcome Reception at Portico to kick off the week at IBM Systems Technical University. Please RSVP to join us and feel free share with others in your organization that will be in Hollywood!

VIP Reception at IBM Systems Technical University
RSVP: Monday, October 15th from 7-10PM
Portico at The Diplomat Beach Resort
3460 South Ocean Drive | Hollywood, FL

Join the team from Galileo Performance Explorer for a Welcome Reception at Portico!

Join the team from Galileo for a Welcome Reception at Portico during the IBM Systems Tech U in Hollywood!

What’s new in Hollywood?

IBM is always looking to further the skills and knowledge of their partners and customers, and empower the IT community with the tools necessary to adapt and optimize their infrastructures for tomorrow. At the IBM Systems Technical University in Hollywood, there are a couple of new training sessions to take advantage of:

Professional and Leadership Development Training
Whether you are already a leader or aspiring to be one, you can find content to support your professional and leadership growth.

  1. Stay up to date on hot topics like IBM Design Thinking and Digital Transformation.
  2. Give your left brain a break during TechU to work on your soft skills. You can grow your career with a focus on things like presentation techniques and communication tips, and even how to make your IT project successful.
  3. Check out the current list of topics at your preferred TechU.

Cognitive Systems Track
Cognitive solutions are set to transform the world in dramatic ways. At TechU, you can leverage the experts to go deep on AI on IBM Power Systems. IBM experts will share their deep technical expertise in applied AI, model tuning, GPUs and distributed systems, workload provisioning and management. You’ll find concepts of AI, machine learning and deep learning along with industry use cases, best practices, development frameworks and industry tools. TechU offers focused, in-depth technical training sessions and labs. They will share how to :

  1. Leverage the capabilities of POWER9 for AI
  2. Deploy a fully optimized AI platform with IBM PowerAI
  3. Collaborate on AI projects with the IBM Data Science Experience
  4. Integrate and manage your AI, Spark and Hadoop big data workloads
  5. Use medical image classification with Machine Learning technology
  6. Leverage a high performance file system with IBM Spectrum Scale
  7. Optimize distributed learning models with IBM Deep Impact

Ready for IBM Storage

Galileo Performance Explorer recently announced a partnership with the Ready for IBM Storage Program to empower IBM clients to proactively monitor, manage and optimize their IT environments. Galileo is now available as a validated IBM PartnerWorld solution, enabling Galileo to support customers, VARs, MSPs and enterprise partners worldwide. The Ready for IBM Storage program integrates and validates business partner offerings with IBM technology to enable end-to-end use cases, simplify deployment and reduce risk for clients.

Galileo support an array of IBM servers, storage, systems and cloud, ensuring users have the most insight into the capacity and performance of their most crucial systems. Galileo provides monitoring for IBM server, storage and cloud systems including:

  1. IBM AIX
  2. IBM i
  3. IBM Spectrum Scale
  4. IBM DS3000, DS4000 and DS5000
  5. IBM DS8000
  6. IBM FlashSystem
  7. IBM SONAS
  8. IBM Spectrum Virtualize
  9. IBM V7000 Unified
  10. IBM VIX
  11. IBM Power
  12. IBM z
  13. IBM Cloud

Other solutions have gaps in monitoring, especially when it comes to cloud infrastructure. Galileo monitors your entire infrastructure, including on-site and cloud-based systems. To find out more about what Galileo can do for your unique IT systems, join us at IBM Systems Technical University in Hollywood or connect with us to schedule a demo or start a free trial.

Stay in the know!

Tech U features skill-building sessions that are in demand and appeal to attendees who share a technical vision and curiosity. Your event experience can be enhanced by following and discussing the topics you want to know about related to IBM Systems. Join the IBM Systems Technical Universities (TechU) LinkedIn Group to view event highlights, connect with attendees and tech experts and stay on top of the latest technology trends.