Here's Who Made Gartner's 2016 Magic Quadrant For Distributed File Systems And Object Storage

Storage In The Age Of Big Data

The rapid and unabated growth of so-called "unstructured" data has led research firm Gartner to establish a new Magic Quadrant for distributed file systems and object storage.

Such systems seek to provide cheap, scale-out solutions to customers that are tasked with dealing with vast and growing pools of data like Facebook photos and songs provided by music streaming services.

Gartner estimates that data growth is exceeding 40 percent annually in many enterprises, pushing business leaders to demand storage products that be applied to a growing number of use cases at lower prices. Enterprises are also demanding features and capabilities common in big data cloud infrastructures.

Software-defined storage deployed on commodity hardware is emerging as a threat to traditional external controller-based storage arrays in environments with a steep growth of unstructured data, Gartner says. Click through to see which vendors Gartner says have been most successful in developing distributed file systems and object storage products that solve the cost and scalability challenges that can't be handled by traditional storage environments.

Leader: Dell EMC

Dell EMC, one of three vendors in the "leaders" quadrant, has two products in this area: Isilon and Elastic Cloud Storage (ECS). Isilon is a mature, scale-out NAS platform with deployment across many enterprises and service provider verticals and geographies. ECS is an object storage platform for big data and distributed file systems and can store billions of objects for cloud-native applications. The Isilon OneFS operating system is in its eighth generation of software release and has more than 7,000 customer deployments for use cases such as analytics, file sharing, backup and archiving. ECS became available in June 2015 and now has 250-plus customers, many with over 1PB in their deployments. ECS has compatibility with HDFS, OpenStack Swift and Amazon S3, as well as APIs for Centera and Atmos.

Dell EMC

Strengths: Isilon's enterprise adoption from traditional file system support to additional use cases is growing, including video surveillance, big data analytics, backup and active archive. Elastic Cloud Storage has a track record of large production deployments across a variety of enterprises and is becoming an attractive price/performance platform for a variety of end users, Gartner said. The research firm also noted that Dell EMC's broad portfolio, aggressive pricing and new software-only, leasing and subscription-based purchasing models are particularly attractive.

Cautions: Gartner said customers might feel the impact of sales and support changes as Dell EMC adjusts to life post-acquisition. Also, the current version of Isilon OneFS is not flash-optimized, and is not suited for small files and metadata-intensive workloads with low-latency requirements, Gartner said. Also, ECS can be expensive, and some customers may seek out lower-cost object storage alternatives.

Leader: IBM

IBM's acquisition of Cleversafe in the fourth quarter of 2015 gave IBM, another "leader" – a competitive, mature, proven and scalable product in the fast-growing object storage market. Gartner looked at two IBM products: IBM Spectrum Scale and IBM Cloud Object Storage. IBM Spectrum Scale is based on General Parallel File System (GPFS), a distributed file system built by IBM researchers in the late 1990s. IBM Spectrum Scale is highly scalable and delivers competitive concurrent throughput performance due to its decentralized architecture and efficient data placement across the nodes. GPFS has been sold as an integrated hardware platform, the recent focus on software-defined deployment models — both on-premises and in the IBM Cloud — has the potential to increase the attractiveness of the product to prospective buyers, Gartner said.

IBM

Strengths: IBM Spectrum Scale and IBM Cloud Object Storage are mature products with flexible deployment options and hundreds of PB-scale customers. IBM Cloud Object Storage delivers a differentiated way to secure data at rest. It's all or nothing (AONT) encryption is integrated with its information dispersal algorithm to lower the potential for data breaches, and it obviates the need for external key management. IBM Spectrum Scale delivers competitive throughput performance for both small and large files; provides broad support for interfaces, including NFS, SMB, REST (S3, Swift) and HDFS; and can tier data to Amazon S3 and the IBM Cloud Object Storage.

Cautions: IBM Cloud Object Storage product lacks native support for file access and needs third-party tools to enable it, Gartner said. Gartner clients have also reported difficulty understanding IBM licensing, which presents customers with a mix of licensing types based on the role performed by each node. While the acquisition of Cleversafe by IBM's SoftLayer division is a key strength for the business, it could also steer future product development toward SoftLayer's needs rather than those of enterprise IT customers, Gartner said.

Leader: Scality

Venture-backed Scality, a "leader" in the space, according to Gartner, released its Ring scale-out, peer-to-peer distributed software that can be deployed on any x86 commodity hardware in 2010, signing HPE and Dell as resellers. By the end of last year, HPE had also made an investment in Scality. Ring has more than 100 large customers, including service providers, public sector, life sciences, financial services, research institutions and big businesses. IT leaders have implemented Ring for email hosting, content distribution, active archive and private storage cloud.

Scality

Strengths: Scality has strong channel partnerships and OEM relationships and provides a native, unified file and object platform for enterprise customers, Gartner said, noting that Ring also has a reputation for good performance with small files, which gives it the opportunity to replace traditional NAS products for unstructured data.

Cautions: High turnover means the Scality sales and support experience in North America has been mixed, Gartner said. Scality Ring is not designed as a replacement for general-purpose NAS solutions, as it is not optimized for heavy metadata and random access workloads, and does not have all the features of enterprise NAS, Gartner said. Scality is also the largest independent distributed file system and object storage vendor that does not sell its solution as a pre-integrated hardware appliance, making it an acquisition target for larger vendors seeking to add products and expertise focused on an emerging market.

Visionary: Red Hat

Red Hat, deemed a "visionary" by Gartner, has two open-source solutions it gained through the acquisitions of Inktank in 2014 and Gluster in 2011. Red Hat Ceph Storage delivers unified block, file and object access in a single system with an ability to scale out to PBs of data on commodity hardware. Ceph's architecture is underpinned by RADOS, a distributed object storage service, which uses a unique algorithm to handle data placement that balances availability and performance. Red Hat Gluster Storage is a scale-out, multiprotocol (NFS, RESTful APIs, SMB), open-source storage software solution, with PB-scale capacity and competent data protection, Gartner said.

Red Hat

Strengths: Red Hat has a proven commercial model for open source software with a differentiated storage portfolio that is focused on bimodal requirements of enterprise IT, Gartner said, adding that the Ceph project has an active community of coders and a growing ecosystem of OEM partners including Cisco, Dell EMC, Fujitsu, SanDisk and Supermicro. In particular, Red Hat Gluster Storage offers customers flexible deployment options — bare-metal install, virtual storage appliance on KVM, VMware hypervisors, AWS and Microsoft Azure public clouds, and as a Docker image. It can also be deployed as a hyper converged appliance through third-party server OEMs.

Cautions: Ceph deployments require a major effort from enterprise users, calling for careful instrumentation to optimize and fine-tune cluster performance and reliability for chosen hardware, Gartner said. The Ceph Filesystem (Ceph FS) isn't mature enough to be deployed for production use cases in the enterprise due to limited data protection features. Customers can be confused by product overlap between Gluster and Ceph.

Visionary: SwiftStack

Another Magic Quadrant "visionary," SwiftStack, is the main contributor to Swift, an open-source object storage project in the OpenStack suite of cloud infrastructure software. SwiftStack develops a product that allows enterprises to better deploy and manage a SwiftStack cluster, which first started shipping in 2013. Although the foundation of SwiftStack is based on OpenStack Swift, the vendor has also adopted the Amazon S3 API because of the sheer size of the S3 ecosystem of integrated tools and the market opportunity, compared to Swift, Gartner said. SwiftStack supports backup and archiving to its platform using Commvault and Veritas' NetBackup.

SwiftStack

Strengths: SwiftStack's main strength is its work to reduce the costs of implementing and operating an OpenStack Swift cluster, Gartner said. Customers consistently rate SwiftStack highly for reliability, their overall relationship with the vendor and satisfaction with sales and deployment services. Customers are beginning to consider SwiftStack alongside larger, more established vendors, Gartner said.

Cautions: Swift requires a lot of tuning to optimize performance for particular hardware and workloads, limiting its appeal to enterprise buyers, Gartner said. SwiftStack has focused much of its efforts on the Amazon S3 API due to the sheer size of the S3 ecosystem and the market opportunity, leaving its commitment to OpenStack Swift API origins in question. The vendor has wide support for server and storage hardware that may result in support issues for configurations that are not commonly deployed or tested, Gartner said.

Challenger: Hitachi

Gartner put Hitachi all alone in the "Challengers" quadrant. The Hitachi Content Platform (HCP) is an object storage product with enterprise file synchronization and sharing (EFSS) and a cloud storage gateway. HCP was first made available in 2006, is a mature product with deployments across a range of industries, including financial services, health care and public-sector enterprises. HCP's maturity and legacy as a compliance-focused archiving platform are supported by a broad range of ISV solutions, including content archiving, medical imaging, cloud storage gateways and enterprise content management (ECM) vendors, Gartner said. Customers can build cloud-native applications using Amazon S3-compatibility supported by HCP. HCP Anywhere supports file synchronization clients on Windows, Mac, iOS, Android and Windows Phone.

Hitachi

Strengths: HCP is a stable product supported by consistently positive marks by customers in terms of reliability, Gartner said. The product has a large number of customers across several industries and supports mobile access and remote offices, making the platform a one-stop-shop.

Cautions: HCP deployments are typically smaller than those taken on by the quadrant's leaders, Gartner said. Also, Hitachi "does not serve the distributed file system segment of the market well and is significantly behind the leaders in terms of capabilities in this area," according to Gartner. HCP doesn't have mind share among developers building modern, Mode 2 applications.

Niche Player: Caringo

Caringo, a "niche player" in the quadrant, has been successful with customers that require a scalable and secure solution for storing data for regulatory compliance purposes with built-in search capabilities, Gartner said.

Caringo

Strengths: Caringo Swarm is a mature object storage system that offers flexible, all-inclusive pricing on either a license or subscription basis.

Challenges: Caringo is yet to be profitable. It is also largely self-funded, limiting its ability to aggressively invest in the product, Gartner said. Also, Caringo has a weak partnership with server OEMs, which limits its enterprise appeal due to a lack of wide reference architectures and integrated support from the hardware vendors.

Niche Player: Cloudian

Gartner put Cloudian in the "niche player" quadrant. The company is tight with Amazon, and develops HyperStore, an Amazon S3-compatible object storage platform that has been available since March 2011. HyperStore is delivered as a turnkey integrated appliance or as a software-only product that can run on industry-standard hardware. HyperStore offers on-premises storage and data management, as well as replication, erasure coding and multitenancy services, and tiering to the Amazon Web Services (AWS) cloud.

Cloudian

Strengths: HyperStore maintains closer compatibility with the Amazon S3 API than much of its competition, Gartner said. Cloudian's multitenancy and quality of service features resonate well with large enterprises and CSPs, and HyperStore appliances provide robust, modular, high-density and resilient hardware options for a wide range of end-user capacities, according to Gartner.

Cautions: Gartner clients outside of Japan and the U.S. are rarely evaluating HyperStore due to Cloudian's limited market traction, Gartner said. Cloudian's marketing efforts sometimes outstrip the company's ability to deliver and the company's vision includes hybrid cloud storage that tiers to Amazon S3, but market adoption of such features is "negligibly low," Gartner said.

Niche Player: DDN

Gartner includes DDN in the "niche players" quadrant. The vendor offers Web Object Scaler (WOS), an object storage product that supports a native web services interface in addition to providing Amazon S3 and Swift compatibility. DDN has sold WOS as a turnkey hardware appliance since 2008 and has more recently shifted to allowing WOS to be deployed as software with broad hardware support, Gartner said. DDN offers reference architectures of WOS with Dell EMC, Hewlett Packard Enterprise, Supermicro and the Open Compute Project (OCP).

DDN

Strengths: DDN has a heritage in the HPC market and has the resources in place to serve these customers well with WOS, Gartner said. Enterprises can select from a variety of WOS hardware and software deployment options that suits infrastructure, cost and management preferences. DDN offers seamless integration between WOS and its parallel file system appliances, GRIDScaler and EXAScaler, for handling warm and cold datasets between these environments transparently.

Cautions: WOS prioritizes performance over security features, and lacks server-side encryption and HTTP authentication, Gartner notes. Also, WOS's S3 API compatibility is implemented on an architecture that does not scale well beyond a single site. Some customers have rated DDN poorly on delivering on sales promises in addition to solving challenges with respect to deploying and operating complex WOS-based infrastructures, Gartner said.

Niche Player: Huawei

China-based Huawei is another of Gartner's niche players. The company offers the OceanStor 9000 product, which has been available since 2013.Delivered as a hardware appliance, the OceanStor 9000 is particularly well-suited for the media and entertainment industries and video surveillance, Gartner said. There are now more than 90 customers with greater than 1PB deployments. OceanStor 9000 is a scale-out distributed file system product providing shared storage for unstructured data. OceanStor 9000 provides file services via NFS and CIFS shares, and object storage through Amazon S3 and OpenStack Swift interfaces.

Huawei

Strengths: The OceanStor 9000's highlights are high scalability and resilience, supporting up to 100PB of storage on up to 288 nodes based on a symmetrically distributed architecture, Gartner said. The system is particularly suitable for enterprises in the Asia/Pacific region, as Huawei has expanded its presence and witnessed a triple-digit increase in its year-over-year revenue, Gartner noted.

Cautions: Huawei's long-term storage product roadmap remains uncertain for enterprise IT buyers, Gartner said, pointing to the company's decision to discontinue its OceanStor UDS, its dedicated object storage product line that had struggled to gain traction. Also, Huawei's service, support and reseller network for storage products continues to be weak in the North American market, where it still struggles to grow its market share due to geopolitical and brand perception issues, Gartner said.

Niche Player: NetApp

NetApp remains one of the leaders in ECB block and file storage, and was placed in Gartner's "niche players" quadrant. NetApp's StorageGRID Webscale is an object storage product providing API-accessible, distributed storage with the ability to scale across geographies. StorageGRID Webscale supports the major object protocols (and offers NAS access) on dedicated hardware or as VMs in either VMware or OpenStack environments. The origin of NetApp StorageGRID Webscale comes from the acquisition of Bycast in 2010, with a strong footprint in healthcare records storage and management, Gartner said.

NetApp

Strengths: NetApp has a mature service and support infrastructure, and a presence in 100 countries worldwide, Gartner said, adding that the new StorageGRID Webscale product has been redesigned to improve metadata handling, such as scalability and resilience. It also has identity and access management features that work seamlessly between public cloud and on-premises storage environments.

Cautions: Substantial parts of StorageGRID Webscale have been rewritten and are currently lacking large-scale reference customers, Gartner said. Also, the product doesn't integrate with NetApp's Ontap product line and lacks strategic focus.

Niche Player: Panasas

Another "niche player," Gartner noted that Panasas has primarily focused on the high-performance computing storage market. Its flagship product is the ActiveStor appliance, which is based on its PanFS distributed file system. The vendor's focus has been on building high-performance storage products with high scalability, availability and ease of use.

Panasas

Strengths: Panasas is an established leader in the high-performance storage space, providing integrated hardware and software support with deep workflow integration with applications in industries such as life sciences, manufacturing, energy and the public sector, Gartner said. Panasas' object-based file system architecture and distributed approach to erasure codes, means it can protect data at a file level, resulting in lower-capacity overheads and faster rebuilds, Gartner said.

Cautions: Panasas' historical focus on the lower-margin HPC market can impact its ability to scale investment in the highly competitive commercial scale-out NAS market, Gartner said, adding that Panasas' data services are limited and its integration with back-up and recovery and archiving vendors is weak.

Niche Player: SUSE

Germany-based SUSE was also called a "niche player" by Gartner. The company is a contributor to Ceph, an open source storage software project, on which SUSE Enterprise Storage (SES) is based. It was first made available in 2015. SUSE is focused on adding installation and management capabilities to its Ceph contributions.

SUSE

Strengths: SES is aggressively priced, based on nodes rather than capacity, and, as a result, customers can deploy cost-efficient object storage using commodity servers with densely configured disk storage, Gartner said. SUSE is also focused on making OpenStack more enterprise-consumable through efforts such as improving backup and recovery with Ceph. Gartner noted that SES may have appeal for existing customers of SUSE Linux Enterprise (SLES), a frequently deployed operating system, particularly for Mode 1 applications.

Cautions: The majority of SES deployments are in Europe, and the product has virtually no adoption in North America, reflecting SUSE's recent entry in the market for enterprise storage and a lack of credibility in the space, Gartner said. The analyst firm added that SUSE has overly ambitious plans with a forthcoming release that includes support for Ceph FS, which is largely considered not to be enterprise-ready by other upstream contributors to Ceph.