A Year with Enterprise Flash Drives – Lessons Learned

9 February 2010

http://www.futuregov.net/articles/2010/feb/09/year-enterprise-flash-drives-lessons-learned/

In February of 2008 EMC, the information infrastructure company, introduced enterprise flash drives into the mainstream enterprise disk array market. The enterprise flash drives are packaged in the same form factor as the traditional magnetic drives and where first introduced into EMC’s high end disk array, the Symmetrix DMX 4. Since the announcement EMC has introduced enterprise flash drives to its sister array the CLARiiON UltraScale Cx-4 family of products.

Photos

View photos

A Bit about Enterprise Flash

Enterprise flash drives are different from the traditional consumer flash in that the technology used inside of the drives uses chips with higher capacities and also higher performance. For enterprise flash there is also additional circuits installed to avoid wearing down a single chip (wear leveling) as well as error detection and correction if a chip should exhibit bit error. There are some other circuitry added outside of the flash chips themselves which further adds to the performance and reliability of the enterprise flash drives.

All of these technologies serve to make a single enterprise flash drive about 30 times faster than a high performance magnetic drive while exhibiting the same or better reliability than magnetic drives. This suggests that a maximum configured DMX 4 with 960 drives is the performance equivalent of a DMX 4 with 32 flash drives! To ensure that a single tier zero drive doesn’t use up all resources (unless required) the DMX 4 platform has been enhanced with priority controls as well as cache partitioning to ensure that the right resources are assigned to the right data in case of a resource conflict.

At this time the price per GB of flash drives are relatively high vs. traditional magnetic drives, however if the metric price vs. performance is used (price per IOP) then these drives are less expensive than traditional magnetic disk drives. Flash drives was originally expected to follow Moore’s law when it comes to capacity and cost, roughly doubling the capacity every three years at the same cost, however recently intense innovation and competition in this segment is suggesting that the capacity will double every two years at the same cost.

Real world experience with flash in Symmetrix DMX 4

Customers in Asia Pacific and Japan have universally embraced flash as a new type of tier in their storage infrastructure. This new tier, tier zero or the performance tier is typically used on a very small amount of the data which is critical to the customer’s business process. What we have found in real life is that working closely with the application and business users to identify this process is critical to business and deployment success. Typically this is done by first deciding on a business process that is critical, examples of critical tasks could be a specific report, query or task that users do which gives significant business benefit if it could be accelerated. Once the task is identified the storage admin works closely with the application administrator to identify the data which is used by the identified task. Typically this is found in a transactional RDBMS environment and our experience suggests that this is typically a few tables in the database which requires high performance random reads. Tools such as oracle statspack/AWR are helpful to assist in identifying which Tables and corresponding LUN’s are being used the most as well as identifying how long we need to wait for a response for the critical business tasks. Once these data objects have been identified we can migrate their data to a tier zero disks, and immediately gain the performance benefits.

Historically log disks and transaction journals have been the object which many storage subsystems have tried to accelerate. It may be tempting to attempt to try to solve these performance problems with flash disks. In real world this is not an optimum use of flash since writes performance is accelerated via array caching mechanisms. The tier zero flash attempts to accelerate read performance and since log disks rarely do reads our experience is that log drives is not the essential data object to put on flash drives. In our experience assigning cache memory to LUN’s containing log data resolves this performance problem effectively and combined with cache partitioning ensures that these parts of the database gets the right priority even if there is heavy load against the overall storage system.

In real world applications where the above method has been applied our experience shows that we typically identify a relatively low number of data base tables which are frequently read physically from disk. After we moved these tables to flash drives we’ve seen average application throughput measured as response time, as well as the number of business transactions per section improve by about a factor 3-5 times. Since each business transaction often is composed of several SQL transactions the database transaction per second rate is increased at a multiple of the business transaction rate.

There has been some discussion about wear of flash drives. The enterprise flash drives employ wear leveling technology, ensuring that a new bank of flash is written to upon each write so that the wear is evenly distributed across all physical addresses. This is done because flash drives behaves much like EPROM’s, to complete a write the target bank in the flash chip has to be erased prior to being written. Our experience shows that this wear leveling combined with the increased write cycles (number of times a single bank can be erased and written to) is on par or better than what we find in magnetic disk, and in fact our experience with these drives in the real world has been that these types of drives are as or more reliable than traditional magnetic drives.

Conclusion

Enterprise flash is an exciting technology which can significantly improve the performance of business transactions. Due to its relatively high initial cost the technology should be deployed strategically in the areas where it can be leveraged the most, and working closely with application administrators and developers is key to identifying hotspots which flash can resolve. Flash by itself can consume the entire available performance capability of a very large enterprise array so technologies for prioritisation and partitioning of resources when there are resource conflicts are essential components if a good architecture.

We have seen a tremendous uptake of flash drives in Asia Pacific and Japan and we believe this is a result of customers in these markets requiring the benefits of flash technology due to their rapid business expansion which drives their IT requirements. Although we see a slowdown in IT spending we still believe that customers will continue to make strategically investments into tier zero flash drive technologies for the areas where they require the business benefit.