Mid-sized company in the telecommunications industry. Over a thousand employees, partners and customers distributed globally with the majority in North America.
Problem #1: Backup Window too long and cumbersome to manage
- System backups were taking too long, creating a high probability for error, interference with production workloads and ultimately impacting the successive weekly backup.
- Storing backups on tape was still a viable option due to its attractive cost, but presented problems during recovery of the data because it was cumbersome and slow.
Problem #2: Application “time-outs” due to latency and disk contention from filer
- Many application workloads required faster performance which meant higher IOPS from the storage system; this created disk contention for performance, particularly during overlapping workloads competing for the higher performing disks.
- Simply adding disk shelves to their existing storage system to improve performance had become very inefficient; it created too much stranded capacity while it added additional areas of contention at the switching layers.
- Higher productivity; 15x improvement in IOPS without any added disk.
- Risk of operational error and data loss minimized; backup window reduced from 8 days to 2 hours.
- IT operations expense reduced: Cost of nightly offsite tape operations reduced 80%.
Solution #1: Implement a Disk to Disk (D2D) Backup strategy
NuSpective Solutions, along with our vendor partner NetApp, collaborated with the customer’s IT staff and decided to implement a disk to disk backup solution using NetApp’s SnapVault technology. This solved several problems and added several benefits in parallel.
- Problems Solved:
- backup windows were taken down from a week to hours
- reliance on tape was now relegated to archival purposes only and the frequency of this chore reduced to once per month; with the potential to move to once per quarter in the future
- Benefits Added:
- SnapVault technology allowed the customer to take the backups offsite to a secondary datacenter while very using very little incremental bandwidth across their WAN
- Efficiently transporting copies of the backups to the off-site location also provided an added layer of data resiliency against disaster
- Taking point-in-time copies of the data at the secondary site also allows the customer to move their remaining archive workload off of their production datacenter
NuSpective and its professional services team worked closely with the customer to appropriately size their secondary site target array with enough High Density Drives (HDD) to handle the increased data storage requirements. The team also helped execute the data migration required to initialize the first sync of the data between the sites. The professional service also included training and education seminars to ensure post project success.
Solutions #2: Design a Performance Tier with Solid State technology using Flash
Once again teaming with NuSpective and NetApp, the customer was educated on the benefits of “Tiering” their existing production storage system using Flash and Solid State Disk (SSD) technology. The respective engineering teams conducted a deep-dive performance analysis of their current workloads on their existing storage system. This process ensured the recommended “Data Tiering” design was optimized for their specific application usage profile. The final design, utilizing Flash technology, solved their performance problem and provided several additional benefits:
- Problem Solved:
- The unacceptable latencies that were created by high disk contention were eliminated. Reducing the latencies that were averaging in the mid 60’s to low single digits, as measured in milliseconds. Application & VM timeouts were eliminated as a result as well.
- Benefits Added:
- With the Flash technology absorbing much of the “read” IOPS demand from the applications, much of the “stranded” SAS disk now provides significant headroom to handle forecasted data growth.
- Because the Flash technology exists in the controller, the customer doesn’t have to micro-manage the allocation of the IOPS resources; the controller has native system intelligence to automatically allocate the Flash technology to the application with the most urgent demands.
The team worked diligently to implement the new Flash technology seamlessly; as well as re-organize workloads and back-end processes for optimal utilization of the array and Flash technology. The implementation utilized vendor specified best-practices to help avoid future issues.