Guruprasad S – Director, Information Management & Governance Products
I am sure all of us at some point in time would have felt a need to have an infinite storage space so that there is no need to worry about adding new disks or cleaning up old/unwanted files to create more space. The fact is, the data sprawl we are seeing now will only increase year over year and so we need to find ways to manage the storage space. Managing storage space has its own set of challenges; the other important aspect to be considered is the cost associated with the addition of storage be it in DAS, SAN, NAS, or any other media.
If we look closely at the data, one can make out that most of the data over a period becomes less relevant, accessed rarely and at times not required to be even maintained. In fact not all the data that is generated is actually used, a second time.
In the data management parlance, data that is accessed frequently is called hot, rarely accessed is called warm and not accessed is called cold data. This visualization of the temperature (hot, warm or cold) of data, so to speak, is important. There are many studies that give a good indication of the amount of cold and hot data in any environment. Though there might be some variations depending on the organization and the kind of data it holds, this also follows the Pareto principle of 80-20, with 80% of data being cold and 20% being hot or warm. So, on one hand, there is a huge amount of cold data in the environment, while on the other hand, due to continuous data sprawl we have a never-ending need to increase the storage capacity.
There has been a lot of studies around where to store the data, especially cold data, on premises or in the cloud, & which one of the two is cost effective. Apart from the cost, we need to understand the pros and cons of both the methods, some of the factors to be considered are, the control of data, availability, security, maintenance, and so on. Looking at the cost alone, a general observation is, there could be anywhere between 40% and 50% of cost savings by moving cold data to the cloud. There are no one-size fits all and this can vary from organization to organization, but if there is a cost benefit, definitely this is something to be looked at and planned.
Now that we have come this far, we get a couple of questions:
- Can I get a visual representation of the cold and hot data in my environment?
- Can I free up space and reduce the storage cost by putting cold data in less costly media?
- Can I easily move my cold data without impacting anything?
- Can I easily get the cold data back when needed?
- What are my cost savings if I move the cold data out to the cloud?
- and may be some more questions
Yes, all the above and more is possible with Cloud Integrated Storage (CIS) which is available as technical preview and is part of Micro Focus® Open Enterprise Server (OES) 2018. CIS is a hybrid solution that provides a secure gateway to store, manage, and access data across private or public cloud. Below is a pictorial representation of the full environment. Visit the CIS Documentation page for more information.
CIS gives a clear picture of the temperature of the data in a given environment. A further granular drill-down is also possible as depicted in the following two pictures:
CIS provides an easy way to ensure hot data (frequently used) is stored on fast and high-quality storage, while the cold data (less accessed) is placed on a cloud storage with a relatively slower access. CIS allows data to be moved from traditional on-premises storage to any cloud/object store and vice-versa.
CIS provides various policies that govern the movement of data between the different media based on one or more parameters like last accessed time, size of file, type, etc. This ensures expensive storage is reserved for hot data, thus bringing down the overall cost of data maintenance as most of the cold data can be moved to low-cost cloud solutions.
There are pros and cons to both on-premises and cloud-based solution, but it’s very clear that there is a definite cost benefit in moving cold data to the cloud environment. CIS as part of OES 2018 is readily available to show the amount of cold data in the environment and also help in easily moving the data to a different store based on the needs of the organization. If you have not tried this out, do try it out or if you have already tried and have some questions, reach out to the OES Product Management team. If you have any ideas around this feature you can always put it in the OES Ideas Portal.
More information (write-ups and videos) showcasing the use and benefits of Cloud Integrated Storage (CIS) will be available soon.