How to create data management policies for unstructured data

Join online with today’s leading executives at the Data Summit on March 9th. Register here.


This article was contributed by Randy Hopkins, VP of Global Systems Engineering and Competencies at Compress.

If data is the lifeblood of your organization – customer relationship success, product launches, cross-selling and upselling and employee productivity – then you need to manage it strategically. That strategy should include a program for data management policies. This ensures that data is always stored in the right environment according to its use, age, value and business priority. And those parameters are always changing.

For example, the electric car manufacturer wants to understand how its vehicles operate in different weather conditions. Therefore, they want to create a data management policy to pull trace files from the car into the data lake and analyze it at regular intervals. Once the study is completed, that policy will expire and the moved data will be deleted or moved to Deep Archive storage. The hospital may have a policy of retaining medical images for the patient’s life and may determine where and when those images go into cold storage. Some organizations need to delete former employees’ files immediately or after a period of time. Given the scope of data stored in enterprises today, doing this manually is no longer a viable option.

Data is growing at an unprecedented rate, accounting for 30% or more of the total IT budget on its storage, now is the time to consider the idea of ​​data management policy automation. The lion’s share of all data is unstructured – all kinds of files including images, log files, trace files, output files, video and audio and which is spreading like wildfire. Data management policies must address the efficient movement and management of unstructured data.

The benefits of adopting a systematic approach to creating, managing, and managing data policies include:

  • Automated policies align data strategies with business goals;
  • Simplifies data management by minimizing manual effort and ad hoc decisions;
  • Cold data tearing delivers the means of maximizing cost savings by constantly moving to less expensive storage;
  • Ensures compliance with industry regulations;
  • Adds ransomware protection by copying data from primary storage to object lock storage where it cannot be compromised;
  • Automatically feeds data pipelines into data lakes and tools for analysis and AI programs.

The notion of data management policy is not new, but historically, this activity took place in storage vendor technology. The storage vendor-centric approach was very good and efficient before data reached petabytes and today’s increasing levels and before organizations used multiple storage vendors and the cloud to manage their data. But now, the storage-centric approach to policy management creates vendor lock-ins and silos, making it difficult to move data quickly and across different storage technologies and services as needed to manage data-efficiently and support users, big data analysis Initiatives and costs. – Savings command. IT leaders know that in the digital age, data needs more than security. It requires complete lifecycle management and is where modern data management policies are implemented.

Consideration for the management of an unstructured data management policy

Access anywhere. Distributed workforce now needs instant access to data – regardless of where it is stored – with a transparent user experience. Data professionals should prioritize these requirements as they formulate policies to save money, secure data, and enable access controls.

Automate as much as you can, The declarative approach is the goal. While many options are now available using independent data management software to manage policies across storage, many organizations still employ IT managers and spreadsheets to create and track policies. The worst part of this bespoke manual effort is to search for files with certain features and then move or delete them. These efforts are ineffective, incomplete and hinder the pursuit of policies; It’s painful to maintain them, and IT professionals have a lot of competitive preferences. In addition, this approach limits the potential for strategic AI and ML projects to use policies to continuously curate and move data in Data Lex. Instead, find a solution with an intuitive interface to build and execute on a schedule that runs in the background without human intervention.

Measure and refine the results. Any data management policy should be mapped to specific goals, such as cost savings on storage and backup. He should measure those results and let you know their status so that if those goals are not being met, you can change plans accordingly. This is similar to a smoke detector that always checks its own battery and then warns you when it comes time to replace it. Data management tools do a heavy lifting for you and should let you know when something is not working or if there is a problem to fix it.

For example, if you have a data management plan that tires data into object storage in the cloud after reaching the age of one, you would expect a certain percentage of savings. However, if this cold data is frequently withdrawn to local applications and storage, you will face a higher exit fee which will resist those savings. At that point, you may want to consider a different tiring model, or better yet, a data management solution that identifies the latest access activity and trends for cold data and applies it to the right place.

Align employee roles. Data management policies should be managed by a team within the organization that identifies how the policy is created, accessed and used. The team is also responsible for managing, implementing and refining policies, and communicating with employees who need to know employees. Large enterprises should consider creating a data management policy team consisting of top executives who contribute to discussions on data governance, protection and monetization. The team will align with business units to ensure that maintenance and safety considerations are consistent.

Metadata management for unstructured data: Another consideration is to simplify the search in all file metadata from the unified file index. Technology should also enable copy, move, archive, tire and reporting actions on unstructured data files.

In conclusion, Enterprise Data does not belong to any individual or business unit; It is owned by the enterprise and needs to be managed holistically and strategically to meet the needs of critical stakeholders and align with broad organizational goals. Users don’t have to worry about where the data resides. Data should be accessible to users, no matter where they live. Ultimately, a data management policy should guide your organization’s philosophy to manage both structured and unstructured data as a valuable enterprise asset.

Randy Hopkins is VP of Global Systems Engineering and Competencies Content.

DataDecisionMakers

Welcome to the VentureBeat community!

DataDecisionMakers is where experts, including tech people working on data, can share data-related insights and innovations.

If you would like to read about the latest ideas and latest information, best practices and the future of data and data tech, join us at DataDecisionMakers.

You might even consider contributing to your own article!

Read more from DataDecisionMakers

Similar Posts

Leave a Reply

Your email address will not be published.