Creative tips and tricks

Permanent Storage Memory for Long-Term Use in the age of containers (r)

Apr 21, 2023
Illustration showing persistent storage devices like hard drives and cards.

Share on

Persistent storage refers the storage of data using a non-volatile technique in order to guarantee that data can be accessed even after a device or application powers off or restarts. The storage and retrieval capabilities of information allows apps on the web to save user information and state information and operate with a high degree of reliability.

For monolithic systems storage access is simple as the server and storage are connected. Systems that are dispersed geographically complicate access because storage must be available to all components around the world.

The issue of containerization is further complicated since containers are light as well as stateless and impermanent -- characteristics that do not make them suitable to store the documents. So, any storage system must be compatible with containers. It adds an additional layer of complexity.

This article explores persistent storage by presenting a review about the different storage types as well as their structure and use scenarios. Furthermore, it gives examples that illustrate the distinctions between persistent and volume storage. Storage volume using Docker.

Storage types which are in use

There are a variety kinds of permanent storage. They include conventional spinning disks (hard disk drives, also known as HDDs) Solid-state drive (SSDs) and network-attached storage (NAS), and storage area networks (SANs).

  • HDDs are data storage electromechanical devices for storing and retrieving digital data using spinning disks made of magnetic media. The disks are equipped with magnetic heads that are placed on an arm that can are able to read and write data.
  • SSDs often referred to as solid-state devices, semiconductors or solid-state disks. They use integrated circuit assemblies that store the data for long durations of time. They usually use interconnected flash devices with no moving components. The fact that they're stationary makes them significantly more reliable and long-lasting as compared to HDDs.
  • Network-attached Storage is a collection of HDDs as well as SSDs, or both connected via an internal network that uses some of the existing file system like New Technology File System (NTFS) or the fourth extended filesystem (EXT4).
  • SANs are networked, high-speed block-level storage devices like disk arrays or tape libraries. The connections they make appear to the operating system's storage as local. It isn't accessible via the local area network (LAN).

Persistent Storage Architecture

There are three options to ensure persistent storage each one with its own specific uses and limitations.

Object Persistent Architecture

The approach of object-permanent architecture makes use of object-relational mapping (ORM) to build objects using data from an existing key-value or relationship database. This approach is useful for situations where data isn't governed by a schema, because the ORM will be responsible for the retention and retrieval.

Block Persistent Architecture

Block persistent architecture utilizes block-level storage devices. They are suitable to store large documents. This technique is helpful for those who need to keep huge amounts of data as you can use multiple blocks to boost storage capacity.

Persistent Filestore Architecture

A persistent Filestore is useful for applications that require frequent retrieval of data and require an interface for managing the filesystem.

Persistent Storage Cases for Use in the Storage

This section will explain some of the uses of each type of storage.

 OPS is Object Persistent Storage

  • Analytics using big data Obscure storage can be used in big data analytics for the storage and management of huge databases that are commonly used for data analysis as well as machine learning and AI. It allows access to data quickly and efficiently, making it an integral component of architectures for big data.

Block Persistent Storage

  • HDC (HPC): HPC environments for the rapid and efficient processing of large amounts of data. Block persistent storage allows HPC clusters to store and access massive data sets like mathematical simulations, weather models along with financial analysis. Block storage is usually the preferred option to HPC because it provides the best performance with the lowest latency data access, as well as allowing simultaneous input/output (I/O) processes that could dramatically speed up processing time.
  • Editing videos: Video editing applications demand high-performance and fast access to huge video file formats. They also need to accommodate large numbers of I/O operations per second, as well as have a minimal latency for editing and rendering videos in real-time. Block storage can provide these features, making it the perfect solution for workflows for editing videos.
  • Gaming Games also require high-performance and low latency for accessing game assets as well as player data. Block storage efficiently stores and retrieves huge volumes of data. This ensures that gaming environment are loaded quickly and remain fluid during play.

 permanent filestore storage

  • Media and entertainment: Animation, editing and rendering software generally make use of persistent storage. They require low-latency and high-performance access to large media files including audio, video and photos. Filestore is a file sharing system which can be accessed by several applications. This makes the perfect storage system for these applications.

Storage in containers that can be used for a long time

Containers are lightweight, portable as well as secure and easy, offering an interface between various applications. Containers must be equipped with a method to store data in between reboots as well as their elimination. Containers can store files and a file system similar to typical applications. When you construct containers with new features that are not persistent, they will lose all data.

Containers can be used for creating volume storage or mounting an entire storage volume. The storage volumes contained in containers are regarded as an archive. Anything that's written to the volume gets in the host's system for file storage.

Containers with persistent storage have to function as such since restarting the container will create an entirely new instance and discards the existing instance. If the container does not have the same view of data, it is lost whenever the container is restarted. A storage volume preserves data across sessions and restarts, which allows the container to remain in its status even after it's moved or restarted.

Volume vs Persistent Volume

Containers offer two methods to store persistent data by using volumes, as well as persistent volumes. There's a distinction between them. Containers are able to store large amounts of data. When you close a container, the data remains available and is re-accessible after restarting. But, when you remove or remove a container, the data disappears because you've removed the original volume storage.

Bind mounts can be described as a method of keeping data out of the system's filesystem. This data isn't lost in the event that you delete the container. Data is stored until the entire container is deleted manually.

This section will show the two types of volume using examples.

A Storage Container Persistent Demo

It's a simple application which has two fields to permit input from the user:

  • Title
  • Text of the Document
Screenshot: The demo application's feedback form graphical interface.
Demo Application's GUI with fields for Text and Text of the Document fields.

Once you have saved the input of the user After saving the input, it is accessible by accessing the file in the feedback directory, with the name in the field titled field. The input from the text of the document field is the actual file's contents.

How to Utilize Volume Storage

When you've downloaded the app on your personal system, it'll have the ability to access the option of volume storage according to the Dockerfile.

Screenshot: Contents of the Docker file, including a VOLUME attribute.
Dockerfile which demonstrates the use of storage volume.

Then, you make the image, and then start the program. To do so, execute these instructions.

docker build -t feedback-node:volumes . docker run -d -p 3000:80 --name feedback-app feedback-node:volumes
Screenshot: Terminal window showing results of the docker build command with volume storage.
Making the application using a utilization of volumes of storage.
Screenshot: Terminal window after executing the docker run command with volume storage.
Its running shows it's managing the volume of storage.

After the application has been activated, visit localhost.3000 to leave feedback.

Screenshot: Submitting feedback via the demo application's graphical interface.
Giving feedback to the app.

Click Save and navigate to localhost:3000/feedback/test.txt to see if the input is stored successfully or not.

Screenshot: A browser with the submitted test.txt file open.
Feedback received successfully confirmed.

The container should be started and removed from it to see if the entry is still there.

docker stop feedback-app docker start feedback-app

Returning to the same website You will see that you still see the same feedback. What happens when you take the container away and try to start it again?

docker stop feedback-app docker rm feedback-app docker run -d -p 3000:80 --name feedback-app feedback-node:volumes
Screenshot: Browser reporting failure to open test.txt file.
Feedback information has disappeared.

To avoid this and ensure that the data is retained after you have removed the container, you can use persistent volume storage also known as name storage. The first step is to remove the containers as well as images.

docker stop feedback-app docker rm feedback-app docker rmi feedback-node:volumes

How to Utilize Persistent Volume Storage

Before you try this, ensure that you get rid of the VOLUME property in the Dockerfile and rebuild the image.

Screenshot: Dockerfile edited to remove VOLUME attribute.
The Dockerfile was modified to get rid of this attribute. VOLUME attribute.
docker build -t feedback-node:volumes . docker run -d -p 3000:80 --name feedback-app -v feedback:/app/feedback feedback-node:volumes

This is apparent with the second command, you may utilize"the "-v" flag to define the volume that remains outside of the container, which remains regardless of the removal of the container.

As with the earlier step, try adding feedback. You can connect to it after having been able to stop, removed it, and restart the container.

Screenshot: Entering text in the demo application's feedback form.
Incorporating new feedback to allow for testing of persistence.
docker stop feedback-app docker rm feedback-app docker run -d -p 3000:80 --name feedback-app -v feedback:/app/feedback feedback-node:volumes

It is evident that, even after removing and stopping the container, the information remains accessible, and available.

Screenshot: Browser that has successfully opened the second test file.
When you have removed and shut off the container, information is not completely.

Summary

Persistent storage is vital in containers since it permits the storage of data that isn't connected to the lifespan of the container. The most common types of storage for containerized applications include binding mounts and volume. Both have their advantages and uses.

Volumes are kept in the containers' file systems. while bind mounts are directly accessed on the host machine.

Persistent storage lets data be shared across containers, making it possible to develop complex, multi-tiered apps. Persistent storage is vital in ensuring the stability and longevity of containerized applications offering a secure and flexible way to save crucial data.

  • Simple management and setup Simple setup and management My Dashboard. My dashboard
  • Support is available 24 hours a day.
  • The most effective Google Cloud Platform hardware and network is powered by Kubernetes for maximum scalability
  • A high-end Cloudflare integration to speed up and security
  • Global audience reach with the possibility of 35 data centers and more than 275 PoPs spread across the globe

Article was posted on here