We all know that Java JVM automatically allocates and cleans the application memory. The well-known Garbage Collector mechanism is responsible for all of it. As a result, a programmer doesn’t even have to be familiar with the mechanisms of operational memory usage.
Although the GC algorithms are constantly improving, they are still based on the frequent copying and optimizing of memory blocks.
Indeed, everything depends on various factors, but it’s assumed that a single application instance should not occupy more than 2GB of memory bytes. Larger amounts may cause performance drops, and with a size above 10GB, serious problems can be expected.
Meanwhile, operational and mass memory has become cheaper for several years and is no longer such a problem as before.
This is shown in movie files, which previously couldn’t reach even 1GB and now have increased to over 2GB for 4K resolution.
That’s why today I would like to show you how to reconcile fire and water and deal with a vast storage need while still having a small, high-performance application in Java.
The Challenge
Write a platform that enables efficient video streaming for many online users. The movie database can contain several hundred items of various sizes, from 0.5 to 2GB each. Total storage is 1TB. Operational memory is not a problem.
Pitfalls
Storing large videos on external resources is inefficient. Reading takes a lot of time, data is copied through various buffers, and we use memory, threads and external resources massively. Such an application is inefficient.
Storage HEAP memory exceeding 10GB may cause performance problems when the garbage collector works. That may cause significant performance issues. It’s unacceptable.
Solution
I will store videos in Non-Heap memory and provide multi-threaded access to resources without copying them. Then, I will enable concurrent read-only access and stream videos via Spring Rest API. The application will use 1TB of Non-Heap memory but just megabytes of Heap.
What is Non-Heap memory?
As we all know, the JVM creates its memory pool called heap storage. This is what the Garbage Collector takes care of. There is only one problem with it – when the memory area is too large, the work of GC, which is based mainly on constantly copying and optimizing memory blocks, can cause severe performance problems.
However, there are other possibilities: Non-Heap memory, also known as Off-Heap memory. This is the memory that the system allocates to the application process, making it separate and safe. This memory is released after the application closes. GC can’t see Non-Heap, and because of this, the app has to manage it. What can I say – it’s usually much more trouble than it’s worth. Also, Java only provides a few functionalities to handle such a memory.
Nevertheless, my case is specific – I only need to allocate vast areas and don’t even need to free them, making this memory a perfect fit. Garbage Collector problems will go away.
In addition, Non-Heap memory has another critical advantage – its size is limited only by physical hardware resources (the memory can be limited, but by default, it’s not). Therefore, I can put huge amounts of data into it, even 1TB of movies, and my application will remain light and efficient.
Non-Heap – allocation and access
To allocate Non-Heap memory, I will use the allocateDirect method in the ByteBuffer class. Each movie will create its own ByteBuffer, to which I will save items. I will combine all the movies in a map where the key will be the movie’s name, and the value will be a specific ByteBuffer object. The decorator for the map will be the Storage class – Spring singleton, which I will refer to from other places in my application.
I will add a few more useful methods to the Storage class, e.g. counting the amount of memory allocated for storing the movies
Stream endpoint
Issuing a streaming endpoint requires me to convert a ByteBuffer to a Spring InputStreamResource. Neither Java nor Spring supports it, and I had to create my own ByteBufferBackedInputStream implementation. It is essential that I don’t copy the item area from Non-Heap to Heap memory here. I only change the form of access to the Non-Heap byte array, which is actually a movie.
To expose the video stream, media must be listed as MediaType.APPLICATION_OCET_STREAM
Frontend
For the purposes of my test application, I will create a frontend to upload videos and watch them.
I will display everything on SpringBoot’s embedded Tomcat.
To enable the upload of larger files, I will set larger buffers in the properties.
I run my application at 9000 port.
Working example
I have created a simple POC application which allows for video file uploading and streaming on the web page. Because of Java’s internal limitations, I had to limit the size of the uploaded videos to 2GB. Using larger files is possible but needs additional complications, which I wanted to avoid in the test application. For testing purposes, I have loaded 5 movies and consumed 7.3GB of Non-Heap memory. Next, I opened 4 web pages. Each of them played a different movie. JConsole showed less than 50MB of real Heap usage.
Source Codes
Please find the source codes on GitHub here.
Klaudiusz Wojtkowiak
Backend Developer, DAC.digital
Klaudiusz is an experienced Senior Developer passionate and focused on backend improvements. Privately, he develops a Daobab library, which brings a unique ORM approach in Java and Kotlin. A father and husband who likes historical travels and gun shooting practice.
