Gosala Kulupana (BBC R&D)
High dynamic range (HDR) technology provides a large range of luminance values and consequently elevates the perceived quality in video content. At the same time, HDR content consumes a large amount of power when displayed on compatible devices. We have analysed the power consumption of the display across different HDR content and a few different HDR TV sets and proposed Just Noticeable Difference (JND) based algorithms to reduce the power consumption with a minimal impact on the user perception.
The flow chart below illustrates the architecture of the proposed algorithms. An autoregression model is first employed for each pixel to calculate two components: orderly components (the ones that are predictable from the surrounding), and disorderly components (the ones that are not predictable from the surrounding). The orderly components generally correspond to low-frequency parts of the image and hence are more sensitive to human eyes than the disorderly components. They are later used as inputs to compute several visual masks that are later applied on video frames.
Framework of the proposed power-saving algorithms
Display power consumption and image luma values have a monotonic relationship, which means as one variable goes up or down, the other variable goes in the same direction. Therefore, the above derived orderly components are used to compute the deduction in luma values corresponding to one JND (i.e., the first visual mask). In essence, they represent the maximum power savings attainable in the HDR displays for the smallest perceptible quality difference. During this process, we make sure that human skin tones are not affected, as the eye is very sensitive to skin tone (i.e., pixels that correspond to human skin tones are not modified).
A region-of-interest-based mask is also applied, to keep the objects within images minimally impacted and consequently deliver most of the power savings from background regions within images. A pre-trained model is deployed for this task.
Finally, a spatial correction mask is employed to preserve the edges of the objects, as the human eye is also sensitive to variations along object boundaries. A series of gradient, gaussian and box filters are used here to detect the edges and subsequently compute spatial correction masks.
Fifteen video sequences in three categories were tested to evaluate the performance of the proposed methods: sports, drama and nature. All were in UHD Hybrid-Log Gamma (HLG) HDR. Three HDR TV sets were used during the experiments (LG-C1 OLED TV, LG-C2 OLED TV and Sony Bravia 65” ZD9 LCD TV). For the LG sets, “Filmmaker” and “Standard” modes were used, whereas for the Sony TV, “Cinema Home” and “Standard” modes were used.
The experimental findings confirm that power consumption in display devices is highly dependent on the type of content they display (e.g., bright videos consume more power). Furthermore, the “Standard” mode in OLED TVs with extra processing of video images is linked to ~30% higher power consumption than the equivalent “Filmmaker” mode. Importantly, by employing the proposed power saving algorithms, the power consumption can be reduced up to 18% in certain scenarios. Further work will be conducted on embedding chrominance information and conducting formal subjective studies.
Jayasingam Adhuran also contributed to the work described here.
This article first appeared in issue 59 of EBU tech-i magazine.