Posted on

The State of #HDR in Broadcast and OTT – CES 2016 update

By Yoeri Geutskens

This article was first published in December 2015, but has been updated post-CES 2016 (corrections on Dolby Vision, UHD Alliance's "Ultra HD Premium" specification and the merging of Technicolor and Philips HDR technologies).

A lot has been written about HDR video lately, and from all of this perhaps only one thing becomes truly clear – that there appear to be various standards to choose from. What’s going on in this area in terms of technologies and standards? Before looking into that, let’s take a step back and look at what HDR video is and what’s the benefit of it.

Since 2013, Ultra HD or UHD has come up as a major new consumer TV development. UHD, often also referred to as ‘4K’, has a resolution of 3,840 x 2,160 – twice the horizontal and twice the vertical resolution of 1080p HDTV, so four times the pixels. UHD has been pushed above all by TV manufacturers looking for new ways to entice consumers to buy new TV sets. To appreciate the increased resolution of UHD, one needs to have a larger screen or a smaller viewing distance but it serves a trend towards ever larger TV sizes.

While sales of UHD TV sets are taking off quite prosperously, the rest of the value chain isn’t following quite as fast. Many involved feel the increased spatial resolution alone is not enough to justify the required investments in production equipment. Several other technologies promising further enhanced video are around the corner however. They are:

  • High Dynamic Range or HDR
  • Deep Color Resolution: 10 or 12 bits per subpixel
  • Wide Color Gamut or WCG
  • High Frame Rate or HFR: 100 or 120 frames per second (fps)

As for audio, a transition from conventional (matrixed or discrete) surround sound to object-based audio is envisaged for the next generation of TV.

Of these technologies, the first three are best attainable in the short term. They are also interrelated.

So what does HDR do? Although it’s using rather different techniques, HDR video is often likened to HDR photography as their aims are similar: to capture and reproduce scenes with a greater dynamic range than traditional technology can, in order to offer a more true-to-life experience. With HDR, more detail is visible in images that would otherwise look either overexposed, showing too little detail in bright areas, or underexposed, showing too little detail in dark areas.

HDR video is typically combined with a feature called Wide Color Gamut or WCG. Traditional HDTVs use a color space referred to as Rec.709, which was defined for the first generations of HDTVs which used CRT displays. Current flat panel display technologies like LCD and OLED can produce a far wider range of colors and greater luminance, measured in ‘nits’. A nit is a unit for brightness, equal to candela per square meter (cd/m2). To accommodate this greater color gamut, Rec.2020 color space was defined. No commercial display can fully cover this new color space but it provides room for growth. The current state of the art of color gamut for displays in the market is a color space called DCI-P3 which is smaller than Rec.2020 but substantially larger than Rec.709.

To avoid color banding issues that could otherwise occur with this greater color gamut, HDR/WCG video typically uses a greater sampling resolution of 10 or 12 bits per subpixel (R, G and B) instead of the conventional 8 bits, so 30 or 36 bits per pixel rather than 24.

Sony-Color-space

Color/luminance volume: BT.2020 (10,000 nits) versus BT.709 (100 nits); Yxy
Image credit: Sony

The problem with HDR isn’t so much on the capture side nor on the rendering side – current professional digital cameras can handle a greater dynamic range and current displays can produce a greater contrast than the content chain in between can handle. It’s the standards for encoding, storage, transmission and everything else that needs to happen in between that are too constrained to support HDR.

So what is being done about this? A lot, in fact. Let’s look at the technologies first. A handful of organizations have proposed technologies for describing HDR signals for capture, storage, transmission and reproduction. They are Dolby, SMPTE, Technicolor, Philips, and BBC together with NHK. Around the time of CES 2016, Technicolor and Philips have announced they are going to merge their HDR technologies.

Dolby’s HDR technology is branded Dolby Vision. One of the key elements of Dolby Vision is the Perceptual Quantizer EOTF which has been standardized by SMPTE as ST 2084 (see box: SMPTE HDR Standards) and mandated by the Blu-ray Disc Association for the new Ultra HD Blu-ray format. The SMPTE ST 2084 format can actually contain more picture information than TVs today can display but because the information is there as manufacturers build better TVs the content has the potential to look better as the new, improved display technologies come to market. Dolby Vision and HDR10 use the same SMPTE 2084 standard making it easy for studios and content producers to master once and deliver to either HDR10 or, with the addition of dynamic metadata, Dolby Vision. The dynamic metadata is not an absolute necessity, but using it guarantees the best results when played back on a Dolby Vision-enabled TV. HDR10 uses static metadata which ensures it will still look good – far better than Standard Dynamic Range (SDR). Even using no metadata at all, SMPTE 2084 can work at an acceptable level just as other proposed EOTFs without metadata do.

For live broadcast Dolby supports both single and dual layer 10-bit distribution methods and has come up with a single workflow that can simultaneously deliver an HDR signal to the latest generation and future TVs and a derived SDR signal to support all legacy TVs. The signal can be encoded in HEVC or AVC. Not requiring dual workflows will be very appealing to all involved in content production and the system is flexible to let the broadcaster choose where to derive the SDR signal.  If it’s done at the head-end they can choose to simply simulcast it as another channel or convert the signal to dual-layer single stream signal at the distribution encoder for transmission.  Additionally the HDR-to-SDR conversion can be built into set-top boxes for maximum flexibility without compromising the SDR or HDR signals. Moreover, the SDR distribution signal that’s derived from the HDR original using Dolby’s content mapping unit (CMU) is significantly better in terms of detail and color than one that’s captured natively in SDR, as Dolby demonstrated side by side at IBC 2015. The metadata is only produced and multiplexed into the stream at the point of transmission, just before or in the final encoder – not in the baseband workflow. Dolby uses 12-bit color depth for cinematic Dolby Vision content to avoid any noticeable banding but the format is actually agnostic to different color depths and works with 10-bit video as well. In fact, Dolby recommends 10-bit color depth for broadcast.

High-level overview of Dolby Vision dual-layer transmission for OTT VOD

High-level overview of Dolby Vision dual-layer transmission for OTT VOD;
other schematics apply for OTT live, broadcast, etc. 
Image credit: Dolby Labs Dolby Vision white paper

Technicolor has developed two HDR technologies. The first takes a 10-bit HDR video signal from a camera and delivers a video signal that is compatible with SDR as well as HDR displays. The extra information that is needed for the HDR rendering is encoded in such a way that it builds on top of the 8-bit SDR signal but SDR devices simply ignore the extra data.

Technicolor

Image credit: Technicolor

The second technology is called Intelligent Tone Management and offers a method to ‘upscale’ SDR material to HDR, using the extra dynamic range that current-day capture devices can provide but traditional encoding cannot handle, and providing enhanced color grading tools to colorists. While it remains to be seen how effective and acceptable the results are going to be, this technique has the potential to greatly expand the amount of available HDR content.

Having a single signal that delivers SDR to legacy TV sets (HD or UHD) and HDR to the new crop of TVs is also the objective of what BBC’s R&D department and Japan’s public broadcaster NHK are working on together.  It’s called Hybrid Log Gamma or HLG. HLG’s premise is an attractive one: a single video signal that renders SDR on legacy displays but HDR on displays that can handle this. HLG, BBC and NHK say, is compatible with existing 10-bit production workflows and can be distributed using a single HEVC Main 10 Profile bitstream.

Depending on whom you ask HLG is the best thing since sliced bread or a clever compromise that accommodates SDR as well as HDR displays but gives suboptimal results and looks great on neither. The Hybrid Log Gamma name refers to the fact that the OETF is a hybrid that applies a conventional gamma curve for low-light signals and a logarithmic curve for the high tones.

HLG

Hybrid Log Gamma and SDR OETFs; image credit: T. Borer and A. Cotton, BBC R&D

Transfer functions:

  • OETF: function that maps scene luminance to digital code value; used in HDR camera;
  • EOTF: function that maps digital code value to displayed luminance; used in HDR display;
  • OOTF: function that maps scene luminance to displayed luminance; a function of the OETF and EOTF in a chain. Because of the non-linear nature of both OETF and EOTF, the chain’s OOTF also has a non-linear character.

 BBC_WorkFlow

Image credit: T. Borer and A. Cotton, BBC R&D

The EOTF for Mastering Reference Displays, conceived by Dolby and standardized by SMPTE as ST 2084 is ´display-referred'.  With this approach, the OOTF is part of the OETF, requiring implicit or explicit metadata.

Hybrid Log Gamma (HLG), proposed by BBC and NHK, is a 'scene-referred' system which means the OOTF is part of the EOTF. HLG does not require mastering metadata so the signal is display-independent and can be displayed unprocessed on an SDR screen.

The reasoning is simple: bandwidth is scarce, especially for terrestrial broadcasting but also for satellite and even cable, so transmitting the signal twice in parallel, in SDR and HDR, is not an attractive option. In fact, most broadcasters are far more interested in adding HDR to 1080p HD channels than in launching UHD channels, for exactly the same reason. Adding HDR is estimated to consume up to 20% extra bandwidth at most, whereas a UHD channel gobbles up the bandwidth of four HD channels. It’s probably no coincidence HLG technology has been developed by two broadcast companies that have historically invested a lot in R&D. Note however that the claimed backwards compatibility of HLG with SDR displays only applies to displays working with Rec.2020 color space, i.e. Wide Color Gamut. This more or less makes its main benefit worthless.

ARIB, the Japanese organization that’s the equivalent of DVB in Europe and ATSC in North America, has standardized upon HLG for UHD HDR broadcasts.

The DVB Project meanwhile has recently announced that UHD-I phase 2 will actually include a profile that adds HDR to 1080p HD video – a move advocated by Ericsson  and supported by many broadcasters. Don’t expect CE manufacturers to start producing HDTVs with HDR however. Such innovations are likely to end up only in the UHD TV category, where the growth is and any innovation outside of cost reductions takes place.

This means consumers will need a HDR UHD TV to watch HD broadcasts with HDR. Owners of such TV sets will be confronted with a mixture of qualities – plain HD, HD with HDR, plain UHD and UHD with HDR (and WCG), much in the same way HDTV owners may watch a mix of SD and HD television, only with more variations.

The SMPTE is one of the foremost standardization bodies active in developing official standards for the proposed HDR technologies. See box ‘SMPTE HDR standards’.

SMPTE HDR Standards

ST 2084:2014 - High Dynamic Range EOTF of Mastering Reference Displays

  • defines 'display referred' EOTF curve with absolute luminance values based on human visual model
  • called Perceptual Quantizer (PQ)

ST 2086:2014 - Mastering Display Color Volume Metadata supporting High Luminance and Wide Color Gamut images

  • specifies mastering display primaries, white point and min/max luminance

Draft ST 2094:201x - Content-dependent Metadata for Color Volume Transformation of High-Luminance and Wide Color Gamut images

  • specifies dynamic metadata used in the color volume transformation of source content mastered with HDR and/or WCG imagery, when such content is rendered for presentation on a display having a smaller color volume

One other such body is the Blu-ray Disc Association (BDA). Although physical media have been losing some popularity with consumers lately, few people are blessed with a fast enough broadband connection to be able to handle proper Ultra HD video streaming, with or without HDR. Netflix requires at least 15 Mbps sustained average bitrate for UHD watching but recommends at least 25 Mbps. The new Ultra HD Blu-ray standard meanwhile offers up to 128 Mpbs peak bit rate. Of course one can compress Ultra HD signals but the resulting quality loss would defy the entire purpose of Ultra High Definition.

Ultra HD Blu-ray may be somewhat late to the market, with some SVOD streaming services having beat them to it, but the BDA deserves praise for not rushing the new standard to launch without HDR support. Had they done that, the format may very well have been declared dead on arrival. The complication, of course, was that there was no single agreed-upon standard for HDR yet. The BDA has settled on the HDR10 Media Profile (see box) as mandatory for players and discs with Dolby Vision and Philips’ HDR format as optional for players as well as discs.

HDR10 Media Profile

  • EOTF: SMPTE ST 2084
  • Color sub-sampling: 4:2:0 (for compressed video sources)
  • Bit depth: 10 bit
  • Color primaries: ITU-R BT.2020
  • Metadata: SMPTE ST 2086, MaxFall (Maximum Frame Average Light Level), MaxCLL (Maximum Content Light Level)

Referenced by:

  1. Ultra HD Blu-ray spec (Blu-Ray Disc Association)
  2. HDR-compatible display spec (CTA; former CEA)

UHD Alliance ‘Ultra HD Premium’ definition Display Content Distribution
Image resolution 3840×2160 3840×2160 3840×2160
Color Bit Depth 10-bit signal Minimum 10-bit signal depth Minimum 10-bit signal depth
Color Palette Signal input: BT.2020 color representation

Display reproduction: More than 90% of P3 color space

BT.2020 color representation BT.2020 color representation
High Dynamic Range SMPTE ST 2084 EOTF

A combination of peak brightness and black level either:

More than 1000 nits peak brightness and less than 0.05 nits black level
or

More than 540 nits peak brightness and less than 0.0005 nits black level

SMPTE ST 2084 EOTF

Mastering displays recommended to exceed 1000 nits in brightness, less than 0.03 black level, minimum of DCI-P3 color space

SMPTE ST 2084 EOTF

The UHD Alliance mostly revolves around Hollywood movie studios and is focused on content creation and playback, guidelines for CE devices, branding and consumer experience). At CES 2016, the UHDA has announced a set of norms for displays, content end ‘distribution’ to deliver UHD with HDR, and an associated logo program. The norm is called ‘Ultra HD Premium’ (see box). Is it a standard? Arguably, yes. Does it put an end to any potential confusion over different HDR technologies? Not quite – while the new norm guarantees a certain level of dynamic range it does not specify any particular HDR technology, so all options are still open. 

The Ultra HD Forum meanwhile focuses on the end-to-end content delivery chain including production workflow and distribution infrastructure.

In broadcasting we’ve got ATSC in North America defining how UHD and HDR should be broadcast over the air with the upcoming ATSC 3.0 standard (also used in South Korea) and transmitted via cable. Here, the SCTE comes into play as well. Japan has the ARIB (see above) and for most of the rest of the world, including Europe, there’s the DVB Project, part of the EBU, specifying how UHD and HDR should fit into the DVB standards that govern terrestrial, satellite and cable distribution.

In recent news, the European Telecommunications Standards Institute (ETSI) has launched a new Industry Specification Group (ISG) “to work on a standardized solution to define a scalable and flexible decoding system for consumer electronics devices from UltraHD TVs to smartphones” which will look at UHD, HDR and WCG. Founding members include telcos BT and Telefónica. The former already operates a UHD IPTV service; the latter is about to launch one.

Then there are CTA (Consumer Technology Association, formerly known as CEA) in the US and DigitalEurope dealing with guidelines and certification programs for consumer products. What specifications does a product have to support to qualify for ‘Ultra HD’ branding? Both have formulated answers to that question. It has not been a coordinated effort but fortunately they turn out to almost agree on the specs. Unity on a logo was not as feasible, sadly. The UHD Alliance has just announced they’ve settled on a definition of Ultra HD they’ll announce at CES, January 4th, 2016. One can only hope this will not lead to yet more confusion (and more logos) but I’m not optimistic.

By now, the CTA has also issued guidelines for HDR. DigitalEurope hasn’t yet. It’d be great for consumers, retailers and manufacturers alike if the two organizations could agree on a definition as well as a logo this time.

Ultra HD display definition CTA definition DigitalEurope definition
Resolution At least 3840x2160 At least 3840x2160
Aspect ratio 16:9 or wider 16:9
Frame rate Supporting 24p, 30p and 60p 24p, 25p, 30p, 50p, 60p
Chroma subsampling Not specified 4:2:0 for 50p, 60p

4:2:2 for 24p,25p, 30p

Color bit depth Minimum 8-bit Minimum 8-bit
Colorimetry BT.709 color space; may support wider colorimetry standards Minimum BT.709
Upconversion Capable of upscaling HD to UHD Not specified
Digital input One or more HDMI inputs supporting HDCP 2.2 or equivalent content protection. HDMI with HDCP 2.2
Audio Not specified PCM 2.0 Stereo
Logo  CTA Logo UHD  DigitalEur Logo UHD

CTA definition of HDR-compatible:

A TV, monitor or projector may be referred to as a HDR Compatible Display if it meets the following minimum attributes:

  1. Includes at least one interface that supports HDR signaling as defined in CEA-861-F, as extended by CEA-861.3.
  2. Receives and processes static HDR metadata compliant with CEA-861.3 for uncompressed video.
  3. Receives and processes HDR10 Media Profile* from IP, HDMI or other video delivery sources. Additionally, other media profiles may be supported.
  4. Applies an appropriate Electro-Optical Transfer Function (EOTF), before rendering the image.

CEA-861.3 references SMPTE ST 2084 and ST 2086.

What are consumers, broadcasters, TV manufacturers, technology developers and standardization bodies to do right now?

I wouldn’t want to hold any consumer back but I couldn’t blame them if they decided to postpone purchasing a new TV a little longer until standards for HDR have been nailed. Similarly, for broadcasters and production companies it only seems prudent to postpone making big investments in HDR production equipment and workflows.

For all parties involved in technology development and standardization, my advice would be as follows. It’s inevitable we’re going to see a mixture of TV sets with varying capabilities in the market – SDR HDTVs, SDR UHD TVs and HDR UHD TVs, and that’s not even taking into consideration near-future extensions like HFR.

Simply ignoring some of these segments would be a very unwise choice: cutting off SDR UHD TVs from a steady flow of UHD content for instance would alienate the early adopters who bought into UHD TV already. The CE industry needs to cherish these consumers. It’s bad enough that those Brits who bought a UHD TV in 2014 cannot enjoy BT Sport’s Ultra HD service today because the associated set-top box requires HDCP 2.2 which their TV doesn’t support.

It is not realistic to cater to each of these segments with separate channels either. Even if the workflows can be combined, no broadcaster wants to spend the bandwidth to transmit the same channel in SDR HD and HDR HD, plus potentially SDR UHD and HDR UHD.

Having separate channels for HD and UHD is inevitable but for HDR to succeed it’s essential for everyone in the production and delivery chain that the HDR signal be an extension to the broadcast SDR signal and the SDR signal be compatible with legacy Rec.709 TV sets.

Innovations like Ultra HD resolution, High Dynamic Range, Wide Color Gamut and High Frame Rate will not come all at once with a big bang but (apart from HDR and WCG which go together) one at a time, leading to a fragmented installed base. This is why compatibility and ‘graceful degradation’ are so important: it’s impossible to cater to all segments individually.

What is needed now is alignment and clarity in this apparent chaos of SDOs (Standards Defining Organizations). Let’s group them along the value chain:

Domain Production Compression Broadcast Telecom Media/ Streaming CE
SDO SMPTE, ITU-R MPEG , VCEG ATSC, SCTE, EBU/DVB, ARIB, SARFT ETSI BDA, DECE (UV), MovieLabs CTA, DigitalEurope, JEITA

Within each segment, the SDOs need to align because having different standards for the same thing is counterproductive. It may be fine to have different standards applied, for instance if broadcasting uses a different HDR format than packaged media; after all, they have differing requirements. Along the chain, HDR standards do not need to be identical but they have to be compatible. Hopefully organizations like the Ultra HD Forum can facilitate and coordinate this between the segments of the chain.

If the various standardization organizations can figure out what HDR flavor to use in which case and agree on this, the future is looking very bright indeed.

Further reading:

Yoeri Geutskens has worked in consumer electronics for more than 15 years. He writes about high-resolution audio and video. You can follow him on Ultra HD and 4K on twitter @UHD4k.

Posted on

User experience gain must trump resource consumption for UHD success

This opinion blog is about 3 things that could derail UHD if User Experience lets them.

Ok so you can already tell that I’m biased. I believe in UHD and its five components that will change user experience:

  1. Higher resolution (4K)
  2. Higher Dynamic Range (ability to see details in both brighter whites and darker blacks simultaneously)
  3. Better colour (more colours, closer to human perception)
  4. Better sound (more channels than speakers, object based surround sound)
  5. Higher refresh rate (especially for action which can otherwise look choppy at very high resolution without higher refresh rates)

My crystal ball hasn’t confirmed that this is the order in which these components will arrive, or to what extent it’ll be a big bang approach, or even if some components might get left by the wayside. I’ll delve into that in another blog. In the last 15 years or so, I’ve witnessed HD succeed and I have written several times about why I believe UHD’s time is now (recently here or here in early 2014 for example). I have nothing to sell and no vested interest in UHD, I’m simply driven by my geeky fascination with the promise of a great new experience brought to TV and IP technologies that I’ve been working with for so long.

But just in case I am wrong, here are 3 things that that some of us fret about and still could prevent UHD success.

Thing Description Issue Why it won’t stop UHD
Fragmentation Vendors pulling in different directions Device & content incompatibility Industry bodies like the Ultra HD Forum or the UHD Alliance
Energy Extra brightness, more pixels and more images consume more power Regulation, consumer reluctance, UHD perceived as not Politically Correct Technology progress has often consumed more power (e.g. HD vs SD). Better efficiency means extra power required is less than extra user experience delivered. CPE power issue is more in standby mode than peak consumption. Need not consume much more power with HD-only signal.
Bandwidth UHD can require over 4 times HD bandwidth / file size. Channel and content distribution issues. Monthly data caps will be an issue for OTT households. Networks grow in quantum leaps, UHD will help spur the next one. All-fibre connections and future 5G networks will provide more bandwidth than UHD can consume. A new generation of low-orbit satellites is also on its way.

The driving force providing the impetus to overcome challenges such as those mentioned above is User Experience. This is the part of the equation I have to rely on gut feeling or faith for. My premise is that UHD ushers in a great new User Experience with a sensation of realism and immersion.

If it actually turned out that UHD didn’t bring that “wow” effect so many of us in the industry believe in, then any one of the above “things” could alone derail UHD from becoming a market success and we’ll have to find another game changer in the TV industry.  My experience so far suggests that UHD will be that game change but also that there are still niggles that need ironing out.

As it happens I’ve been watching quite a lot of 4K TV via Amazon and Netflix in the last few months. Landscapes and close-ups are all pretty amazing, but I do have a nagging worry over some indoor scenes, which despite being shot by top-of the range pros (e.g. Amazon’s Transparent, or Breaking Bad, …) leave a strange feeling that something isn’t quite right in 4K resolution. It occurs when there is some mild camera movement yet when most of the scene is in focus. I get this counterintuitive sensation that there is something maybe amateurish in the composition. This could be due to the shooting not having been properly thought out by the director and cameraman for 4K TV playback, or maybe it’s just me not yet being used to processing so much data on screen. If either of these is true, which I suspect is the case, the issue will quickly disappear. But this highlights my only real concern over UHD’s success: will it be consistently “wow” enough to overcome resistances like the three issues stated above? If so I have no doubts that vendors, content providers and operators, as personified in the Ultra HD Forum, will be insure that the whole UHD movement is not derailed by relatively minor teething troubles.

Posted on

@nebul2’s 14 reasons why 2015 will be yet another #UHD #IBCShow

Ultra HD or 4K has been a key topic of my pre and post IBC blogs for over 5 years. I’ve recently joined the Ultra HD Forum, serving on the communications working group. That’s a big commitment and investment, as I don’t have any large company paying my bills. I’m making it because I believe the next 18 months will see the transition from UHD as the subject of trials for big operators and precursor launches to something no operator can be without. Time to get off the fence. I once wrote that the 3D emperor didn’t have any clothes on; well, the UHD emperor is fully clothed.

Of course much still needs to be achieved before we see mass adoption. I don’t know if HDR and 4K resolution will reach market acceptance one at a time or both together, and yes, I don’t know which HDR specification will succeed. But I know it’s all coming.

Below is a list of 14 key topics ordered by my subjective (this is a blog remember) sense of comfort on each. I start with areas where the roadmap to industrial strength UHD delivery is clear to me and end with those where I’m the most confused.

Note on vocabulary: 4K refers to a screen resolution for next gen TV whereas UHD includes that spatial resolution (one even sees UHD phase 2 documents refer to an 8K resolution) but also frame rate, HDR and next generation Audio.

So as I wander round IBC this year, or imagine I’m doing that, as probably won’t have time, I’ll look into the following 14 topics with growing interest.

1. Broadcast networks (DVB)

I doubt I’ll stop by the big satellite booths for example, except of course for free drinks and maybe to glimpse the latest live demos. The Eutelsat, Intelsat or Astras of this world have a pretty clear UHD story to tell. Just like the cableCos, they are the pipe and they are ready, as long as you have what it takes to pay.

2. Studio equipment (cameras etc.)

As a geek, I loved the Canon demos at NAB, both of affordable 4K cameras and their new ultra sensitive low-light capabilities. But I won’t be visiting any of the studio equipment vendors, simply because I don’t believe they are on the critical path for UHD success. The only exception to this is the HDR issues described below.

 3. IP network; CDN and Bandwidth

Bandwidth constricts UHD delivery; it would be stupid to claim otherwise. All I’m saying is that by putting this issue so high on the list everything is clear in the mid-term. We know how fast High-Speed Broadband (over 30MPS) is arriving in most markets. In the meantime, early adopters without access can buy themselves a UHD Blu-ray by Christmas this year and use progressive download services. The Ultra HD Alliance has already identified 25 online services, several of which support PDL. Once UHD streams get to the doorstep or the living room, there is still the issue of distributing them around the home. But several vendors like AirTies are addressing that specific issue, so again, even if it isn’t fixed, I can see how it will be.

 4. Codecs (HEVC)

The angst around NAB this year when V-nova came out with a bang has subsided. It seems now that even if such a disruptive technology does come through in the near-term, it will complement not replace HEVC for UHD delivery.

The codec space dropped from a safe 2 in my list down to 4 with the very recent scares on royalties from the HEVC Advance group that wants 0.5% of content owner & distributor's gross revenue. Industry old-timers have reassured me that this kind of posturing is normal and that the market will settle down naturally at acceptable rates.

 5. Head-ends (Encoders, Origins, etc.)

I always enjoy demos and discussion on the booths of the likes of Media Excel, Envivio, Harmonic, Elemental or startup BBright and although I’ll try to stop by, I won’t make a priority of them because here again, the mid-term roadmaps seem relatively clear.

I’ve been hearing contradictory feedback on the whole cloud-encoding story that has been sold to us for a couple of years already. My theory – to be checked at IBC – is that encoding in the cloud really does make sense for constantly changing needs and where there is budget. But for T2 operators running on a shoestring – and there are a lot of them – the vendors are still mainly shifting appliances. It’s kind of counterintuitive because you’d expect the whole cloud concept of mutualizing resources to work better for the smaller guys. I must have something missing here, do ping me with info so I can update this section.

 6. 4K/UHD resolutions

While there is no longer any concern on what the screen resolutions will be, I am a little unclear as to the order in which they will arrive. With heavyweights like Ericsson openly pushing for HDR before 4K, I’m a little concerned that lack of industry agreement on this could confuse the market.

 7. Security for UHD

Content owners and security vendors like Verimatrix have all agreed that better security is required for UHD content. I see no technical issues here - just that if the user experience is adversely affected in any way (remember the early MP3 years), we could see incentive for illegal file transfer grow, just when legal streaming seems to be taking of at last.

 8. TV sets & STBs

Well into second half of my list, we’re getting into less clear waters.

When it’s the TV set that is doing the UHD decoding, we’re back at the product cycle issue that has plagued smart TVs. It’s all moving too fast for a TV set that people still would like to keep in the living room for over 5 years.

On the STB side, we’ve seen further consolidation since last year’s IBC. Pace for example is no longer; Cisco is exiting STBs etc. It seems that only players with huge scale will survive. Operators like Swisscom or Orange can make Hardware vendors’ lives harder by commoditizing their hardware using software-only vendors such as SoftAtHome to deliver advanced features.

 9. Frame rates

This is a really simple one but for which consensus is needed. At a 4K screen resolution the eye/brain is more sensitive to artifacts. Will refresh rates standardize at 50Hz or 60Hz? Will we really ever need 120Hz?

It’s clear that doubling a frame rate does not double the required bandwidth as clever compression techniques come to play. But but I haven’t seen a consensus on what the bandwidth implication of greater frame rate will actually be.

10. Next Gen Audio

There are only a few contenders out there, and all have compelling solutions. I’m pretty keyed up on DTS’s HeadphoneX streamed with Unified Streaming packagers because I’m helping them write an eBook on the subject. Dolby is, of course, a key player here but for me it’s not yet clear how multiple solutions will cohabit. It isn’t yet clear how if and when we’ll move from simple channel-based to scene based or object based audio. Will open source projects like Ambiophonics play a role and what about binaural audio.

11. HDR

High Dynamic Range is about better contrast. Also, the brain perceives more detail when contrast is improved, so it’s almost like getting more pixels for free. But the difficulty with HDR and why it’s near the bottom of my list is that there are competing specifications. And even once a given specification is adopted, its implementation on a TV set can vary from one CE manufacturer to another. I final reservation I have is the extra power consumption it will entail that goes against current CE trends.

12. Wide Color Gamut

As HDR brings more contrast to pixels WCG brings richer and truer colors. Unlike with HDR, the issue isn’t about which spec to follow, as it is already catered for in HEVC for example. No, it’s more about when to implement it and how the color mapping will be unified across display technologies and vendors.

 13. Work flows

Workflow from production through to display is a sensitive issue because it is heavily dependant on skills and people. So it’s not just a mater of choosing the right technology. To produce live UHD content including HDR, there is still no industry standard way of setting up a workflow.

 14. UHD-only content

The pressure to recoup investments in HD infrastructure makes the idea of UHD content that is unsuitable for HD downscaling taboo. From a business perspective, most operators consider UHD as an extension or add-on rather than something completely new. There is room for a visionary to coma and change that.

Compelling UHD content, where the whole screen is in focus (video rather than cinema lenses) gives filmmakers a new artistic dimension to work on. There is enough real estate on screen to offer multiple user experiences.

In the world of sports a UHD screen could offer a fixed view on a whole football pitch for example. But if that video were seen on an HD screen, the ball probably wouldn’t be visible. Ads that we have to watch dozens of times could be made more fun in UHD as their could be different storied going on in different parts of the screen, it would almost be an interactive experience …

Posted on

My Ultra High Definition #NAB15

There is always plenty to see at NAB and if I liked Las Vegas, I’d surely come more often. But the last time I was here was in 2001 when the show was dubbed the convergence NAB.

I’m going again this year for one main reason: Ultra HD. Sure 4K has been a key topic for at least the last five NABs, but now is different and here’s why.

Gut feelings aren’t very useful in making business decisions, but sometimes, once all evidence has been considered, that’s all you have to go by. Intuitions can even be dangerous when you don’t have enough information, like in 1993 when I advised my cousin not to join an Internet start-up because I “felt” that the Web was going to stay the realm of geeks and early adopters. Thank god he didn’t heed my stupid advice. I also missed the SMS boat with a strong gut feeling that such a barbaric user experience would never make it mainstream. But those were spot decisions made without enough background knowledge let alone understanding of what was really going on under the surface, driving user behaviour. Daniel Kanneman, in his excellent book Thinking Fast and Slow explains how any experts intuitions are mainly bunk anyway.

So I’ll attempt o explain why UHD will be real and will start now with as much argument and fact. I will own up to what is no more than opinion. To prove that this isn’t just consulting BS (yes I admit I’m also a consultant) I’m spending a few grand of my own money and taking 5 days off to go to NAB, which represents over my annual investment as one man band. Now is the time when the industry will really launch UHD, and I want to be there. The table below lists the main reasons given by those who advocate waiting for the time being.

Issue

Description

Facts

Opinion

for

against

Content creation, postproduction & workflows 4K content requires powerful hardware and four times the storage space. Studios already shoot in 4K, as do high-end smartphones. 4K cameras already used to produce several feeds in many studios. Cost, lack of standards, creates risk of needing to re-invest. To produce the best possible HD it’s already better to work in UHD then downscale. Be first out of the stalls for UHD deployment.
Content availability Most libraries are not UHD Anything shot on 70mm film can be re-mastered.

Up-scaling any content improves the HD experience.

It will take time to reach critical mass of native UHD content. This is also a chicken and egg situation. Offer and demand will prod each other forwards. Shorter content shelf life means more new content.
Colour depth and refresh rates UHD shows everything better including flaws. 30 fps looks jittery in UHD. These issues will be addressed irrespective of UHD. HD needs them too. UHD hardware may not support future HDR and HFR specs. Can cause legitimate delays. However, improved HD creates awareness of picture quality and fuels desire for UHD.
Device readiness Require more power to decode. With Moore’s law still in effect this problem will disappear shortly Impossible to leverage existing hardware in the field Smartphones shoot 4K that users will want to watch (Apples 5K iMac sets the scene). UHD is a premium feature consumers will pay for.
Distribution and Bandwidth Require 3 to 4 times the HD bandwidth Fibre and 4G deployments in full swing, HEVC is here Volumes required for ABR file storage will explode. Beyond HEVC, a new wave will come (I’l keeps tabs on the buzzing V-Nova).
Screen size and viewing distance To be at least 6 feet away from screen it must be at least 55” In urban homes shorter viewing distances make sense. Huge screen sizes are only popular in some markets As long as consumers perceive benefits they will adapt, they always do.

I wont dwell in this blog on the benefits of UHD, but unlike with other technical (r)evolutions such as 3D, all content will benefit from UHD. I also see an opportunity for a new kind of video story telling. High resolution content with shorter viewing distances lets different parts of the screen tell a different story depending on how and where you watch. Video can become more immersive. Choosing what part of the screen to watch is almost an interactive experience. Whether 3D technology is present or not will become a technical detail.

All the issues discussed above will benefit from branding, standardization and end-to-end interoperability testing which is why I will be reporting from the launch of the UHD Forum in Vegas. I’ll also look into the UHD alliance which has already launched a consumer facing Web site.

I’ll write a post-NAB blog, so far I intend to meet up with:

  • the Ultra HD Forum gang,
  • The Ultra HD Alliance (if I find them),
  • V-Nova that boasts UHD stream at HD bitrates,
  • BBright that offers an entry-level UHD play-out system that simplifies trials,
  • Verimatrix, that is launching a new UHD focussed security suite,
  • Sony to get a feel of the latest 4K cameras and TVs,
  • Please comment if you have other suggestions.

You only need to see UHD twice for it to make sense UHDUHD ;o)

[Update, just got back rather that a new blog here is the word cloud of my impressions walking around the halls and listening to conferences (see my twitter for more details)] : wordle 7

Posted on

Real UHD deployments in 2016 says Thierry Fautier, UHD Forum will help

Most of the video ecosystem is agreed on one thing: Ultra HD or 4K will happen, but none of us agree yet on when and how. It is clear is that the standards will play a key role in determining the timeframe. In previous cases, say with DASH for example, an industry body above competing standards has been the most effective way to speed things up. It seems like two separate initiatives are coalescing independently, which may be a good thing. CES 2015 was the place to be and many UHD issues where addressed. To get a clearly picture, I spoke to someone at the heart of it all. Here is my interview with Thierry Fautier VP of Video Strategy for Harmonic Inc.

Q: First of all Thierry can you confirm that UHD was a prominent them in Las Vegas this year?

A: Most certainly, Ultra HD was one of the most prominent topics at CES 2015. This was the first major show since some key announcements of Ultra HD services in late 2014:
- UltraFlix and Amazon that offer OTT services on connected TVs,
- DirecTV that announced a push VoD satellite service (through its STB that stores and then streams with decoding in the Samsung UHD TV),
- Comcast that announced a VoD streaming service directly through the Samsung TV, with content from NBC.

Q: But these services require UHD decoding on a Smart TV?

A: Yes that is a first takeaway from CES: TVs are the ones decoding UHD for now, STBs will start doing so from second half of 2015.

Q: Beyond the few services just described, what signs did you see that UHD might really start becoming available to all from 2015?

A: Several, for example the announcement that Warner Bros has decided to publish UHD titles using Dolby’s Vision process. Netflix also announced that its Marco Polo series would be re-mastered in HDR (but without announcing which technology). So on content and services side, things are moving on HDR.

Q: Do you see HDR as one of the first challenges to solve for UHD to succeed?

A: I do. The plethora of HDR demonstrations by all UHD TV manufacturers was impressive. I will not go into the details of the technologies used, it would take too much time and this may change (due to the standardization effort of HDR). The only thing I would say is that there is a consensus in the industry to produce UHD TV, it will be around 1,000 nits (against 10,000 for the MovieLabs spec) [a NIT is a measurement of light where a typical skylight lets in about 100 Million Nits and a florescent light about 4,000 Nits]. On the technology side, LG is the outsider with its OLED technology that was shown in 77 inches, while the rest of the industry seems to focus on the quantum dot (Samsung announced in 2014 that it was abandoning OLED).
This suggests that we will have HDR in 2015; the real question is on which spec HDR will be based? You now understand the eagerness of studios to standardize HDR.

Q: so is HDR a complete mess?

A: HDR is actually already in the process of standardization, but with more or less synchronized work:

- ITU began a call for a technology which was answered by Dolby, BBC, Philips and Technicolor.
- EBU / DVB is working on a standardization of HDR mainly for live broadcast applications. The goal is to finalize the spec in 2015.
- SMPTE is defining the parameters required for the production of HDR content. A first spec (ST 2036 for HDR EOTF and ST 2086 for Metadata) has already been ratified.
- MPEG is currently defining what to add to the existing syntax to HDR in a single layer. The outcome is expected in July 2015.
- Blu-ray is finalizing its HDR (single layer) specification and also hopes to freeze it mid-2015 to optimistically hoping to launch services in time for Christmas 2015. Blu-ray is working in coordination with MPEG and SMPTE. Note that Bly Ray will then follow specifications for streaming / download under Ultra Violet.
- The Japanese stakeholders, through NHK, announced they would now develop their own HDR for 8K.

So you see the diversity of the various proposals that exist, the new “Ultra HD Alliance” should bring some order here. The clue I can give is that to have a Blu-ray UHD service in 2015, this must be done with chips that are already in production in 2015. I think we will see more clearly at NAB (April) and that by IFA (September) everything will be decided, at least for the short term, aligned hopefully with DVB / EBU Ultra HD-1 Phase 2.

Q: I gather what is now called the “Ultra HD Alliance” is actually something different to what I described in my last blog and that it’s first challenge is getting HDR sorted out?

A: Indeed Ben, the Ultra HD Alliance is a group of 10 companies primarily from Hollywood and the world of TV in addition to Netflix and DirecTV on the operator’s side. The first goal of this group is to get HDR (High Dynamic Range) specifications under control (see diagram below) and the quality measurement from the output of the UHDTV. In this regard, Netflix will launch a certification of the quality of HDR streaming; HD to start and we can imagine that this will be extended to UHD. Note that no manufacturers have yet been invited, which is surprising as they are the ones actually going to do most of the job!

Q: So the organization we spoke about last time is something else?

A: Yes Harmonic, with a group of 40 other companies have proposed to create an Ultra HD Forum to take care of the complete UHD chain from end-to-end, including OTT, QoS, Push-VoD, nVOD, adaptive streaming, Live and on-demand. After various meetings that took place at CES, discussions are on going to ensure that the two groups (UHD Alliance and UHD Forum) work closely together.

Q: so as in other areas would you see the need for at least two governing bodies to manage UHD standards?

A: In the short term yes. The UHD Alliance is focussing a single blocking factor at the moment i.e. HDR/WCG/Audio , but will have a broader marketing and evangelization remit. The UHD Forum on the other hand is starting out with and ambition of end-to-end ecosystem impact. In the longer term there is no reason the two entities might not merge, but from where we stand today it seems most efficient to have the two bodies with the different focuses.

Q: Does HDR make sense without HRF (High Frame Rate)?

A: Well I'd say on the chip side there is still a challenge as 2 times more computing power is required; HDMI is also a limiting factor as bandwidth increases. Early services might get away with just a 25% increase. Most encoder providers are not yet convinced that the effort will produce improvements justifying the disruption brought by the doubling of frame rate. We have been asking for 60/120 fps formal testing but we’ll need to wait for the new generation cameras especially in sport, as opposed too currently used cameras often equipped with low shutter speeds coming from the film world where 24 fps is the norm. At IBC’14, Harmonic together with Sigma Designs, was showing encoding of UHD p50 and up conversion in a Loewe Ultra HD TV set to 100 fps, with a motion compensated frame up conversion powered by Sigma Designs. Visitors from the EBU saw the demonstration and were pleased with the result. This will be one of the most contentious topics in the months to come, as the value might not be able to counterbalance the impact on the ecosystem.

Q: What about the chipset makers?

A: I visited Broadcom ViXS, STM, Sigma Designs who all had demos at different maturity levels to support different types of HDR. They are all waiting for a standard for HDR.

Q: So to wrap up can you zoom out of the details and give us the overall picture for UHD deployment?

Ultra HD is a technology that will revolutionize the world of video. Making UHD requires a complete rethinking of the workflow, from video capture, production to the presentation. This will take several years. I’m not even talking about spectrum issues to get this on the DTT network....
As you can see, the specifications are still in flux when we talk about “real Ultra HD”, the technologies are being set up and should be ready in 2016 to make live large scale interoperability testing during the Rio Olympics and also have the first services to OTT or on Blu-ray Disc that supports the HDR and WCG (Wide Color Gamut).

(Disclaimer 1: Thierry is a friend and is passionate about Ultra HD, he was invited speaker at both NAB and IBC last year on UHD, disclaimer 2: Although I have written for Harmonic in the past, I’m not under any engagement from them).

diagram UHD2

To be continued....

Posted on

Ultra HD ecosystem getting organized, alliance on the way

I attended the French HD Forum meeting on UHD last week in Paris, which hosted by Eutelsat. France prides itself on being innovative, often with government or strong regulator incentive. How this actually works out is a matter for politicians as in the case of the Minitel that predated the Internet. There is still no consensus on whether it was a good thing for France, with French people becoming used to eCommerce before the term even existed or whether on the contrary it made France miss the first Internet wave.

When it comes to TV standards similar debates rage. Much ink was spilt over the terrestrial switch-over which was completed here in 2011. The transition from SD to HD was always a political hot potato and is still underway with spectrum scarcity in the current DVB-T1 setup restricting HD to 5 of the 23 FTA channels.

Unsurprisingly, when the French get talking about UHD, there’s palpable tension with all the differing agendas. Does it make more sense to finish upgrading the end-to-end environment to HD before playing around with UHD, or on the contrary would it be more economical to avoid two upgrades and go straight to the ultimate target of full UHD? Should TV stations wait for customer demand or try to stimulate it with UHD services in as early as 2015?

Beyond these legitimate debates, there is also some confusion that is artificially created by a lack of information and sharing across the Ultra HD video ecosystem.

The risk of confusion

As UHD TV gradually tips over it’s peak of inflated expectations the TV industry at large, through the diversity of its reactions, will undoubtedly lead it down to the depths of disillusionment. Some TV stations publicly doubt if 4K will ever be a sound business proposition, while satellite operators and many technology vendors have bet their future on UHD success. Sometimes, even within the same industry group UHD is being pulled in several different directions at once as for example between the different UHD specifications of ITU, EBU, Digital Europe and CEA. Some key differences and commonalities are:

Feature

ITU EBU (phase 1) CEA Digital Europe

Resolution 

4320p 2160p 2160p 2160p
Frame rate 120/60 60 60/30/24 60/30/24
Color space BT2020 BT 709
HDMI NA 2.0 2.0
Bit depth 10/12 10 8 minimum 8 minimum
HDR Under standardization Phase 2 Not mentioned Not mentioned

The standardization of UHD has so far been much less chaotic than it was for say 3D technology at a similar stage.

Some clear standards emerging from:

  • the telecoms sector with the (ITU-R) recommendation from ITU’s Study Group 6 (more at: http://www.tvtechnology.com/news/0086/itu-issues-uhdtv-standards-recommendations-/213615#sthash.DXG9J7bU.dpuf),
  • the video technology space, which is also active with MPEG-HEVC having published a specification in January 2013 that can use used for UHD and that is now looking actively at HDR,
  • the consumer electronics industry that provided a vital part of the Ultra HD requirements with the standardization of HDMI 2.0,
  • the broadcasters, with the DVB/EBU ultra high definition broadcast format (UHD-1 Phase 1) specification for example.

But UHD’s success will rely on much more than just increased bandwidth and resolution and many of the other elements are still under discussion like for example the required increases in both color sensitivity and contrast with HDR (High Dynamic Range) or refresh rates with HFR (High Frame Rate). Norms for carrying higher definition audio with a greater number of channels have been standardized by ETSI with AC-4 that is actively promoted by Dolby. The MPEG standards body is currently in the process of creating an object based audio encoding standard with MPEG-H. The IP encapsulation techniques defined by SMPTE (2022-6) are still to be universally accepted by the industry.

To succeed faster, at a lower the cost for early adopters, UHD doesn’t need yet another body defining standards, but one that explains them, helps ensure their interoperability and promotes successful business cases.

After the failure of 3D, the industry needs to regroup around UHD to ensure its success, in a similar way the DASH Industry forum (dashif.com) has rallied all the DASH energies.

The Ultra HD ecosystem is quite complex and we provide here (courtesy of Harmonic) an end to end diagram for Ultra HD:

diagram UHDTo speed up the process of getting through the trough of disillusionment or maybe is it to cross the chasm, I learnt in Paris that a few market leading companies are in talks to set up an alliance. Its intended scope is to cover all parts of the content lifecycle from production to display, encompassing contribution, distribution, post-production and play-out. The Alliance’s stated goal will be to promote interoperable specifications, propagate effective business models, provide forecasting and share all successful application models.

The alliance would identify, describe and share specifications relevant to all parts of the distribution chain in close collaboration with standardization bodies.

Interoperability will be a key driver for all the alliance’s work, defining the system level interop points, organizing interop plug fests and publishing and promoting the results.

The Alliance would also deliver business models for both live and on-demand content, sharing any industry success stories and ensuring any mistakes are only ever made once.

An Ultra HD Alliance would promote existing industry reports but also pool real market data from its members and use projections to obtain the most accurate forecasts for critical market dynamics. The number of deployed UHD capable CPE, the readiness of live TV workflows or the extent of UHD VoD assets will be closely monitored and projected. The alliance also intends to show how UHD can be used in different application domains such as VoD, Live TV, Linear play-out, Push VoD, etc. presenting the benefits of UHD over HD with operator feedback.

To successfully promote Ultra HD, the alliance would be represented at trade shows and conferences. The alliance’s website would encourage interaction with blogging and social media. Webinars and various publications including whitepapers will also shorten UHD’s time-to-market.

The alliance would be open to companies from all parts of the ecosystem. Content providers, broadcasters, production houses, operators, playout companies, encoder vendors, audio specialists, security providers, chipset makers and UHD device manufacturers would all be able to join. Other organisations such as the HD Forum, EBU, DVB, etc. would be welcome too.

The setup of the alliance is still at the stage of informal talks, but the first formal meeting will take place at the CES in Las Vegas in January 2015.

Stay tuned for an update after the show (previous 4K blog on 7 Reasons why UHD/4K makes sense here)