Vips – 8.0: Using > Processing UltraHDR images
Description: Reference for Vips-8.0: Using > Processing UltraHDR images

#### Using > Processing UltraHDR images [src]

libvips can process HDR images encoded with UltraHDR. These are ordinary SDR images, but with a gainmap embedded within them — SDR displays will just show the regular SDR image, but for an HDR display, the extra gainmap can be used as an exponent to recover the full HDR range. This ability to show the same file in good quality on both SDR and HDR displays makes the format very useful.

libvips uses Google’s libultrahdr for UltraHDR load and save. The current version of this library only supports UltraHDR JPEG images; the next version is expected to add support for a wider range of image formats.

There are two main paths for UltraHDR images in libvips: as an SDR image with a separate gainmap, and as a full HDR image. The separate gainmap path is relatively fast but you will sometimes need to update the gainmap during processing. The full HDR path does not require gainmap updates, but can be slower, and will usually lose the original image’s tone mapping.

## UltraHDR as SDR with a separate gainmap

libvips will detect JPEG images with an embedded gainmap and automatically invoke the `vips_uhdrload()` operation to load them. This operation attaches the gainmap (a small JPEG-compressed image) as the `"gainmap-data"` metadata item, plus some other gainmap tags.

### Load and save

For example:

    $ vipsheader -a ultra-hdr.jpg
    ultra-hdr.jpg: 3840x2160 uchar, 3 bands, srgb, uhdrload
    width: 3840
    height: 2160
    bands: 3
    format: uchar
    coding: none
    interpretation: srgb
    xoffset: 0
    yoffset: 0
    xres: 1
    yres: 1
    filename: ultra-hdr.jpg
    vips-loader: uhdrload
    icc-profile-data: 588 bytes of binary data
    gainmap-data: 31738 bytes of binary data
    gainmap-max-content-boost: 100 100 100
    gainmap-min-content-boost: 1 1 1
    gainmap-gamma: 1 1 1
    gainmap-offset-sdr: 0 0 0
    gainmap-offset-hdr: 0 0 0
    gainmap-hdr-capacity-min: 1
    gainmap-hdr-capacity-max: 100
    gainmap-use-base-cg: 1

This gainmap metadata is copied unmodified through any processing operations. If you save an image with gainmap metadata to a JPEG file, libvips will do the write with the `vips_uhdrsave()` operation, embedding the gainmap and the associated metadata in the output image.

Intermediate operations which change the image geometry will also need to update the `"gainmap-data"` metadata item, the mechanisms for doing this are described below. The other gainmap fields should probably not be changed unless the intention is to alter the image appearance.

### High-level libvips operations

Two high-level libvips operations will automatically update the gainmap for you during processing: `vips_dzsave()` and `vips_thumbnail()`.

`vips_dzsave()` always strips all metadata by default, so you’ll need to set `keep="gainmap"` to write the gainmap to the tiles. For example:

    $ vips dzsave ultra-hdr.jpg x --keep gainmap

### À la carte processing

Other operations will NOT automatically update the gainmap for you. If you call something like `vips_crop()`, an operation which changes the image geometry, the gainmap and the image will no longer match. When you save the cropped image, the gainmap is likely to be incorrect.

A helper function, `vips_image_get_gainmap()`, makes updating the gainmap relatively easy: it returns a `VipsImage` for the gainmap, and attaches the image pointer as the metadata item `"gainmap"`. Once you have updated the gainmap, you can overwrite this value.

For example, in C you could write

    VipsImage *image = ...;
    VipsImage *out;
    int left, top, width, height;
    if (vips_crop(image, &out, left, top, width, height, NULL))
        return -1;

    // also crop the gainmap, if there is one
    VipsImage *gainmap;
    if ((gainmap = vips_image_get_gainmap(out))) {
        // the gainmap can be smaller than the image, we must scale the
        // crop area
        double hscale = (double) gainmap->Xsize / image->Xsize;
        double vscale = (double) gainmap->Ysize / image->Ysize;

        VipsImage *new_gainmap;
        if (vips_crop(gainmap, &new_gainmap, left * hscale, top * vscale,
            width * hscale, height * vscale, NULL))
            return -1;
        g_object_unref(gainmap);

        // vips_image_set_image() modifies the image, so we need to make a
        // unique copy ... you can skip this step if you know your image is
        // already unique
        VipsImage *new_out;
        if (vips_copy(out, &new_out, NULL)) {
            g_object_unref(new_gainmap);
            return -1;
        }
        g_object_unref(out);
        out = new_out;

        // update the gainmap
        vips_image_set_image(out, "gainmap", new_gainmap);

        g_object_unref(new_gainmap);
    }

Or with ruby-vips:

    def crop_image_and_gainmap(image, left, top, width, height)
      image = image.crop left, top, width, height
      gainmap = image.get_gainmap
      unless gainmap.nil?
        hscale = gainmap.width / image.width
        vscale = gainmap.height / image.height
        new_gainmap = gainmap.crop left * hscale, top * vscale, width * hscale, height * vscale
        image = image.mutate do |mutable|
          mutable.set_type! Vips::IMAGE_TYPE, "gainmap", new_gainmap
        end
      end

      image
    end

### Performance and quality considerations

Doing gainmap processing explicitly like this has two big advantages: first, you have exact control over this processing, so you can make sure only the gainmap transformations that are strictly necessary take place. Secondly, since you supply the gainmap to the UltraHDR save, you can also be certain any user tone mapping is preserved.

The disadvantage is the extra development work necessary. The second UltraHDR path in libvips avoids this problem.

## Full HDR processing

You can transform an UltraHDR SDR plus gainmap image to full HDR with `vips_uhdr2scRGB()`. This will compute an scRGB image: a three-band float with sRGB primaries, black to white as linear 0-1, and out of range values used to represent HDR.

For example:

    $ vips uhdr2scRGB ultra-hdr.jpg x.v
    $ vipsheader x.v
    x.v: 3840x2160 float, 3 bands, scrgb, uhdrload
    $ vips max x.v
    15.210938

If you save an scRGB image as JPEG, it will be automatically written as UltraJPEG. Any associated gainmap is thrown away and basic tonemapping performed to make a new gainmap for SDR display.

Full HDR processing with scRGB is simple, but potentially slower than the separate gainmap path, and will not preserve any user tone map.