The New Test Images – Image Compression Benchmark

10 points by Zababa 10 months ago | 15 comments
  • andrewla 10 months ago
    I understand why it can be problematic to use humans in test images, but this is in conflict with the purpose of test images. The purpose, briefly, is to allow human eyes to easily spot defects in image processing. While we can use automated methods as well especially for formal analysis of fidelity, in the end the reason why we are displaying images is for human consumption.

    We don't need to to Lena or the other traditional images, but it should be possible to find a usable example -- even if it's just the grimacing guy from the meme or something.

    • refulgentis 10 months ago
      It's not that it's problematic to use humans, it's that it's problematic to use images from 1970s Playboy.

      There's a theoretical argument to make that this rando site should also have a picture of a human, but it's weak -- something something evolution over millenia, something something parsing faces, therefore we'll notice if its off quicker.

      It's a weak argument because we don't expect ex. facial symmetry to change, and faces are capable of being quite asymmetrical.

      In practice its much more valuable for the random website to have ex. the color test card, deep dark night with streetlights to stretch dynamic range, depth of field and details of fine spider web above blurry background, etc. It's fairly easy to find a picture of a human when needed, especially one that we're familiar with and see small details change :)

      • lmpdev 10 months ago
        I don’t work on image processing but do own $400 professional pencils and use an sRGB monitor and have been producing “high end hobby” level works for a couple of decades

        I would urge away from using an entire human portrait. Decontextualised elements of the human form register a lot more accurately than an entire face. I spend a great deal of time trying to escape viewing works as a face holistically as it tends (from my subjective experience) to trigger a different part of my brain than the area responsible for visual differentiation/assessment

        I would think that something similar to this would be better for researchers to quickly and accurately gauge DSP effects than a whole portrait: https://en.wikipedia.org/wiki/File:Missing_Link_by_Nick_Murp...

        Obviously much better curated than the referenced image

      • dylan604 10 months ago
        Part of the use of having a human was meant to see how well skin tones looked after being developed. The big problem was that it was only a single skin tone that was ever used.
        • Lerc 10 months ago
          That seems like a poor argument for Lina given how off the colour balance is. It is a very pink image.

          The merit comes more from the combination of sharp edges, gradients, and details that humans are sensitive to (face) and details that they are not (feathers).

          Spending bits to place an eye at just the right place is more important than spending the same bits to place a bit of fluff covering the same number of pixels.

          Something like the photo in test card F at high resolution would be good. (maybe with a fluffier toy)

      • hoten 10 months ago
        More context, here is what the old "Lena" test image is all about: https://womenlovetech.com/losing-lena-why-we-need-to-remove-... (nsfw, it's a Playboy pic)
        • Zababa 10 months ago
          A bit more context from the page:

          > The images historically used for compression research (lena, barbra, pepper etc...) have outlived their useful life and its about time they become a part of history only. They are too small, come from data sources too old and are available in only 8-bit precision.

          > These high-resolution high-precision images have been carefully selected to aid in image compression research and algorithm evaluation. These are photographic images chosen to come from a wide variety of sources and each one picked to stress different aspects of algorithms. Images are available in 8-bit, 16-bit and 16-bit linear variations, RGB and gray.

          > You are encouraged to use these images for image compression research and algorithm evaluation. Suggestions for further improvements are always welcome.

          I am currently looking at different image formats/png optimizers and trying to compare them, so having a set of images to compare is nice.

          I do feel like they're not exactly representative of what an image can be on the web or on my computer. I was thinking of adding a few things for my own use: a screenshot, an image macro, a page of manga.

          • brudgers 10 months ago
            I am not defending the use of Lena nor advocating for its continued use.

            A similarly useful replacement should probably contain similar types and degrees of detail so that image algorithms can be tested. The hair, skin, fabric etc. in Lena readily surface an algorithm's compression and convolution artifacts. The Lena image was chosen in part for its technical properties.

            Unlike the Utah Teapot also used for computer imaging, the Lena image is socially and academically inappropriate. Though suitable replacements can be found or made, most images are not suitable technical replacements. Doing so will probably be non-trivial due to the volume of existing technical decisions based on the Lena image. https://www.tandfonline.com/doi/full/10.1080/09500340.2016.1...

            • Zababa 10 months ago
              Very good point that I hadn't considered, thanks for sharing that!
          • sliq 10 months ago
            This page was last updated in 2015, 9 years ago, and ALOT has happened in image compression and on the web in these 9 years. And the samples are just so totally weird, they are not representative for anything.
            • munchler 10 months ago
              > each one picked to stress different aspects of algorithms

              Really? Photographically, these images are kinda crappy - the kind of casual snapshot one might take and then delete. The cathedral image, for example, is both underexposed and overexposed. I guess this isn't too surprising, since it was taken on a Nikon D70 from 2004, which had pretty limited dynamic range. This hardly seems like a good example to choose for testing image compression, since it lacks a lot of useful detail. Or maybe this was a deliberate choice? What were the actual criteria used to choose these images?

              • jjmarr 10 months ago
                My first question was "what should I be looking for when this image is compressed?" A good compression algorithm should preserve important qualities of the image, but I can't evaluate that without knowing the qualities preserved.
                • 1970-01-01 10 months ago
                  I totally agree. None of them show passionate photography, and I hope that's not intentional, as photography is an art. We should at minimum benchmark photography that is stimulating for a human, otherwise what is the point to any of this?
                • entropie 10 months ago
                  is nightshot_iso_1600 and nightshot_iso_100 interchanged? that looks not right to me. Iso 100 looks way brighter.
                  • throwaway7679 10 months ago
                    (2008)
                    • aaron695 10 months ago
                      [dead]