CPNG, a backwards compatible fork of PNG
237 points by dirkf 1 year ago | 120 comments- solardev 1 year agoWhat happens if this actually picks up steam, and suddenly PNG is no longer one format, but a bunch of incompatible ones that look somewhat similar, whose fidelity depends on your renderer?
Early in PNG's history, we already had this issue with alpha channels, progressive rendering, and hit-or-miss support for APNG (animated PNGs, meant to replace GIFs but never happened).
It was also an issue for a long time for PSDs and SVGs, where the same file never looked the same on two browsers/devices/apps/versions.
I would bet that these days, generating or decoding PNGs is the bottleneck almost nowhere, but extending the format would cause problems everywhere in real-world usage. Apps and companies can no longer tell whether there's something wrong with their image or if somewhere in the pipeline, some new graphics designer decided to use a bleeding-edge version of a 30-year-old graphics format that nobody else accounted for, and it looks "broken" in half the browsers now. A format can still look broken even if it's "backward compatible", just by virtue of having some features (like HDR) that are only displayable in some renderers but not others.
Why not just make a new format instead and have browsers & devices fall back as necessary, like we already do with webp and srcsets?
- hinkley 1 year agoPNG was always set up for extension. In particular, it has a clever way of allowing ancillary data sections to be marked as unimportant, so a decoder knows whether it can skip them or report the file as unreadable.
I suspect the big thing these days would be to support brotli and zstd.
- crq-yml 1 year agoA problem that often comes up with extensible formats is that whomever comes along and implements them assumes exactly the test cases they came up with, which can often mean "just the files I have for this project" or "just the output of the encoder I have".
So there will be formats that can reorder the chunks, and those minimum-viable readers will all break when they encounter a file from a different source, because they hardcoded a read order. This leads to an experience on the user's end where they need a "fixer" tool to reorder it to deal with the bad decoder.
There were tons of old binary formats that were like this. It can happen with text, but it's less likely to, because a larger proportion of textual formats build over a container like XML or JSON to offload the text parsing, and then they end up with some pre-structured data.
- chriswarbo 1 year ago> There were tons of old binary formats that were like this. It can happen with text, but it's less likely to, because a larger proportion of textual formats build over a container like XML or JSON to offload the text parsing, and then they end up with some pre-structured data.
Note that PNG also "build[s] over a container", since it's a descendant of IFF.
- chriswarbo 1 year ago
- solardev 1 year agoMany formats have stuff like that (like cover art in MP3 ID3 tags), but usually they're used for, well, ancillary purposes.
It's dangerous to use this to change the actual primary output of the file (the image), especially in a way that users and editors can't easily detect.
- sspiff 1 year agoI would say at least in the context of extra data to extend the bit depth for HDR, that data could be considered ancillary?
We've been rendering images in SDR forever, and most people don't have HDR capable hardware or software yet, so I don't know how you could consider it as broken to render the image without the HDR data?
- sspiff 1 year ago
- Retr0id 1 year agoYou could add an "ignorable" zstd-compressed IDAT variant, but that wouldn't give you backwards-compat in any useful way - the zlib-compressed IDAT still has to be present, and unless you fill it full of zeroes or a placeholder image, the overall file is going to be larger.
- crq-yml 1 year ago
- mmastrac 1 year agoAPNG is extremely well-supported these days: https://caniuse.com/apng
Fun fact: APNG is better supported than JSON.stringify
- Andrex 1 year agoAnd yet I have yet to see one in the wild. It's all WebM and Gifs these days.
- keep_reading 1 year agoIf you've seen a video slot machine you've seen APNGs :)
- keep_reading 1 year ago
- Andrex 1 year ago
- smcameron 1 year ago> I would bet that these days, generating or decoding PNGs is the bottleneck almost nowhere
The bulk of my game's startup time is spent decoding PNGs via libpng. There are some alternatives to libpng like fpng[1], or alternate image formats like QOI[2]
These both exist because png is slow and/or complicated.
[1] https://github.com/richgel999/fpng [2] https://github.com/phoboslab/qoi (discussed here: https://news.ycombinator.com/item?id=29661498)
- doophus 1 year agoMost games ship textures in a GPU-friendly format such as DXT to avoid problems like this.
- doophus 1 year ago
- gred 1 year ago> these days, generating or decoding PNGs is the bottleneck almost nowhere
Anecdotal, but I'm familiar with a system which spends ~50% of CPU cycles on PNG encoding (most of which is actually spent in zlib compression).
The other approaches I've seen involve creating performance-focused forks of zlib (e.g. zlib-chromium and zlib-cloudflare). This has benefits beyond PNG encode / decode:
https://aws.amazon.com/blogs/opensource/improving-zlib-cloud...
- Nyan 1 year agoWhy not use fpnge?
- gred 1 year agoIt's a Java system, so not quite so simple. Maybe it's worthwhile to create some Java bindings? Recent JDKs make it feasible to swap out the underlying zlib implementation, so swapping out zlib-madler with zlib-cloudflare or zlib-ng might provide the best cost/benefit.
- gred 1 year ago
- Nyan 1 year ago
- chaxor 1 year agoMaybe this is the genius in it. In order to get everyone to move to better formats, you just break everything in the one everyone uses, so they have to move.
Like Twitter becoming X to push everyone to mastodon. Mastodon is better in every way, so it's a net win.
- sp332 1 year agoBut most people didn’t move to Mastodon, and a bunch moved to incompatible networks like Bluesky, cohost, etc. Which is really the kind of problem you don’t want to have when you're just posting a photo.
- Sardtok 1 year agoPlease don't use PNG for photos.
- Sardtok 1 year ago
- cortesoft 1 year ago> Mastodon is better in every way
We might wish this to be true, but it isn't. There are more people on Twitter/X, which is the most important part of a social network.
- Fauntleroy 1 year agoLess people is actually a very, very good thing in my mind. Twitter is a great example of what happens when too many people are "talking" at once.
- Fauntleroy 1 year ago
- tetris11 1 year agoI think idiocy and short-term self-interest is more at play here, than any 4D chess shenanigans.
- bawolff 1 year agoEmbrace, extend, extinguish. Not just for microsoft anymore!
- bsder 1 year ago> Mastodon is better in every way, so it's a net win.
Making easily disprovable statements is not the way to win people to your side.
Searchability is weaker. Login is all over the map. Links don't work. etc.
These are the standard problems with non-centralized software.
I really don't understand why Mastodon didn't use/create something like the "magnet" links used for torrents. That way, even if you lost the server, as long as you touched something that had the appropriate hashes, you can access the information.
I use Mastodon, but it is not better in every way.
- sp332 1 year ago
- skybrian 1 year agoIt seems like it’s not going to look broken? Unlike, say, the difference between black & white and color TV, or an animated image that doesn’t animate, it will be a subtle difference that most users won’t notice or care about. Some designers may be annoyed, but it doesn’t seem like that big a deal.
- pwdisswordfishc 1 year agoThough doesn't that mean the feature is less likely to be implemented in the first place?
Nobody gave a shit about Unicode grapheme clusters until EEMAWDJI came about. Sadly.
- skybrian 1 year agoWhat's "EEMAWDJI?" There are no Google search results for that acronym.
- skybrian 1 year ago
- pwdisswordfishc 1 year ago
- zimbatm 1 year agoHe said that it would be backwards-compatible. It's in the name of the project.
- solardev 1 year agoSorry, but did you read my post? It's only backward-compatible in the sense that existing renderers can display SOMETHING -- but it's not the same image.
From the article:
> [...] like how color TV was introduced but it still worked on B&W TV's. New features can be added, as long as existing decoders ignore them and return a viewable image
Keyword "ignore them". To my reading, this means that CPNGs will contain several images inside them: A full-featured version with "color" (or HDR, or or whatever) for newer renderers, and a fallback one (in "black and white" in his example) for existing renderers.
It's not really "backward compatible", but more like "fallback-able".
- edflsafoiewq 1 year agoThe first two extensions (constrained deflate, multithreaded encoding/decoding) only present optimization opportunities for the decoder. It is still the same image.
Single-threaded decoding of large PNGs is enough of a bottleneck that Apple has their own version of PNG with multithreading support.
- brookst 1 year ago> CPNGs will contain several images inside them
That was not my read. When discussing HDR it’s clear that ONLY the extra bits are stored in a new data structure and are applied to the base image at display time.
So that gets you a bit-for-bit compatible image on old systems and HDR on new systems without duplicating data.
I believe that pattern applies throughout: a baseline, standard PNG image plus extra instructions and data for adding information to it.
- edflsafoiewq 1 year ago
- solardev 1 year ago
- pwdisswordfishc 1 year ago> Early in PNG's history, we already had this issue with alpha channels, progressive rendering, and hit-or-miss support for APNG (animated PNGs, meant to replace GIFs but never happened).
Don't forget pixel aspect ratio. Oh wait, most viewers still ignore that.
- hinkley 1 year ago
- ebb_earl_co 1 year ago> Why continue messing with PNG at all? Because if you can figure out how to squeeze new features into PNG in a backwards compatible way, it's instantly compatible with all browsers, OS's, engines, etc. This is valuable
What a brilliant paragraph. I wish this developer all the success in the world.
- unconed 1 year agoThe backwards compatibility is okay, but the author should also plan a more optimal replacement encoding that ditches the legacy compatibility, and require new implementations to support both.
Otherwise there is no way to sunset the hacks.
- unconed 1 year ago
- Voultapher 1 year agoThat's really cool! I stumbled across libpng being 10+x slower to encode than jpg and tiff at work. The LOGLUV32 part is very clever. I particularly like the tonemapped fallback and general idea to build on top instead of reinventing. That said I hope these format extensions don't end up in compatibility hell, where viewing the full info image is hit or miss between different CPNG decoders.
- gumby 1 year agoI loved reading this even though I personally have zero need myself. I enjoyed the rationale and he engineering.
The world needs more work like this. I’m talking about the thoughtful image format but also that applies to the write up too.
- fbdab103 1 year agoWhat is the modern power ranking on image formats?
For lossless, what is typically the most efficient size wise? Decompression speed?
For lossy?
I am not in a situation where these micro-optimizations mean much to me, and always default to png, but curious to know where the state of the art is today.
- solardev 1 year agoAVIF and WebP, two modern replacements for both JPEG and GIF on the web, both support lossy and lossless encoding.
WebP is mature and has more browser support, but AVIF is getting there (notably only lacking Edge support). Both can compress in either a lossy JPEG-like fashion or in a lossless PNG-like fashion.
If you use a image CDN like Imgix, it'll just auto-detect and serve whatever the most optimal format is anyway: https://docs.imgix.com/apis/rendering/auto/auto#format. Cloudinary too: https://cloudinary.com/documentation/image_optimization#how_...
For non-web, there's also JPEG XL to look at, but if you're not doing rendering an image for redistribution, it's probably better to keep it as raw as possible anyway (i.e. camera raw images plus photoshop layers, or whatever).
- ComputerGuru 1 year agoWebP and AVIIF (and, to a much lesser extent, HEIC, which AVIF is basically a rip off of) absolutely suck for color management since they are a) virtually never original source formats, b) are video codecs. WebP technically supports two different color profile techniques (traditional embedded ICC - broken in every mainstream batch image processor I’ve tried - and nlx video-based color profiles). Unlike WebP and all the other image formats, untagged AVIF can’t be assumed to be sRGB (in part because there is no actual sRGB for video, though close variants exist) and every image processor or image editor will open it with a different base color profile assigned. WebP doesn’t even support exif, making it absolutely horrible for “lossless” operations that effectively aren’t lossless since they necessarily destroy metadata.
HEIC is also a video codec at heart but has a default color space that also isn’t sRGB (which is a good thing; it’s about time we moved on), untagged HEIC images can (though often aren’t in any default workflow) be assigned Display P3. Assigning/assuming sRGB will absolutely break your images, of course.
- eyegor 1 year agoThe worst part about avif support in edge is it was added as an optional feature flag ~8 months ago but it still isn't enabled as a default. Nearly every other browser supports avif by default these days.
https://winaero.com/avif-support-is-now-available-in-microso...
- ComputerGuru 1 year ago
- ksec 1 year ago>What is the modern power ranking on image formats?
I will assume this can be outside the web and consider only image format / codec that is state of the art.
>For lossless, what is typically the most efficient size wise? Decompression speed?
In terms of lossless, JPEG-XL is the one to look at. Both in terms of size and decompression speed. You will already see communities from professional photographers using this format.
>For lossy?
That depends on what sort of quality you are looking for. In some edge cases you could have JPEG XL being better at ultra low bit per pixel, like 0.1 bpp. Otherwise in 95% of cases, at 0.1 bpp to 0.5 bpp, it is HEIF / AVIF. HEIF based on VVC / H.266 Encoder likely to be the state of the art. With current reference FVC / H.267 implementation bringing another 10 - 30% improvement.
However the higher the quality you go, i.e 0.8 to 1.5 bpp, the more favourable to JPEG XL.
- jjcm 1 year agoAs others have mentioned, the pool is webp, avif, and jpeg-xl.
If you’re building something today, webp likely has the best tradeoff of encoding speed, efficiency, and compatibility.
For pure tiering, here’s how I’d rank them:
*Efficiency:*
Webp: B tier
AVIF: A tier
JXL: S tier
*Encoding Speed:*
Webp: B tier
AVIF: D tier
JXL: A tier
*Compatibility*
Webp: A tier
AVIF: B tier
JXL: D tier
- eyegor 1 year agoWhat about mozjpeg?
- eyegor 1 year ago
- solardev 1 year ago
- summerlight 1 year agoWhat does it exactly mean by "100% backward compatible"? It looks like some optimizations could be backported to the existing encoder/decoder without breaking the format but this is more of an optimization. My impression is that this is backward compatible in a way similar to APNG (it will return some reasonable images if the file is using a new functionality), but I'm not sure if I understand it correctly.
- Retr0id 1 year agoI've also been pondering a backwards-compatible fork of PNG - but rather than a fork, mine would be a subset. Specifically, it would be an as-simple-as-possible uncompressed* bitmap format, suitable for use in low-complexity embedded systems etc. (e.g. bootloaders, wink wink). By being backwards compatible, you get the added benefit of retaining compatibility with existing image viewers, but without having to implement "all of PNG" in decoders and encoders. Now, the base PNG spec isn't even that big, but the more you constrain the spec, the easier it is to be sure you've implemented it securely.
* If you're wondering how that works in a backwards-compatible way, DEFLATE already supports uncompressed blocks.
- pwg 1 year ago> I've also been pondering a backwards-compatible fork of PNG - but rather than a fork, mine would be a subset. Specifically, it would be an as-simple-as-possible uncompressed* bitmap format, suitable for use in low-complexity embedded systems etc. (e.g. bootloaders, wink wink).
Look at the NetPBM formats (PPM, PGM, PGM). They are about as simple as they can possibly get (a tiny, ASCII, header, followed by binary bitmap data), and are also uncompressed.
- Retr0id 1 year agoThey're simple, but they're nowhere near as widely supported as PNG (or BMP)
- Retr0id 1 year ago
- colejohnson66 1 year agoBMP already exists for uncompressed bitmap data.
- Retr0id 1 year agoThis is true, but BMP became a bit of a kitchen-sink format, supporting all sorts of pixel formats, and optionally, compression.
i.e. you'd still want to pick a subset to implement. To be honest, you're probably right - BMP would be more sensible starting point, but I'm interested to see how far PNG can be pushed.
- Retr0id 1 year ago
- jezek2 1 year agoYou may want to look at my PNG/DEFLATE implementation:
http://public-domain.advel.cz/
It contains various implementations of the compression, from simple uncompresssed to more complex variants and a quite small PNG loader. There is also a minimalistic PNG writer with uncompressed data.
- ack_complete 1 year agoI would think that something like a bootloader would have fixed input and be able to rely on that, with signing if necessary, rather than the robustness of the decoder. Otherwise, someone who could replace the image would probably be able to replace the code as well.
Without Deflate compression, PNG would have no compression at all, as the predictor mechanism gives no savings on its own. TARGA with RLE would be a better choice than PNG-0.
- Retr0id 1 year agoI thought that too, but then the "LogoFAIL" vulnerabilities came out.[1]
It's possible to define a subset of the deflate bitstream that only does simple RLE compression, and I've done so: https://gist.github.com/DavidBuchanan314/b223bce114ec715a66f...
But, for logo/icon sized bitmaps, you can generally get away without needing compression at all. A 128x128 1bpc image is a mere 2KB.
[1] discussed here https://news.ycombinator.com/item?id=38515571
- Retr0id 1 year ago
- doubloon 1 year agook thats pretty wild, you would take the zlib deflate/inflate code, (for example in a library like lodepng) and then like chunk 95% of it in the garbage? so basically every block would just be uncompressed? kind of funny but it would probably work pretty well and your code size could get down way way smaller than the current typical png code.
seems like the downside is that this is "worse than nothing" compression, the image file would be bigger than the original blit of the data. for example 1024x1024x32bit color means 3 megabytes for one image. or do i miss something?
- RaisingSpear 1 year agoMany PNG compressors allow you to specify the zlib compression level, where 0 = no compression. This will effectively give you an uncompressed image, perhaps with some format overhead.
Your math is a bit off - a 1024x1024 at 32bpp would be 4MB, ignoring overhead.
I've actually done something like this in the past - create PNGs with 0 compression, then compress it with something stronger than Deflate (like LZMA). Because the PNG filtering is still applied, this allows greater compression relative to PNG by itself (or a BMP compressed with LZMA).
- doubloon 1 year agoright but the "compression/decompression" code essentially becomes, like what, 10 lines of C? down from several thousand
- doubloon 1 year ago
- Retr0id 1 year agoThere's no need to take anyone else's code, emitting uncompressed DEFLATE blocks is trivial. I'm not sure what you mean by garbage?
> for example 1024x1024x32bit color means 3 megabytes for one image.
You do miss something, that's 4 megabytes, plus any header/format overhead - but you'd get similar performance out of any uncompressed format, that's just the tradeoff.
- doubloon 1 year agoi mean the PNG code, youd have to have some PNG code in addition to the DEFLATE code yes?
- doubloon 1 year ago
- RaisingSpear 1 year ago
- pwg 1 year ago
- tedunangst 1 year agoNo mention of JPEG XT or JPEG-HDR?
- notfed 1 year agoThis sounds pretty amazing.
For CPNG-aware libraries, the performance improvements sound impressive.
For old (CPNG-unaware) libraries: should I expect any performance drop reading a CPNG image compared to if the image had remained PNG? Similarly, how much larger will a CPNG be than a PNG?
- TheFuzzball 1 year agoMaybe next we'll get eXtensible Compatible Network Graphics: XCPNG
- mkl 1 year agoPNG is already extensible; that's part of how CPNG is working.
- TheFuzzball 1 year ago's a joke
- TheFuzzball 1 year ago
- mkl 1 year ago
- brookst 1 year agoVery cool, and I hope it sees adoption.
Also speaks to either wise or lucky design of PNG itself, that it can support these kinds of backwards-compatible extensions.
- lifthrasiir 1 year agoPNG had more focus on backward and forward compatibility, but the fact that PNG can be "extended" in this way is not that unusual for file formats composed of multiple sections (chunks in PNG). Especially considering that other aspects of PNG effectively failed, for example it is technically possible to add new compression method or color type to IHDR, but that would give you a file completely unreadable by existing decoders. CPNG essentially works by reinterpreting PNG in a different way when certain conditions are met.
- lifthrasiir 1 year ago
- ericskiff 1 year agoThis is wonderful. What a great way to continue innovation without facing the adoption hurdles of a new format
- jancsika 1 year agoHow different can the fallback be?
Could you do an image of SBF that falls back to an image of Madoff?
- ComputerGuru 1 year agoThere’s no mention of what effect these changes have on file size. It seems to me all the non-HDR changes will blow up file sizes for all but the largest of images.
- snshn 1 year agoIf APNG couldn't pick up steam and get widespread adoption, not sure how this will. But hopefully I'm wrong.
- jeroenhd 1 year agoAPNG is supported in every browser and all the video encoding tools I've used. It's not used all that often, but support for it is built into many software libraries.
- jeroenhd 1 year ago
- vzaliva 1 year agoHow this compares to WebP?
- kibwen 1 year agoI didn't even realize that WebP had an optional lossless compression mode.
- Dwedit 1 year agoWebP's lossless compression mode usually beats PNG by a lot, and even decompresses faster. I consider lossless WebP to completely obsolete PNG. Lossless JXL often beats WebP in compression, but loses in decompression time.
Except for indexed color images. PNG beats WebP on those images. Meanwhile JXL beats PNG on indexed color images.
- Dwedit 1 year ago
- kibwen 1 year ago
- jbverschoor 1 year agoWhy not just invest in jxl
- Exoristos 1 year agoSeeping?
- topsycatt 1 year agoIf only it was called GNP...
- phront 1 year agomeet a new bunch of security holes
- KingLancelot 1 year ago[dead]