Learn More: https://www.dkriesel.com/start?do=search&id=en%3Aperson&q=Xe...

Brief: Xerox machines used template matching to recycle the scanned images of individual digits that recur in the document. In 2013, Kriesel discovered this procedure was faulty.

Rationale: This method can create smaller PDFs, advantageous for customers that scan and archive numerical documents.

Prior art: https://link.springer.com/chapter/10.1007/3-540-19036-8_22

Tech Problem: Xerox's template matching procedure was not reliable, sometimes "papering over" a digit with the wrong digit!

PR Problem: Xerox press releases initially claimed this issue did not happen in the factory default mode. Kriesel demonstrated this was not true, by replicating the issue in all of the factory default compression modes including the "normal" mode. He gave a 2015 FrOSCon talk, "Lies, damned lies and scans".

Interesting work!

Many core codecs are pretty good at adhering to reference implementations, but are still open to similar issues so may not be bit exact.

With a DCT or wavelet transform, quantisation, chroma subsampling, entropy coding, motion prediction and the suite of other techniques that go into modern media squishing it’s possible to mostly reason about what type of error will come out the other end of the system for a yet to be seen input.

When that system is replaced by a non-linear box of mystery, this ability is lost.

It is relevant only when you assume that lossy compression has no way to control or even know of such critical changes. In reality most lossy compression algorithms use a rate-distortion optimization, which is only possible when you have some idea about "distortion" in the first place. Given that the error rarely occurred in higher dpis, its cause should have been either a miscalculation of distortion or a misconfiguration of distortion thresholds for patching.

In any case, a correct implementation should be able to do the correct thing. It would have been much problematic if similar cases were repeated, since it would mean that it is much harder to write a correct implementation than expected, but that didn't happen.

> Majority of traditional compressions would make text unreadable when compression is too high or the source material is too low-resolution. They don't substitute one number for another in an "unambiguous" way (i.e. it clearly shows a wrong number instead of just a blurry blob that could be both).

Traditional compressions simply didn't have much computational power to do so. The "blurry blob" is something with lower-frequency components only by definition, and you have only a small number of them, so they were easier to preserve even with limited resources. But if you have and recognize a similar enough pattern, it should be exploited for further compression. Motion compensation in video codecs were already doing a similar thing, and either a filtering or intelligent quantization that preserves higher-frequency components would be able to do so too.

----

> 2. "Lower DPI" is extremely common if your definition for that is 300dpi. At my company, all the text document are scanned at 200dpi by default. And 150dpi or even lower is perfectly readable if you don't use ridiculous compression ratios.

I admit I have generalized too much, but the choice of scan resolution is highly specific to contents, font sizes and even writing systems. If you and your company can cope with lower DPIs, that's good for you, but I believe 300 dpi is indeed the safe minimum.

The cartoons & illustrations lose all of their gradations in feeling & tone with every outline a harsh edge. The landscapes lose any sense of lushness and atmosphere, instead taking a high-clarity HDR look. Faces have blemishes inserted the original actor never had. Fruit is replaced with wax imitation.

As an artist, I would never run any of my art through anything like this.

https://replicate.com/philz1337x/clarity-upscalerhttps://replicate.com/philz1337x/multidiffusion-upscaler

Could you explain what you mean by "not safe for archiving"? The standard is published and there are multiple implementations, some of which are open-source. There is no danger of it being a proprietary format with no publicly available specification.