This is in a way prompted by the whole digital photography thing.
RAW = digital equivalent of the old film negative. 16.8 MB
JPEG = “Lossy” 3-4 MB
“lossy” means it is a one way street, once you throw away data to go from the raw to the jpeg, there is literally no way to get that data back, it is lost, forever, and no, it is not a question of having the right software or the right amount of computing power or the right meta-data or anything else.
This is a random shot taken from my sofa of some crap lying on the floor about 8 feet away.
One – basic raw pic, 16.8 MB, white balance done, exported as a 16 bit TIFF 68.8 MB to AfterShot, no editing in aftershot, just saved as a JPEG with 90% quality. 3.46 MB… BTW all these four thumbs link to full size versions.
Two – same pic, the jpeg now, pick red as a spot colour, adjust the fuzziness to get all the reds I want, save as jpeg, 2.94 MB
Three – same pic, pick second spot colour (uncheck first but leave it unchanged) of yellow, adjust fuzzy etc, save as jpeg, 3.27 MB
Four – same pic, just check both red and yellow spot colours, save as jpeg, 3.43 MB
It is in many ways a pretty little pic, the spot colours and texture of the cord make it pretty, but note how re-enabling just the red spot colour, while leaving the yellow spot colour unchanged, also apparently alters subtly the shades of the yellow, compare the yellow in shots three and four.
But the fact is, you can’t get from the second or third or fourth pic to the first, no way, not possible.
Sure, if you are *incredibly* patient, you can colourise it by hand, and it may end up looking quite realistic, but it will not be the same… it will just be a guess.
In fact if you look at pic four, at 100% zoom, you’ll see each red spot on the cord surrounded by a bluish line where the red spot colour has mixed with the yellow spot colour.
This is not merely a case of information that has been LOST, but new information introduced THAT NEVER EXISTED IN THE ORIGINAL.
This latter point is an all too overlooked and vital part of ALL “lossy” techniques, it doesn’t *just* apply to digital pictures of some coloured cord on the floor of some random guy on the internet, it applies in every single case you use any kind of “lossy” process, as opposed to a bit perfect verbatim unedited clone copy.
Teracopy uses MD5 hashes to ensure that all copy and move processes produce bit perfect copies… for example… Windows file copy / move does not…. Windows file copy / move is much faster than Teracopy, and 99.99% of the time, it doesn’t matter.
The definition of “it doesn’t matter” is what matters here.
“Lossy” doesn’t matter (within limits) if you are just taking a picture of a small router / switch to record what patch leads go where.
“Lossy” does matter, a lot, if you are looking at an X-ray of your thorax and looking for anomalies, not only can a lossy process “lose” a genuine anomaly, the lossy process itself could “create” a non existent one, such as the blue border on the red spots of the rope.
The UK NHS uses all digital now for storage of X-rays, if you ask for a copy of your medical files you get a CD, documents are scanned as photos, no OCR, and pictures such as X-rays are stored as DICOM/HIPAX, obviously you don’t want your X-rays being stored as lossy jpegs, with quality set to some low but very common on the intertubes figure such as 65%
I always preferred FIF, or fractal image format, over jpeg or gif, because you could zoom to 400% and it still looked pretty damn good, not all minecraft / lego, but this was just a visual thing, it wasn’t actual data.
Everything from Blade Runner to CSI to everything else gets this shit wrong, zoom in, enhance, zoom in more, enhance more, they say.
Utter and total fucking bullshit, take a 200 page paperback book, tear it in half and throw the last 100 pages away, sure, you can write several credible alternative endings, but you will never ever ever get a word and layout perfect rendition of the original, no matter how much compute power you throw at the first 100 pages.
The opposite of the “lossy” are the “modelers”, they have models, they have interpolation, they have algo’s up the fucking wazzoo, tear a phone book in half and ask them to get even the next single entry correct, never mind the rest of the fucking book…
Thing is, until you actually try and look up Willy Wonka’s phone number, or dial a number at random eg 17th one down on page 247, it will look like a pretty good phone book. Add one more level of “difficulty” eg do it with a Chinese phone book, or a Berlin one to someone who cannot make international calls to Germany, and it’s indistinguishable.
If you hand me your PC for repair, the very first thing I do is make an image of the hard disk in Macrium, but here is the thing, it is (by default, I have the option to do otherwise, I just don’t, unless specifically requested to do so, or it is a server, then I do it by default) lossy, because it ignores every single sector on the hard drive which is listed as free space.
Hard drives don’t actually delete data, they just delete the index meta data, which is how data recovery works, a bit wise clone of your hard disk, we can do data recovery on, but even here, we still lose *something* in the copying, a normal “ignore free space” copy of your hard disk, we cannot do data recovery on, or, to be specific, if we do, we will recover what was written to THAT disk prior to your image being copied there, not what was on the free space on your old disk.
You see why even “police forensic” computer disk analysis is “lossy”, and indeed, unless the entire disk surface itself has been DOD wiped prior to use on that particular case, just like the earlier examples, it can introduce data that was never present in the original.
I once knew a sculptor, he described what he did as cutting away everything that was not the object inside, trying to get out, an example of a lossy technique that creates something more appealing, out of something less appealing, but, if he goes too far, tap tap, the right arm falls off, nobody in the future can say what the right hand was doing, how the hand and fingers were laid out, or were they even finished by the sculptor?
Lossy is prevalent in our own senses and brains and bodies, we simply discard most of the data coming in.
Lossy is everywhere.
That, however, is no reason to simply ignore it or discount it.
Nor even to adopt it as SOP.
