Remembrance Of Things—Part 2

By Ctein

(When we last left our hero, he had a mere 45 GB of photographs to sort through. How did he cope?! We now return to the story in progress...)

There were a whole bunch of obvious duplicates I could weed out. Tee's work habit was to stock iPhoto libraries with all the original camera files in subfolders named "Originals." Those that interested her got copied into the parent folder of "Originals." I used Bridge light table windows (screenshot below) to compare the contents of each dated folder with its Originals sub-folder. Why Bridge? Because it happened to come with my copy of Photoshop. Really, any image viewing program that lets you display large numbers of photographs on one screen with associated metadata, lets you sort them by different criteria, and lets you select ones to move or delete ought to work just as well.

This is where it occasionally got tricky. Most of the time files of the same name in those two windows would be the same photograph, but they'd have slightly different file sizes. Tee's workflow involved re-saving the photographs rather than merely duplicating them. Doing some careful, direct pixel-by-pixel comparisons in Photoshop, I found only the most trivial differences, so I kept the larger version. Occasionally I would find files of the same name where the "copy" was markedly larger. This was a guarantee that Tee had made some sort of alteration. I always preserved both versions of those photos.

This sounds a lot worse than it really was. It was entirely visual work; I'd queue up some jazz, put on some headphones, and zone out to the music while I mowed down duplicates. After a week or so, I'd made my way through all the library folders, and Tee's hard drive had shrunk by another 8–9 GB. Now came the hard work.

Half of the photos resided various project and works-in-progress folders; they usually duplicated or were derived from the iPhoto files. Fortunately Tee often carried the camera's image number through the naming process. I could use Mac OS's Spotlight utility to search on a string like "0194" and scan the list to match photographs across folders. Some projects derived from a couple of specific days' photographs. Once I'd paired up two folders this way, I'd usually find they had a whole bunch of photographs in common. (I also wrote down these folder connections for future scholars.) Others derived from years of photographs and I had to do wild-card searches on frame numbers for most of the individual photographs to find their sources.

Derived photographs could be very subtly different. Tee reworked images in stages, and I needed to preserve all the intermediates. I wrote a time-saving Action in Photoshop to compare photographs. It pasted one photograph as a Difference layer over a second photograph and added a curves adjustment layer that increased the contrast four-fold (below). If there were any differences, they jumped right out.

Once, there were two photographs that had different version letters, but I honestly couldn't see any difference between them. The Action showed that Tee had removed four birthmarks from the subject—not exactly obvious!

I'm pretty well convinced I didn't miss more than 1% of the duplicates and didn't mistakenly delete even a handful of unique images. Now the hard drive was a svelte 30 GB. After I wrote "readme" RTF's for the files in the new archive, explaining to future scholars what I had done and how I organize all the files, I wrote two copies of the archives back onto the hard drive that had been provided. Then I burnt four sets of DVD's on Kodak Preservation DVD blanks. Nothing is certain, but these are as good as you can get. One set stayed with me, one with the executor of the state, a backup copy went to my friend Laurie Edison's home, and the "official" copy to the University.

A few files got munched when I burnt my DVD's, probably 1 in 10,000. Bless Photoshop's Contact Sheet II function for catching them for me. I created an Errata folder, containing fresh copies of the munched photos for inclusion in a catalog CD. I also copied my "readme's" and my folder cross-reference notes to that CD. The bulk of the CD was a set of about high-resolution JPEG files—the full set of of contact sheets generated from the DVD's of Tee's work. 12 images across, 12 images down, and about 130 sheets in all, as illustrated last time. The sum total of an artist's digital life. In two handfuls of gold disks. A strange thing to contemplate.

And equally strange—viewing all those photographs in chronological sequence, over and over, was a little like viewing a stop-motion encapsulation of someone's life. By no means the sum total of those years of living. Still, an important subset of them, sitting on my hard drive, presenting itself on my screen as I scrolled through page after page of images, finally arriving at the very last folder, shown above. Dated 7/27/06, it contained mostly photos of Tee. More self portraits? I have no idea. Either way, they're the last photos in her files.

Tee's life ended exactly one month later.

____________________

Ctein

Comments

[stephen connor]

Good friends are hard to find. Evidently, Tee got lucky.

[Kenneth Sky]

So how did you deal with an image like 6147 that seems so obviously out of place with the others around it?

[Andrew Gilchrist]

Not sure whether Tee's workflow would have hampered this, but often the original filename (from camera) remains in the image's metadata. Perhaps some scripting with Exiftool might group together images that had the same origin.

[dyathink]

Hi Ctein:

As i am right now, as we speak, cleaning out my own digital closets following a very rare Mac crash (read warning) the other night *sigh*..i read your entire post, start to finish, to pick up how-to hints. Most of this was pretty dry stuff though you made me chuckle a time or two. Then i got to the end. The chilling end. The final photos, self portraits, and her death a month later. I wasn't prepared for this. I thought it was just a task you undertook and completed. Job well done and all that. But it became more than that to you. An existential journey through this artist's personal creative moments, all frozen as is on her hard drive the day she departed this world. Like the dirty dishes in the sink, the car keys in their basket on the counter next to the eyeglasses, the dog's leash hanging on the hook by the door, this morning's newspaper still spread open on the dining room table, pillow still dented from the head that laid on it the night before. For all photographers whose digital files encompass the heart and soul of their creative energies, this was a chilling post to read. My hard drive will probably find itself parked by the curb waiting for the garbage pick-up the day after i'm gone. Tee was certainly fortunate that her work lived on after she passed away and that her digital files were in hands as trustworthy as yours.

[Martin Doonan]

I have a question for Ctein following on from this effort: is there advice you'd give the rest of us on annotating our filing systems?

Assuming we'd wish others to keep our work after we're gone, is there some way to let others know what's there that would be helpful to them?

dyathink's comments make me wonder how many large archives are disappearing because those coming after us don't understand them.

[Ctein]

Dear Kenneth,

I didn't have to! My job wasn't to evaluate the quality of the images but merely to preserve them in a form that made it easiest for future scholars to work with them. I just left that file where it was, just as I did with frame number 6138. I'm pretty sure that one was just an accidental firing of the shutter, but wasn't my place to decide that, and storage is cheap.

~ pax \ Ctein
[ please excuse any word salad. MacSpeech in training! ]
======================================
-- Ctein's online Gallery http://ctein.com
-- Digital restorations http://photo-repair.com
======================================

[Ctein]

Dear Andrew,

That would be a really good idea if I'm saddled with a task like this in the future. I've only had this new machine for half a year, though, so I'm really not up to speed on scripting under MacOS X. I'm not saying it's that hard to learn, but if one hasn't any spare time to learn it, well... … anyway, it's good advice and it's on the (rather too long) to-do list.

pax / Ctein
==========================================
-- Ctein's Online Gallery http://ctein.com
-- Digital Restorations http://photo-repair.com
==========================================

[Ctein]

Dear Dya,

Thank you! I was hoping to invoke in the reader the same emotional reaction I had when I got to that last folder. At that point I was so immersed in the process that momentarily I couldn't figure out why I wasn't able to view the next month's iPhoto folder. The realization of the why was a mild but brutal shock.

Historical biographers have talked of something like this. After they spend so much time reading over a subject's papers and letters and interviews and documents, they become a kind of passenger traveling in the subject's life. They feel connected to it in a way that is more than only factual, it starts to have an essence of reality that is more than vicarious.

And I am so sorry to hear about your disk crash! The only time that ever happened to me I was fortunate enough to have backups that were 12 hours old and it was still upsetting and aggravating.

~ pax \ Ctein
[ please excuse any word salad. MacSpeech in training! ]
======================================
-- Ctein's online Gallery http://ctein.com
-- Digital restorations http://photo-repair.com
======================================

[Ctein]

Dear Martin,

I am afraid I have no useful advice to give because I have done no annotations whatsoever. All my work is carefully filed (and everything proof-sheeted) in chronological order and labeled by date, but there are no annotations to indicate the import of any of it. I do preserve all my correspondence, though, and I have sufficiently frequent correspondence that that would allow anyone who cared to figure out what 99+% of the photos were about.

Tee was an enthusiastic correspondent, too, and I'm guessing that if someone were to read through all her documents they could make similar correlations.

It would probably make a huge amount of sense for me to start adding annotations to my work. I'm almost 100% certain that I will not ever bother to do it, though.

~ pax \ Ctein
[ please excuse any word salad. MacSpeech in training! ]
======================================
-- Ctein's online Gallery http://ctein.com
-- Digital restorations http://photo-repair.com
======================================

[Aaron J. Grier]

Ctein, I'm wondering if you have any thoughts on digital archive maintenance? you don't seem to mention much about what happens _after_ all this enormous amount of work is done. the trick with digital archives seems to be periodic verification of data integrity and copying through to whatever the next common storage media is.