Hello all!
These are your NDSR Boston 2015 residents taking over. We’re only in week two, but are each feeling excited about the direction of our projects and eager to learn from our participation in the NDSR program over the next nine months. We’ll use this blog to discuss facets of our projects, professional development opportunities, successes, challenges, and lessons learned. Stay tuned for more about what we’ve experienced so far, and about whats ahead!
June 3, 2015 – The National Digital Stewardship Residency Boston (NDSR-Boston) program, a program to develop professionals in digital stewardship through post-graduate residencies, welcomed a new group of residents and hosts for the 2015/16 cohort. Each resident will have the opportunity to work on an exciting and challenging project at one of the Boston area institutions beginning in September 2015.
Andrea Goethals, NDSR Boston Project Director and Harvard Library Mentor welcomed the new group, saying, “We are fortunate to be in the position of having another really great group of residents. All of us are looking forward to welcoming them when they start in September, and getting to know them as they work with us on projects within our institutions.”
Alexandra Curran – MIT Libraries
Alexandra Curran’s interests include audio/visual archives, digital curation, and the preservation of multimedia collections in libraries and cultural heritage institutions. While earning her B.S. in Digital Cinema from DePaul University, she explored editing, compositing, data management and the long-term management of digital assets. Her M.A. in Library and Information Science and Graduate Certificate in Museum Studies, both from the University of South Florida (USF), prepared her for curatorial duties with interviews in the USF Tampa Library’s Oral History Program, including the significant Speaking Out Against Genocide digital oral history collection. While interning at the National Archives and Records Administration’s Motion Picture Preservation Lab she prepared materials for long-term storage and digital conversion, and also learned the principles and techniques of photochemical and digital restoration. As an intern for the City of Tarpon Springs, Florida, she created a community-based website for the Greek Community Documentation Project. Current volunteer activities include assisting a local historical society with preservation issues for video oral histories in older formats and ingesting content into an institutional repository. A fan of tea, cinema, and spy novels, Alexandra currently works in a tea store. She is looking forward to her residency with MIT Libraries.
Jeffrey Erickson – University of Massachusetts at Boston
Jeffrey Erickson is 2015 graduate of the School of Library and Information Science at Simmons College with a focus on Archives and Cultural Heritage Informatics. Jeff was attracted to the Cultural Heritage concentration because of its emphasis on technology and digital preservation. Building on his extensive professional career in IT, Jeff’s interest was to become an archivist so he could work on issues related to digital materials. Jeff has studied digital preservation, preservation management, digital asset management, XML and metadata at Simmons and applied his newly acquired knowledge to solve issues related to digital materials and technology at the Massachusetts Historical Society, the Forbes House Museum, and the John F. Kennedy Presidential Library and Museum. Jeff has a personal interest in preservation; he has restored an antique building, has collected artifacts from his family’s dairy farm in New Bedford, MA and is building a family archive to pass along his family’s heritage to his children and to future generations of his family. Jeff enjoys working on DIY projects, reading non-fiction, traveling with his family, playing basketball and working on his golf game.
Alice Sara Prael – John F. Kennedy Presidential Library & Museum
Alice Sara Prael recently graduated with an MLS from the University of Maryland, College Park, specializing in the curation and management of digital assets. During that time she served as the Digital Programs and Initiatives Graduate Assistant at University of Maryland Libraries. In this position she discovered a passion for digital preservation and partnered with the Special Collections and University Archives to create a workflow for processing born digital content. She spent the past summer interning with the National Archives and Records Administration where she oversaw the digitization and description of a series of records from the John F. Kennedy Assassination Records Collection. Alice currently lives in Maryland where she hikes, crafts and plays roller derby with the DC Rollergirls. She is thrilled to continue work with digital preservation at the John F. Kennedy Presidential Library through NDSR.
Julie Seifert – Harvard Library
Julie Seifert graduated from the MLS program at the University of North Carolina at Chapel Hill, with a concentration in archives and records management, and a certificate in Digital Curation. While at UNC, she focused on digital preservation and completed a Master’s project on digital forensics. She also received a scholarship to go to the Czech Republic, where she met Czech colleagues and studied digital libraries in the Czech Republic. In her free time, she enjoys paddle boarding, reading, and traveling. She completed an internship in Boston during graduate school and is looking forward to being back! She will carry out her residency at Harvard University.
Stefanie Ramsay – State Library of Massachusetts
Stefanie Ramsay is a recent MLIS graduate from the University of Washington. Drawing upon her background in American history, Stefanie’s focus in graduate school has been on the preservation of historical materials for long-term access and use by diverse user groups. She’s worked in digital collections for the University of Washington Special Collections Library, performed archival processing for the Washington State Jewish Archives, and implemented a pilot digitization program for architectural drawings at the Seattle architecture firm NBBJ. She believes digital preservation is a necessity for modern information organizations, and is eager to learn more about best practices as well as how libraries and archives adopt the appropriate workflows to ensure greater access to and preservation of their important materials. Having previously lived in Los Angeles, New York City, and Seattle, Stefanie is looking forward to walking all over Boston, eating lots of lobster, and cheering on the Red Sox at Fenway Park. Stefanie will carry out her residency with the State Library of Massachusetts.
June 2015 – The National Digital Stewardship Residency Boston (NDSR-Boston) program, a program to develop professionals in digital stewardship through post-graduate residencies, has concluded the first round of residents marking the occasion with a Capstone Event held at Harvard on May 13, 2015. The residents, who were embedded within five Boston area host institutions (Harvard, MIT, Northeastern, WGBH and Tufts), each presented their work in a poster created as the final project of the residency.
The event was well-attended and included representatives from each host institution as well as local and regional professionals. Among the hosts, it was widely agreed that the experience of mentoring a resident was a rewarding one and that the resident contributed valuable work to their institution.
With an eye towards sustaining and expanding the regional digital stewardship network, the program plans to include the first year cohort of residents in training and social events during the second residency year which starts in September 2015.
I’m hitting the phase in my residency now where it’s time to start looking ahead to the future– and the future, right now, means documentation.
For the past several months, I’ve been responsible for designing and operating the process of ingesting AAPB drives crammed chock-full of unique video content onto LTO tape. Across the way, I’ve hit a number of the inevitable roadblocks and complications, and had to adjust my tools and strategy accordingly. That’s all to be expected, but what happens when someone else has to pick up where I left off? Although I’ve made all my decisions in consultation with the team and my mentors at WGBH, there are several nitty-gritty details about the usage of the scripts I’ve built and the workflow I’ve developed – and, especially, the way that it’s changed over time — that right now exist only in my head. While I’ve done some documentation along the way, at this point it’s time for me to take a hard look at my notes and make sure they’re as clear as possible so that the next person who comes along (and the next after that, and the next after that) can base their future preservation decisions on solid data that’s fully understood.
….and when I sat down to start doing that, I remembered that while I’d promised in a previous entry to share my workflow for the AAPB ingest on this blog, I’d also never actually done that either. So here goes! (Fair warning, this may get long.)
This is a picture of one of my standard Trello checklists for the American Archive drives that I’m processing. I developed the checklist pretty early in the process. Looks pretty straightforward, right? Let me go through and provide a little more context.
So that’s the basic workflow – which doesn’t take into account special circumstances that apply only to some drives. For example, a certain number of drives have known failed files on them, which are listed in a separate Excel sheet; I’ve modified my scripts to include a process which pulls those files and move them to separate folders so they don’t take up space on an LTO.
This initial workflow also leaves out a number of smaller QC procedures that I’ve learned to include over time, after stumbling over problems I hadn’t expected. For example, after discovering that some failed files and non a/v-files had been accidentally included in the collection, I started running searches on each batch of PBCore records that I generated to identify any that don’t have some kind of audio or video essence track. Recently, we also discovered that some of the files that we’ve been ingesting are low-quality derivatives that we probably shouldn’t be preserving, so I’m currently in the process of modifying my scripts again to try and weed those extra files out at the pass as well. One of the results of that is that the drives I’m processing now end up looking a little bit different than the drives I was processing at the beginning of the project, when I hadn’t figured all of that out yet.
Whether or not you read through all of those workflow steps, the point I’m trying to get at here is that in my experience workflow is never really ‘basic,’ and it’s never static. It’s constantly changing to take into account new circumstances you didn’t anticipate, and there’s no way to get around that. As archivists, the material that we’re preserving just isn’t a series of square pegs; there’s always going to be that one item that throws the whole plan off-kilter. That’s why it’s so vital to make sure you keep documenting what you’re doing, so that if someone in the future is wondering why the material or metadata they’re looking at suddenly looks a little different than it did before, they have a clear and easy answer as to why, how, and what to do about it.
— Rebecca, signing off, and diving back into documentation!
As my project moves along, our first batch of digitized audiocassettes from the Herb Pomeroy collection have just arrived back at MIT from the vendor. Leading up to this exciting arrival, a smaller group of representatives from the project have been meeting to determine exactly what the Quality Control (QC) steps will be, as distinguished from our Quality Assurance (QA) steps.
Semantics can be flustering sometimes, but nuanced distinctions between words and phrases can contain major implications. For our project – in defining Quality Control and Quality Assurance – the implications affect who is responsible for executing the actions and who needs to be trained on what tools, as the Collections Preservation and Reformatting (CPR) department is executing the Quality Control actions, while the Content Curators (CC) work on the Quality Assurance methods. Below you can see the isolated high-level “Transform Analog to Digital” workflow and the “Manage Digital” workflow and where the QC and QA steps fit.
The way I’ve been conceptualizing the distinctions we made between the two are: Quality Control is more like quantitative data checking and Quality Assurance is qualitative content assessment. So I wanted to discuss these distinctions, the methods that are included under them, and the tools we are currently employing to execute these processes.
Quality Control (QC)
As mentioned above, the Quality Control processes relevant for this project have really geared towards determining whether the data and files requested from the vendor is present and complete. Questions involved in our QC measure include: Are the number of recordings we expected on the hard drive? Are all of the metadata fields we needed filled in? Are the digital audio surrogates the length and stereo specification we requested? Here are some of the QC actions and the tools we are enlisting – though they will certainly evolve as the workflow matures:
| QC Action | Tool |
| Virus check | Sophos |
| Checksum validation | Karen’s Hasher or MDFiveCheck |
| Was reformatting done correctly? | BWF MEtaEdit to check channels and stereo/mono |
| Are all metadata types we requested present? | BWF MetaEdit |
| Correct file formats for preservation and access copies? | DROID |
| Appropriate number and size of files? | N/A – Eyeball the directory |
Quality Assurance (QA)
The Content Curators are then in charge of the QA, listening to the actual audio (content) and determining:
It’s been a very useful conversation for our group, especially instructive for other members as to what ways digital archival content needs to be treated in order to maintain integrity, reliability, and authenticity. As this has all been a part of the group hashing out the workflow, it is another example of how documenting detailed, lower-level pieces of the workflow has clarified responsibilities and where specific tools can be leveraged. I’m very close to finishing documentation of the next portion of the workflow (the D sequence above), and I am excited to show everyone the work soon!
By JHOVE, we’ve done it –
Tricia P
Samantha DeWitt
NDSR Resident | Tufts University Tisch Library & Digital Collections and Archives
What can be done to encourage data preservation among university researchers? U.S. science and technology research agencies and the offices that oversee them have had strong ideas on the subject and have been making pronouncements for over a decade. Twelve years ago, the National Institutes of Health (NIH) began asking grant-seekers to plan for the management of their research data. The National Science Foundation (NSF) followed in 2011. In 2013, the White House superseded both agencies with a memorandum from the Office of Science and Technology Policy (OSTP) announcing the government’s commitment to “increase access to federally funded published research and digital scientific data…”
Money talks. The NSF invests about $7 billion in American research annually and the NIH allocates about $30 billion to medical research. To put these numbers in perspective for Tufts, that translated into about $8 million from the NSF and just under $62 million from the NIH last year.*
Because the agencies that disburse money in the form of federal R&D funds have been mandating data management plans from their applicants, the research universities that rely on those federal funds have responded. Many, including Tufts, have looked to their libraries to provide support in:
Universities have taken other steps as well. Some have created new data repositories, or have augmented their existing institutional repositories in order to accommodate and support the long-term preservation of research data.
Have government data access directives had their intended effect? So far the data are sparse. A 2014 Drexel University study did find NIH mandates, along with those implemented by scientific journals, seemed to be “meeting the goal of increasing the sharing of scientific resources among life science investigators.” But the point I want to make here is that, at the very least, these mandates have served to publicize the issue of data management to a degree that has encouraged debate and discussion among researchers and others within the university.
While the government still provides the greatest portion of funding to U.S. research universities, the automatic spending cuts of the 2013 budget sequestration have reduced the flow of money enough to make grant-seekers nervous. Researchers are increasingly appealing to foundations, corporations and philanthropic organizations to fill in the gaps. I hope you’ll join me in April for Part II, as we “follow the money” and look at which non-government funders are advocating for data management as well!
‘Till then,
Sam
* In research project grants
http://projectreporter.nih.gov/reporter.cfm
http://dellweb.bfa.nsf.gov/Top50Inst2/default.asp
This was our first effort at charting out the current accessioning workflow for all the different material collected by WGBH, step-by-step – from its arrival (in boxes carried over from production departments, or on hard drives dropped off by production assistants, or copied to a server or sent to a dedicated inbox or – you get the idea), through the various complex data and rights management processes designed to make sure that every tape, shot, transcript, and contract is appropriately accounted for, all the way through to physical storage and the burgeoning digital preservation program the department is now putting into place. Peter Higgins (my partner in workflow documentation crime) and I spent a long time trying to make it as clear and easy to follow as possible, but it can’t be denied that it’s kind of a beast.
Don’t bother trying to read the tiny text in the boxes; for now, just relax and enjoy the wash of incomprehensible geometry. After all, as of the point of the writing of this blog post, thirty days after Peter and I presented this document to the rest of the WGBH team, it’s also already outdated. WGBH, as I’ve mentioned before on this blog, is an archive in transition, which is one of the reasons working on this project now was so crucial. The current workflow involves a lot of emailing back and forth, entering information into Excel spreadsheets (unless someone else is using them first), moving folders around in a share drive, and assigning files color-coded labels. As a newbie myself, I can vouch for the fact that this is a difficult system to explain to newcomers, which is a problem for an archive that hosts several new interns every year. It also tends to result in a lot of this:
Everyone at WGBH wants to streamline the current workflow, get rid of unnecessary steps and outdated practices, and figure out better tracking – having an easy way to tell who’s doing what when is key for ensuring that work isn’t duplicated and material doesn’t slip through the cracks. However, two major changes are coming down the pipeline, which may also alter the workflow significantly.
The first is that the Media Archive Research System, aka MARS – the complex FileMaker database which currently stores and links together all of WGBH’s data about its programs, physical media assets, original video content, and licensed media – is being replaced. In the long run, the new database should be friendlier for WGBH production departments, making it easier for them to enter and store metadata for the use of the rights department and the archives, and to retrieve the data they need on the other end. In the short term, however, there are still a lot of question marks about how exactly the new database is going to link up with the current archival workflows around metadata management.
The second factor is the adoption of the HydraDAM system for digital preservation, which I talked about [in a previous blog post linked here.] HydraDAM isn’t intended to be the primary source for content metadata, but again, in theory, it should be able to automate a lot of the processes that archivists are currently doing manually, such as generating and comparing checksums to ensure safe file transfers. But until HydraDAM is ready to kick into full gear, we won’t know for sure how nicely it’s going to play with the rest of the systems that are already in place in the archives.
We want to make sure these transitions go as smoothly as possible, and as we’re cleaning up the workflow to make it easier for the staff, we also want to make sure we’re not making any changes we’ll regret when the new systems come online. That’s why we let workflows eat our brain for most of December, throwing ourselves into the task of creating the monstrously in-depth diagram to represent the current workflow that we possibly could.
Then we used the original beast to build this:
OK, yes, it still looks pretty beastly, but in theory it’s the first step on the road to a better, stronger, faster beast. However, the real key focus here is all those gold-colored boxes on the chart. They represent elements of the workflow that we know we’re going to need, but don’t yet exactly know how they’re going to happen in the oncoming Hydra-PYM future – whether that’s because we haven’t finished developing the tool that’s going to do it, or because we need to research new tools that we haven’t yet got. We’re using this proposed workflow itself as a tool for discussion and planning, to help target the areas to focus on as we move forward in implementing new systems, and make sure that we’re not leaving out any crucial functionality that we’re going to need later down the line.
Although there’s still a long timeframe for the WGBH workflow overhaul, the work we’ve done has already had some immediate results – including the adoption of a new project management system to help the department keep better track of our ingest process. We’ve decided to use Trello, a flexible online application that allows us to create ‘cards’ representing each delivery of production material that comes into the archive, and add lists of tasks that need to be accomplished before the accession process is complete.
When we presented our Trello test space to the MLA team at WGBH, we thought we would have to build in some time for testing and for people to get used to the idea of tracking their work in an entirely different system. However, everyone we spoke to was so excited for the change that we received a pretty much unanimous decision to jump on-board right away.
Inspired by the activity on the main MLA Trello, I’ve also created a Trello board to keep track of my own next phase of NDSR — which will be a topic for my next post, in which I solemnly swear to have minimal geometry.
– Rebecca
Last Thursday, January 22nd, all of the residents, hosts, board members, and a group of interested colleagues convened to hear updates on all of our NDSR Boston projects. We enjoyed refreshments, our project director Andrea Goethals introduced our cohort of Residents, and we each spent about 15 minutes speaking about our project goals, the progress we’ve made, challenges we have faced, and what we have left to accomplish before June 2015. It was a great way to catch up on everyone’s projects as well as an opportunity to meet with people interested in our work.
Since I have been wanting to share some project updates anyhow, I thought I would just post my presentation (with embedded narration) for anyone else interested. You can also find a condensed, written update on my project through the Library of Congress’ digital preservation blog The Signal.
Enjoy! And happy blizzard season!
Tricia P.
Hello navigators of the digital plane!
It has been a long time since my last check-in, but rest assured there has been much movement in the world of formats migration at Harvard. Today I bring you more musings on the integrity of digital images and some of the psychedelic visual processes therein. The theme today is that looks can be deceiving…not unlike one of these!!!
Can you see the hidden image??? Hint: It’s an Elephant
Okay, well maybe the work I’ve been doing is actually quite different, but anyways…
The last month-and-a-half has been devoted to deep analysis of the Photo CD format, a pageant-style selection of potential target formats and tools, and conversion testing and quality assurance between myself and members of the Imaging Services department. Moving into an action phase of a project is always very gratifying after months of planning and preparation, and the importance of carefully and deliberately planning for this phase has fully crystallized for me during this past month. Deciding upon a format’s significant properties (which will be our “wish list” for successfully migrating a format), creating criteria for potential tools and target formats, and creating metrics for judging the success of these conversions requires the involvement of multiple stakeholders who will ensure that nothing is being overlooked. To stage this moment of consensus, it is up to us in Digital Preservation to document our analysis of the format (of files as they exist within the Harvard repository and in technical specifications for the format) and clearly articulate the risks and the options that currently exist for avoiding them. In analyzing the format at large, our definitions of the format’s significant properties were as such:
-A tool that correctly interprets PhotoYCC (luminance and chrominance channels separated) and maps color information to a device-independent color space without clipping available color information is essential. If the output color space is RGB, YCC color information should fit into the color space without clipped levels and with the widest color and grayscale gamut possible.
-Scene Balance Algorithms (SBAs) are different calibration settings for the analog object during the initial scanning process based on the material (type of film stock, slide, object, etc). A conversion tool should be able to account for these adjustments to ensure that highlights are not overblown and the photographic object is understood with accurate chrominance and luminance information.
-Image Pac compression is a very efficient form of mathematically lossless encoding which would store the image at a Base resolution (768 x 512) and would be expanded out to a full resolution image (3076 x 2048) through a PCD-compliant application. Conversion tools will need to be able to unpack Base resolution files to ensure that images are migrated losslessly.
-PCD bit streams are 8 bit-depth allowing for 16 – 24 bits per component, using compression/subsampling for efficient, mathematically-lossless encoding of color information. Ideally a target format should be able to losslessly contain all the same information using an efficient compression algorithm.
-The format contains embedded technical metadata, particularly related to the original scanning process, Scene Balance Algorithm adjustments, and other technical and provenance metadata. If the conversion tool is not able to perform this extraction, other tools should be considered.
Based on these findings, we were then able to conduct an audit of the available tools and possible target format for the project. Each of these criteria was used to score each tool/format based on its system and format specifications. The score sheet looked like this:
To score the acceptability of each target format and tool, a “2” was assigned when the criteria was met, a “1” if some issues were noted with this particular criteria (e.g. some loss of color information noted despite support of the color space) and a “0” when the criteria was not met. In some instances criteria were also weighted if it was an essential component (e.g. correctly interpreting an Image Pac file is absolutely essential in order for us to declare the conversion a success). If the tool did not meet these essential criteria, a negative value was applied. In addition to this score sheet, a report was created which also noted tools and formats that were left out entirely due to lack of ongoing support.
From this assessment we decided on these factors for creating a test set from the assortment of PCD files in the repository as well as a number of migration scenarios including the tool used, custom settings within the tool, target format and color space outputs, and the means for performing quality assurance (QA) on the results. Our test factors were as such:
Tools
-pcdMagic appears to be our best option. Not only does it understand Image Pac compression and Scene Balance Algorithms, it can output ProPhoto RGB and DNG/TIFF files. It also accepts any number of input color profiles and film terms for balancing luminance and chrominance levels in the image.
-pcdtojpeg and ImageMagick can both extract technical and provenance metadata from original PCD files.
Output
-YCC is in dwindling support in most digital photo formats so we wanted to try mapping to a new color space. Our research found that CIELab and ProPhoto RGB would be acceptable color spaces which would introduce minimal (and possibly zero) loss of information. However, no tools on the market can both interpret PCD and map YCC to CIELab. Thus ProPhoto RGB is our only option
-DNG and TIFF are the best options for a target format and are also possible for export within the pcdMagic tool. One downfall with DNG is that it is a raw photo format and is not always natively understood by photo applications. Additionally, pcdMagic outputs to DNG with a color space that is “similar to” RGB ProPhoto. Indeed, characterization tools such as ImageMagick report a color profile name of “pcdMagic DNG profile” as opposed to TIFF which contains a true ProPhoto RGB profile. The one downfall with TIFF is that it does not incorporate efficient lossless encoding – a 6 MB PCD file would need to be expanded out to a 36 MB file in order to contain the same lossless information. This is not necessarily an opposing factor to our ultimate decision around a target format since this is more of an implication on storage space than of a successful conversion but is nonetheless a consideration.
Settings and Scenarios
-Images from the Harrison D. Horblit Collection of early photography (6628 of the 7243 total PCD files) were scanned in an ISO 3664 compliant environment and photographed with standard Color Negative film stock and scanned by Boston Photo. Additionally, the photos were uncropped with color bars for calibration and upon return from Boston Photo were cropped (removing color bars) and rendered as RGB Production Masters also in PCD format in order to generate JPEG deliverables. Images from the Harvard Daguerreotypes project (438 files) were photographed on 4050 E-6 35mm film stock and applied with corresponding Scene Balance Algorithm settings. These were delivered to Harvard as cropped PCD objects. Images from the Richard R. Ree Collection (177 files) were shot on an unspecified film stock. More exploration is being performed to determine what settings were used during the scanning process.
-Imaging Services maintains their own collection of Kodak film terms that are specific to SBA settings and film stocks used during the scanning process which are not included in the suite of input color profile options in pcdMagic. We added these film terms to the available profiles in order to perform additional tests and QA.
-Images will be exported as both TIFF and DNG for additional QA.
In QA-ing the results in PhotoShop, we made some interesting observations. First of all, for the Horblit Collection and Harvard Daguerreotypes, the added Kodak film terms produced results that were wildly superior to the default settings. However, some differences could be seen in what film terms produced the best results. For the Horblit Collection, the “Kodak Color Negative” setting proved to be the most ideal setting across the board (for 5 or 6 images tested). This is indeed good news since when migrating more than 7,000 images we are going to want to have an accurate sense of how to place images in different buckets and apply conversion settings uniformly within a batch. If conversion of all 6628 images in Horblit is best obtained through the same film term setting, then a process of batch-converting all objects with that film term setting will be much more efficient and less resource-intensive. Additionally, ImageMagick reported an SBA setting of “Unknown Film Type” across the board so this was not necessarily in conflict with any of the embedded metadata. However, for the Harvard Daguerreotypes the ideal choice of film term from image to image is less obvious. Some images appear best with the “Kodak Color Negative” setting while others demonstrate infinitesimally superior results in either “Kodak 4050 E-6” and “Kodak 4060 K-14” settings. To make matters more complicated, ImageMagick reports SBA settings of “Kodak Universal E-6” film stock which never appears to be the best choice within pcdMagic.
But how do we gauge success of the transfer anyways? PhotoShop is a great application for QA since it presents pre-rendered histograms of TIFF objects and converts Raw formats (such as DNG) to RGB TIFF for comparison. It also can read RGB channels at any point in the image which is especially useful for images with the color bars. When dragging the color sampler across middle gray on the color bars, the RGB channels should be balanced. If the hue or saturation is off, then the channels will not be as equally balanced. This is an example of acceptably balanced gray-levels:
In the absence of color bars, the histogram readings are our best friend (well, they always will be even in the presence of color bars). An image that displays a histogram without a clipped-levels warning and with a wide and dynamic histogram waveform means that the conversion was a success. In comparing these two readings for one example, we can see that the Kodak Color Negative results in a wider, more ideal color gamut. In instances where the histograms demonstrate similar results, converting the image to grayscale can help to demonstrate which image has a better representation of luminance levels.
These examples demonstrate the importance of using acceptability metrics that go beyond what is seen by the naked eye. It is worth mentioning that I am mildly color blind so it would be difficult for me to judge which image shows more cyan tones than another. Here is an example of where the eye fails to show what is actually present. These two images using different color profiles seem pretty much the same, right? Well, in using PhotoShop’s subtract feature, we see that they actually differ quite significantly in how they are interpreting color as is demonstrated by the ghost image that remains.
But what happens when we subtract a TIFF image from a DNG image with the same output settings? Interestingly enough, the subtraction produced no ghost image. Also, the histograms are almost exactly identical (with no differences in gamut but very slight differences in standard deviation readings…I’ll spare you the details on that). We feel confident after comparing our results that the TIFF and DNG produce basically identical results.
Lastly, what about the Richard R. Ree images? These images are unfortunately even less clear. Though the images demonstrate a wider color gamut with the Kodak film terms, some images appear more washed-out than some of the default color profile settings. Take this example of an image of clouds. According to the histogram reading, the Kodak Color Negative is the more ideal choice. But isn’t the more ideal choice the one in which more details are visible? Perhaps this is more of an issue with how the display monitor is calibrated (what you see is not always what you get) but these images require more analysis.
So, to sum it all up, here’s where we are:
Coming soon will be the diagram workflow which is currently in draft mode. Also, the RealAudio/SMIL project is just kicking off so look for similar analysis updates soon.
-Joey Heinen
Resources Cites:
Burns, P. D., Madden, T. E., Giorgianni, E. J., & Williams, D. (2005). Migration of Photo CD Image Files. IS&T: The Society for Imaging Science and Technology. Retrieved from http://losburns.com/imaging/pbpubs/43Arch05Burns.pdf
pcdmagic. (2014). Retrieved December 19, 2014, from https://sites.google.com/site/pcdmagicsite/
Eastman Kodak Company. (n.d.). Image Pac compression and JPEG compression: What’s the Difference? Retrieved December 08, 2014, from http://www.kodak.com/digitalImaging/samples/imagepacVsJPEG.shtml
Kodak Professional. (2000). Using the ProPhoto RGB Profile in Adobe Photoshop v5.0. Retrieved from http://scarse.sourceforge.net/docs/kodak/ProPhoto-PS.pdf
As a part of the NDSR-Boston program, each institution and resident host a home-turf event. We have only had two so far, but it has been a great way to exhibit everyone’s institution as well as keep individuals in the program engaged with each other as a whole. And this past Thursday, December 18th, it was MIT’s turn to host! All of the residents were there – me, Joey, Jen, Sam, and Rebecca – as well as two of the host mentors – Andrea Goethals from Harvard University and Nancy McGovern from MIT. We were also joined by our intrepid Program Manager, Kristen Confalone, and MIT’s Digital Archivist, Kari Smith.
We began our host event with Kari giving us an in-depth talk and demonstration of the Digital Sustainability Lab. She walked us through some test instances of tools like BitCurator and ArchivesSpace, and we discussed how the lab would fit into our larger project goals. Then I gave everyone a little peek of the workflow diagrams and documentation that I have been working on – which I will probably be sharing on the blog in the new year.
Following the demonstrations, we got a quick tour of some of the MIT terrain. One of the coolest features of the academic buildings is that there are always lots of little innovative exhibits and interactive toys and experiments peppering the hallways. It’s been one of my favorite parts of working here and meandering through the campus.
We ended our night reveling in some pre-holiday cheer at the campus pub. It was a festive way to punctuate the end of an excellent first half of the residency. I hope everyone out there is gearing up for (or perhaps already enjoying) lovely winter break plans. Happy Holidays from NDSR-Boston – see you in 2015!
Happy Holly Daze,
Tricia P
ndsrboston2015 