…though, when is the work ever light? Now in its second year, the Association of Moving Image Archivists (AMIA) and the Digital Library Federation’s (DLF) joint project – Hack Day – once again brought together archivists, programmers, and developers for an intense day of hacking and open-source tool development. Led and organized by AMIA’s Open Source Committee, this year’s menu featured an impressive and diverse line-up of project ideas that all were tackling some rather sinewy problems – a new-and-improved PBCore record generator and validator, a GUI for streamlining selection of target formats/containers during video capture in Black Magic, building out documentation for various tools and addressing some eye-raising oversights in the “Digital Preservation” Wikipedia page, to name a few (see the complete digest of projects here).
Fellow NDSRer Rebecca Fraimow will report on her project involving documentation for FFmpeg. As for me (Joey Heinen), the project for which I jumped on board was generating a report that pulls the results from various video characterization tools in order to point out discrepancies between them, proposed by Kara van Malssen as a continuation of a project started at the Open Repositories 2014 Developer Challenge. The tool functions much in the same way that FITS generates output from several tools at once so as to choose the “best fit”, though addresses FITS lack of robustness for characterization of A/V formats. Tools such as MediaInfo, FFprobe, and ExifTool are designed to parse and analyze A/V files to report on their various embedded descriptive and technical characteristics (e.g. codec, number and type of A/V streams, bit depth, color profiles, etc.) such that archivists can better manage their collections. However, at this year’s American Institute for Conservation of Historic and Artistic Works (AIC) conference, Joanna Phillips, conservator at the Guggenheim Museum, and Agathe Jarczyk, conservator and educator at the University of Bern, delivered a vital call-to-arms paper in which alarming errors in these characterization tools were exposed. Some noteworthy instances include reporting MPEG-4 for Quicktime 10-bit uncompressed video or examples where the timecode was incorrect by minutes (not just seconds). While some of these examples can be justified (particularly the famous MPEG-4 example, which is further explained in this great post by Erik Piil), there is clearly much work to be done to ensure that tools are correctly reporting on these factors or that users are well-informed as to how to interpret this information and to further understand the inner-workings of the tools (also illustrated in Erik’s post).
The group discussed what we considered to be the significant attributes for comparison across the tools and how to design the layout of the resulting csv so as to best facilitate this comparison. We then broke into two groups to complete the task: Group 1 (Ben Fino-Radin, Morgan Morel, Eugene Gekhter) to design the python script for running the analyses through the three tools and formatting the combined output; Group 2 (Kara van Malssen, Nicole Martin, Karen Cariani, and me) to determine the Xpath for the various attributes such that we could be certain that the most accurate field from each tool was summoned.
Group 2 decided to first select diverse and sometimes zany video formats such that we could get a good scope of how the three tools handle different formats: AVCHD, XDCAM, Super 35mm mxf, mkv, uncompressed mov, iPhone mp4. From this list we each generated raw XML reports in MediaInfo, FFprobe, and ExifTool and began poring over the results, using the following command-line prompts for each tool:
MediaInfo: mediainfo -f –Language=raw –Output=XML [filename]
FFprobe: ffprobe -show_format -show_streams -show_data -show_error -show_versions -print_format xml [filename]
ExifTool: exiftool -X [filename]
In examining each of the XML outputs, it didn’t take us long to realize that there were some major differences in how information was being formatted as well as how these attributes were listed, something that would prove to be a headache later on when reconciling the python scripts to locate the respective Xpath. For example, ExifTool would often output track information in a general “track” namespace, leaving the user to interpret whether this track was audio or video based on the given attribute contained therein (e.g. the presence of “Frame Rate” leading the user to conclude that Track 2, for example, was a video track). To complicate matters further, some video formats would output the audio within “Track 1” while others would be in “Track 2.” Because of this issue, Exiftool is still not completely implemented in the group’s github repository – but should be soon! Since ExifTool is most commonly used for still image collections given its initial development within that community, it is not altogether surprising that this tool often proved to be the most problematic for video formats. However, in the example of the iPhone mp4 file, neither FFprobe nor MediaInfo could report on the bit depth of the audio whereas ExifTool could.
While FFprobe and MediaInfo were, on the whole, the most faithful tools for outputting information, we often struggled with FFprobe’s (over)abundance of data, vacillating between attributes such as “codec” and “codec_long_name,” the “long_name” attributes often simply containing data that was formatted in a somewhat different way which, depending on the attribute, was sometimes more congruent with the other tools than the shortened attribute. FFprobe would also often display information that was still correct but would use somewhat indirect formulas to arrive to that conclusion (e.g. a Frame Rate of 30000/1001 to represent 29.97 – both meaning the same thing but presenting the info in a somewhat misleading way). I could continue down this rabbit hole of other minute discoveries, but the point is that behavior of these tools depends on the format and all the complexities found therein. While in some instances there may in fact be bugs in the code which results in false information, in other instances it’s a matter of knowing how each tool parses through the packets of information which can be unique to each format. An interesting follow-up project might be to chart the behaviors of a number of these formats from tool-to-tool to begin to understand how this data is parsed. But for seven people plowing through this work over the course of 7 hours, I’d say that we did our fair share of excavation. Thanks to all the organizers for creating such a great event and to all the amazing minds that came together for the cause!
This is Rebecca, jumping in. I was hanging out over in the Wiki Edit-a-thon portion of Hack Day — a new aspect of the event spearheaded by Kathryn Gronsbell of AVPreserve, focused on building out some of the documentation available around what we do as digital media preservation specialists.
Staring at text documents while your neighbors at the next table are discovering how to digitize video using only their brains might seem like a slightly dry way to spend the day, but adding an editing and documentation aspect to Hack Day is really a pretty great idea for a number of reasons. The digital archiving world — and, specifically for AMIA, the a/v archiving world — has a solid component of amazing and skilled coders and tech wizards who are completely comfortable breaking tools apart and seeing how they run. Still, for every archivist who has the piece of technical knowledge that they need to find a solution to their problem, there are ten or a hundred people who don’t even know where to start looking. As AVPreserve’s Chris Lacinak pointed out in this great post by Shira Peltzman, documentation is key to making sure that our toolsets are findable, accessible and usable. It’s also another way for people who might not have the training in Python or Ruby to still feel like they’re making a solid contribution at Hack Day, so it becomes an even more engaging event for the broader community.
This time, the Edit-A-Thon was composed of three teams. Kathryn Gronsbell and crew set to Wiki-ing on the highest level, tackling Wikipedia’s coverage of digital preservation as a broad topic. Among other tasks, they buckled down on providing really solid information about preservation fundamentals and links to resources for anyone who might come to Wikipedia looking for help (which of course is the first place most people go.) Meanwhile, Kristin MacDonough, of BAVC and Video Data Bank, recruited a group to help her boost up the usability of the A/V Artifact Atlas — a fantastic informational resource which documents the artifacts and errors that are likely to show up in converting analog video to digital formats. Most of the people who worked on that project don’t work with video in their day jobs, which made them the perfect team to act as test users and focus in on ways to make the site more understandable to the target audience of people who may not know very much about video.
As for me, I sat down with Erica Titkemeyer to tackle some documentation of FFmpeg, an open-source video characterization and transcoding program that forms the backbone of most of the open-source software options available for archivists working with video. FFmpeg can be a really useful tool in and of itself when working with digital video for all kinds of functions, including playing arcane formats, taking screenshots or clips, capturing metadata, and transcoding. However, since it’s only accessible through the command line — and the official documentation is designed for developers planning to integrate the tool into other programs, rather than for laypeople who just want to know how to make a YouTube file — many archivists become intimidated when they sit down to try and use it in their regular workflow.
At last year’s Hack Day, a team of intrepid FFmpegers created a wiki on GitHub to explain the installation and use of FFmpeg for laypeople. Erica and I built on that project, creating use cases for some of the most common functions and breaking down the code into its component parts so people could really see what was going on with the program and make it work to serve their needs. Common Use Cases and Transcoding Use Cases are the pages to check out there, if you’re interested. I learned a ton that I didn’t know about FFmpeg while trying to find ways to explain the stuff that I did know, and I plan to keep working on the project throughout the year. Something else to keep an eye on is Ashley Blewer’s work-in-progress, an app called ffmproviser which should allow users to make requests and generate lines of ffmpeg code for common commands; it’s not quite ready for full use yet, but stay tuned, because it’s got a ton of potential as a tool for a/v archivists.
This was my first time participating in Hack Day, and it was a pretty great experience — it’s easy to get caught up in the collective thrill of working with a team who are all dedicated to pushing the field a little further, and exciting to know that the work you do is going to be of direct use. It was also a great way to kick off AMIA, which focused very heavily this year on open-source digital tools, but that’s a topic for another post.