I am back in Cambridge today after spending last week in beautiful Chapel Hill, North Carolina, where I attended the International Digital Preservation Conference, along with fellow residents Alex and Alice. I enjoyed the opportunity to meet colleagues and to engage in discussions about common digital preservation issues – it was really helpful to hear about issues others have faced and how they tackled them. I’d like to say thank you to the organizers for making it happen, and also for allow us residents to volunteer!
There was a lot of discussion about audits and self-assessments at the conference, which I found especially helpful, since my project is focused on that. I thought I’d use my blog post today to talk about where I am in the process of self-assessment, and mix in a little bit of what I’ve learned from others so far.
Right now, I am working on the “gap analysis” aspect of the project. This means figuring out which of the ISO 16363 metrics we meet, and whether we have documentation to show that we do. For those not familiar at all with audits – it’s basically like a really complicated matching game. Sort of. The ISO 16363 lays out a bunch of metrics (over 100 of them), each one describing a different thing that a trustworthy repository should do or have (i.e. a mission statement, a preservation plan, etc.) So, the repository needs to go through and decide if they meet those standards – and then provide documentation that shows that they do. After all, you can’t just say that you’re doing something, right? So, you match the documentation you have to the standards – or, sometimes, you make new documentation. If you’re not familiar, it might be helpful to just glance over the ISO 16363 standard.
Because there are so many metrics, I’ve found – and heard from others – that just getting organized can be really complicated. I realized I needed to have a space where I could put down all the metrics and then take notes about whether the documentation to meet those metrics existed. Inspired by the lessons from CLOCKSS audit process and their wiki, I created a wiki to help with this. I made a separate wiki page for each metric. I also spoke with fellow NDSR resident Jessica Tieman, of the DC cohort, who said she used a Sharepoint site for her gap analysis – so I thought a website seemed like a good way to go!
However, while at iPres, I heard from Maureen Pennock of the British Library, who also used a wiki for her audit. She said that the wiki was helpful during the audit process. She was able to use it as kind of a “scratch pad,” and record information she gathered related to a certain standard. For example, she said she took notes on the different wiki pages as she interviewed people in her organization. However, she added that while the wiki was helpful during the process, when it came time to create the final report of her findings, the wiki made things more difficult because all the information was on different pages.
Also, here is the link to the notes from the iPres audit workshop, for those interested. Notes from most of the sessions are available online.
I can see how the wiki would make writing the report difficult. However, I’ve decided to continue on with the wiki, for now. I think it will be helpful for taking notes as I proceed throughout the project, and I like that it’s something that people could continue to add to and work on. I also like the structure of it – how you can nest the sub-sub-metrics within the sub-metrics. (That is possibly the nerdiest sentence I’ve ever written).
Additionally, to help me further visualize what documentation we have and what we don’t, I’m in the process of creating an Excel document where I’m listing the metric and supporting document – or a document that could be created to support that document. If there is more than one document to go along with a metric, I list them individually. That way, once I’m finished, I could use the “Sort” filter on Excel to list by document type, if I wanted to see how one document can apply to many standards. This will hopefully allow us to see which documents will have the most impact. It’s likely that one document could support many standards (they did that in the CLOCKSS audit, if you look at their wiki) so if we need to create new documentation, we might be able to create just a few documents instead of, you know, a hundred.
Finally, I am planning to create a Word document with some really high-level, general summaries of the metrics. I’m imagining that I might write a short paragraph summarizing each of the three sections, and then suggest documentation that could provide evidence for those sections. I think it’s very possible that a few pieces of documentation could meet many requirements – I saw this in the CLOCKSS audit.
In summary, I’m hoping that by getting organized, I can figure out what documentation we’re missing, and whether one or two new documents could effectively fill all those gaps.
One last thought: Is there something ironic about creating new documents to organize all your documentation? Possibly…Also, do any of you have any thoughts on how to get organized for an audit? Finally, here’s some more pictures of Chapel Hill, just because: