Open Metadata Handbook

Preface

The Open Metadata Handbook is a joint effort of the Public Domain Working Group and the Open Bibliographic Data Working Group of the Open Knowledge Foundation.

Introduction

This book is intended to give the non-expert an overview of standards and best practises related to publishing metadata about works. Its primary focus is metadata from cultural heritage institutions - i.e. GLAM institutions (galleries, libraries, archives and museums).

The book was started to help us get to grips with diverse collections of metadata which we were interested in using to figure out which works have entered the public domain in which different countries. At the OKF, we have been working on the developement of automated calculation to determine the public domain status of a work (see http://publicdomain.okfn.org/calculators), and we soon realized that we often do not have the necessary metadata to accurately determine whether or not a work is in the public domain. We have obtained data from different sources, e.g. BBC, British National Library, but we need to combine this data in meaningful ways in order to achieve a more comprehensive set of metadata. This required us to engage in the process of vocabulary alignment, removing duplicate entries, understanding whether similar fields actually mean the same thing, and figuring out whether different data models are compatible with each other.

This gave rise to a whole new series of questions:

what are the different standards for metadata?
how do they each relate to different types of works?
what are their corresponding pros and cons?
are these standards interoperable with each other?
are they cross-border ? cross-sector ?
how can we integrate data taken from different sources?

This book aims to answer some of these questions in a superficial but nonetheless exhaustive way.

Goals

The Open Metadata Handbook is NOT:

a universal guide to metadata designed for people with specialized technical knowledge.
yet another series of best practices or recommendations on how to set up an open and interoperable metadata model.

The Open Metadata Handbook is:

a simple guide to tools and standards that already exist, an attempt at mapping metadata structures used by different institutions, organisations and projects, including, but not limited to, GLAM institutions.
a series of guidelines to help people navigate the huge amount of work that has been done in this area, to help them achieve proper document discovery and analysis by means of metadata.

The ultimate goal of the Open Metadata Handbook is:

to help anyone learn about metadata, even if they have no education on data management.
to encourage metadata development and integration in an interoperable way.
to make it easier to harvest and process bibliographic metadata from a variety of sources.
to foster the community involvement in the drafting of this document.

Scope

The Open Metadata Handook is not only concerned with literary works (books, articles, etc), but also with artistic works, musical works, audiovisual works, and so forth. While the focus is generally on bibliographic works, there is a significant amount of metadata for film/video, sound and cartographic data, as well as metadata for datasets (census, etc). We do not plan to provide a detailled overview of every single standard, but rather to illustrate what is the state of the art in different domains with a series of pointers on where to find more accurate information. It important to explore how metadata is employed to describe different types of works, in order to identify whether (and to what extent) does the use of metadata differ according to the type of works taken into consideration. This would also allow us to determine what is the most preferred standard for each type of bibliographic work and why.

Introduction providing a definition of the most basic concepts:
- what is metadata?
- why is metadata useful?
- how is it produced?
- how is it used?
What is Open Metadata
- what does 'open' mean ? (opendefinition.org)
- why open up metadata? (cf. Open Bibliographic principles @ http://openbiblio.net/principles - but not as strong)
- Legal Issues
  - default position of the law
  - open licences
- Technical Issues
  - Accessibility
  - Interoperability
  - Reusability
- case studies: http://obd.jisc.ac.uk/examples
Technical overview of different Metadata standards:
- anatomy of Metadata records from different fields (3-5 examples)
- why are there different elements & different standards
- overview of the most common elements used for films, artworks, literary works, etc
- overview of currently available standards
  - their objectives, historical context, their corresponding pros & cons + "personal note"
- overview of different serialization schema
- examples of different fields of application (who uses what) - with e.g. screenshots
Data Integration
- how to integrate data from different standards, how do different standard interact with each others? e.g. protocols, interoperability.
- how to understand different elements from different standards, from different fields, for different requirements? e.g. vocabulary alignment.
- what are the tools available for the discovery, identification, location, and deduplication of metadata
- examples of e.g. collection trusts.
Appendix
- list of resources, mailing lists, books
- glossary

Addressees

The Open Metadata Handbook is intended for:

GLAM institutions: to encourage them to open up their metadata, and to help them understand the challenges that must be addressed in order to provide interroperability between different metadata standards. Many institutions do not know about the metadata systems of other institutions.

This guide will point them to all the currently existing information. The goal s therefore not to harmonise the use of metadata, but to provide guidance on how can different standards interact with each others.

People who are interested in reusing open metadata from cultural heritage organisations, but who may not know much about how this is usually structured. The Handbook must therefore be accessible for and useful to non-technical users, as well as non-specialists who are interested in consuming open data, perhaps from a variety of different sources.
Anyone who is interested in getting a broad overview of standards and best practices related to publishing metadata about works. As a textbook, the Open Metadata Handbook should include case studies, and content based around Encyclopedia studies.

Main Contributors

Primavera de Filippi
Andrea Marchesini
Jim Pitman
Karen Coyle
Claudio Gnoli
John Mark Ockerbloom
Owen Stephens
Pascal Christoph
Antoine Isaac
Rakesh Kumar Pandey

Previous Material

Most of the material below is now obsolete, given the new direction which has been taken by the Open Metadata Handbook. However, it might contain useful material that should be incorporated into the new guide:

Metadata Elements: Core metadata elements, for discovery, identification, and description of different types of work
See also http://openbiblio.okfnpad.org/metadata-elements

Metadata Standards: State of the art in terms of currently available standards and their different fields of application (who uses what) + pros/cons of the most popular standards concerning bibliographic works
See also http://openbiblio.okfnpad.org/metadata-art and http://openbiblio.okfnpad.org/metadata-procons

Recommendations: Conclusions / recommendations on what we consider to be the most appropriate standard(s) for the purpose of open bibliography. This should take the form of a decision-tree, where different data providers can answer simple questions in order to find out what are best-practices for them, in terms of exchange format and metadata format.
See also http://openbiblio.okfnpad.org/metadata-conclusions

Useful Links

Wikipedia has related information at Metadata standards