Cinergie – Il cinema e le altre arti. N.20 (2021), 69–83
ISSN 2280-9481

FiRe2: a Call for a Film Repository of Technical Data and Memories for Photo and Movie Restoration

Alice PlutinoUniversity of Milan (Italy)

Alice Plutino obtained a PhD in Computer Science at University of Milan (2021), where now works as Research Fellow. Her research interests are: Color Science, Colorimetry, Image Enhancement, Image Digitization and Archiving, with a particular interest in Cultural Heritage applications. She is author of the book “Tecniche di Restauro Cinematografico” edited by Dino Audino and of several journal and conference papers of national and international relevance. She is Adjunct professor at Università degli Studi di Milano and Centro Sperimentale di Cinematografia, teaching digital film restoration and digital media conservation. She has been part of the Organining Committee of many national and international conference, she is Associate Editor of Color Culture and Science Journal and member of the Italian color group (Gruppo del Colore).

Arianna CrespiUniversity of Milan (Italy)

After a Bachelor degree in Physics, she graduated in Cultural Heritage Conservation Science. She’s a maths and physics teacher and the co-creator of Photo FiRe2, the database for film restoration developed during her master thesis project.

Giulia MorabitoUniversity of Milan (Italy)

After the master degree in Cultural Heritage Conservation Science, currently she is a PhD student at the departement of Earth Science at the University of Milan. Her research interests range from the study and restoration of photographic and cinematographic material and the development of new analytical techniques for the material investigation.

Beatrice SartiUniversity of Milan (Italy)

Beatrice Sarti is currently a PhD student at the University of Milan (Italy) at the Department of Computer Science, after obtaining a Master degree in Cultural Heritage Conservation Science. She performed a master degree thesis in collaboration with the Xlim research institute (Poitiers) concerning with the Hyperspectral Imaging characterization for pigment identification on ancient paintings. Her PhD research is focused on film restoration with a particolar interest in Chemical and Physical Diagnostic Analyses, Data Digitization, Image Processing and Enhancement.

Alessandro RizziUniversity of Milan (Italy)

Alessandro Rizzi is Full Professor at the Department of Computer Science at the University of Milano (Italy), teaching Multimedia, Colorimetry and Film restoration. He is doing research since 1990 in the field of digital imaging with a particular interest on color, visualization, photography, HDR, VR and on the perceptual issues related to digital imaging, interfaces and lighting. He is the head of the MIPS Lab at the Department of Computer Science. He has been one of the founders of the Italian Color Group, Secretary of CIE Division 8, IS&T Fellow and Vice President. In 2015 he received the Davies medal from the Royal Photographic Society. He is co-chair of the IS&T Conference “Color Imaging: Displaying, Processing, Hardcopy and Applications”, Topical Editor for “Applied Color Science” of the Journal of Optical Society of America A, Associate Editor of Journal of Electronic Imaging, member of several program committees of conferences related to color and digital imaging, and author of about 400 scientific works.

Submitted: 2021-06-21 – Revised version: 2021-10-28 – Accepted: 2021-11-12 – Published: 2021-12-20

Abstract

Throughout history, thousands of different kinds of film formats, emulsions and colour systems have been used, to reproduce natural colours and as mean of expression for specific characters and scenes. In recent years, the advent of digital mediums has improved post-production technology and the film restoration process. Moreover, the support migration exposed different issues, such as colour reproducibility in digital systems that present the same problems as those posed by analogue. Furthermore, the lack of physical information on film dyes, emulsions and sensitivity renders the reconstruction of different film process colours impossible, and so too their comparison with the analogue or digital copies. In this contribution we aim to increase film experts’ and restorers’ awareness of colour reproduction problems that always affect the practice of digitization and digital restoration, and we propose the contribution of Fire2, an open access database of cinematographic and photographic technical data and materials, to support the work of conservators, restorers, and researchers.

Keywords: Film Restoration; Film Database; Film Colors; Sensitometry; Audiovisual Memory.

1 Introduction

Throughout history, filmmakers have used thousands of different kinds of film formats, emulsions and colour systems; from hand colouring to modern chromogenic positives, different dyes, emulsions, and filters have been used to represent colours on films. Nicola Mazzanti’s experience at the laboratory “L’immagine ritrovata” and at the Royal Belgian Film Archive (Cinematek) gave him the opportunity to collate numerous technical data on distinctive, historical colour films. He then tried to assess which ones modern digital technologies could correctly reproduced. Mazzanti catalogued around 280 colour supports and he found out that just 2% of them could be digitally reproduced (Mazzanti 2019). The low amount of reproducible films was also due to the historical lack of sensitometry documentation and technical reports, but also due to the physical limitation of recent emulsions and modern digital technologies for reproducing older ones, as confirmed by the experimental work of Harald Brendel from ARRI (Brendel 2005).

In recent years, the advent of digital mediums has improved post-production technology and the film restoration process. Due to the high manageability and reversibility of digital intermediate, many archives made huge digitization campaigns to make copies of stored films more accessible and to increase their usability. This support migration could be a great advantage in film historical research, philology and material sharing, but has prompted different issues regarding the reliability of the digital medium, since colour reproducibility in digital systems present the same problems as the those in analogue. In this context, the lack of physical information on film dyes, emulsions and sensitivity makes the reconstruction of different film processes, and therefore their comparison with digital mediums, impossible (i.e., the set of colours which can be reproduced by a specific device/media). For this reason, today visual comparison between films and digital media is more commonly used. Visual comparison has some practical advantages, but its main limitation is that it cannot precisely and objectively identify how much the reproduction and the original differ. If colorimetric models and simulations are not available, it is practically impossible to precisely reproduce a film gamut.

The issues linked to film digital migration have consequences in many fields of film studies, from philology to conservation, and require an in-depth analysis of all aspects related to the change of the film material. In this contribution, we will focus mainly on the technical issues relating to digital film reproduction, since our aim is to increase film experts’ and restorers’ awareness of issues with colour reproduction problems, and to proposing Fire2, an open access database of cinematographic and photographic technical data and materials, as a potential solution for supporting curators, restorers and researches in their work. With this project we aim to not only build an online open-source dataset that is accessible for everyone, but also create an online knowledge-sharing space, aimed at promoting the creation of a multidisciplinary network of competencies.

2 Colour Reproduction and Simulation

Before discussing digital colour, colorimetry, and video editing, it is fundamental to specify that all the film copies that the film restoration process can produce (both analogue and digital) can never replace the original copy. This statement might seem obvious, but in the actual practice of film restoration and digitization it is not. The most evident fact is that, today, the digitization process, and its related advantages, are leading to a transposition of many materials from analogue to digital, often without a proper colour management. The issue in this process is that, in many libraries and archives the films are sent to other experts or laboratories already digitized, to be studied, analysed, or restored.

In this context, we tend to live in limbo, where we know that a digital image (a matrix of numbers) is intrinsically different from the original analogue image (a combination of colourants from an inert base), but since the digital media is easily accessible and manageable, we wrongly consider it as a reliable substitute. Anyway, as Nicola Mazzanti would say “A horse is a horse, and a camel is a camel” (Mazzanti 2019), meaning that the analogue film and the digital media will always be different, also if they can be used for similar proposals. The biggest issue in considering the analogue and the digital as interchangeable media is the loss of materiality of the film, but also the loss of the data’s integrity.

Nevertheless, cinema has always been based on reproduction, in fact it is not like other kinds of tangible cultural heritage e.g., paintings or statues, which are meant to be unique. Cinema is meant to spread like a savannah fire, not to remain closed in an archive or a museum. Consequently, copies and reproductions are part of cinema and film restoration, but only if they are produced within a coherent technological system. Every technological system developed from the Lumiére’s and Edison up until today was characterized by specific film stocks, cameras, printing machines, projection technologies, and each one produced a different final image, which was the result of the interaction of all those elements. If today we take just a part of the original technological system, like a projection technology (e.g., the carbon lamp), and we use it on modern films, we cannot obtain a coherent reproduction by this incoherent system, but just a simulation. Therefore, the only way to obtain a coherent reproduction of a film, is to completely use the same technological system. To better understand this concept, Figure 1 shows the sensitivity curves of two different film stocks. Here, the dotted lines identify the sensitivity peaks of yellow, magenta and cyan emulsions, which are clearly different, meaning that the same image obtained through those two stocks will differ and colour aberrations may occur. In the same way, the use of a digital medium in the film restoration workflow could produce errors in colour reproduction, since not all of the modern displays and projection devices can reproduce the colours of an analogue film. Figure 2 shows an example of this concept; in fact it displays the CIE Chromaticity Diagram coordinates (CIE 1932, Lindbloom 2017) of a CRT monitor's primary RGB lights, a DLP projector and the RGB and CYM colours of a typical motion picture print film projected with a Xenon lamp. Looking at the diagram it is possible to see that both the monitor and the projector cannot correctly reproduce the film’s colours, which has a wider gamut. In the practice of film digital acquisition and restoration this means that a full set of colours (e.g., in the green-cyan region) cannot be correctly reproduced by monitors and projectors, thus they are changed and approximated. As a consequence, the film aesthetic is altered, and the digital medium is an intrinsically different version.

Figure 1. Sensitivity curves of two different film stocks. Figure reproduced from (Brendel, 2005).
Figure 2. The CIE chromaticity diagram which represents the gamut of a CRT monitor (solid lines) and the gamut of a DLP projector (dashed lines). The asterisks mark the locations of the red, yellow, green, cyan, blue, and magenta colour of a typical motion picture print film projected with a Xenon lamp. Figure reproduced from (Brendel 2005).

Consequently, cinema is reproduction, but it is very hard to precisely reproduce its visual aura, the colour process and its chromatic characteristics. The only operation that we can do is to simulate the perception of the original characteristics of the film (Plutino and Rizzi 2020). Currently, even if different attempts to reproduce colour film perception have been made (Gschwind and Frey 1997, Chambah, Besserer and Courtellemont 2002, Chambah, Besserer and Courtellemont 2001, Chambah and Rizzi 2010, Berolo et al. 2012, Plutino et al. 2019), we do not have scientific and mathematical models on which we can base the film simulations (both in analogue and digital) for most characteristics, including colour. Thus, even today, where colours are concerned, we have only a few or even no information about the original colours of hundreds colour systems used across time (Flückiger, Hielscher and Wietlisbach 2020, Flüeckiger 2012). Consequently, we cannot map the original analogue colours into modern digital systems (e.g., DCP), and we can only make visual (and subjective) comparisons (Plutino 2020). This approach is the most commonplace in all the biggest restoration laboratories, and has some advantages, but also some limits. Firstly, this approach depends strictly on the expertise and competence of the colourist, but also on the instrument available to see and observe the originals and the copies. In fact, today the use of a prestigious (and expensive) scanner or software seems to be the only considered parameter when evaluating the quality of a digitization. In the real digitization practice there is always the constant need to monitor the quality of the results, to regularly calibrate the instruments and their components, and to keep the software upgraded. Without a robust and continuous maintenance, upgrading and evaluation, the acquisitions will be worthless, the restoration uncontrolled and the projection misrepresented. For example, if we consider a stencil colouring made with aniline-based dyes from the 1924, digitized through a Northlight scanner and encoded to jpeg 24bit, what would be the best way to project it in order to simulate the original film? The answer is that this image does not match the analogue film colours (either on film or in digital), thus we cannot simulate the original film. From a technical point of view, this is caused by the different methods which are used to produce colours, from the aniline subtractive process on the film, to the additive synthesis when the film is projected and from the colour signal transduction (i.e., from physical spectral curve to a matrix of numbers) to the additive process to generate the colour signal from red, green and blue lights. Thus, the signals generated, and the process involved are so complex and so varied that, in the majority of the cases, we do not even have the data nor the instruments to measure exactly how much they differ.

In this complex and uncertain context, we aim firstly at raising the awareness of restorer and film expert on the colour reproduction problems which always affect the practice of digitization and digital restoration. Thus, the cooperation between film experts e.g., archivists, restorers, humanities researchers, historians, with technicians, scientific researchers and conservation scientists is fundamental to produce new approaches to film restoration. Furthermore, we aim to provide a preliminary instrument, called Fire2, which could support and increase the cooperation among experts.

3 Film Databases

The Fire2 project is part of a broad overview of existing databases and organizations, which aims to preserve and classify old film materials. The most important institutions are the National Film Preservation Board (Library of Congress 2021), operating in the United States, and Lost Film (Deutsche Kinemathek 2021), a part of the Deutsche Kinemathek based in Berlin. Thanks to those organizations, it has been possible to recover, catalogue and make available to the public thousands of films considered lost, using modern technologies. E.g., the CollectiveAccess software (Whirl-i-Gig 2021) used by Lost Films, which allows the user to update and complete the archive film data with textual documents, photographs, audio, and videos in a virtual space. Furthermore, Lost Films provides users with a set of unknown photographs and videoclips, to support identification involving the public.

In addition to these major organizations, many other cinematheques and libraries have their own collections and are working to digitize and share them through diverse methods. The foundation of ACE – The Association of European Cinematheques (Association des Cinémathèques Européennes), is an outstanding example of collaboration among institutions, and has facilitated connections between 49 European national and regional film archives (Association of European Cinematheques 2021). This association, as a branch of FIAF (Fédération Internationale des Archives du Film / International Federation of Film Archives), has the main role of safeguarding European film heritage in order to guarantee its survival and ensure its visibility through the use of new technologies.

In this context, many projects have been made to guarantee public access to the audiovisual heritage collected and preserved by archives and libraries, e.g., European Film Gateway (Artini et al. 2012) and Europeana (Petras et al. 2017). Other interesting examples of online dataset of audiovisual material are provided by BFI (The British Film Institute) (BFI 2021), which made its wide collection of movies and videos available online for its users, and by EYE (EYE 2021) which has digitized about 15% of its collection, now accessible via their YouTube and Open Beelden channels. In Italy, the Turconi Collection Database is a project powered by Cineteca del Friuli and Le Giornate del Cinema Muto and has a record of 23,491 clippings collected by Davide Turconi, from the Josef Jove Collection and other sources (Cineteca del Friuli 2017). The database contains mostly 35mm nitrate film frame clippings from the years of the early cinema, which are now available online (Usai e Yumibe 2011).

The associations, institutions and projects presented in this section are just an overview of the many different digital databases which improve film materials’ accessibility. Alongside the most important archives and libraries there are a plethora of institutions and private collections which are still inaccessible. Recently, the effects of the COVID-19 pandemic in many countries led to heavy digitization campaigns that improved the accessibility to archival materials, but there is still a lot of work to be done.

Considering the recent evolution of access to digital materials and the problems in film simulation presented in the previous section, nowadays there is a lack of information regarding film technical data. Alongside the official databases there are many amateur websites collecting cinematographic and photographic images, but without providing information about the original materials, the film process, the acquisition pipeline or the potential restoration workflow. In this way, the film experts, cannot be sure about the reliability of the frames or of colours, because no singular technical information is provided. Furthermore, even if some data about the film process and materials are provided, it is quite impossible to retrieve and find the technical spreadsheets concerning a specific film, because film production companies do not have online open-source databases. Thus, a lot of technical information about film sensitivity, emulsion densities, colourants, photographic development processes or even film acquisition conditions have been lost, and this makes film reproduction even harder (if not impossible). Even if many online film archives exist and are accessible to the public, there is still the need to create databases and sources to access the technical film data, in order to support the work of the restorer in simulating the original film tones and colours.

The Fire2 project inserts itself into this context, aiming to provide an open-source platform to all film restoration experts, where the available film technical data are collected. In this way, it would be possible to have original technical data to support the research in film restoration and open new directions for a more objective approach.

4 Fire2 Project

In the previous Section we presented the many online databases which aim to promote accessibility to different audio-visual materials, but many other works have been published in order to catalogue and collect examples of alternative film processes (Flüeckiger 2012) or different film stocks (Filmstocks.info 2021). Nevertheless, even if some efforts have been made, the films’ technical information concerning sensitometry, densitometry or colour reproduction are often unavailable and completely lost (Mazzanti 2019).

Starting from this need, in Fire2 project, we have laid the foundations for the creation of a new open-source database of film technical data, available for every researcher. Therefore, this project aims to give the researchers access to all of the films data available; promote restoration that is more faithful to the original materials; collect and archive technical information; encourage all film experts to share their research; and create a network between experts from different domains.