Cinergie – Il cinema e le altre arti. N.20 (2021), 7–25
ISSN 2280-9481

Comparative Analysis of Colour Film Style by Computational Means

Olivia Kristina StutzUniversity of Zurich, Department Film Studies (Switzerland)

She is a PhD student based in the ERC Advanced Grant FilmColors project at the Department of Film Studies, University of Zurich, under the supervision of Prof. Dr. Barbara Flueckiger. Her research focuses on the relation between historical colour film processes and their aesthetics from the 1890s to the mid-1930s, with a special interest in the application of digital humanities tools. She has been an active contributor to the Timeline of Historical Film Colors since 2016, and has completed archival internships at the EYE Filmmuseum in Amsterdam (NLD) and the BFI National Archive in Berkhamsted (GBR).

Submitted: 2021-06-15 – Revised version: 2021-10-20 – Accepted: 2021-11-20 – Published: 2021-12-20

Abstract

This paper uses two different colour versions of the early film The Life and Passion of Christ (La Vie et la Passion de Jésus Christ, 1907) by Ferdinand Zeccato demonstrate how computer-based techniques for analysing digitized film materials, using a combination of close and distant readings, can facilitate the ‘text’-critical, comparative analysis of colour film styles as practised in the fields of both film studies and film archives, a process that ultimately benefits their different research and knowledge interests. The case study illustrates the use of the annotation and visualization software VIAN, as well as the VIAN WebApp, by focusing mainly on its automatic colour analysis tools and its computationally produced colour film visualizations, which function as epistemic images. VIAN thereby becomes the scientific framework that provides the examiner with an objective, evidence-generating system for comparing colour films. The strategy of using contemporary digital humanities tools in these fields thus makes the invisible visible and the impossible possible.

Keywords: Digital Humanities; VIAN; Film Colours; Comparative Film Analysis; Visualizations.

Acknowledgements

This project has been funded by the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme, grant agreement N. 670446 FilmColors.

1 Introduction

The comparative analysis of film versions, or various editions of the same film, has traditionally been a ‘text’-critical philological method practised both in the fields of film studies and film archives. However, their paths and end goals are not necessarily the same. In very general terms, the film historian, whom I regard as a subcategory of the film scholar, is typically interested in the diachronic developments of the medium, following a tradition of thought focused either on style and aesthetics or on contextualization within film history. The film archivist, on the other hand, may have a background in film studies, for example, but is much more bound to the institutional framework of the archive, which illustrates a difference in focus regarding film as cultural heritage: film identification, film provenance, film documentation, cataloguing, film preservation, film restoration. The parameter of film colour, which is a fundamentally important object of comparative examinations, is thus both a film aesthetic and film philological category. In practice, however, the comparison of colour film versions is characterized by several problems: first and foremost, probably, the lack of scientific evidence. In this essay, I use the specific case study of Ferdinand Zecca’s early colour film The Life and Passion of Christ (La Vie et la passion de Jésus Christ, 1907) to demonstrate why contemporary digital humanities tools — here, the video annotation and visualization software VIAN — could be regarded as an effective extension of the professional’s traditional comparative toolboxes (Olesen 2017: 199–203, Bohn 2020: 210–11 ) for examining film colour. Thus, the case study of this film, on which much has already been published, reveals typical problems as well as possible solutions to them, of a kind that were not previously available using traditional methods. By focusing mainly on questions of style and aesthetics, I will show how VIAN can be used for powerful, computer-based, qualitative and quantitative comparative analysis of colour film style (and, at least to a certain degree, film ‘text’) in a way that ultimately benefits the research and knowledge interests of the two fields.

2 Digital Humanities, Methods, and Tools

Early colour films — that is, early films that juxtapose the applied colour film processes such as tinting, toning, hand-colouring, and stencil-colouring — are an illustrative example demonstrating the traditional methods and/or tools used in the fields of film studies and film archives for the comparative analysis of multiple film versions. This is because their editing structures, and thus their colour schemes, have varied greatly and from edition to edition — for reasons of historical film distribution and exhibition practices, film provenance, and film reconstruction, for example (Lameris 2017: 109).

A traditional film scholar, or a film historian for that matter, may compare the stylistic differences between early colour film versions based on alternative editions available as DVDs or Blu-rays, and by making use, for example, of the classicist-philologist transcription tool of the viewing log (Bohn 2020: 200–02). The latter has been a popular, descriptive film analysis tool since the 1970s and 1980s, one that pursued the goal of providing more evidence when trying to communicate audiovisual sensations, thus helping to better establish film studies as a university subject (Kanzog 1997: 11–2). It can be situated within other new empirical methods for researching film that were developed during this period as a means to formalize and quantify film aesthetics (Olesen 2017: 41–3). A key category, for instance, was the parameter of editing, a so-called ‘hard fact’ of the film ‘text’. Colour, on the other hand, is not. In film studies, it typically belongs to the category of film style, which denotes a formal, context-dependent system that refers on a textual level to the patterning arrangement of cinematic devices, and thus to a structured set of aesthetic systems and their overall function within a film ‘text’ (Bordwell 1985: 9–10). To date, however, film colour has remained difficult to grasp or verbalize. I will come back to this issue below. Yet a film scholar/historian’s use of DVDs or Blu-rays (i.e. digitized analogue colour film in our case) as source material for his aesthetic or historical analysis brings with it some other serious philological problems. Epistemologically speaking, digital and analogue materiality differ substantially. On a very basic level, the digital colour film image is determined by pixels and the bit depth, whereas the aesthetics of an analogue colour film image is characterized by its silver grain and/or colour components (Flueckiger 2012: 139–48). Furthermore, digitization processes also vary in different ways und thus significantly alter the aesthetics of the digital file (Flueckiger et al. 2018: 7–10). To make matters worse, many of these (home video) products very often disregard philological norms and ethical principles (Catanese 2014) by, for instance, not disclosing the exact material foundations or the chosen digitization process they were working with, thus destroying any kind of traceability of the film’s provenance (Flueckiger et al. 2020: 80). Nevertheless, the use of these sources is still a widespread practice today that is hardly reflected in film studies — often not even in film historical research. Of course, there are also those film historians who are lucky enough to be able to examine the actual historical film material closely on the film inspection bench at the film archive. In our case of the early colour film, this film historian may then strive to reconstruct the film’s genealogy (e.g. its premiere colour version) and its historical context of production and exhibition. The film archivist, similarly, also practises this traditional qualitative notion of ‘text’ criticism (Bohn 2020: 202–09) by running the nitrate prints, for instance, synchronized on a viewing table, which automatically counts the film frames. The differently coloured nitrate elements that characterize our early film’s colour scheme are thus amenable to a direct visual comparison. In total, the comparative analysis process in both fields is often a tedious task, one that is extremely time-consuming and therefore costly. The tools utilized can change depending on the area, institution, or even operator, which can influence the quality, uniformity, and thus the significance of the results. These inherent deficiencies in the practice and tools are particularly evident in the realm of film colours, which are not handled in a standardized manner. A basic problem is that of the categorization of colour: unlike editing styles, it cannot be regarded as a hard fact, and it is not a language (Flückiger 2011: 48–9). As an aesthetic phenomenon, colour is a function of subjective physiological and phenomenological human perception, which is additionally determined by other factors such as knowledge, culture, language, etc. (Zollinger 1999). Accordingly, colour cannot be expressed on an inter-subjective level by means of traditional ‘text’-critical methods. Depending on their expertise, the investigator can certainly determine the amount of colour process(es) deployed and their philological order throughout the film, but its mere technical listing, as a blue tone or a pink tint, for example, lacks accuracy and thus doesn’t denote the actual shade of that blue or pink precisely — that is, its hue, saturation, and lightness that characterizes the film prints. The qualia of colour is thus extremely difficult to put into words — for film scholars, film historians, and film archivists alike. But even a photograph of the film material (e.g. made with a mobile phone) cannot be used as a parameter for comparison, as these photographic processes are not standardized either. On the other hand, in conjunction with the VIAN WebApp (https://vian.app/), the desktop software VIAN provides the means to solve these problems.

Developed since 2017 under the aegis of the ERC Advanced Grant FilmColors (https://www.film.uzh.ch/de/research/projects/verbund/ercfilmcolors.html) and the Visualization and MultiMedia Lab at the University of Zurich, the software allows an in-depth investigation of large-scale video datasets on a micro, meso, and macro level (Flueckiger and Halter 2018, 2020, Halter et al. 2019). Alongside the manual annotation, classification, and (semi-)automatic segmentation of the films, VIAN operates with a (semi-)automatic screenshot manager, metric colour widgets, and other deep-learning tools that allow the determination of stylistic patterns of colour and the visualization of this aesthetic data along different parameters of choice, such as colour schemes, colour processes, and genres. It was originally intended for the analysis of film colours and narration but is currently being expanded to include other tools for analysing films (e.g. speech recognition, eye tracking). Consequently, the point is that, unlike traditional methods of comparative colour film analysis, the software can create actual scientific evidence, vividness, and immediacy by means of computationally generated colour film visualizations, functioning as epistemic images. Although VIAN operates on the same level of sense perception here, the visual representations provide the investigator with an objective system and thus a scientific framework for comparing colour films (Flueckiger and Halter 2020: 113), which supports the objectives of the two fields mentioned at the beginning. The possibilities offered by the software become especially meaningful when we considerthat the comparative study of colour film versions as described above is, like the method of comparative vision in art history (Dünkel 2008: 24–8), at its core a basic method of distinction (differentiation) and classification. Accordingly, the determination of a colour film’s style is a classification process that is initially based on comparisons from within a comprehensive corpus of films. Looked at from this perspective, it is no longer surprising that in digital art history, contemporary machine-learning techniques — namely, the visual classification of images using computer vision — have started to accurately classify the stylistic features of images, and thus to identify epochs, styles of painting, and artists (Saleh and Elgammal 2015, Seguin et al. 2016, Impett and Moretti 2017, Bell and Ommer 2018, Arnold and Tilton 2019). The algorithms operate by using pattern recognition and are trained on large-scale collections of images with similar visual features. As such, the analysis and visualization of style have become an automated process based on the concept of similarity and the ‘comparative vision’ of the computer, taking it beyond earlier diagrammatic approaches (Cortjaens and Heck 2014). These current developments in digital art history are framed by the recent calls for a ‘visual digital turn’ in the digital humanities (Münster and Terras 2020, Wevers and Smits 2020) and the increased application of computational visual analysis for cultural ‘data’(Olesen et al. 2016, Heftberger 2016, Manovich 2020, Arnold and Tilton 2020). In digital film studies, contemporary video annotation and visualization software such as VIAN, which have adopted complex deep-learning and machine-vision algorithms from the field of visual analytics, have recently started to catch up with these developments and begun to explore film content and style in new ways (Pustu-Iren et al. 2020). In doing so, they bypass earlier empirical or statistical methods of film (style) analysis dating from the 1970s and 1980s — which, as mentioned above, typically lack descriptive models for the sensuous features of images such as the dimensions of colour — or later, around 2005, the Cinemetrics tool developed by Yuri Tsivian, which investigates shot lengths as a means to quantify film style (Olesen 2017: 43–7). For the purposes of digital film analysis, however, one is forced to work with born-digital or, in the case of the early colour film, digitized film materials, even though they are, strictly speaking, less ‘philologically correct’, as explained above.

To visualize these problems and at the same time demonstrate the advantages of VIAN in the following case study, I draw, therefore, on two epistemologically different early colour film sources, namely a DVD with almost no given information provided about its provenance or digitization data and a manually photographed film print directly from the nitrate vaults of a film archive.

3 One Early Colour Film, Two Versions: The Material Sources in Comparison

The Life and Passion of Christ (1907) is a prestigious and well-known Pathé-Frères production. The French studio, which belonged to one of the early twentieth century’s leading film production and stencilling colourization companies (O’Brien 2012: 298–301), distributed the film in Europe and the US (Friesen 2016: 86–7). As the title of the film suggests, it is about the youth, the miracles, and the Passion of Jesus Christ. There are many reasons why this film was chosen for this analysis: as it is precisely the early films that are characterized by manifold colour schemes in different editions, they also tend to combine a spectrum of (applied) colour processes with diverse material-aesthetic characteristics, which is relevant to our study. Stencilling by Pathé, for instance, was typical of its historical context, and especially for this sort of prestige film, as colour produced by this ‘luxurious’ technique was regarded as a fitting embellishment for such culturally ‘valuable’ and prominent subject matter (Gaudreault 2016: 15). Moreover, the genre of the religious film, to which Zecca’s film belonged, was, due to the subject’s historical popularity, also especially suited to film versions or variants: in fact, Pathé marketed at least four versions of the film, in 1899, 1902, 1907, and 1913 (Blot-Wellens 2017: 17–8). And although the 1907 version is considered the best-documented (Boillat and Robert 2016: 27, Friesen 2016), the studies nevertheless lack illustrated colour references or in-depth colour comparisons. Last but not least, the choice of the particular film is also based on pragmatic reasons relating to the availability of different early colour film versions — i.e. as purchasable DVDs or Blu-rays — or to the much more limited accessibility of historical materials in film archives for a film scholar like me, who had the good fortune to do two trainee ships in the field of nitrate film (identification) as well and was thus granted access to this cultural heritage within the framework of the FilmColors project.

The first version is based on a 42-minute DVD release by Passport Video in 2004, accompanied by a musical score by Los Angeles composer Shawn Alan Klaiber. On the back of its sleeve, it claims to be a reproduction of a 1905 version of the — apparently — hand-coloured film. However, as Alain Boillat and Valentine Robert (2016: 31) have shown, the DVD’s source material was, in fact, based on the 1907 version of The Life and Passion of Christ. Moreover, it also becomes obvious, when analysing the film using VIAN, that the film is not hand-coloured but contains stencil-coloured, tinted, and toned segments alongside 36 orange-red tinted English title-cards.Typically, and as mentioned above, no further details about the film material or the digitization process (here, possibly from a telecine of a safety print) are given on the DVD either, thus making a ‘philologically correct’ investigation nearly impossible and complicating the analysis of the film’s aesthetics. This circumstance illustrates the difficult position in which film scholars find themselves even today. The DVD serves, therefore, as an exemplary (but insufficient) source for the comparative investigation. For the sake of simplicity, I will refer to the ‘DVD version’ in the following.

The source material for the second version of The Life and Passion of Christ is based on a 976.9-metre (about a 35-minute long) historical 35 mm coloured nitrate print preserved at the EYE Film museum in the Netherlands. According to the archive’s chief curator of silent film, Elif Rongen-Kaynakçi, the particular nitrate print arrived at EYE in 1957 as part of the Jean Desmet archive, which is one of the key components of the institution’s collection today (Lameris 2017: 37–40). According to the information provided in the archive’s catalogue, the film is supposed to be hand-coloured.1 However, the on-site inspection of the fragmentary five-reel film has shown that this is most likely a stencil-coloured and tinted nitrate print that also includes black-and-white Dutch intertitles, which may be a sign of an exhibitor’s intervention, as well as mostly orange-red tinted (and ultimately toned) Dutch title cards, some of which contain the famous Pathé logo, the rooster. For the sake of simplicity, I will refer to the ‘EYE version’ in the following. To work with the nitrate print in a digital humanities’ workflow, I photographed the entire colour scheme of the film — that is, all the tableaux, title cards, and intertitles — and fed the photos into the VIAN pipeline.2 The 252 individual high-resolution photographs of the nitrate print include the edge-marking area and were taken manually with a standardized photography set-up to maintain uniform lighting conditions (Flueckiger 2015: 27–9).Thus, in contrast to conventional scanning methods in film archives, the film frames were selected individually, but in a consistently uniform manner, as the main goal is to capture the entire colour scheme of the film for VIAN. A large selection of the photographs of the film is currently exhibited on the Timeline of Historical Film Colors (https://filmcolors.org/galleries/la-vie-et-la-passion-de-jesus-christ-1907/), the comprehensive web resource of historical colour film processes, which has been curated by Barbara Flueckiger since 2012 and is an integral part of the FilmColors project. The photographed nitrate print in VIAN serves as an example of how un-digitized archival film materials (i.e. most early films) can be easily absorbed into the software’s pipeline. Compared to the DVD version, therefore, the nitrate copy in VIAN, even if it has been digitized, can be regarded as a much more ‘philologically correct’ version for the following comparative analysis of colour film style.

4 Comparing Early Film Colour, Or How to Generate Evidence With VIAN

As mentioned earlier, I approach the material aesthetic investigation of the two film versions using VIAN primarily from the perspective of colour and therefore focus on some of the software’s automatic colour analysis tools in order to generate scientific evidence. I start with a qualitative analysis on a micro level (individual screenshots of temporal colour film segments), move to the film’s meso level (affording a distant reading of the film), and then briefly zoom out to a macro level in the VIAN WebApp, which takes a whole corpus of colour films into account.

4.1 Micro level

When investigating the aesthetic qualities and use of colour in The Life and Passion of Christ on a micro level, a comparison between two versions of the same tableau using selector-dependent colorimetry promises meaningful results. Colorimetry — the quantification of the perception of colour — is a computational colour analysis widget in VIAN that computes the colour feature of each frame using the three-dimensional CIE-L*a*b colour space (Flueckiger and Halter 2020: 76), which is perceptually uniform. It expresses colour by means of the L-axis (i.e. the perceptual lightness of colour): this is perpendicular to the colour plane a*, which displays the hue and saturation of greens and reds, and plane b*, which indicates that of blues and yellows. The widget thus makes film colour scientifically measurable and displays it relative to the human visual system. Selector-dependent colorimetry in VIAN conveys a frame’s colour scheme by means of different live-feedback visualizations, namely with the “Cluster Palette” and the “Space Palette”. Both are based on SEEDS superpixels followed by a subsequent agglomeration of said superpixels (Halter et al. 2019: 125). In the case of the Cluster Palette, the colours of one or more levels of the agglomeration tree are displayed using colour bars and frequencies conveyed by their relative width. In contrast, Space Palettes embed the agglomeration tree at one specific depth into the LAB colour space and employ a jitter effect to indicate the size of a cluster. The latter will be significantly reduced in the EYE version, since, in contrast to the DVD version, fewer frames (i.e. photographs as visual information) were provided to the software for the computational process. In VIAN, the two visualizations are directly juxtaposed with the respective selected frame to facilitate a comparison. If we use these tools to compare, for instance, the well-known tableaux of ‘The Last Supper’ in both film versions, their colour schemes, on the level of individual frames, are immediately apparent. The colour visualizations of the DVD version are marked by distinct pastel pinks or purples and dark browns with a cool undertone in an almost tone-on-tone manner, which, when focus is put exclusively on the Cluster Palette, are falsely reminiscent of a typical tinting or toning colour scheme (Fig. 1). Whereas this loss of colour diversity is certainly related to the quality of the DVD scan that influences the expressiveness of the visualizations, this example also illustrates how both the Cluster Palette and the Space Palette are needed to accurately visualize colour processes. Indeed, the Space Palette gives a more meaningful indication of the stencilling technique here, as the colours are plotted on different sections of the plane, suggesting that more than one dye was used in the colorization process. The visualizations of the EYE version, on the other hand,are distinguished by a more diverse colour scheme of warm, honey-infused browns and a very distinct apricot and purple, particularly evident in the colour bars of the Cluster Palette, which can also be retraced in selected frames, namely in the apostle on the left and in the wall hanging (Fig. 2).

Figure 1: DVD version: colour scheme visualization in VIAN of the stencil-coloured tableau ‘The Last Supper’
Figure 2: EYE version: colour scheme visualization in VIAN of the stencil-coloured tableau ‘The Last Supper’ (Eye Filmmuseum, Desmet Collection)

Although both colouring schemes appear different, they are still remarkably similar in terms of their basic choice of colours. Indeed, it could be that they only vary because of their material sources or their digitization processes respectively. If we look closer at the apostle on the left in the DVD version, a very faint orange-pink, like the distinct apricot in the EYE version, becomes visible. Could it be that the tableau of the DVD version is in fact based on an element of the EYE version, allowing us to draw conclusions about the provenance and restoration of the film? Indeed, the colour scheme of purple-browns is not reused in this manner in the rest of the DVD version and thus appears to be unique. Its ‘Crucifixion’ tableau, for instance, contains a classical combination of blue-violet and yellow — the ultramarine and gold of early colour films — which creates vivid complementary contrasts (Fig. 3). This colour scheme is used in similar ways throughout the film, the most similar being the tableau ‘Baptizing Christ’ (Fig. 4), indicating that in the DVD version the colour combination is a recurring stylistic element.

Figure 3: DVD version: colour scheme visualization in VIAN of stencil-coloured tableau ‘The Crucifixion’
Figure 4: DVD version: colour scheme visualization in VIAN of stencil-coloured tableau ‘Baptizing Christ’

After the death of Christ, the blue-violet-yellow colour scheme, however, suddenly changes to shades of azure blue (Fig. 5).

Figure 5: DVD version: colour scheme visualization in VIAN of tinted tableau ‘Jesus Taken from the Cross’

By plotting the hue in the blue-green section of the colour space only, ranging from very dark greens to turquoise and light cyan, the Space Palette now beautifully illustrates the ‘monochrome’ aesthetics of the tinting technique (which only varies in saturation and lightness in relation to the underlying black-and-white silver image). The particular distribution of colour dots once again allows us to deduce the process employed. In contrast, the tableau in the EYE version shows a similar or recurring cream-brown-grey-pink stencil colour scheme, as seen earlier in its tableau ‘The Last Supper’ (Fig. 6).

Figure 6: EYE version: colour scheme visualization in VIAN of stencil-coloured tableau ‘Jesus Taken from the Cross’ (Eye Filmmuseum, Desmet Collection)

In fact, when comparing the two tableaux of ‘Jesus Taken from the Cross’ in a qualitative close reading, we can not only see differences in colour, but also in the staging of the characters. Throughout the scene, the DVD version never displays the same frame as the EYE version, even though the set designs coincide. This means that there are not just two versions of the 1907 film, as Dwight H. Friesen (2016: 79) discovered, but probably (at least) three.

4.2 Meso and Macro Levels

At the meso level of film analysis, the focus turns to a visualization of the colour schemes of the complete films. In VIAN, there are several ways of doing this, but at this point I am interested in the ‘Time Palette’ and the interactive “Colorspace 3D”. The former, also called ‘Movie Barcode’, displays the film’s colour scheme over the entire screen time — on the x-axis — and is based on the clustering principle of the Space Palette using (selector-independent) film colorimetry (Halter et al. 2019: 124–5). By visualizing both copies using this method, the two film colour schemes become comparable on a very basic level (i.e. how colourful the versions are). The DVD version, for instance, is marked by a relatively diverse colour scheme that has a distinct number of green-blues and beige-browns, whereas the EYE version is (broadly) marked by pink-browns, and only four segments contain a large amount of blue (Figs. 7 and 8). In this sense, the Time Palettes confirm what we assumed earlier at the micro level.

Figure 7: DVD version: VIAN colour scheme visualization over the film’s screening time
Figure 8: EYE version: VIAN colour scheme visualization over the film’s screening time

The clustering method of the visualization, however, also allows us to draw conclusions about the relationship between the colour processes, the content, and philological order of the tableaux. The distinct blue parts at the end of the DVD version’s Time Palette, for instance, which denotes the aforementioned tinted sequence in ‘The Crucifixion’, can only be so blue if a large area of the film image is dyed in that colour, typically by stencilling an area like the sky that takes up almost half of the image, or by tinting. Throughout the DVD version, however, we find other blue segments — two at the beginning and in the middle, and a blue-greenish part about three-quarters of the way through and towards the end. The EYE version, on the other hand, only contains two large blue segments, approximately halfway and three-quarters of the way through. So, could the colour blue have been applied to the same two tableaux in both films? And were particular tableaux historically associated with a specific hue or colour process and are therefore recognizable as a distinct coloured segment at a meso level? Indeed, when retracing the blue segments in both copies at a micro level, the tableaux ‘Christ Walks on the Waters’ (Fig. 9) and ‘The Wonderful Draught of Fishes’ (Fig. 10) adhere to the common colour convention for water, whereas the tableau ‘The Kiss of Judas’ follows the convention for a night-time tint (Fig. 11). The aesthetics of the blue tint in the DVD version is thereby a typical example of the inadequate quality of many scanning methods with regard to certain colours (Flueckiger et al. 2018: 18–22).