Finding references to smell in artworks

Identifying visual references of olfactory phenomena in artworks is an important way to uncover how Europe may have smelled in the past and how smell was represented. The computer-vision team of the Odeuropa project is currently working on methods which would automatically extract these references from various large collections of European artworks by applying, modifying, and extending state-of-the-art object detection methods. In order to collect and extract these olfactory references using computer recognition, it is necessary to first identify how smell is visually represented or depicted in historical artworks.

To provide an example of how this works, we used the print Smell (1581-1656) by Nicolaes de Bruyn, which is currently housed at the Rijksmuseum in Amsterdam.

Smell (1581-1656) by Nicolaes de Bruyn

In the sixteenth century, the pairing of a woman with a dog was used as a visual depiction or personification of the sense of smell. Since the object detection method was able to identify the dog and the woman, this would seem like an effective system. However, there are certain challenges which come with this detection. Firstly, not all pairings of people or women with dogs are ‘olfactory’, for example in other centuries a dog on the lap or feet of a woman represents fidelity, as seen in Jan van Eyck’s Arnolfini Portrait (1434).

Jan van Eyck’s Arnolfini Portrait (1434)
This presents us with the challenge of distinguishing when a dog is or is not ‘olfactory’ in nature. A second challenge is that the olfactory gesture of the woman smelling the flowers was also not detected by computer recognition. This poses further limitations on detecting olfactory elements in paintings.

Many olfactory-related narratives can also be found in the Bible, the Sacrifice of Noah (Genesis 8:20) for example. The print, Sacrifice of Noah after the Flood by Casper Luyken, shows Noah creating a burnt offering of animals, combined with the usual “Covenant of the Rainbow” in the background.

Sacrifice of Noah after the Flood by Casper Luyken

These types of olfactory narratives reveal more limitations of existing object detectors, while the people and animals were easily detected but the rainbow and cloud of smoke were not, hence overlooking the olfactory element of the artwork. This could be because these object detection systems are limited to the data with which they have been trained, leading to two problems. Firstly, since the detectors are trained with photographic data, their effectiveness decreases when applied to images with an artistic style such as historical paintings and prints. Secondly, it could be that certain objects (like smoke and rainbows) were either underrepresented or not at all part of the detector’s training data.

In order to tackle these issues of computer recognition, we will apply and modify domain adaptation techniques in order to improve the detection abilities on artistic image domains. After implementing a working object detection system, we plan to incorporate art historical knowledge which would also enable our system to recognize complex and context-specific olfactory references.

Seeing by Smelling – How to Enhance the Experience for Blind and Low Sighted People in a Museum of ‘Visual’ Art

This post by Caro Verbeek originally appeared on Futurist Scents

In 2015 I embarked on one of the greatest adventures of my life. IFF (Hilversum), the Rijksmuseum (Amsterdam) and I (Vrije Universiteit Amsterdam) reconstructed a dozen historical and other scents for our joint project ‘In Search of Lost Scents’. All scents were related to art works from the so called ‘Highlight tour’: a unique project combining the best of (very different) worlds.

Right before the lockdown (due to corona) I was able to introduce some of these scents to a group of blind and near sighted people during a tour at the Rijksmuseum, organized by Hannes Wallrafen of Stichting Geluid in Zicht and Cathelijne Denekamp. Denekamp is manager of accessibility in the museum and is convinced the senses can play an important role in inclusivity:

“In order to make objects come to life, touch and smell are essential tools for blind people or people with low vision. The Rijksmuseum is considered a very visual museum. As a museum we acknowledge our responsibility of giving blind people or people with low vision access to art and history without using their eyes. Scent and story-telling enable us to do that.”

Olfactory tour for blind and near-sighted people at the Rijksmuseum.
Olfactory tour for blind and near-sighted people at the Rijksmuseum. Some (including me) are wearing especially designed scent necklaces which are a co-creation and -design by Caro Verbeek, Justus Tomlow and Bernardo Fleming (IFF).

Understanding rituals by the sense of smell – The Adoration of the Magi

We started our multi-sensory tour in the dimly lit vaulted rooms that store medieval and renaissance art works. We halted at the painting ‘Adoration of the Magi’ by Geertgen tot Sint Jans. The artist confronts us with lavishly dressed men holding precious gifts in front of a new-born child sitting on the lap of his mother. After this description the participants recognized the biblical story of the three kings offering frankincense, myrrh and gold to Jesus without even hearing the title of the painting. Not mentioning such details leaves more room for the mind to explore and imagine.

Frankincense and myrrh were meant as burnt offerings to pay honour to kings and gods. In antiquity  resins were burnt ‘per fumum’ (through smoke) to make its fragrant emanations reach the nostrils of divine beings. Every contemporary viewer of this painting would have understood that by offering these fragrances these royals symbolically acknowledged the divine and profane superiority of Christ; a meaning that got lost somewhere over the past centuries. And so did the meaning and aromatic quality of one of the scents.

The participants were enabled to actually smell myrrh as part of a story-telling technique. Its aromatic quality (bitter) is connected to its historical meaning: besides being an offering to a god, the bitterness of this resin was considered prophetic and referred to the future suffering of Christ. This means that sensory engagement can actually lead to a better understanding of past rituals. Here’s what participant Emilie De Lanoy Meijer recalled months later:

“Because of the scent of myrrh I instantly felt transported to the story and time it took place.”

Scents are in fact known to elicit intense historical sensations, according to neuro-scientist Richard Stevenson; even more so than images or sounds.

Geertgen tot Sint Jans, Adoration of the Magi, 1480 – 1485.
Geertgen tot Sint Jans, Adoration of the Magi, 1480 – 1485. Two of the gifts offered to Jesus were scents. Myrrh literally translates as bitter and smells accordingly.

“As a fully blind individual I do not feel any connection to paintings, but rather to stories”

We then took a small detour to a so-called ‘pomander’ in the department of the ‘special collections’. Pomanders are metal jewels filled with fragrances, worn by people of high status such as doctors and lawyers. They were hung from a chain or ‘chatelaine’ so the scents could be taken to the nose at will. Fragrances were supposed to help protect people from diseases, which were thought to be caused by bad smells (think of ‘malaria’ which literally means ‘bad air’). Baring in mind the story surrounding the three magi: a pomander could also function as a prayer nut which contents were an offering or wordless prayer in itself.

Read more