On creating benchmark datasets

In the Odeuropa project, we are creating several benchmark datasets. These datasets are used to evaluate how well our computer vision or language technology tools perform on a certain task. Last week, Stefano Menini presented our text benchmark of 6 languages at the LChange workshop in Dublin, held in conjunction with the Association for Computational Linguistics’ Annual Meeting:

The dataset contains parts of texts in English, Italian, French, German, Dutch, and Slovene that have been marked by humans for references to smell. For each sentence, we don’t only mark the fact that there is a reference to a smell, but also what emotions are evoked by the smell, what location the smell is perceived at, any qualitative remarks (did the perceiver like the smell or not) etc. The annotation format was previously presented in the paper “FrameNet-like Annotation of Olfactory Information in Texts“.

The benchmark contains over 20,000 annotations and spans 4 centuries (1620 – 1920) across 10 different text genres. This allows us to investigate how smells are referenced in different settings over time. Historians worked with computational linguistics to provide a historical background to the linguistic aspects of smells that are investigated. We hope that linguists and historians alike will find it useful.

You can find the paper at: https://aclanthology.org/2022.lchange-1.1/

The dataset is openly available so other researchers can also evaluate their olfactory information extraction tools against it.

 

Follow Your Nose: A Guided Tour with Smell

Written by: Christina Kotsopoulou

On the 5th of April, 2022, at 19.00, Museum Ulm in Germany launched the olfactory guided tour, Follow Your Nose. A Guided Tour with Smell, which was designed in collaboration with Odeuropa and International Flavours and Fragrances (IFF). The aim of this tour is to pair historically informed scents with eight artworks from the museum’s permanent collection and engage visitors into a multisensory and ‘nose-opening’ experience with art. The launch was a two-hour event that included introductory talks from both Museum Ulm and Odeuropa, enlightening discussions with the perfumers of IFF who were present that evening and engaging short versions of the olfactory guided tour in English and German. 

All the scents of the olfactory tour at the museum office of Dr. Eva Leistenschneider where she welcomed the Odeuropa team before the launch started. Photo taken by Christina Kotsopoulou.
Dr. Eva Leistenschneider prepared Sofia Ehrich and Lizzie Marx with scented blotters and instructions on how the guided tours would be conducted by the three of them that evening. Photo taken by Christina Kotsopoulou.

Once the launch began, deputy director of the museum, Dr. Eva Leistenschneider, the museum’s director, Stefanie Dathe, and the mayor of Ulm gave a word of thanks to everyone involved in this project, explained the challenges of planning this project during the Covid -19 lockdowns in 2021 and emphasized on the importance of introducing smell into museums. On behalf of Odeuropa, Sofia Ehrich and  Lizzie Marx briefly spoke about the nature and goals of the Odeuropa project. Afterwards, the four guided tours that would be given in English and German were introduced as well as the IFF perfumers who were there that evening to answer people’s questions about the making of a scent. 

Stefanie Dathe, the director of Museum Ulm, giving a word of thanks to all the involved members of the project. Photo taken by Christina Kotsopoulou.

My experience of the launch started with an insightful discussion with the IFF perfumers in which I was able to learn their thinking process when creating a scent to pair with a painting. In particular, based on their perspective, it became clear that a perfumer’s approach depends first and foremost on the type of scent. Working with an abstract painting and thus, an abstract scent can be more creative for perfumers who can then use raw materials by colour to achieve the best result. On the contrary, when a scent is historically informed it must be based not only on the art historical background of the painting but also the historical period it was made to determine what kind of materials were available back then. Interestingly, the IFF perfumers worked with at least fifty raw material scents to achieve the final scents for the olfactory tour in the Museum Ulm mainly because of the malodorous aspects of a few paintings, such as the artwork of Dieter Roth, Kleine Landschaft that depicts the natural decay of food materials. However, the biggest challenge during their process of creating a scent for an artwork was meeting the expectations of the perfumer and museum. Therefore, the IFF perfumers emphasized on the importance of a good and constant communication between both sides during the whole creating process as well as on the instrumental role of physically experiencing the artworks that will be translated into scents: an experience that the IFF perfumers unfortunately could not have in the early stage of their thinking process due to the general lockdowns of Covid -19 in Germany.

The IFF perfumers’ table at the Museum hall with scent samples and books about the making of scents for the public. Photo taken by Sofia Ehrich.

The two English olfactory tours which followed the introductory talks of the launch were given by Lizzie Marx and Sofia Ehrich and each one of them included three different paintings. Depending on the artwork and how recognisable the theme was, both Lizzie and Sofia interacted with their small groups by asking them what was depicted every time and what scent could be linked with each artwork before handing participants the scented blotters and informing them about the art historical and olfactory aspects of the painting. It was only in the case of Ellsworth Kelly’s Orange Blue, that the group was asked  to smell the blotters in advance of seeing the artwork so that they could confirm whether or not they smelled any of the colours  after viewing the artwork. The public’s reactions to the question about what they smell each time was very positive and almost every participant tried to guess the ingredients of the scents with great success. The most interesting reaction, though, was when the public’s expectations would be fulfilled or not after seeing and ‘smelling’ a painting, particularly the malodorous choices of the collection. For instance, the scent of hell was expected to be more foul but  instead it was even found pleasant by a few participants. All in all though, the public responded very enthusiastically to the pairing of a painting with scent from the first moment saying that a “scent can convey more messages about an  artwork than the image alone”.

Sofia Ehrich giving an olfactory tour for the scent of hell in front of the painting Christ in Limbo. Photo taken by Christina Kotsopoulou.
Lizzie Marx giving an olfactory tour in English for the scent of tanned leather in front of the Portrait of Helena Schermar. Photo taken by Christina Kotsopoulou.

In sum, the launch of the olfactory guided tour, Follow Your Nose. A Guided Tour with Smell, at the Museum Ulm was a very enlightening experience about the public’s olfactory reactions and the IFF perfumers’ perspectives. Learning about a perfumer’s approaches and challenges when creating a pairing scent for an artwork, was a unique opportunity that the event offered and one that I particularly enjoyed due to my limited knowledge on the subject. Since the official opening of the olfactory guided tours to the public, according to Dr. Eva Leistenschneider, the tours have been very positively received especially by non-regular museum visitors  who found their experience with art more relatable through the scents. Hopefully, these olfactory tours will continue to be well received by the public and attract more curious participants from around the world during the next months that they will be conducted at the Ulm Museum! 

Please find further information about the available dates for the olfactory guided tour, Follow Your Nose. A Guided Tour with Smell, at Museum Ulm here.