On creating benchmark datasets

In the Odeuropa project, we are creating several benchmark datasets. These datasets are used to evaluate how well our computer vision or language technology tools perform on a certain task. Last week, Stefano Menini presented our text benchmark of 6 languages at the LChange workshop in Dublin, held in conjunction with the Association for Computational Linguistics’ Annual Meeting:

The dataset contains parts of texts in English, Italian, French, German, Dutch, and Slovene that have been marked by humans for references to smell. For each sentence, we don’t only mark the fact that there is a reference to a smell, but also what emotions are evoked by the smell, what location the smell is perceived at, any qualitative remarks (did the perceiver like the smell or not) etc. The annotation format was previously presented in the paper “FrameNet-like Annotation of Olfactory Information in Texts“.

The benchmark contains over 20,000 annotations and spans 4 centuries (1620 – 1920) across 10 different text genres. This allows us to investigate how smells are referenced in different settings over time. Historians worked with computational linguistics to provide a historical background to the linguistic aspects of smells that are investigated. We hope that linguists and historians alike will find it useful.

You can find the paper at: https://aclanthology.org/2022.lchange-1.1/

The dataset is openly available so other researchers can also evaluate their olfactory information extraction tools against it.

 

Smell, Latin, and Authority

by Aron Ouwerkerk

Besides the analysis of historical texts written in several different European vernaculars (see for example Sanne Steen’s post about annotating Dutch texts), Odeuropa also looks at Latin sources dating between the sixteenth and eighteenth century. When hearing Latin, most people will immediately think about the ancient Romans, and perhaps recall the names of famous classical authors like Virgil, Ovid and Cicero. The authority of these writers looms large over the literary history of Europe. This especially holds true for Neo-Latin literature, that is the texts that were written in Latin during the early modern period (between c. 1500 to around 1800).

The Neo-Latin texts that we study nicely illustrate the widespread importance of these literary giants from antiquity. Not only in literary texts like Erasmus’ Adagia (1536) do we encounter their names and (sometimes hidden) references to antiquity, but also in medical and scientific treatises many authors use their work to show their learnedness and support their arguments when it comes to smell. In a rather technical and dull treatise by Andreas Repplerus (Disputatio physica de odoratu, 1703), for instance, the author points out that “between so many thousands of people we seldom find two people who look exactly the same, like once in the story Amphitruo looked similar to Jupiter, and Sosia to Mercury” (inter tot hominum myriades, rarissime conspiciantur duo, ita prorsus sibi similes, ac olim in fabula Amphitruoni erat Jupiter, & Sosiae Mercurius). He refers to a comedy by Roman playwright Titus Maccius Plautus (254 – 184 BCE), in which the gods Mercury and Jupiter change their appearance to deceive the friends and relatives of two mortal men. The author then continues to make his point about smell: “therefore, it automatically follows that emanations that for one cause a light trembling and tickling in the olfactory organs, for another could cause a more vehement sensation” (Posito autem eo, […] sponte sequitur, quod eadem effluvia, quae apud unum hominem lenem tremorem & titillationem in odoratûs organis excitant, apud alium e contrario vehementiorem motum […] producere […] queant).

Apart from referring to classical texts, our medical and scientific authors often use the authority from other writers too. This way, they try to make their arguments more convincing. On the excellent sense of smell of dogs, for example, an author named Fridericus Koeler writes in 1794:

“Here follows an example told to me by the eminent teacher Blumenbach [Johann Friedrich Blumenbach, 1752-1840]. About two hundred years ago, a noble Swiss man named De Zollikofer who was traveling from Altenklingen Castle to Paris, ordered his beloved dog to be locked up at home. After two weeks, however, the dog got free, and followed his boss for a hundred and twenty leagues (60 miles) on the roads, until he found him in Paris.”

Exemplum mihi hic adferre liceat, quod ex ore Ill. Blumenbach, praeceptoris pie colendi hausi. Ante ducentos circiter annos nobilis quidam Helvetus De Zollikofer ab arce sua Altenklingen ad Lutetias Parisiorum iter faciens, canem dilectissimum domi includere iussit. Iste vero duabus elapsis septimanis libertatem nactus, per viarum trivia dominum centum et viginti leucis (60 Meilen) distantem persecutus est, donec Parisiis illum invenit.

While science from the seventeenth and eighteenth century (the period of the Enlightenment) is often thought of as moving away from traditional authority and toward empiricism, passages like these from above ask for some nuance. The earlier mentioned author Repplerus certainly valued experiments, as he wrote somewhere: “But this is my own conjecture, and if anyone could demonstrate that it opposes healthy reasoning  and experiments, I will reject it more easily than I accepted it” (Sed haec mea saltem est conjectura, quam si ullo modo cum sana ratione, & experimentis pugnare, aliquis mihi demonstraverit, majori facilitate rejiciam, quam admisi). At the same time, however, many early modern scientists still engaged with classical authors when analyzing and thinking about smell. Moreover, the authority of ‘eminent scholars’ continued to be used to turn anecdotes into arguments. We may end with a final passage from the same ‘empiricist’ Repplerus:

“If the wind is blowing in the opposite direction, we perceive smells breathing forth from opposite regions with more ease and strength than with tailwinds. This is what the most eminent Kenelmus Diby, a courtier of England and chancellor of Great Britain, said. Sailing by the coastline of Spain, he had observed three or four times that the sailors could flawlessly grasp that they were still another thirty or fourty miles away from certain regions, by the most fragrant smell of rosemary. He also added that he himself had perceived this smell equally strong, as if he were holding a plucked twig of rosemary in his hands. But note that the most illustrious man also pointed out that he enjoyed headwinds.”

Contrario nobis vento, odores e regione opposita spirantes, facilius & validius percipimus, quam vento nobis secundo. Sic illustrissimus Kenelmus Digby, Angliae quondam Comes, & Magnae Britanniae Cancellarius, refert: se maritimas Hispaniae oras praeternavigando, ter vel quater observasse, nautas, duobus vel tribus ante diebus, quam in conspectum istarum terrarum proveherentur, ex fragrantissimo rorismarini odore, ad unguem quasi tenuisse, quod triginta vel quadraginta adhuc milliaribus ab istis regionibus remoti essent additque se ipsum quoque hunc odorem aeque valide percepisse, ac si avulsum rorismarini ramum manibus tenuisset; hoc tamen simul notat illustrissimus vir, quod vento tum temporis usus sit adverso.

In conclusion, the Neo-Latin olfactory literature included in Odeuropa displays an enormous wealth of historical anecdotes, (pseudo-)scientific arguments, emotions, and attitudes toward smell. As a language that was used widely across Europe until fairly recently, this literature therefore proves an invaluable source to incorporate in our ongoing effort to collect, analyze, and safeguard our European olfactory heritage.

 

Two types of rosemary (rosmarinum silvestre and rosmarinum silvestre minus). From: Pietro Andrea Mattioli, Commentarii in VI libros Pedacii Dioscoridis Anazarbei de medica materia (1598), p. 576

 

Odeuropa Team Milestone: Our First In-Person, Hands-On Meeting

 

Parisian street art: “Pendre au nez. Les murs ont des oreilles” or “Hang onto the nose. The walls have ears.” Photo: Inger Leemans.

After more than nine months of remote working due to the pandemic, we finally brought part of the Odeuropa team together. This October, 11 Odeuropa team members, representing all work packages and almost every project partner, met in Paris to smell things, co-create annotated data, bridge gaps between core concepts, and even challenge each other to a game of foosball. This lockdown period has been a burden for all research teams, but for a project researching smells and olfactory heritage, the audiovisual-biased world of online working has been a severe challenge.

Odeuropa team members Lizzie Marx and Victoria-Anne Michel smelling books during the smell walk at the Bibliothèque Nationale de France. Photo: Inger Leemans.

We approached this first in-person gathering ‘nose-first’ with the help of our PhD student, Victoria-Anne Michel who organized an olfactory workshop and smell walk for the team. During the workshop, we were able to smell perfumes and raw materials which we then categorized and made associations with through our sense of touch. During our smell walk through the different spaces within the Bibliothèque Nationale de France (site Richelieu) we stuck our noses into a few books, old and new. While walking around the place, we first inhaled the general atmosphere, which sensory researcher Kate McLean calls “smell catching.” We then went “smell hunting” in specific spots. Through this exercise, we felt the contrasts of olfactory atmospheres, or “smellscapes”; from the sleek and modern Performing Arts reading room, the wax wooden rotunda, the futuristic corridor leading to the old-fashioned Manuscripts reading room, and, finally, the impressive, bright, cathedral-looking Labrouste reading room. The discussions that followed were a mix of poetic impressions, technical considerations like the different types of air control systems, and personal scent memories. With this, olfaction functions as both an emotional and spatial sense! Special thanks to the curators Sylvie Bourel, Hervé Grosdoit-Artur and Mathieu Lescuyer, without whom this smellwalk would not have been possible.

This trip not only allowed us to get to know each other and train our noses, but also to truly invest in interdisciplinary knowledge exchanges within Odeuropa and beyond. Throughout the few days, we were able to  meet with researchers, heritage groups and other parties involved in the perfume and scent culture industry.

Cecilia Bembibre in the flamingo room at the press launch of the Sensory Odyssey experience. Photo: Sofia Ehrich.

Some highlights from our trip:

  • We received a warm welcome at the Osmothèque – the historic perfume conservatory based in Versailles – discussing future collaborations around olfactory heritage with the President, Head of Scientific Committee and Communications director of Osmothèque (and smelling some of the nose-boggling perfumes and odorants safeguarded by the institute);
  • We attended the press launch of Sensory Odyssey, a multisensory immersive event in the Natural History Museum of Paris;
  • We met with the representatives of the Centre des Monuments Nationaux to discuss olfactory approaches to heritage representation;
  • We had drinks with the acclaimed French historian Annick Le Guérer, to discuss her participating in one of our future Smellinars, among other things (more to be announced later);
  • We visited the Voyages Immobiles exhibition to see how smells were presented and communicated in an event for general publics
  • We annotated 371 perfumed gloves in paintings;
  • Lastly, we had four trips together to the supermarket, ate loads of croissants and had some adventures with Vélib bikes.

Alas, three days was surely not enough for all our ambitions – so much is going on in the realm of olfactory heritage in France. So: we’ll be back!

 

Part of the Odeuropa team having lunch together at a Parisian restaurant. Photo: Marieke van Erp.