Daniel Hershcovich

Tenure-Track Assistant Professor at CoAStaL, Natural Language Processing section, Department of Computer Science, University of Copenhagen, Denmark.

Research interests:

Adapting and generalizing language models and data across cultures and languages.
Combining explicit representation of natural language semantics and pragmatics into neural models and their analysis.
Promoting socially and environmentally responsible development of language technology.
Investigating language and identity with respect to food and promoting sustainable and diverse diets.

Our paper What's the Meaning of Superhuman Performance in Today's NLU? won an Outstanding Paper Award at ACL 2023!
Our paper Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical Study was picked up by several Danish media outlets, including Politiken, Børsen, Ekstra Bladet, P1 Morgen, TV 2!

Haptic Captioning: Using Natural Language Models to Design Haptic Experiences. Villum Experiment project with Hasti Seifi.
Mining the Meaning. Data+ project with Jens Bjerring-Hansen (Department of Nordic Studies and Linguistics).

(Google Scholar, Semantic Scholar)

Probing for Hyperbole in Pre-Trained Language Models.
Nina Skovgaard Schneidermann, Daniel Hershcovich and Bolette Sandford Pedersen. ACL Student Research Workshop (SRW) 2023.

proceedings pdf abstract bib

Hyperbole is a common figure of speech, which is under-explored in NLP research. In this study, we conduct edge and minimal description length (MDL) probing experiments for three pre-trained language models (PLMs) in an attempt to explore the extent to which hyperbolic information is encoded in these models. We use both word-in-context and sentencelevel representations as model inputs as a basis for comparison. We also annotate 63 hyperbole sentences from the HYPO dataset according to an operational taxonomy to conduct an error analysis to explore the encoding of different hyperbole categories. Our results show that hyperbole is to a limited extent encoded in PLMs, and mostly in the final layers. They also indicate that hyperbolic information may be better encoded by the sentence-level representations, which, due to the pragmatic nature of hyperbole, may therefore provide a more accurate and informative representation in PLMs. Finally, the inter-annotator agreement for our annotations, a Cohen’s Kappa of 0.339, suggest that the taxonomy categories may not be intuitive and need revision or simplification.
On Evaluating Multilingual Compositional Generalization with Translated Datasets.
Zi Wang and Daniel Hershcovich. ACL 2023.

proceedings preprint pdf abstract bib

Compositional generalization allows efficient learning and human-like inductive biases. Since most research investigating compositional generalization in NLP is done on English, important questions remain underexplored. Do the necessary compositional generalization abilities differ across languages? Can models compositionally generalize cross-lingually? As a first step to answering these questions, recent work used neural machine translation to translate datasets for evaluating compositional generalization in semantic parsing. However, we show that this entails critical semantic distortion. To address this limitation, we craft a faithful rule-based translation of the MCWQ dataset from English to Chinese and Japanese. Even with the resulting robust benchmark, which we call MCWQ-R, we show that the distribution of compositions still suffers due to linguistic divergences, and that multilingual models still struggle with cross-lingual compositional generalization. Our dataset and methodology will be useful resources for the study of cross-lingual compositional generalization in other tasks.
What does the Failure to Reason with "Respectively'' in Zero/Few-Shot Settings Tell Us about Language Models?.
Ruixiang Cui, Seolhwa Lee, Daniel Hershcovich and Anders Søgaard. ACL 2023.

proceedings preprint abstract bib

Humans can effortlessly understand the coordinate structure of sentences such as "Niels Bohr and Kurt Cobain were born in Copenhagen and Seattle, respectively". In the context of natural language inference (NLI), we examine how language models (LMs) reason with respective readings (Gawron and Kehler, 2004) from two perspectives: syntactic-semantic and commonsense-world knowledge. We propose a controlled synthetic dataset WikiResNLI and a naturally occurring dataset NatResNLI to encompass various explicit and implicit realizations of "respectively". We show that fine-tuned NLI models struggle with understanding such readings without explicit supervision. While few-shot learning is easy in the presence of explicit cues, longer training is required when the reading is evoked implicitly, leaving models to rely on common sense inferences. Furthermore, our fine-grained analysis indicates models fail to generalize across different constructions. To conclude, we demonstrate that LMs still lag behind humans in generalizing to the long tail of linguistic constructions.
What's the Meaning of Superhuman Performance in Today's NLU?.
Simone Tedeschi, Johan Bos, Thierry Declerck, Jan Hajic, Daniel Hershcovich, Eduard H. Hovy, Alexander Koller, Simon Krek, Steven Schockaert, Rico Sennrich, Ekaterina Shutova and Roberto Navigli. ACL 2023. Outstanding Paper Award.

proceedings preprint abstract bib

In the last five years, there has been a significant focus in Natural Language Processing (NLP) on developing larger Pretrained Language Models (PLMs) and introducing benchmarks such as SuperGLUE and SQuAD to measure their abilities in language understanding, reasoning, and reading comprehension. These PLMs have achieved impressive results on these benchmarks, even surpassing human performance in some cases. This has led to claims of superhuman capabilities and the provocative idea that certain tasks have been solved. In this position paper, we take a critical look at these claims and ask whether PLMs truly have superhuman abilities and what the current benchmarks are really evaluating. We show that these benchmarks have serious limitations affecting the comparison between humans and PLMs and provide recommendations for fairer and more transparent benchmarks.
Pay More Attention to Relation Exploration for Knowledge Base Question Answering.
Yong Cao, Xianzhi Li, huiwen liu, Wen Dai, shuai chen, Bin Wang, Min Chen and Daniel Hershcovich. Findings of ACL 2023.

proceedings preprint abstract bib

Knowledge base question answering (KBQA) is a challenging task that aims to retrieve correct answers from large-scale knowledge bases. Existing attempts primarily focus on entity representation and final answer reasoning, which results in limited supervision for this task. Moreover, the relations, which empirically determine the reasoning path selection, are not fully considered in recent advancements. In this study, we propose a novel framework, RE-KBQA, that utilizes relations in the knowledge base to enhance entity representation and introduce additional supervision. We explore guidance from relations in three aspects, including (1) distinguishing similar entities by employing a variational graph auto-encoder to learn relation importance; (2) exploring extra supervision by predicting relation distributions as soft labels with a multi-task scheme; (3) designing a relation-guided re-ranking algorithm for post-processing. Experimental results on two benchmark datasets demonstrate the effectiveness and superiority of our framework, improving the F1 score by 5.7% from 40.5 to 46.3 on CWQ and 5.8% from 62.8 to 68.5 on WebQSP, better or on par with state-of-the-art methods.
Sentiment Classification of Historical Danish and Norwegian Literary Texts.
Ali Al-Laith, Kirstine Nielsen Degn, Alexander Conroy, Bolette S. Pedersen, Jens Bjerring-Hansen and Daniel Hershcovich. NoDaLiDa 2023.

proceedings preprint abstract bib

Sentiment classification is valuable for literary analysis, as sentiment is crucial in literary narratives. It can, for example, be used to investigate a hypothesis in the literary analysis of 19th-century Scandinavian novels that the writing of female authors in this period was characterized by negative sentiment, as this paper shows. In order to enable a data-driven analysis of this hypothesis, we create a manually annotated dataset of sentence-level sentiment annotations for novels from this period and use it to train and evaluate various sentiment classification methods. We find that pre-trained multilingual language models outperform models trained on modern Danish, as well as classifiers based on lexical resources. Finally, in classifier-assisted corpus analysis, we confirm the literary hypothesis regarding the author's gender and further shed light on the temporal development of the trend. Our dataset and trained models will be useful for future analysis of historical Danish and Norwegian literary texts.
Cross-Cultural Transfer Learning for Chinese Offensive Language Detection.
Li Zhou, Laura Cabello, Yong Cao and Daniel Hershcovich. C3NLP 2023.

proceedings preprint abstract bib

Detecting offensive language is a challenging task. Generalizing across different cultures and languages becomes even more challenging: besides lexical, syntactic and semantic differences, pragmatic aspects such as cultural norms and sensitivities, which are particularly relevant in this context, vary greatly. In this paper, we target Chinese offensive language detection and aim to investigate the impact of transfer learning using offensive language detection data from different cultural backgrounds, specifically Korean and English. We find that culture-specific biases in what is considered offensive negatively impact the transferability of language models (LMs) and that LMs trained on diverse cultural data are sensitive to different features in Chinese offensive language detection. In a few-shot learning scenario, however, our study shows promising prospects for non-English offensive language detection with limited resources. Our findings highlight the importance of cross-cultural transfer learning in improving offensive language detection and promoting inclusive digital spaces.
Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical Study.
Yong Cao, Li Zhou, Seolhwa Lee, Laura Cabello, Min Chen and Daniel Hershcovich. C3NLP 2023.

proceedings preprint abstract bib

The recent release of ChatGPT has garnered widespread recognition for its exceptional ability to generate human-like responses in dialogue. Given its usage by users from various nations and its training on a vast multilingual corpus that incorporates diverse cultural and societal norms, it is crucial to evaluate its effectiveness in cultural adaptation. In this paper, we investigate the underlying cultural background of ChatGPT by analyzing its responses to questions designed to quantify human cultural differences. Our findings suggest that, when prompted with American context, ChatGPT exhibits a strong alignment with American culture, but it adapts less effectively to other cultural contexts. Furthermore, by using different prompts to probe the model, we show that English prompts reduce the variance in model responses, flattening out cultural differences and biasing them towards American culture. This study provides valuable insights into the cultural implications of ChatGPT and highlights the necessity of greater diversity and cultural awareness in language technologies.
A Two-Sided Discussion of Preregistration of NLP Research.
Anders Søgaard, Daniel Hershcovich and Miryam de Lhoneux. EACL 2023.

proceedings pdf preprint abstract bib slides poster

Van Miltenburg et al. (2021) suggest NLP re- search should adopt preregistration to prevent fishing expeditions and to promote publication of negative results. At face value, this is a very reasonable suggestion, seemingly solving many methodological problems with NLP re- search. We discuss pros and cons—some old, some new: a) Preregistration is challenged by the practice of retrieving hypotheses after the results are known; b) preregistration may bias NLP toward confirmatory research; c) prereg- istration must allow for reclassification of re- search as exploratory; d) preregistration may in- crease publication bias; e) preregistration may increase flag-planting; f) preregistration may increase p-hacking; and finally, g) preregistra- tion may make us less risk tolerant. We cast our discussion as a dialogue, presenting both sides of the debate.
A Dataset of Sustainable Diet Arguments on Twitter.
Marcus Astrup Hansen and Daniel Hershcovich. Workshop on NLP for Positive Impact 2022.

proceedings pdf abstract bib slides poster data

Sustainable development requires a significant change in our dietary habits. Argument mining can help achieve this goal by both affecting and helping understand people's behavior. We design an annotation scheme for argument mining from online discourse around sustainable diets, including novel evidence types specific to this domain. Using Twitter as a source, we crowdsource a dataset of 597 tweets annotated in relation to 5 topics. We benchmark a variety of NLP models on this dataset, demonstrating strong performance in some sub-tasks, while highlighting remaining challenges.
Can AMR Assist Legal and Logical Reasoning?.
Nikolaus Schrack, Ruixiang Cui, Hugo A. López and Daniel Hershcovich. Findings of EMNLP 2022 (long paper).

proceedings pdf abstract bib code slides poster

Abstract Meaning Representation (AMR) has been shown to be useful for many downstream tasks. In this work, we explore the use of AMR for legal and logical reasoning, proposing neural architectures that utilize linearised AMR graphs in combination with pre-trained language models. While these models are not able to outperform text-only baselines, they correctly solve different instances than the text models, suggesting complementary abilities. Error analysis further reveals that AMR parsing quality is the most prominent challenge, especially regarding inputs with multiple sentences. We conduct a theoretical analysis of how logical relations are represented in AMR and conclude it might be helpful in some logical statements but not for others.
Towards Climate Awareness in NLP Research.
Daniel Hershcovich, Nicolas Webersinke, Mathias Kraus, Julia Anna Bingler and Markus Leippold. EMNLP 2022 (long paper).

proceedings pdf preprint abstract bib code slides

The climate impact of AI, and NLP research in particular, has become a serious issue given the enormous amount of energy that is increasingly being used for training and running computational models. Consequently, increasing focus is placed on efficient NLP. However, this important initiative lacks simple guidelines that would allow for systematic climate reporting of NLP research. We argue that this deficiency is one of the reasons why very few publications in NLP report key figures that would allow a more thorough examination of environmental impact. As a remedy, we propose a climate performance model card with the primary purpose of being practically usable with only limited information about experiments and the underlying computer hardware. We describe why this step is essential to increase awareness about the environmental impact of NLP research and, thereby, paving the way for more thorough discussions.
Compositional Generalization in Multilingual Semantic Parsing over Wikidata.
Ruixiang Cui, Rahul Aralikatte, Heather Lent and Daniel Hershcovich. TACL, 2022.

proceedings online preprint abstract bib code and data

Semantic parsing (SP) allows humans to leverage vast knowledge resources through natural interaction. However, parsers are mostly designed for and evaluated on English resources, such as CFQ (Keysers et al., 2020), the current standard benchmark based on English data generated from grammar rules and oriented towards Freebase, an outdated knowledge base. We propose a method for creating a multilingual, parallel dataset of question-query pairs, grounded in Wikidata. We introduce such a dataset, which we call Multilingual Compositional Wikidata Questions (MCWQ), and use it to analyze the compositional generalization of semantic parsers in Hebrew, Kannada, Chinese and English. While within-language generalization is comparable across languages, experiments on zero-shot cross-lingual transfer demonstrate that cross-lingual compositional generalization fails, even with state-of-the-art pretrained multilingual encoders. Furthermore, our methodology, dataset and results will facilitate future research on SP in more realistic and diverse settings than has been possible with existing resources.
Generalized Quantifiers as a Source of Error in Multilingual NLU Benchmarks.
Ruixiang Cui, Daniel Hershcovich and Anders Søgaard. NAACL 2022 (long paper).

proceedings preprint abstract bib

Logical approaches to representing language have developed and evaluated computational models of quantifier words since the 19th century, but today's NLU models still struggle to capture their semantics. We rely on Generalized Quantifier Theory for language-independent representations of the semantics of quantifier words, to quantify their contribution to the errors of NLU models. We find that quantifiers are pervasive in NLU benchmarks, and their occurrence at test time is associated with performance drops. Multilingual models also exhibit unsatisfying quantifier reasoning abilities, but not necessarily worse for non-English languages. To facilitate directly-targeted probing, we present an adversarial generalized quantifier NLI task (GQNLI) and show that pre-trained language models have a clear lack of robustness in generalized quantifier reasoning.
Evaluating Deep Taylor Decomposition for Reliability Assessment in the Wild.
Stephanie Brandl, Daniel Hershcovich and Anders Søgaard. ICWSM 2022.

proceedings preprint abstract bib

We argue that we need to evaluate model interpretability methods in the wild, in situations where professionals make critical decisions, and models can potentially assist them. We present an in-the-wild evaluation of token attribution based on Deep Taylor Decomposition, with professional journalists performing reliability assessments. We find that using this method in conjunction with RoBERTa-Large, fine-tuned on the Gossip Corpus, led to faster and better human decision-making, as well as a more critical attitude toward news sources among the journalists. We present a comparison of human and model rationales, as well as a qualitative analysis of the journalists' experiences with machine-in-the-loop decision making.
Challenges and Strategies in Cross-Cultural NLP.
Daniel Hershcovich, Stella Frank, Heather Lent, Miryam de Lhoneux, Mostafa Abdou, Stephanie Brandl, Emanuele Bugliarello, Laura Cabello Piqueras, Ilias Chalkidis, Ruixiang Cui, Constanza Fierro, Katerina Margatina, Phillip Rust and Anders Søgaard. ACL 2022 (long paper).

proceedings pdf preprint abstract bib slides poster

Various efforts in the Natural Language Processing (NLP) community have been made to accommodate linguistic diversity and serve speakers of many different languages. However, it is important to acknowledge that speakers and the content they produce and require, vary not just by language, but also by culture. Although language and culture are tightly linked, there are important differences. Analogous to cross-lingual and multilingual NLP, cross-cultural and multicultural NLP considers these differences in order to better serve users of NLP systems. We propose a principled framework to frame these efforts, and survey existing and potential strategies.
Scaling Creative Inspiration with Fine-Grained Functional Facets of Product Ideas.
Tom Hope, Ronen Tamari, Hyeonsu Kang, Daniel Hershcovich, Joel Chan, Aniket Kittur and Dafna Shahaf. CHI 2022.

proceedings preprint abstract bib

Web-scale repositories of products, patents and scientific papers offer an opportunity for creating automated systems that scour millions of ideas and assist users in discovering inspirations and solutions. Yet the common representation of ideas is in the form of raw textual descriptions, lacking important structure that is required for supporting creative innovation. Prior work has pointed to the importance of functional structure -- capturing the mechanisms and purposes of inventions -- for allowing users to discover structural connections across ideas and creatively adapt existing technologies. However, the use of functional representations was either coarse and limited in expressivity, or dependent on curated knowledge bases with poor coverage and significant manual effort from users. To help bridge this gap and unlock the potential of large-scale idea mining, we propose a novel computational representation that automatically breaks up products into fine-grained functional facets. We train a model to extract these facets from a challenging real-world corpus of invention descriptions, and represent each product as a set of facet embeddings. We design similarity metrics that support granular matching between functional facets across ideas, and use them to build a novel functional search capability that enables expressive queries for mechanisms and purposes. We construct a graph capturing hierarchical relations between purposes and mechanisms across an entire corpus of products, and use the graph to help problem-solvers explore the design space around a focal problem and view related problem perspectives. In empirical user studies, our approach leads to a significant boost in search accuracy and in the quality of creative inspirations, outperforming strong baselines and state-of-art representations of product texts by 50-60%.
A Multilingual Benchmark for Probing Negation-Awareness with Minimal Pairs.
Mareike Hartmann, Miryam de Lhoneux, Daniel Hershcovich, Yova Kementchedjhieva, Lukas Nielsen, Chen Qiu and Anders Søgaard.
CoNLL 2021.

proceedings abstract bib

Negation is one of the most fundamental concepts in human cognition and language, and several natural language inference (NLI) probes have been designed to investigate pretrained language models' ability to detect and reason with negation. However, the existing probing datasets are limited to English only, and do not enable controlled probing of performance in the absence or presence of negation. In response, we present a multilingual (English, Bulgarian, German, French and Chinese) benchmark collection of NLI examples that are grammatical and correctly labeled, as a result of manual inspection and editing. We use the benchmark to probe the negation-awareness of multilingual language models and find that models that correctly predict examples with negation cues often fail to correctly predict their counter-examples {\em without} negation cues, even when the cues are irrelevant for semantic inference.
Can Language Models Encode Perceptual Structure Without Grounding? A Case Study in Color.
Mostafa Abdou, Artur Kulmizev, Daniel Hershcovich, Stella Frank, Ellie Pavlick and Anders Søgaard.
CoNLL 2021.

proceedings preprint abstract bib

Pretrained language models have been shown to encode relational information, such as the relations between entities or concepts in knowledge-bases -- (Paris, Capital, France). However, simple relations of this type can often be recovered heuristically and the extent to which models implicitly reflect topological structure that is grounded in world, such as perceptual structure, is unknown. To explore this question, we conduct a thorough case study on color. Namely, we employ a dataset of monolexemic color terms and color chips represented in CIELAB, a color space with a perceptually meaningful distance metric. Using two methods of evaluating the structural alignment of colors in this space with text-derived color term representations, we find significant correspondence. Analyzing the differences in alignment across the color spectrum, we find that warmer colors are, on average, better aligned to the perceptual color space than cooler ones, suggesting an intriguing connection to findings from recent work on efficient communication in color naming. Further analysis suggests that differences in alignment are, in part, mediated by collocationality and differences in syntactic usage, posing questions as to the relationship between color perception and usage and context.
Great Service! Fine-grained Parsing of Implicit Arguments.
Ruixiang Cui and Daniel Hershcovich.
IWPT 2021.

proceedings preprint abstract bib code

Broad-coverage meaning representations in NLP mostly focus on explicitly expressed content. More importantly, the scarcity of datasets annotating diverse implicit roles limits empirical studies into their linguistic nuances. For example, in the web review "Great service!", the provider and consumer are implicit arguments of different types. We examine an annotated corpus of fine-grained implicit arguments (Cui and Hershcovich, 2020) by carefully re-annotating it, resolving several inconsistencies. Subsequently, we present the first transition-based neural parser that can handle implicit arguments dynamically, and experiment with two different transition systems on the improved dataset. We find that certain types of implicit arguments are more difficult to parse than others and that the simpler system is more accurate in recovering implicit arguments, despite having a lower overall parsing score, attesting current reasoning limitations of NLP models. This work will facilitate a better understanding of implicit and underspecified language, by incorporating it holistically into meaning representations.
Lexical Semantic Recognition.
Nelson F. Liu, Daniel Hershcovich, Michael Kranzlein, Nathan Schneider.
MWE 2021.

proceedings preprint abstract bib code

In lexical semantics, full-sentence segmentation and segment labeling of various phenomena are generally treated separately, despite their interdependence. We hypothesize that a unified lexical semantic recognition task is an effective way to encapsulate previously disparate styles of annotation, including multiword expression identification / classification and supersense tagging. Using the STREUSLE corpus, we train a neural CRF sequence tagger and evaluate its performance along various axes of annotation. As the label set generalizes that of previous tasks (PARSEME, DiMSUM), we additionally evaluate how well the model generalizes to those test sets, finding that it approaches or surpasses existing models despite training only on STREUSLE. Our work also establishes baseline models and evaluation metrics for integrated and accurate modeling of lexical semantics, facilitating future work in this area.
It’s the Meaning That Counts: The State of the Art in NLP and Semantics.
Daniel Hershcovich and Lucia Donatelli.
KI - Künstliche Intelligenz, 2021.

pdf online abstract bib

Semantics, the study of meaning, is central to research in Natural Language Processing (NLP) and many other fields connected to Artificial Intelligence. Nevertheless, how semantics is understood in NLP ranges from traditional, formal linguistic definitions based on logic and the principle of compositionality to more applied notions based on grounding meaning in real-world objects and real-time interaction. “Semantic” methods may additionally strive for meaningful representation of language that integrates broader aspects of human cognition and embodied experience, calling into question how adequate a representation of meaning based on linguistic signal alone is for current research agendas. We review the state of computational semantics in NLP and investigate how different lines of inquiry reflect distinct understandings of semantics and prioritize different layers of linguistic meaning. In conclusion, we identify several important goals of the field and describe how current research addresses them.
How far can we get with one GPU in 100 hours? CoAStaL at MultiIndicMT Shared Task.
Rahul Aralikatte, Héctor Ricardo Murrieta Bello, Daniel Hershcovich, Marcel Bollmann, Anders Søgaard.
MultiIndicMT Shared Task 2021.

proceedings abstract bib

This work shows that competitive translation results can be obtained in a constrained setting by incorporating the latest advances in memory and compute optimization. We train and evaluate large multilingual translation models using a single GPU for a maximum of 100 hours and get within 4-5 BLEU points of the top submission on the leaderboard. We also benchmark standard baselines on the PMI corpus and re-discover well-known shortcomings of translation systems and metrics.
Moses and the Character-Based Random Babbling Baseline: CoAStaL at AmericasNLP 2021 Shared Task.
Marcel Bollmann, Rahul Aralikatte, Héctor Murrieta Bello, Daniel Hershcovich, Miryam de Lhoneux, Anders Søgaard.
AmericasNLP Shared Task 2021.

proceedings abstract bib

We evaluated a range of neural machine translation techniques developed specifically for low-resource scenarios. Unsuccessfully. In the end, we submitted two runs: (i) a standard phrase-based model, and (ii) a random babbling baseline using character trigrams. We found that it was surprisingly hard to beat (i), in spite of this model being, in theory, a bad fit for polysynthetic languages; and more interestingly, that (ii) was better than several of the submitted systems, highlighting how difficult low-resource machine translation for polysynthetic languages is.
An autonomous debating system.
Noam Slonim, Yonatan Bilu, Carlos Alzate, Roy Bar-Haim, Ben Bogin, Francesca Bonin, Leshem Choshen, Edo Cohen-Karlik, Lena Dankin, Lilach Edelstein, Liat Ein-Dor, Roni Friedman-Melamed, Assaf Gavron, Ariel Gera, Martin Gleize, Shai Gretz, Dan Gutfreund, Alon Halfon, Daniel Hershcovich, Ron Hoory, Yufang Hou, Shay Hummel, Michal Jacovi, Charles Jochim, Yoav Kantor, Yoav Katz, David Konopnicki, Zvi Kons, Lili Kotlerman, Dalia Krieger, Dan Lahav, Tamar Lavee, Ran Levy, Naftali Liberman, Yosi Mass, Amir Menczel, Shachar Mirkin, Guy Moshkowich, Shila Ofek-Koifman, Matan Orbach, Ella Rabinovich, Ruty Rinott, Slava Shechtman, Dafna Sheinwald, Eyal Shnarch, Ilya Shnayderman, Aya Soffer, Artem Spector, Benjamin Sznajder, Assaf Toledo, Orith Toledo-Ronen, Elad Venezian and Ranit Aharonov.
Nature, 2021.

pdf online abstract bib website

Artificial intelligence (AI) is defined as the ability of machines to perform tasks that are usually associated with intelligent beings. Argument and debate are fundamental capabilities of human intelligence, essential for a wide range of human activities, and common to all human societies. The development of computational argumentation technologies is therefore an important emerging discipline in AI research1. Here we present Project Debater, an autonomous debating system that can engage in a competitive debate with humans. We provide a complete description of the system’s architecture, a thorough and systematic evaluation of its operation across a wide range of debate topics, and a detailed account of the system’s performance in its public debut against three expert human debaters. We also highlight the fundamental differences between debating with humans as opposed to challenging humans in game competitions, the latter being the focus of classical ‘grand challenges’ pursued by the AI research community over the past few decades. We suggest that such challenges lie in the ‘comfort zone’ of AI, whereas debating with humans lies in a different territory, in which humans still prevail, and for which novel paradigms are required to make substantial progress.
Joint Semantic Analysis with Document-Level Cross-Task Coherence Rewards.
Rahul Aralikatte, Mostafa Abdou, Heather Lent, Daniel Hershcovich and Anders Søgaard.
AAAI 2021 (long paper).

preprint abstract bib code

Coreference resolution and semantic role labeling are NLP tasks that capture different aspects of semantics, indicating respectively, which expressions refer to the same entity, and what semantic roles expressions serve in the sentence. However, they are often closely interdependent, and both generally necessitate natural language understanding. Do they form a coherent abstract representation of documents? We present a neural network architecture for joint coreference resolution and semantic role labeling for English, and train graph neural networks to model the 'coherence' of the combined shallow semantic graph. Using the resulting coherence score as a reward for our joint semantic analyzer, we use reinforcement learning to encourage global coherence over the document and between semantic annotations. This leads to improvements on both tasks in multiple datasets from different domains, and across a range of encoders of different expressivity, calling, we believe, for a more holistic approach for semantics in NLP.
Refining Implicit Argument Annotation For UCCA.
Ruixiang Cui and Daniel Hershcovich.
DMR 2020.

pdf proceedings preprint abstract bib slides data

Predicate-argument structure analysis is a central component in meaning representations of text. The fact that some arguments are not explicitly mentioned in a sentence gives rise to ambiguity in language understanding, and renders it difficult for machines to interpret text correctly. However, only few resources represent implicit roles for NLU, and existing studies in NLP only make coarse distinctions between categories of arguments omitted from linguistic form. To better understand the behaviour of implicit roles and their characteristics, in this paper, we design a typology for fine-grained implicit argument annotation on top of Universal Conceptual Cognitive Annotation's foundational layer. The proposed implicit argument categorisation is driven by theories of implicit role interpretation and consists of six types: Deictic, Generic, Genre-based, Type-identifiable, Non-specific, and Iterated-set. We exemplify our design by revisiting part of the UCCA EWT corpus, providing a new dataset annotated with the refinement layer, and making a comparative analysis with other schemes both in terms of quantity and quality.
Comparison by Conversion: Reverse-Engineering UCCA from Syntax and Lexical Semantics.
Daniel Hershcovich, Nathan Schneider, Dotan Dvir, Jakob Prange, Miryam de Lhoneux and Omri Abend.
COLING 2020.

pdf proceedings preprint abstract bib poster converter code parser code

Building robust natural language understanding systems will require a clear characterization of whether and how various linguistic meaning representations complement each other. To perform a systematic comparative analysis, we evaluate the mapping between meaning representations from different frameworks using two complementary methods: (i) a rule-based converter, and (ii) a supervised delexicalized parser that parses to one framework using only information from the other as features. We apply these methods to convert the STREUSLE corpus (with syntactic and lexical semantic annotations) to UCCA (a graph-structured full-sentence meaning representation). Both methods yield surprisingly accurate target representations, close to fully supervised UCCA parser quality---indicating that UCCA annotations are partially redundant with STREUSLE annotations. Despite this substantial convergence between frameworks, we find several important areas of divergence.
MRP 2020: The Second Shared Task on Cross-Framework and Cross-Lingual Meaning Representation Parsing.
Stephan Oepen, Omri Abend, Lasha Abzianidze, Johan Bos, Jan Hajic, Daniel Hershcovich, Bin Li, Tim O'Gorman, Nianwen Xue and Daniel Zeman.
CoNLL 2020 shared task.

pdf proceedings abstract bib slides website

The 2020 Shared Task at the Conference for Computational Language Learning (CoNLL) was devoted to Meaning Representation Parsing (MRP) across frameworks and languages. Extending a similar setup from the previous year, five distinct approaches to the representation of sentence meaning in the form of directed graphs were represented in the English training and evaluation data for the task, packaged in a uniform graph abstraction and serialization; for four of these representation frameworks, additional training and evaluation data was provided for one additional language per framework. The task received submissions from eight teams, of which two do not participate in the official ranking because they arrived after the closing deadline or made use of additional training data. All technical information regarding the task, including system submissions, official results, and links to supporting resources and software are available from the task web site at: http://mrp.nlpl.eu
HUJI-KU at MRP 2020: Two Transition-based Neural Parsers.
Ofir Arviv, Ruixiang Cui and Daniel Hershcovich.
CoNLL 2020 shared task.

pdf preprint proceedings abstract bib slides TUPA code HIT-SCIR code multitask code

This paper describes the HUJI-KU system submission to the shared task on Cross-Framework Meaning Representation Parsing (MRP) at the 2020 Conference for Computational Language Learning (CoNLL), employing TUPA and the HIT-SCIR parser, which were, respectively, the baseline system and winning system in the 2019 MRP shared task. Both are transition-based parsers using BERT contextualized embeddings. We generalized TUPA to support the newly-added MRP frameworks and languages, and experimented with multitask learning with the HIT-SCIR parser. We reached 4th place in both the cross-framework and cross-lingual tracks.
Køpsala: Transition-Based Graph Parsing via Efficient Training and Effective Encoding.
Daniel Hershcovich, Miryam de Lhoneux, Artur Kulmizev, Elham Pejhan and Joakim Nivre.
IWPT 2020 shared task.

pdf preprint proceedings abstract bib slides code

We present Køpsala, the Copenhagen-Uppsala system for the Enhanced Universal Dependencies Shared Task at IWPT 2020. Our system is a pipeline consisting of off-the-shelf models for everything but enhanced graph parsing, and for the latter, a transition-based graph parser adapted from Che et al. (2019). We train a single enhanced parser model per language, using gold sentence splitting and tokenization for training, and rely only on tokenized surface forms and multilingual BERT for encoding. While a bug introduced just before submission resulted in a severe drop in precision, its post-submission fix would bring us to 4th place in the official ranking, according to average ELAS. Our parser demonstrates that a unified pipeline is effective for both Meaning Representation Parsing and Enhanced Universal Dependencies.
MRP 2019: Cross-Framework Meaning Representation Parsing.
Stephan Oepen, Omri Abend, Jan Hajič, Daniel Hershcovich, Marco Kuhlmann, Tim O'Gorman, Nianwen Xue, Jayeol Chun, Milan Straka and Zdeňka Urešová.
CoNLL 2019 shared task.

pdf proceedings abstract bib slides website

The 2019 Shared Task at the Conference for Computational Language Learning (CoNLL) was devoted to Meaning Representation Parsing (MRP) across frameworks. Five distinct approaches to the representation of sentence meaning in the form of directed graph were represented in the training and evaluation data for the task, packaged in a uniform abstract graph representation and serialization. The task received submissions from eighteen teams, of which five do not participate in the official ranking because they arrived after the closing deadline, made use of additional training data, or involved one of the task co-organizers. All technical information regarding the task, including system submissions, official results, and links to supporting resources and software are available from the task web site at: http://mrp.nlpl.eu
TUPA at MRP 2019: A Multi-Task Baseline System.
Daniel Hershcovich and Ofir Arviv.
CoNLL 2019 shared task.

pdf proceedings abstract bib poster slides code

This paper describes the TUPA system submission to the shared task on CrossFramework Meaning Representation Parsing (MRP) at the 2019 Conference for Computational Language Learning (CoNLL). TUPA provides a baseline point of comparison and is not considered in the official ranking of participating systems. While originally developed for UCCA only, TUPA has been generalized to support all MRP frameworks included in the task, and trained using multi-task learning to parse them all with a shared model. It is a transition-based parser with a BiLSTM encoder, augmented with BERT contextualized embeddings.
Rewarding Coreference Resolvers for Being Consistent with World Knowledge.
Rahul Aralikatte, Heather Lent, Ana Valeria Gonzalez, Daniel Hershcovich, Chen Qiu, Anders Sandholm, Michael Ringgaard and Anders Søgaard.
EMNLP-IJCNLP 2019 (short paper).

pdf preprint proceedings abstract bib code

Unresolved coreference is a bottleneck for relation extraction, and high-quality coreference resolvers may produce an output that makes it a lot easier to extract knowledge triples. We show how to improve coreference resolvers by forwarding their input to a relation extraction system and reward the resolvers for producing triples that are found in knowledge bases. Since relation extraction systems can rely on different forms of supervision and be biased in different ways, we obtain the best performance, improving over the state of the art, using multi-task reinforcement learning.
The Language of Legal and Illegal Activity on the Darknet.
Leshem Choshen*, Dan Eldad*, Daniel Hershcovich*, Elior Sulem* and Omri Abend.
ACL 2019 (long paper).

pdf preprint proceedings abstract bib slides code

The non-indexed parts of the Internet (the Darknet) have become a haven for both legal and illegal anonymous activity. Given the magnitude of these networks, scalably monitoring their activity necessarily relies on automated tools, and notably on NLP tools. However, little is known about what characteristics texts communicated through the Darknet have, and how well off-the-shelf NLP tools do on this domain. This paper tackles this gap and performs an in-depth investigation of the characteristics of legal and illegal text in the Darknet, comparing it to a clear net website with similar content as a control condition. Taking drug-related websites as a test case, we find that texts for selling legal and illegal drugs have several linguistic characteristics that distinguish them from one another, as well as from the control condition, among them the distribution of POS tags, and the coverage of their named entities in Wikipedia.
Argument Invention from First Principles.
Yonatan Bilu, Ariel Gera, Daniel Hershcovich, Benjamin Sznajder, Dan Lahav, Guy Moshkowich, Anael Malet, Assaf Gavron and Noam Slonim.
ACL 2019 (long paper).

pdf preprint proceedings abstract bib

Competitive debaters often find themselves facing a challenging task -- how to debate a topic they know very little about, with only minutes to prepare, and without access to books or the Internet? What they often do is rely on ''first principles'', commonplace arguments which are relevant to many topics, and which they have refined in past debates. In this work we aim to explicitly define a taxonomy of such principled recurring arguments, and, given a controversial topic, to automatically identify which of these arguments are relevant to the topic. As far as we know, this is the first time that this approach to argument invention is formalized and made explicit in the context of NLP. The main goal of this work is to show that it is possible to define such a taxonomy. While the taxonomy suggested here should be thought of as a ''first attempt'' it is nonetheless coherent, covers well the relevant topics and coincides with what professional debaters actually argue in their speeches, and facilitates automatic argument invention for new topics.
SemEval 2019 Task 1: Cross-lingual Semantic Parsing with UCCA.
Daniel Hershcovich, Zohar Aizenbud, Leshem Choshen, Elior Sulem, Ari Rappoport and Omri Abend.
SemEval 2019 shared task.

pdf preprint proceedings abstract bib slides website

We present the SemEval 2019 shared task on Universal Conceptual Cognitive Annotation (UCCA) parsing in English, German and French, and discuss the participating systems and results. UCCA is a cross-linguistically applicable framework for semantic representation, which builds on extensive typological work and supports rapid annotation. UCCA poses a challenge for existing parsing techniques, as it exhibits reentrancy (resulting in DAG structures), discontinuous structures and non-terminal nodes corresponding to complex semantic units. The shared task has yielded improvements over the state-of-the-art baseline in all languages and settings. Full results can be found in the task's website.
Syntactic Interchangeability in Word Embedding Models.
Daniel Hershcovich, Assaf Toledo, Alon Halfon and Noam Slonim.
RepEval 2019.

pdf preprint proceedings abstract bib poster slide code

Nearest neighbors in word embedding models are commonly observed to be semantically similar, but the relations between them can vary greatly. We investigate the extent to which word embedding models preserve syntactic interchangeability, as reflected by distances between word vectors, and the effect of hyper-parameters---context window size in particular. We use part of speech (POS) as a proxy for syntactic interchangeability, as generally speaking, words with the same POS are syntactically valid in the same contexts. We also investigate the relationship between interchangeability and similarity as judged by commonly-used word similarity benchmarks, and correlate the result with the performance of word embedding models on these benchmarks. Our results will inform future research and applications in the selection of word embedding model, suggesting a principle for an appropriate selection of the context window size parameter depending on the use-case.
Content Differences in Syntactic and Semantic Representations.
Daniel Hershcovich, Omri Abend and Ari Rappoport.
NAACL 2019 (long paper).

pdf preprint proceedings abstract bib supplementary material poster code data source examples

Syntactic analysis plays an important role in semantic parsing, but this role remains a topic of ongoing debate. The debate has been constrained by the scarcity of empirical comparative studies between syntactic and semantic schemes, which hinders the development of parsing methods informed by the details of target schemes and constructions. We target this gap, and take Universal Dependencies (UD) and UCCA as a test case. After abstracting away from differences of convention or formalism, we find that most content divergences can be ascribed to: (1) UCCA's distinction between a Scene and a non-Scene; (2) UCCA's distinction between primary relations, secondary ones and participants; (3) different treatment of multi-word expressions, and (4) different treatment of inter-clause linkage. We further discuss the long tail of cases where the two schemes take markedly different approaches. Finally, we show that the proposed comparison methodology can be used for fine-grained evaluation of UCCA parsing, highlighting both challenges and potential sources for improvement. The substantial differences between the schemes suggest that semantic parsers are likely to benefit downstream text understanding applications beyond their syntactic counterparts.
Universal Dependency Parsing with a General Transition-Based DAG Parser.
Daniel Hershcovich, Omri Abend and Ari Rappoport.
CoNLL 2018 UD Shared Task.

pdf preprint proceedings abstract bib poster code source

This paper presents our experiments with applying TUPA to the CoNLL 2018 UD shared task. TUPA is a general neural transition-based DAG parser, which we use to present the first experiments on recovering enhanced dependencies as part of the general parsing task. TUPA was designed for parsing UCCA, a cross-linguistic semantic annotation scheme, exhibiting reentrancy, discontinuity and non-terminal nodes. By converting UD trees and graphs to a UCCA-like DAG format, we train TUPA almost without modification on the UD parsing task. The generic nature of our approach lends itself naturally to multitask learning.
Multitask Parsing Across Semantic Representations.
Daniel Hershcovich, Omri Abend and Ari Rappoport.
ACL 2018 (long paper).

pdf preprint proceedings abstract bib supplementary material poster code data source

The ability to consolidate information of different types is at the core of intelligence, and has tremendous practical value in allowing learning for one task to benefit from generalizations learned for others. In this paper we tackle the challenging task of improving semantic parsing performance, taking UCCA parsing as a test case, and AMR, SDP and Universal Dependencies (UD) parsing as auxiliary tasks. We experiment on three languages, using a uniform transition-based system and learning architecture for all parsing tasks. Despite notable conceptual, formal and domain differences, we show that multitask learning significantly improves UCCA parsing in both in-domain and out-of-domain settings.
A Transition-Based Directed Acyclic Graph Parser for UCCA.
Daniel Hershcovich, Omri Abend and Ari Rappoport.
ACL 2017 (long paper). Outstanding Paper Award.

pdf preprint proceedings abstract bib supplementary material slides poster code demo data source

We present the first parser for UCCA, a cross-linguistically applicable framework for semantic representation, which builds on extensive typological work and supports rapid annotation. UCCA poses a challenge for existing parsing techniques, as it exhibits reentrancy (resulting in DAG structures), discontinuous structures and non-terminal nodes corresponding to complex semantic units. To our knowledge, the conjunction of these formal properties is not supported by any existing parser. Our transition-based parser, which uses a novel transition set and features based on bidirectional LSTMs, has value not just for UCCA parsing: its ability to handle more general graph structures can inform the development of parsers for other semantic DAG structures, and in languages that frequently use discontinuous structures.
Automatic Claim Negation: Why, How and When.
Yonatan Bilu, Daniel Hershcovich, Noam Slonim.
NAACL HLT 2015 (long paper).

proceedings abstract bib

The main goal of argumentation mining is to analyze argumentative structures within an argument-rich document, and reason about their composition. Recently, there is also interest in the task of simply detecting claims (sometimes called conclusion) in general documents. In this work we ask how this set of detected claims can be augmented further, by adding to it the negation of each detected claim. This presents two NLP problems: how to automatically negate a claim, and when such a negated claim can plausibly be used. We present first steps into solving both these problems, using a rule-based approach for the former and a statistical one towards the latter.
Context Dependent Claim Detection.
Ran Levy, Yonatan Bilu, Daniel Hershcovich, Ehud Aharoni and Noam Slonim.
COLING 2014 (long paper).

proceedings abstract bib

While discussing a concrete controversial topic, most humans will find it challenging to swiftly raise a diverse set of convincing and relevant claims that should set the basis of their arguments. Here, we formally define the challenging task of automatic claim detection in a given context and discuss its associated unique difficulties. Further, we outline a preliminary solution to this task, and assess its performance over annotated real world data, collected specifically for that purpose over hundreds of Wikipedia articles. We report promising results of a supervised learning approach, which is based on a cascade of classifiers designed to properly handle the skewed data which is inherent to the defined task. These results demonstrate the viability of the introduced task.
Claims on Demand–an Initial Demonstration of a System for Automatic Detection and Polarity Identification of Context Dependent Claims in Massive Corpora.
Ehud Aharoni, Carlos Alzate, Roy Bar-Haim, Yonatan Bilu, Lena Dankin, Iris Eiron, Daniel Hershcovich, Shay Hummel.
COLING 2014 (system demonstration).

proceedings abstract bib

While discussing a concrete controversial topic, most humans will find it challenging to swiftly raise a diverse set of convincing and relevant claims that should set the basis of their arguments. Here, we demonstrate the initial capabilities of a system that, given a controversial topic, can automatically pinpoint relevant claims in Wikipedia, determine their polarity with respect to the given topic, and articulate them per the user's request.
A Benchmark Dataset for Automatic Detection of Claims and Evidence in the Context of Controversial Topics.
Ehud Aharoni, Anatoly Polnarov, Tamar Lavee, Daniel Hershcovich, Ran Levy, Ruty Rinott, Dan Gutfreund, Noam Slonim.
Workshop on Argumentation Mining at ACL 2014.

proceedings abstract bib

We describe a novel and unique argumentative structure dataset. This corpus consists of data extracted fro m hundreds of Wikipedia articles using a meticulously monitored manual annotation process. The result is 2,683 argument elements, collected in the context of 33 controversial topics, organized under a simple claim-evidence structure. The obtained data are publicly available for academic research.
Verification of Transactional Memory in POWER8.
Alon Adir, Dave Goodman, Daniel Hershcovich, Oz Hershkovitz, Bryan Hickerson, Karen Holtz, Wisam Kadry, Anatoly Koyfman, John Ludden, Charles Meissner, Amir Nahir, Randall R Pratt, Mike Schiffli, Brett St Onge, Brian Thompto, Elena Tsanko, Avi Ziv.
DAC 2014.

pdf abstract bib

Transactional memory is a promising mechanism for synchronizing concurrent programs that eliminates locks at the expense of hardware complexity. Transactional memory is a hard feature to verify. First, transactions comprise several instructions that must be observed as a single global atomic operation. In addition, there are many reasons a transaction can fail. This results in a high level of non-determinism which must be tamed by the verification methodology. This paper describes the innovation that was applied to tools and methodology in pre-silicon simulation, acceleration and post-silicon in order to verify transactional memory in the IBM POWER8 processor core.

Does injecting linguistic structure into language models lead to better alignment with brain recordings?.
Mostafa Abdou, Ana Valeria Gonzalez, Mariya Toneva, Daniel Hershcovich and Anders Søgaard.

preprint abstract bib code

Neuroscientists evaluate deep neural networks for natural language processing as possible candidate models for how language is processed in the brain. These models are often trained without explicit linguistic supervision, but have been shown to learn some linguistic structure in the absence of such supervision (Manning et al., 2020), potentially questioning the relevance of symbolic linguistic theories in modeling such cognitive processes (Warstadt and Bowman, 2020). We evaluate across two fMRI datasets whether language models align better with brain recordings, if their attention is biased by annotations from syntactic or semantic formalisms. Using structure from dependency or minimal recursion semantic annotations, we find alignments improve significantly for one of the datasets. For another dataset, we see more mixed results. We present an extensive analysis of these results. Our proposed approach enables the evaluation of more targeted hypotheses about the composition of meaning in the brain, expanding the range of possible scientific inferences a neuroscientist could make, and opens up new opportunities for cross-pollination between computational neuroscience and linguistics.

Universal Semantic Parsing with Neural Networks.
Daniel Hershcovich.
PhD dissertation, Hebrew University of Jerusalem, 2019.

pdf abstract bib slides source

A major scientific effort is dedicated to natural language understanding, which aims to be able to comprehend text, reason about it, and act upon it in an intelligent way. While specific use-cases or benchmarks can be solved with relatively simple systems, which either ignore word order ("bag-of-words" models) or treat it as a simple linear structure (such as the popular sequence-to-sequence framework allowing neural networks to learn tasks in an end-to-end fashion), understanding human language in general requires a hierarchical representation of meaning. Constructing this representation from text has been the goal of an extensive line of work in semantic parsing. While many semantic representation schemes have been proposed, they share many of their basic distinctions, such as between predicates (relations, states and events) and arguments (participants). This thesis focuses on a particular semantic representation scheme called Universal Conceptual Cognitive Annotation (UCCA), whose main design principles are support for all major linguistic semantic phenomena, cross-linguistic applicability, stability across translations, ease of annotation (even by those who are not experts in linguistics), and a modular architecture supporting multiple layers of semantic annotation. A fully automatic parser is presented, and evaluated on multiple languages (English, French and German). The parser, titled "TUPA" (transition-based UCCA parser), is able to learn very general graph structures: directed acyclic graphs over token sequences with non-terminal nodes for complex units, where these may cover discontinuous terminal yields. This general class of graphs covers the structures annotated in UCCA, as well as other representation schemes. TUPA is implemented as a transition-based parser, whose transition system supports these structural properties. Its transition classifier is a neural network equipped with a bidirectional long short-term memory (BiLSTM) module for calculating feature representations for the input. In an extensive comparison to conversion-based methods, as well as other classifier implementations, TUPA is shown to outperform all baselines in the task of UCCA parsing in both in-domain and out-of-domain settings in three languages. The parser is subsequently applied to two other semantic representation schemes, DM and AMR, and to syntactic dependencies in the Universal Dependencies (UD) scheme. This demonstrates that the flexible parser is usable not just for UCCA parsing. Furthermore, training TUPA in a multitask setting on all of these schemes improves its UCCA parsing accuracy, by effectively learning generalizations across the different representations: a shared model is thus able to apply semantic distinctions in one task, which have been learned for another. Finally, in an empirical comparison of the content of semantic and syntactic representations, we discover several aspects of divergence, i.e., differences in the content captured by these schemes. These have profound impact on the potential contribution of syntax to semantic parsing, and on the usefulness of each of the approaches for semantic tasks in natural language processing. I see semantic parsing as a means for computers to learn language. While different representations focus on different distinctions and do so with formally different structures, they share an overall goal, which is to support natural language processing applications, such as classifying text into categories, tagging it for linguistic properties, performing inference and reasoning, and generating new text according to some constraints (e.g., machine translation). The combined datasets annotated in every representation are an invaluable resource, which, used effectively, can greatly boost our achievements in language understanding and processing.

Cross-lingual Semantic Representation for NLP with UCCA.
Omri Abend, Dotan Dvir, Daniel Hershcovich, Jakob Prange and Nathan Schneider.
COLING 2020 tutorial.

proceedings abstract bib slides videos

This is an introductory tutorial to UCCA (Universal Conceptual Cognitive Annotation), a cross-linguistically applicable framework for semantic representation, with corpora annotated in English, German and French, and ongoing annotation in Russian and Hebrew. UCCA builds on extensive typological work and supports rapid annotation. The tutorial will provide a detailed introduction to the UCCA annotation guidelines, design philosophy and the available resources; and a comparison to other meaning representations. It will also survey the existing parsing work, including the findings of three recent shared tasks, in SemEval and CoNLL, that addressed UCCA parsing. Finally, the tutorial will present recent applications and extensions to the scheme, demonstrating its value for natural language processing in a range of languages and domains.

Assessing Knowledge of Cultural Values in ChatGPT
Kulturministeriets Tech-Kontor, Ministry of Culture, Denmark (09/2023).

slides pptx
Recipe Adaptation with Language Models
Department of Food Science, University of Copenhagen (06/2023).

abstract slides pdf slides pptx

As people explore cuisines from different cultures, it becomes important to adapt recipes to cater to different ingredient availability and preparation methods. In addition, individuals with dietary restrictions, due to health, religious or other reasons, require customized recipe adaptations. In ongoing research, my colleagues and I explore the application of large language models such as ChatGPT to address these issues, automatically adapting recipes according to various constraints. In one project, we use them to generate coherent recipes while adapting to cultural differences. In another, we focus on the challenge of adapting non-vegan recipes to vegan ones while preserving the flavor, cultural style, and aesthetic of the dish. In this presentation, I will highlight the challenges this task raises, as high-quality adaptations require more than simple phrase substitutions – for example, the models are required to infer necessary modifications to preparation methods or introduce creative ingredient substitutions that are both appealing and interesting. This opens the way for a whole new field in Artificial Intelligence, bridging Natural Language Processing and Food Science.
AI 101 – on what ChatGPT is(n’t). Large Language Models and their Potential and Limitations for Language Learning
Forum on AI in foreign language programmes, University of Copenhagen (05/2023).

slides pdf slides pptx
Cultural Adaptation with and of Language Models
Invited talk at the EACL 2023 workshop Cross-Cultural Considerations in NLP (05/2023).

abstract slides pdf slides pptx

Large language models provide a unique opportunity to adapt content to the user's culture. However, this raises the risk of perpetuating cultural biases and homogenizing cultural diversity, as many of these models are currently centralized and trained on a limited set of languages and cultures. In this talk, I will emphasize the importance of adapting language models to various cultures to promote linguistic and cultural diversity. This involves not only adapting existing language models but also creating new models and resources that incorporate culturally diverse knowledge and perspectives. This approach will require collaboration and exchange of knowledge across linguistic and cultural boundaries, and has the potential to support intercultural cross-fertilization in various fields, such as literature, anthropology, and beyond. By adapting language models to reflect cultural diversity, we can enable more equitable access to information and foster greater intercultural understanding.
Finding Meaning in Data across Languages
Sprogteknologisk Konference 2022, Centre for Language Technology, University of Copenhagen (11/2022)

abstract slides pdf slides pptx

What is common to complex database queries and beautiful artistic masterpieces? Artificial intelligence can generate both from simple English prompts, and large language models have been instrumental in these achievements. I will show how we can understand the patterns such models find in data using systematic meaning representations, and how to create these representations so we can adapt language models across languages and cultures.
Natural Language Processing for Sustainable Diets
Research Seminar at the Center for Applied Ecological Thinking (CApE). University of Copenhagen (11/2022)

slides pdf
Generalization and Representation in Multilingual Semantic Parsing
Workshop on Ten Years of BabelNet and Multilingual Neurosymbolic Natural Language Understanding, Sapienza University of Rome (07/2022)

slides pdf recording
Argument Mining for Green Nutrition
SODAS-Climate meeting, University of Copenhagen (05/2022)

slides pdf
Challenges and Strategies in Cross-Cultural NLP
LIIR Journal Club, KU Leuven (04/2022)

slides pdf
Cultural and Environmental Considerations in Natural Language Processing
Seminar at University of Haifa, Department of Information Systems (03/2022)

slides pdf
Challenges and Strategies in Cross-Cultural NLP
NLP Workshop, IT University of Copenhagen (03/2022)

slides pdf
Meaning Representation and Parsing in Natural Language Processing
Seminar at the Department of Food and Resource Economics, University of Copenhagen (05/2021)

slides pdf
Meaning Representation Parsing
DIKU Bits, Department of Computer Science, University of Copenhagen (02/2020)

slides pdf
Universal Meaning Representation Parsing
Seminar in Computational Linguistics, Department of Linguistics and Philology, Uppsala University (11/2019)

slides pdf
Universal Semantic Parsing with Neural Networks
Georgetown University (06/2019)

slides pdf
A Transition-Based Directed Acyclic Graph Parser for Universal Conceptual Cognitive Annotation
Tel Aviv University, NLP Seminar (01/2018)

slides pdf
A Transition-Based Directed Acyclic Graph Parser for Universal Conceptual Cognitive Annotation
University of Washington, The Paul G. Allen Center for Computer Science & Engineering (07/2017)

slides pdf
A Transition-Based Directed Acyclic Graph Parser for Universal Conceptual Cognitive Annotation
Technion, Computational Data Science Seminar (06/2017)

slides pdf
Broad-Coverage Transition-Based UCCA Parsing
Hebrew University, Computer Science Learning Club (11/2016)

slides pdf
Broad-Coverage Semantic Parsing: A Transition-Based Approach
The Israeli Seminar on Computational Linguistics, ISCOL 2016 (05/2016)

slides pdf

Copenhagen Summer University course on Climate Friendly AI
Natural Language Processing (teacher and course coordinator): 2019, 2020, 2021, 2022, 2023

materials
Introduction to Data Science (teacher and course coordinator): 2023, 2024
Introduction to Machine Learning (teacher): 2020

Human Language from a Computational Perspective (TA): 2014, 2015, 2016, 2017

materials

EMNLP 2022 Workshop Co-chair.
HPLT & NLPL Winter School 2023 Program Co-chair.
Co-organized ISCOL 2017, the annual meeting of the Israeli seminar on computational linguistics, as well as SemEval 2019 Task 1 on Cross-lingual Semantic Parsing with UCCA, and the CoNLL 2019 and 2020 shared tasks on Cross-Framework Meaning Representation Parsing.
Co-presented a tutorial on Cross-lingual Semantic Representation for NLP with UCCA at COLING 2020.
Guest editor for the Künstliche Intelligenz Special Issue on NLP and Semantics.
Area chair for NAACL-HLT 2021, ACL-IJCNLP 2021 and *SEM 2021.
Action editor for ACL Rolling Review since October 2021.
Reviewer for ACL (2015, 2016, 2017, 2019, 2020, 2022, 2023), EMNLP (2017, 2018: Best Reviewer Award, 2019, 2020: Outstanding Reviewer, 2023), IJCNLP 2017, COLING (2018, 2020, 2022), NAACL-HLT 2019, *SEM (2019, 2020, 2022), DMR (2019, 2020, 2023), NoDaLiDa (2019, 2021, 2023), EurNLP 2019, AAAI 2020, IJCAI 2020, IWPT (2020, 2021) AACL-IJCNLP 2020, CoNLL (2020, 2022, 2023), TLT 2020, EACL 2021 and the journals Computational Linguistics, Computer Speech & Language and Language Resources and Evaluation.

Visions of a Connected Future, roundtable on AI, Copenhagen Institute for Futures Studies (08/01/2023)
Sustainable Diet Arguments on Twitter:
- Kunstig intelligens 'udvinder' god kommunikation om klimavenlige madvalg af 30.000 tweets, Klimamonitor (13/01/2023)
- Forskere bruger AI til at få viden om bæredygtig kost på Twitter, CSR.dk (17/01/2023)
- Nuggets mined from thousands of tweets can persuade us to eat more climate-friendly, UCPH SCIENCE News (30/01/2023)
- Kunstig intelligens kan måle folkestemningen: Nyttigt for firmaer og politikere, men kan det misbruges?, Videnskab.dk (30/01/2023)
- University Of Copenhagen Researchers Demonstrate AI’s Role In Sustainable Food, India Education Diary (11/02/2023)
- ‘Argument mining’: Guldkorn fra tusindvis af tweets kan overbevise os om at spise mere klimavenligt, POV International (27/04/2023)
Cultural Bias in ChatGPT:
- Samtalerobot er et redskab for amerikansk kulturimperialisme, Politiken (10/07/2023)
- ChatGPT fremmer amerikanske normer og værdier, Ekstra Bladet (10/07/2023)
- Studie: ChatGPT udbreder amerikanske normer, P1 Morgen (10/07/2023)
- ChatGPT har amerikansk bias, TV 2 Kosmopol (10/07/2023)
- Nyt studie: Populær chatbot promoverer amerikanske værdier og normer, Børsen (25/08/2023)
Danish Researchers Look To AI For Sustainable Future, Grady Newsource, Georgia, USA (21/07/2023)

I practice Kendo (Japanese fencing). I was even at the World Championships in Japan (2015) and Korea (2018).

Daniel Hershcovich
Department of Computer Science
University of Copenhagen
Lyngbyvej 2
DK-2100 Copenhagen Ø, Denmark
d... @di.ku.dk

(Google Scholar, Semantic Scholar)

Peer-Reviewed Publications

Preprints

Dissertations

Tutorials

University of Copenhagen

Hebrew University of Jerusalem

News

Projects

Publications(Google Scholar, Semantic Scholar)

Peer-Reviewed Publications

Preprints

Dissertations

Tutorials

Talks

Teaching

University of Copenhagen

Hebrew University of Jerusalem

Community

Media

Miscellaneous

Contact

(Google Scholar, Semantic Scholar)