HG7017: Computational Lexical Semantics
Thursday 9:30-13:30 HSS Computer Lab 2 (HSS-01-05)
In this course students will become familiar with how to represent
word meanings computationally and a variety of methods for
automatically determining the meaning of words. These include
dictionary based methods, such as LESK, graph based methods such as
UKB, supervised methods such as sequence classification and best paths
as well as vector space methods. We will finish with a discussion of
how to provide feedback from word sense disambiguation to meaning
The course is a seminar course, with participants choosing and
presenting papers each week. Because of this, content varies from
year to year depending on the composition of the class.
The course includes a strong computational component, students
will experiment with implementing systems and algorithms, in addition
to learning about them. By the end of this course, graduate students
will have an advanced knowledge of current computational approaches to
lexical meaning. Students should be able to evaluate lexical resources
for coverage and quality as well as to design and implement systems
for determining meaning in text.
This year (2104) we will try to cover at least:
- Lexical Semantics in Definitions (LESK)
- Distributional Approaches (e.g. Vector Spaces)
- Structural Semantics (e.g. Semantic Dependencies)
- Graph based approaches (e.g. personalised page rank)
BabelNet (a much richer graph)
- Structure + Distributional Semantics (Socher)
- Structural + Lexical (AMR)
- Knowledge Acquisition (e.g. learn from definitions or text
snippets; also learning regular expression from examples)
Final Deadline 2014-12-01
PROJECT TWO DUE 2014-12-01 11:59 (email is fine)
- Seminar 1 2014-08-14
- Seminar 2 2014-08-21
- Semantic Dependencies (Rose Chen:
- Deep Learning over distributional spaces applied to learning
predominant senses (Huizhen Wang:
- Seminar 3 2014-08-28
- Recursive Deep Learning (Egor:
- Cross-lingual sense projection (Giulia:
- BabelNet & WSD (Luis:
- R. Navigli and S. Ponzetto. 'BabelNet: The Automatic Construction, Evaluation and Application of a Wide-Coverage Multilingual Semantic Network. Artificial Intelligence, 193, Elsevier, 2012, pp. 217-250.
- R. Navigli, D. A. Jurgens, D. Vannella. SemEval-2013 Task 12: Multilingual Word Sense Disambiguation. Proc. of 7th International Workshop on Semantic Evaluation (SemEval), in the Second Joint Conference on Lexical and Computational Semantics (*SEM 2013), Atlanta, USA, June 14-15th, 2013, pp. 222-231.
- Seminar 4 2014-09-04 (FCB has a clash 10:30-12:30)
- Seminar 5 2014-09-11
- Learning Ontologies from Definitions (David:
- Word Sense Disambiguation (Tuan Anh:
- Seminar 6 2014-09-18
- Seminar 7 2014-09-25
- Seminar 8 2014-10-16
- Combining Resources (David:
- The Integrated Semantic Framework (Francis:
- Seminar 9 2014-10-23 (short class)
PROJECT ONE DUE 2014-10-24 11:59 (email is fine)
- Seminar 10 2014-10-30 (+ invited talk)
- Seminar 11 2014-11-06 (no presentations
--- individual meetings about your projects as necessary)
- Seminar 12 2014-11-13 (Late start: 12:30–)
- Final discussion:
What are the most important problems in your field?
Presentation One (20%): assigned subject
Present a paper (or series of papers), along with an extended
motivation/background and some discussion of applications. We will
discuss both the content of the paper, and also the form of your
presentation. One goal of the course is to learn how to present
Presentations should take an hour, with 30-40 minutes of
presentation and 30-20 minutes discussion. You should give me the
corrected slides within a week of the presentation, and I will add
them to the web page.
- Seminar 2: Rose Chen; Huizhen Wang
- Seminar 3: Egor, Giulia
- Seminar 4: Yukun, Luis
- Seminar 5: David, Tuan Anh
Presentation Two (20%): your choice (must be OKed: should fit
in with your research)
Present a paper (or series of papers), along with an extended
motivation/background and some discussion of applications. Can be
your own work (if it is ready).
- Seminar 7: Tuan Anh, Mike
- Seminar 8: David
- Seminar 9: ???
- Seminar 11: ???
Project One (30%): assigned topic
Sample: critically evaluate one lexical resource (such as WordNet
and tagged corpus). The evaluation will include writing code to
summarize its properties and compare it to other resources.
The project must include a computational component, a presentation,
and a written component. In the computational component the student
will be expected to write to code analyse data and solve problems. In
the presentation, they will present the results to their lecturer and
peers. In the written component they will explain their results in the
form of a short paper. They should incorporate any feedback from their
presentation in the final paper. The final paper should follow
guidelines for TACL.
- Tuan Anh: Extended Simplified LESK with the wordnet gloss corpus
- Egor: Examining compositionally with vector-spaces
OR phrase clustering with
- Giulia: Cross-lingual mapping: projection vs intersection
- David: ? identifying and exploiting definitions in Indonesian
and maybe other languages
- Yukun: Named entity
Project Two (30\%): your choice (must be OKed: should fit in with your research)
Sample: implement (or extend an existing implementation of) a word
sense disambiguation system.
As there is no textbook which covers the topics to the depth required, we
will rely mainly on readings. the following textbooks are recommended for
- Representing Word Meaning
- Word Sense Disambiguation
- Dictionary based methods
- Supervised Methods
- Cross lingual disambiguation
- Graph based methods
- Vector Space Methods
- Method Combination
- WSD and applications
- Machine Translation
- Intelligent Indexing and the Semantic Web
- Computational Lexical Semantics Relational meaning and WordNets; Vector Spaces; Ontologies (Stevenson, 2003, ch 3); (Miller, 1998), (Vossen, 1998, ch2)
- Splitting or Lumping: Granularity; Meaning across languages; Multiword expressions (Navigli, 2006),(Stevenson, 2003, ch 2)
- Dictionary based methods for WSD LESK, ALT (Lesk, 1986; Ikehara et al., 1996; Baldwin et al., 2010)
- Supervised methods for WSD. CL Special Issue on WSD (www.aclweb.org/anthology-new/J/J98/)
- Cross lingual disambiguation for WSD (Dagan and Pereira, 1994; Zhong and Ng, 2009)
- Graph based methods for WSD (Agirre and Soroa, 2009; Agirre et al., 2009)
- Vector space methods for WSD (Widdows, 2004, ch 2–4)
- Method Combination (Stevenson, 2003, ch 6)
- Machine Translation (Carpuat et al., 2006; Chan et al., 2007)
- Intelligent Indexing and the Semantic Web (Shadbolt et al., 2006)
- WSD and the Representation of Meaning (Stevenson, 2003, ch 7)
- Agirre, Eneko and Aitor Soroa. (2009) .Personalizing pagerank for word sense disambiguation. In 12th conference of the European chapter of the Association for Computational Linguistics (EACL-2009), Greece.
- Agirre, Eneko, Oier Lopez de Lacalle, and Aitor Soroa. (2009). Knowledge-based WSD and specific domains: performing over supervised WSD. In International Joint Conference on Artificial Intelligence (IJCAI-2009), Pasadena.
- Baldwin, Timothy, Su Nam Kim, Francis Bond, Sanae Fujita, David Martinez, and Takaaki Tanaka. (2010). A reexamination of MRD-based word sense disambiguation. ACM Transactions on Asian Language Information Processing (TALIP). (to appear).
- Chan, Yee Seng, Hwee Tou Ng, and David Chiang. (2007).Word sense disambiguation improves statistical machine translation. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 33–40, Prague, Czech Republic. Association for Computational Linguistics. URL http://www.aclweb.org/anthology/P/P07/P07-0005.
- Carpuat, Marine, Pascale Fung, and Grace Ngai. (2006). Aligning word senses using bilingual corpora. ACM Transactions on Asian Language Information Processing (TALIP), 5(2):89–120, ISSN 1530-0226. doi: http://doi.acm.org/10.1145/1165255.1165256.
- Dagan, Ido and Fernando Pereira. (1994). Similarity-based estimation of word cooccurrence probabilities. In Proceedings of the 32nd Annual Meeting of the Association for Computational Linguistics, pages 272–278, Las Cruces, New Mexico, USA. Association for Computational Linguistics. doi: 10.3115/981732.981770. URL http://www.aclweb.org/anthology/P94-1038.
- Ikehara, Satoru, Satoshi Shirai, and Francis Bond. (1996). Approaches to disambiguation in ALT-J/E. In International Seminar on Multimodal Interactive Disambiguation: MIDDIM-96, pages 107–117, Grenoble.
- Lesk, Michael (1986). Automatic sense disambiguation: How to tell a pine cone from an ice cream cone. In Proceedings of the 1986 SIGDOC Conference, pages 24–26, New York, ACM.
- Miller, George. (1998). Nouns in WordNet. In Christine Fellbaum, editor, WordNet: An Electronic Lexical Database, chapter 1, pages 23–46. MIT Press.
- Navigli, Roberto. (2006). Meaningful clustering of senses helps boost word sense disambiguation performance. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, pages 105–112, Sydney, Australia.
- Shadbolt, Nigel, Wendy Hall, and Tim Berners-Lee. (2006). The semantic web revisited. IEEE Intelligent Systems, pages 1541–1672. (http://eprints.ecs.soton.ac.uk/12614/1/Semantic_Web_Revisted.pdf).
- Stevenson, Mark. (2003). Word Sense Disambiguation. CSLI Publications.
- Vossen, Piek, editor. (1998). Euro WordNet. Kluwer.
- Widdows, Dominic. (2004). Geometry and Meaning. CSLI Publications.
- Zhong, Zhi and Hwee Tou Ng. (2009). Word sense disambiguation for all words without hard labor. In International Joint Conference on Artificial Intelligence (IJCAI-2009), pages 1616–162.
One of the core areas of study in computational linguistics is the
study of how words and sentence meaning is represented. This course
covers both linguistics and computational issues, with an emphasis on
the latter. The course also offers a grounded introduction to some
general natural language processing techniques, such as sequence
labeling, graph search, comparison of similarity and method
Aims and objectives
This course is designed to provide an advanced knowledge of current
computational approaches to lexical meaning. Students should be able
to evaluate lexical resources for coverage and quality; design and
implement systems for determining meaning in text; and present their
results effectively. Students will do two small research projects
during the course.
On completion of this course, the students should be able to:
- Critically evaluate lexical resources for cover and accuracy.
- Understand the main approaches to representing word meaning computationally
- Read, understand and present other people's research
- Be able to implement or enhance an existing method for word sense disambiguation
- Conduct independent research on word sense disambiguation
Computational Linguistics Lab
Division of Linguistics and Multilingual Studies
Nanyang Technological University
Level 3, Room 55, 14 Nanyang Drive, Singapore 637332
Tel: (+65) 6592 1568; Fax: (+65) 6794 6303