Skip to main content

First-year seminar

Time: Tue 2021-12-14 16.15

Location: Fantum and Zoom

Video link: https://kth-se.zoom.us/j/63790435562

Lecturer: Ulme Wennberg

Abstract:
Welcome to my first year seminar. In this seminar, I will talk about
some of the challenges of my first year of my PhD. I will devote some
time to presenting our paper on positional information in transformer
architectures that appeared at ACL this summer, where we analyze
traditional position embeddings and show that they memorize linguistic
dependencies in a sub-optimal fashion, requiring far longer training
times than necessary. In the paper, we present a new way to view
positions in transformer architectures, that models linguistic impacts
more directly than preceding BERT-like models, yielding improved results
even without additional language modeling training. I will also address
some implementational challenges and plans for the second year of my PhD.
Belongs to: Speech, Music and Hearing
Last changed: Dec 14, 2021