(5/5) We then propose two approaches to improve robustness: MAS augments input data with synonyms to increase lexical variation during training, and ADV uses adversarial training. Both are pretty effective at improving the performance. See the paper for more detail:

(4/5) Now we can test how much substituting in synonymous material affects state-of-the-art models. Accuracy drops dramatically!

(3/5) To test how big this effect is, we created a new dataset, Spider-Syn (based on Spider), in which we substitute words and phrases with synonymous material:

We now have a Twitter account for the Computational Linguistics / NLP lab at @QMEECS / @QMUL - watch this space for updates! compling.eecs.qmul.ac.uk/ #NLProc