Clinical Machine Learning at University Hospital Zurich / He, Him
Joined October 2015
- Tweets275
- Following1.2K
- Followers366
- Likes1.3K
*The meaning of prompts* (2)
With regards to soft-prompt tuning, Khashabi et al. (2022) found that good-performing soft prompts can be mapped to arbitrary text as the mapping between continuous and discrete spaces is *not* one-to-one.
*The meaning of prompts* (1)
Webson and Pavlick (2022) state that it is unlikely that a "models’ improvement is derived from models understanding task instructions in ways analogous to humans’ use of task instructions"
*Correct labeling is less important*
Min et al. (2022, EMNLP) show that replacing gold labels with random labels only marginally hurts performance; trend is consistent over nearly all models tested and independent of prompt template
*Various biases lead to high result variance*
Zhao et al. (2021) demonstrate that result variance can be caused by phenomena such as recency bias, majority label bias and common token bias
*Results are sensitive to order*
As shown by Lu et al. (2022) and Zhao et al. (2021), in-context example order has a strong influence on the few-shot performance and there seems to be an “optimal” order of in-context examples (Kumar and Talukdar, 2022)
Automatically adding such information is challenging because the required context is different for each instance - we acknowledge the fact that this will introduce noise and use a dual-encoder architecture with cross-attention to inject contextual information in a robust manner.
In the example below, it is challenging
to follow the reasoning of the text if the meaning
of school spirit is left unclear. The provided context is not part of the dataset and was extracted from a
large language model via prompting.
Interesting: it seems like the more information (e. g. Code, hyperparams tuning, statistical significance tests) you provide about your experiments, the more likely it will be accepted. @JesseDodge #ACL2022
Copying paper titles from PDFs often results in missing the first letter and I always wonder how much difference it makes when using #googlescholar #NLProc
