Lapata, Mirella and Frank Keller. 2004. The Web as a Baseline: Evaluating the Performance of Unsupervised Web-based Models for a Range of NLP Tasks. In Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics, 121-128. Boston.
Previous work demonstrated that web counts can be used to approximate bigram frequencies, and thus should be useful for a wide variety of NLP tasks. So far, only two generation tasks (candidate selection for machine translation and confusion-set disambiguation) have been tested using web-scale data sets. The present paper investigates if these results generalize to tasks covering both syntax and semantics, both generation and analysis, and a larger range of n-grams. For the majority of tasks, we find that simple, unsupervised models perform better when n-gram frequencies are obtained from the web rather than from a large corpus. However, in most cases, web-based models fail to outperform more sophisticated state-of-the-art models trained on small corpora. We argue that web-based models should therefore be used as a baseline for, rather than an alternative to, standard models.
@InProceedings{Lapata:Keller:04,
author = {Mirella Lapata and Frank Keller},
title = {The Web as a Baseline: Evaluating the Performance of
Unsupervised Web-based Models for a Range of {NLP} Tasks},
crossref = {NAACL:04},
pages = {121--128}
}
@Proceedings{NAACL:04,
title = {Proceedings of the Human Language Technology Conference of the North
American Chapter of the Association for Computational Linguistics},
booktitle = {Proceedings of the Human Language Technology Conference of the North
American Chapter of the Association for Computational Linguistics},
year = 2004,
address = {Boston}
}