Yansong Feng and Mirella Lapata. 2008. Automatic Image Annotation Using Auxiliary Text Information. In Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, 272-280. Columbus, OH.

The availability of databases of images labeled with keywords is necessary for developing and evaluating image annotation models. Dataset collection is however a costly and time consuming task. In this paper we exploit the vast resource of images available on the web. We create a database of pictures that are naturally embedded into news articles and propose to use their captions as a proxy for annotation keywords. Experimental results show that an image annotation model can be developed on this dataset alone without the overhead of manual annotation. We also demonstrate that the news article associated with the picture can be used to boost image annotation performance.

@InProceedings{feng-lapata:2008:ACLMain,
  author    = {Feng, Yansong  and  Lapata, Mirella},
  title     = {Automatic Image Annotation Using Auxiliary Text Information},
  booktitle = {Proceedings of ACL-08: HLT},
  year      = {2008},
  address   = {Columbus, Ohio},
  pages     = {272--280},
}