NTCIR Temporal Information Access (Temporalia) Task

Overview


Abstract

The objective of NTCIR Temporal Information Access (Temporalia) Task is to foster research in temporal information access. Given the fact that time plays a crucial role in estimating information relevance and validity we believe that successful search engines must consider temporal aspects of information in greater detail. At NTCIR-11, we built a test collections for temporal query intent classification a nd temporal adhoc retrieval. A total of 35 runs were submitted by 9 teams in Asia, Europe, and North America. NTCIR-12 will expand this achievement by considering the technological challenges such as query ambiguity detection and search results diversification in the context of temporal information retrieval. We also expand our language scope to include Chinese dataset in our collection.

Motivation

Temporal Information Retrieval has been gaining a lot of interest in IR and related research communities. It can be defined as a subset of document retrieval in which time plays crucial role in estimating document relevance. The objective of this task is to systematize various requirements in Temporal IR and offer a standardized challenge based on which competing systems can be compared and analyzed. Our recent analysis (Joho, et al., 2013) suggests that although many of temporal information needs seek for recent (fresh) information, a good proportion of them also search for information about past incidents as well as future incidents. Although there are several evaluation tasks that involve search and filtering over time (e.g., TDT, NTCIR GeoTime, TREC Temporal Summarization), there is no test collection to measure the performance of search applications across temporal information needs categories such as Past, Recent, Seasonal, and Future (cf. Khodaei and Alonso, 2012) in a systematic way.

Reference