NTCIR Temporal Information Access (Temporalia) Task

Call for participation


###############################################################
Call for participation

NTCIR-11 Temporal Information Access (Temporalia) Task
http://ntcirtemporalia.github.io/
http://ntcir.nii.ac.jp/Temporalia/NTCIR-11-Temporalia/

Registration:
http://ntcir.nii.ac.jp/NTCIR11Regist/

Important Dates:
Jan 20, 2014:registration due
Mar 30, 2014:testing data for formal run released
Jun 30, 2014:formal run submissions due
Aug 1, 2014:evaluation results released
Sep 1, 2014:participant draft papers due
Nov 1, 2014:camera ready papers due
Dec 9-12, 2014:NTCIR-11 conference @ NII, Tokyo, Japan
###############################################################

===OVERVIEW===

The objective of Temporalia is to foster research in temporal information access. Given the fact that time plays crucial role in estimating information relevance and validity we believe that successful search engines must consider temporal aspects of information in greater detail. We propose a challenge that establishes common grounds for designing and analyzing temporally-aware information access systems. We plan to focus on Information Retrieval first at NTCIR-11 with potential extension to other applications at NTCIR-12 (hence the task name).

Temporal Information Retrieval has been gaining a lot of interest in IR and related research communities. It can be defined as a subset of document retrieval in which time plays crucial role in estimating document relevance. The objective of this task is to systematize various requirements in Temporal IR and offer a standardized challenge based on which competing systems can be compared and analyzed. Although many of temporal information needs seek for recent (fresh) information, a good proportion of them also search for information about past incidents as well as future incidents. Although there are several evaluation tasks that involve search and filtering over time (e.g., TDT, NTCIR GeoTime, TREC Temporal Summarization), there is no test collection to measure the performance of search applications across temporal information needs categories such as Past, Recent, and Future in a systematic way.

===TASKS====

Temporalia at NTCIR-11 will offer two subtasks to address temporal information access technologies as follows. Interested researchers and research groups can participate in either or both of the subtasks.

- Temporal Query Intent Classification (TQIC) subtask
TQIC subtask asks participants to classify a given query string to one of temporal classes: Past, Recency, Future, or Atemporal. Participants will receive a set of query strings and query submitting date, and develop a system to classify each of the query strings to one of temporal classes. Participants were allowed to use any external resources to complete the TQIC subtask as long as the detail of external resource usage is described in their report. This subtask does not require to index our document collection.
You can see the example queries for each class here:
http://ntcirtemporalia.github.io/NTCIR-11/taskdescription.html

- Temporal Information Retrieval (TIR) subtask
TIR subtask asks participants to retrieve a set of documents from a provided document collection in response to a search topic that incorporates time factor. In addition to a typical search topic description (i.e., title, description, and sub topics), TIR search topic description will contain a query submitting date information to indicate the relationship between the query and searcher. This subtask requires to index the document collection.
You can see the example search topics here:
http://ntcirtemporalia.github.io/NTCIR-11/taskdescription.html

Training data for dry runs for both the tasks is expected to be released by Jan 30, 2014.

Please note:
Task participants must submit a paper for the NTCIR-11 meeting (unrefereed) and at least one member of participated groups must attend the meeting at NII, Tokyo in December 2014.

Please visit Temporalia website for more information about the task:
http://ntcirtemporalia.github.io
http://ntcir.nii.ac.jp/Temporalia/NTCIR-11-Temporalia/ (Mirror)

===ORGANIZERS===

Hideo Joho (University of Tsukuba, Japan)
Adam Jatowt (Kyoto University, Japan)
Roi Blanco (Yahoo! Research Barcelona, Spain)


===CONTACT===

tc4fia at googlegroups dot com