Call for participation
###############################################################
Call for participation
NTCIR-11 Temporal Information Access (Temporalia) Task
http://ntcirtemporalia.github.io/
http://ntcir.nii.ac.jp/Temporalia/NTCIR-11-Temporalia/
Registration:
http://ntcir.nii.ac.jp/NTCIR11Regist/
Important Dates:
| Jan 20, 2014: | registration due |
| Mar 30, 2014: | testing data for formal run released |
| Jun 30, 2014: | formal run submissions due |
| Aug 1, 2014: | evaluation results released |
| Sep 1, 2014: | participant draft papers due |
| Nov 1, 2014: | camera ready papers due |
| Dec 9-12, 2014: | NTCIR-11 conference @ NII, Tokyo, Japan |
###############################################################
===OVERVIEW===
The objective of Temporalia is to foster research in temporal information access.
Given the fact that time plays crucial role in estimating information relevance and
validity we believe that successful search engines must consider temporal aspects of
information in greater detail.
We propose a challenge that establishes common grounds for designing and analyzing
temporally-aware information access systems.
We plan to focus on Information Retrieval first at NTCIR-11 with potential extension
to other applications at NTCIR-12 (hence the task name).
Temporal Information Retrieval has been gaining a lot of interest in IR and related
research communities.
It can be defined as a subset of document retrieval in which time plays crucial role
in estimating document relevance.
The objective of this task is to systematize various requirements in Temporal IR and
offer a standardized challenge based on which competing systems can be compared and analyzed.
Although many of temporal information needs seek for recent (fresh) information,
a good proportion of them also search for information about past incidents as well as future
incidents.
Although there are several evaluation tasks that involve search and filtering over
time (e.g., TDT, NTCIR GeoTime, TREC Temporal Summarization),
there is no test collection to measure the performance of search applications across
temporal information needs categories such as Past, Recent, and Future in a systematic way.
===TASKS====
Temporalia at NTCIR-11 will offer two subtasks to address temporal information access
technologies as follows.
Interested researchers and research groups can participate in either or both of the subtasks.
- Temporal Query Intent Classification (TQIC) subtask
TQIC subtask asks participants to classify a given query string to one of temporal classes:
Past, Recency, Future, or Atemporal.
Participants will receive a set of query strings and query submitting date,
and develop a system to classify each of the query strings to one of temporal classes.
Participants were allowed to use any external resources to complete the TQIC subtask
as long as the detail of external resource usage is described in their report.
This subtask does not require to index our document collection.
You can see the example queries for each class here:
http://ntcirtemporalia.github.io/NTCIR-11/taskdescription.html
- Temporal Information Retrieval (TIR) subtask
TIR subtask asks participants to retrieve a set of documents from a provided document
collection in response to a search topic that incorporates time factor.
In addition to a typical search topic description (i.e., title, description, and sub topics),
TIR search topic description will contain a query submitting date information to indicate
the relationship between the query and searcher.
This subtask requires to index the document collection.
You can see the example search topics here:
http://ntcirtemporalia.github.io/NTCIR-11/taskdescription.html
Training data for dry runs for both the tasks is expected to be released by Jan 30, 2014.
Please note:
Task participants must submit a paper for the NTCIR-11 meeting (unrefereed) and at least
one member of participated groups must attend the meeting at NII, Tokyo in December 2014.
Please visit Temporalia website for more information about the task:
http://ntcirtemporalia.github.io
http://ntcir.nii.ac.jp/Temporalia/NTCIR-11-Temporalia/ (Mirror)
===ORGANIZERS===
Hideo Joho (University of Tsukuba, Japan)
Adam Jatowt (Kyoto University, Japan)
Roi Blanco (Yahoo! Research Barcelona, Spain)
===CONTACT===
tc4fia at googlegroups dot com