Call for participation
###############################################################
Call for Task Participation
NTCIR-12 Temporal Information Access (Temporalia-2) Task
Website
http://ntcirtemporalia.github.io/
Facebook
https://www.facebook.com/ntcirtemporalia
Twitter
https://twitter.com/ntcirtemporalia
Registration:
http://ntcir.nii.ac.jp/NTCIR12Regist/
Important Dates (Tentative):
| Feb 28, 2015 | NTCIR-12 Kick-off Event |
Jun 30, 2015
Jul 30, 2015 |
Task Registration due
**Extended** |
| Jul 1, 2015 | Final task guideline released
Document Collection release |
Jul 1, 2015
Jul 7, 2015 |
Dry run testing data releaased
**Extended** |
| Sep 1, 2015 | Dry run submission due |
| Oct 1, 2015 | Dry run evaluation results released |
Nov 1, 2015
Nov 8, 2015 |
Formal run testing data released
**Extended** |
Dec 1, 2015
Dec 10, 2015 |
Formal run submissions due
**Extended** |
| Feb 1, 2016 | Formal run evaluation results released
Early task overview paper due |
| Mar 1, 2016 | Participant draft papers due |
| May 1, 2016 | Camera ready papers due |
| Jun 7-10, 2016 | NTCIR-12 conference @ NII, Tokyo, Japan |
###############################################################
===OVERVIEW===
The objective of NTCIR Temporal Information Access (Temporalia) Task is
to foster research in temporal information access.
Given the fact that time plays a crucial role in estimating information relevance
and validity we believe that successful search engines must consider temporal aspects
of information in greater detail.
At NTCIR-11, we built a test collections for temporal query intent classification
and temporal adhoc retrieval.
A total of 35 runs were submitted by 9 teams in Asia, Europe, and North America.
NTCIR-12 will expand this achievement by considering the technological challenges
such as query ambiguity detection and search results diversification in the context
of temporal information retrieval.
We also expand our language scope to include Chinese dataset in our collection.
===TASKS====
Temporalia-2 at NTCIR-12 offer two subtasks in English and Chinese to address temporal
information access technologies as follows.
Interested researchers and research groups can participate in either or both of
the subtasks in any combination of languages.
- Temporal Intent Disambiguation (TID) Subtask
TID subtask asks participants to estimate a distribution of four temporal intent classes
(Atemporal, Past, Recent, or Future) for a given query.
This is an upgraded task from
Temporal Query Intent Classification (TQIC) subtask at NTCIR-11,
where participants were only asked to estimate the best (single) temporal intent category for a given query.
Like the TQIC subtask, participants will receive a set of query strings and submission date,
and develop a system that estimates a distribution among four temporal intent classes.
TID will employ test queries more likely to be temporally ambiguous than those used in TQIC.
The answer distribution is estimated from the voting crowd workers.
- Temporally Diversified Retrieval (TDR) Subtask
TDR subtask will require participants to retrieve a set of documents relevant to
each of four temporal intent classes for a given topic description.
Participants are also asked to return a set of documents that is temporally diversifie for the same topic.
Participants will receive a set of topic descriptions, query issuing time,
and indicative search questions for each of temporal classes (Past, Recency, Future, and Atemporal).
Indicative search questions show one possible subtopic under a particular temporal class. Participants
are asked to develop a system that can produce a total of five search results per topic
(Past, Recency, Future, Atemporal, and Diversified).
This is an upgraded task from
Temporal Information Retrieval (TIR) subtask at NTCIR-11
where participants were asked to retrieve documents for separate temporal subtopics.
TDR is more challenging since the system needs to optimise ranking of documents for up to
four different temporal classes for a given topic.
Please note:
Task participants must submit a paper to the NTCIR-12 Conference (unrefereed) and at
least one member of participated groups must attend the conference at NII, Tokyo in June 2016,
to present your work.
===ORGANISERS===
Hideo Joho (University of Tsukuba, Japan)
Adam Jatowt (Kyoto University, Japan)
Roi Blanco (Yahoo Labs, Barcelona, Spain)
Haitao Yu (University of Tsukuba, Japan)
Shuhei Yamamoto (University of Tsukuba, Japan)
===CONTACT===
tc4fia at googlegroups dot com