All TID and TDR runs (a run refers to the results for a subtask) must be compressed (zip). Each team can submit up to 3 runs per subtask-language pair. As the run files must be encoded in UTF-8.
Please submit your run file to tc4fia at googlegroups.com with the subject "Temporalia-2 Dry Run (or Formal Run) Submission". We will send a confirmation message as soon as possible.
Run files should be named as "<Group-ID>-[TID/TDR]-[CE]-[1/2/3].txt"
<Group-ID> is exactly the Group ID when you registered in NTCIR-12.
[TID/TDR]: Use either TID or TDR. TID for Temporal Intent Disambiguation runs, TDR for Temporally Diversified Retrieval runs.
[CE]: C for Chinese runs, E for English runs.
[1/2/3]: Use a number inside {1,2,3} to denote the priority of the run (1>2>3). Due to limited resources, we may not include all submitted runs in the result pool. Therefore, it is important for you to point out in which order we should take your runs into consideration for result pool construction.
For all runs in TID, the first line of the run file must be of the form:
<SYSDESC>[insert a short description of your model in English here]</SYSDESC>The rest of the file should contain tab-separated lines of the form (probability value: 3 digits to the right of the decimal point):
[TopicID] [Probability of Past] [Probability of Recency] [Probability of Future] [Probability of Atemporal] [RunName]\nFor example, a segment of an English TID run should look like this:
074 0.250 0.250 0.250 0.250 ORG-TID-E-1
075 0.250 0.250 0.250 0.250 ORG-TID-E-1You can return up to 100 documents for each subtopic.
For all runs in TDR, the first line of the run file must be of the form:
<SYSDESC>[insert a short description of your model in English here]</SYSDESC>The rest of the file should contain tab separated lines of the form:
[id] [rank] [doc_id] [score] [RunName]\n"id" is the query id, which is a concatenation of the query id and "p", "r", "f", "a" or "d" that indicate the past, recency, future, atemporal subtopics and diversified ranking respectively. "rank" is the rank of the document for that query.
For example, a segment of an English TDR run should look like this:
001d 1 doc_i 0.25 ORG-TDR-E-1
001d 2 doc_k 0.2 ORG-TDR-E-1