CSE 422 · Information Retrieval · Track B
Text Retrieval on
CISI Dataset
Comparing Boolean, TF-IDF, and BM25 retrieval models with live search and full evaluation metrics on 1,460 library-science documents.
Tolga Biçer · Durmuş Burak Dirlikli · Şakir Yılmaz Öğüt
Dataset & Index Statistics
Loading stats…
System Pipeline
1
Parsing
CISI.ALL, CISI.QRY,
CISI.REL, CISI.BLN
→
2
Preprocessing
Lowercase · strip punct
Stopword removal
→
3
Inverted Index
term → {doc_id: tf}
82,404 postings
→
4
Retrieval
Boolean / TF-IDF / BM25
→
5
Evaluation
P@10 · Recall@10 · MAP
Retrieval Models
Boolean
Exact-match retrieval using AND, OR, NOT operators over the inverted index. The 35 CISI.BLN queries are evaluated; returns an unordered result set.
#and(term1, #or(term2, term3))P@10: 0.2486Recall@10: 0.0785MAP: 0.0775
TF-IDF
Scores relevance using Term Frequency (TF) and Inverse Document Frequency (IDF). Employs log normalization and cosine similarity (√doc_len). Evaluated on all 76 queries.
w(t,d) = (1+log tf) × log(N/df)P@10: 0.3184Recall@10: 0.1409MAP: 0.2118
BM25
Introduces term saturation (k₁=1.5) and length normalization (b=0.75). Performs worse than TF-IDF here because CISI's short, uniform documents limit the benefit of length normalization.
IDF × tf(k₁+1) / (tf + k₁(1−b+b·dl/avgdl))P@10: 0.2737Recall@10: 0.0991MAP: 0.1598