Pembentukan Passage dalam Question Answering System untuk Dokumen Bahasa Indonesia

Abstract

Indonesia. Supervised by AHMAD RIDHA. Passages are used by question answering system to get pieces of relevant documents. This research compared various aspects of passages: overlapping and non-overlapping passages, sentencebased and word-based passages, and passage formation time (before and after indexing). Types of question in this research are siapa (who), di mana (where), kapan (when), and berapa (how many). For indexing and retrieval process, we used BM25 and proximity algorithms from Sphinx. Top documents or passages were re-weighted using rules to get passages containing answers candidate. Answer extraction was performed using the smallest distance between query and candidate answers. Evaluation was conducted using mean reciprocal rank and answer accuracy (four criteria: Right, Unsupported, Wrong, and Null). The best result was obtained using BM25 for two kinds of passage, namely, 20 overlapping words with 80% accuracy and 30 overlapping words with 77.5% accuracy, where both considered one tag as one word and were formed after indexing. The best result for proximity were obtained three kinds of passages, namely, 2 overlapping sentences with 77.5% accuracy, 2 non-overlapping sentences with 77.5% accuracy, and 20 overlapping words with 77.5% accuracy, they also considered one tag as one word and were formed after indexing. The average performance based on mean reciprocal rank for passage by using BM25 and Proximity are 75.1% and 76.1%, respectively. The passages formed after indexing have better accuracy which indicates retrieving relevant documents is important for question answering system.