Identifying the optimal set of parameters for new topic identification through experimental design

Abstract

Users are interested in multiple topics during a search session, and identifying the boundaries of search sessions is an important task. This study proposes to use neural networks for defining the topic boundaries in search engine transaction logs, and is a part of ongoing research on automatic new topic identification. The objective of the study is to determine the best set of parameters for neural networks that are designed to perform automatic new topic identification. Sample data logs from FAST (currently owned by Yahoo) and Excite (currently owned by IAC Search & Media) search engines were analyzed. The findings show that neural networks are fairly successful in identifying topic continuations and shifts in search engine transaction logs. The choice of the neural network structure depends on which performance measure is more important to the user. For a certain performance measure, there is a set of parameters of neural networks that will increase the performance of new topic identification in search engine transaction logs. In addition, the threshold value of the output level of neural networks is the most influential parameter on the performance of new topic identification.