Haystacks and hypotheses

• Published in: Proceedings of the American Society for Information Science and Technology Vol. 40; no. 1; pp. 59 - 64
• Main Authors: Demaine, Jeffrey, Martin, Joel, De Bruijn, Berry
• Format: Journal Article
• Language: English
• Published: Hoboken Wiley Subscription Services, Inc., A Wiley Company 01.10.2003
• Subjects: Bibliometrics; Biomedicine; Information Retrieval; Information Seeking; Information Systems; Linkage Analysis; Linking Agents; Linking Structures; Medical Research; Medicine; Relevance (Information Retrieval)
• ISSN: 0044-7870; 1550-8390; 1550-8390
• DOI: 10.1002/meet.1450400107

This paper describes the EurekaSeek bibliometric technique for automated linked‐literature analysis. The MEDLINE database of biomedical literature is iteratively searched in order to identify research opportunities in the form of conceptual linkages between terms. As a tool for identifying undiscovered public knowledge, EurekaSeek is a variation on the techniques of Swanson and Smalheiser. EurekaSeek uses medical subject headings instead of text analysis in a fully automated search process, thereby eliminating the reliance on expert input during the process of linking literatures. In this paper, the EurekaSeek process is tested by retroactively examining the co‐occurrence of terms in the published literature. The hypothesis tested in this paper is whether this tool, had it existed in the past, could have identified conceptual linkages that occurred only later in the literature. In addition, EurekaSeek is compared against a process that considers all potential term‐to‐term relationships. The list of terms that EurekaSeek produces is a subset of all potential linked literature terms. The experiment shows that EurekaSeek produces a higher percentage of likely hypotheses than when all terms are considered. While the proportion of identified linkages generated is still too small for the process to be a practical aid to research, statistically significant results were achieved. Metaphorically speaking, EurekaSeek identifies a higher proportion of needles per haystack.