Patient records in the patient administration department at Naval Medical Center San DiegoWIKIMEDIA, AMANDA L. KILPATRICKThe increased use of electronic medical records (EMRs) is supporting widespread data-mining efforts to uncover trends in health, disease, and treatment response data. But a significant chunk of information in EMRs remains stored as text, unusable by conventional data-mining methods. These semi-structured or unstructured data include clinical notes, certain categories of test results such as echocardiograms and radiology reports, and other important documentation. To take full advantage of EMRs, we need to utilize both data- and text-mining techniques to explore patient outcomes.
Text and data mining have much in common; underlying each is the assumption that knowledge lies buried in a scattered mass of information. But whereas data mining predominately relies on statistical methods to uncover trends in structured data, text-mining techniques seek to make sense of information that is unstructured, such as a doctor’s scribbles on a patient’s chart. For example, much of the available clinical data are in narrative form as a result of transcription of dictations, direct entry by providers, or use of speech-recognition applications. This “free-text” form is convenient to express concepts and events, but is difficult to search, summarize, and analyze. Fortunately, text-mining techniques can help code these data for analysis.
Text mining normally requires a pre-processing phase such as spell checking, ...
