Of the Top 10 most-searched medical conditions on Wikipedia, nine contained errors, according to a soon-to-be-published study led by Campbell Medical School’s regional associate dean, Dr. Robert Hasty. His article will be the nation’s first published study on the accuracy of the world’s most-used medical source.
BY BILLY LIGGETT
The question first entered the mind of Dr. Robert Hasty two years ago in a room full of young residents at a teaching hospital in Florida. The vice chairman of Nova Southeastern University’s Internal Medicine Department and the founding director of Palmetto General Hospital’s residency program at the time, Hasty posed a question to soon-to-be physicians and noticed many of them immediately breaking out their smart phones and tablets to find the answer.
Their online destination? Wikipedia — today considered the “single leading source” of health care information for medical students, professionals and even patients, according to a 2014 study by the IMS Health Institution.
Hasty’s question? Just how reliable of a source can Wikipedia be when the information can be written and edited by anybody?
“Wikipedia is a bit like flatulence for clinicians. Everyone does it, but no one wants to admit to it,” Hasty says. “It’s an easy-to-access source for anybody, but who’s editing it? That’s the scary question. It can come from anywhere — a first-year med student or a professional who could stand to gain by promoting one treatment over another. Those are valid concerns.”
Hasty, now the vice president of medical education and regional associate dean for the Jerry M. Wallace School of Osteopathic Medicine, is the lead author of the country’s first published study that examines the accuracy of content for common medical conditions on Wikipedia. “Wikipedia in Comparison to Peer-Reviewed Medical Knowledge in the 10 Most Common Medical Conditions” will appear in May in the Journal of the American Osteopathic Association. In their study, Hasty and 16 co-authors — most of them residents during Hasty’s time in Florida — discovered that of the 10 most-searched medical conditions, nine of the 10 Wikipedia articles on those conditions contained “statistically significant errors.”
“The idea behind Wikipedia is that the masses make it a better product over time, but our research says it still has a long way to go,” Hasty says. “That’s not to say there aren’t errors in peer-reviewed literature as well. But there’s more cause for concern when the material online doesn’t have the appropriate peer-vetting process.”
Significant Discordance
To get an idea of the behemoth Wikipedia has become in its 13-year history, consider this: on any given day, 15 percent of all internet users visit the site at least once. The online encyclopedia has 1,600 times as many articles as its forefather, the Encyclopedia Britannica, and it is the world’s fifth-most popular website with editions in 287 languages.
If you were to print out Wikipedia — a task groups like Pedia Press and IndieGoGo are ready to undertake — the result would be almost 1.2 million pages long.
A large chunk of those pages would be dedicated to health care. According to the IMS Health Institute, the top 100 English language health care-related pages on Wikipedia were accessed 1.9 million times in 2013. In its recent study, the group reported that information on these pages is subject to constant change, often overseen by “informal or formal working groups.”
The lack of formal peer review is what spawned Hasty’s study.
“The slightest change or miswording can change public health fairly significantly,” Hasty says. “[Physicians and students] have developed confidence in using Wikipedia, thinking it’s a well-reviewed peer-referenced publication. For patients, it’s the first thing that pops up on a Google search.”
Hasty and his team took the Top 10 most common conditions in terms of expenditure in the U.S. and found their corresponding Wikipedia articles for their research — coronary artery disease, concussions, lung cancer, depression, arthritis, chronic obstructive pulmonary disease, hypertension, diabetes, back pain and hyperlipidemia. In a blinded process, two randomly assigned students reviewed each article and identified all assertions [defined as “a confident statement of fact or belief”] made in the article. They compared those assertions made in Wikipedia with references made in UpToDate, the peer-reviewed medical encyclopedia source purchased by Campbell’s med school and currently used by its students and faculty.
Once each article was reviewed, the findings were tabulated by two different independent reviewers, and the data was sent off to a statistician. The result: Hasty and his co-authors found errors in nine out of the 10 Wikipedia articles, with “concussions” the only entry found to be error-free.
“I think Wikipedia has great potential, and our study shows that more work needs to be done to improve accuracy for a resource used by the majority of clinicians.”
According to Hasty, most of the errors were “relatively minor.” One example of a significant error was with the diagnosis of hypertension, where Wikipedia stated that traditionally, the diagnosis is required to have “three separate sphygmomanometer measurements at one monthly interval.” The best available evidence and guidelines only require two measurements.
“The extra reading could potentially cause patients to go longer without treatment of their high blood pressure,” Hasty says.
Another error involved coronary artery disease. Wikipedia’s article states that family history is not an important risk factor in the disease, but according to Hasty’s study, “Multiple studies confirm or support the importance of family history of CAD in determining a patient’s risk.”
The study’s conclusion: Caution should be used when using Wikipedia to answer questions regarding patient care. But Hasty says that doesn’t mean there isn’t room for improvements.
“I think Wikipedia has great potential, and our study shows that more work needs to be done to improve accuracy for a resource used by the majority of clinicians,” Hasty says. “Already, there have been interesting reports since we started this. There’s a group of med students at UC-San Francisco working on a project to improve the site.”
Hasty’s study will be published in May, but already doctors around the country are raising the red flag on Wikipedia’s content. UCSF professor of psychiatry Dr. Amin Azzam told NPR in January he believes Wikipedia is a “double-edged sword.”
“Because anyone can edit, we don’t necessarily know the expertise of the people doing the editing,” he said. “One the other hand, the reason it’s so popular is because everyone can contribute.”
Hasty says he hopes the study leads to changes on the website.
“I hope Wikipedia looks at this and decides to make a more robust peer-review system,” he says. “Some of the errors we found were subtle, but some could change the way a future physician might practice medicine. There’s a reason why there’s a high standard. A slight deviation could seriously harm people.”