藍色情懷

大部分的科學研究因草率分析而被玷污

Most Science Studies Appear to Be Tainted By Sloppy Analysis
http://online.wsj.com/article/SB118972683557627104.html

WSJ, September 14, 2007; Page B1
By ROBERT LEE HOTZ

每個人都會犯錯，而且，假如你相信醫界學者 John Ioannidis 的話，那麼科學家所犯的錯誤比起他們真正的貢獻（fair share）要來的多。經由他的計算，大份已發表的研究發現都是錯的。

We all make mistakes and, if you believe medical scholar John Ioannidis, scientists make more than their fair share. By his calculations, most published research findings are wrong.

Ioannidis 博士是一位流行病學家，他在希臘 Ioannina 大學醫學院與麻州 Medford 的 Tufts 大學研究「研究方法」。在一系列有影響力的分析報告中，他記載在每年數千篇經同儕審核（ peer-reviewed）而發表的研究報告中，有很多都與所見不同（so much less than meets the eye）。

Dr. Ioannidis is an epidemiologist who studies research methods at the University of Ioannina School of Medicine in Greece and Tufts University in Medford, Mass. In a series of influential analytical reports, he has documented how, in thousands of peer-reviewed research papers published every year, there may be so much less than meets the eye.

這些有瑕疵的發現，在大多數情況下，並非根源於騙局 （fraud）或形式上的不規矩（ formal misconduct），而是來自於世俗的不良品行（misbehavior）：錯算（miscalculation）、粗劣的研究設計（poor study design）或自私的資料分析（self-serving data analysis）。 "現代研究中有愈來愈多的顧慮，在已出版的研究主張中，謬誤的發現可能佔了多數或甚至是大多數，" Ioannidis 博士說。 "關於一個研究發現的新主張，很有可能是假的而非真的。"

These flawed findings, for the most part, stem not from fraud or formal misconduct, but from more mundane misbehavior: miscalculation, poor study design or self-serving data analysis. "There is an increasing concern that in modern research, false findings may be the majority or even the vast majority of published research claims," Dr. Ioannidis said. "A new claim about a research finding is more likely to be false than true."

該研究領域愈熱門，對於其所發表的發現更應該以存疑心態檢視，他果斷的說。

The hotter the field of research the more likely its published findings should be viewed skeptically, he determined.

以這項發現為例：男性與女性之間疾病發生的風險也許不同，端賴他們的基因。研究已在高血壓、精神分裂症與多發性硬化症、還有肺癌與心臟病上顯著地報告性別上 的差異。在上個月於 Journal of the American Medical Association 所發表的研究中，Ioannidis 博士與其同事分析了 432 篇已發表的，關於性別與基因的研究主張。

Take the discovery that the risk of disease may vary between men and women, depending on their genes. Studies have prominently reported such sex differences for hypertension, schizophrenia and multiple sclerosis, as well as lung cancer and heart attacks. In research published last month in the Journal of the American Medical Association, Dr. Ioannidis and his colleagues analyzed 432 published research claims concerning gender and genes.

基於更仔細的檢查，幾乎沒有一篇受到推舉（held up）。只有一篇被複製。

Upon closer scrutiny, almost none of them held up. Only one was replicated.

就統計上來說，科學遭受過度顯著（significance）之苦。過於熱切的研究者經常對他們所分析的統計變量進行過多的修補，以便從他們的資料集（data sets）中得出任何有意義的洞見。(聖經密碼不就是玩著相同的把戲嗎？這是以科學手段來助長迷信的歪風，讓我想到，上星期觀看「福爾摩斯」電影後，對「神權統治」的看法，有更深切的體會！)

Statistically speaking, science suffers from an excess of significance. Overeager researchers often tinker too much with the statistical variables of their analysis to coax any meaningful insight from their data sets. 

"人們玩弄資料以發現任何看起來有意義的事，以彰顯他們發現了某些新奇且不平常的事，" Ioannidis 說。

"People are messing around with the data to find anything that seems significant, to show they have found something that is new and unusual," Dr. Ioannidis said.

在美國，研究是一個每年價值 550 億美元的事業，那是將它的可靠性堆砌在證據的可信賴性上，而 Ioannidis 博士的研究則擊中痛處（strikes a raw nerve）。事實上，他在 2005 年所撰寫的文章，「為何大部分已發表的研究都是錯的（Why Most Published Research Findings Are False）」，仍是 PLoS Medicine 期刊發行以來，下載數量最多的技術性報告。

In the U. S., research is a $55-billion-a-year enterprise that stakes its credibility on the reliability of evidence and the work of Dr. Ioannidis strikes a raw nerve. In fact, his 2005 essay "Why Most Published Research Findings Are False" remains the most downloaded technical paper that the journal PLoS Medicine has ever published.

"他完成系統化的檢視已發表的文獻，並且 憑藉著經驗向我們展現我們內心所深知的東西，" Muin Khoury，美國疾管局 National Office of Public Health Genomics 主任。 "我們更需注意已發表科學結果的「複製（replication）」。"

"He has done systematic looks at the published literature and empirically shown us what we know deep inside our hearts," said Muin Khoury, director of the National Office of Public Health Genomics at the U.S. Centers for Disease Control and Prevention. "We need to pay more attention to the replication of published scientific results."

經由實驗所發現的每一項新事實，代表未知當中的一個立足點。在知識的荒野中，很難從詭計（fraud）中區分錯誤；從欺騙（deception）中區別馬虎；貪婪中辨別熱心 （eagerness），或著愈來愈多地，來自於黨派激情的科學信念（scientific conviction from partisan passion）。

Every new fact discovered through experiment represents a foothold in the unknown. In a wilderness of knowledge, it can be difficult to distinguish error from fraud, sloppiness from deception, eagerness from greed or, increasingly, scientific conviction from partisan passion.

從幹細胞研究到全球暖化的科學發現都變成政治政策戰爭的糧秣時，即便枝微末節的錯誤與校正都會產生巨大的後果。

As scientific findings become fodder for political policy wars over matters from stem-cell research to global warming, even trivial errors and corrections can have larger consequences.

然而，其他研究者告誡，不要害怕所有的錯誤。錯誤一如發現，都是科學的一部份。這是在尋找真理時無法避免的副產品，那必須經由不斷摸索（trial and error）進行下去。"

Still, other researchers warn not to fear all mistakes. Error is as much a part of science as discovery. It is the inevitable byproduct of a search for truth that must proceed by trial and error. 

"你在那裡有新知識領域正在發展，接著科學將會有爭論，會因資料不夠充分而產生的錯誤，或著沒有認出新的事物，" 耶魯大學科學歷史學家 Daniel Kevles 說。相互矛盾的資料與詮釋上的差異都十分常見。

"Where you have new areas of knowledge developing, then the science is going to be disputed, subject to errors arising from inadequate data or the failure to recognize new matters," said Yale University science historian Daniel Kevles. Conflicting data and differences of interpretation are common.

為了要徹底根除錯誤，科學家依賴其他人的警惕。即便如此，這些發現仍然很少被它人檢查，或獨立複製。撤回（retractions），雖然更加常見，但仍較不頻繁。那些被駁斥的發現仍會徘徊於科學文獻中多年，被不知情的其他研究者引用，混合這些錯誤。

To root out mistakes, scientists rely on each other to be vigilant. Even so, findings too rarely are checked by others or independently replicated. Retractions, while more common, are still relatively infrequent. Findings that have been refuted can linger in the scientific literature for years to be cited unwittingly by other researchers, compounding the errors.

受到物理學、生物學與醫學當中的騙局刺激，研究期刊最近採用更嚴厲的預防措施進行保護，至少能對付精心捏造的資料。不過要承認（錯誤）則更加困難，即便那是 正直的錯誤。上個月，中國政府提出一條新法律，讓它的科學家能夠承認疏忽而不用受罰。下週，世界第一場「研究正直風範（research integrity）」的研討會將在（葡萄牙首都）里斯本聚會（譯註：詳見相關報導）。

Stung by frauds in physics, biology and medicine, research journals recently adopted more stringent safeguards to protect at least against deliberate fabrication of data. But it is hard to admit even honest error. Last month, the Chinese government proposed a new law to allow its scientists to admit failures without penalty. Next week, the first world conference on research integrity convenes in Lisbon.

大體上，科技複審者都因工作繁重很難偵測到每項異常之處。平均來說，每年研究者們就遞交約 12000 篇論文到每週經同儕審核的 Science 期刊。去年，Science 就有四篇論文被撤回。還有一打以上的論文經過校正。

Overall, technical reviewers are hard-pressed to detect every anomaly. On average, researchers submit about 12,000 papers annually just to the weekly peer-reviewed journal Science. Last year, four papers in Science were retracted. A dozen others were corrected.

沒有人真的知道，還有多少不正確的研究報告依然未接受挑戰。

No one actually knows how many incorrect research reports remain unchallenged.

在今年初，資訊專家 Murat Cokol 及其哥倫比亞大學的同事，將 U.S. National Library of Medicine 從 1950 - 2004 在 4000 份期刊上所發表的 940 萬篇報告進行分類。經過嚴密計算（raw count），只有 596 篇正式被撤回，Cokol 博士報告。

Earlier this year, informatics expert Murat Cokol and his colleagues at Columbia University sorted through 9.4 million research papers at the U.S. National Library of Medicine published from 1950 through 2004 in 4,000 journals. By raw count, just 596 had been formally retracted, Dr. Cokol reported.

"經過校正亦不是最終真理，" Kevles 教授說。

"The correction isn't the ultimate truth either," Prof. Kevles said.

Email me at ScienceJournal@wsj.com.