Home LiteratureArticle Details
PMID: 25150838 Published · ppublish English Journal Article Research Support, N.I.H., Extramural Research Support, N.I.H., Intramural Research Support, Non-U.S. Gov't

A comprehensive assessment of RNA-seq accuracy, reproducibility and information content by the Sequencing Quality Control Consortium.

Nature biotechnology ·Vol. 32 ·No. 9 ·2014-09-00 ·Pages 903-14

SEQC/MAQC-III Consortium

Abstract

We present primary results from the Sequencing Quality Control (SEQC) project, coordinated by the US Food and Drug Administration. Examining Illumina HiSeq, Life Technologies SOLiD and Roche 454 platforms at multiple laboratory sites using reference RNA samples with built-in controls, we assess RNA sequencing (RNA-seq) performance for junction discovery and differential expression profiling and compare it to microarray and quantitative PCR (qPCR) data using complementary metrics. At all sequencing depths, we discover unannotated exon-exon junctions, with >80% validated by qPCR. We find that measurements of relative expression are accurate and reproducible across sites and platforms if specific filters are used. In contrast, RNA-seq and microarrays do not provide accurate absolute measurements, and gene-specific biases are observed for all examined platforms, including qPCR. Measurement performance depends on the platform and data analysis pipeline, and variation is large for transcript-level profiling. The complete SEQC data sets, comprising >100 billion reads (10Tb), provide unique resources for evaluating RNA-seq analyses for clinical and regulatory settings.

MeSH Terms
Polymerase Chain Reaction Reproducibility of Results Sequence Analysis, RNA/standards
Authors & Affiliations
1 authors, click to expand affiliations / ORCID
SEQC/MAQC-III Consortium
Investigators
162 investigators, click to expand
Su Zhenqiang
Łabaj Paweł P
Li Sheng
Thierry-Mieg Jean
Thierry-Mieg Danielle
Shi Wei
Wang Charles
Schroth Gary P
Setterquist Robert A
Thompson John F
Jones Wendell D
Xiao Wenzhong
Xu Weihong
Jensen Roderick V
Kelly Reagan
Xu Joshua
Conesa Ana
Furlanello Cesare
Gao Hanlin
Hong Huixiao
Jafari Nadereh
Letovsky Stan
Liao Yang
Lu Fei
Oakeley Edward J
Peng Zhiyu
Praul Craig A
Santoyo-Lopez Javier
Scherer Andreas
Shi Tieliu
Smyth Gordon K
Staedtler Frank
Sykacek Peter
Tan Xin-Xing
Thompson E Aubrey
Vandesompele Jo
Wang May D
Wang Jian
Wolfinger Russell D
Zavadil Jiri
Auerbach Scott S
Bao Wenjun
Binder Hans
Blomquist Thomas
Brilliant Murray H
Bushel Pierre R
Cai Weimin
Catalano Jennifer G
Chang Ching-Wei
Chen Tao
Chen Geng
Chen Rong
Chierici Marco
Chu Tzu-Ming
Clevert Djork-Arné
Deng Youping
Derti Adnan
Devanarayan Viswanath
Dong Zirui
Dopazo Joaquin
Du Tingting
Fang Hong
Fang Yongxiang
Fasold Mario
Fernandez Anita
Fischer Matthias
Furió-Tari Pedro
Fuscoe James C
Caimet Florian
Gaj Stan
Gandara Jorge
Gao Huan
Ge Weigong
Gondo Yoichi
Gong Binsheng
Gong Meihua
Gong Zhuolin
Green Bridgett
Guo Chao
Guo Lei
Guo Li-Wu
Hadfield James
Hellemans Jan
Hochreiter Sepp
Jia Meiwen
Jian Min
Johnson Charles D
Kay Suzanne
Kleinjans Jos
Lababidi Samir
Levy Shawn
Li Quan-Zhen
Li Li
Li Li
Li Peng
Li Yan
Li Haiqing
Li Jianying
Li Shiyong
Lin Simon M
López Francisco J
Lu Xin
Luo Heng
Ma Xiwen
Meehan Joseph
Megherbi Dalila B
Mei Nan
Mu Bing
Ning Baitang
Pandey Akhilesh
Pérez-Florido Javier
Perkins Roger G
Peters Ryan
Phan John H
Pirooznia Mehdi
Qian Feng
Qing Tao
Rainbow Lucille
Rocca-Serra Philippe
Sambourg Laure
Sansone Susanna-Assunta
Schwartz Scott
Shah Ruchir
Shen Jie
Smith Todd M
Stegle Oliver
Stralis-Pavese Nancy
Stupka Elia
Suzuki Yutaka
Szkotnicki Lee T
Tinning Matthew
Tu Bimeng
van Delft Joost
Vela-Boza Alicia
Venturini Elisa
Walker Stephen J
Wan Liqing
Wang Wei
Wang Jinhui
Wang Jun
Wieben Eric D
Willey James C
Wu Po-Yen
Xuan Jiekun
Yang Yong
Ye Zhan
Yin Ye
Yu Ying
Yuan Yate-Ching
Zhang John
Zhang Ke K
Zhang Wenqian
Zhang Wenwei
Zhang Yanyan
Zhao Chen
Zheng Yuanting
Zhou Yiming
Zumbo Paul
Tong Weida
Kreil David P
Mason Christopher E
Shi Leming
References (47)
47 references, click to expand
  1. featureCounts: an efficient general purpose program for assigning sequence reads to genomic features.
    Bioinformatics. 2014 Apr 1;30(7):923-30 PMID: 24227677
  2. Variance stabilization applied to microarray data calibration and to the quantification of differential expression.
    Bioinformatics. 2002;18 Suppl 1:S96-104 PMID: 12169536
  3. Summarizing and correcting the GC content bias in high-throughput sequencing.
    Nucleic Acids Res. 2012 May;40(10):e72 PMID: 22323520
  4. A rat RNA-Seq transcriptomic BodyMap across 11 organs and 4 developmental stages.
    Nat Commun. 2014;5:3230 PMID: 24510058
  5. Reproducibility of high-throughput mRNA and small RNA sequencing across laboratories.
    Nat Biotechnol. 2013 Nov;31(11):1015-22 PMID: 24037425
  6. Alternative isoform regulation in human tissue transcriptomes.
    Nature. 2008 Nov 27;456(7221):470-6 PMID: 18978772
  7. A scaling normalization method for differential expression analysis of RNA-seq data.
    Genome Biol. 2010;11(3):R25 PMID: 20196867
  8. RNA-seq: an assessment of technical reproducibility and comparison with gene expression arrays.
    Genome Res. 2008 Sep;18(9):1509-17 PMID: 18550803
  9. edgeR: a Bioconductor package for differential expression analysis of digital gene expression data.
    Bioinformatics. 2010 Jan 1;26(1):139-40 PMID: 19910308
  10. STAR: ultrafast universal RNA-seq aligner.
    Bioinformatics. 2013 Jan 1;29(1):15-21 PMID: 23104886
  11. A new summarization method for Affymetrix probe level data.
    Bioinformatics. 2006 Apr 15;22(8):943-9 PMID: 16473874
  12. AceView: a comprehensive cDNA-supported gene and transcripts annotation.
    Genome Biol. 2006;7 Suppl 1:S12.1-14 PMID: 16925834
  13. A comparison of RNA-Seq and high-density exon array for detecting differential gene expression between closely related species.
    Nucleic Acids Res. 2011 Jan;39(2):578-88 PMID: 20864445
  14. Differential analysis of gene regulation at transcript resolution with RNA-seq.
    Nat Biotechnol. 2013 Jan;31(1):46-53 PMID: 23222703
  15. A systematic comparison and evaluation of high density exon arrays and RNA-seq technology used to unravel the peripheral blood transcriptome of sickle cell disease.
    BMC Med Genomics. 2012 Jun 29;5:28 PMID: 22747986
  16. The External RNA Controls Consortium: a progress report.
    Nat Methods. 2005 Oct;2(10):731-4 PMID: 16179916
  17. Evolving gene/transcript definitions significantly alter the interpretation of GeneChip data.
    Nucleic Acids Res. 2005 Nov 10;33(20):e175 PMID: 16284200
  18. Noisy splicing drives mRNA isoform diversity in human cells.
    PLoS Genet. 2010 Dec 09;6(12):e1001236 PMID: 21151575
  19. Identifying differentially expressed transcripts from RNA-seq data with biological variation.
    Bioinformatics. 2012 Jul 1;28(13):1721-8 PMID: 22563066
  20. Landscape of transcription in human cells.
    Nature. 2012 Sep 6;489(7414):101-8 PMID: 22955620
  21. The Subread aligner: fast, accurate and scalable read mapping by seed-and-vote.
    Nucleic Acids Res. 2013 May 1;41(10):e108 PMID: 23558742
  22. Characterization and improvement of RNA-Seq precision in quantitative transcript expression profiling.
    Bioinformatics. 2011 Jul 1;27(13):i383-91 PMID: 21685096
  23. NCBI Reference Sequences (RefSeq): current status, new features and genome annotation policy.
    Nucleic Acids Res. 2012 Jan;40(Database issue):D130-5 PMID: 22121212
  24. RNA-seq: technical variability and sampling.
    BMC Genomics. 2011 Jun 06;12:293 PMID: 21645359
  25. International network of cancer genome projects.
    Nature. 2010 Apr 15;464(7291):993-8 PMID: 20393554
  26. Targeted next-generation sequencing of a cancer transcriptome enhances detection of sequence variants and novel fusion transcripts.
    Genome Biol. 2009;10(10):R115 PMID: 19835606
  27. RNA-sequence analysis of human B-cells.
    Genome Res. 2011 Jun;21(6):991-8 PMID: 21536721
  28. voom: Precision weights unlock linear model analysis tools for RNA-seq read counts.
    Genome Biol. 2014 Feb 03;15(2):R29 PMID: 24485249
  29. G-stack modulated probe intensities on expression arrays - sequence corrections and signal calibration.
    BMC Bioinformatics. 2010 Apr 27;11:207 PMID: 20423484
  30. SHRiMP2: sensitive yet practical SHort Read Mapping.
    Bioinformatics. 2011 Apr 1;27(7):1011-2 PMID: 21278192
  31. Twenty-five years of quantitative PCR for gene expression analysis.
    Biotechniques. 2008 Apr;44(5):619-26 PMID: 18474036
  32. GENCODE: the reference human genome annotation for The ENCODE Project.
    Genome Res. 2012 Sep;22(9):1760-74 PMID: 22955987
  33. Mapping and quantifying mammalian transcriptomes by RNA-Seq.
    Nat Methods. 2008 Jul;5(7):621-8 PMID: 18516045
  34. mRNA enrichment protocols determine the quantification characteristics of external RNA spike-in controls in RNA-Seq studies.
    Sci China Life Sci. 2013 Feb;56(2):134-42 PMID: 23393029
  35. Assessing technical performance in differential gene expression experiments with external spike-in RNA control ratio mixtures.
    Nat Commun. 2014 Sep 25;5:5125 PMID: 25254650
  36. Evaluating the impact of sequencing depth on transcriptome profiling in human adipose.
    PLoS One. 2013 Jun 24;8(6):e66883 PMID: 23826166
  37. Human transcriptome array for high-throughput clinical studies.
    Proc Natl Acad Sci U S A. 2011 Mar 1;108(9):3707-12 PMID: 21317363
  38. The impact of quantitative optimization of hybridization conditions on gene expression analysis.
    BMC Bioinformatics. 2011 Mar 14;12:73 PMID: 21401920
  39. Hybridization thermodynamics of NimbleGen microarrays.
    BMC Bioinformatics. 2010 Jan 19;11:35 PMID: 20085625
  40. Comprehensive genomic characterization defines human glioblastoma genes and core pathways.
    Nature. 2008 Oct 23;455(7216):1061-8 PMID: 18772890
  41. TopHat2: accurate alignment of transcriptomes in the presence of insertions, deletions and gene fusions.
    Genome Biol. 2013 Apr 25;14(4):R36 PMID: 23618408
  42. Comprehensive evaluation of differential gene expression analysis methods for RNA-seq data.
    Genome Biol. 2013;14(9):R95 PMID: 24020486
  43. Analyzing and minimizing PCR amplification bias in Illumina sequencing libraries.
    Genome Biol. 2011;12(2):R18 PMID: 21338519
  44. Using RNA sample titrations to assess microarray platform performance and normalization techniques.
    Nat Biotechnol. 2006 Sep;24(9):1123-31 PMID: 16964226
  45. Comparison and calibration of transcriptome data from RNA-Seq and tiling arrays.
    BMC Genomics. 2010 Jun 17;11:383 PMID: 20565764
  46. A comprehensive evaluation of normalization methods for Illumina high-throughput RNA sequencing data analysis.
    Brief Bioinform. 2013 Nov;14(6):671-83 PMID: 22988256
  47. The MicroArray Quality Control (MAQC) project shows inter- and intraplatform reproducibility of gene expression measurements.
    Nat Biotechnol. 2006 Sep;24(9):1151-61 PMID: 16964229
Article Info
Journal
Nature biotechnology
Abbr.
Nat Biotechnol
ISSN
1546-1696
Published
2014-09-00
Epub
2014-00-24
Pages
903-14
Language
English
Region
United States
NLM ID
9604648
PMCID
PMC4321899
Subset
IM
Grants
NIGMS NIH HHS · R24 GM102656 · United States
NHGRI NIH HHS · P01 HG000205 · United States
NHGRI NIH HHS · R44HG005297 · United States
NCI NIH HHS · U54CA119338 · United States
NIEHS NIH HHS · Z01 ES102345-04 · United States
NCI NIH HHS · P30 CA015083 · United States
NCI NIH HHS · U54 CA119338 · United States
NCI NIH HHS · R01CA163256 · United States
NINDS NIH HHS · R01 NS076465 · United States
NCI NIH HHS · R01 CA163256 · United States
NIGMS NIH HHS · R24GM102656 · United States
NHGRI NIH HHS · P01HG00205 · United States
NHGRI NIH HHS · R44 HG005297 · United States
NINDS NIH HHS · R01NS076465 · United States
Biotechnology and Biological Sciences Research Council · BB/I000771/1 · United Kingdom
NHGRI NIH HHS · R01HG006798 · United States
NHGRI NIH HHS · R01 HG006798 · United States
NIGMS NIH HHS · P50 GM021700 · United States
NHLBI NIH HHS · R01 HL109054 · United States
Intramural NIH HHS · United States
Databases
GEO
Analysis Services
Analysis Services

Contact

No. 2 Wenbo Road, Zhangqiu District, Jinan, Shandong

Qilu Normal University · Genelibs Bioinformatics Lab

750 Shunhua Rd, Jinan

2F, Bldg F, University Science Park

Tel: 0531-88819269

WeChat Official Account

Follow our WeChat subscription account for real-time updates and the latest in medical and biological research.


Business Email

E-mail: product@genelibs.com