Is the statistical assessment of papers submitted to the "British Medical Journal" effective?

A study was performed to judge the effectiveness of the statistical assessment scheme for papers submitted to the "British Medical Journal." Statistical criticism of the content of 200 submitted papers which had already been seen by a subject referee led directly or indirectly to 73 (37%) being rejected for publication. In most cases (53 out of 73) serious problems requiring more than minor revision were identified. A comparison of reports on subsequently unpublished and published papers showed that adverse statistical assessments--suggesting major problems--were more common in the papers that were not accepted for publication. Moderate, or less, revision was recommended for 63% of published papers but 39% of the remainder. A checklist of relevant questions was used in making a detailed comparison of 12 published papers, six of which had been statistically assessed and six of which had not. This comparison yielded little evidence that the papers that had been assessed were statistically more acceptable than those that had not been assessed but re-emphasised the subjectivity of refereeing and assessment.