Instability of quartet analyses of molecular sequence data by the maximum likelihood method: the Cetacea/Artiodactyla relationships.

By using the maximum likelihood method, we study the reliability of phylogeny based on only four taxa, which have been studied by Philippe and Douzery with the maximum parsimony and neighbor-joining methods. Amino acid sequences of cytochrome b were provided for the analysis to examine the relationships among Ruminantia, Suiformes, and Cetacea with out-group mammals by using one representative species from each group. In accord with Philippe and Douzery's analysis, it was shown that we can find a quartet of species which provides a high bootstrap proportion for each of the three possible trees and that the quartet analyses with only four species can be misleading even with the maximum likelihood. However, if the confidence level of the inferred tree is estimated adequately by the maximum likelihood method and we take a conservative attitude, we can avoid the danger of concluding an erroneous tree by using quartet analysis, although the problem may remain unresolved in many cases. In order to obtain a reliable tree, it is important to carry out analyses based on as many different genes and as many relevant species as possible and to synthesize the results.