Rational use of in vitro P-glycoprotein assays in drug discovery.

P-glycoprotein (Pgp) affects the absorption, distribution, and clearance of a variety of compounds. Thus, identification of compounds that are Pgp substrates can aid drug candidate selection and optimization. Our goal was to evaluate three assays used to determine whether compounds are Pgp substrates. Sixty-six compounds were tested in monolayer efflux, ATPase, and calcein-AM assays. Assay results yielded two categories of compounds. Category I (n = 35) exhibited concordance across the assays. Category II (n = 31) revealed differences among the assays that related to the apparent permeability (P(app)) of the compounds. Within category II, two groups were discerned based on the absence (group IIA, n = 10, nontransported substrates) or presence (group IIB, n = 21, transported substrates) of monolayer efflux. Detection of efflux (group IIB) was associated with compounds having low/moderate P(app) values (mean = 16.6 nm/s), whereas inability to detect efflux (group IIA) was associated with compounds having high P(app) values (mean = 535 nm/s). The calcein-AM and ATPase assays revealed Pgp interactions for highly permeable group IIA compounds but were less responsive than monolayer efflux for low/moderate P(app) compounds of group IIB. All assays detected substrates across a broad range of P(app), but the efflux assay was more prone to fail at high P(app), whereas the calcein-AM and ATPase assays were more prone to fail at low P(app). When P(app) is low, efflux is a greater factor in the disposition of Pgp substrates. The efflux assay is more reliable at low/moderate P(app) and is the method of choice for evaluating drug candidates despite low throughput and reliance on liquid chromatography with tandem mass spectrometry.