Tamoxifen (TXF), a triphenylethylene antiestrogen, is the major therapeutic agent for breast cancer. In rare cases, TXF treatment appears to increase incidence of endometrial cancer. Also in rats, TXF was found to induce hepatocellular carcinoma. Previous studies suggested that metabolism of TXF may contribute to its antiestrogenic and anticancer activity. The current study demonstrates a novel route of TXF metabolism. TXF is metabolized by rat and human liver microsomes into a reactive intermediate (txf*) which binds irreversibly to microsomal proteins. The binding requires NADPH and O2 and is inhibited by CO, inhibitors of P-450, and antibodies to rat NADPH-P450 reductase, indicating catalysis by P450. Phenobarbital treatment of rats markedly increases binding, suggesting the involvement of induced P450s. Sodium dodecyl sulfate-polyacrylamide gel electrophoresis of proteins from incubation of [14C] TXF with phenobarbital-treated microsomes exhibits a major radiolabeled zone which corresponds to a molecular weight of approximately 54,000, suggesting binding to a P-450. Cysteine and glutathione inhibited the binding of TXF without significantly affecting P-450-mediated metabolism of TXF, possibly by reacting with txf* or by competing for the same binding sites. Exposure of phenobarbital-treated microsomes and control-microsomes to 50 degrees C for 90 s, which inactivates the flavin-containing monooxygenase (FMO), diminished binding and pH 8.6 enhanced binding. Also, alternate FMO substrates inhibited binding. These findings indicate that P-450 and possibly FMO catalyze the reactions leading to the formation of txf*. However, incubations with single-labeled and dual-radiolabeled tamoxifen or with [14C]TXF-N-oxide demonstrated that monodesmethyl-TXF and TXF-N-oxide, the principal P-450 and FMO-mediated metabolites, respectively, are not on the major route of txf* formation, indicating that txf* could not be an aldehyde derived from tamoxifen nitrone. Thus, though the structure of txf* was not characterized, certain possibilities were excluded. Speculations on the structure of txf* and on its possible pharmacological and toxicological activity are presented.