Rapid identification and quantification of opioid drugs are of significant importance and an urgent need in drug regulation and control, considering the serious social and economic impact of the opioid epidemic in the United States. Unfortunately, techniques for accurate detection of these opioids, particularly for fentanyl, an extremely potent synthetic drug of abuse and a main perpetrator in the opioid crisis, are often not readily accessible. Therefore, a fast, highly sensitive, and preferably quantitative technique, with excellent portability, is highly desirable. Such a technique can potentially offer timely and crucial information for drug control officials, as well as health professionals, about drug distribution and overdose prevention. We therefore propose a portable surface-enhanced Raman scattering (SERS) approach by pairing an easy to perform yet reliable SERS protocol with a compact Raman module suitable for rapid, on-site identification and quantification of trace fentanyl. Fentanyl spiked in urine control was successfully detected at concentrations as low as 5 ng/mL. Portable SERS also enabled detection of trace fentanyl laced in recreational drugs at mass concentrations as low as 0.05% (5 ng in 10 μg total) and 0.1% (10 ng in 10 μg total) in heroin and tetrahydrocannabinol (THC), respectively. Drug interaction with the nanoparticle surface was simulated through molecular dynamics to investigate the molecular adsorption mechanism and account for SERS signal differences observed for opioid drugs. Furthermore, resolution of fentanyl in binary and ternary opioid mixtures was readily achieved with multivariate data analysis. In sum, we developed a rapid, highly sensitive, and reliably quantitative method for trace fentanyl analysis by synergizing a streamlined SERS procedure and a portable Raman module at low cost.