Repeated cognitive screening of functionally independent older adults may help identify decline before performance crosses conventional thresholds for cognitive impairment, but interpreting longitudinal change requires distinguishing true change from measurement error. The Reliable Change Index (RCI) is commonly used for this purpose. Conventional RCI methods based on classical test theory (CTT), however, typically assume that measurement error is constant across examinees, an assumption that may be particularly problematic for screening instruments such as the Mini-Mental State Examination (MMSE), which often show pronounced ceiling effects. We compared conventional CTT- and difference-score-based item response theory (IRT) RCIs in 1136 cognitively unimpaired older adults with elevated brain amyloid from the A4 clinical trial. The two approaches were strongly correlated overall (r = .83). However, among the 92 participants identified as showing reliable decline by at least one method, 44 (47.8%) received discordant classifications. Thus, substantial overall correspondence between CTT- and IRT-based RCIs can co-occur with meaningful disagreement about which individuals have experienced reliable cognitive decline. These findings highlight the importance of accounting for variation in measurement precision when interpreting longitudinal change on cognitive screening measures in older adults. They also provide an important cautionary note for clinicians who rely on repeated cognitive screening, as detection of reliable decline may be particularly challenging among individuals with relatively preserved cognition who may be in the earliest stages of cognitive deterioration.