Alzheimer's disease (AD) is a major neurodegenerative condition that affects millions around the world. As one of the main biomarkers in the AD diagnosis procedure, brain amyloid positivity is typically identified by positron emission tomography (PET), which is costly and invasive. Brain structural magnetic resonance imaging (sMRI) may provide a safer and more convenient solution for the AD diagnosis. Recent advances in geometric deep learning have facilitated sMRI analysis and early diagnosis of AD. However, determining AD pathology, such as brain amyloid deposition, in preclinical stage remains challenging, as less significant morphological changes can be observed. As a result, few AD classification models are generalizable to the brain amyloid positivity classification task. Blood-based biomarkers (BBBMs), on the other hand, have recently achieved remarkable success in predicting brain amyloid positivity and identifying individuals with high risk of being brain amyloid positive. However, individuals in medium risk group still require gold standard tests such as Amyloid PET for further evaluation. Inspired by the recent success of transformer architectures, we propose a geometric deep learning model based on transformer that is both scalable and robust to variations in input volumetric mesh size. Our work introduced a novel tokenization scheme for tetrahedral meshes, incorporating anatomical landmarks generated by a pre-trained Gaussian process model. Our model achieved superior classification performance in AD classification task. In addition, we showed that the model was also generalizable to the brain amyloid positivity prediction with individuals in the medium risk class, where BM alone cannot achieve a clear classification. Our work may enrich geometric deep learning research and improve AD diagnosis accuracy without using expensive and invasive PET scans.

Magnetic resonance (MR) images are often acquired as anisotropic volumes in clinical settings. Such volumes have a worse through-plane resolution than in-plane resolution, hampering results in many processing pipelines that expect isotropic resolutions. Super-resolution (SR) is a promising methodology to address this problem, but there is concern whether the estimated high-resolution (HR) image suffers from egregious hallucinations, especially with deep learning methods that produce aesthetically pleasing results. One approach to restrict the impact of hallucinations is to guarantee that the estimated HR image is exactly cycle-consistent with the low-resolution observation. The denoising diffusion null space model (DDNM) achieves this through a range null space decomposition, but the specific design of the forward map is left to the application. In this work, we analyze the forward problem in 2D MR acquisition and construct an appropriate linear map A . We train a denoising diffusion probabilistic model on T1-weighted (T1-w) head MR images from multiple datasets and implement DDNM using A for the SR task. We show that the approach yields exact cycle-consistent solutions that are also realistic. We evaluated the approach in a wide variety of T1-w MR datasets, including withheld subjects from training sites and two sites outside of the training domain. We achieve excellent qualitative and quantitative results according to both distortion and perceptual metrics.

Magnetic resonance (MR) tagging is an imaging technique for noninvasively tracking tissue motion in vivo by creating a visible pattern of magnetization saturation (tags) that deforms with the tissue. Due to longitudinal relaxation and progression to steady-state, the tags and tissue brightnesses change over time, which makes tracking with optical flow methods error-prone. Although Fourier methods can alleviate these problems, they are also sensitive to brightness changes as well as spectral spreading due to motion. To address these problems, we introduce the brightness-invariant tracking estimation (BRITE) technique for tagged MRI. BRITE disentangles the anatomy from the tag pattern in the observed tagged image sequence and simultaneously estimates the Lagrangian motion. The inherent ill-posedness of this problem is addressed by leveraging the expressive power of denoising diffusion probabilistic models to represent the probabilistic distribution of the underlying anatomy and the flexibility of physics-informed neural networks to estimate biologically-plausible motion. A set of tagged MR images of a gel phantom was acquired with various tag periods and imaging flip angles to demonstrate the impact of brightness variations and to validate our method. The results show that BRITE achieves more accurate motion and strain estimates as compared to other state of the art methods, while also being resistant to tag fading.

Contrastive Language-Image Pre-training (CLIP) demonstrates strong potential in medical image analysis but requires substantial data and computational resources. Due to these restrictions, existing CLIP applications in medical imaging focus mainly on modalities like chest X-rays that have abundant image-report data available, leaving many other important modalities under-explored. Here, we propose one of the first adaptations of the full CLIP model to mammography, which presents significant challenges due to labeled data scarcity, high-resolution images with small regions of interest, and class-wise imbalance. We first develop a specialized supervision framework for mammography that leverages its multi-view nature. Furthermore, we design a symmetric local alignment module to better focus on detailed features in high-resolution images. Lastly, we incorporate a parameter-efficient fine-tuning approach for large language models pre-trained with medical knowledge to address data limitations. Our multi-view and multi-scale alignment (MaMA) method outperforms state-of-the-art baselines for three different tasks on two large real-world mammography datasets, EMBED and RSNA-Mammo, with only 52% model size compared with the largest baseline.

This study proposes a novel heterogeneous graph convolutional neural network (HGCNN) to handle complex brain fMRI data at regional and across-region levels. We introduce a generic formulation of spectral filters on heterogeneous graphs by introducing the k-th Hodge-Laplacian (HL) operator. In particular, we propose Laguerre polynomial approximations of HL spectral filters and prove that their spatial localization on graphs is related to the polynomial order. Furthermore, based on the bijection property of boundary operators on simplex graphs, we introduce a generic topological graph pooling (TGPool) method that can be used at any dimensional simplices. This study designs HL-node, HL-edge, and HL-HGCNN neural networks to learn signal representation at a graph node, edge levels, and both, respectively. Our experiments employ fMRI from the Adolescent Brain Cognitive Development (ABCD; n=7693) to predict general intelligence. Our results demonstrate the advantage of the HL-edge network over the HL-node network when functional brain connectivity is considered as features. The HL-HGCNN outperforms the state-of-the-art graph neural networks (GNNs) approaches, such as GAT, BrainGNN, dGCN, BrainNetCNN, and Hypergraph NN. The functional connectivity features learned from the HL-HGCNN are meaningful in interpreting neural circuits related to general intelligence.

Can we use sparse tokens for dense prediction, e.g., segmentation? Although token sparsification has been applied to Vision Transformers (ViT) to accelerate classification, it is still unknown how to perform segmentation from sparse tokens. To this end, we reformulate segmentation as a sparse encoding → token completion → dense decoding (SCD) pipeline. We first empirically show that naïvely applying existing approaches from classification token pruning and masked image modeling (MIM) leads to failure and inefficient training caused by inappropriate sampling algorithms and the low quality of the restored dense features. In this paper, we propose Soft-topK Token Pruning (STP) and Multi-layer Token Assembly (MTA) to address these problems. In sparse encoding, STP predicts token importance scores with a lightweight sub-network and samples the topK tokens. The intractable topK gradients are approximated through a continuous perturbed score distribution. In token completion, MTA restores a full token sequence by assembling both sparse output tokens and pruned multi-layer intermediate ones. The last dense decoding stage is compatible with existing segmentation decoders, e.g., UNETR. Experiments show SCD pipelines equipped with STP and MTA are much faster than baselines without token pruning in both training (up to 120% higher throughput) and inference (up to 60.6% higher throughput) while maintaining segmentation quality. Code is available here: https://github.com/cvlab-stonybrook/TokenSparse-for-MedSeg.

The Event Based Model (EBM) is a probabilistic generative model to explore biomarker changes occurring as a disease progresses. Disease progression is hypothesized to occur through a sequence of biomarker dysregulation "events". The EBM estimates the biomarker dysregulation event sequence. It computes the data likelihood for a given dysregulation sequence, and subsequently evaluates the posterior distribution on the dysregulation sequence. Since the posterior distribution is intractable, Markov Chain Monte-Carlo is employed to generate samples under the posterior distribution. However, the set of possible sequences increases as N! where N is the number of biomarkers (data dimension) and quickly becomes prohibitively large for effective sampling via MCMC. This work proposes the "scaled EBM" (sEBM) to enable event based modeling on large biomarker sets (e.g. high-dimensional data). First, sEBM implicitly selects a subset of biomarkers useful for modeling disease progression and infers the event sequence only for that subset. Second, sEBM clusters biomarkers with similar positions in the event sequence and only orders the "clusters", with each successive cluster corresponding to the next stage in disease progression. These two modifications used to construct the sEBM method provably reduces the possible space of event sequences by multiple orders of magnitude. The novel modifications are supported by theory and experiments on synthetic and real clinical data provides validation for sEBM to work in higher dimensional settings. Results on synthetic data with known ground truth shows that sEBM outperforms previous EBM variants as data dimensions increase. sEBM was successfully implemented with up to 300 biomarkers, which is a 6-fold increase over previous EBM applications. A real-world clinical application of sEBM is performed using 119 neuroimaging markers from publicly available Alzheimer's Disease Neuroimaging Initiative (ADNI) data to stratify subjects into 6 stages of disease progression. Subjects included cognitively normal (CN), mild cognitive impairment (MCI), and Alzheimer's Disease (AD). sEBM stage is differentiated for the 3 groups (χ2p-value<4.6e-32). Increased sEBM stage is a strong predictor of conversion risk to AD (p-value<2.3e-14) for MCI subjects, as verified with a Cox proportional-hazards model adjusted for age, sex, education and APOE4 status. Like EBM, sEBM does not rely on apriori defined diagnostic labels and only uses cross-sectional data.

Deep learning based models for registration predict a transformation directly from moving and fixed image appearances. These models have revolutionized the field of medical image registration, achieving accuracy on-par with classical registration methods at a fraction of the computation time. Unfortunately, most deep learning based registration methods have focused on scalar imaging modalities such as T1/T2 MRI and CT, with less attention given to more complex modalities such as diffusion MRI. In this paper, to the best of our knowledge, we present the first end-to-end geometric deep learning based model for the non-rigid registration of fiber orientation distribution fields (fODF) derived from diffusion MRI (dMRI). Our method can be trained in a fully-unsupervised fashion using only input fODF image pairs, i.e. without ground truth deformation fields. Our model introduces several novel differentiable layers for local Jacobian estimation and reorientation that can be seamlessly integrated into the recently introduced manifold-valued convolutional network in literature. The results of this work are accurate deformable registration algorithms for dMRI data that can execute in the order of seconds, as opposed to dozens of minutes to hours consumed by their classical counterparts.

Convolutional neural networks (CNN) have been broadly studied on images, videos, graphs, and triangular meshes. However, it has seldom been studied on tetrahedral meshes. Given the merits of using volumetric meshes in applications like brain image analysis, we introduce a novel interpretable graph CNN framework for the tetrahedral mesh structure. Inspired by ChebyNet, our model exploits the volumetric Laplace-Beltrami Operator (LBO) to define filters over commonly used graph Laplacian which lacks the Riemannian metric information of 3D manifolds. For pooling adaptation, we introduce new objective functions for localized minimum cuts in the Graclus algorithm based on the LBO. We employ a piece-wise constant approximation scheme that uses the clustering assignment matrix to estimate the LBO on sampled meshes after each pooling. Finally, adapting the Gradient-weighted Class Activation Mapping algorithm for tetrahedral meshes, we use the obtained heatmaps to visualize discovered regions-of-interest as biomarkers. We demonstrate the effectiveness of our model on cortical tetrahedral meshes from patients with Alzheimer's disease, as there is scientific evidence showing the correlation of cortical thickness to neurodegenerative disease progression. Our results show the superiority of our LBO-based convolution layer and adapted pooling over the conventionally used unitary cortical thickness, graph Laplacian, and point cloud representation.

The increasing availability of large-scale neuroimaging initiatives opens exciting opportunities for discovery science of human brain structure and function. Data-driven techniques, such as Orthonormal Projective Non-negative Matrix Factorization (opNMF), are well positioned to explore multivariate relationships in big data towards uncovering brain organization. opNMF enjoys advantageous interpretability and reproducibility compared to commonly used matrix factorization methods like Principal Component Analysis (PCA) and Independent Component Analysis (ICA), which led to its wide adoption in clinical computational neuroscience. However, applying opNMF in large-scale cohort studies is hindered by its limited scalability caused by its accompanying computational complexity. In this work, we address the computational challenges of opNMF using a stochastic optimization approach that learns over mini-batches of the data. Additionally, we diversify the stochastic batches via repulsive point processes, which reduce redundancy in the mini-batches and in turn lead to lower variance in the updates. We validated our framework on gray matter tissue density maps estimated from 1000 subjects part of the Open Access Series of Imaging (OASIS) dataset. We demonstrated that operations over mini-batches of data yield significant reduction in computational cost. Importantly, we showed that our novel optimization does not compromise the accuracy or interpretability of factors when compared to standard opNMF. The proposed model enables new investigations of brain structure using big neuroimaging data that could improve our understanding of brain structure in health and disease.

Surface reconstruction of cortical and subcortical structures is crucial for brain morphological studies. Existing deep learning surface reconstruction methods, such as DeepCSR and Vox2Surf, learn an implicit field function for computing the isosurface, but do not consider mesh topology. In this paper, we propose a novel and efficient deep learning mesh deformation network, called MeshDeform, to reconstruct topologically correct surfaces of subcortical structures using brain MR images. MeshDeform combines features extracted from a U-Net encoder with mesh deformation blocks to predict surfaces of subcortical structures by deforming spherical mesh templates. MeshDeform is able to reconstruct in less than 10 seconds the surfaces of a left-right pair of subcortical structures with subvoxel accuracy. Reconstruction of all 17 subcortical structures takes less than one and a half minutes. By contrast, Vox2Surf takes about 20-30 minutes for all subcortical structures. Visual and quantitative evaluation on the Human Connectome Project (HCP) dataset demonstrate that MeshDeform generates accurate subcortical surfaces in limited time while preserving mesh topology.

Atlases are crucial to imaging statistics as they enable the standardization of inter-subject and inter-population analyses. While existing atlas estimation methods based on fluid/elastic/diffusion registration yield high-quality results for the human brain, these deformation models do not extend to a variety of other challenging areas of neuroscience such as the anatomy of C. elegans worms and fruit flies. To this end, this work presents a general probabilistic deep network-based framework for atlas estimation and registration which can flexibly incorporate various deformation models and levels of keypoint supervision that can be applied to a wide class of model organisms. Of particular relevance, it also develops a deformable piecewise rigid atlas model which is regularized to preserve inter-observation distances between neighbors. These modeling considerations are shown to improve atlas construction and key-point alignment across a diversity of datasets with small sample sizes including neuron positions in C. elegans hermaphrodites, fluorescence microscopy of male C. elegans, and images of fruit fly wings. Code is accessible at https://github.com/amin-nejat/Deformable-Atlas.

Non-mydriatic retinal color fundus photography (CFP) is widely available due to the advantage of not requiring pupillary dilation, however, is prone to poor quality due to operators, systemic imperfections, or patient-related causes. Optimal retinal image quality is mandated for accurate medical diagnoses and automated analyses. Herein, we leveraged the Optimal Transport (OT) theory to propose an unpaired image-to-image translation scheme for mapping low-quality retinal CFPs to high-quality counterparts. Furthermore, to improve the flexibility, robustness, and applicability of our image enhancement pipeline in the clinical practice, we generalized a state-of-the-art model-based image reconstruction method, regularization by denoising, by plugging in priors learned by our OT-guided image-to-image translation network. We named it as regularization by enhancing (RE). We validated the integrated framework, OTRE, on three publicly available retinal image datasets by assessing the quality after enhancement and their performance on various downstream tasks, including diabetic retinopathy grading, vessel segmentation, and diabetic lesion segmentation. The experimental results demonstrated the superiority of our proposed framework over some state-of-the-art unsupervised competitors and a state-of-the-art supervised method.

Longitudinal analysis is a core aspect of many medical applications for understanding the relationship between an anatomical subject's function and its trajectory of shape change over time. Whereas mixed-effects (or hierarchical) modeling is the statistical method of choice for analysis of longitudinal data, we here propose its extension as hierarchical geodesic polynomial model (HGPM) for multilevel analyses of longitudinal shape data. 3D shapes are transformed to a non-Euclidean shape space for regression analysis using geodesics on a high dimensional Riemannian manifold. At the subject-wise level, each individual trajectory of shape change is represented by a univariate geodesic polynomial model on timestamps. At the population level, multivariate polynomial expansion is applied to uni/multivariate geodesic polynomial models for both anchor points and tangent vectors. As such, the trajectory of an individual subject's shape changes over time can be modeled accurately with a reduced number of parameters, and population-level effects from multiple covariates on trajectories can be well captured. The implemented HGPM is validated on synthetic examples of points on a unit 3D sphere. Further tests on clinical 4D right ventricular data show that HGPM is capable of capturing observable effects on shapes attributed to changes in covariates, which are consistent with qualitative clinical evaluations. HGPM demonstrates its effectiveness in modeling shape changes at both subject-wise and population levels, which is promising for future studies of the relationship between shape changes over time and the level of dysfunction severity on anatomical objects associated with disease.

Contrastive learning has shown great promise over annotation scarcity problems in the context of medical image segmentation. Existing approaches typically assume a balanced class distribution for both labeled and unlabeled medical images. However, medical image data in reality is commonly imbalanced (i.e., multi-class label imbalance), which naturally yields blurry contours and usually incorrectly labels rare objects. Moreover, it remains unclear whether all negative samples are equally negative. In this work, we present ACTION, an Anatomical-aware ConTrastive dIstillatiON framework, for semi-supervised medical image segmentation. Specifically, we first develop an iterative contrastive distillation algorithm by softly labeling the negatives rather than binary supervision between positive and negative pairs. We also capture more semantically similar features from the randomly chosen negative set compared to the positives to enforce the diversity of the sampled data. Second, we raise a more important question: Can we really handle imbalanced samples to yield better performance? Hence, the key innovation in ACTION is to learn global semantic relationship across the entire dataset and local anatomical features among the neighbouring pixels with minimal additional memory footprint. During the training, we introduce anatomical contrast by actively sampling a sparse set of hard negative pixels, which can generate smoother segmentation boundaries and more accurate predictions. Extensive experiments across two benchmark datasets and different unlabeled settings show that ACTION significantly outperforms the current state-of-the-art semi-supervised methods.