Robotic-assisted surgery (RAS) is established in clinical practice, and automated surgical skill assessment utilizing multimodal data offers transformative potential for surgical analytics and education. However, developing effective multimodal methods remains challenging due to the task complexity, limited annotated datasets and insufficient techniques for cross-modal information fusion. Existing state-of-the-art methods rely exclusively on RGB video and are evaluated only in dry-lab settings, failing to bridge the substantial domain gap to real clinical procedures. In clinical RAS cases such as Robot-assisted Hysterectomy (RAH) and Robot-Assisted Radical Prostatectomy (RARP), background tissue peristalsis and strong specular reflections from instruments, needles, and catheters introduce significant visual challenges. This work introduces RAH-skill and RARP-skill, two clinical datasets for RAS skill assessment, comprising 279,691 and 70,661 RGB frames from 37 and 33 suturing videos, both paired with expert M-GEARS annotations and automatically generated optical flow and segmentation masks. We propose SurgFusion-Net, a novel multimodal fusion network that integrates RGB frames, optical flow, and tool segmentation masks to address background motion noise and instrument specular reflections in RAS skill assessment. The core component, Divergence Regulated Attention (DRA), is a scene-aware fusion mechanism that adaptively regulates cross-modal feature alignment to address the visual challenges of clinical cases, ensuring accurate and robust RAS skill assessment across heterogeneous surgical settings. Validated on the JIGSAWS benchmark, RAH-skill, and RARP-skill datasets, our approach outperforms recent baselines with SCC improvements of 0.02 in LOSO, 0.04 in LOUO across JIGSAWS tasks, and 0.0538 and 0.0493 gains on RAH-skill and RARP-skill, respectively. Our source code and dataset is available at https://github.com/HRL-Mike/SurgFusion-Net.

Surgical full scene segmentation is essential for laparoscopic assistance but remains challenging due to the high visual similarity among anatomical structures, and illumination variations caused by single moving light source. Moreover, accurately segmenting thin, elongated instruments is still difficult, especially when they appear at oblique orientations. Although Mamba-based segmentation methods effectively model long-range spatial relationships through state-space updates, their 2D selective scan strategy is limited in capturing the oblique spatial distributions of surgical instruments. To address these challenges, a holistic vision Mamba block (HVMamba) is proposed with a Holistic Directional Selective Scan module (HSSD) module to integrate anisotropic spatial features from multiple directions while simultaneously modeling cross-channel dependencies to address ambiguous visual features and complex illumination. Specifically, HSSD comprises an Attention-Guided Holistic Directional Selective Scan (AHSD) for efficient integration of horizontal and oblique spatial features under the guidance of cross-coordinate relationships, and a Channel-aware Directional Selective Scan (CASD) to model bidirectional cross-channel dependencies and enhance responses in ambiguous regions. Based on HVMamba, a hierarchical hybrid network, SurgMamba, is further developed by combining an HVMamba branch with a convolutional neural network branch to jointly capture global and local representations. The two types of representations are adaptively fused by Local-Global Feature Coupling Units (LG-FCUs) in the encoder and an Attention-Aware Gating Mechanism (AGM) in the prediction head. Experiments on two public datasets demonstrate that SurgMamba achieves superior performance over state-of-the-art methods, particularly in challenging cases involving obliquely oriented thin instruments, strong specular highlights, and low-contrast tissue boundaries. Code is available at https://github.com/hailinhhh/SurgMamba.

Vision Transformer has achieved significant performance improvements in natural image segmentation tasks owing to its superior global modeling capabilities. However, applying vision Transformers to 3D medical image segmentation is challenging because of the quadratic computational complexity of the self-attention mechanism and their limited generalization on small-scale datasets. To address these limitations, we propose a hybrid CNN-Transformer architecture guided by channel attention, referred to as CACFormer, for 3D medical image segmentation. Specifically, we design a simple and effective channel attention module to guide the fusion of local and global features in each channel. This module adaptively assigns weights to each channel based on its semantic contribution to accurate segmentation. Meanwhile, we introduce a novel linear Transformer variant that integrates a linear attention mechanism with tanh activation. This design encourages the model to focus on the target regions and produce robust segmentation outcomes. The effectiveness and competitive generalization of the proposed framework are validated across five benchmark datasets. On AMOS2022, CACFormer achieves an average Dice score of 89.71%, outperforming 3D UX-Net (89.30%) while reducing inference time from 3.77 s to 2.49 s (a 33.95% reduction). On BraTS2021, CACFormer attains an average Dice score of 90.20%, comparable to TransBTS (90.33%), with 28.54% fewer parameters and 15.22% faster inference time (from 0.46 s to 0.39 s), demonstrating a favorable trade-off between performance and efficiency. Moreover, CACFormer demonstrates competitive cross-dataset generalization, achieving an average Dice score of 86.50% on BraTS2021 when trained on BraTS2019, significantly out-performing TransBTS (47.90%). Index Terms-3D Medical Image.

Computational analysis of breast histopathological images is critical for reliable computer-aided diagnosis and treatment planning. Owing to the ultra-high resolution of whole-slide images (WSIs), most existing WSI segmentation methods rely on patch-wise processing. However, independently processing isolated patches breaks spatial continuity and weakens global tissue context, ultimately limiting segmentation performance. To overcome these limitations, we propose CAT-WSI, a context-aware trajectory learning framework for breast pathology WSI segmentation. Rather than treating patches as unordered samples, CAT-WSI organizes them into structured transverse trajectories across each slide, thereby preserving long-range spatial dependencies while reducing the directional bias and boundary fragmentation inherent in conventional patch-based pipelines. To further enhance global positional awareness, CAT-WSI augments these trajectory representations with a paired downsampled whole-slide thumbnail, enabling explicit global-local contextual modeling over the entire slide. We evaluate CAT-WSI on the CAMELYON16 and Breast-HER2+ datasets across multiple magnification levels. Extensive experiments demonstrate that CAT-WSI achieves consistently strong performance across multiple magnification levels, attaining the best overall results on the evaluated benchmarks in our experimental setting.

Histopathological analysis of stained tissue remains central to biomedical research and clinical care. Virtual staining (VS) offers a promising alternative, with potential to reduce costs and streamline workflows, yet hallucinations pose serious risks to clinical reliability. Here, we formalize the problem of hallucination detection in VS and propose a scalable post-hoc baseline method: Neural Hallucination Precursor (NHP), which explores the generator's latent space to preemptively identify hallucinations. Extensive experiments across diverse VS tasks show NHP is both effective and robust. Critically, we also find that models with fewer hallucinations do not necessarily offer better detectability, exposing a gap in current VS evaluation and underscoring the need for hallucination detection benchmarks.

Automatic tooth segmentation is essential for computer-aided diagnosis and treatment planning in dentistry. Proposal generation plays a pivotal role in accurately delineating instance boundaries and is critical for improving segmentation accuracy. However, existing methods suffer from dispersive offset bias, spatial information loss, and spillover-induced centroid shift, all of which are difficult to alleviate in Euclidean space. This limitation commonly leads to missed detections or excessive merging of instances, particularly in cases of missing or crowded teeth. To overcome these issues, we introduce a novel proposal generation framework based on Topological Morphological Clustering (TMC), which integrates edge logits fields with topological morphology. The key insight is that morphological erosion applied to tooth and edge masks enables the separation of individual tooth instances. To further enhance the continuity of tooth-tooth edges, we design an edge enhancement module that fuses the alpha-shape algorithm with our proposed KNN-based iterative contour approximation (KNN-ICA) method, thereby constructing a supplementary geometric edge logits field. Extensive experiments show that our method achieves superior detection performance over existing proposal generation techniques, and outperforms prior instance segmentation methods on two public benchmarks. Our code is available at https://github.com/jfzhuang314/TMELFNet.

Generalizable dense feature matching in endoscopic images is crucial for robot-assisted tasks, including 3D reconstruction, navigation, and surgical scene understanding. Yet, it remains challenging due to difficult visual conditions (e.g., weak textures, large viewpoint variations) and scarce annotated data. To address these challenges, we propose EndoMatcher, a generalizable endoscopic image matcher via large-scale, multi-domain data pre-training. EndoMatcher employs a two-branch Vision Transformer to extract multi-scale features, enhanced by dual interaction blocks for robust correspondence learning under difficult visual conditions. To overcome data scarcity and improve domain diversity, we construct Endo-Mix6, the first multi-domain dataset for endoscopic matching. Endo- Mix6 consists of approximately 1.2M real and synthetic image pairs across six domains, each corresponding to a distinct endoscopic data source or imaging scenario. Correspondence labels are generated through Structure-from-Motion and simulated transformations. The scale and cross-domain heterogeneity of Endo-Mix6 introduce new training challenges due to significant variations in dataset sizes, distribution shifts, and label noise. To ensure stable optimization, we adopt a progressive multi-objective training strategy that adaptively balances supervision across domains. This enhances feature learning and enables generalization to unseen organs and imaging conditions in a zero-shot fashion. Extensive zero-shot matching experiments demonstrate that EndoMatcher increases the number of inlier matches by 140.69% and 201.43% on the Hamlyn Centre and Bladder Tissue datasets over state-of-the-art methods, respectively, and improves the Matching Direction Prediction Accuracy (MDPA) from 76.0% to 85.4% on the Gastro-Matching dataset, achieving dense and accurate matching under challenging endoscopic conditions. The code is publicly available at https://github.com/Beryl2000/EndoMatcher.

Automatic medical image segmentation, as a prerequisite for clinical quantitative analysis, forms the basis of computer-aided diagnosis. However, blurry object boundaries caused by factors such as imaging quality and inherent physiological properties of tissues or lesions are the main causes of imprecise segmentation. This aligns with the common understanding that high uncertainty and misclassification tend to occur at boundaries in segmentation. To address the challenge, we investigate this phenomenon and explore the connection between uncertainty and tissue boundaries by analysing various tissues. Then an Evidential Uncertainty-Guided Boundary (EUGB) loss is further proposed to demonstrate that uncertainty information can indeed facilitate combating boundary segmentation errors. The proposed EUGB loss not only emphasizes challenging pixels along blurry boundaries using evidential uncertainty, but also introduces a regularization term that constrains uncertainty learning by penalizing incorrect predictions and reinforcing correct ones. The effectiveness of the proposed EUGB loss is verified in the public LIDC-IDRI, ISIC 2018, and OCTA-500 datasets with two classic medical image segmentation networks (U-Net and TransU-Net). Experimental results demonstrate that the proposed loss outperforms seven other segmentation loss functions in terms of boundary segmentation, while maintaining competitive region-level segmentation accuracy. Beyond introducing a new loss function, this paper provides empirical insights for selecting appropriate loss functions across different application scenarios. We systematically analyze the strengths and limitations of existing losses from multiple perspectives, including reliability and dataset characteristics. This analysis offers practical insights that enable researchers and practitioners to optimize segmentation performance based on specific data attributes.

Quantitative imaging is an important feature of spectral X-ray and CT systems, especially photon-counting CT (PCCT) imaging systems, which is achieved through material decomposition (MD) using spectral measurements. In this work, we present a novel framework that makes the PCCT imaging chain end-to-end differentiable (differentiable PCCT), with which we can leverage quantitative information in the image domain to enable cross-domain learning and optimization for upstream models. Specifically, the material decomposition from maximum-likelihood estimation (MLE) was made differentiable based on the Implicit Function Theorem and inserted as a layer into the imaging chain for end-to-end optimization. This framework allows for an automatic and adaptive solution of a wide range of imaging tasks, ultimately achieving quantitative imaging through computation rather than manual intervention. The end-to-end training mechanism effectively avoids the need for direct-domain training or supervision from intermediate references as models are trained using quantitative images. We demonstrate its applicability in two representative tasks: correcting detector energy bin drift and training an object scatter correction network using cross-domain reference from quantitative material images.

Pretrained segmentation models for cardiac magnetic resonance imaging (MRI) often fail to generalize across imaging sequences due to substantial contrast variations. These variations arise from different imaging protocols, yet fundamentally, all contrasts are governed by the same underlying tissue properties, primarily captured by three components: the magnetization strength (M0), T1, and T2. Building on this insight, we introduce Reverse Imaging, a physics-driven framework for data augmentation and domain generalization in cardiac MRI. Our method infers tissue properties from observed MR images with annotation by solving an ill-posed nonlinear inverse problem, regularized by a generative prior. The prior is learned from the multiparametric saturation-recovery single shot acquisition (mSASHA) dataset for joint cardiac T1 and T2 mapping. In inference, we characterize imaging sequences as weak, moderate, or strong observations according to the physical information they provide and the degree of ill-posedness. This motivates an iterative prior-learning strategy that uses moderate T1-mapping observations to alleviate mSASHA data scarcity via pseudo tissue-property estimates. We further integrate MRI physics into posterior inference by expressing the sequence model as a likelihood term guiding the reverse diffusion process. For widely used but weak cine observations, we develop a sequence-specific ControlNet to improve efficiency and spatial consistency. Extensive experiments on eight unseen cardiac MRI sequences with markedly different contrast mechanisms show that Reverse Imaging yields plausible tissue-property estimates, supports synthesis of diverse yet physically consistent contrasts, and improves segmentation robustness under severe cross-sequence shifts.

Accurate T-staging is crucial for guiding personalized treatment strategies for laryngopharyngeal cancer. However, current clinical practice relies on invasive biopsy procedures, whereas CT-based staging remains challenging due to the complex patterns of tumor invasion. Recent computer-aided approaches face two key challenges: 1) Structural relationship modeling: existing methods underrepresent anatomically structured patterns of tumor invasion, as they either process whole CT volumes without tumor-specific anatomical constraints or rely on labor-intensive tumor segmentation. 2) Fine-grained cross-modal alignment: while radiology reports contain organ-specific invasion details, current methods that apply global feature fusion struggle to accurately align individual anatomical structures with their corresponding textual descriptions. To address these issues, we propose an anatomy-aware multimodal framework that integrates organ-level CT context and radiology reports into a unified representation for laryngopharyngeal T-staging. The framework first constructs an Anatomy-Structured Organ Graph (AOG) that captures invasion patterns between primary sites and surrounding organs, then performs Organ-Anchored Cross-Modal Alignment (OCA) so that each organ node aggregates textual evidence from the radiology report, and finally refines this graph representation by injecting organ-specific invasion cues extracted from the report via Report-Enhanced Graph-Refinement (REG), yielding a multimodal organ graph that combines spatial and textual evidence. Extensive experiments demonstrate that the proposed framework achieves superior performance in T-staging of laryngopharyngeal cancer.

The increasing use of vision-language models (VLMs) in healthcare applications presents great challenges related to hallucinations, in which the models may generate seemingly plausible results that are in fact incorrect. Such hallucinations can jeopardize clinical decision making, potentially harming the diagnosis and treatments. In this work, we propose MedHallTune, a large-scale benchmark designed specifically to evaluate and mitigate hallucinations in medical VLMs. Comprising over 100,000 images and 1,000,000 instruction pairs, MedHallTune includes both hallucination and non-hallucination samples. While the dataset is primarily constructed using GPT-based generation and filtering, the evaluation split is manually verified by medical students and physicians to ensure annotation correctness and clinical reliability. We conduct a comprehensive evaluation of current medical and general VLMs using MedHallTune, assessing their performance across key metrics, including clinical accuracy, relevance, detail level, and risk level. The experimental results show that fine-tuning with MedHallTune successfully improves the ability of several existing models to manage hallucinations and boost their zero-shot performance on downstream visual-question-answering tasks, making them more reliable for practical medical applications. Our work contributes to the development of more trustworthy VLMs. The benchmark, evaluation codes and data construction prompts will be made publicly available at MedHallTune.

Accurate MRI-based brain tumor analysis requires not only tumor subtype classification but also localization at an anatomical granularity that is consistent with radiology reports. Most vision-only methods address localization and classification as separate label-prediction tasks, and therefore provide limited alignment with the fine-grained anatomical semantics used in routine reporting. To address this limitation, we propose LIGHT (Learning Image-text Grounding for Hierarchical Tumor analysis), a 3D vision-language framework that formulates brain tumor localization and subtype classification as image-text retrieval in a shared embedding space. Given multimodal MRI inputs, LIGHT retrieves coarse anatomical regions, fine-grained subregions, and tumor subtypes in a unified coarse-to-fine procedure. The framework has three main components: (1) a hierarchical retrieval vocabulary containing 21 coarse-grained regions, 563 fine-grained subregions, and 5 tumor subtypes; (2) large-scale foundation pretraining on a curated patient-disjoint in-house dataset of 99,813 MRI-report pairs; and (3) grounded task fine-tuning that uses segmentation-guided tumor crops and multi-template prompt supervision to improve tumor-prompt alignment. Across 11,034 annotated 3D brain tumor MRI cases from in-house, external clinical, and public datasets, LIGHT achieved 72.1% accuracy for coarse-grained localization, 70.1% top-1 accuracy for fine-grained localization, and 76.0%, 83.7%, and 84.6% accuracy for subtype classification on three 3D datasets, respectively. Additional public 2D subtype-classification experiments provide further evidence of cross-setting subtype recognition. These results suggest that report-grounded image-text retrieval can provide interpretable anatomical and diagnostic outputs for brain tumor MRI analysis. Code: https://github.com/qiuzhaoyu/LIGHT.

Iodine plays a central role in thyroid physiology, and knowledge of its spatial distribution may aid diagnostic framing and follow-up of neoplastic thyroid pathologies. Conventional histology is the gold standard for thyroid assessment but consumes biological material, limiting volumetric context and further analyses, and offers limited specificity for iodine. We present a virtual histology (VH) workflow using a compact microtomography X-ray system with a small-pixel spectral detector, combining phase-contrast and spectral imaging to produce volumetric iodine maps together with morphological images. Four formalin-fixed, paraffin-embedded human thyroid specimens (benign and malignant neoplasms and Hashimoto's thyroiditis) were imaged with this system and benchmarked against a synchrotron reference. The voxel-wise comparison showed strong agreement between laboratory and synchrotron iodine maps (mean correlation r = 0.89, range from 0.75 to 0.96). Average iodine content varied among samples (about 0.25 to 1.15 mg/ml), while discrepancies between laboratory and synchrotron quantification were always below 20%. Comparison with conventional histology indicates that pathologically relevant features, such as distinction between lesion and normal parenchyma, macrofollicles, thyroid capsule, capsular invasion, are visible in spectral VH. This work shows that laboratory spectral VH non-destructively provides volumetric morphology and iodine quantification with fidelity close to synchrotron measurements, and holds promise as an adjunct to histopathology for thyroid lesion characterization.

We propose a Geometric Energy Field (GEF) supervision framework to enhance the robustness and structural consistency of retinal artery/vein (A/V) segmentation. We introduce two geometrically complementary energy fields: the Distance Energy Field (DEF), which encodes soft, vessel-type-specific spatial territories by modeling pixel-wise proximity to arteries and veins, thereby explicitly capturing their spatial coupling, and the Orientation Energy Field (OEF), which models vessel elongation and directional continuity to enforce label consistency along entire vascular trajectories and suppress spurious artery/vein label flipping. These geometry-aware energy fields provide explicit supervisory signals that guide A/V segmentation beyond local appearance cues. Extensive experiments on six retinal datasets demonstrate that the proposed method outperforms state-of-the-art approaches, achieving A/V segmentation accuracies of 97.9%, 97.1%, 98.7%, 98.5%, 97.7% and 99.1%, respectively. The results indicate that the proposed GEF-enhanced architecture produces more coherent, stable, and clinically plausible artery/vein segmentation results.