SurgClip: Accelerated Domain-Adaptive CLIP for Surgical Keyframe-Text Retrieval

Surgical video archives offer rich procedural information but remain difficult to index due to high visual complexity and heterogeneous imaging conditions that confound standard retrieval models. While vision-language models like CLIP provide a scalable foundation, they lack surgical domain knowledge and fail to meet the strict latency and memory constraints of clinical deployment. To overcome these barriers, we present SurgClip, a domain-adaptive keyframe-text retrieval framework for surgical videos. First, we introduce a lightweight Phase Adapter trained on Cholec80 workflow annotations to align CLIP's visual embedding space with surgical semantics, boosting Top-1 accuracy from 0.18 to 0.52, Top-5 from 0.84 to 0.96. Second, to enable deployment efficiency, we implement INT8 Dynamic Quantization on the adapter and Product Quantization (PQ) to large-scale image embeddings. This strategy achieves a 4× model size reduction and 150× embedding compression with minimal degradation in cosine similarity (MAE < 0.005). Finally, we propose a Reliability Analysis Framework to evaluate robustness under synthetic distribution shifts (brightness and blur) and identify phase-specific degradation patterns. SurgClip provides a reproducible, end-to-end solution that bridges the gap between state-of-the-art foundation models and practical clinical application. The code has been publicly released at https://github.com/YueranCao2001/surgclip