Efficient Backbone Exploration and Fine-Grained Alignment for Text-Based Person Search

Abstract

Text-Based Person Search (TBPS) requires discriminative cross-modal embeddings that can capture fine-grained visual attributes such as clothing patterns, accessories, and footwear and remain robust to changes in camera viewpoint, changes in resolution, and occlusion. Standard CLIP-style vision language models offer good global semantic alignment but have limited patch-level grounding, and are computationally expensive to run in real-time surveillance systems. This thesis explores two complementary directions: backbone exploration, comparing transformer based ViT encoders with state-space model (SSM) backbones, specifically MambaVisionfine grained alignment, training a patch-level CLIP pipeline based on cropped regions of images aligned with textual phrases, and the Cross-Modal Adaptive Dual Association (CADA) module integrated with a MambaVision backbone. On three canonical TBPS benchmarks (CUHK-PEDES, ICFG-PEDES and RSTPReid), experiments are performed. Results indicate that: (a) the MambaVision SSM backbone consistently outperforms ViT-B/32 baselines on Recall@1 by approximately 10-15 percentage points on CUHK-PEDES, ICFG-PEDES, and RSTPReid respectively. The results provide a feasible design solutions to CLIP based pedestrian retrieval systems.