Accelerating DETR Convergence via Semantic-Aligned Matching
Gongjie Zhang, Zhipeng Luo, Yingchen Yu, Kaiwen Cui, Shijian Lu
The recently developed DEtection TRansformer (DETR) establishes a new object
detection paradigm by eliminating a series of hand-crafted components. However,
DETR suffers from extremely slow convergence, which increases the training cost
significantly. We observe that the slow convergence is largely attributed to
the complication in matching object queries with target features in different
feature embedding spaces. This paper presents SAM-DETR, a
Semantic-Aligned-Matching DETR that greatly accelerates DETR's convergence
without sacrificing its accuracy. SAM-DETR addresses the convergence issue from
two perspectives. First, it projects object queries into the same embedding
space as encoded image features, where the matching can be accomplished
efficiently with aligned semantics. Second, it explicitly searches salient
points with the most discriminative features for semantic-aligned matching,
which further speeds up the convergence and boosts detection accuracy as well.
Being like a plug and play, SAM-DETR complements existing convergence solutions
well yet only introduces slight computational overhead. Extensive experiments
show that the proposed SAM-DETR achieves superior convergence as well as
competitive detection accuracy. The implementation codes are available at
this https URL