Dual triplet network for image zero-shot learning

As a cross-modal task, zero-shot learning (ZSL) is generally achieved by aligning the semantic relationships between different modalities. It is a key issue in the alignment to accurately measure the multi-modal data distances. Although metric learning has been employed in many image ZSL approaches, few of them make full use of the data information. To address this issue, we propose a novel deep metric learning framework called Dual-Triplet Network (DTNet) for image ZSL. The DTNet first projects the semantic information into the visual space with a mapping network and then employs two triplet networks for learning the visual-semantic mapping. Specifically, one triplet network focuses on negative attribute features, and the other pays special attention to negative visual features, which guarantees the sufficient discovery and utilization of data information. Extensive experiments on three benchmark datasets demonstrate that our proposed DTNet achieves the state-of-the-art results on both traditional and generalized image ZSL tasks. Especially, on the H measurement of generalized image ZSL, DTNet has improvements of 18% on AwA, 1.9% on CUB, and 12.9% on aPY, respectively.