Multi-colour sketch-based image retrieval with an explicable feature embedding

Sketch-based image retrieval (SBIR) is a cross-domain matching problem that has been gaining continuous attention in the computer vision community. Currently, SBIR techniques mainly focus on dealing with black-and-white sketches and ignore the utilization of multi-colour information of images. This leads to insufficient retrieval performance since they cannot distinguish between images that have the same shape but different colours. To address this problem, a multi-colour sketch-based image retrieval (MCSBIR) method using a two-stage network architecture is proposed. A novel feature embedding for explicably describing the shape and colour information is designed. A triplet loss function is developed to learn the feature embedding, in which a new distance metric is proposed to separate the shape and colour features. In addition, the first multi-colour sketch-image dataset is built to achieve the MCSBIR task and a user interface is designed to visually present the MCSBIR method. The effectiveness of the MCSBIR method is demonstrated by comprehensive experiments.