Sketch Based Image Retrieval for Architecture Images with Siamese Swin Transformer

Yuxin Xu, Yuyao Yan, Yiming Lin, Xi Yang, Kaizhu Huang*

*Corresponding author for this work

Research output: Contribution to journalConference articlepeer-review

1 Citation (Scopus)


Sketch-based image retrieval (SBIR) is an image retrieval task that takes a sketch as input and outputs colour images matching the sketch. Most recent SBIR methods utilise deep learning methods with complicated network designs, which are resource-intensive for practical use. This paper proposes a novel compact framework that takes the siamese network with image view angle information, targeting the SBIR task for architecture images. In particular, the proposed siamese network engages a compact SwinTiny transformer as the backbone encoder. View angle information of the architecture image is fed to the model to further improve search accuracy. To cope with the insufficient sketches issue, simulated building sketches are used in training, which are generated by a pre-trained edge extractor. Experiments show that our model achieves 0.859 top-one accuracy exceeding many baseline models for an architecture retrieval task.

Original languageEnglish
Article number012035
JournalJournal of Physics: Conference Series
Issue number1
Publication statusPublished - 1 Jun 2022
Event2022 6th International Conference on Machine Vision and Information Technology, CMVIT 2022 - Virtual, Online
Duration: 25 Feb 2022 → …


Dive into the research topics of 'Sketch Based Image Retrieval for Architecture Images with Siamese Swin Transformer'. Together they form a unique fingerprint.

Cite this