Achelous: A fast unified water-surface panoptic perception framework based on fusion of monocular camera and 4d mmwave radar: A Fast Unified Water-Surface Panoptic Perception Framework Based on Fusion of Monocular Camera and 4D mmWave Radar

Research output: Chapter in Book or Report/Conference proceedingConference Proceedingpeer-review

Abstract

Current perception models for different tasks usually exist in modular forms on Unmanned Surface Vehicles (USVs), which infer extremely slowly in parallel on edge devices, causing the asynchrony between perception results and USV position, and leading to error decisions of autonomous navigation. Compared with Unmanned Ground Vehicles (UGVs), the robust perception of USVs develops relatively slowly. Moreover, most current multi-task perception models are huge in parameters, slow in inference and not scalable. Oriented on this, we propose Achelous, a low-cost and fast unified panoptic perception framework for water-surface perception based on the fusion of a monocular camera and 4D mmWave radar. Achelous can simultaneously perform five tasks, detection and segmentation of visual targets, drivable-area segmentation, waterline segmentation and radar point cloud segmentation. Besides, models in Achelous family, with less than around 5 million parameters, achieve about 18 FPS on an NVIDIA Jetson AGX Xavier, 11 FPS faster than HybridNets, and exceed YOLOX-Tiny and Segformer-B0 on our collected dataset about 5 mAP50-95 and 0.7 mIoU, especially under situations of adverse weather, dark environments and camera failure. To our knowledge, Achelous is the first comprehensive panoptic perception framework combining vision-level and point-cloud-level tasks for water-surface perception. To promote the development of the intelligent transportation community, we release our codes in https://github.com/GuanRunwei/Achelous.

Original languageEnglish
Title of host publication2023 IEEE 26th International Conference on Intelligent Transportation Systems, ITSC 2023
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages182-188
Number of pages7
ISBN (Electronic)9798350399462
DOIs
Publication statusPublished - 14 Jul 2023
Event26th IEEE International Conference on Intelligent Transportation Systems, ITSC 2023 - Bilbao, Spain
Duration: 24 Sept 202328 Sept 2023

Publication series

NameIEEE Conference on Intelligent Transportation Systems, Proceedings, ITSC
ISSN (Print)2153-0009
ISSN (Electronic)2153-0017

Conference

Conference26th IEEE International Conference on Intelligent Transportation Systems, ITSC 2023
Country/TerritorySpain
CityBilbao
Period24/09/2328/09/23

Keywords

  • Fusion of vision and radar
  • Unified panoptic perception
  • USV-based water-surface perception

Fingerprint

Dive into the research topics of 'Achelous: A fast unified water-surface panoptic perception framework based on fusion of monocular camera and 4d mmwave radar: A Fast Unified Water-Surface Panoptic Perception Framework Based on Fusion of Monocular Camera and 4D mmWave Radar'. Together they form a unique fingerprint.

Cite this