Maverickp

Paper - One RING to Rule Them All:
      Radon Sinogram for Place Recognition, Orientation and Translation Estimation

2022.03

• [The 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2022)]
  LiDAR-based global localization is a fundamental problem for mobile robots. It consists of two stages, place recognition and pose estimation, and yields the current orientation and translation, using only the current scan as query and a database of map scans. Inspired by the definition of a recognized place, we consider that a good global localization solution should keep the pose estimation accuracy with a lower place density. Following this idea, we propose a novel framework towards sparse place-based global localization, which utilizes a unified and learning-free representation, Radon sinogram (RING), for all sub-tasks. Based on the theoretical derivation, a translation invariant descriptor and an orientation invariant metric are proposed for place recognition, achieving certifiable robustness against arbitrary orientation and large translation between query and map scan. In addition, we also utilize the property of RING to propose a global convergent solver for both orientation and translation estimation, arriving at global localization. Evaluation of the proposed RING based framework validates the feasibility and demonstrates a superior performance even under a lower place density.
• CV 3D Paper Code

Paper - Translation Invariant Global Estimation of Heading Angle
      Using Sinogram of LiDAR Point Cloud

2021.10

• [The 2022 IEEE International Conference on Robotics and Automation (ICRA 2022)]
  Global point cloud registration is an essential module for localization, of which the main difficulty exists in estimating the rotation globally without initial value. With the aid of gravity alignment, the degree of freedom in point cloud registration could be reduced to 4DoF, in which only the heading angle is required for rotation estimation. In this paper, we propose a fast and accurate global heading angle estimation method for gravity-aligned point clouds. Our key idea is that we generate a translation invariant representation based on Radon Transform, allowing us to solve the decoupled heading angle globally with circular cross-correlation. Besides, for heading angle estimation between point clouds with different distributions, we implement this heading angle estimator as a differentiable module to train a feature extraction network end- to-end. The experimental results validate the effectiveness of the proposed method in heading angle estimation and show better performance compared with other methods.
• CV 3D Paper

Paper - Radar-to-Lidar: Heterogeneous Place Recognition
      via Joint Learning

2021.05

• [2021 Frontiers in Robotics and AI]
  Place recognition is critical for both offline mapping and online localization. However, current single-sensor based place recognition still remains challenging in adverse conditions. In this paper, a heterogeneous measurement based framework is proposed for long-term place recognition, which retrieves the query radar scans from the existing lidar (Light Detection and Ranging) maps. To achieve this, a deep neural network is built with joint training in the learning stage, and then in the testing stage, shared embeddings of radar and lidar are extracted for heterogeneous place recognition. To validate the effectiveness of the proposed method, we conducted tests and generalization experiments on the multi-session public datasets and compared them to other competitive methods. The experimental results indicate that our model is able to perform multiple place recognitions: lidar-to-lidar (L2L), radar-to-radar (R2R), and radar-to-lidar (R2L), while the learned model is trained only once.
• DL 3D Paper Code

Paper - CORAL: Colored structural representation
      for bi-modal place recognition

2021.02

• [The 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2021)]
  Place recognition is indispensable for a drift-free localization system. Due to the variations of the environment,place recognition using single-modality has limitations. In this paper, we propose a bi-modal place recognition method, which can extract a compound global descriptor from the two modalities, vision and LiDAR. Specifically, we first build the elevation image generated from 3D points as a structural representation. Then, we derive the correspondences between 3D points and image pixels that are further used in merging the pixel-wise visual features into the elevation map grids. In this way, we fuse the structural features and visual features in the consistent bird-eye view frame, yielding a semantic representation, namely CORAL. And the whole network is called CORAL-VLAD. Comparisons on the Oxford RobotCar show that CORALVLAD has superior performance against other state-of-theart methods. We also demonstrate that our network can be generalized to other scenes and sensor configurations on crosscity datasets.
• DL 3D CV Paper

Paper - DiSCO: Differentiable Scan Context with Orientation

2020.10

• [2021 IEEE Robotics and Automation Letters (RAL)]
  Global localization is essential for robot navigation, of which the first step is to retrieve a query from the map database. This problem is called place recognition. In recent years, LiDAR scan based place recognition has drawn attention as it is robust against the environmental change. In this paper, we propose a LiDAR-based place recognition method, named Differentiable Scan Context with Orientation (DiSCO), which simultaneously finds the scan at a similar place and estimates their relative orientation. The orientation can further be used as the initial value for the down-stream local optimal metric pose estimation, improving the pose estimation especially when a large orientation between the current scan and retrieved scan exists.
• DL 3D Paper Code

Paper - Deep Phase Correlation for End-to-End Heterogeneous Sensor
      Measurements Matching

2020.06

• [2020 The Conference on Robot Learning (CoRL)]
  The crucial step for localization is to match the current observation to the map. When the two sensor modalities are significantly different, matching becomes challenging. In this paper, we present an end-to-end deep phase correlation network (DPCN) to match heterogeneous sensor measurements. In DPCN, the primary component is a differentiable correlation-based estimator that back-propagates the pose error to learnable feature extractors, which addresses the problem that there are no direct common features for supervision. Also, it eliminates the exhaustive evaluation in some previous methods, improving efficiency. With the interpretable modeling, the network is light-weighted and promising for better generalization.
• DL CV Paper Code

Paper - Collaborative Localization of Aerial and Ground Mobile Robots
      through Orthomosaic Map

2020.04

• [2020 International Conference on Real-time Computing and Robotics (RCAR)]
  With the deepening of research on the SLAM system, the possibility of cooperative SLAM with multi-robots has been proposed. This paper presents a map matching and localization approach considering the cooperative SLAM of an aerial-ground system. The proposed approach aims to help precisely matching the map constructed by two independent systems that have large scale variance of viewpoints of the same route and eventually enables the ground mobile robot to localize itself in the global map given by the drone.
• CV Paper

Paper - GEM: Online Globally Consistent Dense Elevation Mapping for
      Unstructured Terrain

2020.02

• [2020 IEEE Transactions on Instrumentation and Measurement (TIM)]
  Online dense mapping gives a representation of the unstructured terrain, which is indispensable for safe robotic motion planning. In this article, we propose such an elevation mapping system, namely GEM, to generate a dense local elevation map in constant real time for fast responsive local planning, and maintain a globally consistent dense map for path routing at the same time. We model the global elevation map as a collection of submaps. When the trajectory estimation of the robot is corrected by simultaneous localization and mapping (SLAM), only relative poses between submaps are updated without rebuilding the submap. As a result, this deformable global dense map representation is able to keep the global consistency online. Besides, we accelerate the local mapping by integrating traversability analysis into the mapping system to save the computation cost by obstacle awareness. The system is implemented by CPU-GPU coordinated processing to guarantee constant real-time performance for in-time handling of dynamic obstacles. Substantial experimental results on both simulated and real-world data set validate the efficiency and effectiveness of GEM.
• SLAM Paper Code

Paper - Real-time instance-aware semantic mapping

2019.11

• [Journal of Physics: Conference Series (2019 ICDT Conference)]
  The semantic information helps robots to understand its surroundings like human beings and enables robots to achieve human-robot interaction. In recent years, there have been many interests in semantic mapping. Numerous approaches manage to build a semantic map and achieve good accuracy, but the existing mapping methods which create the metric semantic map ignore the subsequent applications of the semantic map. However, the metric map with the simple semantic class label has no direct benefit to localization. In this paper, we propose an approach to construct an object-centric map with promising applications.
• SLAM

Paper - GPU accelerated real-time traversability mapping

2019.09

• [2019 IEEE International Conference on Robotics and Biomimetics (ROBIO)]
  Dense map representation of the robot surroundings, which contains detailed information of the drivable region can be easily used for motion planning. To build a dense map on mobile robots, the main challenge is that the system has to be efficient due to the limited computational resources. In this paper, we propose a novel approach to generate a dense map with drivable information. First, the dense map with elevation information is generated by the proprioceptive localization results acquired from kinematic and inertial measurement, as well as the accumulated raw data from the range sensor.
• CUDA SLAM Paper Code

Graduate Thesis - Semantic Mapping

2019.03

• In order to improve the perception ability of mobile robots, there is a fashion to import object infomation in the process of SLAM, a most common one is semantic mapping. In my Graduate thesis, I developed a mapping system based on the RNN and semantic segmentation.
• CV Thesis

Intern @Nanjiang Robotics

2018.08

• As a Intern Algorithm Engineer, I developed a feature-based 3d mapping method using sensor - Quanergy M8. The feature-based map is used to describe the depot environment which mainly consists of walls and girders. Test code can be found on github.
• PCL Github

Course - Flight Vehicle Navigation and Control Technology

2018.07

• In this course, I had the chance to get more basic knowledge of the plane and the quadrotor. I was attracted by the knowledge of the quadrotor for I just had finished some application projects on this platform. In order to have a deeper understanding, I modeled a quadrotor with motion equations and force constraints and built a rather practical model in Matlab Simulink. I even managed to control the quadrotor using PID and cascade PID.
• PID

ICRA 2018 DJI RoboMaster AI Challenge

2018.05

• I attended ICRA 2018 DJI RoboMaster AI Challenge as a member of ZMART Team, ZJU. Our team got 12/21 in the final competition.
  In this Challenge, the task was to defeat the official AI cars by building our own cars. In this situation, we built a robot based on ROS(Robot Operation System) with modules like decision, navigation, localization and so on. I was in charge of the detecting and technical coordination.
• View

UAV Precise Landing using infrared beacon

2018.05

• This project is based on the previous work on putting out a fire using UAV.
  I developed a robust two-step landing strategy to assure the landing precision. First, the UAV detects the AR Markers around the target and roughly goes to the top of it. Then, the infrared beacon continuously gives the robust signal to the UAV, so it can adjust its pose during the landing.
• CV

Course - Advanced Experiment of Microprocessors & Interfacing

2018.01

• This project demonstrates how a quadrotor can put out a fire. I used Raspberry Pi (model 3b) as a companion computer of the PIXHAWK (with apm firmware). There was a camera on the Pi to get the downward vision. With the pictures we got, we calculated the best throw-point and throw our "water ball" to the point.
• Github

Course - Robots Technology

2018.01

• This project was developed on the V-REP simulation environment. The task was to build a robot to grab a block and put it on the manipulator platform.
  I loaded a model of a ready-made manipulator and modified it to a mobile manipulator. A camera and a distance sensor was installed on the end-effector to achieve visual servo.