Drifting Error Comparision of ARCore and ARKit Visual Odometry

ARCore from Google and ARKit from Apple are tools to build AR applications on Android and IOS, respectively. Both tools have some same features in common, such as visual odometry, SLAM, 3D rendering, and augmented image. More information about the tools can be found:

Build new augmented reality experiences that seamlessly blend the digital and physical worlds

ARKit - Augmented Reality - Apple Developer

In this post, a simple evaluation of the drifting error on both platforms are presented. The test is carried out in a corridor, respectively on ASUS Zenfone AR and iPhone X. The test starts by moving forward for 19.1 meters and then make three and a half turn around its own spot. This process is repeated four times for each test which means the second half of the test follows exactly same route as the first half. Besides, three tests are done on each platform to increase the accuracy of the estimate.

Corridor used for testing

Data Acquisition

The position data during the test is logged out while taking the starting point as reference (0,0,0). The raw data is described in the following plots, the subplots from top to bottom are respectively x position, y position, z position, and angle. The distance is in meters and the angle is in radians:

ARCore first test

ARCore second test

ARCore third test

ARKit first test

ARKit second test

ARKit third test

Result Analysis

Due to a lack of benchmarks, only the start and the end of the corridor are evaluated. It shows that, after moving 19.1 meters, ARCore has an error of around 0.5 meters and ARKit has an error of around 1.5 meters. In terms of turning, ARCore has an error of around 0.06 rad (3.44 degrees) and ARKit has an error of around 0.15 rad (8.59 degree) after turning three and a half turns.

There are also other aspects that need to be noticed. Firstly, it can be seen from the plots that there are more jumps in ARCore than ARKit. It means ARCore relies on SLAM more often than ARKit to produce a better result whereas ARKit has the potential to provide a more continuous user experience. Secondly, the SLAM might be confusing if the mapping is wrong. Last but not least, ARCore stops working if there is no feature in the view whereas it does not affect ARKit too much.

Funding

The project was executed by Alexandra Institute and was partially funded by InfinIT (the innovation network for IT in Denmark) and by a performance contract from the Danish Ministry of Higher Education and Science.

Reference

Alapetite, Alexandre, Zhongyu Wang, John Paulin Hansen, Marcin Zajączkowski, and Mikołaj Patalan. “Comparison of Three Off-the-Shelf Visual Odometry Systems.” Robotics 9, no. 3 (2020): 56.