Drifting Error Comparision of ARCore and ARKit Visual Odometry

ARCore from Google and ARKit from Apple are tools to build AR applications on Android and IOS, respectively. Both tools have some same features in common, such as visual odometry, SLAM, 3D rendering, and augmented image. More information about the tools can be found:

In this post, a simple evaluation of the drifting error on both platforms are presented. The test is carried out in a corridor, respectively on ASUS Zenfone AR and iPhone X. The test starts by moving forward for 19.1 meters and then make three and a half turn around its own spot. This process is repeated four times for each test which means the second half of the test follows exactly same route as the first half. Besides, three tests are done on each platform to increase the accuracy of the estimate.

Image for post
Image for post
Corridor used for testing

Data Acquisition

The position data during the test is logged out while taking the starting point as reference (0,0,0). The raw data is described in the following plots, the subplots from top to bottom are respectively x position, y position, z position, and angle. The distance is in meters and the angle is in radians:

Image for post
Image for post
ARCore first test
Image for post
Image for post
ARCore second test
Image for post
Image for post
ARCore third test
Image for post
Image for post
ARKit first test
Image for post
Image for post
ARKit second test
Image for post
Image for post
ARKit third test

Result Analysis

Due to a lack of benchmarks, only the start and the end of the corridor are evaluated. It shows that, after moving 19.1 meters, ARCore has an error of around 0.5 meters and ARKit has an error of around 1.5 meters. In terms of turning, ARCore has an error of around 0.06 rad (3.44 degrees) and ARKit has an error of around 0.15 rad (8.59 degree) after turning three and a half turns.

There are also other aspects that need to be noticed. Firstly, it can be seen from the plots that there are more jumps in ARCore than ARKit. It means ARCore relies on SLAM more often than ARKit to produce a better result whereas ARKit has the potential to provide a more continuous user experience. Secondly, the SLAM might be confusing if the mapping is wrong. Last but not least, ARCore stops working if there is no feature in the view whereas it does not affect ARKit too much.

Funding

The project was executed by Alexandra Institute and was partially funded by InfinIT (the innovation network for IT in Denmark) and by a performance contract from the Danish Ministry of Higher Education and Science.

Reference

Alapetite, Alexandre, Zhongyu Wang, John Paulin Hansen, Marcin Zajączkowski, and Mikołaj Patalan. “Comparison of Three Off-the-Shelf Visual Odometry Systems.” Robotics 9, no. 3 (2020): 56.

Written by

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store