Augmented reality (AR) Viewer

[greymfm]

Hi,
Wouldn't it be nice to add an augmented reality (AR) viewer to OpenVSLAM? :-)
I'm trying to add an augmented reality (AR) viewer to OpenVSLAM based on an AR viewer example found in ORB_SLAM2.

The general approach is:

1. Create OpenGL window and viewport (using Pangolin library)
2. Create Image Texture for viewport, assign camera frame image to it
3. Get camera matrix (fx, fy, cx, cy) and set OpenGL projection with it.
4. Get camera pose (rotation, translation) and set OpenGL modelview with it.
5. Draw virtual things (e.g. 3D cube at world position 0,0,0)

To make the code reproducable (and not getting into bugs due to wrong hardware/calibration setup etc.), I'm using EuroC dataset ('MH_01_easy'), stereo config ('EuRoC_stereo.yaml') and example ('run_euroc_slam.cc'). To run the AR viewer, copy 'viewer_ar.cc/.h' into folder 'pangolin_viewer' and replace 'viewer' class by 'viewerAR' class in 'run_euroc_slam.cc'.

Now my problem: All points above seem to work except that the 3D cube is not at a fixed position - it moves with the camera.
I cannot see any error in the example code. Maybe you are interested in trying this code :-)

Any hint, any idea or just looking at my example code would be appreciated ;-)
Regards,
Alexander

viewer_ar.zip

[greymfm]

Side-note: I got the ORB-SLAM2 AR viewer example working using ORB2-SLAM (I modified the code, so it can run without ROS) - And using ORB2-SLAM as visual SLAM engine, the cubes are fixed at their locations...

viewer_ar_orbslam.zip