ORB-SLAM2

环境 #

1. Ubuntu 20.04，用22编译完运行也会core dump

整体架构 #

1. 三大板块 Tracking, Local Mapping, Loop Closing
2. 线程间通信，Local Mapping和Loop Closing 轮询检查是否有新的关键帧
3. Tracking检测到新的关键帧后，把关键帧放到Local Mapping的队列中。

知识点 #

描述子 #

1. 特征点有像素坐标和灰度信息，不足以全面描述一个特征点
2. 比较特征点周围的点对并统计它们的灰度差异，得到一个包含更多信息的描述子
3. 有助于在进行特征匹配时，评价两个特征点之间的“距离”。
4. ORB算法对每个特征点计算一个方向，并根据这个方向来旋转BRIEF的采样模式（Oriented BRIEF Descriptor）

Tracking #

提取特征点，计算描述子 #

1. 特征点尽量每个图像区域都有，有利于追踪
2. 逐层按块儿提取特征点
3. 非极大值抑制（区域分裂，取区域内最好的）
4. 计算特征点灰度质心，与几何中心连线，得到主方向
5. 计算旋转点对，比较点对计算得到描述子

跟踪（单目初始化） #

1. 没参考帧，设置参考帧
2. 有参考帧，判断是否有足够的特征点，没有丢弃，有则计算匹配点
3. 匹配点数量不够，丢弃，有则计算位姿
4. 使用RANSAC算法区分inliners和outliers，排除outliers
5. 计算单应矩阵和基础矩阵，进行重投影误差卡方检验
6. 去除三角化匹配失败的点

Ransac分为三步 #

1. 小样本随机采样，把他们当作inliners，计算模型
2. 用模型计算所有点的误差，小于阈值$\epsilon$的记为inliers，得到inliners
3. 评分，inliners数量越多，分数越高
4. 重复1-3，直到找到最好的模型，或者达到最大迭代次数，此时模型的inliners就是最终的inliners。

局部建图 #

1. 参考帧，当前帧计算词袋信息（描述子聚类的信息，相同词袋内的特征点，才有可能成为一对匹配，是用于加快匹配的信息）

Mapping #

局部地图的点有地图点的描述子，是所有关键帧对应描述子的平均值，P3P的时候就可以用描述子之间的“距离”来匹配

SVD #

1. SVD分解是一种矩阵分解方法，将一个矩阵分解为三个矩阵的乘积

BoW #

1. transform turn descriptor into a word
2. A Keyframe has many features points, and each feature point has a descriptor, and each descriptor has a word
3. A histogram of words(BoW) is used to represent the keyframe

Keyframe #

1. Frame has keypoints, keypoints can be used to calculate descriptors.
2. descriptors are then used to calculate BoW, which show the distribution of words in the frame. The transform method that calculates BoW also provide a invert index, which can be used to find keypoints that are mapped to the same word.
3. With Bow, we can compare two frames, and find the similarity between them. This is useful in loop closing.

Keyframe Database #

1. A database that stores keyframes
2. It stores an inverted index. We can find keyframes that share the same word.

Map Point #

1. Has a position in the world
2. has a member map observation that stores the <keyframe,w index> pair to store which keyframe observe this point as well as the descriptor index for that point.
3. Map point also has a “descriptor” that is the average of all the good descriptors that observe the map point.
4. The “average” is determined by the descriptor that has the least median distance to all others.MapPoint::ComputeDistinctiveDescriptors()

Bundle Adjustment #

xushangnjlh commented on Jan 10, 2018

I think this may be used for Shur Complement for the sparse matrix decomposition, to > accelerating solve Hx = b problem. In visual SLAM, the mappoints always need setMarginalize() to tremendously decrease the dimension of H.

ghost commented on Jun 3, 2018

check the Thruns book “probabilistic robotics” (chapter1). There, it is clearly explained, why and how to “marginalize” some variables out of the optimization problems.