OpenMPR: Recognize places using multimodal data for people with visual impairments

Ruiqi Cheng, Kaiwei Wang, Jian Bai, Zhijie Xu

Research output: Contribution to journalArticle

Abstract

Place recognition plays a crucial role in navigational assistance, and is also a challenging issue of assistive technology. The place recognition is prone to erroneous localization owing to various changes between database and query images. Aiming at the wearable assistive device for visually impaired people, we propose an open-sourced place recognition algorithm OpenMPR, which utilizes the multimodal data to address the challenging issues of place recognition. Compared with conventional place recognition, the proposed OpenMPR not only leverages multiple effective descriptors, but also assigns different weights to those descriptors in image matching. Incorporating GNSS data into the algorithm, the cone-based sequence searching is used for robust place recognition. The experiments illustrate that the proposed algorithm manages to solve the place recognition issue in the real-world scenarios and surpass the state-of-the-art algorithms in terms of assistive navigation performance. On the real-world testing dataset, the online OpenMPR achieves 88.7% precision at 100% recall without illumination changes, and achieves 57.8% precision at 99.3% recall with illumination changes. The OpenMPR is available at https://github.com/chengricky/OpenMultiPR.
LanguageEnglish
Number of pages13
JournalMeasurement Science and Technology
Early online date10 May 2019
DOIs
Publication statusE-pub ahead of print - 10 May 2019

Fingerprint

Visual Impairment
impairment
Lighting
Image matching
Descriptors
Illumination
Cones
illumination
Navigation
Assistive Technology
Visually Impaired
Image Matching
Recognition Algorithm
navigation
Leverage
Testing
Assign
cones
Cone
Query

Cite this

@article{ee556367f2d244c29ee9aeec8a06a1be,
title = "OpenMPR: Recognize places using multimodal data for people with visual impairments",
abstract = "Place recognition plays a crucial role in navigational assistance, and is also a challenging issue of assistive technology. The place recognition is prone to erroneous localization owing to various changes between database and query images. Aiming at the wearable assistive device for visually impaired people, we propose an open-sourced place recognition algorithm OpenMPR, which utilizes the multimodal data to address the challenging issues of place recognition. Compared with conventional place recognition, the proposed OpenMPR not only leverages multiple effective descriptors, but also assigns different weights to those descriptors in image matching. Incorporating GNSS data into the algorithm, the cone-based sequence searching is used for robust place recognition. The experiments illustrate that the proposed algorithm manages to solve the place recognition issue in the real-world scenarios and surpass the state-of-the-art algorithms in terms of assistive navigation performance. On the real-world testing dataset, the online OpenMPR achieves 88.7{\%} precision at 100{\%} recall without illumination changes, and achieves 57.8{\%} precision at 99.3{\%} recall with illumination changes. The OpenMPR is available at https://github.com/chengricky/OpenMultiPR.",
keywords = "Visual Localization, Computer Vision, Navigational Assistance, Assistive Technology",
author = "Ruiqi Cheng and Kaiwei Wang and Jian Bai and Zhijie Xu",
year = "2019",
month = "5",
day = "10",
doi = "10.1088/1361-6501/ab2106",
language = "English",
journal = "Measurement Science and Technology",
issn = "0957-0233",
publisher = "IOP Publishing",

}

OpenMPR : Recognize places using multimodal data for people with visual impairments. / Cheng, Ruiqi; Wang, Kaiwei; Bai, Jian; Xu, Zhijie.

In: Measurement Science and Technology, 10.05.2019.

Research output: Contribution to journalArticle

TY - JOUR

T1 - OpenMPR

T2 - Measurement Science and Technology

AU - Cheng, Ruiqi

AU - Wang, Kaiwei

AU - Bai, Jian

AU - Xu, Zhijie

PY - 2019/5/10

Y1 - 2019/5/10

N2 - Place recognition plays a crucial role in navigational assistance, and is also a challenging issue of assistive technology. The place recognition is prone to erroneous localization owing to various changes between database and query images. Aiming at the wearable assistive device for visually impaired people, we propose an open-sourced place recognition algorithm OpenMPR, which utilizes the multimodal data to address the challenging issues of place recognition. Compared with conventional place recognition, the proposed OpenMPR not only leverages multiple effective descriptors, but also assigns different weights to those descriptors in image matching. Incorporating GNSS data into the algorithm, the cone-based sequence searching is used for robust place recognition. The experiments illustrate that the proposed algorithm manages to solve the place recognition issue in the real-world scenarios and surpass the state-of-the-art algorithms in terms of assistive navigation performance. On the real-world testing dataset, the online OpenMPR achieves 88.7% precision at 100% recall without illumination changes, and achieves 57.8% precision at 99.3% recall with illumination changes. The OpenMPR is available at https://github.com/chengricky/OpenMultiPR.

AB - Place recognition plays a crucial role in navigational assistance, and is also a challenging issue of assistive technology. The place recognition is prone to erroneous localization owing to various changes between database and query images. Aiming at the wearable assistive device for visually impaired people, we propose an open-sourced place recognition algorithm OpenMPR, which utilizes the multimodal data to address the challenging issues of place recognition. Compared with conventional place recognition, the proposed OpenMPR not only leverages multiple effective descriptors, but also assigns different weights to those descriptors in image matching. Incorporating GNSS data into the algorithm, the cone-based sequence searching is used for robust place recognition. The experiments illustrate that the proposed algorithm manages to solve the place recognition issue in the real-world scenarios and surpass the state-of-the-art algorithms in terms of assistive navigation performance. On the real-world testing dataset, the online OpenMPR achieves 88.7% precision at 100% recall without illumination changes, and achieves 57.8% precision at 99.3% recall with illumination changes. The OpenMPR is available at https://github.com/chengricky/OpenMultiPR.

KW - Visual Localization

KW - Computer Vision

KW - Navigational Assistance

KW - Assistive Technology

U2 - 10.1088/1361-6501/ab2106

DO - 10.1088/1361-6501/ab2106

M3 - Article

JO - Measurement Science and Technology

JF - Measurement Science and Technology

SN - 0957-0233

ER -