OpenMPR

Recognize places using multimodal data for people with visual impairments

Ruiqi Cheng, Kaiwei Wang, Jian Bai, Zhijie Xu

Research output: Contribution to journalArticle

Abstract

Place recognition plays a crucial role in navigational assistance, and is also a challenging issue in assistive technology. Place recognition is prone to erroneous localization owing to various changes between database and query images. Aiming at a wearable assistive device for visually impaired people, we propose an open-sourced place recognition algorithm, OpenMPR, which utilizes multimodal data to address the challenging issues of place recognition. Compared with conventional place recognition, the proposed OpenMPR not only leverages multiple effective descriptors, but also assigns different weights to those descriptors in image matching. Incorporating GNSS data into the algorithm, cone-based sequence searching is used for robust place recognition. The experiments illustrate that the proposed algorithm manages to solve the place recognition issue in real-world scenarios and surpasses state-of-the-art algorithms in terms of assistive navigation performance. On the real-world testing dataset, the online OpenMPR achieves 88.7% precision at 100% recall without illumination changes, and achieves 57.8% precision at 99.3% recall with illumination changes. The OpenMPR is available at https://github.com/chengricky/OpenMultiPR.

Original languageEnglish
Article number124004
Number of pages11
JournalMeasurement Science and Technology
Volume30
Issue number12
Early online date10 May 2019
DOIs
Publication statusPublished - 30 Sep 2019

Fingerprint

Visual Impairment
impairment
Lighting
Image matching
Descriptors
Illumination
Cones
illumination
Navigation
Assistive Technology
Visually Impaired
Image Matching
Recognition Algorithm
navigation
Leverage
Testing
Assign
cones
Cone
Query

Cite this

@article{ee556367f2d244c29ee9aeec8a06a1be,
title = "OpenMPR: Recognize places using multimodal data for people with visual impairments",
abstract = "Place recognition plays a crucial role in navigational assistance, and is also a challenging issue in assistive technology. Place recognition is prone to erroneous localization owing to various changes between database and query images. Aiming at a wearable assistive device for visually impaired people, we propose an open-sourced place recognition algorithm, OpenMPR, which utilizes multimodal data to address the challenging issues of place recognition. Compared with conventional place recognition, the proposed OpenMPR not only leverages multiple effective descriptors, but also assigns different weights to those descriptors in image matching. Incorporating GNSS data into the algorithm, cone-based sequence searching is used for robust place recognition. The experiments illustrate that the proposed algorithm manages to solve the place recognition issue in real-world scenarios and surpasses state-of-the-art algorithms in terms of assistive navigation performance. On the real-world testing dataset, the online OpenMPR achieves 88.7{\%} precision at 100{\%} recall without illumination changes, and achieves 57.8{\%} precision at 99.3{\%} recall with illumination changes. The OpenMPR is available at https://github.com/chengricky/OpenMultiPR.",
keywords = "Visual Localization, Computer Vision, Navigational Assistance, Assistive Technology",
author = "Ruiqi Cheng and Kaiwei Wang and Jian Bai and Zhijie Xu",
year = "2019",
month = "9",
day = "30",
doi = "10.1088/1361-6501/ab2106",
language = "English",
volume = "30",
journal = "Measurement Science and Technology",
issn = "0957-0233",
publisher = "IOP Publishing",
number = "12",

}

OpenMPR : Recognize places using multimodal data for people with visual impairments. / Cheng, Ruiqi; Wang, Kaiwei; Bai, Jian; Xu, Zhijie.

In: Measurement Science and Technology, Vol. 30, No. 12, 124004, 30.09.2019.

Research output: Contribution to journalArticle

TY - JOUR

T1 - OpenMPR

T2 - Recognize places using multimodal data for people with visual impairments

AU - Cheng, Ruiqi

AU - Wang, Kaiwei

AU - Bai, Jian

AU - Xu, Zhijie

PY - 2019/9/30

Y1 - 2019/9/30

N2 - Place recognition plays a crucial role in navigational assistance, and is also a challenging issue in assistive technology. Place recognition is prone to erroneous localization owing to various changes between database and query images. Aiming at a wearable assistive device for visually impaired people, we propose an open-sourced place recognition algorithm, OpenMPR, which utilizes multimodal data to address the challenging issues of place recognition. Compared with conventional place recognition, the proposed OpenMPR not only leverages multiple effective descriptors, but also assigns different weights to those descriptors in image matching. Incorporating GNSS data into the algorithm, cone-based sequence searching is used for robust place recognition. The experiments illustrate that the proposed algorithm manages to solve the place recognition issue in real-world scenarios and surpasses state-of-the-art algorithms in terms of assistive navigation performance. On the real-world testing dataset, the online OpenMPR achieves 88.7% precision at 100% recall without illumination changes, and achieves 57.8% precision at 99.3% recall with illumination changes. The OpenMPR is available at https://github.com/chengricky/OpenMultiPR.

AB - Place recognition plays a crucial role in navigational assistance, and is also a challenging issue in assistive technology. Place recognition is prone to erroneous localization owing to various changes between database and query images. Aiming at a wearable assistive device for visually impaired people, we propose an open-sourced place recognition algorithm, OpenMPR, which utilizes multimodal data to address the challenging issues of place recognition. Compared with conventional place recognition, the proposed OpenMPR not only leverages multiple effective descriptors, but also assigns different weights to those descriptors in image matching. Incorporating GNSS data into the algorithm, cone-based sequence searching is used for robust place recognition. The experiments illustrate that the proposed algorithm manages to solve the place recognition issue in real-world scenarios and surpasses state-of-the-art algorithms in terms of assistive navigation performance. On the real-world testing dataset, the online OpenMPR achieves 88.7% precision at 100% recall without illumination changes, and achieves 57.8% precision at 99.3% recall with illumination changes. The OpenMPR is available at https://github.com/chengricky/OpenMultiPR.

KW - Visual Localization

KW - Computer Vision

KW - Navigational Assistance

KW - Assistive Technology

UR - http://www.scopus.com/inward/record.url?scp=85075656642&partnerID=8YFLogxK

U2 - 10.1088/1361-6501/ab2106

DO - 10.1088/1361-6501/ab2106

M3 - Article

VL - 30

JO - Measurement Science and Technology

JF - Measurement Science and Technology

SN - 0957-0233

IS - 12

M1 - 124004

ER -