This article is part of the series Image and Video Processing for Disability.

Open Access Research Article

Color Targets: Fiducials to Help Visually Impaired People Find Their Way by Camera Phone

James Coughlan1* and Roberto Manduchi2

Author Affiliations

1 Rehabilitation Engineering Research Center, Smith-Kettlewell Eye Research Institute, San Francisco, CA 94115, USA

2 University of California, Santa Cruz, CA 95064, USA

For all author emails, please log on.

EURASIP Journal on Image and Video Processing 2007, 2007:096357  doi:10.1155/2007/96357


The electronic version of this article is the complete one and can be found online at: http://jivp.eurasipjournals.com/content/2007/1/096357


Received:16 January 2007
Revisions received:10 May 2007
Accepted:2 August 2007
Published:28 August 2007

© 2007 Coughlan and Manduchi

This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

A major challenge faced by the blind and visually impaired population is that of wayfinding—the ability of a person to find his or her way to a given destination. We propose a new wayfinding aid based on a camera cell phone, which is held by the user to find and read aloud specially designed machine-readable signs, which we call color targets, in indoor environments (labeling locations such as offices and restrooms). Our main technical innovation is that we have designed the color targets to be detected and located in fractions of a second on the cell phone CPU, even at a distance of several meters. Once the sign has been quickly detected, nearby information in the form of a barcode can be read, an operation that typically requires more computational time. An important contribution of this paper is a principled method for optimizing the design of the color targets and the color target detection algorithm based on training data, instead of relying on heuristic choices as in our previous work. We have implemented the system on Nokia 7610 cell phone, and preliminary experiments with blind subjects demonstrate the feasibility of using the system as a real-time wayfinding aid.

References

  1. J Coughlan, R Manduchi, H Shen, Cell phone-based wayfinding for the visually impaired. The 1st International Workshop on Mobile Vision, May 2006, Graz, Austria

  2. W Crandall, B Bentzen, L Myers, J Brabyn, New orientation and accessibility option for persons with visual impairment: transportation applications for remote infrared audible signage. Clinical and Experimental Optometry 84(3), 120–131 (2001). PubMed Abstract | Publisher Full Text OpenURL

  3. AM Ladd, KE Bekris, AP Rudys, DS Wallach, LE Kavraki, On the feasibility of using wireless ethernet for indoor localization. IEEE Transactions on Robotics and Automation 20(3), 555–559 (2004). Publisher Full Text OpenURL

  4. MS Uddin, T Shioyama, Bipolarity- and projective invariant-based zebra-crossing detection for the visually impaired. Proceedings of the 1st IEEE Workshop on Computer Vision Applications for the Visually Impaired (CVAVI '05), June 2005, San Diego, Calif, USA, 22

  5. P Silapachote, J Weinman, A Hanson, R Weiss, MA Mattar, Automatic sign detection and recognition in natural scenes. Proceedings of the 1st IEEE Workshop on Computer Vision Applications for the Visually Impaired (CVAVI '05), June 2005, San Diego, Calif, USA, 27

  6. Y Cho, U Neumann, Multi-ring color fiducial systems for scalable fiducial tracking augmented reality. Proceedings of the IEEE Virtual Reality Annual International Symposium (VRAIS '98), March 1998, Atlanta, Ga, USA, 212

  7. D Claus, AW Fitzgibbon, Reliable fiducial detection in natural scenes. Proceedings of the 8th European Conference on Computer Vision (ECCV '04), May 2004, Prague, Czech Republic 4, 469–480

  8. L Naimark, E Foxlin, Circular data matrix fiducial system and robust image processing for a wearable vision-inertial self-tracker. Proceedings of the International Symposium on Mixed and Augmented Reality (ISMAR '02), September-October 2002, Darmstadt, Germany, 27–36

  9. D Scharstein, AJ Briggs, Real-time recognition of self-similar landmarks. Image and Vision Computing 19(11), 763–772 (2001). Publisher Full Text OpenURL

  10. A State, G Hirota, D Chen, W Garrett, M Livingston, Superior augmented reality registration by integrating landmark tracking and magnetic tracking. Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '96), August 1996, New Orleans, La, USA, 429–438

  11. A Zandifar, PR Duraiswami, A Chahine, LS Davis, A video based interface to textual information for the visually impaired. Proceedings of the 4th IEEE International Conference on Multimodal Interfaces (ICMI '02), October 2002, Pittsburgh, Pa, USA, 325–330

  12. J Liang, D Doermann, H Li, Camera-based analysis of text and documents: a survey. International Journal on Document Analysis and Recognition 7(2-3), 84–104 (2005). Publisher Full Text OpenURL

  13. X Chen, AL Yuille, Detecting and reading text in natural scenes. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '04), June-July 2004, Washington, DC, USA 2, 366–373

  14. J Gao, J Yang, An adaptive algorithm for text detection from natural scenes. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '01), December 2001, Kauai, Hawaii, USA 2, 84–89

  15. AK Jain, B Yu, Automatic text location in images and video frames. Pattern Recognition 31(12), 2055–2076 (1998). Publisher Full Text OpenURL

  16. H Li, D Doermann, O Kia, Automatic text detection and tracking in digital video. IEEE Transactions on Image Processing 9(1), 147–156 (2000). PubMed Abstract | Publisher Full Text OpenURL

  17. P Viola, M Jones, Rapid object detection using a boosted cascade of simple features. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '01), December 2001, Kauai, Hawaii, USA 1, 511–518

  18. BV Funt, GD Finlayson, Color constant color indexing. IEEE Transactions on Pattern Analysis and Machine Intelligence 17(5), 522–529 (1995). Publisher Full Text OpenURL

  19. R Manduchi, C Tomasi, Distinctiveness maps for image matching. Proceedings of the 10th International Conference on Image Analysis and Processing, September 1999, Venice, Italy, 26–31

  20. A Baumberg, Reliable feature matching across widely separated views. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '00), June 2000, Hilton Head Island, SC, USA 1, 774–781

  21. DG Lowe, Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60(2), 91–110 (2004)

  22. C Schmid, R Mohr, C Bauckhage, Evaluation of interest point detectors. International Journal of Computer Vision 37(2), 151–172 (2000). Publisher Full Text OpenURL

  23. J Shi, C Tomasi, Good features to track. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '94), June 1994, Seattle, Wash, USA, 593–600

  24. G Sharma, HJ Trussell, Digital color imaging. IEEE Transactions on Image Processing 6(7), 901–932 (1997). PubMed Abstract | Publisher Full Text OpenURL

  25. J Coughlan, R Manduchi, Functional assessment of a camera phone-based wayfinding system operated by blind users. Conference of IEEE Computer Society and the Biological and Artificial Intelligence Society (IEEE-BAIS), Research on Assistive Technologies Symposium (RAT '07), April 2007, Dayton, Ohio, USA