Exploratorium_8.1.1
Uses of AI
- to Enhance Sensing

Annotated Bibliography

Category
  • Visions of the Future
    • PAST Visions of the Future
  • Input
    • using Text
    • using Pointing Devices
    • using eyes
    • using Voice or Sign
    • using Machine Vision
    • using 1D Continuous Controls
    • using 2D to 6D Continuous Controls
    • using Gestures on a Surface (2D)
    • using Gestures in 3-space
    • Unique Combinations of Input Devices
    • Enhancing/Accelerating Input
  • Sensing (by device)
    • Biological
    • Mechanisms for Sensing
    • Environmental
    • Virtual sensors
    • Other
  • Output
    • to Visual sense
    • to Tactile or Pain sense
    • as Language (speech, text, sign-lang)
    • to Auditory sense
    • to Directly Activate/Control user
  • Altering/Enhancing Sensory Input
    • what is Seen
    • Translating Between Senses
  • Direct Brain Interfaces
    • Unique Combinations of DBI types
    • Mechanisms for Direct Brain Interfaces
    • DB Output, to Control or Project
    • DB Input - into Brain
    • Direct Brain Monitoring
  • Virtual & Augmented Reality (xR)
    • Mechanisms for AR/VR/xR
    • Uses of AR/VR/xR
  • Uses of AI
    • to Enhance Input
    • as Assistant (voice, text, etc.)
    • to Enhance/Adapt Output
    • to Enhance Sensing
    • Other uses of AI in interfaces
  • Accessibility + Digital Divide
    • Accessibility & HCI
    • Digital Divide - Digital Affinity
    • Digital Divide - Disability
  • Other
Search

COPY currently shown bibliography

Girju, R. (2021). Adaptive multimodal and multisensory empathic technologies for enhanced human communication. arXiv preprint arXiv:2110.15054.

  • The paper discusses the need for a principled framework that includes senses (intuitive, adaptive, and effective empathic) in the digital design process of telehealth and the use of Artificial Intelligence within it. It presents features and identifies some challenges that need to be considered in the process.

Gomez, Juan Diego and Mohammed, Sinan and Bologna, Guido and Pun, Thierry. 2011.Toward 3D scene understanding via audio-description: Kinect-iPad fusion for the visually impaired. In The proceedings of the 13th international ACM SIGACCESS conference on Computers and accessibility (ASSETS '11). Association for Computing Machinery, New York, NY, USA, 293–294. https://doi.org/10.1145/2049536.2049613

  • Computer vision based framework for real time object localization and audio description to provide color and depth perception using Microsoft kinect 3D motion sensor.

Knoll, Alois. "Researchers transmit digital twin of road traffic situation into vehicles: Bird's-eye view improves safety of autonomous driving". Artificial Intelligence, Mobility. Technical University of Munich. Sept. 13, 2022.

  • This paper demonstrates how multiple cameras can be combined to give a person a view they would not otherwise have

Machine translation of cortical activity to text with an encoder–decoder framework https://www.nature.com/articles/s41593-020-0608-8

  • Training a recurrent neural network to encode each sentence-length sequence of neural activity into an abstract representation, and then to decode this representation, word by word, into an English sentence.

Z. Lu, S. Cai, B. Chen, Z. Liu, L. Guo and L. Yao, "Wearable Real-Time Gesture Recognition Scheme Based on A-Mode Ultrasound," in IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 30, pp. 2623-2629, 2022, doi: 10.1109/TNSRE.2022.3205026. Accessed at: https://ieeexplore.ieee.org/abstract/document/9881558

  • This study tested a portable gesture recognition system using ultrasound with AI that can be used offline, finding it has a 96.92% accuracy rate, showing the technology has great potential for use in gesture-based interfaces in many contexts.