Machine Learning on Multi-modal Sensor Data

Digital self-tracking technology has become more accessible to the public in recent years with the development of connected portable devices (such as smart phones, smart watches, smart bands, and other personal biological monitoring devices), human biosensors, as well as information management systems designed for monitoring, storing and analyzing human self-tracking data. The proliferation of such technology has made it much easier than any time before to collect biological and physiological signals, such as the electrocardiogram (ECG), oxygen saturation (SpQ2), heart rate (HR), the electroencephalogram (EEG), Galvanic Skin Response (GSR), blood pressure/oxygen level, body temperature, etc. These self-tracking data can help us better understand each individual’s health conditions by monitoring and analyzing such data.  As a result, mining such sensor data has been gaining significant attention from both industry and academia in recent years.

This presentation focuses on technologies for machine learning platform on analyzing self-monitored data for real-world applications. As an example, a project about multi-modal sensor data for activity recognition will be introduced. In this project, we first collect the sensor data from smart home setting in Bristol (UK), and learn the patterns of human behaviors by machine learning technology. The built machine learning system has the capability to predict the activities of daily living and posture and ambulation. These prediction efforts have practical impact on identifying and alerting stroke, falls, and other emergency scenarios for clinical professionals and caregivers, such as elder people healthcare.

Outline:

1. Background of Machine Learning on Multi-modal Sensor Data
   1. Sensors in real-world scenarios
   2. Machine Learning on Sensor Data
   3. Challenges of learning from multi-modal sensor data
2. Machine learning platform and structure
   1. Overall System architecture
   2. Preprocessing and feature engineering
   3. Modeling and Offline evaluation
   4. Online deployment, evaluation, and model updating
3. Challenges and insight discoveries
   1. Data sparsity and missing issue
   2. Imbalanced data
   3. Sequential behavior pattern over time
4. Summary and Application Discussion
   1. Application scenario #1: Sequential activity recognition for auto-monitoring system in police station
   2. Application scenario #2: Machine learning on multi-modal sensor data for elder people emergency alert system

参考译文：

随着如手机、智能手表和生物传感器等便携式设备的普及，数字化自跟踪技术近几年已经获得了长足发展。同样，用于监测、存储和分析人类自跟踪数据的信息管理系统设计也在不断的优化。

和以往任何时候相比，由于技术的快速发展，使得收集生物数据和生理数据这个过程变得越来越容易，例如心电图(ECG)，血氧饱和度(SpQ2)，心率(HR)，脑电波信号(EEG)，皮肤电反应(GSR)，血压，含氧量，体温等等。这些自跟踪数据可以帮助我们通过监测和分析这些数据，来更好地理解个体的健康状况。正因为此，挖掘传感数据中的隐含价值受到越来越多工业界和学术界的关注。

本次演讲会聚焦在介绍机器学习平台在传感数据中的应用，详细对利用多源传感器数据结合机器学习技术来检测人体行为项目做详细介绍。在这一项目中，首先收集位于Bristol (UK)的智能家庭传感器数据，包括重力传感器，环境传感器和三维视觉传感器，之后通过机器学习技术来构建人类行为监测模型。这一机器学习系统能通过分析传感数据，动态准确的检测日常生活里的人体行为活动。这一系统对于识别或者预测中风，摔倒以及其他危险事情有很切实的意义，尤其是在临床案例和老人照顾方面。

演讲提纲

1. 机器学习在多源传感数据上应用的背景介绍

• 现实生活中传感器的应用
• 机器学习和传感器数据的结合
• 机器学习应用于多源传感数据的技术挑战

2. 机器学习平台和架构

• 系统架构
• 预处理和特征工程
• 建模和离线性能评估
• 在线部署、评估和模型更新

3. 挑战和项目收获

• 稀疏数据与数据缺失问题
• 数据分布不均衡问题
• 时序行为模式分析

4. 总结和应用挑战

• 案例：警察局自动监测系统里的连续行为识别
• 案例：老年人紧急状况监测，预警系统