Building a cyber-physical fall detection system for seniors
The population of senior citizens is on the rise, and the number staying at home longer is increasing. However, a leading source of injury for seniors is falling (over a third of seniors older than 65 fall each year1), with a large number of serious injuries or even death. Seniors living at home, often by themselves, are at high risk for delays in receiving necessary medical help. We have developed a Senior Collapse Detection (SCD) system as a domestic cyber-physical system to monitor for and detect falls.2 SCD communicates with the senior, and in cases of no response or an affirmative response, the system has an emergency response protocol. SCD could help seniors get more immediate attention when they do have falls, and as such could possibly prevent death and reduce the severity of long-term injury.
Detecting falls is not an easy task. Simply enumerating all of the different ways a person could fall was the subject of a survey from Mubashir,3 and we have further identified a number of additional fall types not in the survey. Each fall type has certain characteristics that our computer vision algorithms are coded to detect. These detection algorithms have to contend with the nearly constant difficulty of missing data, for instance, parts of a person's body hidden by their orientation to the sensor or by standing behind a piece of furniture. In addition, processing 3D depth data into skeleton tracking is not easy, resulting in joint locations that are by no means perfect, and sometimes completely erroneous. Our application tracks users as they move freely about their home. We have used the vast research on fall indication factors4–8 in SCD's fall detection components of head motion fall detection, floor prone detection, and horizontal detection.
Significant acceleration of the head is a strong indicator of falling,5–8 and this is monitored in the skeleton data by the head motion fall detection component. The floor prone detection component uses the common-sense notion that most falls end with the senior prone on the floor. A senior is classed as on the floor if a significant number of skeleton joints are within a small distance of the calculated 3D ground plane. Finally, the horizontal detector component detects relatively horizontally positioned skeletons and checks the distance of the individual's head to the floor. As with other ‘medically inclined’ systems, we favor false positives over missed fall events. For example, in our system, the movements an individual makes when moving to lie down on the floor can be detected as a fall. By calculating the distance from various joints to the floor, our detection algorithms are able to detect real fall events while ignoring fall-like normal daily activities. A great advantage of our system is that it is autonomous and no devices are worn by the senior.
Raw skeleton data can be unstable. Consequently, SCD has the option to use reliability metrics for each detected joint position and angle based on comparison to physical models. These models are derived from physical therapy data, including range-of-motion parameters.9 Currently, SCD uses a single model, but we are examining the possibility of matching the individuals detected in the video frame to the most appropriate physical therapy data based on age, height, and other physical properties.
The SCD algorithms have performed well with our recorded data of simulated falls. The next step will be to test SCD in a real senior home environment.
Lynne Grewe is a professor in computer science. She is founder and director of iLab working on vision and media data understanding with web and mobile infrastructures.
Steven Magaña-Zook, a member of iLab, is a graduate student in computer science interested in data science. He is also a computer scientist at Lawrence Livermore National Laboratory.