The Digital Family Portrait, developed in prototype form by Associate Professor of Computing Beth Mynatt, is one application that is forging ahead. Its goal is to give adult children a virtual awareness of the condition and activity level of their independently living parent, who often resides in another town.
The portrait – actually a computer screen with the older adult's picture displayed in a digital frame – might hang on a mantel over a fireplace in the adult child's home. Sensing data from the parent's home feeds the portrait's frame with 28 butterfly icons that change in size, indicating the parent's daily activity levels for a 28-day period.
If the adult child wants more information about a particular day, she touches that icon and gets a more detailed visualization for that day. In this screen, information indicates the weather and temperature at the older adult's home, and a bar graph reflects the person's room-to-room activity.
Sensors in the Broadband Institute Residential Laboratory can already track movement of people from room to room, giving researchers a broad sense of activity based on an inhabitant's motion. Mynatt has experimented with using this data in the portrait application. In addition, she has conducted studies involving actual families in their own homes using information from interviews with the senior adult to supply data for the portrait. Mynatt plans more experiments like these, both in the Residential Laboratory with actual human subjects and in the field with additional families whose houses will be instrumented with simple sensing systems.
"The sensing information will provide a visualization for a unique footprint of a person's daily life," Mynatt explains. "We might find out that the older adult got up during the night a couple of times to go to the bathroom. Then they got up and had breakfast, and then there was a lull of activity followed by another burst of activity around dinner and bedtime. So without breaching the privacy boundary, you can still get a good sense of a person's overall activity level using the Digital Family Portrait. If you know that person well, you gain a sense over time about whether today looks like a normal day."
With the Digital Family Portrait well under way, Mynatt and colleague Wendy Rogers, a professor of psychology, want to develop technologies to address memory declines in older adults. So they are investigating interfaces that make up for short-term memory impairments, as well as systems to provide near-term reminders. If an older adult is interrupted or distracted from a task, he might forget what he was doing. An aware computing system could continuously gather information and visually present information back to the resident to jog his memory.
photo by Gary Meek
|
|
In a prototype technology demonstration called "What was I cooking?", cameras under the kitchen cabinets record a computer science student, Mahmudul Jilani, putting white ingredients into a bowl. Researcher Quan Tran uses the sensing information to create a cartoon-strip-format display of the student's recent actions. Then when the student's activity is interrupted and they forget what ingredients were added to the bowl, the display provides a reminder.
(300-dpi JPEG version - 398k)
|
|
Mynatt, Rogers and Gregory Abowd, an associate professor of computing and research director of the Aware Home project, developed a prototype interface called "What was I cooking?" In a demonstration of the technology, cameras under the kitchen cabinets record a person putting white ingredients into a bowl. For the prototype only, containers are also instrumented with sensors. Researchers use the sensing information to create a cartoon-strip-format display of the person's recent actions. Then when the person's activity is interrupted and they forget what ingredients were added to the bowl, the display provides a reminder.
A similar visual reminder system could prompt recall when older adults start a task in one room, then must go to another room to complete it. When they get to the other room, they often forget why they needed to go there. Rogers also wants to develop interfaces for prospective memory – memory to do something in the future (e.g., remembering to take medication). "So the house could became a surrogate memory system," Mynatt adds.
To promote social interaction between seniors and their grandchildren, Mynatt and visiting researcher Itiro Siio and graduate student Jim Rowan have developed an application called Peek-A-Drawer. It provides virtual shared drawers for grandparents who want to share a part of their living space with grandchildren who live far away. When a user puts something in an upper drawer of a chest and closes it, a photograph is taken automatically. Then the image appears on a monitor in the lower drawer of a chest in the relative's house. "The operation is so simple that even children can communicate with their grandparents," the researchers report. "Also, as the camera only takes pictures of objects inside the drawer, privacy is assured."
Meanwhile, Associate Professors Irfan Essa and Aaron Bobick are focusing on sensing technology, with the Residential Laboratory serving as the test bed. "The goal is to build an infrastructure so an aware home can see and hear its inhabitants and interact with them like a normal person would," Essa explains.
Two prototype sensing systems exist in the Residential Laboratory, though neither is continuously operating for now. In the kitchen, optical sensors in the ceiling can locate a person in the room and approximately track where they are headed (e.g., toward the refrigerator or oven). In another room, cameras and microphones are embedded in the walls. Researchers are primarily concerned with data collection for now, rather than data processing, Essa says. But they can interpret audio data with commercial speech recognition software.
photo by Gary Meek
|
|
To promote social interaction between seniors and their grandchildren, Associate Professor Beth Mynatt, visiting researcher Itiro Siio and graduate student Jim Rowan have developed an application called Peek-A-Drawer. It provides virtual shared drawers for grandparents who want to share a part of their living space with grandchildren who live far away.
|
|
Later, researchers want the system to recognize a person's location when they speak. Then if the person is speaking in the dark, the cameras can to move where the person is located. Next, researchers will instrument the Residential Laboratory with sensors focused on specific areas, like the sink in the kitchen to gather data on when an inhabitant is washing the dishes, for example.
Altogether, the video sensing systems in the Residential Laboratory will provide generic location information – where people are and when – and then add basic activity recognition capabilities to the system, Bobick says. "One simple example is sitting on the sofa," he explains. "To assess your cognitive engagement, it matters a lot whether you are watching TV, reading the newspaper or sleeping. So once the tracking system identifies a person on the couch, I want to have more computer vision to tell me which level of activity they are engaged in."
Though prototype sensing systems may be apparent to users, future versions will not be. "We don't want people to feel tracked without giving any value back to the individual," Abowd says. "We have a desire to make sensing invisible in the house. We want to leave it to the house to be aware, and that is primarily a sensing challenge. But we also want people to be comfortable about what is being known about them.... We have found that people are willing to give up some privacy to have the ability to maintain their quality of life and stay in their own home."
It is in these issues where technology and real life intersect. Aware Home researchers have anticipated this intersection and are crossing it in advance of technology design.
– Jane M. Sanders
Contents
Research Horizons
GT Research News
GTRI
Georgia Tech
Send questions and comments regarding these pages to
webadmin@edi.gatech.edu
Last updated: July 25, 2002
