April 2016
« May    




Innovative Applications of Computing to Energy and Health: Shwetak Patel

January 29, 2013

Recent trends in technology development are producing some amazing new applications and areas of research. People that are able to cover a bunch of synergistically related disciplines suddenly find themselves in a great position to do ground breaking work. One of those people is Shwetak Patel. I was fortunate to attend his recent talk at a Computer Science Seminar at the University of Toronto, as well as have lunch with him. Here’s some basic information about him from his Wikipedia page:

“He is an assistant professor at the University of Washington in Computer Science & Engineering and Electrical Engineering, where he joined in 2008. His technology start-up company on energy sensing, Zensi, was acquired by Belkin International, Inc. in 2010. He was named a 2011MacArthur Fellow.”

Shwetak Patel was born and raised in Alabama (yes, I was surprised) and he did his Ph.D with Greg Abowd at Georgia Tech. “Patel focuses on developing easy-to-deploy sensing technologies, activity recognition, and applications for energy monitoring. He also has developed novel interaction techniques for mobile devices, mobile sensing systems, and wireless sensor platforms, many of which in collaboration with Microsoft Research, where is also a visiting researcher.”


Shwetak Patel’s research is in the interesting confluence between ubiquitous computing, human-computer interaction, and sensor systems ( His four main areas of research interest are:

sustainability sensing
health sensing
low-power sensing
novel interaction techniques

Shwetak Patel is pretty handy, being both a licensed electrician and a licensed plumber. He developed those skills as a teenager when he worked with Habitat for Humanity ( to help build “simple, decent, and affordable housing”).

In his recent talk Patel spent about two thirds of the time talking about energy and water sensing, finishing up by presenting results from a mobile health sensing project.

The aim of the energy sensing project was to help consumers reduce their residential consumption. This has been a problem up to now because homeowners and tenants are provided with very little information about energy consumption at the appliance level. This is a significant societal problem since 21% of US energy use is residential and similar proportions of residential energy use are likely in other developed countries.

Another figure that was surprising to me is that 56% of treated, potable water use is also residential.

Patel contrasted what is shown on energy bills with grocery bills. Unlike grocery bills that are broken down by item (and maybe even by category in some stores) energy bills have only one price on them typically (no breakdown).

To Patel, an energy bill with no breakdown makes no sense and is a problem that needs to be fixed.

Currently the electrical meter is designed for billing purposes only, and it is not designed for consumers. The obvious solution of using distributed direct sensing to determine how much power is used by each device or appliance is not practical, since it would be to difficult to retrofit all the appliances and outlets.

Patel discussed an alliterative approach, infrastructure mediated sensing. This is where the utility infrastructure is used as the sensor. He then described the elecrisense system that he has developed. This is a single sensor on the power line that can detect the use of electrical devices, and it is homeowner installable. One problem is that modern devices all tend to look similar in terms of power information (60-100W), thus an alternative sensing criterion had to be found. Fortunately, all devices generate EMI noise and this acts like an appliance heartbeat that can identify which appliance is producing each burst of noise in the line.

Thus looking at the electrical noise from a voltage level allows you to infer devices switching on and off. These days most devices use switch mode power supplies, and while these devices are small and efficient, they also produce EMI noise. Patel demonstrated how, in the frequency domain (kilohertz,) you can differentiate devices based on the dominant frequency of the EMI noise that they produce. This happens because different devices have different power supplies which have different load characteristics.

Here is a chart showing how different devices are inferred from the frequency components of the EMI noise.


Some of this work has major privacy implications. Patel gave the example of inferring what movie a person is watching based on patterns of EMI noise. You can do this by looking at how the power supply is switching to keep up with the backlighting of the TV (cycling between light and dark). By looking up stored brightness switching data from a movie database (e.g., Netflix) it is then possible to determine which movie is being watched with just a few minutes of EMI noise (power switching) data.

Patel’s research relies a lot on Machine learning methods. ln developing some of his sensors used a combination of nearest neighbor classification and Support Vector Machines. The resulting sensor interface then consisted of:the following steps:

power line interface (high pass filter)
digitizer and frequency transform
event processor and classifier

The methods described above can be used to identify devices, but now the problem becomes “how can I get the power information from each device?” The “contact-less whole-house current consumption sensor” was offered as a solution to this problem. It detects the magnetic field behind the metal at the fuse box (if I understood correctly) within 4% of the “true power” and it also provides additional features for classification. Patel then did a validation study to evaluate how well his inferred measurements matched the gold standard of directly monitoring the power utilization of each device. To do this he installed monitoring equipment on about 100 devices in each of 7 homes. I asked him about the problem of getting ethics approval for this work and he said that it was quite difficult and took a few months (I”m not surprised).

He referred to the sensors placed behind devices as ground truth sensors (I don’t think it was meant to be a pun).

He found that the average accuracy for identifying appliance events was 91.7%, which I think is amazingly good considering the indirect measurements that he was relying on.

The next part of the talk looked at Modeling water use. In this case, instead of looking at EMI noise he was looking at the “water hammer”. This is the resonance in the water column that result from sudden local changes in water pressure as valves open and close. Just as in the case of EMI noise, a signature may be inferred that is dependent on the fixture type, the valve type, and the valve location in the home.

Using a similar approach as with Electrisense, he installed ground truth sensors on every device within five highly instrumented homes. There were 103 water valves used and a total of 14, 960 water events (over 5 weeks of observation).

The obtained accuracy was 97% for isolated events.

The accuracy dropped to a still respectable 89.5% for compound (>2) events (e.g., when the rinse cycle for the washing machine coincided with someone washing their hands).

Using this information he could then develop water utilization user interfaces and break down usage by different activities. One of the interesting points he made was that even with moderate levels of accuracy, feedback on amounts and types of usage can be very effective in encouraging more conservation.

Here is an image showing how Elecrisense can be used to provide a much more informative power bill.


The utilization data also yielded some surprising insights. People tend to think of their kitchen appliances as consuming a lot of energy, but he actually found the proportion of total household use by those appliances was less than 5%. Not surprising perhaps, a pool pump was taking 30% of the power consumption in one home. But the other big surprise was that a digital video recorder (DVR) was accounting for 11% of the household power utilization. He attributed that to both the fact that the device was always on, and to the inefficiency of its design (in the US, entertainment devices didn’t have to be energy star compliant).

I found an interesting summary of Patel’s energy sensing work which I quote below (
“Imagine receiving your next credit statement and discovering that your credit card company will no longer provide you with an itemized list of your monthly charges. Who in their right mind would pay a bill with no breakdown of the charges? Yet for decades, around the world, consumers have blindly paid their electric and water bills without having the slightest inkling of how much each device in their home costs them to run each month.

Shwetak Patel, a 29-year-old MacArthur Fellow and currently an assistant professor of computer science and engineering at the University of Washington, has found a better way. Patel recognized that every device in our homes has a unique digital signature. This signature can be detected with simple wireless sensors on existing infrastructure such as gas lines, electrical wiring, plumbing, and ventilation. Patel’s smart algorithms, combined with a small sensor that can be plugged in anywhere in a home, or screwed into a hose spigot for water, can inexpensively provide visual feedback revealing how much of the resource each appliance consumes. Armed with this information, consumers will be able to see which devices are the biggest electricity or water waster and suggest ways to conserve.

In the photo below, Patel and his cousin Sy, with his wife, Varsa, and their children, Sejal, Rishi, and Jaisal, stand behind their Hayward, California, home amid their appliances and installations. Patel holds water- and electrical-use sensors above an iPad showing the typical usage of his cousin’s household. As it turns out, electric pool pumps (in the foreground, with lights) consume the most electricity, along with digital video recorders, which eat up 11 percent of household power, since they are left on continuously.”


Patel mentioned that he had to stand on top of a cherry picker for a couple of hours waiting for the right lighting conditions to take this photo. The things we do for science!

Mobile health sensing

By the time we got to the mobile health sensing part of the talk I think that my attention had started to wander. For me at least, there is only so much high quality stuff I can absorb at one time. Be that as it may, here’s a quick summary of the health sensing work which focused on one particular problem, that of measuring lung function (spirometry). For diabetics measuring their own blood sugar levels with glucometers has become standard practice, but sprirometry has yet to become a ubiquitous self-monitoring tool in the same way. However, there is clearly a need for self-management tools of a variety of chronic diseases, and not just diabetes.

SpiroSmart is a mobile phone spirometer application. It uses the phone’s microphone to measure lung function. Traditional spirometers use a flow sensor (e.g., a turbine). This isn’t feasible with a mobile phone so Patel found a way to use the microphone to measure the acoustic data that reflects flow based resonances in the vocal tract.

So in this case the noise produced in the vocal tract as transmitted by the microphone is the sensor. But in this case, what would be noise for an automated speech recognition system becomes the signal for spirometry. This signal can be used as the signal for lung function modeling. Spirometry is an effort dependent test and Patel looked at the problem of how the phone should be held and what fixtures if any should be used. He found that with minimal training people could hold the phone in a suitable position and perform the test. He then did an evaluation (I like this empirical testing) by comparing phone based spirometry vs. measurement by a standard spirometer in a group of 52 patients. The accuracy was on average within 5% of the measures produced by a clinical spirometer.

The following images show SpiroSmart being used (top left) vs. conventional spirometry (top right). Sample results for each technique are plotted below.


The bottom line for me is that this was a fascinating talk and showed the power of combining machine learning methods with creative uses of sensing technologies to solve what seem to be hard problems in innovative ways.

Posted in: Events | Tags: , , , , , , , , , ,

Leave a Reply

Follow Us!

Latest Posts