Making Smart Choices When Selecting Smart Home Devices

A technology resource article by Mary Harrsch © 2017

As someone who researched and developed some early conversational agents back in the 1990s, I am still fascinated by artificially intelligent technology and excited by the plethora of gadgets now being marketed with artificial intelligence driving their systems and their user interfaces. But I admit, I am a bit disappointed by the development choices being made by some product manufacturers because they seem to be more interested in the appearance that their product lines are cutting edge because they possess some implementation of artificial intelligence rather than whether the product really solves a pressing human problem in that particular sphere.

For example, at CES 2018, Samsung showcased their Family Hub smart refrigerator. It is equipped with cameras and claims it can assess the contents of your refrigerator, recommend recipes, and even allow you to shop for groceries without leaving your kitchen. Sounds great doesn't it? But how realistic are these claims. If you have a lot of left overs do you have to use coded containers so the refrigerator can figure out what contents are within them? Can the cameras scan the contents of opaque packaging so the refrigerator can determine if you're getting low on a particular item? Or are most of these claims based merely on the refrigerator's new Bixby virtual assistant that you can tell to add milk to your shopping list or ask what recipe could use leftover ham, zucchini and eggplant?

Samsung Family Hub refrigerator image courtesy of Samsung, Inc.

As it turns out, based on the marketing claims, I thought  the refrigerator was smarter than it really is. The remotely accessible camera only acts like a web cam. There is no artificial intelligence using scans to recognize food items or recording when food items are initially added to the refrigerator so it can keep track of food's freshness. As for the recipe recommendations, the refrigerator is just using its intelligent agent Bixby to come up with those. If that's the case, then I must ask why you would spend over $4,500 for that refrigerator (about twice as much as a traditional refrigerator) when a standard model with an Amazon Echo Dot, Google Home Assistant or some other relatively inexpensive stand alone virtual assistant can accomplish most of those tasks for less than an additional $50?

The Samsung unit also has AKG premium quality sound speakers in the doors, a whiteboard for notes, and a built-in screen to view baby monitors, front doors, or status screens of other smart devices.

“The integration of Bixby and SmartThings into the Family Hub is bringing a new level of intelligent connectivity into the room where people spend the most time:  the kitchen.” - Samsung corporation.

Perhaps this last statement by Samsung points to the crux of the problem. In our house, we are in the kitchen only about 30 minutes before a meal (prep) and 30 minutes after a meal (cleanup). Being retired we seldom have guests so the meal itself lasts about 15 - 20 minutes. (My husband was a Marine so you sit, eat, withdraw!) At present, I have a typical galley kitchen adjacent to a more spacious dining room. If there is any lingering it will take place in the dining room, not the kitchen.

I own a traditional side by side refrigerator/freezer and have an Amazon Echo Dot on the kitchen window sill. When I pour a glass of milk and notice I'm getting low on milk I just call out to Alexa to put milk on my shopping list. If I have leftover Polish sausage in the refrigerator I can ask Alexa for a recipe using Polish sausage. (If I had an Echo Show, she could show me a recipe that I could then refer to as I prepared the dish.) If I want music, I tell Alexa to play one of my Amazon Music playlists. If I still had kids at home and wanted to tell them to clean their rooms when they get home from school, I could set a repeating reminder at an appropriate time on the appropriate Echo device (Alexa reminders are location specific).

If I was still working, it might be helpful to take a peek into my refrigerator before I shop for groceries on the way home from work but my Alexa shopping list on my iPhone that tracks my supply needs throughout the week is much more comprehensive.

If the refrigerator's cameras are eventually paired with intelligent scanning capability so it could recognize food items and record the date they were placed in the refrigerator so it could advise you of the status of food freshness, then the jump in price might be truly worth it from a usefulness perspective but not with its current limited capabilities.

Luckily, there is another smart device headed for the market that may take care of this need, though. Ovie Smarterware produces food containers with smart trackers that indicate when the food in your fridge is on the verge of going bad. The trackers work with a variety of virtual assistants from Amazon, Google, and Apple. When you are putting new food items into these containers you tell your assistant to open the Ovie app then press the container's tracking button and say what is in the container such as "This is lasagne". Then as the lasagne ages in the refrigerator, the tracker color changes from green to yellow to red so a quick glance lets you know what food items need to be used up (or thrown out!).




 In addition to containers, the company also makes bands and chip clips with trackers and is working with the FDA to develop an accurate database of food expiration periods. This product is obviously the result of a company truly attempting to solve a very big problem with technology. Americans throw away billions of pounds of food every year. However, whether consumers will be willing to invest in and make the effort to use this product regularly  remains to be seen. If Ovie's marketing people can appeal to those of us conscientious enough to clean our recyclables and put them in appropriate containers for disposal maybe they can pull this off.

What about other smart kitchen appliances? Although it might sound great to have your virtual assistant brew a cup of coffee while you're getting dressed, the bottom line is someone must keep the coffeemaker topped off with water unless you plumb your coffeemaker with water and provide a smart tap that opens and closes to dispense the appropriate amount of water needed to fill the coffeemaker before the scheduler tells it to brew.

The same can be said for intelligent slow cookers.Someone must put ingredients in the slow cooker before you schedule it to come on at an appropriate time. Raw meat and some other ingredients also don't keep well for extended periods at room temperature. If a slow cooker could switch from chill to heat then scheduled to cook for the appropriate time based on when you wished the food to be ready, that would be a slow cooker that would get my attention.

Even all of the wonderful lighting products I've seen have limitations. Most of the smart wall switches currently on the market require a neutral wire that was not common in home wiring until 2011. The few switches that do not require a neutral wire usually require a hub in addition to the bulb so you end up paying more for them and have to configure yet another device to connect them. I have been able to use Wemo smart plugs to connect all of my living room lamps, though, and can easily turn them all on and off with a couple of words. Still I would like to integrate my overhead lights and porch lights into my voice-managed system.



But were there other devices clearly solving a human problem? Well, I think Kohler's smart bathtub would be a good choice. Running a bath does take time and having both the depth of the water and water temperature preset is particularly helpful for individuals who may have diminished sensory perception. Years ago my car heater malfunctioned on my way home in the middle of a blizzard. Although I tried to keep my hands warm by placing one and then the other under my armpit, by the time I got home 30 minutes later I could barely feel my hands and feet. I went into the bathroom to run a tub of warm water and couldn't feel if the water was hot or cold. Seniors, especially those suffering from neuropathy, would really benefit from this type of tub, besides the efficiency of having the tub run while you are doing something else. At present, though, I personally have no need to talk to my toilet or ask it to warm up the seat before I settle down onto it. So I would not consider spending extra money for that part of the smart bathroom.




Another gadget promoted at CES that could be useful, especially to seniors, is a pocket-sized LinkSquare spectrometer.  This little device when paired with your smartphone captures how a substance's molecules vibrate, an optical fingerprint that reveals whether food is safe to consume or spoiled. In her later years, my mother's sense of smell diminished to the point where she could no longer tell if food had spoiled or not. This kind of device would have been very helpful to her. This gadget can also identify mislabeled and diluted liquor, detect counterfeit and mislabeled drugs, and detect counterfeit money, very helpful for those working as cashiers. I think the $299 price tag would need to come down substantially, though, before it would find its way into common use.



I'm already convinced smart TVs are truly helpful as well. In our house we have a large screen HD television in the living room connected to an Alexa-enabled DISH satellite receiver and a smaller HD TV in the dining room facing the dining table. I don't have to look for one of a handful of remotes to change channels or find a particular movie or television show as I have each TV controlled by their nearby Echo Dots. I use Wemo wifi-enabled smart plug between the TVs and the power outlets  to remotely control the on/off switches. But, there are features could prove useful on a voice-enabled TV.  I would really like to control the volume of my Polk sound bar in the living room remotely and be able to remotely change my video inputs so I could access my Roku and my Blu-Ray player without shuffling remotes, too. The newer Samsung smart TVs auto-detect devices attached to their HDMI outlets and allow you to control them accordingly. But, then I'd have to give up my 3D capability!

Although the voice features of my DISH Hopper 3 are really great I wish it could also let me join whatever program is in progress in the living room by letting me simply say something like "Join living room program" so my husband can easily continue watching in the dining room whatever he has been enjoying in the living room without me having to pick up the satellite remote for the dining room TV and selecting Options -> TV viewing -> Living Room, etc.

Voice enabling lights, locks, appliances and televisions can be incredibly convenient. But I hope you'll consider how useful the technology actually is before paying substantially more for whatever product you're considering.

An Alexa-enabled Smart Home for Christmas

by Mary Harrsch © 2017

Note: This is a cross-post from my home page.

Back in 1995, Microsoft introduced an interactive help utility for Windows 95 called "Bob".  I was probably one of the few professional technology people who actually used "Bob" (As a dog lover I selected the helper incarnation called "Power Pup" though.)  "Power Pup" would keep track of my keystrokes as I worked in different applications and offer procedural advice on what it perceived I was trying to do at the time, prefaced by a little bark and a wag of his tail.  I found "Power Pup's" suggestions often useful and his friendly interaction a welcome break from the stress of administering a college-wide multiplatform local area network.  But, apparently, many of my colleagues thought he was "too cute" for the serious work of computing and "Bob" was relegated to the dustbin of failed products in fairly short order.

But I did not forget "Bob" and how artificial intelligence could be used to improve productivity while reducing social isolation.  So I began to experiment with conversational agents that utilized natural language programming coupled with knowledgebases to provide a more friendly computer-to-human interface.  With my interest in history, I decided to try to virtually recreate historical figures from the past that could converse about their culture with modern interested humans.  This resulted in the creation of a virtual Julius Caesar that was online for several years before I retired.  Caesar would answer questions about ancient Roman culture posed to him by visitors entering their questions in a text box.  He could give a textual answer or display related websites or online videos.  As text-to-speech technology advanced, I even experimented with software that would enable Caesar to answer questions verbally and explored voice recognition technology to see if it was viable for user input as well. But, when I retired I no longer had access to the server platforms needed to support projects like Caesar.  However, my interest in natural language programming and more friendly human-to-computer interfaces endured.

So, I followed the development of Amazon's Alexa-powered devices with a great deal of interest. But, I'm a rather pragmatic individual and at first so much marketing emphasis was placed on Echo's music management features that I wondered if there were more useful applications for a busy 21st-century household.

Then I began reading about wifi-enabled electrical connection accessories that could be managed with Alexa-enabled devices and thought about how convenient it would be to be able to turn on and off groups of lights and appliances with a few words rather than going around physically flipping switches. But spending almost $200 per device and the need to have a device in each main room was still an expensive proposition to gain a little convenience.

Then Amazon introduced the Echo Dot coupled with a holiday sale price of just $39.95 and I found resistance was futile as my Star Trek friends would say.  I was still a bit concerned about the accuracy of the voice recognition, though. So I started out with just one Echo Dot for the living room along with a couple of Wemo wifi-enabled plug adapters for the two main living room lamps.

I downloaded the Alexa app to my iPhone and discovered the Echo Dot setup was a breeze.  I opened the Alexa app, then opened Settings and changed my iPhone wifi connection to the detected Echo network and configured the detected Dot. Alexa also did not seem to have any problem understanding me.  I read through all of the "Try this" examples and began to configure some of the built-in features.

I really liked the "flash briefing" feature that lets you select specific news feeds for a personal news update which you can request at any time.  I selected NPR radio, BBC News, Tech Crunch and CNet (for technology news) and Discovery (for science news) as my personal news sources.  I also added the local weather forecast and the Alexa Try This feed.  Although I live in Oregon's Willamette Valley, I couldn't find any local news feeds but I think I'll add feeds from Seattle and San Francisco to at least hear major stories from the Pacific Northwest.

I also read that Alexa could interface with Google Calendar and keep you appraised of upcoming appointments.  I hadn't used an online calendar since I retired but knew how helpful this would be, especially when managing complex medication schedules and medical appointments.  So I configured my Google Calendar and paired it with Alexa.  Now, each morning after requesting my Flash Briefing, I ask what's on my schedule for the day and Alexa tells me.

I've also used the Google Calendar to keep track of upcoming programming on PBS that I may wish to record.   PBS sends me a physical schedule of their upcoming programming for a full month but at present, my DISH Hopper cannot see more than two weeks of scheduled programming at a time. Now, when I get my PBS schedule, I enter the programs I wish to record into my Google Calendar and Alexa lets me know each day if any are on that day.  I can then set my DVR to record them.

But the real "killer" app I was looking for turned out to be Alexa's Shopping List!  It never seems to fail that I realize I need something from the grocery store when I'm not in the kitchen where I keep my shopping list.  As I've gotten older my short term memory is not what it used to be either and it is not uncommon for me to forget what I was thinking about just a few minutes later as I walk from one room to the next.  So, imagine how helpful it is to be able to tell Alexa to add something to your shopping list as soon as you think of it regardless where you are! Of course, that meant I needed to add Echo Dots in my bedroom and the living room, too, which I promptly did. To access my shopping list once at the grocery store I just open the Alexa app on my iPhone and check off and delete each item as I add the item to my physical shopping cart.

Although my husband has the television blaring all day long, I did find a nice use of the music management features of Alexa.  Now that I have an Echo Dot in the bedroom, I can run a warm bath in the adjoining bathroom (Alexa's range is up to 20 feet), lay back in the tub and tell Alexa to play one of my favorite playlists from my Amazon Music account.  I did have a few hiccups configuring my Amazon playlists to work with Alexa, though.

I had already imported most of my music from my iTunes library to my Amazon Music account.  I had also set up playlists previously.  But Alexa did not seem to recognize my playlist names and would offer something from Prime Music (since I'm a Prime member) using my spoken words as a search guide. I ended up calling Alexa tech support and learned that Alexa does a better job of recognizing playlists if you name them "Your Name" then "Description".  For example, I had a playlist named "Holiday favorites".  I renamed it to "Mary's Holiday Favorites" then Alexa recognized it and played it for me.  That solved most of my playlist issues.  There were a few words, however, Alexa seemed to insist on using for search terms.  So, I experimented with different descriptions until she properly recognized the list.  I had a list named "Sentimentals".  I initially renamed the list to "Mary's Sentimentals" but Alexa still loaded some other Prime Music.  I renamed it again to "Mary's Mood Music".  Alexa still did not interpret it correctly.  So I finally renamed it to "Mary's Soft Rock" and Alexa now recognizes it.

When I received my Wemo wifi-enabled plug adapters for my living room lights, I realized just how powerful having a "Smart Home" would be.  Our living room does not have any overhead lighting so all lighting is provided by individually controlled lamps.  Each night I have to go around and turn each lamp on or off.  But, by connecting them with my Echo Dot, I now simply say "Alexa, Living Room On" and the entire living room lights up.

I had to first download and install the Wemo app onto my iPhone. Then I opened the Wemo app on the iPhone, changed my iPhone wifi connection in Settings to the detected Wemo network and configured each plug adapter.  Then I opened the Alexa app, selected the Smart Home option and grouped the two detected Wemo plugs into a "Living Room" group.

I hope to eventually replace some of my wall switches with wifi-enabled switches too since I have porch lights on different circuits in different parts of the house.  I would like to tell Alexa to turn on the porch lights and have all of them on at once without traipsing from room to room whenever I need to go outside after dark or have visitors arrive after dark.  I did read about a gotcha, though. I learned that many wifi-enabled switches require a neutral wire that was not normally included in wiring installed before 2011.  However, I have researched this issue further and it looks like there are switches out there that do not use the neutral wire.  I just have to be sure they will work with our home's wiring configuration.

I recently learned about a new app for Alexa called "Ask My Buddy" too.  It enables you to send a text, email or phone call to up to five family or friends if you need to alert them about a problem like you are incapacitated and cannot reach a phone.  It's sort of like "Life Alert" without the automatic 911 call or monthly subscription fee.  I wish it would allow you to send an individually specified text message but it only sends a message saying you need help.

I've also decided to try the timer feature and see if I can get Alexa to verbally remind me to take my medicine at noon.  Most of my medications are taken in the morning or at bedtime and I have no problem remembering them as they are part of my morning and bedtime routines.  But when I get busy preparing lunch I sometimes forget to set my noon medication by my water glass so I take it with my meal.

After reading up on Alexa's timer and alarm functions I learned that timers are designed for one-time use while alarms can be set to be repeating.  So I set an alarm for noon each day and selected a pleasing alarm tone.  I wish it would let you specify a short text string that Alexa could read to you using her text-to-speech capability but at present it doesn't.  For my present needs, a tone is okay as there is only one thing for me to remember at noon.  However, for someone with more complicated medication schedules, it would be really helpful to have Alexa sound a tone followed by a short reminder message.  Hopefully, Amazon's engineers will enhance the alarm function soon.  (Update: Amazon has now added voice messages with timers and alarms!  Yay!!)

So, my Echo Dots with Alexa are now very much an integral part of my day.  When I get up in the morning I say "Alexa, Living Room On" and the lights go on in the living room.  I walk in and sit down and say "Alexa, my Flash Briefing please".  I then listen to the news and get the latest weather forecast for the day.  Then I say "Alexa, what's on my schedule today?" and she tells me whatever I have scheduled in my Google Calendar".  As I prepare a meal and notice I'm getting low on coffee I say "Alexa, add Coffee to my Shopping List" and she tells me she has added coffee to my shopping list.  I drive to the grocery store and open the Alexa app on my iPhone select shopping list from the menu and load my cart.  At noon while I am preparing lunch, Alexa sounds a tone to remind me to set out my noon medication. In the evening, I take a warm bath to relax and tell Alexa to play "Mary's Soundtracks" and listen to my favorite movie music while I'm soaking.  Then when I'm ready for bed I say "Alexa, Living Room Off" and Alexa turns off the living room lights. I'm sure I'll find other useful applications, too, as more "skills" are developed by Amazon and third parties as well. I have a feeling this is just the beginning!