Samsung Smart Interaction: Hands

Related Articles

Sonos Era 100 Review: The Best Smart Speaker Gets Even Better

March 28, 2023

‘House of the Dragon’ Season Finale: That Killer Ending Explained

October 24, 2022

‘Ant

October 24, 2022

At CES this year one of the most interesting announcements involved Samsung’s Smart Interaction, a new feature on its 2012 TVs that utilizes a built-in camera and microphone to enable you to control the boob tube just by speaking and/or gesturing to it. I’ve spent the last few days doing just that, much to the amusement of my co-workers, and boy are my arms (and at least one finger) tired.

My takeaway? Smart Interaction has promise but feels half-baked and more like a gimmick than a compelling upgrade. Once the novelty wears off, its usefulness is limited (at best) to those times you don’t have a remote in-hand.

And most home theater enthusiasts — you know, the people likely to be able to afford the hefty step-up premium Samsung is charging for this feature — have their universal remotes planted firmly in- or close at-hand when watching TV. If they’re anything like me, those viewers will quickly find that a thumb on the remote can do nearly everything Smart Interaction can do, and do it better and more efficiently.

Now playing:
Watch this:

Samsung Smart Interaction: hands-on with voice and gesture…

5:09

Smart Interaction is available on the PNE8000 series plasma (I tested the 60-incher for this hands-on; full review coming soon) as well as the UNES7500 and UNES8000 LED-based LCDs. The least expensive TV with the feature is the 51-inch PN51E8000 plasma ($1800). No other TV manufacturer offers voice and gesture recognition, although Microsoft’s Xbox 360 Kinect features like Bing Video Search come closest to the experience. A few of LG’s 2012 sets–those with “4 mode” remotes–do support voice control, but I haven’t tested them yet.

What is Smart Interaction?
In addition to the built-in camera and microphone along the top frame of the TV, Smart Interaction TVs come with a Bluetooth-to-IR blaster and two different remotes: a standard multibutton clicker and a new touch pad number with just a few flush buttons and another microphone. Samsung intends the latter to be your main controller for the TV, but in practice its touch pad was so unresponsive — and its buttons so sparse — that I ended up using the standard clicker whenever I could. I wish it had a mic, too, since it really helped accuracy during voice commands.

The software to handle voice and gesture control isn’t terrible, but I faced a steep learning curve and some frustrating design issues. To prevent normal conversation or movement in the room from triggering unwanted commands, I had to consciously enter voice and/or gesture command mode by saying “Hi, TV” or waving at the screen, respectively. If you forget to do so nothing happens. I ended up saying “Hi, TV” on a seemingly constant basis because the system, by design, exits gesture or voice mode rather quickly. I could never just say “volume up”; I had to say “Hi, TV…volume up.”

Once engaged, contextual prompts pop up to guide you to available voice commands (e,g., “TV power off,” “exit,” “search,” etc.) or gestures you can use. The general rule is that if the command isn’t visible, either along the bottom of the screen or in the More Commands menu, it won’t work. Voice/gesture control is available in most areas, including standard TV watching and the Smart Hub section (with all of the apps like Netflix, YouTube, etc). But with the exception of the browser and a few Samsung-specific apps, you can’t use voice/gesture control from within an app yet.

Voice control
The two main facets, voice and gesture, share some commands and can also work in concert. I’ll start with voice. It’s best used for powering up/down, muting sound, entering the Smart TV menu or activating a few apps, and as long as I kept my commands to the few available options (here’s a list), it usually worked.

That said, when speaking to the TV’s mic from the couch I often had to repeat commands, and any extra noise in the room was liable to interfere. The touch-pad remote’s mic was more accurate, but if I’m already holding a remote anyway, I’d prefer to press a button for most actions (aside from searches, see below) than use a voice command.

Voice controlled volume (on the TV only) and channel changes (via TV or cable box) are possible by saying “volume up” or “channel down” etc, but they happen one step at a time so it takes forever. Rapidly saying “volume up” caused the system to misinterpret me, launching “Skype” or “My mirror,” for example. The system would almost invariably misinterpret the number I said for direct channel access; “200” (whether I said “two, oh, oh” or “two, zero, zero” or “two hundred”) became something different seemingly every time. I was surprised to find that on my review sample many basic apps lacked commands; saying “Netflix” from the main Smart TV screen did nothing, for example.

Even when I made sure to stick to the onscreen prompts I ran into difficulties. Within an app, voice control options change; for example in Samsung’s custom Family Story app the only voice options are exit and TV power off; I couldn’t adjust volume on the video. Within the Web browser, saying “search” didn’t search for a Web page; instead it caused the browser to close, opened up a whole other “search all” application that hit all the of the TV’s services. The browser doesn’t allow you to enter a URL via voice yet. All of these issues combined to make using voice for all but the most basic commands pretty frustrating.

Voice search
The one situation where voice control could be potentially be easier than using a remote is during searches. On a TV that means skipping the on-screen keyboard used for entering keyword searches, and instead saying something like “Game of Thrones” directly at the TV.

Like many Smart TV suites, Samsung’s includes numerous video services you can search individually (but not via voice)–the only major omission is Amazon Instant. Samsung also offers a Your Video meta-search that hits a bunch of those services at once, including Netflix and Vudu but excluding Hulu Plus and your own cable/satellite TV listings. The built-in Web browser is better than any other TV-based browser, but still worse than the one on your laptop, tablet or phone (one of which is probably in the same room as the TV and just as easy to access).

I tested Samsung’s voice search with both Your Video and the Web browser. Check out the occasionally hilarious results here, but the short story is that I searched successfully for a video program about 41 percent of the time and a Web term about 56 percent of the time when I talked (sternly) to the TV. Using the touch-pad remote’s built-in mic improved accuracy to 74 percent and 90 percent, respectively.

Responses seemed a bit slower than on Kinect’s Bing Video Search, there were frequent “please wait” and “processing” notifications and a couple of outright crashes. Between these issues, the hit-or-miss success rates and the infrequency with which most people use video search or a the TV’s browser at all, I don’t consider voice search a good reason to get a TV with Smart Interaction. That’s especially true when virtual keyboard searches on many TVs and even cable boxes are made significantly less tedious by auto-fills based on common keywords.

Gesture control

Gesture control is activated by holding a hand up in front of the TV, which usually worked under brighter lighting but wasn’t tolerant of shadows. Again prompts pop up to show you what gestures work, and there’s just a few — namely moving your hand to guide a cursor around the screen and making a fist to make a selection.

The problem was, despite excellent lighting, my attempts to activate gesture control were often ignored and I ended up waving foolishly at the TV. When it did work, navigation was inexact and frustrating — think of a coarse version of a Wii-mote — and after a minute or so of it, my arm became tired. I guess that means gesture control is a good workout.

My fist-to-click didn’t register as often as it should have and I ended up flapping my hand open and closed repeatedly in an attempt to “click” an item on the screen. At this point, I seriously considered using my fist to do something else to the TV screen.

Again the Web browser might seem like a place where gesture could be useful, but between awkward-to-impossible scrolling, inconsistent registering of virtual “clicks” and balky responses that seemed like crashes until I “reset” the system by holding my hand up, useful isn’t the word I’d choose.

External device control via IR blaster

Samsung included a wireless blaster that accepts Bluetooth commands from the TV and translates them into infrared commands for your box or a Blu-ray player, but the company neglected to provide for control of an AV receiver or other external audio device. Among the high-end systems likely to house these TVs, that means voice and gesture control can’t control volume or mute. Moreover, the commands for cable and satellite boxes are exceedingly basic — just channel changes and power.

In short, don’t depend on Samsung’s IR blaster to provide nearly the level of external device control as even the cheapest Harmony remote. Until Samsung adds audio device control and more commands (“fast forward” and “30-second skip” for DVRs comes to mind), voice and gesture control of external devices is basically a no-show.

If you deign to employ your thumb, the Touch remote works with the blaster to enable significantly more functionality, for example menu access. It’s still not nearly as good as an actual universal remote, however.

Bottom line: Is it worth it?
If you’ve read this far, you can probably guess: no. The premium for Smart Interaction in Samsung’s line is at least $300 currently, but in my view it’s definitely not worth paying even $100 extra to get voice and gesture control, the touchpad remote and the IR blaster. A simple Harmony remote, for example, can do everything but Web browsing and (arguably) searching better and cheaper. If you really want to use the built-in browser, get Samsung’s $99 keyboard.

I’ll give it this: Smart Interaction does make a fun demo, especially when it misinterprets your speech. When another co-worker came by and realized that yes, the TV was actually reacting to my voice and gestures, he said “Wow, I feel like I’m living in the future.” And yes, the fact that it works at all is just as impressive from a technological standpoint as, say, Microsoft Kinect. But Samsung’s version doesn’t work nearly as well and until it does, owners of these TVs will be far happier keeping a tight grip on their universal remotes.