When an end user with a HIT-enabled “Smart Phone” or mobile device is located near a HIT enabled display, the mobile HIT application establishes 2-way communication with Hand Eye software on the local HIT server.  The HIT server can be either a Set Top Box (STB) controlling the TV, or in the case of a HIT-enabled collaborative application, a local server/PC controlling the video display, projector, etc.

After a connection is established (via Local Area Network or the wireless carrier’s 3G network, etc.) and the Smart Phone/mobile device is registered, interactivity using the HIT platform can begin.

HIT uses the camera on a mobile device as a targeting system, in the same way that the mobile device’s camera would be employed to take a picture.  Hand Eye “sees what the user sees,” enabling highlighting and manipulation of on-screen objects, via an easy-to-use visual interface.

When the end user highlights or selects a HIT-enabled object – a “HotSpot” – on screen, messages or images are sent to the HIT server.  The HIT server reads the relevant XML data that has been encoded (via HotSpot) in the Interactive TV programming or HIT-enabled collaborative application.  For Interactive TV use, HotSpots can be embedded in standard or High-Definition (HDTV) broadcasts, DVR playback or DVD content.

The HIT server then interprets messages and/or images sent by the mobile device.  Using Hand Eye’s image processing technology, the HIT server responds to the mobile device with data defined (the HotSpot) by the screen location.  Alternatively, the server may respond with a menu/list of all available HotSpots allowing the user to directly choose the preferred HotSpot.  The menu/list method of identifying HotSpots is effective for Smart Phones that do not have a camera, such as certain models of the RIM Blackberry, or for basic mobile devices that lack a camera feature.

Based on the data definitions within the specific HotSpot, the HIT system can direct the Smart Phone/mobile device to take various actions, as defined by the message data.

These actions can include:

Manipulating objects on screen: Select (text, objects) Drag-and-drop Insert/Delete Inputting text or annotations Drawing on screen Accomplishing specific e-Commerce tasks: Click To Buy Click To Search Click To Dial Click To Download

Hand Eye’s drawing application (Video Graffiti) does not require XML data.  Instead, the HIT system enables drawing directly on-screen via tracing the movement of the camera, and displaying those movements as “drawn” lines on the screen.