Windows 10: HoloLens Interaction Model

Brink · Jan 21, 2016

Before we dive into talking about the interaction model, we wanted to share that today is the day we open up Twitter voting to select the winner for the Share Your Idea campaign!

The Share Your Idea campaign has generated over 5000 ideas which are now available to the developer community as an idea bank. If you want to build a holographic app, but aren’t sure what you want to build, this is a great place to go to be inspired by the incredibly creative people who have chosen to participate.

With that out of the way, let’s dig in a little deeper into the ways that you can interact with HoloLens.
There are three key elements to input on HoloLens:

Gaze – What you’re looking at, and how you target

Gesture – An “air-tap” gesture that HoloLens will recognize, and which allows you to drive selection

Voice – Full access to the Windows 10 speech engine

Gaze is one of the most natural forms of interaction that we have. On HoloLens, at any time, the system knows where your head is in space, and what is directly in front of you. We leverage gaze to understand what you’re paying attention to at any given time, and to establish what both gesture and voice should be targeting.

As an example, in the Airquarium finalist, one of the core ideas is that you can look at anything and issue a tap to find out more about the animal that you selected. The way the system understands what to give facts and stats about when you issue a command is from gaze.

Gaze really is that simple, and that powerful. Of course, while you wear the HoloLens, you are generally moving around, so your gaze is also moving. The easiest way to think about it is as having a raycast from the device and which you can determine what object (real world as represented in the spatial mapping mesh or holographic) that ray intercepts with. Once you’re gazing at the animal, and you want to find out more, you need to tell the HoloLens to take an action. This brings us to gesture.

Gesture is the way to take a basic action in HoloLens. Simply raise your hand with your index finger raised, and tap down with your index finger. Now you can target (with gaze), and act (with gesture). We use these two primitives to build up all of the standard interactions that you would expect to have with a screen today, but now freed to interact with holograms embedded into your world.

Voice can really shine when you want to drive deeper interactions. These make voice a powerful input method for HoloLens:

Because HoloLens is a full Windows 10 device, the complete speech engine from Windows 10 is available to you as a developer.

Because the device is head mounted, and we actually understand where your mouth is, we have been able to build array microphones into the device which produce a very high quality audio signal for the purpose of doing speech recognition.

Because gaze is present, you actually have a better user context than is possible to attain for voice driven applications today. It is now possible to understand the object or area of interest which a voice command is intended to target. Because the device can provide this context, the user doesn’t need to try to preface each command with what they’re looking at, which will allow deeper, easier voice driven interactions than have been possible to date.

As an easy illustration, consider the Galaxy Explorer finalist. One of the core ideas is that at any time, you can navigate through the universe using natural voice commands.

Here you establish context (a planet or other feature) based on gaze, and then use voice to actually drive what you want to have happen. The user doesn’t need to explain that they’re targeting a particular world, because gaze is already telling us that. In this way, gaze will combine seamlessly with voice commands to allow you to explore the universe in a way you’ve never been able to before.

It is particularly satisfying to be able to open up voting on the anniversary of our original product announce, when we first got to share a bit about the amazing technology that we’re working on with the world. We’ve been blown away by the level of enthusiasm that all aspects of our program have received, and look forward to continuing to engage the community as we drive towards the launch of the Development Edition. If you’re interested in a trip down memory lane, you can see a bunch of the content that we’ve released over the last year here.
Click to expand...

Source: HoloLens Interaction Model | Building Apps for Windows

:)

up-paul-o-gy · Jan 21, 2016

Why???

I think Microsoft wants to have Windows as current as possible throughout the world, partly in an effort to make their new
Hololens (an augmented reality device) ubiquitous, a device which requires Win10.

Just as the internet, cell phones and smart phones each fundamentally
changed the way we communicate, interact, and get things done, Augmented Reality will do the same. People will be willing to pay for the Hololens (and its services), and I believe Microsoft will make a "killing" from it. In some ways, Hololens will likely replace
smart phones, laptops, and tablets (though they'll still have their uses).

Of course Microsoft will be profiting from this move, but I believe their intentions are also honorable. Because the Hololens is a 3D, interactive, augmented reality device, Microsoft's move can do
a lot to move all technology forward -- not just computer technology -- and more seamlessly connect each of us with information and our friends,
even far more than now!

To me, this move will totally bring us much closer to that envisioned utopian future that we've always dreamed about. Imagine having a 3-dimensional computer display all around you, everywhere you go! (Sometimes 2 dimensions just doesn't cut it.)

Ronnie Vernon · Jan 21, 2016

Can the hololense work in the real world?

Hello my name is Scott and I would like to be a tester for the hololens to see if it works in the real as in the outside world. To find out what they can and can't do.
Click to expand...

Hi Scott

The Hololens is not available to the general public yet.

The Hololens program that Microsoft is offering is for the developers who will be developing apps for the Hololens.

Go to the following website. Click the links on the Blue Menu Bar. Enjoy.

Hololens Main:

Microsoft HoloLens

Regards

Windows 10: HoloLens Interaction Model

HoloLens Interaction Model

HoloLens Interaction Model

HoloLens Interaction Model - Similar Threads - HoloLens Interaction Model

Interactive Shell

Hololens will not power on

Hololens will not power on

Hololens will not power on

Idea quantum hololens and monogame

Hololens

Hololens sensors driver Bug

keyboard interaction

This is Microsoft HoloLens