Windows 10: Creating a new lens on reality: The HoloLens System Design team

Brink · Apr 4, 2018

It was classified as top secret.

“The most difficult mechanical problem you will ever work on and it’s going to be similar to inventing and creating antigravity boots” is how it was described to Roy Riccomini, the first mechanical product designer hired for the project.

This thrill of a seemingly impossible task intrigued Roy, a 30-year product design veteran who developed iBook during his 10 year stint at Apple. In September 2012, roughly 20 others like him were also drawn in by the irresistible challenge to build the hardware for Microsoft’s first ever mixed-reality device, HoloLens.

So how did they build a holographic computer to wear on the head? Much like they built the hardware team itself: with true grit, from the ground up.

The team, replete with experienced engineers and designers from both startups and large companies in Silicon Valley, brought its creativity, talent, experience, and unique perspective to shape the foundation of the innovative HoloLens design.

‘Imagineering’ what the future will hold

Microsoft had a vision for HoloLens: to help people collaborate, communicate, create, and learn by providing new ways to visualize and interact with 3D data.

“Microsoft invests a lot of time thinking about how to empower the people who use our products,” said Scott Fullam, senior director of new technology integration. Scott has several startups, Jawbone, 2Wire, and Apple sprinkled throughout his almost 30 years working in Silicon Valley. “A large part of the design process is drawn from both personal inspiration and what our customers need—not only today, but imagining about what the future will hold, what will make their jobs better.”

The team had a never-before-attempted assignment: develop a lightweight, wearable, head-mounted, self-contained system that gives users a unique mixed-reality experience.

HoloLens is an immensely complicated product. Specialized components—multiple sensors, advanced optics, and a custom holographic processing unit—all work together to birth a hologram with no external cameras, no wires, no phone, and no connection to a PC needed.

“It’s no easy feat for a human to see the real world, to place holographic objects alongside real world objects, and to walk around with a headset on without feeling sick,” said Patrick Codd, who leads the electrical engineering arm of the project. “The research that went into what makes humans nauseated, how to prevent that, and how to make images look sharp was not a trivial task.”

Patrick Codd, left, and Nagina Bhandary, right, try on two early prototypes of the HoloLens.

The team got to work and came up with several prototypes. The first iterations of HoloLens in the conceptual phase were bulky and intimidating.

“Trying to put something like that on your head is very intimidating,” said Roy. “The Microsoft team broke through tons of barriers, had tons of ideas for algorithms and actually made workable hardware that didn’t look nice and was plugged into the wall, but worked,” he said. “We were brought in to make it a shippable product.”

The product design and engineering teams joined forces to transform that experience.

There were so many iterations while engineering HoloLens that Michael Nikkhoo, senior director of Core Technology, got nearly 100 patents under his name alone for research related to HoloLens. “This shows the level of rigor we went through to come up with new ideas and incorporate them into the product design,” said the former NASA satellite engineer.

As an example, the team went to great lengths to make sure HoloLens could fit comfortably on a wide range of head sizes and shapes and take eyewear into account. Human bodies are complex. Some foreheads are angled, and the skin breathes and sweats.

It took months to come up with a design that put minimal pressure on sensitive veins and arteries in human foreheads and temples.

“We have traveled around the world to find out that there are only five or six companies that have this technology, but none of them can go down to sub 1 pound per square inch of pressure,” Michael said, who came to Microsoft via companies like Nortel, Cisco, and Apple. “This was a tall, challenging order.”

Another challenge was to create a durable device, light yet strong. Inside the device is an optical mount sitting on a chassis, head tracking, inertial systems, battery, CPUs, and GPUs (a full Windows 10 computer).

Building a team of Silicon Valley technical talent

The HoloLens Silicon Valley team was carefully selected; they had to be willing to jump in and work in startup mode: pick up a broad spectrum of activities, scrape ideas together from very little, traverse gray areas, and test, test, test—all to make technology that had never been made before.

“In a lot of other companies, you’re mostly taking evolutionary small steps based on previous product design. We had to start from scratch,” Patrick said.

These engineers also had to be skilled in taking a product from concept to mass production. The hiring managers for HoloLens were tasked with finding engineers with vertically integrated product experience, a technical talent especially prevalent in Silicon Valley, one that means a person has the ability and experience to take a project into mass production, from design all the way through to manufacturing and shipping to a customer. Roy said that trying to find engineers with that experience is really hard, but it’s a skill he learned during his time at Apple.

“There is a lot of talent that we can find in Silicon Valley itself, being a hub for a lot of technological research, work, and talent. A large part of the system design team is from Silicon Valley—due to the talent pool available here,” said Nagina Bhandary, director for system validation on HoloLens. “Plus, we have other hardware teams, like the Silicon team here, and it makes it very easy to collaborate.”

The HoloLens System Design team works across hardware research, firmware development, board design, electrical engineering, mechanical engineering, and software development.

Members of the Silicon Valley HoloLens leadership team, from left to right: Patrick Codd, Michael Nikkhoo, Roy Riccomini, Scott Fullam, Rune Jensen, and Nagina Bhandary.

While some technology companies take a siloed approach when developing a new product, the HoloLens team pulled everyone in early, during weekly design reviews, to seek input from teams working on all parts of the product.

“That is definitely a lot different from some of the cultures I have worked in in the past,” said Roy.

On multiple occasions, Patrick remembers having to rip up an entire functioning electrical design to make it work for the rest of the system.

The high-energy Rune Jensen, right, is at the helm of the Silicon Valley HoloLens team.

“If I only had to worry only about the electrical engineering, it would be a no brainer,” Patrick said, a former iPhone and iPad developer with Apple. But the team focused instead on a holistic, cross-functional development approach.

Rune Jensen, GM, HoloLens System Design, and the rest of the managers strive to create an environment where everyone can find deep meaning in his or her work and is motivated to tackle everyday challenges and opportunities.

“The culture we have is that we give everybody an opportunity to grow their skill set and their career,” said Nagina, who began her career at Microsoft 12 years ago with a background in chip verification. When the opportunity arose to move from working on the chips that went into HoloLens to working on systems validation, she jumped at the chance to learn something new.

“There’s a lot of investigation, trial, and error, and we encourage people to take those risks for experimentation,” she said.

The team has demonstrated a willingness to learn from mistakes and stand back up after a difficult outcome, ready to meet future challenges.

Building the device itself was a big risk. Would people use it in their everyday lives? Would it transform education, work, and play like imagined?

“I’m very excited about the risks that Microsoft is willing to take, and I think all the engineers here are as well,” said Patrick. “Microsoft looks way ahead—10, 20, maybe even 30 years ahead sometimes.”

Roy Riccomini shows how HoloLens can be integrated into the workplace, part of the driving vision behind the HoloLens movement.

That vision has paid off.

Today, Japan Airlines uses HoloLens to allow trainee engineers to view working holograms of jet engines without having to visit a hangar or take an expensive working component out of service. Volvo uses it to allow customers to view modifications on a 3D model of the car, as well as for internal training purposes. Trimble, a software developer for architecture firms, uses the device to help visualize buildings in 3D. Ford is reinventing the way they do product design by blending 3D holograms digitally with both clay models and physical production vehicles. And at Case Western University, students take advantage of HoloLens to examine full-scale human bodies without the need for cadavers.

“Once you’ve done all your design work,” said Scott, “and you’ve spent months and months and months collaborating with the software teams, the big a-ha moment is seeing the final hologram on the display. There’s nothing like it.”

Lead photo: Members of the Silicon Valley HoloLens team.
Click to expand...

Source: Creating a new lens on reality: The HoloLens System Design team | Microsoft Bay Area

:)

LCRT · Apr 4, 2018

hololens powered by the 950 and 950xl

you don't seem to understand what hololens is, and why it can't be done with a phone.

hololens isn't "virtual reality", it is "augmented reality". it is not comparable to any of the other systems you might be thinking of.

LCRT · Apr 4, 2018

hololens powered by the 950 and 950xl

if you understood it, you wouldn't mention hololens at all. it has nothing to do with what you are talking about.

it is not an affordable version of hololens - because it is NOT the same type of tech. you clearly do not understand the difference.

HOLOLENS is not the same type of product. do not associate VR using a phone with an augmented reality headset. they're not related at all.

Windows 10: Creating a new lens on reality: The HoloLens System Design team

Creating a new lens on reality: The HoloLens System Design team

Creating a new lens on reality: The HoloLens System Design team

Creating a new lens on reality: The HoloLens System Design team - Similar Threads - Creating lens reality

Firefox Reality for HoloLens 2 now available in Microsoft Store

Mozilla Bringing Firefox Reality to HoloLens 2

New Fluent Design Navigation for Visual Studio Team Services

Mixed Reality momentum continues and Microsoft HoloLens expands

HoloLens welcomes six new countries to the world of mixed reality

Computer Aided Design (CAD) enters the era of mixed reality

Ford brings Microsoft HoloLens to Design Studio

Trimble brings mixed reality to life for architecture and design

Microsoft HoloLens in space: Making science fiction (mixed) reality