Windows 10: HoloLens facilitates computer vision research with Research Mode

Brink · Jun 18, 2018

Microsoft HoloLens is the world’s first self-contained holographic computer. Remarkably, in Research Mode, available in the newest release of Windows 10 for HoloLens, it’s also a potent computer vision research device. Application code can not only access video and audio streams but can also at the same time leverage the results of built-in computer vision algorithms such as SLAM (simultaneous localization and mapping) to obtain the motion of the device as well as the spatial-mapping algorithms to obtain 3D meshes of the environment. These capabilities are made possible by several built-in image sensors that complement the color video camera normally accessible to applications.

Specifically, HoloLens has four gray-scale environment tracking cameras and a depth camera to sense its environment and capture gestures of the user. As shown in Figure 1, two of the gray-scale cameras are configured as a stereo rig capturing the area in front of the device so that the absolute depth of tracked visual features can be determined through triangulation. Meanwhile the two additional gray-scale cameras help provide a wider field of view to keep track of features. These synchronized global-shutter cameras are significantly more light-sensitive than the color camera and can be used to capture images at a rate of up to 30 frames per second (FPS).

The depth camera uses active infrared (IR) illumination to determine depth through time-of-flight. The camera can operate in two modes. The first mode enables high-frequency (30 FPS) near-depth sensing, commonly used for hand tracking, while the other is used for lower-frequency (1-5 FPS) far-depth sensing, currently used by spatial mapping. In addition to depth, this camera also delivers actively illuminated IR images that can be valuable in their own right because they are illuminated from the HoloLens and reasonably unaffected by ambient light.

Figure 1 – The HoloLens additional built-in gray scale image sensors complement the color camera normally available to applications.

With the newest release of Windows 10 for HoloLens, researchers now have the option to enable Research Mode on their HoloLens devices to gain access to all these raw image sensors streams, shown in Figure 2. Researchers can still use the results of the built-in computer vision algorithms but can now also choose to use the raw sensor data for their own algorithms. The sensors’ streams can either be processed on device or transferred wirelessly to another PC or to the cloud for more computationally demanding tasks. This opens a wide range of new computer vision applications for HoloLens. In egocentric vision, HoloLens can be used to analyze the world from the perspective of a user wearing the device. For these applications, HoloLens abilities to visualize results of the algorithms in the 3D world in front of the user can be a key advantage. HoloLens sensing capabilities can also be very valuable for robotics where these can, for example, enable a robot to navigate its environment.

Figure 2 – A sample HoloLens application that displays any of the Research Mode streams in real time.

These new HoloLens capabilities will be demonstrated at a tutorial on June 19th, 2018, at the IEEE/CVF International Conference on Computer Vision and Pattern Recognition (CVPR) in Salt Lake City. The next generation HoloLens depth sensing capabilities, which will also be made available through Project Kinect for Azure, will also be demonstrated at this tutorial.

We hope to see you there!
Click to expand...

Source: Microsoft HoloLens facilitates computer vision research by providing access to raw image sensor streams with Research Mode - Microsoft Research

:)

Brink · Jun 19, 2018

How Microsoft researchers are moving beyond keyboard and mouse

Kfir Karmon imagines a world in which a person putting together a presentation can add a quote or move an image with a flick of the wrist instead of a click of a mouse.

Jamie Shotton envisions a future in which we can easily interact in virtual reality much like we do in actual reality, using our hands for small, sophisticated movements like picking up a tool, pushing a button or squeezing a soft object in front of us.

And Hrvoje Benko sees a way in which those types of advances could be combined with simple physical objects, such as a few buttons on a piece of wood, to recreate complex, immersive simulators – replacing expensive hardware that people use today for those purposes.

Microsoft researchers are looking at a number of ways in which technology can start to recognize detailed hand motion — and engineers can put those breakthroughs to use in a wide variety of fields.

The ultimate goal: Allowing us to interact with technology in more natural ways than ever before.

“How do we interact with things in the real world? Well, we pick them up, we touch them with our fingers, we manipulate them,” said Shotton, a principal researcher in computer vision at Microsoft’s Cambridge, UK, research lab. “We should be able to do exactly the same thing with virtual objects. We should be able to reach out and touch them.”

This kind of technology is still evolving. But the computer scientists and engineers who are working on these projects say they believe they are on the cusp of making hand and gesture recognition tools practical enough for mainstream use, much like many people now use speech recognition to dictate texts or computer vision to recognize faces in photos.

That’s a key step in Microsoft’s broader goal to provide more personal computing experiences by creating technology that can adapt to how people move, speak and see, rather than asking people to adapt to how computers work.

“If we can make vision work reliably, speech work reliably and gesture work reliably, then people designing things like TVs, coffee machines or any of the Internet of Things gadgets will have a range of interaction possibilities,” said Andrew Fitzgibbon, a principal researcher with the computer vision group at the UK lab.

That will be especially important as computing becomes more ubiquitous and increasingly anticipates our needs, as opposed to responding to our commands. To make these kinds of ambient computing systems truly work well, experts say, they must be able to combine all our senses, allowing us to easily communicate with gadgets using speech, vision and body language together – just like we do when communicating with each other...

[youtube]QTz1zQAnMcU[/youtube]

[youtube]Jwk4iwpeVRU[/youtube]
Click to expand...

Read more: Talking with your hands: How Microsoft researchers are moving beyond keyboard and mouse - Next at Microsoft

Brink · Jun 19, 2018

MSRC 2017 list of Top 100 security researchers

Security researchers play an essential role in Microsoft’s security strategy and are key to community-based defense. To show our appreciation for their hard work and partnership, each year at BlackHat North America, the Microsoft Security Response Center highlights contributions of these researchers through the list of “Top 100” security researchers reporting to Microsoft.

This list ranks security researchers reporting directly to Microsoft according to the quantity and quality of all reports for which we’ve issued fixes. While one criteria for the ranking is volume of reports a researcher has made, the severity and impact of the reports is very important to the ranking. Higher-impact issues carry more weight than lower-impact ones. While this list does not include security researchers who report to our partners ZDI and iDefense as we do not always have full information to recognize their efforts, we very much appreciate the partnership with ZDI and iDefense as they ensure that we know about any reports affecting Microsoft products.

Given the number of individuals reporting to Microsoft, anyone ranked among the Top 100 is among some of the top talent in the industry. Regardless of where security researchers are ranked in this list, we appreciate their active and ongoing participation with the Microsoft Security Response Center, and encourage new researchers to report potential vulnerabilities to us at L: secure@microsoft.com. We’re excited to see who’s going to be on the list next year.

MSRC team
Click to expand...

Source: The MSRC 2017 list of “Top 100” security researchers MSRC

btarunr · Jun 19, 2018

23 Universities and Institutions Added As NVIDIA CUDA Research/Teaching Centers

Today, NVIDIA welcomes 23 universities and research institutions to its family of CUDA Research Centers and CUDA Teaching Centers.

CUDA Teaching Centers empower tens-of-thousands of students each year with the knowledge and expertise to take advantage of GPUs and their parallel-processing capabilities. These centers receive teaching kits, textbooks, software licenses and NVIDIA CUDA architecture-enabled GPUs for teaching lab computers, as well as academic discounts for additional hardware.

CUDA Research Centers are recognized research institutions that embrace and utilize GPU computing across multiple research fields. They are at the forefront of some of the world’s most innovative scientific research. These centers have access to exclusive events with key researchers and academics, a designated NVIDIA technical liaison, and specialized online and in-person training sessions.

Here are some examples of CUDA-related work taking place at the newest centers:

Gdansk University of Technology (Poland)
The CUDA Research Center at Gdansk University of Technology focuses on computational electromagnetics and photonics, which involve numerical solution of Maxwell’s equations.

Indian Institute of Technology Bombay (India)
Researchers at the Indian Institute of Technology Bombay are leveraging the power of CUDA and GPUs for advanced rendering, image understanding, and performance-driven data structures. Their goals are to broaden GPU computing to other research frontiers and bring parallel programming into mainstream education.

Joanneum Research – Digital (Austria)
The DIGITAL – Institute for Information and Communication technologies is a leading international research partner and center of expertise in the area of information and communication technology. The Institute’s focus includes web and internet technologies, image, video and acoustic signal processing along with remote sensing, communication and navigation technologies. Several research groups within DIGITAL have successfully explored the potential of GPU computing for large-scale computer vision problems.

The other new CUDA Research Centers include:

Bergische Universität Wuppertal (Germany)

Friedrich-Alexander-University Erlangen-Nuremberg (Germany)

Kazan Federal University (Russia)

Middle East Technical University (Turkey)

University of Malaga (Spain)

The new CUDA Teaching Centers include:

CV Raman College of Engineering (India)

European University of Lefke (Turkey)

Friedrich-Alexander-Universität Erlangen-Nürnberg (Germany)

Perm State University (Russia)

Pontifical Catholic University (Peru)

Providence University (Taiwan)

Taiyuan University of Technology (China)

Texas A&M University-Kingsville (US)

Texas State University-San Marcos (US)

Universidade Federal Fluminense (Brazil)

Università degli Studi di Milano (Italy)

University of Colorado (US)

University of Newcastle (Australia)

University of Plymouth (UK)

For more information on NVIDIA research activities and these programs, please visit the NVResearch site.

Windows 10: HoloLens facilitates computer vision research with Research Mode

HoloLens facilitates computer vision research with Research Mode

HoloLens facilitates computer vision research with Research Mode

HoloLens facilitates computer vision research with Research Mode

HoloLens facilitates computer vision research with Research Mode - Similar Threads - HoloLens facilitates computer

Windows Research

Windows Research

Microsoft Research Network Speed Test

Microsoft Research Collage maker

First Steps in Hyper-V Research

Windows Defender Research

Windows Defender Research

Announcing Microsoft Research Open Data

Research only in the local computers