Windows 10: Introducing NVIDIA Jarvis - Multimodal AI SDK

Brink · Oct 22, 2019

At Mobile World Congress Los Angeles, NVIDIA Co-founder and CEO Jensen Huang announced in his keynote – NVIDIA Jarvis – an SDK for building and deploying AI applications that fuse vision, speech, and other sensors into one system.

Introducing NVIDIA Jarvis – Multimodal AI SDK

NVIDIA Jarvis offers a comprehensive workflow to build, train and deploy AI systems that uses speech and visual cues such as gestures and gaze in context. For example, lip movement can be fused with speech input to identify the active speaker. Gaze can be used to understand if the speaker is engaging the AI agent or other people in the scene. This enables simultaneous multi-user, multi-context conversations with the AI system that need a deeper understanding of the context.

The new SDK provides several base modules for speech tasks such as such as intent and entity classification, sentiment analysis, dialog modeling, domain and fulfillment mapping.

For vision, modules include person detection and tracking, detection of key body landmarks and body pose, gestures, lip activity and gaze.

With Jarvis, you can also use custom modules or fine-tune to adapt for your use case.

For edge and IoT use cases, Jarvis runs on the NVIDIA EGX stack, which is compatible with all commercially available Kubernetes infrastructure.

Availability

Apply now for early access to learn more about the SDK and early availability here.
Click to expand...

Source: Announcing NVIDIA Jarvis, Combining Speech, Vision and Other Sensors into One AI Platform - NVIDIA Developer News CenterNVIDIA Developer News Center

Tweet

— Twitter API (@user) View on Twiiter
Click to expand...

:)

Brink · Oct 22, 2019

NVIDIA introduces open source PhysX SDK 4.0

NVIDIA PhysX, the most popular physics simulation engine on the planet, is going open source.

We’re doing this because physics simulation — long key to immersive games and entertainment — turns out to be more important than we ever thought.

Physics simulation dovetails with AI, robotics and computer vision, self-driving vehicles, and high-performance computing.

K1rotbzekf0

It’s foundational for so many different things we’ve decided to provide it to the world in an open source fashion.

Meanwhile, we’re building on more than a decade of continuous investment in this area to simulate the world with ever greater fidelity, with on-going research and development to meet the needs of those working in robotics and with autonomous vehicles.

Free, Open-Source, GPU-Accelerated

PhysX will now be the only free, open-source physics solution that takes advantage of GPU acceleration and can handle large virtual environments.

It will be available as open source starting Monday, Dec. 3, under the simple BSD-3 license.

PhysX solves some serious challenges.

In AI, researchers need synthetic data — artificial representations of the real world — to train data-hungry neural networks.

In robotics, researchers need to train robotic minds in environments that work like the real one.

For self-driving cars, PhysX allows vehicles to drive for millions of miles in simulators that duplicate real-world conditions.

In game development, canned animation doesn’t look organic and is time consuming to produce at a polished level.

In high-performance computing, physics simulations are being done on ever more powerful machines with ever greater levels of fidelity.

The list goes on.

PhysX SDK addresses these challenges with scalable, stable and accurate simulations. It’s widely compatible, and it’s now open source.

NVIDIA PhysX scales to large numbers of interacting bodies.

PhysX SDK is a scalable multi-platform game physics solution supporting a wide range of devices, from smartphones to high-end multicore CPUs and GPUs.

It’s already integrated into some of the most popular game engines, including Unreal Engine (versions 3 and 4) and Unity3D.

You can also find the full source code on GitHub. Dig in.
Click to expand...

Source: NVIDIA Extends PhysX for High Fidelity Simulations, Goes Open Source | The Official NVIDIA Blog

Tweet

— Twitter API (@user) View on Twiiter
Click to expand...

Brink · Oct 22, 2019

NVIDIA introduces open source PhysX SDK 4.0

NVIDIA PhysX, the most popular physics simulation engine on the planet, is going open source.

We’re doing this because physics simulation — long key to immersive games and entertainment — turns out to be more important than we ever thought.

Physics simulation dovetails with AI, robotics and computer vision, self-driving vehicles, and high-performance computing.

K1rotbzekf0

It’s foundational for so many different things we’ve decided to provide it to the world in an open source fashion.

Meanwhile, we’re building on more than a decade of continuous investment in this area to simulate the world with ever greater fidelity, with on-going research and development to meet the needs of those working in robotics and with autonomous vehicles.

Free, Open-Source, GPU-Accelerated

PhysX will now be the only free, open-source physics solution that takes advantage of GPU acceleration and can handle large virtual environments.

It will be available as open source starting Monday, Dec. 3, under the simple BSD-3 license.

PhysX solves some serious challenges.

In AI, researchers need synthetic data — artificial representations of the real world — to train data-hungry neural networks.

In robotics, researchers need to train robotic minds in environments that work like the real one.

For self-driving cars, PhysX allows vehicles to drive for millions of miles in simulators that duplicate real-world conditions.

In game development, canned animation doesn’t look organic and is time consuming to produce at a polished level.

In high-performance computing, physics simulations are being done on ever more powerful machines with ever greater levels of fidelity.

The list goes on.

PhysX SDK addresses these challenges with scalable, stable and accurate simulations. It’s widely compatible, and it’s now open source.

NVIDIA PhysX scales to large numbers of interacting bodies.

PhysX SDK is a scalable multi-platform game physics solution supporting a wide range of devices, from smartphones to high-end multicore CPUs and GPUs.

It’s already integrated into some of the most popular game engines, including Unreal Engine (versions 3 and 4) and Unity3D.

You can also find the full source code on GitHub. Dig in.
Click to expand...

Source: NVIDIA Extends PhysX for High Fidelity Simulations, Goes Open Source | The Official NVIDIA Blog

Tweet

— Twitter API (@user) View on Twiiter

Brink · Oct 22, 2019

NVIDIA RTX Broadcast Engine Brings Twitch Livestreams to Life with AI

Leading into TwitchCon — the world’s top gathering of livestreamers — we’re announcing the RTX Broadcast Engine, a new set of RTX-accelerated software development kits that use the AI capabilities of GeForce RTX GPUs to transform livestreams.

Powered by dedicated AI processors called Tensor Cores on RTX GPUs, the new SDKs enable virtual greenscreens, style filters and augmented reality effects — the kind of techniques used by major broadcast networks — all using AI and without the need for special equipment.

Livestreaming of video games has become a cultural phenomenon. Over 750 million people around the world tune in to watch people play video games. TwitchCon is where this global movement comes together. More than 50,000 streamers and fans will converge in San Diego this weekend to meet their favorite gamers and learn about the future of livestreaming.

RTX Brings AI to Livestreaming

NVIDIA GPUs are already the most popular choice to power the PC games played by streamers. They’re also used to encode and stream video to platforms such as Twitch, YouTube, Mixer, Huya and Douyu.

With the RTX Broadcast Engine’s AI-powered capabilities, NVIDIA is announcing a new way that RTX GPUs can enable more immersive livestreams — all without special cameras or physical props.

The new SDKs include:

RTX Greenscreen, to deliver real-time background removal of a webcam feed, so only your face and body show up on the livestream. The RTX Greenscreen AI model understands which part of an image is human and which is background, so gamers get the benefits of a greenscreen without needing to buy one.

RTX AR, which can detect faces, track facial features such as eyes and mouth, and even model the surface of a face, enabling real-time augmented reality effects using a standard web camera. Developers can use it to create fun, engaging AR effects, such as overlaying 3D content on a face or allowing a person to control 3D characters with their face.

RTX Style Filters, which use an AI technique called style transfer to transform the look and feel of a webcam feed based on the style of another image. With the press of a hotkey, you can style your video feed with your favorite painting or game art.

ZGxaAhIOgho

NVIDIA and OBS Bringing RTX Greenscreen to Gamers

In addition, we’re working with OBS, one of the leading livestreaming applications, to integrate RTX Greenscreen. With it, livestreamers will be able to remove their background environment or instantly teleport themselves anywhere — in this world or in virtual ones. This feature will be showcased at TwitchCon for the first time and available in the coming months.

“NVIDIA has been at the top of my list when it comes to streaming and recording equipment. I’m continually impressed with what they’re doing,” said Hugh Bailey, author, OBS. “And their technology is impressive with RTX features like RTX Greenscreen.”

Livestreaming Ecosystem Supports NVIDIA Broadcast SDKs

The RTX Broadcast Engine will enable streaming applications throughout the ecosystem to create immersive tools and effects for broadcasters to engage audiences and drive viewership.

“The new RTX Broadcast Engine is an exciting advancement that will allow developers in our app store to create powerful new tools for streamers with NVIDIA RTX GPUs,” said Ali Moiz, CEO of Streamlabs. “We’re thrilled to continue working with NVIDIA as they introduce new features to the Streamlabs developer community, and look forward to implementing this new technology.”

“We have collaborated with NVIDIA over the years on many projects and the introduction of the NVIDIA RTX Broadcast Engine is by far the most exciting,” said Miguel Molina, director of developer relations at XSplit. “For the XSplit team, we are excited to integrate these new tools into our suite of apps, enabling our users to create better content by maximizing the potential of NVIDIA GeForce RTX.”

In addition to RTX Broadcast Engine, leading applications such as OBS, XSplit, Huya, Douya and Streamlabs have deployed the NVIDIA Video Codec SDK for fast, high-quality streaming. Three new integrations made their debut this month:

Twitch Studio, a new, easy-to-use application for new livestreamers currently in beta, has integrated the Video Codec SDK to enable high-quality livestreaming.

Discord, the world’s leading gaming chat application, just released a new group broadcasting feature called “Go Live,” which uses NVIDIA GPUs and the Video Codec SDK to accelerate broadcasting games in Discord.

Elgato is one of the world’s leading manufacturers of video capture cards for gaming. It recently integrated the Video Codec SDK into the software of its new 4K60 Pro MK.2 capture card for recording 4K at 60fps video in High Dynamic Range.

Developers can learn more about the RTX Broadcast Engine and apply for early access at developer.nvidia.com/broadcastengine. Or stop by the OBS booth at TwitchCon, booth 1823, where we’ll be showing off RTX Greenscreen in OBS, new RTX Studio laptops and upcoming RTX games.
Click to expand...

Source: A Stream Come True: NVIDIA RTX Broadcast Engine Brings Twitch Livestreams to Life with AI | The Official NVIDIA Blog NVIDIA RTX Broadcast Engine Brings Twitch Livestreams to Life with AI

Windows 10: Introducing NVIDIA Jarvis - Multimodal AI SDK

Introducing NVIDIA Jarvis - Multimodal AI SDK

Introducing NVIDIA Jarvis - Multimodal AI SDK

Introducing NVIDIA Jarvis - Multimodal AI SDK

Introducing NVIDIA Jarvis - Multimodal AI SDK - Similar Threads - Introducing NVIDIA Jarvis

Introducing the next wave of AI at Scale innovations in Bing

NVIDIA introduces GeForce GTX 1650 Turing GPUs

CES 2019: NVIDIA introduces GeForce RTX Laptops

CES 2019: Introducing the NVIDIA GeForce RTX 2060

NVIDIA introduces open source PhysX SDK 4.0

NVIDIA Introduces GeForce GTX 1050 and 1050 Ti

Introducing NVIDIA TITAN V (Volta)

#MSIgnite 2017 - Introducing Dynamics 365 AI Solutions

NVIDIA Introduces GeForce GTX 1080