Windows 10: WebRTC UWP & callstats.io integration sample released

Brink · Jan 9, 2019

Early last year, we announced support for Real-Time Communications on the Universal Windows Platform based on a fork of Google’s WebRTC.org repo. The project enables native UWP developers to build Chrome compatible and feature equivalent RTC apps for all Windows 10 platforms, including Desktop, HoloLens, Xbox and ARM powered laptops. Since that time we’ve seen tremendous community interest in the project including 26K NuGet downloads and hundreds of developers reaching out on GitHub and L: email. We’ve also made numerous improvements to both WebRTC and ORTC UWP and announced our contribution of UWP support back to the Google WebRTC.org repo – more details will be shared in the coming months.

One thing we’ve heard consistently from developers is that they want better insight into the quality of WebRTC calls on their applications. This could be a call from HoloLens to a Desktop browser or a real-time video streaming application for Xbox. To that end, we have partnered with callstats.io to provide platform portable .NET integration for their WebRTC monitoring and analytics service!

Today we are announcing the release of a .NET Standard 2.0 NuGet package for callstats.io, along with a code sample based on the PeerCC UWP source. As a .NET Standard library it is available to all supported .NET implementations including the .NET Framework, .NET Core, UWP, Xamarin and Unity platforms (check the documentation for version requirements). The PeerConnection sample demonstrates a UWP application that establishes a 2-way video call and sends the WebRTC stats to the callstats.io service in 10 second intervals.

Here is an example dashboard showing statistics for the entire service:

Kognitiv Spark, a leader in mixed reality systems and a Windows 10 developer, is using the callstats.io service with its RemoteSpark application.
“Our operations team had faced challenges helping customers resolve network problems that can disrupt a mixed reality session,” said Ryan Groom, cofounder and CTO, Kognitiv Spark. “With native support in Windows 10, we were able to turn-up the callstats.io service in a few hours and begin using the rich data available in its dashboard to detect and expedite problem resolution.”
Let’s take a look at the steps for integration.

1) Create a callstats.io account.

Go to https://dashboard.callstats.io/register and create the Organization

Navigate to the apps tab to create an app and enter the SDK as other “UWP”

From the left side panel, click on “App Settings” and navigate to the “Security” tab

Click on “New credentials” and choose a label for the key

Select the key type as “ECDSA” and enter the public key

Under the key details, click on “view” for the credential

2) To authenticate with callstats.,io it is necessary to generate cryptographic keys and upload the public key to the service. Follow the instructions in the readme.md file here. After completing this step, you should have a Config class and .p12 certificate in your project. That’s all you need to authenticate to the callstats.io REST API.

3) Reference the Org.WebRtc.Callstats NuGet package available here in your project.

4) If you are using PeerCC sample you can find classes for collecting data from PeerCC and WebRTC in the Stats folder, you can also use this code as an example for your implementation. Add the files in the folder of PeerCC-Sample repo to your project:

PeerConnectionControllerStateChange.cs: Tracks RTCPeerConnection state changes and collects data from peer connection states for submission to callstats.

StatsController.cs: Collects data from the entire PeerCC sample and uses the Org.WebRtc.Callstats library to send prepared data to the callstats.io REST API.

WebRtcStats.cs: Prepares statistics obtained from RTCPeerConnection for communication to callstats.io via the REST API.

5) Initialize the callstats.io service and send SDP and application errors via the StatsController. For an example, see Conductor.cs in PeerCC-Sample.

6) Handle ICE state changes on RTCPeerConnection and use PeerConnectionStateChange to send data to the service:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28 _peerConnection.OnIceGatheringStateChange += async() =>

{

await PeerConnectionStateChange.StatsOnIceGatheringStateChange(_peerConnection);

Debug.WriteLine("Conductor: Ice connection state change, gathering-state=" + _peerConnection.IceGatheringState.ToString().ToLower());

};

_peerConnection.OnIceConnectionStateChange += async () =>

{

if (_peerConnection != null)

await PeerConnectionStateChange.StatsOnIceConnectionStateChange(_peerConnection);

else

{
await PeerConnectionStateChange.PeerConnectionClosedStateChange();

ClosePeerConnection();

}

};
7) Send application level events for audio muting and unmuting to StatsController. For an example, see MainViewModel.cs in PeerCC-Sample.

The sample should now be fully wired up to the callstats.io service and ready for testing. This is only the beginning of the possibilities for improving call quality, future work will focus on improving user experience with additional metrics and AI. We are excited to hear feedback from the community on uses for the library. Let us know what you think on GitHub or in email!
Click to expand...

Source: WebRTC UWP callstats.io integration sample released! - Windows Developer Blog

:)

Brink · Jan 9, 2019

Real-Time Communications on the UWP with WebRTC and ORTC

Readers of this blog interested in Real-Time Communications are probably familiar with Google’s WebRTC project. From the WebRTC site:

“WebRTC is a free, open project that provides browsers and mobile applications with Real-Time Communications (RTC) capabilities via simple APIs. The WebRTC components have been optimized to best serve this purpose.”

At Microsoft, we’ve seen tremendous support grow for WebRTC over the past five years. One of the most pivotal uses of WebRTC is building native video chat apps, which now reach more than one billion users.

Google’s native supported platforms for WebRTC include iOS, Android and traditional Win32 desktop apps. On Windows, Microsoft Edge already supports ORTC APIs and now supports WebRTC 1.0 APIs in Insider Preview builds on Desktop devices. For example, if you need to build a WebRTC app in HTML/JS targeted at desktop browsers or desktop web apps using the Web App Template, then Microsoft Edge and Windows web platform are a great choice.

But what if you want to write in C# or C++ and run WebRTC on Xbox, HoloLens, Surface Hub or Windows Phone, or write in HTML/JS and run on Raspberry Pi? What if you are using Google’s iOS and Android libraries and need bit-for-bit compatibility for your UWP application? What if you modify WebRTC source in your application and need to use those modifications in your Universal Windows Platform (UWP) application?

To fulfill these additional scenarios, we have ported and optimized WebRTC 1.0 for UWP. This is now available as an Open Source project on GitHub as well as in binary form as a NuGet package. The project is 100 percent compatible with Google’s source, enabling scenarios such as a WebRTC video call from Xbox running UWP to a Chrome browser on the Desktop.

WebRTC ChatterBox sample running as a native Windows 10 application.

Microsoft has also long been a supporter of the ORTC APIs and we work closely with the Open Peer Foundation to ensure optimal support of ORTC for UWP apps. ORTC is an evolution of the WebRTC API, which gives developers fine-grained control over the media and data transport channels, and uses a standard JSON format to describe peer capabilities rather than SDP, which is unique to WebRTC.

ORTC was designed with WebRTC interoperability in mind and all media is wire-compatible with WebRTC. ORTC also includes an adapter that converts SDP to JSON and exposes APIs that match WebRTC. Those two considerations make it possible for developers to migrate from WebRTC to ORTC at their own pace and enable video calls between WebRTC and ORTC clients. ORTC for UWP is available both as an Open Source project on GitHub as well as a NuGet package.

The net result of combined UWP and Edge support for WebRTC 1.0 and ORTC is that all Windows 10 platforms support RTC and developers can choose the solution they prefer.

Let’s take a look at an example from our samples repository on GitHub.

DataChannel via ORTC

The DataChannel, part of both the WebRTC and ORTC specs, is a method for two peers to exchange arbitrary data. This can be very useful in IoT applications – for example, a Raspberry Pi may collect sensor data and relay it to a Mobile or HoloLens peer in real-time. Keep in mind that while the sample code below uses ORTC APIs, the same scenario is possible via WebRTC.

To exchange messages between peers in ORTC, a few things must happen first (see MainPage.OpenDataChannel() in the sample code):

The peers must exchange ICE candidates, a successful pair of which will be used to establish a peer-to-peer connection.

The peers must exchange ICE parameters and start an ICE transport session – the underlying data path used for the peers to exchange data.

The peers must exchange DTLS parameters. which includes encryption certificate and fingerprint data used to establish a secure peer-to-peer connection, and start a DTLS transport session.

The peers must exchange SCTP capabilities and start an SCTP transport session. At this stage, a secure connection between the peers has been established and a DataChannel can be opened.

It’s important to understand two things about the above sequence. First, the data exchanges are in simple JSON, and as long as two peers can exchange strings, they can exchange all necessary data. Second, the identification of the peers and the exchange of these parameters, called signaling, is outside of the specification of ORTC and WebRTC by design. There are plenty of mechanisms available for signaling and we won’t go into them, but NFC, Bluetooth RFCOMM or a simple TCP socket server like the one included in the sample code, would suffice.

With the SCTP transport session established, the peers can open a Data Channel. The peer initiating the call creates an instance of an RTCDataChannel() passing the SCTP transport instance, and the remote peer receives the event RTCSctpTransport.OnDataChannel. When the remote peer receives this event, the Data Channel has been established and the peers can send messages to each other.

The code below is an excerpt from MainPage.Signaler_MessageFromPeer() in the sample code. The string message contains data received from the peer via the signaling method (in this case, the TCP socket server):
[table][tr][td]1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20[/td] [td]var sctpCaps = RTCSctpCapabilities.FromJsonString(message); if (!_isInitiator) { // The remote side will receive notification when the data channel is opened. Send SCTP capabilities back to the initiator and wait. _sctp.OnDataChannel += Sctp_OnDataChannel; _sctp.Start(sctpCaps); var caps = RTCSctpTransport.GetCapabilities(); _signaler.SendToPeer(peer.Id, caps.ToJsonString()); } else { // The initiator has received SCTP caps back from the remote peer, which means the remote peer has already // called _sctp.Start(). It's now safe to open a data channel, which will fire the Sctp.OnDataChannel event on the remote peer. _sctp.Start(sctpCaps); _dataChannel = new RTCDataChannel(_sctp, _dataChannelParams); _dataChannel.OnMessage += DataChannel_OnMessage; _dataChannel.OnError += DataChannel_OnError; }[/td] [/tr] [/table]

When the DataChannel has been established, the remote peer receives the OnDataChannel event. The parameter data for that event includes a secure DataChannel which is open and ready to send messages:
[table][tr][td]1 2 3 4 5 6 7 8[/td] [td]private void Sctp_OnDataChannel(RTCDataChannelEvent evt) { _dataChannel = evt.DataChannel; _dataChannel.OnMessage += DataChannel_OnMessage; _dataChannel.OnError += DataChannel_OnError; _dataChannel.SendMessage("Hello ORTC peer!"); }[/td] [/tr] [/table]

You can now freely exchange encrypted messages between the peers over the DataChannel. The signaling server is no longer required and that connection can be closed.

Real-time peer connectivity in Universal Windows Applications enables many exciting scenarios. We’ve seen developers use this technology to enable a remote peer to see what HoloLens users sees in real-time and interact with their 3D environment. Xbox developers have used the DataChannel to enable low-latency FPS style gaming. And one of our close collaborators, Blackboard, relies on the technology to stream classroom video feeds and enable collaboration in their Windows app. Check out our Universal Windows samples and the library source on GitHub – we look forward to your PRs!
Click to expand...

Source: https://blogs.windows.com/buildingap...m-webrtc-ortc/

Brink · Jan 9, 2019

UWP porting guidance for Android and iOS developers

Today, we are pleased to introduce a new app development concept map for app developers that aligns software development concepts across the iOS, Android, and Windows platforms. App developers new to Windows 10, who are already familiar with Android or iOS, will be able to use this reference to quickly orient themselves on building for the UWP.

Like the 2nd century B.C. Rosetta Stone tablet, which enabled translation between three different ancient languages, this Windows app concept map can be used to translate ideas between Android, iOS, and Windows, making porting much easier between platforms. You can use the concept map side-by-side with other useful resources found in our porting apps to Windows 10 section on MSDN. Reviewing this guide will help you port your apps to Windows and expose them to a larger user base.

Excerpt from the guidance document: a side-by-side map of data concepts in Android, iOS, and Windows.

The concept mapping guidance is broken out into 15 sections to make navigation between concepts and platforms simple:

User-interface – overview of controls and styling– overview of controls and styling

Navigation and app structure – toolbars, layouts and back buttons

Custom inputs – speech recognition and beyond

Data – databases, REST calls and web sockets

Tooling – comparing IDEs

App lifecycle – suspend and resume events as well as background tasks

Performance – threading, async calls and performance best practices

Monetization – in-app purchases and trials

Adapting to multiple platforms – responsive layout and device capability testing

Notifications – push and scheduled notifications as well as display widgets

Media capture and rendering – video and audio

Sensors – accessing raw and processed sensor data

Location and mapping – maps, geofencing and geocoding

App-to-app communication – talking between different apps

Software design – overview of preferred design patterns by platform

At a glance, you’ll be able to see the difference between platforms and identify which features and functionality are available to each. The guidance is laid out in a convenient table that let you easily locate and learn more about the platform details you are most interested in. Android supports app widgets, for example, but iOS does not. iOS supports badges, on the other hand, while Android does not. Windows supports badges and also has a tile system.

Beyond development concept mapping, it’s worth noting that the Dev Center provides a large array of additional resources to help app developers bring their existing code to Windows 10. Here are just a few links to help get you started:

What’s a Universal Windows Platform (UWP) app?

Getting Started with UWP for iOS developers

Introduction to UWP Design

Porting Apps to Windows 10

Bringing your mobile apps to the Universal Windows Platform (UWP) provides new reach for your existing code beyond mobile. For iOS developers, it’s also worth looking at the Windows Bridge for iOS, which makes it easier to use your existing Objective-C codebase on Windows.
Click to expand...

Source: UWP porting guidance for Android and iOS developers | Building Apps for Windows

Brink · Jan 9, 2019

UWP Experiences – App Samples

The UWP App Experiences are beautiful, cross-device, feature-rich and functional app samples built to demonstrate realistic app scenarios on the UWP platform across PC, Tablet, Xbox and more. Besides being open source on GitHub, each sample is accompanied by at least one blog post and short overview video, and will be published on the Windows Store in the upcoming month to provide easier access for developers.

The News Experience

( source | blog post | video )

Fourth Coffee is a news app that works across desktop, phone, and Xbox One, and offers a premium experience that takes advantage of each device’s strengths including tailored UI for each input modality such as controller on Xbox, touch on tablet and mouse on Desktop.

The Weather Experience

( source | blog post | video )

Atmosphere is a weather app that showcases the use of the popular Unity Engine to build beautiful UWP apps. In addition, the app implements UWP app extensions to enable other developers to extend certain areas of the app, as well as exposes an app service that enables other apps to use that weather information, as illustrated by Fourth Coffee.

The Music Experience

( source | blog post | video )

Backdrop is a cross-platform music app sharing code between UWP and other platforms using Xamarin. It supports background audio on UWP devices and cross-platform device collaboration using SignalR.

The Video Experience

( source | blog post | video )

South Ridge Video is a hosted web application built with React.js and hosted on a web server. The app can easily be converted to a UWP application that takes advantage of native platform capabilities, and can be distributed through the Windows Store as with any other UWP app.

The IoT Experience

( source | blog post | video )

Best For You is a fitness UWP app focused on collecting data from an IoT device using Windows IoT Core, Azure IoT Hub, Azure Event Hub and Azure Stream Analytics for processing.

The Social Experience

( source )

Adventure Works is a cross-device UWP application for sharing adventures and experiences with fictional friends. It is separated into three parts:

Camera and Cognitive Services Vision APIs ( blog post | video )

Project Rome and Social Integration ( blog post | video )

Speech and Inking ( blog post | video )

About the samples

These samples have been built and designed for multiple UWP devices and scenarios in mind from the start and are meant to showcase end to end solutions. Any developer can take advantage of these samples regardless of the device type or features they are targeting, and we are looking forward to hearing about your experience on the official GitHub repository.

Happy coding!
Click to expand...

Source: UWP Experiences – App Samples - Building Apps for Windows

Windows 10: WebRTC UWP & callstats.io integration sample released

WebRTC UWP & callstats.io integration sample released

WebRTC UWP & callstats.io integration sample released

WebRTC UWP & callstats.io integration sample released

WebRTC UWP & callstats.io integration sample released - Similar Threads - WebRTC UWP callstats

TPM: TSS.MSR the "NV UWP" sample Project.

Help with WebRTC

Apple releases iOS and iPadOS 13.4.1

Apple releases iOS and iPadOS 13.4

Real-Time Communications on the UWP with WebRTC and ORTC

GameAnalytics SDK for Microsoft UWP Released

UWP porting guidance for Android and iOS developers

UWP Experiences – App Samples

OOKLA releases new Speedtest app (UWP)