Windows 10: Microsoft Introduces Web APIs for Dual Screen and Foldable Devices

Brink · Sep 14, 2020

The browser is one of the most used apps on any device, and today itâs running on a completely new hardware category: dual-screen and foldable devices.

With a variety of new dual-screen and foldable devices coming to market, including last weekâs launch of the new Surface Duo, thereâs never been a better time to start thinking about how your site can embrace these form factors.

Today, we are pleased to announce the availability of two new experimental features that will help web developers effectively lay out content in a browser window that spans multiple display regions and create responsive websites that are a natural fit for this new class of device:

CSS screen-spanning media feature and a set of environment variables to describe the geometry of the fold.

JavaScript window segments enumeration API, a companion API to the CSS primitives, useful when working with non-DOM targets like Canvas2d and WebGL.

The foldable device class

Broadly speaking, there are two variants of foldable devices: dual-screen devices, and single-screen devices that take advantage of the flexible display technology. Both have a lot in common: they are portable, multi-posture devices that allow users to rotate, flip and fold.

On this form factor, applications can reside on one side, or can be spanned across both display regions. A website that responds to this spanned state integrates with the semantics and intent of logically splitting the presented content.

The expansive screen real-estate and unique postures enabled by this device category allow web developers to unlock unprecedented experiences on the web in a device that can fit in your pocket or purse.

Transitioning from traditional contiguous screens to dual-screens and foldables

Although existing websites will continue to work out-of-the-box, making sites aware of the foldable nature of the device can greatly enhance the user experience.

To better illustrate the opportunity and demonstrate how the newly created browser capabilities work, weâll take you through enhancing an example e-mail client layout.

Showing a list-view of the inbox and the content of an email side-by-side is a common pattern that is naturally great for wider viewing areas. When the browser window is spanned across the two display regions on a dual-screen device, the overall viewport width is likely to be comparable to that of a traditional landscape tablet device.

Unmodified, the email client will continue to work as it always has. However, the experience could be dramatically improved if we could align the inbox and the email columns with the fold, keeping each within the boundaries of a single display region. That way, neither content area will be cut or masked by the device hinge or rendered over the fold of a flexible display.

To achieve the desired layout, we are introducing a new screen-spanning media feature and a set of pre-defined environment variables that allow web developers to treat foldable devices as another responsive web design target. Developers can now create layouts that adapt to every device category without taking a hard dependency on specific hardware parameters. This flexibility improves scalability because it does not require duplication of work for each new device type.

Detecting display regions

The CSS screen-spanning media feature will help web developers test if the root viewport is being spanned across multiple adjacent display regions, and provide details about the configuration of these adjacent display regions (e.g. stacked or side-by-side).

Syntax

The screen-spanning media feature is specified as a value describing the number of folds (or hinges) the device has, as well as their postures. If the device is not a foldable device, the value is none. If it is a foldable, it can have one of these two values:

Single-fold-vertical: matches a device with a single fold (two display regions) and the fold posture is vertical.

Single-fold-horizontal: matches a device with a single fold (two display regions) and the fold posture is horizontal.

Calculating display regions geometry

Itâs not safe to assume that when in the screen-spanning state, the fold is always going to divide the viewport exactly in half. Additionally, some window managers may choose to mask the web content behind the fold. To help web developers calculate the size of each display region and ensure they know how much their content, if any, needs to be padded to avoid the mask, we are adding four predefined CSS environment variables.

env(fold-top)

env(fold-left)

env(fold-width)

env(fold-height)

The values of these variables are expressed in CSS pixels and are relative to the layout viewport (i.e. are in the client coordinates, as defined by CSSOM Views). When evaluated in content that is not in one of the spanning states, these values will be treated as if they donât exist, and the browser will use the fallback value as passed to the env() function.

Enhancing our e-mail example app for dual-screen and foldable experiences

Letâs put the CSS screen-spanning media feature and the fold geometry environment variables into practice and enhance the reader view of our email client:

Code:

@media screen and (min-width: 799px) {
/* rules specific to screens that are wider than tablet */
}

@media screen and (min-width: 799px) and (screen-spanning: single-fold-vertical) {
/* main is an element wrapping the 3 flex items highlighted in the figure above */
main {
display: flex;
flex-direction: row;
}
.navigation {
/*
** flex direction is row, so flex-basis acts like this flex item width
** according to the design, the desired width on foldables / dual-screens is 60px
*/
flex-basis: 60px;

flex-grow: 0;
flex-shrink: 0;
}

.inbox {
/*
** inbox width consumes the entire width of the first display region, ex
** inbox width = display-region-1-width - 60px (navigation column width)
*/
flex-basis: calc( env(fold-left) - 60px );

/*
** some devices have a mask, so we need to add a margin or a gap after this column
** env(fold-width) = 28 CSS-pixels on surface Duo.
** env(fold-width) = 0 CSS-pixels on devices that does not mask content.
*/
margin-inline-end: env(fold-width);

flex-grow: 0;
flex-shrink: 0;
}

.email-content {
/*
** the email content column should "grow" to fill the rest of the space
** but to demonstrate how to calculate the width of the 2nd display region
** we will manually set the width
*/
flex-basis: calc( 100vw - (env(fold-left) + env(fold-width)) );

flex-grow: 0;
flex-shrink: 0;
}

}
Click to expand...

Enumerating the window segments in JavaScript

When working with non-DOM targets like Canvas2d or WebGL, you can get the geometry of each display region using the new Window Segments Enumeration API.

getWindowSegments() is a method on the Window object that returns an array of 1 or more DOMRects representing the geometry and position of each display region.

The returned array is an immutable snapshot of the display regionsâ state at the time the method was called. If the user transitions from a spanning state to a non-spanning state, or rotates the device, the window segments previously retrieved will be invalid.

Code:

const segments = window.getWindowSegments();

// case 1: desktops, traditional touch screen devices, foldable device not spanned
console.log(segments.length) // 1

// case 2: dual-screen and foldable
console.log(segments.length) // 2
Click to expand...

Pages should listen to window resize event or orientationchange event to detect whether the browser was resized, or the device was rotated and retrieve the updated display regions.

Code:

let segments = window.getWindowSegments();

// state 1: browser is spanned across 2 displays and fold is vertical.
console.log(segments.length); // 2

// state 2: user decided to rotate the device, browser is still spanned but fold is now *horizontal*
// on window resize, both resize and orientationchange events fire
// resize events will also fire when user go in or out of spanned state.
window.addEventListener('resize', () => {
// segments we initially retrieved are no longer
// update with latest information representing segments the 2 when the fold is horizontal
segments = window.getWindowSegments();
});
Click to expand...

There are no explicit methods to learn whether the fold posture is vertical or horizontal, as this information can easily calculated from the DOMRects returned:

Code:

function isSingleFoldHorizontal() {
const segments = window.getWindowSegments();

// single fold means the device has 1 fold and 2 display regions
if( segments.length !== 2 ) {
return false;
}

// horizontal fold means 1st segment top is smaller than 2nd segment top
if( segments[0].top < segments[1].top ) {
return true;
}

// if we reach this point, the fold is vertical
return false;
}
Click to expand...

Same applies for the fold-width, web developers can use information provided by getWindowSegments() to learn whether the window manager is masking content rendered behind the fold, and the fold width is greater than zero pixels, or not.

Code:

function foldWidth() {
const segments = window.getWindowSegments();

// if there's 1 segment, fold mask is not applicable, return 0
// if there's more than 2 segments, we don't deal with this kind of device, yet, return 0
if( segments.length !== 2 ) {
return 0;
}

// fold is vertical
// device looks like this: [][]
if( segments[0].top === segments[1].top ) {
return segments[1].left - segments[0].right;
}

// if we reach this point, the fold is horizontal
return segments[1].top - segments[0].bottom;
}
Click to expand...

Future-proofing

As developers, we tend to plan for the future when creating for today, so a minimum amount of refactoring is needed to unlock scenarios that are likely to arise down the road.

An imaginary device with 2 folds and 3 screens

Unlike CSS, JavaScript has the concepts of arrays, loops and conditionals which make the mapping between the Window Segments Enumeration API and a device with N display regions more direct. For the imaginary device illustrated above, when the browser is spanned across all 3 display regions, calling the getWindowSegments() method will return an array of 3 DOMRects and using simple language primitives like a loop or the built-in array methods you could learn more about how the display regions are configured (e.g. are all screen of the same width?, etc.)

In CSS, the current plan is to simply add new values to the screen-spanning media feature representing new screen topologies.

Start enhancing your websites for foldable experiences today

The CSS screen-spanning media feature and the Window Segment Enumeration API are available behind an experimental flag, you can enable them at edge://flags/#enable-experimental-web-platform-features.

Starting in Microsoft Edge 86, web developers can use Microsoft Edge DevTools to emulate dual-screen & foldable devices on Windows and Mac desktop platforms. Alternatively, you can download and install the new Surface Duo emulator preview (version 2020.806.1 or newer) to test and debug using the built-in Edge browser after enabling experimental platform features flag.

Both the JavaScript Window Segments Enumeration API and the CSS screen-spanning media features are available as Origin Trials, where you can acquire tokens and safely experiment with these new primitives in production in exchange for providing us with feedback about the APIs. Sign up for an Origin Trial if youâre interested in testing out these APIs!

The road ahead

These APIs are available for you to experiment with today after many iterations and improvements with the collaboration of the Chromium project, Google, Intel, W3Câs CSSWG, Second-screen WG and many others.

We have contributed both the CSS and JavaScript primitives for desktop platforms to the Chromium open-source project and now the DevToolsâ foldable and dual-screen devices emulation is available not only in Edge, but also in Chrome and soon in other Chromium-based browsers. We are currently working on upstreaming the Android implementation so all Chromium-based browsers on Android OS can support web developers delivering exciting new experiences for this flexible device category.

If you have feedback about either one of the APIs, please let us know by opening an issue on GitHub or alternatively you can reach out to us on Twitter (@_zouhir or @MSEdgeDev).

â Zouhir Chahoud, Program Manager
â Daniel Libby, Principal Software Engineering Manager
Click to expand...

Source: https://blogs.windows.com/msedgedev/...able-web-apis/

Tweet

— Twitter API (@user) View on Twitter
Click to expand...

:)

Brink · Sep 14, 2020

Developing new category of dual-screen devices for mobile productivity

Last month we shared our vision for dual-screen devices, designed to help people get more done on smaller and more mobile form factors. Today, we are going to share how developers can unlock this new era of mobile creativity. There are two stages to optimize for dual-screen devices:

1) Your websites and apps work

Your code is important, and you will not have to start anew on these devices. Our goal is to make it as easy as possible for your existing websites and apps to work well on dual-screen devices.

Windows 10X is an expression of Windows 10 and will be available on dual-screen and foldable PCs, including the Surface Neo and devices from several partners. Developers will be able to use existing investments and tools for Web, UWP, and Win32 on these devices.

The Surface Duo will bring together Android apps, OS, and Surface hardware. Your current websites and Android apps will continue to work and run on a single screen. You can also stay in your current workflow and continue to use the same tools you do now.

2) Embrace dual-screen experiences: introducing a common model

The excitement for this new device category creates a great opportunity for developers to innovate and reach new customers – enabling them to be more productive and engaged while on-the-go. We are in the process of identifying key postures and layouts across dual-screen and foldable PCs so that you can take advantage of both.

For native app developers, our goal is to develop a common model layered onto existing platform-specific tools and frameworks for Windows and Android. Of course, APIs to access this model will be tailored to the developer platform for each operating system. For example, you can use APIs to enhance your apps to use dual-screen capabilities and features like the 360-degree hinge.

Web will continue to follow the standards-based model. And we are committed to building the right web standards and APIs to allow web developers to take advantage of cross-platform dual-screen capabilities. Web developers can use the browser or web-based app model of their choosing to take advantage of these capabilities.

Early access

We are excited to start working with developers, and for those who want to adopt early please reach out to L: [email protected] to learn more. Thank you for your continued support and interest in this new device category. We cannot wait to share more details with developers in early 2020.
Click to expand...

Source: Developing for the new category of dual-screen devices built for mobile productivity - Windows Developer Blog

How to Install Windows 10X Dual Screen Emulator in Windows 10

Tweet

— Twitter API (@user) View on Twitter
Click to expand...

Monica Cam · Sep 14, 2020

BT pairing code for Microsoft Foldable Keyboard not displayed

Hi Adrian,

We would like to know more about your concern. Kindly provide us more details by answering the following questions:

Has the Microsoft Foldable Keyboard in Windows 10 computer ever worked?

What specific troubleshooting have you done so far?

Meanwhile, to help resolve your concern, we suggest that you run the Hardware and Devices troubleshooter. To do this, refer to these steps:

On the Start menu, type Troubleshooting, and then hit
Enter.

.Choose Hardware and Sound.

Select and run the Hardware and Devices troubleshooter.

You can also refer to the steps provided in this article on how to set up a Bluetooth keyboard in Windows 10.

Should you have further questions, let us know.

Brink · Sep 14, 2020

Announcing dual-screen preview SDK for Microsoft Surface Duo

In November, we shared our vision for dual-screen devices and how this new device category will help people get more done on smaller and more mobile form factors. Today, we are excited to give you an update on how you can get started and optimize for dual-screen devices by:

Exploring preview SDKs and standards proposals for apps and websites

Embracing dual-screen experiences

Learning more at Microsoft 365 Developer Day

1) Exploring preview SDKs and standards proposals for apps and websites

We are happy to announce the availability of the preview SDK for Microsoft Surface Duo, and availability in the coming weeks for the preview SDK for Windows 10. We are also excited to announce new web standards proposals to enable dual-screen experiences for websites and PWAs on both Android and Windows 10X. These new web standards proposals will provide you with the capabilities and tools you need for dual-screen devices.

Download the preview SDK for Microsoft Surface Duo

Today, developers can download the preview SDK for Surface Duo, access documentation and samples for best practices, see UX design patterns, and more. The preview SDK gives developers a first look at how you can take advantage of dual-screen experiences.

This includes:

Native Java APIs to support dual-screen development for the Surface Duo device, including the DisplayMask API, Hinge Angle Sensor, and new device capabilities.

An Android Emulator with a preview Surface Duo image that is integrated into Android Studio so you can test your app without a physical device. The emulator simulates postures, gestures, hinge angle, mimicking the seam between the two screens, and more. We’ll continue to add functionality over time.

Requirements: For the Android Studio and Android Emulator.

We will have more announcements and discussion in the coming months and look forward to hearing your feedback.

Figure 1: The Android Emulator with a preview Surface Duo image

An early look at developing for Windows 10X

In the coming weeks, developers will have access to a pre-release version of the Windows SDK through the standard Insider builds. Our intent is to provide you with the Microsoft® Emulator on February 11th as well as new APIs for dual-screen support, documentation, and code samples.

This includes:

Native Windows APIs for dual-screen development to enable your app to span the two screens, detect the hinge position, and take advantage of Windows 10X.

Microsoft Emulator is a dual-screen Hyper-V emulator so you can deploy your existing Universal Windows Platform (UWP) and Win32 apps and test in both single-and dual-screen scenarios. The emulator simulates the physical device so you can see how your apps interact with Windows 10X.

Requirements: A recent Windows Insiders preview build of 64-bit Windows 10 (Pro, Enterprise, or Education), 64bit CPU with 4 cores, 8GB minimum (16GB of RAM recommended), Hyper-V enabled and dedicated GPU that supports Direct X 11.0 or later.

Figure 2: Microsoft Emulator showing Windows 10X

Build dual-screen experiences on the web

The new Microsoft Edge released last week, provides a powerful and compatible foundation for website and web app experiences across devices, powered by Chromium. We are actively incubating new capabilities that enable web content to provide a great experience on dual-screen devices, whether it’s running in the browser or installed as an app.

New web standards for dual-screen layout: We are proposing CSS primitives for dual-screen layouts and a JavaScript Window Segments Enumeration API to provide web platform primitives for web developers to detect multiple displays and lay out content across them. We expect to provide an experimental implementation of these features in preview builds of the browser soon.

Dual-screen polyfills: As the above features progress through the web standards process, we’ve published polyfills that you can write against as you begin to explore dual-screen development. You can find the polyfills and associated documentation at:

The JavaScript Window Segments enumeration polyfill enumerates segments (regions on separate displays) where logically separate pieces of content can be placed.

The CSS ‘spanning’ media feature polyfill allows you to lay out content in a window that spans multiple displays.

Progressive Web Apps are supported out of the box in the new Microsoft Edge, which can be installed directly from the browser on Windows 10X and Android. PWAs will support the same dual-screen layout features and tools as the browser.

We’ll have more to share about building for dual-screen devices with web technologies over the coming months – watch the Microsoft Edge blog for more details.

2) Embracing dual-screen experiences

Dual-screen devices creates an opportunity for your apps to delight people in a new and innovative way. To help you get started, we are providing you with basic support checklists for touch and pen and drag and drop and initial app pattern ideas to ensure your apps work great on dual-screen devices.

Figure 3: Dual-screen app patterns

Your app by default will occupy a single screen, but users can span the app to cover both screens when the device is in a double-portrait or double-landscape layout. You can programmatically enable full-screen mode for your app at any time, but spanning is limited to user activity for now.

Figure 4: Dual-screen orientation and layout.

For those who are interested in native cross-platform development using React Native or Xamarin.Forms, we are working on improvements to those frameworks and code samples. You can find the all dual-screen checklists, app patterns, and new code samples as they become available on our dual-screen documentation site. Please reach out to us at L: [email protected] so we can work with you to idealize and innovate great dual-screen experiences together.

3) Learning more at Microsoft 365 Developer Day – Dual-Screen Experiences

Please join us online for the Microsoft 365 Developer Day, focused on dual-screen experiences on Tuesday, February 11th at 8:30 AM PDT. The keynote and sessions will show how to:

Get the most out of these SDKs and emulators

Use cross platform tools and languages

Design apps for dual-screen devices

Build dual-screen experiences on the web

Connect your apps with Microsoft 365

We hope that you will join us, and we are excited to see what dual-screen experiences you build.
Click to expand...

Source: https://blogs.windows.com/windowsdev...developer-day/

How to Install Windows 10X Dual Screen Emulator in Windows 10

Tweet

— Twitter API (@user) View on Twitter
Click to expand...

Windows 10: Microsoft Introduces Web APIs for Dual Screen and Foldable Devices

Microsoft Introduces Web APIs for Dual Screen and Foldable Devices

Microsoft Introduces Web APIs for Dual Screen and Foldable Devices

Microsoft Introduces Web APIs for Dual Screen and Foldable Devices

Microsoft Introduces Web APIs for Dual Screen and Foldable Devices - Similar Threads - Microsoft Introduces Web

Introducing web capture for Microsoft Edge

Introducing a new and improved Twitter API

Introducing the Storage Access API for Microsoft Edge

New Intel Lakefield processors for dual screen and foldable devices

Introducing Web Authentication in Microsoft Edge

SDK reveals how apps could run on a dual-screen foldable Windows 10 device

Introducing Web Media Extension Package for Microsoft Edge

Introducing the WebVR API in Chrome for Android

Simpler web payments: Introducing the Payment Request API