Windows 10: Windows 10 Memory Compression

Discus and support Windows 10 Memory Compression in Windows 10 News to solve the problem; Tweet — Twitter API (@user) date[/quote] With the announcement of the Memory compression feature, we have received a lot of queries from... Discussion in 'Windows 10 News' started by Brink, Aug 20, 2015.

Page 1 of 2

Brink New Member

Windows 10 Memory Compression

Tweet

— Twitter API (@user) date[/quote]

With the announcement of the Memory compression feature, we have received a lot of queries from the insiders asking for more information. This post tries to go in-depth into the inner working of the Memory Manager and the memory compression engine built within it.

One of the primary goals of any operating system is to manage and share resources efficiently. How well an operating system arbitrates competing memory requirements is particularly important, because it impacts the user’s perception of system responsiveness.

In Windows 10, we enabled the Memory Manager (MM) to compress infrequently-accessed memory pages which reduces the amount of reads and writes to the disk (pagefile) and therefore improves responsiveness. By compressing memory we reduce the amount of memory used per process, allowing the operating system to maintain more applications in physical memory at a time before enacting traditional paging.

When Memory Manager’s policy does dictate paging, the data being written to or read from the disk is already compressed to typically around 40 percent of its original size. This compression improves the performance of other unrelated operations on the system by reducing a major source of interfering IO load. The end result is a snappier, more responsive experience with improved app launch performance and consistency.

Before explaining the Windows 10 changes in detail, I’ll describe some of the fundamental concepts in Windows memory management and give an overview of how it was performed prior to implementing compression.

The first concept is a working set which is the actively-used physical memory of a process. A private working set represents physical memory that is unique to a process. Processes also touch other pages of physical memory that are “shared,” which several processes can reference. When you look at the Processes view in Task Manager, the memory for a specific process is actually its current private working set. Note that for simplicity when I refer to a “working set” in this post, I mean a “private working set.”

Some subset of a process’s working set pages are likely to be used only infrequently, and since memory is a limited resource that needs to be used efficiently, those pages must be reclaimed and made available for use by other processes. When MM decides a page has been inactive for long enough, it puts it on the modified list. An asynchronous process writes pages in the modified list to the pagefileon disk and then - in memory - moves them from the modified list to the standby list. This write happens so that if the memory is reclaimed from the standby list by another process, but the original process comes looking for its page, the data can be retrieved from the pagefile.

However, pages on the standby list haven’t yet been reclaimed, so they retain their data until MM repurposes them for another process’s working set. They can be reused by the app that previously owned them if that app asks for the page before it is reclaimed. This is called a soft fault. But if a page has been repurposed, and the original process asks MM for that page, a hard fault occurs. The original process is now asking for memory that has been taken by another process, and the OS must issue disk read IO to retrieve it.

As explained earlier, MM, when faced with memory pressure, determines which pages should be moved to the modified list. This is called trimming. But Windows 8 added the ability for Universal Windows Platform (UWP) applications to be swapped out in their entirety - i.e. at app rather than at page granularity. This is done because UWP apps are suspended when not in the foreground meaning they won’t be running until the user switches back them. No page will be touched until the app is brought back in its entirety, so it’s safe to swap out the whole application. The disk IO required to swap the application back in is then entirely sequential reads, which are dramatically faster than random reads. For more information about the swapping of UWP apps in Windows 8 MM, see this blog post by Bill Karagounis.

Windows 10 introduces a layer between the pagefile and normal paging activity which allows for the compression of all pages moved to the modified list. This means that when MM feels memory pressure, it will compress unused pages instead of writing them to disk. MM now only performs disk IO when compression isn’t enough to relieve memory pressure. Using this strategy, fewer disk operations are required since - thanks to compression - more data can be stored in memory than before.

This is achieved by using a concept called a compression store, which is an in-memory collection of compressed pages. There’s a single system store for legacy Windows desktop apps or any other non-UWP app memory usage, and then one store for each Windows Runtime app (in order to enable trimming and swapping accordingly). Instead of writing pages in the modified list to the pagefile, MM compresses them and places them in the appropriate store.

The stores live in the system process’s working set. Note that in the diagram above we’ve displayed the System Working Set separately from the other Working Sets for formatting reasons, but the System Working Set is a working set like any other. Though perhaps counterintuitive at first glance, since the system process holds the stores in memory, its working set grows larger exactly when memory is being made available to other processes. This is visible in the task manager.

First, observe the amount of memory used by the System process:

Then, start and suspend some applications:

And the System process’s working set will grow:

Each store manages fragmentation by compacting itself when necessary to ensure a good compression ratio. In practice, compressed memory takes up about 40% of the uncompressed size, and as a result on a typical device running a typical workload, Windows 10 writes pages out to disk only 50% as often as previous versions of the OS did. This will reduce the amount of time spent waiting for IO on all devices, but it will also improve the expected lifespan of systems featuring flash-based hard drives.

Though compression affords the OS more space in memory for a given application load, paging to disk may still be necessary. In Windows 10, this works by moving pages from the stores in the system process to the modified list. If pages come to the modified list from the system process stores, MM knows they’ve already been compressed and are ready to be written to the pagefile. The writes to disk will be smaller since the pages are necessarily compressed, i.e. the only pages MM will write to disk are ones which have been compressed and stashed in a store in the system process.

So now let’s talk about how a process gets its data back either from a compression store or from the pagefile on disk. In the former case where a process faults on a page which is currently in a compressed store, MM simply decompresses the page and puts it back in the requesting process’s working set - a soft compression fault. But in the latter case, MM tries the former and finds that the compressed data for the page is not in memory. The system process working set then hard faults on the compressed data for the page and MM reads it back in from disk. Decompression and adding to the faulting process’s working set follow.

Decompression is a highly parallelizable operation, so Windows can be efficient when moving multiple pages at once. Because of that, swapping UWP applications (as opposed to trimming) performs particularly well. But it’s not just the decompression: Windows combines that parallelizability with the sequential reads designed into the swapping procedure to read pages into memory extremely quickly; it reads a block at a time, and, while waiting for the next IO operation to complete, it’s simultaneously decompressing the data it just read in parallel using multiple CPUs. Where before a platform might have felt sluggish due to the transfer rate of its disk, running Windows 10 the same hardware appears much more responsive.

In summary, Memory compression allows Windows 10 to make better use of available memory on all systems by reducing physical memory utilization, resulting in fewer pagefile IO operations and a reduced reliance on disk. Furthermore, it improves the speed of writing to and reading from disk when MM determines paging must be enacted. In sum, we expect users to experience improved overall responsiveness and usability on all devices running Windows 10.

Thanks,
Ethan Creeger – Program Manager in the Base Platform team. I have been in Windows (and Microsoft) for two years work on the hardware aspects of the Kernel such as ACPI and PCIE.

:)
Click to expand...

Brink, Aug 20, 2015

#1
SIMONTEZZO

simontezzo Win User

System and compressed memory, messing up everything help please

I've tried all the suggestions listed, but the usage is still high. system
and compressed memory

simontezzo, Aug 20, 2015

#2
DENETTEBOYD-KING

DenetteBoyd-King Win User

System and compressed memory high CPU usage

hi, Adrian check this System and Compressed Memory Service High CPU Usage on Windows 10.

Answer to your question can be found on this page.

{FIX} System and Compressed Memory Service High CPU Usage on Windows 10. - wintips.org - Windows Tips & How-tos

Cheers!

DenetteBoyd-King, Aug 20, 2015

#3
ROCKY

Rocky Win User

Windows 10 Memory Compression

Thanks, that is a good read.

Rocky, Aug 20, 2015

#4
69MATRIX69

69matrix69 Win User

That;s interesting. I have come to love yet hate some of the new hidden items in win10

69matrix69, Aug 20, 2015

#5
ROCKY

Rocky Win User

That;s interesting. I have come to love yet hate some of the new hidden items in win10
Click to expand...

Welcome to the forum.

Rocky, Aug 20, 2015

#6
GARY

Gary Win User

Yes indeed thanks, a very interesting read. I hope it makes it to the RTM soon.

Gary, Aug 20, 2015

#7
LEE

Lee Win User

Windows 10 Memory Compression

. . .no big deal, along with no problems. . .*Smile

Lee, Aug 20, 2015

#8
DANIELSON

danielson Win User

Will this help Firefox to startup faster and be more performing on the web too?

danielson, Aug 20, 2015

#9
VICTEK

Victek Win User

"In summary, Memory compression allows Windows 10 to make better use of available memory on all systems by reducing physical memory utilization, resulting in fewer pagefile IO operations and a reduced reliance on disk. Furthermore, it improves the speed of writing to and reading from disk when MM determines paging must be enacted. In sum, we expect users to experience improved overall responsiveness and usability on all devices running Windows 10."

The universal "no free lunch" principal implies that there must be a trade-off here. I'm assuming that memory compression is going to add to CPU load ( ? ) Also I would think that the benefit of memory compression would be seen mostly on systems with less installed ram.

Now that this feature is available in the latest insider build I'm looking forward to third party reviews.

Victek, Aug 20, 2015

#10
TRAIJAN

Traijan Win User

Each store manages fragmentation by compacting itself when necessary to ensure a good compression ratio. In practice, compressed memory takes up about 40% of the uncompressed size, and as a result on a typical device running a typical workload, Windows 10 writes pages out to disk only 50% as often as previous versions of the OS did. This will reduce the amount of time spent waiting for IO on all devices, but it will also improve the expected lifespan of systems featuring flash-based hard drives.
Click to expand...

Isn't the average lifespan of flash based hard drives already something like 87 years?

Can we safely assume that if one has a lot of RAM (16GB or more) that this memory compression really isn't going to mean much to them?

Traijan, Aug 20, 2015

#11
DANIELSON

danielson Win User

I find their statement a bit ambiguous since it doesn't explicitly mention Windows 10 Preview.
They've introduced it to Windows 10, what else could it be except for Windows 10 or Windows 10 Preview?

Memory Manager Improvements:

In Windows 10, we have added a new concept in the Memory Manager called a compression store, which is an in-memory collection of compressed pages...
Announcing Windows 10 Insider Preview Build 10525 | Blogging Windows

danielson, Aug 20, 2015

#12
JAIRUS24

Jairus24 Win User

Windows 10 Memory Compression

I find their statement a bit ambiguous since it doesn't explicitly mention Windows 10 Preview.
They've introduced it to Windows 10, what else could it be except for Windows 10 or Windows 10 Preview?

Memory Manager Improvements:

In Windows 10, we have added a new concept in the Memory Manager called a compression store, which is an in-memory collection of compressed pages...
Announcing Windows 10 Insider Preview Build 10525 | Blogging Windows
Click to expand...

It was introduced in Windows 10 Insider 10525. If you compare it in task manager/resource monitor, the way Windows 10 handles RAM in build 10525 is a little bit different than 10240, it uses less page file and build 10525 is more snappier while doing the same task. Results may differ from user to user but in my case it is snappier.

Jairus24, Aug 20, 2015

#13
TAIRIKUOKAMI

TairikuOkami Win User

So Windows 10 still heavily relies on pagefile?! My system is at 120 kB with pagefile disabled. *Rolleyes

TairikuOkami, Aug 20, 2015

#14
LMILLER7

LMiller7 Win User

So Windows 10 still heavily relies on pagefile?! My system is at 120 kB with pagefile disabled. *Rolleyes
Click to expand...

The pagefile is and has always been an optimization allowing RAM to be used more efficiently. It is a place used to offload rarely used data so more RAM is available for more important purposes. Disabling the pagefile forces such data to remain in memory at all times, even if it is hardly ever used, even if it will never be used again. That is a crime against performance.

Compression allows this rarely used data to take up less space in RAM. And in the event that this data is written to the pagefile more data can be read or written in less time. This is a good thing. Yes, it will take time to compress this data but the extra CPU load is quite small and the benefits well worth the cost.

LMiller7, Aug 21, 2015

#15

(You must log in or sign up to reply here.)

Page 1 of 2

Thema:

Windows 10 Memory Compression

Loading...

Windows 10 Memory Compression - Similar Threads - Memory Compression
In use Compressed Memory using all of my ram.
in Windows 10 Gaming
In use Compressed Memory using all of my ram.: My compressed ram keeps increasing to use all of my ram. When I restart, it my computer works for a while then becomes unusable. I have had ram issues in the past bluescreens, crashing, pc shutting down without warning but I replaced my motherboard and RAM which seemed to...
In use Compressed Memory using all of my ram.
in Windows 10 Software and Apps
In use Compressed Memory using all of my ram.: My compressed ram keeps increasing to use all of my ram. When I restart, it my computer works for a while then becomes unusable. I have had ram issues in the past bluescreens, crashing, pc shutting down without warning but I replaced my motherboard and RAM which seemed to...
Windows 11 Memory Compression
in Windows 10 Gaming
Windows 11 Memory Compression: My memory compression is turned on but when i open "TERMINAL" as administrator or powershell + cmd , all of them say "Invalid Namespace" so i cannot disable it. I have also tried "Get-MMAgent" and it is the same thing , the cmd I used to try disable it was "Disable-MMAgent...
Windows 11 Memory Compression
in Windows 10 Software and Apps
Windows 11 Memory Compression: My memory compression is turned on but when i open "TERMINAL" as administrator or powershell + cmd , all of them say "Invalid Namespace" so i cannot disable it. I have also tried "Get-MMAgent" and it is the same thing , the cmd I used to try disable it was "Disable-MMAgent...
Windows 11 standby list and memory compression
in Windows 10 Gaming
Windows 11 standby list and memory compression: Win 11I9 - 12900KF @5.0Ghz32 GB DDR5 @5200MhzRTX 3080Original thread:https://answers.microsoft.com/en-us/windows/forum/all/windows-11-standby-list-and-memory-compression/4cc5956b-a8d2-42a3-8d4e-9e280bbc7699per the steps in the original thread on the last reply,"As a temporary...
Windows 11 standby list and memory compression
in Windows 10 Software and Apps
Windows 11 standby list and memory compression: Win 11I9 - 12900KF @5.0Ghz32 GB DDR5 @5200MhzRTX 3080Original thread:https://answers.microsoft.com/en-us/windows/forum/all/windows-11-standby-list-and-memory-compression/4cc5956b-a8d2-42a3-8d4e-9e280bbc7699per the steps in the original thread on the last reply,"As a temporary...
memory compression
in Windows 10 BSOD Crashes and Debugging
memory compression: memory compression1memory compression2 3 https://answers.microsoft.com/en-us/windows/forum/all/%E8%83%BD%E5%90%A6%E4%BB%8B%E7%BB%8D%E4%B8%80/00b611dd-4262-41f5-b82e-1daaf905ddbc
memory compression
in Windows 10 Gaming
memory compression: memory compression1memory compression2 3 https://answers.microsoft.com/en-us/windows/forum/all/%E8%83%BD%E5%90%A6%E4%BB%8B%E7%BB%8D%E4%B8%80/00b611dd-4262-41f5-b82e-1daaf905ddbc
memory compression
in Windows 10 Software and Apps
memory compression: memory compression1memory compression2 3 https://answers.microsoft.com/en-us/windows/forum/all/%E8%83%BD%E5%90%A6%E4%BB%8B%E7%BB%8D%E4%B8%80/00b611dd-4262-41f5-b82e-1daaf905ddbc

Log in or Sign up

Search