Windows 10: Confused about effects of pagefile usage on real-time performance

Hello,

I'm using a couple of PCs to stream audio for virtual musical instruments. Basically, there's a master PC that receives commands from a piano keyboard then sends those commands to the slave PCs to play back the audio associated with each key press. That process needs to happen quickly in order to "feel" correct when playing.

Here's my dilemma: every time I load up the audio data on the slave, it gets within 5-10 GB of the total amount of physical RAM (64 GB) and then it starts dumping data to the page file, thereby reducing the RAM usage. That creates disruptions in the audio stream because it can't pull the data from the page file as quickly as it can from RAM (at least I presume that's what's happening). The pagefile usage goes to about 10 GB (as reported by perfmon.exe) and the memory usage goes to about 47 GB (as reported by task manager). So the total memory requirement is about 57 GB, leaving me about 7 GB to spare.

But why is it using the page file instead of using physical memory? My understanding is that the OS tries to free up memory that it thinks I won't need. But how does it decide what RAM is less necessary? In my case it always seems to guess wrong because I can drop the page file down to 1 GB and the system performs better.

But even with a tiny page file I still get 5-7 GB of RAM that is "modified" and, apparently, not available for use.

It seems to me that the best setup for a real-time application like this one is to have no page file because there are no other apps loaded or executed - the memory gets filled at boot up and then never changes. So why would you ever need a pagefile? But even with a tiny page file Windows won't let me access all my RAM and it puts a bunch of it into some kind of quarantine that prevents me from making use of it.

Is there a solution to this problem? Or do I just have to accept that my 64 GB of RAM is really only about 55 GB, and I've lost the other 9 GB to Windows because it thinks it's smarter than I am?

Thanks in advance,



:)

 

MsMpEng extreme and almost constant disk usage after Windows update (aprox.2016-08-03)

MsMpEng and System constantly (mostly) reads disk at high rate. The problem has started couple of days ago. The only thing that was installed during that time was windows update (ok and Adobe and Steam updates).

After some time the disk activity will stop, however it will resume sometime latter.
And no, I had not installed any third party security applications.

In resource monitor it can be seen that Diks activity is 2-8 mb/sec and process name is either System or MsMpEng .

Sometimes there are significant pagefile usage (computer has 4gb) and nothing is currently running (like now I am running only Internet Explorer with this Microsoft page and my Windows are using pagefile like there is no tomorow) . When pagefile usage creeps
up then all kinds of processes are using the pagefile. However resource monitor memory tab shows bigest process beeing MsMpEng with 644kb (note again it should not cause 4gb system to run out of memory). Right now system is writing 2mb/sec (read 0.4kb/sec) to
pagefile and MsMpEng is reading 2mb/sec from pagefile.

And no, running said Windows defender does not show up any viruses.

Basicaly - this is rant about apparently bugy latest Windows update.

 

Windows 10, 100% disk usage

As a note, I've also tried turning off real time detection of Windows Defender (MsMPEng.exe) and while any disk usage from MsMPEng.exe does obviously disappear, it has no noticeable effect on disk usage (still at 100%) or disk stalling, which both remain
due to System.

 

Hi, could you post a screenshot something like this please?



(Ignore the details on mine- my pagefile is on E: )

Also a screenshot of your task manager - the Performance tab, Memory. It would be helpful to see if you are using compressed RAM.

You will most likely find part of your RAM is allocated to your graphics card.

Could you also please complete your system specs (see mine for example)?
System Specs - Fill in at Ten Forums

 

OK - my page file is on my L drive (why does that matter?) and has been set to between 1 and 15 GB. It's an SSD. As I said above, when set to a larger value I get many breaks in the audio stream. The frequency with which those breaks occur decreases with decreasing page file size.

I have tried both with compressed RAM and without and the behavior is the same.

System specs:
i7 6700k on Gigabyte/Asus motherboard
64 GB RAM
Various SSDs and HDDs
One machine uses a $20 AMD video card and the other has a GTX 970.

How does this help answer my questions above? I'm not sure what you do with this extra information.

Thanks,

 

Best place to learn about paging behavior in Windows is in Mark Russinovich et al's outstanding book Windows Internals, 6th edition. Chapter 10 in Part 2 is entitled Memory Management and it explains how modern Windows manages memory and, by extension, how it works with the paging file. Turns out that at least some paging file space is a good idea to maintain on Windows, even on systems with max memory, simply to leave buffering space for pages waiting to be written to memory while pages marked for deletion are being cleaned out. Fascinating stuff!
HTH,
--Ed--

 

A weird thought just surfaced from my MS-DOS 5 days. I frequently created a RAM disk in memory to copy data from floppies before recopying to a blank disk. MS-DOS 6.xx was much better as it would copy to the HDD first. Anyway, the thought was running the Pro version of Windows on a motherboard that supports 128GB RAM and create that RAM disk about half the RAM and designate it for the paging file/Virtual Memory. Or in other word, speed. It would be a bit expensive but the end result may be worth it.

 

Hi Ed - thanks for the info. I'll check it out.

Quick question - it seems to me that leaving the page file intact does, indeed, make sense for most use cases (spreadsheets, web browsing, e-mail, etc). But do you think the same thinking applies to real-time applications like I described above? Does that book specifically discuss how the page fille affects real-time applications?

My understanding is that the pagefile is kind-of a hedge against what the user might do in the future - load other apps, load other data, etc. so Windows tries to guess what's not really necessary and free up memory for what might be necessary in the future. Right?

But in the case I'm describing, there are no other apps or data loaded. Once everything is loaded at boot-up, nothing changes. The computer just responds to requests to dump data that's already in memory into a buffer and transfer it to the master computer.

So, given that the real-time system performance would improve without the pagefile (as evidenced by my testing), doesn't it make sense for Windows to just forget about the pagefile and "modified" memory?

More to the point, how do you force it to do so? I can disable the pagefile but I still lose all that memory to "modified" status, which, as far as I can tell, is Windows waiting for the pagefile size to increase so that it can dump it there.

Thanks again,

 

Always have to keep in mind that other/third-party program installs look for certain things to be available, downside of Microsoft having to "cooperate" with those programmers. Apple not so much, Linux usually even more open than Windows.

 

Dear Rgames:
As I understand it, if you're running real-time stuff it has ways to lock itself into resident memory (the non-paged pool). In such cases, it's only going to use the pagefile when it's either being swapped in before it starts running, or when it swaps out, after it's finished running. While running, paging should not occur for such stuff. That said, I'm not savvy enough about memory allocations available for application use to know if (or how) an application could make such memory reservations. To the best of my knowledge only the OS kernel is able to lock itself into ring 0 status and related memory reservation. Anybody else know different?
Thanks,
--Ed--