Windows 10: Confused about effects of pagefile usage on real-time performance

Discus and support Confused about effects of pagefile usage on real-time performance in Windows 10 Performance & Maintenance to solve the problem; Hello, I'm using a couple of PCs to stream audio for virtual musical instruments. Basically, there's a master PC that receives commands from a piano... Discussion in 'Windows 10 Performance & Maintenance' started by rgames, Dec 27, 2016.

  1. rgames Win User

    Confused about effects of pagefile usage on real-time performance


    Hello,

    I'm using a couple of PCs to stream audio for virtual musical instruments. Basically, there's a master PC that receives commands from a piano keyboard then sends those commands to the slave PCs to play back the audio associated with each key press. That process needs to happen quickly in order to "feel" correct when playing.

    Here's my dilemma: every time I load up the audio data on the slave, it gets within 5-10 GB of the total amount of physical RAM (64 GB) and then it starts dumping data to the page file, thereby reducing the RAM usage. That creates disruptions in the audio stream because it can't pull the data from the page file as quickly as it can from RAM (at least I presume that's what's happening). The pagefile usage goes to about 10 GB (as reported by perfmon.exe) and the memory usage goes to about 47 GB (as reported by task manager). So the total memory requirement is about 57 GB, leaving me about 7 GB to spare.

    But why is it using the page file instead of using physical memory? My understanding is that the OS tries to free up memory that it thinks I won't need. But how does it decide what RAM is less necessary? In my case it always seems to guess wrong because I can drop the page file down to 1 GB and the system performs better.

    But even with a tiny page file I still get 5-7 GB of RAM that is "modified" and, apparently, not available for use.

    It seems to me that the best setup for a real-time application like this one is to have no page file because there are no other apps loaded or executed - the memory gets filled at boot up and then never changes. So why would you ever need a pagefile? But even with a tiny page file Windows won't let me access all my RAM and it puts a bunch of it into some kind of quarantine that prevents me from making use of it.

    Is there a solution to this problem? Or do I just have to accept that my 64 GB of RAM is really only about 55 GB, and I've lost the other 9 GB to Windows because it thinks it's smarter than I am?

    Thanks in advance,



    :)
     
    rgames, Dec 27, 2016
    #1
  2. Rihards Win User

    MsMpEng extreme and almost constant disk usage after Windows update (aprox.2016-08-03)

    MsMpEng and System constantly (mostly) reads disk at high rate. The problem has started couple of days ago. The only thing that was installed during that time was windows update (ok and Adobe and Steam updates).

    After some time the disk activity will stop, however it will resume sometime latter.
    And no, I had not installed any third party security applications.

    In resource monitor it can be seen that Diks activity is 2-8 mb/sec and process name is either System or MsMpEng .

    Sometimes there are significant pagefile usage (computer has 4gb) and nothing is currently running (like now I am running only Internet Explorer with this Microsoft page and my Windows are using pagefile like there is no tomorow) . When pagefile usage creeps
    up then all kinds of processes are using the pagefile. However resource monitor memory tab shows bigest process beeing MsMpEng with 644kb (note again it should not cause 4gb system to run out of memory). Right now system is writing 2mb/sec (read 0.4kb/sec) to
    pagefile and MsMpEng is reading 2mb/sec from pagefile.

    And no, running said Windows defender does not show up any viruses.

    Basicaly - this is rant about apparently bugy latest Windows update.
     
    Rihards, Dec 27, 2016
    #2
  3. Windows 10, 100% disk usage

    As a note, I've also tried turning off real time detection of Windows Defender (MsMPEng.exe) and while any disk usage from MsMPEng.exe does obviously disappear, it has no noticeable effect on disk usage (still at 100%) or disk stalling, which both remain
    due to System.
     
    BucksterMcgee, Dec 27, 2016
    #3
  4. dalchina New Member

    Confused about effects of pagefile usage on real-time performance

    Hi, could you post a screenshot something like this please?

    Confused about effects of pagefile usage on real-time performance [​IMG]

    (Ignore the details on mine- my pagefile is on E: )

    Also a screenshot of your task manager - the Performance tab, Memory. It would be helpful to see if you are using compressed RAM.

    You will most likely find part of your RAM is allocated to your graphics card.

    Could you also please complete your system specs (see mine for example)?
    System Specs - Fill in at Ten Forums
     
    dalchina, Dec 27, 2016
    #4
  5. rgames Win User
    OK - my page file is on my L drive (why does that matter?) and has been set to between 1 and 15 GB. It's an SSD. As I said above, when set to a larger value I get many breaks in the audio stream. The frequency with which those breaks occur decreases with decreasing page file size.

    I have tried both with compressed RAM and without and the behavior is the same.

    System specs:
    i7 6700k on Gigabyte/Asus motherboard
    64 GB RAM
    Various SSDs and HDDs
    One machine uses a $20 AMD video card and the other has a GTX 970.

    How does this help answer my questions above? I'm not sure what you do with this extra information.

    Thanks,
     
    rgames, Dec 27, 2016
    #5
  6. EdTittel Win User
    Best place to learn about paging behavior in Windows is in Mark Russinovich et al's outstanding book Windows Internals, 6th edition. Chapter 10 in Part 2 is entitled Memory Management and it explains how modern Windows manages memory and, by extension, how it works with the paging file. Turns out that at least some paging file space is a good idea to maintain on Windows, even on systems with max memory, simply to leave buffering space for pages waiting to be written to memory while pages marked for deletion are being cleaned out. Fascinating stuff!
    HTH,
    --Ed--
     
    EdTittel, Dec 27, 2016
    #6
  7. Berton Win User
    A weird thought just surfaced from my MS-DOS 5 days. I frequently created a RAM disk in memory to copy data from floppies before recopying to a blank disk. MS-DOS 6.xx was much better as it would copy to the HDD first. Anyway, the thought was running the Pro version of Windows on a motherboard that supports 128GB RAM and create that RAM disk about half the RAM and designate it for the paging file/Virtual Memory. Or in other word, speed. It would be a bit expensive but the end result may be worth it.
     
    Berton, Dec 27, 2016
    #7
  8. rgames Win User

    Confused about effects of pagefile usage on real-time performance

    Hi Ed - thanks for the info. I'll check it out.

    Quick question - it seems to me that leaving the page file intact does, indeed, make sense for most use cases (spreadsheets, web browsing, e-mail, etc). But do you think the same thinking applies to real-time applications like I described above? Does that book specifically discuss how the page fille affects real-time applications?

    My understanding is that the pagefile is kind-of a hedge against what the user might do in the future - load other apps, load other data, etc. so Windows tries to guess what's not really necessary and free up memory for what might be necessary in the future. Right?

    But in the case I'm describing, there are no other apps or data loaded. Once everything is loaded at boot-up, nothing changes. The computer just responds to requests to dump data that's already in memory into a buffer and transfer it to the master computer.

    So, given that the real-time system performance would improve without the pagefile (as evidenced by my testing), doesn't it make sense for Windows to just forget about the pagefile and "modified" memory?

    More to the point, how do you force it to do so? I can disable the pagefile but I still lose all that memory to "modified" status, which, as far as I can tell, is Windows waiting for the pagefile size to increase so that it can dump it there.

    Thanks again,
     
    rgames, Dec 27, 2016
    #8
  9. Berton Win User
    Always have to keep in mind that other/third-party program installs look for certain things to be available, downside of Microsoft having to "cooperate" with those programmers. Apple not so much, Linux usually even more open than Windows.
     
    Berton, Dec 27, 2016
    #9
  10. EdTittel Win User
    Dear Rgames:
    As I understand it, if you're running real-time stuff it has ways to lock itself into resident memory (the non-paged pool). In such cases, it's only going to use the pagefile when it's either being swapped in before it starts running, or when it swaps out, after it's finished running. While running, paging should not occur for such stuff. That said, I'm not savvy enough about memory allocations available for application use to know if (or how) an application could make such memory reservations. To the best of my knowledge only the OS kernel is able to lock itself into ring 0 status and related memory reservation. Anybody else know different?
    Thanks,
    --Ed--
     
    EdTittel, Apr 4, 2018
    #10
Thema:

Confused about effects of pagefile usage on real-time performance

Loading...
  1. Confused about effects of pagefile usage on real-time performance - Similar Threads - Confused effects pagefile

  2. Permanently disable the Real Time Protection to increase performance

    in AntiVirus, Firewalls and System Security
    Permanently disable the Real Time Protection to increase performance: I am trying to permanently disable the Real Time Protection RTP - and I succeed on one Windows10 where I after reboot still have disabled the RTP. On another Windows10 I fail. After reboot the RTP is active again. I can see that the entry I made using the Regedit has...
  3. Confused about Updates

    in Windows 10 Ask Insider
    Confused about Updates: Hi, I've spent a few hours this morning trying to update Windows then furiously back-peddle after reading horror stories online - throw in a Repairing Disc Errors screen... you get the idea. But I'm confused about what I should do now. Under Windows Update I have: Feature...
  4. BitLocker effect on performance ?

    in AntiVirus, Firewalls and System Security
    BitLocker effect on performance ?: Hi I have normal sata HDDs, usb3 drive, and 1 nvme , I want to ask if bitlocker will degrade performance if enabled ? I want to do this to protect my data and my accounts. Thanks 155427
  5. BitLocker effect on performance ?

    in Windows 10 Support
    BitLocker effect on performance ?: Hi I have normal sata HDDs, usb3 drive, and 1 nvme , I want to ask if bitlocker will degrade performance if enabled ? I want to do this to protect my data and my accounts. Thanks 155427
  6. [Help]Is there an application to get mouse effects like Camtasia but in real time?

    in Windows 10 Ask Insider
    [Help]Is there an application to get mouse effects like Camtasia but in real time?: With Camtasia you can make the mouse pointer more visible, add spotlight and click effects, etc. Pic I saw somebody on a Mac OX with really cool mouse effects during a live presentation. Is there something that can do that in Windows 10 but in real time? There are some apps...
  7. pagefile is reduces performance greatly

    in Windows 10 Performance & Maintenance
    pagefile is reduces performance greatly: Because memory leak in windows, you can't get rid of pagefile completely. There are some programs, which are suppose to deal with that. But on occasion i am maxed on ram, even i have 16GB RAM. And also some programs doesn't work without page file. I have pagefile on hdd, but...
  8. Performance monitor confusion.

    in Windows 10 Performance & Maintenance
    Performance monitor confusion.: I have picked out one process here as an example Corsair.Service. Performance Monitor indicates an average of 7.587 while Process Monitor 0.61. Is Performance monitor indicating 7.587% CPU usage? If so how do I interpret Process Monitor's reading 0.61%? Which number is...
  9. Confused about backup

    in Windows 10 Backup and Restore
    Confused about backup: [img] I'm new to Win 10 and yesterday tried to back up my files for the first time. I went to setting>update & security>backup and tried to backup on a flash drive (to transfer to ext. HD), but got a message that files not yet backed up. I then exported my Firefox folders...
  10. Confused about OneDrive

    in Windows 10 Support
    Confused about OneDrive: First post, as a newcomer to Win 10 after 15 years with XP (and all previous Windows OS versions). On this page Files save to OneDrive by default in Windows 10 I'm reading an explanation of OneDrive: "While you can always choose where to save files that you create,...