Windows 10: best way to replace or configure robots.txt to prevent indexing.

Discus and support best way to replace or configure robots.txt to prevent indexing. in Windows 10 Support to solve the problem; System is running Windows 10 and acting as an Apache website host. We have always had good results with using a robots.txt file consisting of only a... Discussion in 'Windows 10 Support' started by questorfla, Jul 15, 2020.

  1. best way to replace or configure robots.txt to prevent indexing.


    System is running Windows 10 and acting as an Apache website host.
    We have always had good results with using a robots.txt file consisting of only a few brief statements;
    The two lines of > "User-agent: *" and "Disallow: /"

    Lately we have noticed occasional reports of some documents turning up in various google cache searches with the contents attributed to the exact folder they are in inside the private website.

    If I wanted to prevent ALL indexing or scanning by ALL robots, what is the best way to accomplish this? These sites are meant to be accessed only by the people who have the link to them. Most contain private content that should never be attributed to anyone. I would prefer a method of keeping all indexing or scanning bots out of the sites.

    Anti-advertising if you will?
    Robots.txt doesn't seem to even stop Google-bots anymore

    :)
     
    questorfla, Jul 15, 2020
    #1
  2. Hrish Win User

    Best way to make a bootable WIndows 10 image on USB with all existing softwares, drivers & configurations to deploy on identical hardwares?

    Best way to make a bootable Windows 10 image on USB with all existing software, drivers & configurations to deploy on exactly identical hardware?

    There is no difference in the hardware we want to deploy that image on.

    Planning to deploy this image on over 4000 desktops of same hardware configuration and make and model.

    Is there a freeware/opensource/free software that we can use for this purpose commercially?

    can you suggest a solution and KB article where it's step by step explained how to achieve it or do it?
     
    Hrish, Jul 15, 2020
    #2
  3. Jsssssssss, Jul 15, 2020
    #3
  4. ¡Firedog Win User

    best way to replace or configure robots.txt to prevent indexing.

    How to prevent index rebuild after each major update

    After each major update, I discover that the Windows Search index is being rebuilt. This is very resource-hungry and means that the index search is unusable for several days. Programs are competing with the indexing service for resources, with a severe degradation
    in performance as a result unless I throttle the process while I'm working, which of course prolongs the agony.

    I installed the update to 10586.164 on 9 March. The computer has been running more or less continuously since then, but there are now a week later still 50,000 items to be indexed of the 2.3 million total.

    • Is this re-indexation really necessary?

    • Is there any way of preventing it from happening?
     
    ¡Firedog, Jul 15, 2020
    #4
Thema:

best way to replace or configure robots.txt to prevent indexing.

Loading...
  1. best way to replace or configure robots.txt to prevent indexing. - Similar Threads - best replace configure

  2. Is there any way to autosave .txt's on shutdown?

    in Windows 10 Ask Insider
    Is there any way to autosave .txt's on shutdown?: Title I am a big fan of the notepad feature. When I turn my computer on in the morning I usually pull up google, steam, discord, a few other things, and my main notepad that I keep open and edit throughout the day. I do my stuff and at the end of the day I go to turn off my...
  3. Best way to update

    in Windows 10 Installation and Upgrade
    Best way to update: What is the most trouble free way to update Windows? Regular ol' Windows Update, Update Assistant or ISO? 139744
  4. best Replacement for Windows Live Mail

    in Browsers and Email
    best Replacement for Windows Live Mail: What is the most similar, free email client app, to replace Windows Live Mail? Thunderbird is useable, but quite complicated compared to WLM which has the menu bar along the top. Mail in Windows 10 doesn't allow emails to be dragged/dropped to desktop, and a few other...
  5. Best Browser & Configuration for Banking?

    in Browsers and Email
    Best Browser & Configuration for Banking?: I've just uninstalled Kaspersky Total Security so I can no longer use the Safe Money browser feature. What's the best way to configure Chrome for safe online banking or are there more secure browsers I should consider? 98874
  6. Best way for Updates

    in Windows 10 Updates and Activation
    Best way for Updates: Hello all, did a Google search and just can't find a lot of answers on this so here I am Right now using WHdownload to keep up with updates,not sure this is the best way to go. Have 3 computer in house so moving updates from one to other to save data downloading. When...
  7. Best Way to Install Updates?

    in Windows 10 Updates and Activation
    Best Way to Install Updates?: I would like to get a mini poll here on how people would install Windows 10 Updates? For now, I set it to Auto Download and Notify to Install from the Group Policy settings since I have the Pro version. Please provide me some feedback on how you guys do the Windows...
  8. Best way to remove trialware?

    in Windows 10 Software and Apps
    Best way to remove trialware?: Hello ALL: We have a new Windows 10 Pro Laptop to be delivered Friday. It is a 17" HP and know it comes with Office trialware along with AV trialware. We own Office 2016 Pro and want to use it, but I want to just remove all of the junque that it comes with. Should I use...
  9. Which is the best replacement to my computer?

    in Windows 10 Software and Apps
    Which is the best replacement to my computer?: I want your opinion on the best replacement for my computer,is xyplorer good or speed commander ? 105845
  10. Best way to update drivers

    in Windows 10 Drivers and Hardware
    Best way to update drivers: Best way to update drivers I recently purchase a laptop and want to know a few things about driver update Firstly i discovered there a few ways by which one can update drivers but i want too know which one of these is better and should be preferred : 1) Driver...