Windows 10: best way to replace or configure robots.txt to prevent indexing.

Discus and support best way to replace or configure robots.txt to prevent indexing. in Windows 10 Support to solve the problem; System is running Windows 10 and acting as an Apache website host. We have always had good results with using a robots.txt file consisting of only a... Discussion in 'Windows 10 Support' started by questorfla, Jul 15, 2020.

  1. best way to replace or configure robots.txt to prevent indexing.


    System is running Windows 10 and acting as an Apache website host.
    We have always had good results with using a robots.txt file consisting of only a few brief statements;
    The two lines of > "User-agent: *" and "Disallow: /"

    Lately we have noticed occasional reports of some documents turning up in various google cache searches with the contents attributed to the exact folder they are in inside the private website.

    If I wanted to prevent ALL indexing or scanning by ALL robots, what is the best way to accomplish this? These sites are meant to be accessed only by the people who have the link to them. Most contain private content that should never be attributed to anyone. I would prefer a method of keeping all indexing or scanning bots out of the sites.

    Anti-advertising if you will?
    Robots.txt doesn't seem to even stop Google-bots anymore

    :)
     
    questorfla, Jul 15, 2020
    #1
  2. Hrish Win User

    Best way to make a bootable WIndows 10 image on USB with all existing softwares, drivers & configurations to deploy on identical hardwares?

    Best way to make a bootable Windows 10 image on USB with all existing software, drivers & configurations to deploy on exactly identical hardware?

    There is no difference in the hardware we want to deploy that image on.

    Planning to deploy this image on over 4000 desktops of same hardware configuration and make and model.

    Is there a freeware/opensource/free software that we can use for this purpose commercially?

    can you suggest a solution and KB article where it's step by step explained how to achieve it or do it?
     
    Hrish, Jul 15, 2020
    #2
  3. Jsssssssss, Jul 15, 2020
    #3
  4. ¡Firedog Win User

    best way to replace or configure robots.txt to prevent indexing.

    How to prevent index rebuild after each major update

    After each major update, I discover that the Windows Search index is being rebuilt. This is very resource-hungry and means that the index search is unusable for several days. Programs are competing with the indexing service for resources, with a severe degradation
    in performance as a result unless I throttle the process while I'm working, which of course prolongs the agony.

    I installed the update to 10586.164 on 9 March. The computer has been running more or less continuously since then, but there are now a week later still 50,000 items to be indexed of the 2.3 million total.

    • Is this re-indexation really necessary?

    • Is there any way of preventing it from happening?
     
    ¡Firedog, Jul 15, 2020
    #4
Thema:

best way to replace or configure robots.txt to prevent indexing.

Loading...
  1. best way to replace or configure robots.txt to prevent indexing. - Similar Threads - best replace configure

  2. Best way for 23H2

    in Windows 10 Gaming
    Best way for 23H2: Since it's been 3 months since the release, I'm now thinking I should use one of the methods to manually update. I seem to have Co-Pilot working but still on version 22H2. So, there's an enablement package, ISO, and Installation Assistant. Which of the three would be...
  3. Which is, the best way?

    in Windows 10 Gaming
    Which is, the best way?: My omputer is a laptop. Running Windows 10.I've been using notepad. But I'm looking, for something better?There are so many choices. Word, Wordpad, Notepad, Notepad 2, Notepad++I prefer Wordpad because you could click on links. But here's the problemafter I open it I copied...
  4. Which is, the best way?

    in Windows 10 Software and Apps
    Which is, the best way?: My omputer is a laptop. Running Windows 10.I've been using notepad. But I'm looking, for something better?There are so many choices. Word, Wordpad, Notepad, Notepad 2, Notepad++I prefer Wordpad because you could click on links. But here's the problemafter I open it I copied...
  5. Best way to backup

    in Windows 10 BSOD Crashes and Debugging
    Best way to backup: Hi, I recently had an issue where my hard drive corrupted and I lost everything. I didnt have a restore point or backup. Now I want to make sure that if this happens again, I have a backup. What is the best way to do this on a regular basis? Would I need a 3rd party...
  6. Best replacement for Instant Backup

    in Windows 10 Backup and Restore
    Best replacement for Instant Backup: I used to use a program called Instant Backup, a paid program that did everything I wanted. I could schedule daily, weekly, and monthly backups and I managed to use it even on Windows 10 until I did a clean reinstall of Windows 10 and the program refused to install. I first...
  7. Is there any way to autosave .txt's on shutdown?

    in Windows 10 Ask Insider
    Is there any way to autosave .txt's on shutdown?: Title I am a big fan of the notepad feature. When I turn my computer on in the morning I usually pull up google, steam, discord, a few other things, and my main notepad that I keep open and edit throughout the day. I do my stuff and at the end of the day I go to turn off my...
  8. Best way to update

    in Windows 10 Installation and Upgrade
    Best way to update: What is the most trouble free way to update Windows? Regular ol' Windows Update, Update Assistant or ISO? 139744
  9. Best Browser & Configuration for Banking?

    in Browsers and Email
    Best Browser & Configuration for Banking?: I've just uninstalled Kaspersky Total Security so I can no longer use the Safe Money browser feature. What's the best way to configure Chrome for safe online banking or are there more secure browsers I should consider? 98874
  10. Best way for Updates

    in Windows 10 Updates and Activation
    Best way for Updates: Hello all, did a Google search and just can't find a lot of answers on this so here I am Right now using WHdownload to keep up with updates,not sure this is the best way to go. Have 3 computer in house so moving updates from one to other to save data downloading. When...