Windows 10: Unreadable non-ANSI characters in Notepad

Discus and support Unreadable non-ANSI characters in Notepad in Windows 10 Software and Apps to solve the problem; The problem: People living in countries, with languages including non-ANSI characters and want a full English Windows environment. If the user sets the... Discussion in 'Windows 10 Software and Apps' started by ddelo, May 21, 2018.

  1. ddelo Win User

    Unreadable non-ANSI characters in Notepad


    The problem:
    People living in countries, with languages including non-ANSI characters and want a full English Windows environment.
    If the user sets the System locale (Language for non-Unicode programs) to the country they live in, then many apps will check this setting and without giving the user any option, are installed with a localized interface, i.e. GUI based on the System locale, which might not be desirable.

    The apparent resolution is to change the System locale to English (US), which solves the apps interface issue, but because we’re talking about Microsoft Windows there is (as always…) an exception. In this case is Notepad…
    Notepad has ANSI (= ASCII & Extended ASCII) as its default setting for saving text files. If the text file contains non-ANSI characters then it gives a warning…which if you accidentally bypass and save the file with the ANSI encoding, all non-ANSI characters become unreadable.

    Being such a user, I have an English (US) installation and to avoid the localized app interface, I have set the System locale to English (United States).
    For some reason, which I haven’t found yet, before version 1803, I could save text documents with Greek (non-ANSI) characters and since I wasn’t getting the encoding warning (at least not that often) when saving the file, a lot of files with Greek characters were saved as ANSI and had no problem.

    This encoding issue has become stricter in 1803. My guess is the “Beta: Use Unicode UTF-8 for worldwide language support” setting that has been added when you change system locale has something to do with it. Either way this is, as stated, still in Beta, thus it doesn’t work as it supposed to, yet!

    So how to read all these text files with ANSI encoding, which contain non-ANSI characters, that are now unreadable?

    The solution:
    Step 1
    Go to: Settings > Time & Language > Region & Language > Related Settings > Administrative Language Settings (opens Control Panel) > System locale (Language for non-Unicode programs)

    Alternatively, for short, type in Windows search/Cortana:
    control.exe /NAME Microsoft.RegionalAndLanguageOptions /PAGE /p:"Administrative"

    and change the “System locale (Language for non-Unicode programs)” to the locale of the country you live in (Greece in my case).
    The system will need to reboot. Click Restart.

    Step 2
    Download the zip file at the end of this post, save and extract it on your Desktop. The zip file contains three scripts:
    CheckFileEncoding.ps1
    ConvertFilesToUnicode.ps1
    ConvertFilesToUnicode_NoBOM.ps1 (for advanced users)

    Step 3
    Open an elevated PowerShell and type the command:
    Code: Set-ExecutionPolicy Bypass -Scope Process -Force[/quote] Then type the following command (provided that you have saved the script in your Desktop):
    Code: $env:USERPROFILE\Desktop\CheckFileEncoding.ps1[/quote] The script will give you a list of all the ANSI text files, in all your user folders, as System.Text.ASCIIEncoding.

    You can check some with non-ANSI characters and verify that they are readable. (They should, since your locale is a non-ANSI one).

    Step 4
    Now you can run the command:
    Code: $env:USERPROFILE\Desktop\ConvertFilesToUnicode.ps1[/quote] The script will:
    1. Create a backup folder in C:\Backup\ASCIItxtBackup and will save a backup of all ANSI files you have in your user folders
    2. Convert all ANSI files you have in your user folders to Unicode.

    After that, you can do again Step 3, to verify that there are no ANSI files in your user folders.

    Step 5
    Go to: Settings > Time & Language > Region & Language > Related Settings > Administrative Language Settings (opens Control Panel) > System locale (Language for non-Unicode programs)

    Alternatively, for short, type in Windows search/Cortana:
    control.exe /NAME Microsoft.RegionalAndLanguageOptions /PAGE /p:"Administrative"

    and change the “System locale (Language for non-Unicode programs)” to the English local of your preference.
    The system will need to reboot. Click Restart.

    That was it. After your computer restarts and since all the text files are now saved in Unicode, they can be read with any System locale.

    Important Note:
    If you want to change either the backup location or the folders where the ANSI text files reside (e.g. search all C:\), open the script “ConvertFilesToUnicode.ps1” and as shown in the red box, in the image below, go to the section where we define the locations and change them according to your needs (e.g. $SourceDirectory = ‘C:\Personal\My Files’). Don’t forget to enclose the folder in quotes (e.g. ‘C:\Backup\My ASCII files’).

    Attachment 189639


    For Advanced Users
    Microsoft Notepad, saves all Unicode files with BOM (Byte Order Mark). In case you don’t want to use BOM in your Unicode text files, use the “ConvertFilesToUnicode_NoBOM.ps1”. It will do exactly what the “ConvertFilesToUnicode.ps1” does, but instead it will save the text file in any Unicode encoding without the BOM.

    Additionally, to change the Unicode encoding, to another format, in the convert section of the script change the Unicode in the “set-content $_.FullName -Encoding Unicode” part to any other of the available values:

    ‘ASCII’: Uses the encoding for the ASCII (7-bit) character set.
    ‘BigEndianUnicode’: Encodes in UTF-16 format using the big-endian byte order.
    ‘BigEndianUTF32’: Encodes in UTF-32 format using the big-endian byte order.
    ‘Default’: Encodes using the default value: ASCII.
    ‘Byte’: Encodes a set of characters into a sequence of bytes.
    ‘String’: Uses the encoding type for a string.
    ‘Unicode’: Encodes in UTF-16 format using the little-endian byte order.
    ‘UTF7:’ Encodes in UTF-7 format.
    ‘UTF8’: Encodes in UTF-8 format.

    :)
     
    ddelo, May 21, 2018
    #1
  2. EckiS Win User

    Notepad in Windows 10 doesn't display Chinese character

    I tested this: I pasted some chinese characters from charmap into notepad.

    On saving, a message box warned me that I have to change the encoding.

    Did you get the message box?

    When I use "Save as", and select "unicdode", reopening the file will display chinese characters.

    BUT: notepad always uses ANSI as default, so is probably not the best choice if you have to do this more often.
     
    EckiS, May 21, 2018
    #2
  3. Notepad issue- English turned into gibberish/unreadable symbols

    Step 2: Windows Update

    • Once again update through Windows Update
    • Follow this steps:
      • Click on Windows Key
      • Click on Settings
      • Click on Update & Security
      • On left panel click on Windows Update
      • On right panel under Windows Update click on Check for Updates

    Step 3: Scan your system

    • Specially D Drive or Notepad folder or Perform Full Scan
      if possible
    • Scan your system by Windows Defender(Recommended) or antivirus program

    Step 4: Unreadable to English

    • Open that Notepad file
    • Select all Crtl +A(Its optional only in Notepad)
    • On menu-bar click on Format menu, then Font...
    • Font window will open
    • Make it as shown below:

    • Unreadable non-ANSI characters in Notepad [​IMG]

    Step 5: Please let us know


    • Please let us know if the issue is Resolved or Not.
    • Please feel free to communicate if you have stuck
     
    Roharashraj™, May 21, 2018
    #3
Thema:

Unreadable non-ANSI characters in Notepad

Loading...
  1. Unreadable non-ANSI characters in Notepad - Similar Threads - Unreadable non ANSI

  2. How to support Non-BMP characters in character map

    in Windows 10 Gaming
    How to support Non-BMP characters in character map: I need help. how to put non-BMP symbols on Character plane? https://answers.microsoft.com/en-us/windows/forum/all/how-to-support-non-bmp-characters-in-character-map/84c42edf-0955-4459-9471-510b72a7eaaa
  3. How to support Non-BMP characters in character map

    in Windows 10 Software and Apps
    How to support Non-BMP characters in character map: I need help. how to put non-BMP symbols on Character plane? https://answers.microsoft.com/en-us/windows/forum/all/how-to-support-non-bmp-characters-in-character-map/84c42edf-0955-4459-9471-510b72a7eaaa
  4. Cannot type alt characters into Notepad

    in Windows 10 Gaming
    Cannot type alt characters into Notepad: Since you changed Notepad on us to v. 11.2112.32.0, I can no longer type alt characters those where you hold down the alt key and use the numeric keypad, such as –, —, é, à. Seems to be a really major omission to disallow basic, common characters like that. How can we enter...
  5. Cannot type alt characters into Notepad

    in Windows 10 Software and Apps
    Cannot type alt characters into Notepad: Since you changed Notepad on us to v. 11.2112.32.0, I can no longer type alt characters those where you hold down the alt key and use the numeric keypad, such as –, —, é, à. Seems to be a really major omission to disallow basic, common characters like that. How can we enter...
  6. Notepad encoding ANSI opening up as UTF-8 Win 10

    in Windows 10 Network and Sharing
    Notepad encoding ANSI opening up as UTF-8 Win 10: Hi All, i'm running Win 10 2004 19041.867 If i save a notepad document Blank as ANSI, it opens as UTF-8. If i use Notepad++ - the same thing happens. Does anyone know why?...
  7. Notepad ANSI/UTF-8

    in Windows 10 Network and Sharing
    Notepad ANSI/UTF-8: I saved my file as .txt ansi and it changed the format to .txt UTF-8. I tried on differnet computers as well, I got the same result. Any idea how to resolve this ? https://answers.microsoft.com/en-us/windows/forum/all/notepad-ansiutf-8/929f9241-b392-487b-afbb-b0f52f493346
  8. win10 .exe unreadable characters from another language.

    in Windows 10 Ask Insider
    win10 .exe unreadable characters from another language.: My Win10 is English, I'm trying to install a Chinese application {ChineseChar}.exe , a printing application, but it says 'the filename,directory name, or volume label syntax is incorrect'. I was able to install it after I changed the application's name to English as myApp.exe...
  9. CE character encoding in Notepad

    in Windows 10 Software and Apps
    CE character encoding in Notepad: I hate Windows feature that enforces installing localized software - e.g. I live in Poland so if I set my location as "Poland" many apps will be installed with Polish interface, which I don't want to. Therefore I've installed my OS in en-US version and left US as my...
  10. Windows 10 Character Map and Notepad

    in Windows 10 Support
    Windows 10 Character Map and Notepad: Have you tried wordpad? Yes he did (HERE)

Users found this page by searching for:

  1. unreadable characters in txt notepad fix

    ,
  2. Unreadable non-ANSI characters in Notepad

    ,
  3. text file notepad unreadable\

    ,
  4. foreign language characters unreadable unicode,
  5. notepad find unreadable character,
  6. notepad chinese characters saved ANSI,
  7. find unreadable chars in notepad ,
  8. ConvertFilesToUnicode.ps1,
  9. convert unreadable text in notepad,
  10. notepad app to read ansi,
  11. notepad non english,
  12. notepad text document unreadable ,
  13. what are Non ansi characters,
  14. ANSI/ASCII color notepad,
  15. notepad unreadable