PinyinTones: type Pinyin with tone marks into any Windows program

[taoyue]

Announcing a new tool for typing Hànyǔ Pīnyīn with tone marks into Windows applications...

PinyinTones is a Windows Text Service for Windows Vista and Windows 7, 32-bit and 64-bit. The 64-bit evolution has truly arrived -- almost half of Windows 7 installations are 64-bit. And 64-bit applications are slowly ramping up -- Microsoft Office 2010 is the first version available natively for 64-bit.

You can download it from: https://www.pinyintones.com/

 

Notes:
 

 

• PinyinTones requires the Visual C++ 2010 Redistributable Package (32-bit and/or 64-bit). The installer will automatically download and install the package from the Microsoft web site if you do not already have it installed.
• Designed for Windows Vista and Windows 7. On these versions of Windows, PinyinTones will work in any application, whether or not it is TSF-aware.
• Not recommended for Windows XP, which does not appear to have a TSF compatibility layer for non-TSF applications. On XP, PinyinTones will work only in TSF-aware applications.
• Be sure to get the version that matches your install of Windows: 32-bit or 64-bit. If you do not know which one you are running, press WindowsKey+Pause to bring up the System information dialog. Next to "System type", it will tell you if you are running a 32-bit or 64-bit operating system.
• PinyinTones registers itself as a Japanese text service. This was deliberately done to avoid the notorious bug in Microsoft Word that causes toned vowels to appear in a different font from surrounding consonants. Word does not do this for Japanese text services.

The source code is available on Codeplex under the Microsoft Public License, a permissive open-source license.

More details about PinyinTones:
 

• PinyinTones is coded to the most modern API available on Windows, the Text Services Framework (TSF). This is the same framework used by Microsoft Pinyin and other Microsoft-supplied language text services, as well as by voice dictation and handwriting recognition.
• Because it is written to the TSF, PinyinTones compiles right off-the-bat for 64-bit and 32-bit. TSF was introduced in Windows 2000 -- the first version of Windows to support a 64-bit processor (the Intel Itanium) -- and all the Microsoft sample code compiles directly for 64-bit without any tweaks.
• PinyinTones makes use of the composition string display that is built into TSF-aware applications. This is how other TSF services operate, including the MS Pinyin text service for typing in Chinese characters. Compositions appear directly in the body of the text, with a dotted underline to indicate that they are still tentative.
• PinyinTones was designed to be as unobtrusive as possible. There is no settings UI, because there are no settings.
• There is no checked input, so there won't be any bugs in the checking algorithm to prevent you from entering Pinyin. If you type correct Pinyin, it will output correct Pinyin (with correct placement of tone marks on vowel combinations). If you type incorrect Pinyin, it will produce ... something. But it will never block you from entering something.
• PinyinTones was designed to be a *very* simple program. I began with the Microsoft TSF sample code and changed fewer than 1000 lines of code. Less code means less that can go wrong.
• Less code also means less functionality. You may have to adjust your input habits to use PinyinTones most efficiently. For example, Backspace will not remove tones. If you're selling something (mai4) and decide you want to buy it instead, just type 3 (mai43), and the tone will change.

 

PinyinTones has been developed independently from the existing Pinyinput IME and does not share any code.

The most notable differences:
 

 

• Pinyinput is coded as an Input Method Editor (IME), a legacy API that dates back to Windows 95. Although Microsoft Windows is famously backward-compatible, there will inevitably be glitches from using a technology that is no longer being updated.
• Pinyinput draws its own toolbar and its own composition window.
• Pinyinput has a bunch of options, and a checked mode. Pinyinput is a more complex program than PinyinTones.

 

I have been in contact with Imron Alston, the author of Pinyinput. We agreed that it would be good to have both programs out there, so that the community can have the benefit of both design approaches.

PinyinTones or Pinyinput? Your choice. If you are running XP or need features such as checked mode, then use Pinyinput. If you want a simpler program that interacts better with newer versions of Windows and works on 64-bit, then use PinyinTones.

 

[obe]

taoyue - thanks for your very timely post!

I downloaded your program and it's working the only thing I'm missing there is the option to combine diacritical marks (which exists in Pinyinput). I realize that you are aiming for a minimal feature-set program, but was still wondering if you have such an option in the works, or if you have some commented out piece of code that could be easily integrated (even if by myself).

thanks again to you and to Imron for all your hard work!

[taoyue]

Hi obe,

I am indeed aiming for a program with a minimum feature set (and also minimum UI). But could you give me a few examples of where you need to use combined diacritical marks? I'm not inclined to add this feature, but I'll at least think about it.

There is no commented-out code in PinyinTones. I really tried not to write any more code than I needed.

[obe]

Hey,

Well to be honest, this is kind of a personal requirement for me... I have written some tool to aid me in my studies, and it does some comparison of pinyin texts... and the entire database was built using "combine diacritical marks".

Anyway, I have already downloaded your code and modified it myself... it seems well written so despite having no experience in working with text services, I was able to easily find a good place to make my adjustments.

By the way, what I wasn't able to do is compile the setup program. It might be lack of understanding on my part, or maybe I don't have some required software installed, but the solution seemed not to load properly, and the setup programs didn't compile... eventually I simply installed with your provided setup, and then manually overwritten the DLL with my compiled DLL...

If you want more input or to discuss anything else and don't think it fits this forum, you are also welcome to email me.

[taoyue]

Yes, I think it's best that you make your own changes to the code.

You need it for a very particular backward-compatibility purpose -- because you already built a database using combining diacritical marks, for a program (your own) that does not do Unicode normalization before string comparisons. I'm willing to leave this feature out for simplicity -- and also because I believe that a general-purpose input tool (i.e., not a tool specifically intended to output Unicode compositions) should output the most normalized form (Unicode Normalization Form C).

Visual Studio changes the project GUID upon opening the .sln. This causes the Setup##.vdproj to refer to a nonexistent project. For now, you can workaround the problem by diff'ing it to see which GUID got changed, and then running a search-and-replace on the Setup##.vdproj. This is obviously not an ideal solution! If you find a way to fix it, please let me know. So far, I've found lots of people discussing the problem at various places, but no solutions that actually work.

Edit: Found the problem. The setup projects now build in revision 54055.

[lwb0706]

My old Windows XP machine crashed, and I was excited to replace it with a brand new 64-bit Windows 7 box. But my heart sank when trusty old Pinyinput didn't work anymore. Fortunately, this tool came to the rescue! Thank you so much for doing this -- it saved me the hassle of buying an old machine just so I could use Pinyinput!

[imron]

Just an update to let you know that Pinyinput is now 64bit compatible too. See the release announcement here, or visit the Pinyinput project page on sourceforge.

[taoyue]

I have just released a new version of PinyinTones, updated to work with Windows 8. It is available both in desktop applications and in Windows Store applications.

Available for download at: http://www.pinyintones.com/

On earlier versions of Windows, there may be improved stability on certain applications that are linked to a different version of the Visual C++ redistributables. PinyinTones is now linked to the system version of the redistributable. This is what Microsoft recommends, and it is what they do for their own IMEs. Thus, it should at least theoretically be an improvement.

[grawrt]

Thank you! I just downloaded this to help with creating my decks and it's really helpful.

Just a note, when downloading it can be totally freaky cause you get a popup that says how its not often downloaded blah blah blah danger. I just got spyware the other day from downloading a chinese input system so I was a little wary but glad I went ahead with it because it's very easy to type with.