HOWTO: Installing Japanese Input and Font Setup in Ubuntu 8.04 (Hardy) using SCIM 日本語

[ryukent]

HOWTO: Installing Japanese Input and Font Setup in Ubuntu 8.04 (Hardy) using SCIM: 日本語

Installing Japanese Input and Superior Font Setup in Ubuntu

Introduction

This is a guide to setting up Japanese for Ubuntu 8.04 Hardy. It is intended as a complete guide encompassing all elements required for using Japanese on any language installation of Ubuntu. It covers input (SCIM-Anthy) and configuring the Japanese fonts. There are other guides around for older versions of Ubuntu or that use the alternative UIM (see other guide). This guide is intended to cover everything. Please note that Kubuntu requires slightly different steps. Please follow the relevant page accordingly. This is an updated version based on the original 6.10 one, but with some sections changed. Please note that if you follow this guide, your fonts will be reconfigured. This might mean losing some font settings you may have made. With each version of Ubuntu, there are certain changes, this guide is not the same as the 7.10 version.

Issues Involved

There are two main issues here:

1.Installing the SCIM input system that will work in a locale other than converting your whole install to Japanese, i.e. you want Japanese input in an English login.

2.The fonts look initially terrible. Therefore a certain amount of customisation is required to make all the Kanji's render in the same style and Hiragana & Katakana to render in a matching style.

Japanese Input with SCIM

This section covers setting up the Japanese input system using SCIM Anthy. This involves, downloading, installing and configuring it so that you can use it in non-Japanese locales (e.g. your system is in English).

Setting Up Repositories

First lets make sure you have the correct repositories installed in order to automatically download the relevant packs. Make sure you have the Universe and Multiverse repositories switched on. This can be done in 'Synaptic Package Manager' under the repositories tab. Also, you need the Japanese repository too. Open the repositories list file:

gksudo gedit /etc/apt/sources.list

Add the following line at the bottom:

deb http://archive.ubuntulinux.jp/ubuntu-ja hardy/

Note that you will need to change 'hardy' if you are using a different version from 8.04. Now update your repos with:

sudo apt-get update

At this stage, you will probably get an error saying that the repository is not validated. Ignore this for now. The following step will correct it. After adding the repository and running the update, you also need to add a keyring for the new location:

sudo apt-get install ubuntu-ja-keyring

Adding Ubuntu Language Support

Go to System / Administration / Language Support and select Japanese. This should install the basics. Make sure you've also turned on support for inputting complex characters.

Making SCIM available under a non-Japanese login

Now you want to make SCIM (Language input system) available in your English (or other language) login and not just the Japanese one. Since 8.04, ubuntu will make it available in GTK applications, but if you want to run non-GTK applications such as KDE or pure X software such as those Java based you'll need to make a few changes. If you are running a US locale, it might work with defaults, but any other locale will almost certainly need registering. First open the scim global settings file:

gksudo gedit ~/.scim/global

Add the line:

/SupportedUnicodeLocales = en_US.UTF-8,en_GB.UTF-8

The above line adds support for US and UK locales. If you are using a different locale, you will need to change / add the relevant locale. You can find out the name of your current locale by entering:

locale

In my case (UK) it returns LANG=en_GB.UTF-8. Add the necessary to the above line.

IMPORTANT NOTE: SCIM is very unforgiving with this line. Note that there is NO SPACE between the "," and "en_GB". If you put a space there, it will ignore everything after. Therefore make sure the following locales are separated by a comma only.

At this stage you'll probably need to log out and back in again. Open a text editor and hit ctrl+space. SCIM should pop up ready to type in Japanese.

Adding handwriting recognition support for looking up Kanjis

After adding the above repository, you should be able to install the 'Tomoe' handwriting recognition addon for scim using:

sudo apt-get install scim-tomoe

Unfortunately, Tomoe is set to load dictionaries that correspond to the locale, so if you're not using a Japanese locale, you'll need to create a link to the dictionary manually.

cd /usr/share/tomoe/recognizer
sudo cp handwriting-ja.xml handwriting-en.xml

Where 'en' corresponds to your locale type. In my case (en_GB.UTF-8) it is 'en'. For you, it might be different. You can look it up as mentioned above.

Now that Tomoe is installed, it is accessible on the SCIM menu under the 'SCIM Command Menu' and listed as 'Handwriting recognition'.

Setting up the system to display Japanese characters properly

OK, now you've got Japanese input installed (hopefully). It might require rebooting xwindows (CTRL+ALT+Backspace). But for me, I really didn't like the horrible fonts that defaulted. Particularly the fact that hiragana / katakana characters are rendered differently from kanjis and the poor quality of smaller sizes annoys me. The main reason for this is that the fonts provided do not always have a full set of kanjis, with default settings the kanjis are rendered as bitmaps and the hiragana and katakana as vectors.

At lower font sizes, it would be impossible to render all the strokes in very complicated characters without blurring and this causes a readability problem. This can be overcome with bitmap alternatives at a low end. Certain strokes are omitted and the shape is actually changed in order to improve readability. It's not simply a case of rendering the same vector in a smaller size. Some true type fonts contain bitmap alternatives that can be automatically substituted at the low end. This is a common approach and is adopted by Windows, MacOS and other electronic devices in Asia such as mobile phones. Here's the next step.

Downloading External Fonts

Unfortunately, I am very disappointed in the Ubuntu selection and you will almost certainly want this to be changed to MSGothic and MSMincho. They contain a superior vector and bitmap selection. These are Microsoft fonts, but they are freely available to use and are actually from a company called Ricoh. They need to be downloaded and installed manually. They can be found at the following page.

http://www.linux.ryukent.co.uk/show.php?id=24

So download and extract the files and you need to copy them into the fonts directory. This will need root privileges and is probably easiest done using the file explorer:

gksudo "nautilus --browser"

That will give you a browser with the right privileges. So copy your downloaded ttf files and paste them into a folder under the fonts tree. I recommend:

/usr/share/fonts/truetype/msjapanesefonts

Rebuilding the font cache

Now we need to rebuild the fonts cache:

sudo fc-cache -f -v

Setting up the font order

OK, so that might well be enough, but I think you'll probably still have your Japanese fonts not running at optimum and the default might be a little ugly. Lets set up the order in which we like the fonts to be selected. Open the “.fonts.conf” file in your home directory:

gksudo gedit ~/.fonts.conf

It should read as follows:

<?xml version="1.0"?>
<fontconfig>
 <alias>
 <family>serif</family>
 <prefer>
 <family>DejaVu Serif</family>
 <family>Times New Roman</family>
 <family>ＭＳ 明朝</family>
 <family>IPAPMincho</family>
 <family>Sazanami Mincho</family>
 <family>Kochi Mincho</family>
 <family>Bitstream Vera Serif</family>
 <family>Thorndale AMT</family>
 <family>Luxi Serif</family>
 <family>Nimbus Roman No9 L</family>
 <family>Times</family>
 <family>Frank Ruehl</family>
 <family>MgOpen Canonica</family>
 <family>AR PL SungtiL GB</family>
 <family>AR PL Mingti2L Big5</family>
 <family>FreeSerif</family>
 <family>Baekmuk Batang</family>
 </prefer>
 </alias>
 <alias>
 <family>sans-serif</family>
 <prefer>
 <family>DejaVu Sans</family>
 <family>Verdana</family>
 <family>ＭＳ ゴシック</family>
 <family>IPAPGothic</family>
 <family>Sazanami Gothic</family>
 <family>Kochi Gothic</family>
 <family>Bitstream Vera Sans</family>
 <family>Arial</family>
 <family>Albany AMT</family>
 <family>Luxi Sans</family>
 <family>Nimbus Sans L</family>
 <family>Helvetica</family>
 <family>Nachlieli</family>
 <family>MgOpen Moderna</family>
 <family>AR PL KaitiM GB</family>
 <family>AR PL KaitiM Big5</family>
 <family>FreeSans</family>
 <family>Baekmuk Dotum</family>
 <family>SimSun</family>
 </prefer>
 </alias>
 <alias>
 <family>monospace</family>
 <prefer>
 <family>DejaVu Sans Mono</family>
 <family>Courier New</family>
 <family>ＭＳ ゴシック</family>
 <family>IPAGothic</family>
 <family>Sazanami Gothic</family>
 <family>Kochi Gothic</family>
 <family>Bitstream Vera Sans Mono</family>
 <family>Andale Mono</family>
 <family>Cumberland AMT</family>
 <family>Luxi Mono</family>
 <family>Nimbus Mono L</family>
 <family>Courier</family>
 <family>Miriam Mono</family>
 <family>FreeMono</family>
 <family>AR PL KaitiM GB</family>
 <family>Baekmuk Dotum</family>
 </prefer>
 </alias>
 <match target="font" >
 <edit mode="assign" name="embeddedbitmap" >
 <bool>true</bool>
 </edit>
 </match>
</fontconfig>

So, save the file and reboot xwindows (CTLR+ALT+Backspace). Now with any luck the order of fonts should have been updated so that the default Japanese type face is actually a clean one first and foremost instead of the ugly first serving. Also it enables the built in bitmap font which can really make kanji's more readable and also enables the bitmap version of hiragana and katakana so that they don't look blurry anti-aliased next to clear bitmap kanjis. For most people this setting will be fine. If you're not happy, by all means leave out the embeddedbitmap setting or change it to false.

To finish things off, I'd suggest making sure in System / Preferences / Appearance / Fonts, you've got subpixel smoothing on and after clicking on details, hinting is set to 'full'.

If you have any additional questions, please don't hesitate to message me in this forum. I'm always happy to help, though I make take a little while responding. 頑張ってください - RyuKent

[tacutu]

thanks for the tutorial.
however, there's something I don't understand: what exactly do we gain by adding the ubuntu-ja repository? Is there any software which is not already available in the ubuntu repos?

[ryukent]

Adding the repos was in previous versions of this guide as they contained fonts that weren't in the normal repos. Some are no longer available, so I've omitted that section. All should work fine without the 'ja' repository, however I find that updates to the Japanese system are released faster in this repository. Also there are a number of other useful programs that you can only find there. In order to keep the guide a complete Japanese setup tutorial, I thought it necessary to leave that section in.

[Zorael]

Awesome tutorial, many thanks. Bitmaps were just the thing I was looking for. (...though by chance happened to enable earlier after furious googling.)

質問!

• Are these Ricoh fonts exact copies of the ones found on a Windows system? Or new-and-improved versions? Because I nicked my whole font directory from my Windows partition, which got me all kinds of peculiar fonts, including MS Mincho and Gothic.
  
  
• After adding the repository, should I install any extra hitherto-deselected-or-unavailable packages? I see it updated scim, anthy and the bridge packages, which is always nice.
  
  
• I made a symlink towards my ~/fonts.conf file at /etc/fonts/fonts/conf.d/99-zor.conf. Will this make the setting system-wide? Should it perhaps be named 01-zor.conf? I figured the higher the number the later it would be included, and as such override any previous settings.
  
  

• Semi-relevant, but how do I enter the Japanese quotation signs? Entering normal " quotes just gives me the suggestions ”, “ and the unmodified ".

  edit: Pah, found it; [ and ] becomes 「 and 」.
  
  
• Any further steps to take for us Kubuntulings? On a side note, I didn't seem to have a .fonts.conf file to begin with, unless my memory is sorely failing me.
  
  
• Must the .fonts.conf entry be ＭＳ 明朝 and ゴシック or does MS Mincho and Gothic suffice? (More out of interest.)
  
  
• The limit at which it stops using bitmaps seems to be at 16px (vectoring above). Is there a way of altering this? (Also out of interest.)
  
  
• Is there a way to get scim enabled at the login screen? Likely not? (Yep, interest.)

[TokyoYank]

RyuKent, you are an asset to the ubuntu and linux community, thanks. (I'll also soon be posting a question to the effective TangoBlaster thread.)

However, some comments:

I was able to get SCIM to work for Japanese input on my English installation simply through System > Administration > Langauge Support , however to date there may have been varying degrees of success with that method: http://ubuntuforums.org/showthread.php?t=783036

Second, do you use a handwriting recognition program to look up kanji? The very useful program tomoe is not in my default Hardy repo's for some reason. (I'm located in Japan, thus use the Japan servers) ..
To be honest, this is one aspect of Hardy which is pushing me back towards Fedora..

http://ubuntuforums.org/showthread.php?t=783673 tomoe and kanjipad

Any thoughts or comments much appreciated

[tacutu]

Browsing the repo recommended by Ryukent I noticed that tomoe is available:
http://archive.ubuntulinux.jp/ubuntu-ja/hardy/

[TokyoYank]

.. Also there are a number of other useful programs that you can only find there. In order to keep the guide a complete Japanese setup tutorial, I thought it necessary to leave that section in.

Appears that tomoe is an example of the discrepancy. And I can verify that "ubuntu-ashisuto.ubuntulinux.jp/ubuntu" does not have it--just because it's a server in japan does not mean it's a ubuntu-ja server I guess.

Any idea why this behavior exists? Am I alone in thinking that the GUI for adding Japanese language support should also add a ubuntu-ja repo also (or at least give a warning about it)

My only guess so far is that the number of users like us who need multi-language support--different from the session locale, I mean--is small, so that we pass under the ubuntu admin radar..


New Topic:

I started thinking about a way to make the instructions GUI only, without involved command line settings. The eventual aim would be instructions to incorporate into an official Hardy How-To document, but I didn't get too far before needing CLI again.

Here's how I updated my repositories and installed tomoe using the GUI

• System > Administration > Software Sources > Third-Party Software > Add
• Text entry box, use the OP entry:
  deb http://archive.ubuntulinux.jp/ubuntu-ja hardy/
  Get a warning about this repository is NOT AUTHENTICATED. OK, can ignore that for now.
• System > Administration > Synaptic Package Manager > Search
  ubuntu-ja-keyring
  Click on box to Mark for Installation, same authentication error, can ignore. ==> click "Apply"
• .. to be honest.. after this point I'd like someone else to confirm the instructions. It's hard for me to go back to a clean install to test this. In Synaptic click "Reload"
• "Search" for tomoe. Mark scim-tomoe for installation, and allow other required packages.

At this point the handwriting-ja.xml should be copied manually. .. perhaps there should/could be a tomoe-scim-compat-multilang (or something) package that does this?

[ryukent]

Sean,

Did you install Hardy from a Hardy CD or is it an upgrade from Gutsy? All the tests I've done are from a fresh install and it is possible old settings somewhere could be interrupting.

Also, as tacutu pointed out, im-switch could help. It was in old versions of my howto, but hardy runs it by default now.

Try running:

im-switch -l

It should say your configuration is pointing to something. This something should be scim-bridge. If it's still not working, can you post the output.


Tokyoyank,

I'm pretty sure us multilinguals get ignored. I've really noticed that there's a huge assumption that the only people who would ever be typing in Japanese are Japanese themselves who would have their entire system in Japanese locale. In my opinion there should be no difference between the repositories. If a newer version of software is released and it has improvements, then why should everyone's system not be updated? Why should software be limited to certain regions? Do people not travel? It's been like this for years.

Plus, why do programmers write software for certain locales? Should not all software run on all ubuntus?? I fully understand the localisation issue as far as language translation is concerned, but beyond that there is no excuse. In my opinion, when you go into the language selection dialogue box and click Japanese (or any other language) on, it should install everything. If fonts or input methods are needed, then of course they should be installed. Who the hell wants half their applications to not work correctly or who wants a partially crippled system with mismatching display?

The day I stop writing in these kind of threads will be the day I consider Ubuntu to truly be worthy as a mainstream OS. I love Ubuntu and will continue to support it fanatically, but I can really understand why some people can't deal with the 'issues'. Until even idiots can set up their system, people will shun linux as an enthusiasts only system. It is not acceptable for this community to look down on people who can't manage complicated setup. It is our responsibility to make it simple for them.

[tacutu]

Venting your frustration, aren't you?
Seriosusly now, I totally agree with you. There shouldn't be any need for "special" repos. Moreover, the said repos are mentioned only on the Japanese page on the Ubuntu website. (which could be a problem for a beginner who could not read so much Japanese but still wants IME and such for learning purposes)

Maybe we should signal the situation as a bug?

[Champy]

i'm trying to get this to work.

i added deb 'http://archive.ubuntulinux.jp/ubuntu-ja jaunty/' to my repositories.

but when I'm in the Synapic Package Manager and i search for 'ubuntu-ja-keyring' i don't get any results. is there a different repository i should add since i'm on 64 bit jaunty jackalope?

I'm new to Ubuntu so I'm having a bit of trouble.