Font Rendering Project - Contributions & Opinions
 

Greetings Everybody :D

This is my first post on Linux Questions and as I was used to posting on BBS a long while ago please take it easy on me; I'm not too familiar with current forum etiquette.

I want to start a few small projects to

a) solve some long standing Linux annoyances
b) make it easy for the 'average joe' to be more involved in making Linux better.

Now, I've not really been much of a GUI user in the past, I enjoy my programming and it's always been functional rather than pretty. After realizing what year it is I decided to trial some of the GUI's for Linux - KDE, XFCE, Gnome, Enlightenment. All seem to have their good points and this is not an attempt at a flame war, so please don't start one. I finally rested on XFCE as it seemed simple enough to work with (dependencies, interface, speed etc.). This is my own personal preference; I don't care which environment you use, we are all working toward the same goal, which brings me to my first project - Font Rendering.

I'm not necessarily looking to rewrite the whole of Linux' font rendering system. What I want to do is use existing software and resources out there to make users displays more readable, easier on the eye. I have seen many, many posts regarding this but they are spread out across the internet and difficult to understand - one persons opinion is converted to a 'give me Mac fonts' tutorial that goes wrong somewhere and your display suddenly becomes very much like that of a ZX Spectrum.

So what do I want here? I want as many LQers as possible to contribute their opinions and knowledge about font rendering on this thread. Lets bring it all into one place, regardless of distro, regardless of favourite environment, no bias unless it's truly warranted. As we progress I will then start up some tutorials and articles about improving the current GUI's. I will also post this to a website that is not forum based along with other projects.

Please contribute, whatever your level of experience with Linux. If you feel your favourite distro has great rendering for example, we can start looking at what makes it work well. If you've lots of experience then help us write scripts etc. so the newbies amongst us can benefit from the work too.

Thanks everyone, if this project starts well, I'll post another pretty soon. I'll announce the website address when the contributions start coming in.

Hm, a good idea to think about it - and even better that you do want to collect data before spending efforts.

I will just write a bit of mostly-well-known things as a starting point.

There is a good font format for Linux fonts, it is called OTF - OpenType Fonts, although TTF support is also OK. There is a good tool to edit/convert fonts in different formats - FontForge. Both OTF and TTF seem to be supported well by FreeType2 library. It should be used together with fontconfig library. Font search directories for the latter are configured using /etc/fonts/fonts.conf ; if a directory with fonts you need is not mentioned inside a <dir></dir> tag pair in the file, something goes wrong (ways of correction depend on distribution; sometimes direct editing works, sometimes it can get overwritten). There is also Pango library, also used for font rendering.

GTK2 contains and uses Pango for text output. Qt depends on FreeType.

Applications that actually want to control rendering often use these libraries directly, like it is done by Gecko and Webkit rendering engines.

You need to use good fonts. There are, for example, RedHat Liberation fonts, Linux Libertine (a good font, with some spices - it uses many OpenType features, and some of them only once), and DejaVu fonts.

If you want to create a document which makes best use of the fonts you have, you have to do it in PDF format, probably.. For that, you can use XeLaΤεΧ (with obvious requirement to be able to use LaΤεΧ) or Scribus (I saw claims that they are working hard on using FreeType to its maximum since quite long ago, but I haven't checked their current progress).

By the way, the font question has many faces. First, there is the question of individual glyph quality. It depends only on font (if we do not consider PDF renderer which renders fonts regardless of pixels and then honestly calculates intersection percent..). Second, there is a question of glyph interaction. Like kerning. It can be ignored by application even when the font supports it. Third, there is a question of coverage, but it is not that important for those who need just LGC (Latin, Greek, Cyrillic). What is lacking, it is a free software font to provide some similar-enough glyphs for all the Unicode standard. Or at least a sane list of 30 fonts that are enough in total. Currently this information is too spread to understand what is total available coverage of a given less-known part of Unicode.

You can test quality of your application's font rendering by a very simple experiment. Set large font size and DejaVu Serif font family. Enter "ff ffi fi fh fo" string. ffi should be one symbol. They must be fused into one glyph. Second f should have different form. Firefox 3 passes such a test; OpenOffice fails it.

It is certainly nice to create a HOWTO on font-related issues, but I don't see where you are haeding at with a "Font rendering" project.

FreeType does a pretty good job at rendering and can be found on most UNIX flavours. Supporting FreeType development with code is great, but rewriting something similar from scratch would be rather an academic exercise with probably not much practical impact.

Based on your description it seems to me that your project rather tends towards font management issues, which is quite different from font rendering itself, even though both are related to fonts. It would be nice to have some font management tool that would work on all distros and all window managers. Then again - installing and un-installing fonts isn't overly complicated currently anyway. The fact that OpenOffice displays and handles fonts differently in certain aspects from other applications has to do with their codebase. It will not be enough to provide some new code library, you will also have to convince the relevant software projects to use that library.

Anyway, my wishlist for a font management application would be:

- installing/removing fonts on system-level (as root)
- doing the same on a user level (in the home directory)
- give feedback on meta data contained within the font
- give feedback on font variants within the font
- display character glyphs or bitmaps of the font (depending on font format)
- display encoding used in the fonts
- convert font formats, encodings (at least the most common ones)
- convert a glyph or a line of text to outline graphics (EPS, PDF)
- define sets of fonts that can be enabled/disabled as a group
- support for PostScript fonts (Type1, Type3, Type42), TrueType, OpenType, Speedo, some bitmap formats
- allow mixed usage of UNIX, Macintosh and Windows fonts (maybe internal conversion)

Linux Archive

As far as I understand, there is a global problem ‘‘Why some applications in some distributions render fonts poorly?’’ and some will to make it go away.

I agree that the code to render fonts well is already written, so if you want to help by code, you need to throw some patches for better integration of existing rendering solutions in existing major applications. And if you would convince any project to accept patches for switching on individual OpenType features, it would be nice (To see what I mean, open Linux Libertine Regular in FontForge, press Ctrl-k to open metrics window or press windows->metrics, look at the left side at the list of OT features, try swiching them on/off and look how ffi, ct or ss may be rendered differently; there are also Qu and 0 to play with).

Yes, and when writing a tool like FranDango described it should be remembered that you want it to have most code in the GUI. FontForge and fontconfig are here, but if you make a wrapper that makes trivial things trivial (for wide spectrum of trivial things...) it will be very useful.

I don't think my Linux experience is particularly basic but I have terrible problems trying to get text to display well on my LCD. When I have followed other tutorials on the net to get good looking fonts the 'rainbow effect' always kicks in. Anti-aliasing, hinting, and subpixel hinting all require such fine adjustment that I felt this project necessary.

As I said, this project doesn't need to be a programming project. If the right tools can come together in combination with the right configurations, and there is a simple, universal way to implement them then we have achieved a good result.

What is great about the last three posts is that we now have areas to concentrate on and work from which at least I had no idea about. Sometimes it's just so hard to find out about stuff that is in current use, even before you start thinking about how it could be better.

KhipuX: what pixel size of letters do you try to get, what fonts are you using, what application versions? Because I am writing it from my notebook with quite a lot of fonts set up, and I do not remember anything about rainbow problems with antialiasing. But big zoom shows that rainbow antialiasing is really used. Maybe your system has wrong idea of your monitor?

Sorry about the 'vacation' guys, not had internet for a couple of weeks..... can you imagine????

Anyway, I've had a full day of poking around the various Linux font threads and a few sites that offer solutions. Here is some of the stuff I've found out and thought about:

- Microsoft has a patent on some technology that renders truetype fonts. Not a huge disaster as it seems like it's related to Cleartype which, in my opinion, looks pretty junk. I have tried Cleartype in Vista and XP and can't get rid of the color-fringe. This is a bigger headache causer for me than uneven lines and bluriness.

- I checked out every option I could within Kubuntu (for KDE's ease of use) for sub-pixel hinting and anti-aliasing. Again, the color-fringing was so bad most of the time it looked like my text was walking.

- Since I have had problems trying to get fonts standardised throughout any distro I'm using, I like the idea of some font management software as FranDango mentioned. It's frustrating to see different fonts used on the same menus in different applications. But I still want to concentrate on the quality of onscreen text.

- Every 'solution' I have come across so far just talks about patching the sub-pixel hinting. I took a look at how this technology works and there doesn't seem to be much that can be changed apart from very fine adjustment of the pixel clock to reduce color-fringing. My technological expertise isn't good enough to understand how to do this so maybe someone else can help here.

- Whilst trying to understand and tryout some sub-hinting methods I had created some test graphics in the Gimp. By mistake I created a black and white image of some small print text which I had smoothed with greyscales (thought I was working in color). The text looked very sharp and very smooth. I managed to create the same effect with blue text and red text using just different shades of the base color. This brings me to thing that there might be a better way of anti-aliasing characters. I didn't necessarily want to start a programming project but this might be a way forward. It would work on CRT screens as well which we have plenty of where I live.

- Firefox and Safari render fantastic text on web pages. Strangely, the text in Firefox's search bars and the menus is awful on my system. If it can be rendered well on a web page then why not on the rest of the system?

I don't know if any of what I said is going to help but it's food for thought and it'll bump this thread which could do with some more contributions. Thanks guys for your posts so far, once again sorry about the response delay.

About Firefox: it most probably uses Freetype for webpage rendering, and uses it correctly. Unfortunately, menus are left to GTK, which (if fresh enough..) uses Pango and probably doesn't choose its best options.

About patents: Microsoft ClearType patents are not the worst part; TrueType hinting patents held by Apple cover more useful things (in USA).

Raskin, what do you think about concentrating on the anti-aliasing, is it worth chasing up?

Also, does Apple's Truetype hinting look better than Microsoft's? Does the hinting only apply to Truetype, or does MS's patent prevent the base technology from being researched before we even get to hinting?

I don't have access to much Apple stuff, we don't have stores anywhere in the country I live in that sells them so I can't go and look. Even more so, I can't find Apple font information on the net.

If the anti-aliasing is worth following I'll make a few improvements to my graphics knowledge and programming skills.

As for anyone else who reads this thread, can you please leave your opinions and if possible a description of how you 'see' text on your Linux screen. This is quite important, if only to grab data that will take too much time for two people to do.

Many thanks.....

Actually, it's not that difficult to configure Linux with proper subpixel hinting and several other nice font options. Even though some stuff is patented, it's in the code and just not enabled from most distributions.

And pango ist for the unicode rendering - it still uses freetype as (one possible) underlying font engine and renders - in most contemporary applications - via Cairo.

I use a certain setting (see this thread - it's not as Slackware specific as it looks, all patches and settings should work on any distribution..) Essentially, it's a patch for Freetype, another for Cairo and some enabling in Xft and there you go with better font rendering. And it makes a HUGE difference.

AFAIK the difference in rendering the menu of Firefox and rendering the webpage itself is due to different styling and displaying mechanisms: until FF2, the menu and stuff came from FF and did only obey a minimal set of Gtk styling options while the webpage inside was rendered with full support of everything Cairo can do (including SVG). This is different with FF3 which obeys fully Gtk theming and if compiled accordingly, will render via your system's Gtk/Cairo/Pango - menu AND webpages themselves. But I may be wrong here, I just looked deeper in FF3, not in FF2.

As far as I know, Microsoft holds some patents that can be ignored. Apple holds relatively honest patents - they patented only a few non-trivial instructions in hinting evaluation VM for TrueType. Unfortunately, most fonts do use these instructions; so Freetype had to also implement auto-hinting.

As for how Apple's own systems look like, there is more in font quality. Currently the best format for fonts is, probably, OpenType.. And some Apple fonts work as showcases for every bit of beauty you can squeeze out of this format. And, obviously, they will always ensure that their subpixel optimization works OK with their monitors.

I can see FF's subpixel rendering on magnified screenshots, and when I look at it on the screen, it just works; and it works nicely. On the other hand, I have nothing against even poor OO.o rendering, but the fact that it fails to make an "ffi" ligature is a pity. The funny thing is that WebKit-GTK, which is port of Webkit, uses Freetype somewhere, passes Acid3 and what not also fails to display a trivial test of ligature support. And yes, I made the test valid XHTML Strict.

By the way, I must say that just choosing a good font family (of course you need sane rendering, but just sane, not particulary good) can really make a difference.

Su-Shee, can you tell me if the solution you provided is already included in Kubuntu 8.04.1? I tried the solution and couldn't get any different effects compared to just playing with the options in the Ksetup. Maybe there is something I'm not doing.

I must admit I'm a little confused as to what needs to be done to use the 'patent' code. So many packages seem to be involved and there is no clear information on the net (Pango, Cairo, Freetype, libXft2 etc.). There's also no indication of what effects can be expected with various solutions - lots of people are still complaining about the font rendering after trying numerous solutions but their expectations will be different I think.

...............

Okay, I've been writing the post in between fiddling for the last three hours, thus the row of periods. Anyway, I seem to have got some reasonable results by switching off the bytecode thingy and using decent fonts as Raskin advised. Only problem is, only certain areas of text have improved. The text in Firefox 3.0's content is great as is my Kmenu (Kubuntu Hardy). The text in this quick reply box is also very clear. However, the menus in my applications are garbage, they look like they're printed on blotting paper. The same is also for the text in OpenOffice documents. Why can't the fonts be rendered thoughout menu's and applications evenly?

I don't know, I don't use Ubuntu and family.

As those are just three patches, you can easily apply them by yourself and compile it.

The patented stuff (in this specific case) is mostly about TrueType rendering mechanisms and in the code included, but disabled, that's all.

"Fonts" under Linux are (nowadays) handled by a family of tools: X.org with the Xft-library for the basic and essential "fonts on X", pango does the Unicode handling of fonts and freetype is the rendering frontend which can delegate the rendering to Cairo (or to the Mac font system or Windows font libs and so on). For the "organization" of fonts, you'll have your fontconfig stuff with the config file fonts.conf and ~./fonts.conf.

And that lies under most contemporary applications with Gtk as widget set or windowmanagers like fvwm or desktops like Gnome and Xfce. How exactly Qt does the font handling, I simply don't know.

All this means: If you want to adjust your fonts nicely, you set config options for fontconfig in your .fonts.conf after you patched certain rendering and displaying capabilities in Xft, Freetype and Cairo.

That's it.

(Besides of using a good font..)

Addition: This is what you want to read and patch.

Cracking post Su-Shee. This has also given me some idea of how all the packages are arranged to provide font rendering.

I'm going to play about with these patches and adjustments for a few days and see what I can come up with. A man in a nearby print shop is running FreeBSD and has the same monitor as me. His fonts are just amazing, so Ive arranged to speak with the person that set up the computer. I will post back when I've heard from him.

Thanks guys.