ICU4J in Eclipse

Doug Felt, IBM

dougfelt@us.ibm.com

What is ICU4J?

Why make ICU4J available in Eclipse?

What is going into Eclipse?

How will ICU4J change?

How will Eclipse change?

How will client code change?

What does it look like?

When will it be available?

Overview

What is ICU4J?

Internationalization library

Sister project of ICU, open source, sponsored by IBM

Unicode Standard compliant, up-to-date

100% Pure Java

Enhances and extends Java functionality

Over six years of continuous development

Based on the original code contributed to Sun

Why make ICU4J available in Eclipse?

Because developers use Eclipse...

ICU supplies CLDR locale data

Over 230 locales (Java 1.5: ~130, Windows XP: ~160)

ICU supports current Unicode Standards

UCA Collation (plus smaller keys, faster comparison)

Character data (4.1.0)

ICU provides additional functionality

Normalization, Calendars, Formatting

What is going into Eclipse?

An ICU4J plugin

All of ICU4J

Modified to run on JDK 1.3, Foundation 1.0 APIs

Data broken into fragments for customizability

An alternative 'thin wrapper' ICU4J plugin

Small subset of ICU4J APIs

Forwards calls to Java implementation, no ICU data

Projects can code to subset without requiring full ICU jar

The full ICU4J plugin

All API classes from ICU4J

Minus a few methods on some classes

Eclipse requires 1.3, Foundation 1.0 compatibility from us

No post-1.3 APIs (e.g. CharSequence, ByteBuffer)

No java BigDecimal (not included in Foundation 1.0)

Modular data

Not modularized by function, unlike standard ICU4J

Data in Eclipse 'fragments' clustered by region (TBD)

The wrapper ICU4J plugin

Forwards ICU4J APIs to Java implementation

Why?

Some Eclipse clients can't accept ICU4J's footprint

But want Eclipse to use ICU4J if available

Small subset of ICU4J APIs, no ICU data

13 classes: BreakIterator, Collator, Formatters...

Remove APIs on them that Java APIs don't support

Defines 'safe' API that all Eclipse code can use

Full API – part 1

com.ibm.icu.text

com.ibm.icu.math

ArabicShaping DictionaryBasedBreakIterator StringPrep

ArabicShapingException IDNA StringPrepParseException

BreakIterator MeasureFormat StringSearch

CanonicalIterator MessageFormat SymbolTable

CharsetDetector Normalizer Transliterator

CharsetMatch NumberFormat UCharacterIterator

ChineseDateFormat RawCollationKey UFormat

ChineseDateFormatSymbols Replaceable UForwardCharacterIterator

CollationElementIterator ReplaceableString UTF16

CollationKey RuleBasedBreakIterator UnicodeCompressor

Collator RuleBasedCollator UnicodeDecompressor

DateFormat RuleBasedNumberFormat UnicodeFilter

DateFormatSymbols SearchIterator UnicodeMatcher

DecimalFormat SimpleDateFormat UnicodeSet

DecimalFormatSymbols StringCharacterIterator UnicodeSetIterator

BigDecimal MathContext

Full API – part 2

com.ibm.icu.util

com.ibm.icu.lang

85 classes + all data, ~3Mb binary (~2Mb is data)

BuddhistCalendar GregorianCalendar SimpleDateRule

ByteArrayWrapper HebrewCalendar SimpleHoliday

Calendar HebrewHoliday SimpleTimeZone

CaseInsensitiveString Holiday StringTokenizer

ChineseCalendar IslamicCalendar TimeZone

CopticCalendar JapaneseCalendar ULocale

Currency LocaleData UResourceBundle

CurrencyAmount Measure UniversalTimeScale

DateRule MeasureUnit ValueIterator

EasterHoliday RangeDateRule VersionInfo

EthiopicCalendar RangeValueIterator

UCharacter UCharacterDirection UProperty

UCharacterCategory UCharacterEnums UScript

Wrapper API

com.ibm.icu.text

com.ibm.icu.util

13 classes, ~54K binary

BreakIterator	DecimalFormat

CollationKey	DecimalFormatSymbols

Collator	NumberFormat

DateFormat	SimpleDateFormat

DateFormatSymbols

Calendar	TimeZone

StringTokenizer	ULocale

How will ICU4J change?

Standard ICU4J API doesn't change

Eclipse version based on ICU4J 3.4.3

Can now run on JDK 1.3

3.4.3 uses ICU time zone data, so independent of JVM

Still requires JDK 1.4 for build, test

Can be built with “FOUNDATION” switch

Matches full version in Eclipse

Just removes some APIs that don't meet minimum req.

How will Eclipse change?

Eclipse uses com.ibm.icu APIs internally

Only wrapper API, implementation depends on plugin

Uses available ICU localization data

Doesn't expose ICU4J in APIs

ViewerSorter has some issues with Collator, TBD

Default Eclipse download contains full plugin

Wrapper version available from Eclipse site

Developers can use all API from ICU4J via the plugin

How will client code change?

Doesn't have to!

Eclipse RCP will use ICU4J for sorting, numbers, dates

Best to use ICU APIs to match Eclipse

Clients can customize their use of ICU

Include only subset of locale data fragments

Include the wrapper version instead of full version

Clients can use full ICU APIs directly

Just import a com.ibm.icu package, and call ICU

Add a dependency on ICU4J plugin to your plugin

Demo

Example: Date/Time Formatting

Left is ICU, right is JVM. Red boxes highlight additional locale support in ICU.

Example: Collation

The left table uses ICU to sort Japanese data conforming to the JIS X4061 standard. The right table uses Java collation to sort the same data (the red boxes highlight the nonstandard Java behavior).

When will it be available?

As of 8 Mar 2006:

Full jar (3.4.2) is already available in Eclipse 3.2M4

Wrapper jar, 3.4.3 to be available in Eclipse 3.2M6

Eclipse internal modifications TBD

How can you get it?

Download plugin from Eclipse site

Download Eclipse 3.2

Will work with Eclipse 3.1, but not supported

References

ICU:

http://www.ibm.com/software/globalization/icu

Eclipse:

http://www.eclipse.org

Java:

http://java.sun.com

http://www.ibm.com/java

CLDR:

http://www.unicode.org/CLDR

Q & A

Questions?

Providing ICU4J via an eclipse plugin makes it easier for developers who use Eclipse to obtain and use ICU.

CLDR provides data for over 230 locales. By comparison, Java 5 provides data for 94 locales (only 22 of which are fully supported) and Windows XP sp2 provides data for 159 locales.

ICU supports Unicode 4.1, including UCA collation. Java does not support UCA collation. ICU's collation generates smaller sort keys, and comparison of large data sets is thereby faster. The character data support in Java depends on the JRE, so it's only as current as your runtime. Most developers have more control over the version of ICU they include than the JRE their clients run on.

ICU provides additional functionality such as Unicode Normalization (NFC, NFD), support for several additional Calendar systems (Arabic, Hebrew, Japanese, Thai Buddhist), and enhancements to NumberFormat and DateFormat.

Two versions of the ICU plugin are being made available.

One is the full ICU4J plugin. This has all the classes that are in the standard ICU4J distribution (3.4.2). It runs on JDK 1.3 and only uses Foundation 1.0 APIs. Data is broken into fragments so that clients can reduce the footprint of the plugin.

The other is a wrapper around Java APIs. This provides only a small subset of the apis, but they are the commonly used ones. Since this implementation calls through to Java internally, it does not make use of ICU data. Projects can code to these APIs without commiting to the full ICU jar, which allows their clients to decide for themselves whether or not to use ICU4J's data.

The plugin contains all the API classes from ICU4J. It is essentially equivalent to the standard ICU4J jar.

But since Eclipse has required us to conform to Foundation 1.0 profile, some of the methods on those classes differ from the standard ICU4J release. In particular, methods that use classes introduced after JDK 1.3 are not present, and neither are methods that use classes that are not in Foundation 1.0 (like BigDecimal). This allows ICU4J to be used in the largest number of applications.

The plugin data has been modularized by region so that clients can choose to include or omit data that they will not be using. The standard ICU4J release provides all the data, and provides clients the ability to build (from the source) modules based on function instead of data. Both approaches are useful, but the data-based approach is easier to use with Eclipse so that's why we chose that.

The wrapper version of the plugin implements the ICU4J apis by forwarding on to their Java counterparts.

Why do this? Footprint. Some clients can't accept even the stripped-down size of ICU4J. But if Eclipse uses ICU4J apis internally, and required the full ICU4J jar, this would prevent these clients from using Eclipse. Hence the wrapper plugin. It allows Eclipse to be coded using ICU4J apis while not requiring the real ICU4J.

The APIs implemented in this fashion are a small subset of the ICU apis. Naturally, no ICU data is present, since none of the ICU implementation is used. These wrapper APIs are based on the ICU APIs, but omit the additions that ICU has made that Java (1.3) cannot support.

A list of the text and math API classes in ICU4J 3.4.2.

You'll note the similarity to the Java text package.

Even more API classes, this time from the util and lang packages.

The full ICU4J jar (including all data) is quite large, over 3Mb. About 2Mb of this is data. About 500K of that data is required by any base implementation of ICU. Thus the minimum ICU installation is about 1.5Mb.

The wrapper API, by way of contrast, is much smaller. It is so small as to be essentially free for most people.

These classes are the most frequently used classes from the java.text and java.util packages.

The standard ICU4J distribution from our web site doesn't change in any significant way. The Eclipse version is based on ICU4J 3.4.3 and is derived from the same sources (with some preprocessing to filter out unsupported APIs).

The standard ICU4J distribution can now run on JDK 1.3, with some care. Depending on the JVM, some classes may not load if they refer to 1.4 APIs. Most JVMs do not resolve such references unless the flow of control requires it.

Although ICU4J will run on JDK 1.3, build and test still requires JDK 1.4 or later.

It is possible to build ICU4J with a switch that enforces Foundation compatibility. The code that results matches the full version of the plugin that appears in Eclipse. This code will load on any 1.3 JVM. This is how the ICU4J full plugin is built.b

Eclipse has been modified in a hundred or so places to use com.ibm.icu.text and com.ibm.icu.util APIs internally. The APIs so used are only the wrapper APIs, as discussed previously, so that any Eclipse product can run without the real ICU4J code and data. If the full version of the ICU4J plugin is installed, ICU's localization data will be used.

Eclipse doesn't expose the ICU4J classes in its APIs (with the possible exception of ViewerSorter-- details are still being worked out). This means that people who call Eclipse APIs will not have to change their code, nor will they have to learn a new set of parallel APIs.

By default the Eclipse download will contain the full ICU4J plugin. This means all Eclipse developers will have access to ICU4J's code and data. Developers can also chose to download and use the wrapper version of the plugin. Since the plugin is just another Eclipse plugin, developers can chose to explicitly depend on it and call any ICU4J APIs they wish.

At the basic level, clients will not need to change their code at all. If they use Eclipse RCP widgets, these will use ICU4J (when present) for sorting and number/date formatting.

Since Eclipse will use ICU data if available, we recommend that developers sorting and formatting in their own code use the ICU4J APIs defined in the wrapper. This ensures the same behavior as Eclipse. By limiting themselves to the wrapper API, developers can avoid requiring the full ICU4J jar.

Clients can omit any data fragments they wish from the full ICU4J plugin if they want to reduce footprint. And of course they can choose to ship the wrapper with their application.

Clients can call ICU APIs directly, just by adding a dependency on the plugin and importing the APIs they wish. This is just like any other plugin in Eclipse.

Show the Eclipse environment with the ICU4J plugin.

An example of the difference in formatting behavior between ICU and Java (1.4).

An example of differences in collation between ICU and Java.

The ICU4J plugin is available now from the Eclipse site. It is part of the 3.2M4 milestone build. The wrapper version (at the time of this writing) will be part of the 3.2M5 milestone build. The target for modifications to Eclipse is also the 3.2M5 build, but that is still pending as of this writing.

Users of Eclipse 3.1 can also use the ICU4J plugin. Eclipse will not support (e.g. not accept bug reports) on this configuration, however.

Some handy references.

Doug Felt

2006-01-24T09:31:43