author Mike Hommey <>
Fri, 25 Jul 2014 07:14:21 +0900
changeset 195945 77804ecb85661104f3c39dca83bcd89b9bbb2824
parent 195902 9230143db3b778f3a248269c7e707d7f13489d8b
permissions -rw-r--r--
Bug 1043351 - Build ShowSSEConfig as a C++ unit test after it was forgotten in bug 894941 and move linker tests to avoid conflicts. r=mshal

Non-standard hyphenation

Some languages use non-standard hyphenation; `discretionary'
character changes at hyphenation points. For example,
Catalan: paral·lel -> paral-lel,
Dutch: omaatje -> oma-tje,
German (before the new orthography): Schiffahrt -> Schiff-fahrt,
Hungarian: asszonnyal -> asz-szony-nyal (multiple occurance!)
Swedish: tillata -> till-lata.

Using this extended library, you can define 
non-standard hyphenation patterns. For example:


or with narrow boundaries:


Note: Libhnj uses modified patterns by preparing
Unfortunatelly, now the conversion step can generate bad non-standard
patterns (non-standard -> standard pattern conversion), so using
narrow boundaries may be better for recent Libhnj. For example, generates a few bad patterns for Hungarian hyphenation
patterns resulting bad non-standard hyphenation in a few cases. Using narrow
boundaries solves this problem. Java HyFo module can check this problem.

Syntax of the non-standard hyphenation patterns


If this pattern matches the word, and this pattern win (see README.hyphen)
in the change region of the pattern, then pattern[start, start + cut - 1]
substring will be replaced with the "change".

For example, a German ff -> ff-f hyphenation:


or with expansion


will change every "ff" with "ff=f" at hyphenation.

A more real example:

% simple ff -> f-f hyphenation
% Schiffahrt -> Schiff-fahrt hyphenation


- Pattern: matching patterns of the original Liang's algorithm
  - patterns must contain only one hyphenation point at change region
    signed with an one-digit odd number (1, 3, 5, 7 or 9).
    These point may be at subregion boundaries: schif3fahrt/ff=,5,1
  - only the greater value guarantees the win (don't mix non-standard and
    non-standard patterns with the same value, for example
    instead of f3f and schif3fahrt/ff=f,5,2 use f3f and schif5fahrt/ff=f,5,2)

- Change: new characters.
  Arbitrary character sequence. Equal sign (=) signs hyphenation points
  for (like in the example). (In a possible German LaTeX
  preprocessor, ff could be replaced with "ff, for a Hungarian one, ssz
  with `ssz, according to the German and Hungarian Babel settings.)

- Start: starting position of the change region.
  - begins with 1 (not 0): schif3fahrt/ff=f,5,2
  - start dot doesn't matter: .schif3fahrt/ff=f,5,2
  - numbers don't matter: .s2c2h2i2f3f2ahrt/ff=f,5,2
  - In UTF-8 encoding, use Unicode character positions: össze/sz=sz,2,3
    ("össze" looks "össze" in an ISO 8859-1 8-bit editor). 

- Cut: length of the removed character sequence in the original word.
  - In UTF-8 encoding, use Unicode character length: paral·1lel/l=l,5,3
    ("paral·lel" looks "paral·1lel" in an ISO 8859-1 8-bit editor).

Dictionary developing

There hasn't been extended PatGen pattern generator for non-standard
hyphenation patterns, yet.

Fortunatelly, non-standard hyphenation points are forbidden in the PatGen
generated hyphenation patterns, so with a little patch can be develop
non-standard hyphenation patterns also in this case.

Warning: If you use UTF-8 Unicode encoding in your patterns, call with UTF-8 parameter to calculate right
character positions for non-standard hyphenation:

./ input output UTF-8


Use hyphenate2() or hyphenate3() to handle non-standard hyphenation.
See hyphen.h for the documentation of the hyphenate*() functions.
See example.c for processing the output of the hyphenate*() functions.

Warning: change characters are lower cased in the source, so you may need
case conversion of the change characters based on input word case detection.
For example, see source

László Németh
<nemeth (at)>