[ << ] [ >> ]           [Top] [Contents] [Index] [ ? ]

10. Producing Binary MO Files

10.1 Invoking the msgfmt Program

 
msgfmt [option] filename.po …

The msgfmt programs generates a binary message catalog from a textual translation description.

10.1.1 Input file location

filename.po …
-D directory
--directory=directory

Add directory to the list of directories. Source files are searched relative to this list of directories. The resulting binary file will be written relative to the current directory, though.

If an input file is ‘-’, standard input is read.

10.1.2 Operation mode

-j
--java

Java mode: generate a Java ResourceBundle class.

--java2

Like –java, and assume Java2 (JDK 1.2 or higher).

--csharp

C# mode: generate a .NET .dll file containing a subclass of GettextResourceSet.

--csharp-resources

C# resources mode: generate a .NET ‘.resources’ file.

--tcl

Tcl mode: generate a tcl/msgcat ‘.msg’ file.

--qt

Qt mode: generate a Qt ‘.qm’ file.

--desktop

Desktop Entry mode: generate a ‘.desktop’ file.

--xml

XML mode: generate an XML file.

10.1.3 Output file location

-o file
--output-file=file

Write output to specified file.

--strict

Direct the program to work strictly following the Uniforum/Sun implementation. Currently this only affects the naming of the output file. If this option is not given the name of the output file is the same as the domain name. If the strict Uniforum mode is enabled the suffix ‘.mo’ is added to the file name if it is not already present.

We find this behaviour of Sun's implementation rather silly and so by default this mode is not selected.

If the output file is ‘-’, output is written to standard output.

10.1.4 Output file location in Java mode

-r resource
--resource=resource

Specify the resource name.

-l locale
--locale=locale

Specify the locale name, either a language specification of the form ll or a combined language and country specification of the form ll_CC.

-d directory

Specify the base directory of classes directory hierarchy.

--source

Produce a .java source file, instead of a compiled .class file.

The class name is determined by appending the locale name to the resource name, separated with an underscore. The ‘-d’ option is mandatory. The class is written under the specified directory.

10.1.5 Output file location in C# mode

-r resource
--resource=resource

Specify the resource name.

-l locale
--locale=locale

Specify the locale name, either a language specification of the form ll or a combined language and country specification of the form ll_CC.

-d directory

Specify the base directory for locale dependent ‘.dll’ files.

The ‘-l’ and ‘-d’ options are mandatory. The ‘.dll’ file is written in a subdirectory of the specified directory whose name depends on the locale.

10.1.6 Output file location in Tcl mode

-l locale
--locale=locale

Specify the locale name, either a language specification of the form ll or a combined language and country specification of the form ll_CC.

-d directory

Specify the base directory of ‘.msg’ message catalogs.

The ‘-l’ and ‘-d’ options are mandatory. The ‘.msg’ file is written in the specified directory.

10.1.7 Desktop Entry mode operations

--template=template

Specify a .desktop file used as a template.

-k[keywordspec]
--keyword[=keywordspec]

Specify keywordspec as an additional keyword to be looked for. Without a keywordspec, the option means to not use default keywords.

-l locale
--locale=locale

Specify the locale name, either a language specification of the form ll or a combined language and country specification of the form ll_CC.

-d directory

Specify the directory where PO files are read. The directory must contain the ‘LINGUAS’ file.

To generate a ‘.desktop’ file for a single locale, you can use it as follows.

 
msgfmt --desktop --template=template --locale=locale \
  -o file filename.po …

msgfmt provides a special "bulk" operation mode to process multiple ‘.po’ files at a time.

 
msgfmt --desktop --template=template -d directory -o file

msgfmt first reads the ‘LINGUAS’ file under directory, and then processes all ‘.po’ files listed there. You can also limit the locales to a subset, through the ‘LINGUAS’ environment variable.

For either operation modes, the ‘-o’ and ‘--template’ options are mandatory.

10.1.8 XML mode operations

--template=template

Specify an XML file used as a template.

-L name
--language=name

Specifies the language of the input files.

-l locale
--locale=locale

Specify the locale name, either a language specification of the form ll or a combined language and country specification of the form ll_CC.

-d directory

Specify the base directory of ‘.po’ message catalogs.

To generate an XML file for a single locale, you can use it as follows.

 
msgfmt --xml --template=template --locale=locale \
  -o file filename.po …

msgfmt provides a special "bulk" operation mode to process multiple ‘.po’ files at a time.

 
msgfmt --xml --template=template -d directory -o file

msgfmt first reads the ‘LINGUAS’ file under directory, and then processes all ‘.po’ files listed there. You can also limit the locales to a subset, through the ‘LINGUAS’ environment variable.

For either operation modes, the ‘-o’ and ‘--template’ options are mandatory.

10.1.9 Input file syntax

-P
--properties-input

Assume the input files are Java ResourceBundles in Java .properties syntax, not in PO file syntax.

--stringtable-input

Assume the input files are NeXTstep/GNUstep localized resource files in .strings syntax, not in PO file syntax.

10.1.10 Input file interpretation

-c
--check

Perform all the checks implied by --check-format, --check-header, --check-domain.

--check-format

Check language dependent format strings.

If the string represents a format string used in a printf-like function both strings should have the same number of ‘%’ format specifiers, with matching types. If the flag c-format or possible-c-format appears in the special comment <#,> for this entry a check is performed. For example, the check will diagnose using ‘%.*s’ against ‘%s’, or ‘%d’ against ‘%s’, or ‘%d’ against ‘%x’. It can even handle positional parameters.

Normally the xgettext program automatically decides whether a string is a format string or not. This algorithm is not perfect, though. It might regard a string as a format string though it is not used in a printf-like function and so msgfmt might report errors where there are none.

To solve this problem the programmer can dictate the decision to the xgettext program (see section C Format Strings). The translator should not consider removing the flag from the <#,> line. This "fix" would be reversed again as soon as msgmerge is called the next time.

--check-header

Verify presence and contents of the header entry. See section Filling in the Header Entry, for a description of the various fields in the header entry.

--check-domain

Check for conflicts between domain directives and the --output-file option

-C
--check-compatibility

Check that GNU msgfmt behaves like X/Open msgfmt. This will give an error when attempting to use the GNU extensions.

--check-accelerators[=char]

Check presence of keyboard accelerators for menu items. This is based on the convention used in some GUIs that a keyboard accelerator in a menu item string is designated by an immediately preceding ‘&’ character. Sometimes a keyboard accelerator is also called "keyboard mnemonic". This check verifies that if the untranslated string has exactly one ‘&’ character, the translated string has exactly one ‘&’ as well. If this option is given with a char argument, this char should be a non-alphanumeric character and is used as keyboard accelerator mark instead of ‘&’.

-f
--use-fuzzy

Use fuzzy entries in output. Note that using this option is usually wrong, because fuzzy messages are exactly those which have not been validated by a human translator.

10.1.11 Output details

--no-convert

Don't convert the messages to UTF-8 encoding. By default, messages are converted to UTF-8 encoding before being stored in a MO file; this helps avoiding conversions at run time, since nowadays most locales use the UTF-8 encoding.

--no-redundancy

Don't pre-expand ISO C 99 <inttypes.h> format string directive macros. By default, messages that are marked as c-format and contain ISO C 99 <inttypes.h> format string directive macros are pre-expanded for selected platforms, and these redundant expansions are stored in the MO file. These redundant expansions make the translations of these messages work with the gettext implementation in the libc of that platform, without requiring GNU gettext's libintl. The platforms that benefit from this pre-expansion are those with the musl libc.

-a number
--alignment=number

Align strings to number bytes (default: 1).

--endianness=byteorder

Write out 32-bit numbers in the given byte order. The possible values are big and little. The default is little.

MO files of any endianness can be used on any platform. When a MO file has an endianness other than the platform's one, the 32-bit numbers from the MO file are swapped at runtime. The performance impact is negligible.

This option can be useful to produce MO files that are optimized for one platform.

--no-hash

Don't include a hash table in the binary file. Lookup will be more expensive at run time (binary search instead of hash table lookup).

10.1.12 Informative output

-h
--help

Display this help and exit.

-V
--version

Output version information and exit.

--statistics

Print statistics about translations. When the option --verbose is used in combination with --statistics, the input file name is printed in front of the statistics line.

-v
--verbose

Increase verbosity level.

10.2 Invoking the msgunfmt Program

 
msgunfmt [option] [file]...

The msgunfmt program converts a binary message catalog to a Uniforum style .po file.

10.2.1 Operation mode

-j
--java

Java mode: input is a Java ResourceBundle class.

--csharp

C# mode: input is a .NET .dll file containing a subclass of GettextResourceSet.

--csharp-resources

C# resources mode: input is a .NET ‘.resources’ file.

--tcl

Tcl mode: input is a tcl/msgcat ‘.msg’ file.

10.2.2 Input file location

file

Input .mo files.

If no input file is given or if it is ‘-’, standard input is read.

10.2.3 Input file location in Java mode

-r resource
--resource=resource

Specify the resource name.

-l locale
--locale=locale

Specify the locale name, either a language specification of the form ll or a combined language and country specification of the form ll_CC.

The class name is determined by appending the locale name to the resource name, separated with an underscore. The class is located using the CLASSPATH.

10.2.4 Input file location in C# mode

-r resource
--resource=resource

Specify the resource name.

-l locale
--locale=locale

Specify the locale name, either a language specification of the form ll or a combined language and country specification of the form ll_CC.

-d directory

Specify the base directory for locale dependent ‘.dll’ files.

The ‘-l’ and ‘-d’ options are mandatory. The ‘.msg’ file is located in a subdirectory of the specified directory whose name depends on the locale.

10.2.5 Input file location in Tcl mode

-l locale
--locale=locale

Specify the locale name, either a language specification of the form ll or a combined language and country specification of the form ll_CC.

-d directory

Specify the base directory of ‘.msg’ message catalogs.

The ‘-l’ and ‘-d’ options are mandatory. The ‘.msg’ file is located in the specified directory.

10.2.6 Output file location

-o file
--output-file=file

Write output to specified file.

The results are written to standard output if no output file is specified or if it is ‘-’.

10.2.7 Output details

--color
--color=when

Specify whether or when to use colors and other text attributes. See The --color option for details.

--style=style_file

Specify the CSS style rule file to use for --color. See The --style option for details.

--force-po

Always write an output file even if it contains no message.

-i
--indent

Write the .po file using indented style.

--strict

Write out a strict Uniforum conforming PO file. Note that this Uniforum format should be avoided because it doesn't support the GNU extensions.

-p
--properties-output

Write out a Java ResourceBundle in Java .properties syntax. Note that this file format doesn't support plural forms and silently drops obsolete messages.

--stringtable-output

Write out a NeXTstep/GNUstep localized resource file in .strings syntax. Note that this file format doesn't support plural forms.

-w number
--width=number

Set the output page width. Long strings in the output files will be split across multiple lines in order to ensure that each line's width (= number of screen columns) is less or equal to the given number.

--no-wrap

Do not break long message lines. Message lines whose width exceeds the output page width will not be split into several lines. Only file reference lines which are wider than the output page width will be split.

-s
--sort-output

Generate sorted output. Note that using this option makes it much harder for the translator to understand each message's context.

10.2.8 Informative output

-h
--help

Display this help and exit.

-V
--version

Output version information and exit.

-v
--verbose

Increase verbosity level.

10.3 The Format of GNU MO Files

The format of the generated MO files is best described by a picture, which appears below.

The first two words serve the identification of the file. The magic number will always signal GNU MO files. The number is stored in the byte order used when the MO file was generated, so the magic number really is two numbers: 0x950412de and 0xde120495.

The second word describes the current revision of the file format, composed of a major and a minor revision number. The revision numbers ensure that the readers of MO files can distinguish new formats from old ones and handle their contents, as far as possible. For now the major revision is 0 or 1, and the minor revision is also 0 or 1. More revisions might be added in the future. A program seeing an unexpected major revision number should stop reading the MO file entirely; whereas an unexpected minor revision number means that the file can be read but will not reveal its full contents, when parsed by a program that supports only smaller minor revision numbers.

The version is kept separate from the magic number, instead of using different magic numbers for different formats, mainly because ‘/etc/magic’ is not updated often.

Follow a number of pointers to later tables in the file, allowing for the extension of the prefix part of MO files without having to recompile programs reading them. This might become useful for later inserting a few flag bits, indication about the charset used, new tables, or other things.

Then, at offset O and offset T in the picture, two tables of string descriptors can be found. In both tables, each string descriptor uses two 32 bits integers, one for the string length, another for the offset of the string in the MO file, counting in bytes from the start of the file. The first table contains descriptors for the original strings, and is sorted so the original strings are in increasing lexicographical order. The second table contains descriptors for the translated strings, and is parallel to the first table: to find the corresponding translation one has to access the array slot in the second array with the same index.

Having the original strings sorted enables the use of simple binary search, for when the MO file does not contain an hashing table, or for when it is not practical to use the hashing table provided in the MO file. This also has another advantage, as the empty string in a PO file GNU gettext is usually translated into some system information attached to that particular MO file, and the empty string necessarily becomes the first in both the original and translated tables, making the system information very easy to find.

The size S of the hash table can be zero. In this case, the hash table itself is not contained in the MO file. Some people might prefer this because a precomputed hashing table takes disk space, and does not win that much speed. The hash table contains indices to the sorted array of strings in the MO file. Conflict resolution is done by double hashing. The precise hashing algorithm used is fairly dependent on GNU gettext code, and is not documented here.

As for the strings themselves, they follow the hash file, and each is terminated with a <NUL>, and this <NUL> is not counted in the length which appears in the string descriptor. The msgfmt program has an option selecting the alignment for MO file strings. With this option, each string is separately aligned so it starts at an offset which is a multiple of the alignment value. On some RISC machines, a correct alignment will speed things up.

Contexts are stored by storing the concatenation of the context, a <EOT> byte, and the original string, instead of the original string.

Plural forms are stored by letting the plural of the original string follow the singular of the original string, separated through a <NUL> byte. The length which appears in the string descriptor includes both. However, only the singular of the original string takes part in the hash table lookup. The plural variants of the translation are all stored consecutively, separated through a <NUL> byte. Here also, the length in the string descriptor includes all of them.

The character encoding of the strings can be any standard ASCII-compatible encoding, such as UTF-8, ISO-8859-1, EUC-JP, etc., as long as the encoding's name is stated in the header entry (see section Filling in the Header Entry). Starting with GNU gettext version 0.22, the MO files produced by msgfmt have them in UTF-8 encoding, unless the msgfmt option ‘--no-convert’ is used.

Nothing prevents a MO file from having embedded <NUL>s in strings. However, the program interface currently used already presumes that strings are <NUL> terminated, so embedded <NUL>s are somewhat useless. But the MO file format is general enough so other interfaces would be later possible, if for example, we ever want to implement wide characters right in MO files, where <NUL> bytes may accidentally appear. (No, we don't want to have wide characters in MO files. They would make the file unnecessarily large, and the ‘wchar_t’ type being platform dependent, MO files would be platform dependent as well.)

This particular issue has been strongly debated in the GNU gettext development forum, and it is expectable that MO file format will evolve or change over time. It is even possible that many formats may later be supported concurrently. But surely, we have to start somewhere, and the MO file format described here is a good start. Nothing is cast in concrete, and the format may later evolve fairly easily, so we should feel comfortable with the current approach.

 
        byte
             +------------------------------------------+
          0  | magic number = 0x950412de                |
             |                                          |
          4  | file format revision = 0                 |
             |                                          |
          8  | number of strings                        |  == N
             |                                          |
         12  | offset of table with original strings    |  == O
             |                                          |
         16  | offset of table with translation strings |  == T
             |                                          |
         20  | size of hashing table                    |  == S
             |                                          |
         24  | offset of hashing table                  |  == H
             |                                          |
             .                                          .
             .    (possibly more entries later)         .
             .                                          .
             |                                          |
          O  | length & offset 0th string  ----------------.
      O + 8  | length & offset 1st string  ------------------.
              ...                                    ...   | |
O + ((N-1)*8)| length & offset (N-1)th string           |  | |
             |                                          |  | |
          T  | length & offset 0th translation  ---------------.
      T + 8  | length & offset 1st translation  -----------------.
              ...                                    ...   | | | |
T + ((N-1)*8)| length & offset (N-1)th translation      |  | | | |
             |                                          |  | | | |
          H  | start hash table                         |  | | | |
              ...                                    ...   | | | |
  H + S * 4  | end hash table                           |  | | | |
             |                                          |  | | | |
             | NUL terminated 0th string  <----------------' | | |
             |                                          |    | | |
             | NUL terminated 1st string  <------------------' | |
             |                                          |      | |
              ...                                    ...       | |
             |                                          |      | |
             | NUL terminated 0th translation  <---------------' |
             |                                          |        |
             | NUL terminated 1st translation  <-----------------'
             |                                          |
              ...                                    ...
             |                                          |
             +------------------------------------------+
[ << ] [ >> ]           [Top] [Contents] [Index] [ ? ]

This document was generated by Bruno Haible on September, 19 2023 using texi2html 1.78a.