Indexing Multilingual Records

Seamless indexing of translated records

Recently we were approached by a member of the Apache Solr for TYPO3 Early Access Program who needed to index translated records of a custom extension they had built for one of their customers.

Translation handling - an important and powerful TYPO3 feature

Many projects out there use TYPO3 because of its powerful translation handling. The translation handling in TYPO3 is easy to understand and use since translations are directly associated to its original language record. A lot of international companies especially value its flexibility of handling language fallbacks should an item not be available for a certain language.

So since translation handling is such an important feature we of course had to support that with Apache Solr for TYPO3.

One of the philosophies we try to follow when developing this TYPO3 extension is to keep everything as simple as possible to the end users and integrators, hiding complexity wherever possible.

For quite a while now the Index Queue, a unique feature of the extension to make indexing of content as easy as it could be, supported automatic indexing of translated pages.

For as long as the page indexer supported that automatism we had an item on our ToDo list to also enable this magic for custom data records as provided by extensions.

Correctly indexing translated records was a must for that mentioned project. So this was the time we eventually had to implement support for this great feature.

How did we do it?

Essentially we took the approach we were using for indexing pages already and moved that to the general indexer. In addition to that we had to do some minor tweaks here and there to make the Index Queue aware of translated records in general. This worked out very easily.

In TYPO3 when extensions support multi language handling it is easy to identify translated versions of original language records. Usually records like news are kept in a folder to organize them.

When you want to translate a page or in this case news records you simply create an Alternative Page Language for that page. As Folders are just pages internally too, you just create a new Alternative Page Language for the storage folder of the news records.

The Index Queue indexer then automatically detects these Alternative Page Languages and uses them to index the translated versions of a (news) record.

This feature is currently available in Apache Solr for TYPO3 1.7.0-dev (trunk, Early Access Program) only.