Indexing in v2.0 - Index Queue

During the T3DD11 we offered a full day tutorial to get you started with Apache Solr for TYPO3. As a big gift back to the TYPO3 community we added the Index Queue to the public version of the extension on TYPO3 forge.

The introduction of the Index Queue means that there won't be "frontend indexing" as we used to know it from EXT:indexed_search and EXT:crawler. The issue with frontend indexing is that you can't differentiate between pages, products, news, or any other custom record. 

With the Index Queue  we provide a way to define an easy mapping of database record fields to Solr index fields. TYPO3 is awesome in how flexible it is in regards to page rendering; in this case this kind of turned out to be a disadvantage at first since you can't (read: want to) "re-implement" all the logic. So to index pages we will still rely on frontend indexing, but it's still way better than the classic way. Even better, the Index Queue will find any access restrictions and translations and take care of indexing those variants automatically, too.

The Index Queue comes with a Record Monitor to track any changes and can then re-index just those changed records or pages. In the same way the Garbage Collector will clean up after your editors when they delete records.

So the Index Queue is an easy and fast way to get your site's pages and extension records indexed. However, raw database data often is not what you want in your index right away, usually you want to do some cleaning or transformation before handing the data to Solr. Because of that the Index Queue also supports mapping TYPO3 content objects (cObj) - or more specifically their output - into index fields. To make it even easier to index custom extension records we came up with custom content objects provided by the Apache Solr for TYPO3 extension.

Currently there are three content objects:

• SOLR_CONTENT - cleans database fields like rich text editor fields from HTML code and formatting
• SOLR_MULTIVALUE - allows to index data into - you guessed it - Solr multivalue fields, acting like the PHP explode() functions
• SOLR_RELATION - resolving relations between records, indexing data into multivalue or singlevalue fields. Useful for news and news categories f.e.

We hope you are looking forward to EXT:solr v2.0 as we do!