Skip to content

Rebuilding the Tracker Search Index

TORO Integrate's Tracker is responsible for recording requests and/or responses, and endpoint invocations that have been sent or received by TORO Integrate. These are indexed using Solr so that their details may be searched and used later on for other purposes such as usage analysis and creating reports.

But what should you do when the Tracker Search Index gets out-of-sync, corrupted, or worse – deleted? Fret not; you may still be able to save your data by rebuilding the index!

The option to rebuild the Tracker Search Index is only toggleable via the Server Admin UI

Unfortunately the only way to trigger a re-index is via the server Admin UI.

The index is rebuilt by pulling the selected entries1 from the Tracker database and then overwriting their corresponding Lucene documents2. As such, it is important to keep TORO Integrate's connection to the Tracker database stable and uninterrupted during the procedure.

The rebuild operation is also run as a background process and thus, TORO Integrate is still capable of handling requests and running other tasks. However, this will also mean processes will be competing for resources which can lead to delays.

During the rebuild process, Solr's auto-commit feature will be turned off. Pending commits will only be pushed after the indexing has completed. This means that the service calls made during the the operation will not be added to the index and will be unsearchable until the process is over and the commits are made. Cancelling the rebuild process would mean a rollback to the previous state of the Tracker Search Index3.

The duration of the process varies and depends on multiple factors such as the number of entries being re-written and the performance of the Solr server(s) holding the data. You may use the configuration options available to customize how the operation is carried out.

Another reason for indexing

Another reason for performing an index is if you only want to have a certain amount of data indexed, or if you only want to keep data index that's less than a certain age.

Procedure

How to rebuild the Tracker Search Index

  1. Vist the Rebuild Tracker Index page by hovering your mouse over the Configuration menu item and then clicking on Rebuild Tracker Index from the appearing sub-menu.

  2. From the Rebuild Tracker Index page, select which type of rebuild you want to perform: Full Rebuild or Partial Rebuild. Tick the Full Rebuild checkbox or fill in the Start Date and End Date fields for a partial rebuild.

    • Full Rebuild

      Rebuilds the entire index, from the oldest entry to the latest one.

    • Partial Rebuild

      Rebuilds a part of the index, specified by a certain date range.

  3. Click the Rebuild button and then wait for the process to finish.

    The progress bar shown when the Tracker Search Index is in the process of rebuilding

Settings

It's possible to tweak and optimize the rebuilding process using the following application properties:

  • solr.reindex.batch.size - indicates the number of records to be fetched and indexed on a single thread

  • solr.reindex.batch.thread - the number of concurrent threads doing the fetching and reindexing


  1. Specified by date ranges 

  2. Solr uses Lucene as its search-engine underneath. Indexed entries are formally called documents. 

  3. Relevant if you have a working tracker search index before you did the rebuilding