Rebuilding the Tracker search index
TORO Integrate's Tracker is responsible for recording requests and/or responses, and endpoint invocations that have been sent or received by TORO Integrate. These are indexed using Solr so that their details may be searched and used later on for other purposes such as usage analysis and creating reports.
But what should you do when the Tracker search index gets out-of-sync, corrupted, or worse – deleted? Fret not; you may still be able to save your data by rebuilding the index.
The option to rebuild the Tracker search index is only toggleable via the Server Admin UI
Unfortunately the only way to trigger a re-index is via the Server Admin UI, which is an interface accessible if you're using a Server Edition of TORO Integrate.
The index is rebuilt by pulling the selected entries1 from the Tracker database and then overwriting their corresponding Lucene documents2. As such, it is important to keep TORO Integrate's connection to the Tracker database stable and uninterrupted during the procedure.
The rebuild operation is also run as a background process and thus, TORO Integrate is still capable of handling requests and running other tasks. However, this will also mean processes will be competing for resources which can lead to delays.
During the rebuild process, Solr's auto-commit feature will be turned off. Pending commits will only be pushed after the indexing has completed. This means that the service calls made during the the operation will not be added to the index and will be unsearchable until the process is over and the commits are made. Cancelling the rebuild process would mean a rollback to the previous state of the Tracker search index3.
The duration of the process varies and depends on multiple factors such as the number of entries being re-written and the performance of the Solr server(s) holding the data. You may use the configuration options available to customize how the operation is carried out.
Why else should the index be rebuilt?
Another reason for performing an index is if you only want to have a certain amount of data indexed, or if you only want to keep data index that's less than a certain age.
Vist the Rebuild Tracker Index page by hovering your mouse over the Configuration menu and then clicking Rebuild Tracker Index from the appearing sub-menu.
From the Rebuild Tracker Index page, select which type of rebuild you want to perform: full rebuild or partial rebuild. Tick the Full Rebuild checkbox or fill in the Start Date and End Date fields for a partial rebuild.
Rebuilds the entire index, from the oldest entry to the latest one.
Rebuilds a part of the index, specified by a certain date range.
Click the Rebuild button and then wait for the process to finish.
It's possible to tweak and optimize the rebuilding process using the following application properties:
solr.reindex.batch.size- indicates the number of records to be fetched and indexed on a single thread
solr.reindex.batch.thread- the number of concurrent threads doing the fetching and reindexing