Tracker search index
In TORO Integrate, you can log data consumed and produced by your integrations via a feature called Tracker. Tracker is a term used to refer to the family of services and resources wired together to record service invocation data. By logging data coming in and out of TORO Integrate, you are granted access to these data again so you could use them when performing audits, analysis, troubleshooting, and when generating reports.
Tracker is a fully searchable, queryable logging engine that's great for looking up the data that your code has dealt with in the past.
Possible use cases
Suppose you have exposed a REST API using TORO Integrate for use in your e-commerce web site. Every time a customer orders something on the website, it contacts TORO Integrate behind the scenes to have the order created in your warehouse. By indexing order data, you can easily create reports about the:
- Average orders received in the previous months
- Orders received per day over the last month
- Orders received which costs greater than $50 in the last week
- Orders the contained certain products
- Orders shipped to a certain postal state
- Orders that have resulted in an error
- Best-selling products in the previous month, or year
You can also do other integrations that compute:
- How often your partners invoke TORO Integrate services
- How often your suppliers send their data for your systems
- The number of products published over the Google Marketplace
- How many new customers have signed up to receive your newsletter in the previous month
- The average value of all invoices, graphed per day for the last month
Internal system design
Under the hood, Tracker is backed by a Solr core of the same name (
linked to your instance's
core package. When a service invocation is triggered by an endpoint or HTTP
call, Tracker will log the details of the service invocation by adding an entry to the
tracker database2. It will then create a Lucene document for that entry and
index it to the aforementioned Solr core's index, which we call the Tracker search index. Querying data against the
index instead of the database allows for faster, finer-grained searches.
Tracking is on by default
TORO Integrate automatically adds a record to tracker every time code is invoked over HTTP/S or via endpoints, unless configured to not do so.
Tracker versus Monitor
Although both Tracker and Monitor log service invocation data, there are still a few key differences between them:
- Tracker logging is optional; Monitor logging is not. If you can get what you need in Monitor, it is better to not use Tracker as this will mean less processes for TORO Integrate to handle.
- Unlike that of the Monitor search index, there are no restrictions in modifying the contents of the Tracker search index. You can add, remove, and modify Tracker documents as you wish.
- With Tracker, you can specify which service invocation data to index (request data only, response data only, or both). Monitor will not provide you such options; the fields it indexes cannot be configured.
trackerSolr core resides in an embedded Solr instance and should be migrated to a remote Solr server instead for performance boost. ↩
TORO Integrate, by default, uses an embeded HSQL database for
tracker. It is recommended that you migrate your data there in a more reliable database when running on production. ↩