Skip to content

Overview

Concepts and Principles

Development

Overview

IDEs

API Explorer

Releases

Release Notes

TORO Integrate

Coder Studio

Coder Cloud

Bug Reports

Search

Creating and Connecting to a Custom Search Index

TORO Integrate supports one of the most popular open source enterprise search platforms, Apache Solr. This means that you are given the ability to link you own Solr cores and add and search documents to/from your local or remote Solr servers (support for embedded, remote and SolrCloud versions of Solr are dependent on your TORO Integrate subscription plan).

This guide will walk you through creating and linking your own local or remote Solr core (also known as a collection).

Prerequisites

Understanding the following topics is a prerequisite to this guide:

Procedures

  1. Set required solr-prefixed instance properties.

    • solr.mode

      This property will tell TORO Integrate where to look for your Solr core (or collection). You have the option to choose from three possible values:

      Mode Description
      local Tells TORO Integrate that it will be using an embedded Solr instance which uses the host's local file system to store indexes.
      remote Tells TORO Integrate to connect to a remote instance of Solr.
      cloud Tells TORO Integrate to connect to a SolrCloud instance.
  2. Create your Solr core or collection using your configuration files.

    1. Prepare the following Solr configuration files for your Solr core:

      • solrconfig.xml
      • schema.xml
      • core.properties (optional for remote and SolrCloud instances)

      Ensure that your Solr core's name follows the format <package_name>_<core_name> where:

      • <package_name> is the name of your Integrate Package
      • <core_name> is the name of your core

      You can do this by setting the name property via the Solr Admin UI or by editing the core.properties file:

      1
      name=<package_name>_<core_name>
      

      If you plan to use SolrCloud, we recommend prefixing your collections' names. The format will then be <prefix>_<package_name>_<core_name>. You will also have to set the solr.prefix instance property.

    2. Create your Solr core or collection using the configuration files you have created above. The process will differ depending on which mode your Solr instance is running on:

      • Local

        1. Package your configuration files such that they follow the structure below:

          1
          2
          3
          4
          5
          <core>
          ├── conf
          │   └── schema.xml
          │   └── solrconfig.xml
          └── core.properties
          
        2. Place this core configuration directory under the target package1's solr directory.

          1
          2
          3
          4
          5
          6
          7
          8
          <package>
          ├── ...
          └── solr
              └── <core>
                  ├── conf
                  │   └── schema.xml
                  │   └── solrconfig.xml
                  └── core.properties
          
      • Remote

        1. Package your configuration files such that they follow the structure below:

          1
          2
          3
          4
          <core>
          └── conf
              └── schema.xml
              └── solrconfig.xml
          
        2. Add this core configuration folder to your Solr instance's configuration directory. Typically, this is in the <solr-home>/server/solr directory.

          1
          2
          3
          4
          5
          6
          <solr-home>/server/solr
          ├── ...
          └── <core>
              └── conf
                  └── schema.xml
                  └── solrconfig.xml
          
        3. Open the Solr admin page, click on the Core Admin tab, click on the Add Core button, and then complete and submit the form to add your Solr core.

          Filled-up form for adding a new Solr core

          The following table describes the fields of this form:

          Field Description
          name The name of the Solr core; must follow the format <package_name>_<core_name>.
          instanceDir The name of the Solr core directory.
          dataDir The name of the directory which will store the index files.
          config The name of the Solr core XML configuration file.
          schema The name of the Solr core XML schema file.
      • Cloud

        1. Zip the configuration files solrconfig.xml and schema.xml together. They must be top-level entries in the archive.

          1
          2
          cd ${solr-home}/server/solr/configsets/${collection}/conf
          zip -r myConfigSet.zip *
          
        2. Upload the resulting .zip file as a ConfigSet via the Solr ConfigSets API to any Solr instance belonging to your target cluster.

          1
          2
          3
          4
          curl -X POST \
          --header "Content-Type:application/octet-stream" \
          --data-binary @myConfigSet.zip \
          "http://<host>:<port>/solr/admin/configs?action=UPLOAD&name=myConfigSet"
          
        3. Open the Solr admin page, click on the Collections tab, click on the Add Collection button, and then fill up and submit the form to add your Solr collection.

          Filled-up form for adding a new Solr collection

          The following table describes the fields of this form:

          Field Description
          name The name of the Solr core; must follow the format <package_name>_<core_name>.
          config set The name of the config set that will be used.
          numShards The number of shards to be created as part of the collection.
          replicationFactor The number of replicas to be created for each shard.
  3. Register your Solr core or collection in esb-package.xml.

    In order to register a custom Solr core or collection to TORO Integrate, you must edit the package1's esb-package.xml file to add a new solr-core element under the solr-cores array element. This new solr-core entry will be used to identify your custom Solr core.

    1
    2
    3
    4
    5
    6
    7
    <esb-package>
        <!-- ... -->
        <solr-cores>
            <solr-core name="<core_name>" enabled="true" />
            <!-- ... -->
        </solr-cores>
    </esb-package>
    

    While we require the format <package_name>_<core_name> or <prefix>_<package_name>_<core_name> for our Solr cores, the new solr-core element's name attribute should only be <core_name> without <prefix> and <package_name> prepended.

  4. Restart your TORO Integrate instance or just the affected Integrate Package, depending on the scope of your changes.

    If you have made modifications on any solr-prefixed instance property, then your TORO Integrate instance will require a restart to pick up the changes. Otherwise, a quick restart of your package will do the trick.

    If your registration was a success, you should have log messages similar to the ones below:

    1
    2
    12/03/18 15:04:05.114 INFO  [<LocalSolrServer|CloudSolrServer|RemoteSolrServer>] Starting core: <package>/<core>
    12/03/18 15:04:25.402 INFO  [<LocalSolrServer|CloudSolrServer|RemoteSolrServer>] Starting core: <package>/<core> completed
    

Gloop Models in Solr

If you have a simple Gloop Model that you would like to index in Solr, you can execute the io.toro.integrate.SolrMethods.gloopModelToSolrSchema service. This will create the required files for Solr that will get you indexing Gloop Models in no time!


  1. Particulary the package which will use the custom core or collection.