Skip to content

Using the Evaluate Field

The Crawler knows where to get API data via CSS selectors. But sometimes, acquired raw data needs further processing. One of the ways you can transform acquired data is via the Evaluate field. The Evaluate field allows the user to transform data via JavaScript.

Available Properties

The Evaluate field allows you access to the following properties in your script:

  • targetEl

    This is the HTML element matching the provided CSS selector.

  • text

    The text obtained from targetEl.

  • metadata

    This is a read-only object containing the current plugin configuration (including crawled data).

  • operation

    This is also a read-only object, but contains API operation data instead; particularly the API operation tied to the field being currently crawled. This is only available in the Request Body, Parameter, and Response tabs.

Available Methods

The Evaluate field allows you access to the following methods in your script:

  • log(String:text) - use this to print text to your browser's console
  • error(String:text) - use this to print text to your browser's console in red to signal the occurrence of an error

Test Evaluate

A Test Evaluate button has been added beside every Evaluate textarea starting v1.2.1. Using this feature, you will be able to test your code in Evaluate fields without having to run the Crawler. The only limitation to this feature would be the inability to test code using operation and targetEl objects.

Test Evaluate button

Upon clicking the Test Evaluate button, a dialog will appear. Here you can evaluate expressions and view logs from the plugin. The plugin will keep track of all evaluated expressions, so you can revert to previous expressions any time.

Using the Test Evaluate dialog

Example

For this example, we're going to look at how we can extract the operation path properly using the Evaluate field; assuming the structure of the documentation page we need to crawl looks like below.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
<div class="main">
    <div class="operations">
        <div class="operation"> <!-- Operation wrapper -->
            <div class="header">
                <div class="title">GET /board</div> <!-- HTTP method and path -->
            </div>
            <!-- ... -->
        </div>
        <div class="operation"> <!-- Operation wrapper -->
            <div class="header">
                <div class="title">DELETE /board/{boardName}</div> <!-- HTTP method and path -->
            </div>
            <!-- ... -->
        </div>
        <div class="operation"> <!-- Operation wrapper -->
            <div class="header">
                <div class="title">POST /board/{boardName}/thread</div> <!-- HTTP method and path -->
            </div>
            <!-- ... -->
        </div>
    </div>
</div>

The HTTP method and URL share the same element; they're only separated by a space character. To get only the URL part for the Operation Url field, we configure our plugin like so:

Label Selector Evaluate
Operation Url div.title return text.split(' ')[1];

This instructs the plugin to look for operation URLs in div.api-title elements. When a match is found, it will:

  1. Get the element's tag content.
  2. Split the obtained string using a space separator character. This will produce an array of two strings.
  3. Return the second item from the array produced in the previous step.

For our example HTML snippet, the Crawler is expected to execute the following transformations:

Raw Value Transformed Value
GET /board /board
DELETE /board/{boardName} /board/{boardName}
POST /board/{boardName}/thread /board/{boardName}/thread