Using the Evaluate Field
The Crawler knows where to get API data via CSS selectors. But sometimes, acquired raw data needs further processing. One of the ways you can transform acquired data is via the Evaluate field. The Evaluate field allows the user to transform data via JavaScript.
Available Properties
The Evaluate field allows you access to the following properties in your script:
-
targetEl
This is the HTML element matching the provided CSS selector.
-
text
The text obtained from
targetEl
. -
metadata
This is a read-only object containing the current plugin configuration (including crawled data).
-
operation
This is also a read-only object, but contains API operation data instead; particularly the API operation tied to the field being currently crawled. This is only available in the Request Body, Parameter, and Response tabs.
Available Methods
The Evaluate field allows you access to the following methods in your script:
log(String:text)
- use this to print text to your browser's consoleerror(String:text)
- use this to print text to your browser's console in red to signal the occurrence of an error
Test Evaluate
A Test Evaluate button has been added beside every Evaluate textarea starting v1.2.1. Using this feature, you will be
able to test your code in Evaluate fields without having to run the Crawler. The only limitation to this feature would
be the inability to test code using operation
and targetEl
objects.
Upon clicking the Test Evaluate button, a dialog will appear. Here you can evaluate expressions and view logs from the plugin. The plugin will keep track of all evaluated expressions, so you can revert to previous expressions any time.
Example
For this example, we're going to look at how we can extract the operation path properly using the Evaluate field; assuming the structure of the documentation page we need to crawl looks like below.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 | <div class="main"> <div class="operations"> <div class="operation"> <!-- Operation wrapper --> <div class="header"> <div class="title">GET /board</div> <!-- HTTP method and path --> </div> <!-- ... --> </div> <div class="operation"> <!-- Operation wrapper --> <div class="header"> <div class="title">DELETE /board/{boardName}</div> <!-- HTTP method and path --> </div> <!-- ... --> </div> <div class="operation"> <!-- Operation wrapper --> <div class="header"> <div class="title">POST /board/{boardName}/thread</div> <!-- HTTP method and path --> </div> <!-- ... --> </div> </div> </div> |
The HTTP method and URL share the same element; they're only separated by a space character. To get only the URL part for the Operation Url field, we configure our plugin like so:
Label | Selector | Evaluate |
---|---|---|
Operation Url | div.title |
return text.split(' ')[1]; |
This instructs the plugin to look for operation URLs in div.api-title
elements. When a match is found, it will:
- Get the element's tag content.
- Split the obtained string using a space separator character. This will produce an array of two strings.
- Return the second item from the array produced in the previous step.
For our example HTML snippet, the Crawler is expected to execute the following transformations:
Raw Value | Transformed Value |
---|---|
GET /board |
/board |
DELETE /board/{boardName} |
/board/{boardName} |
POST /board/{boardName}/thread |
/board/{boardName}/thread |