Using CSS Selectors

The Crawler tab contains a bunch of top-level text fields¹ which expect inputs in the form of CSS selectors. These CSS selectors will be used by the Crawler to locate available API information, which will be extracted and stored² so that they can be used later in creating the API's OpenAPI schema.

Unless stated otherwise, the part which gets extracted and stored is the enclosed tag content of selected elements. Each value will occupy a field in the resulting OpenAPI schema, but for the meantime, they will be mapped to an internal model for storage.

Default crawling strategy specific

The information written in this document is based on the plugin's default behavior. If you're using a custom crawling strategy, the information here may or may not be applicable, depending on how the custom crawling algorithm uses the provided CSS selectors.

Guidelines for Choosing the Right CSS Selectors

• CSS selectors should be reusable

  The Crawler relies on structural consistency in order to work properly. It assumes identical formatting for identical types of data; which is why provided selectors should be capable of selecting all relevant elements, not just a portion of them.

• Select only relevant elements; no more, no less

  A provided CSS rule for a field should, as much as possible, select only the required set of elements and/or data. If it's not possible to isolate elements and/or data from noise, then further processing may be done through the use of the Evaluate, Filter, and Exclude fields.

• Do not prepend the operation wrapper's CSS selector

  The plugin implicitly prepends the provided operation wrapper's CSS selector to all other selectors. This is because the plugin assumes all API operation data is wrapped under the operation wrapper. This is true for all of the fields in the Operation, Request Body, Parameter, and Response tabs.

  What is an operation wrapper?

  An operation wrapper is an HTML element wrapping API operation information like the operation name, path, HTTP method, and description. For the Request Body, Parameter, and Response tabs, the operation wrapper is also expected to contain request body/parameter/response information.

  For example, if the operation wrapper's selector is div.operation and the selector for the operation name is h2, the Crawler will end up looking for the operation name at div.operation h2.

Internal Representation of Extracted Data

When crawling data, a provided selector typically selects more than one element and as a consequence, the plugin ends up collecting multiple values per rule. Internally, these extracted data are stored in arrays._~~

Checking Matches

Clicking on a rule text field prompts the plugin to highlight elements selectable by the provided CSS selector. This feature also takes into account provided values for the Evaluate, Filter, and Exclude fields.

Using the Inspect Page Feature

There is an Inspect Page button for every text field accepting a CSS selector input. This button allows you to quickly determine and enter an element's CSS selector by clicking on the HTML element itself, as presented in the documentation web page.

---

1. Text fields that are shown by default beside the field label; unlike the Evaluate, Filter, and Exclude text fields which only appear when the label is clicked. ↩

2. Extracting and storing of data occurs when the Crawler is run. ↩

3. The Crawler can crawl other pages, if needed, to get the required API information. The Url field is used to indicate where else the Crawler should look instead of the main page. ↩