Skip to content

Flat File Descriptors

Flat File Descriptors are used to define how a particular flat file (whether it's delimited or has fixed width columns) is structured. They, in turn, are used by Gloop to determine how data should be parsed (read) from or serialized (written).

Once a Flat File Descriptor is configured and saved, TORO Integrate will generate a Gloop Model and a Gloop Service. The model will contain the properties specified in the descriptor1 while the Gloop Service can be used to easily open your data, ready for Gloop to process. The generated service returns a Gloop Cursor model array, which is then used to read or write the flat file data.

Creating a Flat File Descriptor

To create a Flat File Descriptor:

Creating a Flat File Descriptor

Creating a Flat File Descriptor

  1. Right click anywhere under a package's code folder and then select New > Flat File Descriptor. Note that the folder you right-clicked on will be the default location of your to-be-created Flat File Descriptor.
  2. In the appearing wizard, populate the following fields:

    • Location2

      Where the flat file descriptor will be stored. By default this will be populated with the location that was right-clicked on.

    • Name

      The name of the descriptor.

    • Type

      Specifies how fields in the flat file are identified. There are two options:

      • Delimited

        Choose this for flat files whose fields are delimited by specific characters (alternatively called separators). CSV and Tab-delimited files are some examples:

        1
        2
        3
        4
        James,Smith
        Thomas,Miller
        Sarah,Clark
        Elizabeth,Taylor
        
      • Fixed Width

        Choose this for flat files which make use of fixed column widths to identify individual fields. If a value can't occupy the entire column, it is typically padded with spaces. For example:

        1
        2
        3
        4
        James     Smith
        Thomas    Miller
        Sarah     Clark
        Elizabeth Taylor
        
  3. Click Finish.

Working with the Flat File Descriptor Editor

The editor has three tabs namely:

Design Tab

The Design Tab is where you define the structure of the flat file.

Flat File Descriptor Editor

Flat File Descriptor Editor

The fields visible in the editor is dependent upon which type (separators versus fixed width delimitation) this descriptor is configured to handle.

Common Configuration Fields

The following fields are both present in descriptors for character-delimited and fixed width flat files:

  • Type

    The type of the descriptor.

  • Input First Data Row

    The row number where reading of data should start. This field is handy if the data has header rows that contain the column names, and you wish to skip them.

  • Encoding

    The text-encoding method used in the flat file.

  • Output Header Type

    Determines whether headers are present or not in the data, when Gloop is writing it.

    Option Description
    Column Names Acknowledge the first row of entries as property names
    Data There are no headers and data is displayed in the first row immediately.

Character-Delimited

The following fields are specific to character-delimited flat file descriptors only:

  • Column Delimiter

    The character used to set fields apart fields in an entry.

  • Text Delimiter

    The character used to quote text; set to double quotes (") by default.

  • Escape Character

    The character used to escape special characters to allow them to be treated as ordinary characters.

  • Comment Maker

    The character used to mark data as a comment.

  • Record Separator

    The character(s) used to separate records.

  • Null String

    The string to be used for null values.

  • Quote Mode

    Indicates how values are quoted in the flat file.

    Option Description
    All For quoting all columns.
    Minimal For quoting only fields which contain special characters such as a delimiter, quote character, or any character used as a record separator.
    Non-Numeric For quoting only non-numeric fields.
    None For not quoting at all. When the delimiter occurs in a field, it must be preceded by the escape character. An exception will be thrown if any character that is unescaped but requires escaping is encountered.
  • Allow Missing Column Names

    Specifies whether missing column names are allowed when parsing the header (first row), if any. This is set to false by default.

  • Ignore Empty Lines

    Specifies whether to ignore empty lines; set to false by default.

  • Ignore Surrounding Spaces

    Specifies whether leading or trailing spaces would be ignored around values; set to false by default.

  • Property

    This is similar to the Gloop Model Editor, this is used to design the model that will represent the flat-file data.

Fixed Width

The following fields are specific to fixed-width flat file descriptors only:

  • Record Separator

    The character(s) used to separate records.

  • Padding Character

    Sets the character used to pad fields in a record.

  • Left Justified

    Specifies whether the columns are left justified or not; set to false by default.

  • Property

    This is similar to the Gloop Model Editor, this is used to design the model that will represent the flat-file data.

  • Width

    Sets the size of each column.

Preview Tab

The Preview Tab enables you to test the Flat File Descriptor with sample data. Once it is fed data, it parses your data displays the results in a table. You can provide data by importing a flat file or by simply adding text in the provided text area.

Flat File Descriptor Preview

Flat File Descriptor Preview

Source Tab

The Source Tab shows the source code of the Flat File Descriptor. It is written in JSON format, parsed by Gloop, and is used to define the Flat File Descriptor.

Flat File Source

Flat File Source

Using a Flat File Descriptor in a Gloop Service

To use a Flat File Descriptor in a Gloop, drag and drop the generated service to the Gloop Service:

Calling a Flat File Descriptor call in Gloop Service

Calling a Flat File Descriptor call in Gloop Service

Once the generated service is added as an Invoke Step, set the arguments required in the call:

  • flatFileData is the flat file to read or write to. This can be a File, InputStream, OutputStream, Reader, or Writer.
  • type is used to indicate whether to read or write data. Set this parameter's value to Input if you need to read data or Output if you need to write data.

Set Flat File Descriptor call's parameters

Set Flat File Descriptor call's parameters

As said earlier, the output of the generated service is a Gloop Cursor. If you've opted to read, the cursor will contain the entries in the flat file; when writing, however, the cursor should receive the entries you want to write to the file. In either scenarios, you would need to use an Iterate Step.

To read each entry, you must iterate over the cursor like you would an array. To do this, create an Iterate Step and then set the cursor as the value of the Iterate Step's Input Array property.

Reading from a Gloop Cursor in an Iterate Step

Reading from a Gloop Cursor in an Iterate Step

To write entries, you must create an Iterate Step as well; but this time, the Input Array should be set to hold the items you want to serialize and the Output Array's value should be the cursor.

Writing to a Gloop Cursor in an Iterate Step

Writing to a Gloop Cursor in an Iterate Step

Quickly creating a Flat File Descriptor call

To quickly generate Gloop code that reads or writes to a flat file:

  1. Press the . key in the Gloop Service Editor.
  2. Type the name of the Flat File Descriptor you want to process.
  3. Select the descriptor to call by moving your arrow keys up or down and then finally pressing once your choice is final.

[start-toggler] Calling a Flat File Descriptor via content assist ![[Coder Studio]] Calling a Flat File Descriptor via content assist ![[Coder Cloud]] [end-toggler]


  1. the flat file's fields 

  2. Option is only available in Coder Studio.