Iterations (loops)

This chapter requires good understanding of project parameters and subprojects explained earlier in this tutorial.

Iterations in EasyMorph are used to arrange loops (cycles) — i.e. when a task is executed multiple times, every time with a new set of input parameters (data). Using iterations you can transform multiple files, process daily data for a given range of dates, evaluate various scenarios (a.k.a. "what-if" analysis) by varying calculation parameters, and arrange other repetitive workflows.

Iterating means calling another project (subproject) multiple times. It is arranged using the "Iterate" transformation. "Iterate" is very similar to the "Call" transformation with two major differences:
  1. 1) Parameters of the called project (subproject) are assigned using column values, not constants.
  2. 2) The subproject is called not once, but as many times as there are rows in the table with the "Iterate" transformation.

The diagram below shows how Project A iterates Project B using a list of files. It runs Project B three times — once per each file name in column [File Name]. Project B has a parameter named Param1. This parameter is assigned by Project A when it runs Project B.

Most frequently, iterations are performed against a list of files, or a list of dates. Such lists can be generated using transformation "Calendar" (generates a sequence of dates) or transformation "List of files" (generates a list of files in a folder). Alternatively, they can be imported from a database or a text file.

Hint: To quickly obtain a list of files in a folder simply drag the folder into EasyMorph.

To create a FOR..NEXT type of loop, generate a sequence of numbers 1..N using "Sequence" transformation. Then iterate across the sequence.

"Iterate" transformation

The "Iterate" transformation runs another project once for each row of its input dataset. Parameters of the iterated project are assigned using field values of the dataset. If a project parameter is not assigned, it keeps the last saved value (the default value).

The "Iterate" transformation can run in two modes:
  • Iterate — do not return anything from the iterated project, just run it.
  • Iterate and append results — return a table (the result dataset) from each iteration, and append all result datasets into one which becomes the output of "Iterate" transformation.
    Appending the result datasets is similar to the "Append" transformation — columns with the same name are concatenated into one column, columns with unique names are extended with empty cells.

Examples

We've preperared a pack of 5 iteration examples. Each example has an illustrated description and annotations in project tables and transformations.

  • Example 1 – load multiple files
  • Example 2 – split one file into many files
  • Example 3 – batch processing of several files
  • Example 4 – iterate and append data into one table
  • Example 5 – iterate another table

Download iteration examples

Advanced topics

"Iterate table" transformation

The "Iterate table" transformation is basically the same as "Iterate" transformation with the only addition that besides assigning parameters of iterated project it also allows passing a table into it. The iterated project should use the "Input" transformation in order to obtain the table. This transformation is helpful when multiple calculations with different parameters have to be performed on the same dataset (e.g. financial stress testing).

Using the "Iterate table" transformation allows arranging two more iteration scenarios:

Nested iterations (not available in the free edition)

An iterated project can in turn iterate another project, effectively forming a nested iteration. In EasyMorph, nested iterations can be of any depth.

Note that in "Iterate" transformation the iterated project itself can be specified using a parameter. Therefore, using nested iterations it is possible to execute a list of EasyMorph projects, collecting their results into one table. This workflow is helpful when you have many projects that have customized transformation logic (e.g. adjusted for each customer), but return standardized results. For instance, you need to collect balance sheets from multiple counterparties, but each counterparty sends it in a different format. Therefore you use a separate EasyMorph project for each counterparty to transform its balance sheet into some uniform representation of balance sheet, and then append all uniform balance sheets into one dataset.


Read next: External programs