Iterations in EasyMorph are used to arrange loops (cycles) — i.e. when part of a calculation is executed multiple times, each time with a new set of input parameters (data). Using iterations you can transform multiple files in a similar fashion, process daily data for a given range of dates, send emails to multiple recepients, and arrange other repetitive workflows. Understanding iterations may require a bit of effort. However, once learned, iterations become a very powerful technique that can be used in many real-life scenarios. From a programming perspective, iterations in EasyMorph allow aranging FOR...EACH, FOR..NEXT, and DO..WHILE/UNTIL types of loops.
In EasyMorph, iterating means calling (executing) another module (or project) multiple times. In this chapter we will be talking mainly about iterating modules. However, it all is equally applicable to iterating projects.
FOR..EACH type of loops in EasyMorph are arranged using the "Iterate" action. The action is very similar to the "Call" action (described in the previous chapter) with two major differences:
The diagram below shows how Module A iterates Module B using a list of file names. Module B has a parameter named Param1. This parameter is assigned by Module A with a new file name every time it runs Module B. Because the input dataset (Table 1) in Module A has 3 rows, the "Iterate" action calls (iterates) Module B exactly 3 times — once per each file name in column [File Name].
Frequently, iterations are performed against a list of files, or a list of dates. Such lists can be generated using the "Calendar" action (generates a sequence of dates) or "List of files" (generates a list of files in a folder). Alternatively, they can be imported from a database or a text file.
Hint: To quickly obtain a list of files in a folder simply drag the folder into EasyMorph.
The "Iterate" action runs another module (or project) once for each row of the action's input dataset. Parameters of the iterated module can be assigned using field values of the dataset. If a module parameter is not assigned, it keeps the last saved value (the default value).
To create a FOR..NEXT type of loop, generate a sequence of numbers from 1 to N using the "Generate sequence" action, then iterate across the sequence using the "Iterate" action.
All calling actions ("Call", "Iterate", "Iterate table", and "Repeat") can run modules in the current project as well as external projects. When designing iteration of a module, it may be confusing to see that the module's parameters that are visible in EasyMorph don't change after a call from another module. That's because during calls/iterations, under the hood, EasyMorph creates an invisible clone of the iterated module and assigns parameters to that invisible clone and runs it. The module that you see remains intact, calls don’t modify it. This is done in order to avoid ambiguity and conflicts because the same module can be called from multiple places simultaneously.
In order to understand how a module would be calculated when called/iterated from another module, set the called module's parameters manually and run it (if Auto-run is off).
We've preperared a pack of 5 iteration examples. Each example has an illustrated description and annotations in project tables and actions.
The "Iterate table" action is basically the same as the "Iterate" action with the only addition being that, besides assigning parameters of the iterated module, it also allows passing a table into it. The iterated module should use the "Input" action in order to obtain the table. This action is helpful when multiple calculations with different parameters have to be performed on the same dataset (e.g. financial stress testing).
Using the "Iterate table" action allows arranging two more iteration scenarios:
Loops created using the actions "Iterate" or "Iterate table" are used in cases when the number of iterations is known in advance and depends on a condition. However, there are cases when the number if iterations is not known ahead of time and depends on a condition. For instance, a web request that fetches paged data from a web API needs to be repeated again and again until no more data is fetched. In programming, such cases are handled using a DO...WHILE/UNTIL loop. In EasyMorph, such loops are arranged using the "Repeat" action.
The "Repeat" action has two modes:
Frequently (but not always), the called module/project may need to have a conditional branching at some point. The condition would serve as the loop exit condition. One of the branches should return an empty (or not empty) dataset, depending on the loop type.
If the called module contains the "Input" action, on the 1st iteration it's populated with the input dataset of the "Repeat" action that calls it. In consequent iterations it's populated with the result dataset from the previous iteration. In other words, the result of previous iteration is automatically passed as input to the next iteration.
Hint:To emulate passing a dataset from previous iteration to next iteration, you can send the output (righ-click the last action, select "Send to sandbox/module") of the default result table to the "Input" action of the same module. If Auto-run is on, the module will recalculate automatically and produce a new result which you can send to "Input" again.
The "Repeat" action can help in numerous cases:
An iterated module can in turn iterate another module, effectively forming a nested iteration. In EasyMorph, nested iterations can be of any depth. However, recursion or cyclical dependencies are not allowed.
Note that in the "Iterate" action the iterated module itself can be specified using a parameter. Therefore, using nested iterations it is possible to execute a list of EasyMorph modules or projects, collecting their results into one table. This workflow is helpful when you have many modules/projects that have customized action logic (e.g. adjusted for each customer), but return standardized results. For instance, you need to collect balance sheets from multiple counterparties, but each counterparty sends it in a different format. Therefore you can use a separate EasyMorph module (or project) for each counterparty to transform its balance sheet into some uniform representation of balance sheet, and then append all uniform balance sheets into one dataset.