Pre-requisites
The actionable Data Catalog (or just Catalog) is a feature of EasyMorph Server. It can be accessed from EasyMorph Desktop or via a web browser (with limited functionality).
If you are a Desktop user, make sure you have configured the Server Link in your EasyMorph Desktop (see the tutorial chapter "Server Link" for more details).
If you're an EasyMorph Server administrator, in order to use the Catalog, configure the following:
- Enable the Data Catalog in the EasyMorph Server space settings (can be disabled by default)
- Make sure Desktop users obtain licenses from EasyMorph Server (see "What is dynamic licensing") or the space has the per-space license for Regular users (described in the Advanced topics below).
Overview
The Data Catalog lists various data assets that your team works with on a daily basis: reference lists, tabular datasets, data extracts, reports, etc. The Catalog can be seen as a "data supermarket" where you can find, retrieve, and manipulate any of these assets no matter where they are located. Note that the Catalog doesn't store the assets. Instead, it knows about them. You can think of the Catalog as a library of "smart bookmarks" that lead to various data used in your organization. Here is what it looks like:
In the Catalog, assets are organized into hierarchical directories, basically, folders. Each directory can contain items (assets) and sub-directories (see the image above). Any item can be retrieved. There are 3 basic types of items listed in the Catalog, all of them can be retrieved:
- Datasets
- Web resources (pages and apps)
- Files
Datasets
Datasets are, basically, unformatted tables (a table can have one or many columns, each with a header and data). Any business data that can be represented in a tabular form can be a dataset in the Catalog. For instance, lists of orders, customers, inventory, transactions, employees, products, accounts, to name a few. Technically, such data can be stored in various forms: spreadsheet tables, database tables, text (CSV) files, records in enterprise applications, SharePoint lists, Salesforce objects, and so on. However, where the data is actually stored doesn't matter. In the Catalog, every dataset has a standard look and feel because the Catalog abstracts away the source of data from the user. Therefore, with the Catalog, you can retrieve business data regardless of its location and source format and regardless of your technical skills.
How does it do it?
When you find the necessary dataset in the Catalog and press the "Retrieve" button (can be seen in the image above), under the hood, EasyMorph Server runs a workflow that retrieves the requested data on the fly. The Catalog then delivers the data from the Server to you and displays it in the Dataset Viewer (see image below) where you can explore it, analyze it, and export to a spreadsheet or one of the other supported file formats. In a way, computed Catalog items can be seen as "smart" Server tasks with a result dataset that is delivered to the user.
Therefore, datasets in the Catalog are dynamic in the sense that they are computed on the fly when retrieved. Also, because EasyMorph workflows can be parameterized, you can specify parameters (such as start and end dates) when retrieving a dataset. Parameters can reduce the retrieved data down to only the needed records. Excluding unnecessary data helps retrieve data faster and reduce the workload on the underlying systems.
Dataset Viewer
When you retrieve a dataset, it opens in the Dataset Viewer. Here, you can do quite a lot with the data:
- Export data into Excel spreadsheet, CSV file, or a few other formats
- Filter rows (drag a column header into the filtering area above)
- Sort rows
- Open it in the Workflow Editor for data transformation
- Re-retrive the dataset with different parameters (e.g. for another date range)
- Update Catalog item properties, such as field metadata and description
- Trigger commands and retrieve related data (to be explained later in this tutorial)
Filtering is an extremely powerful feature of the Dataset Viewer. It allows finding records, exploring relationships in data, and identifying data quality issues. It is especially convenient, that the filters show not only the column values included into the current selection, but also the excluded ones. To add a filter, click a column header and drag it into the filtering area (above the table). Select one or more values in the filter, and press "Apply".
Hint: To quickly create an instant filter with a table value, simply double-click it in the table. To quickly create an instant filter that excludes a table value, Ctrl + double-click the value.
Web pages
Besides datasets, you can add references to various web resources (pages) to the Catalog. Retrieving a web resource is different from retrieving a dataset (described above). When you retrieve a web resource, EasyMorph opens a web browser (the default browser in your computer) with the requested web page. You can add pretty much any web resource to the Catalog. Just a few examples:
- Business Intelligence reports (e.g. Power BI or Tableau)
- Corporate wiki pages (e.g. Confluence)
- Government portals
- Web applications (e.g. Google Sheets)
Since every web resource has a web address (a.k.a. URL), the Catalog can also calculate it on the fly using a visual EasyMorph workflow. It provides a lot of flexibility, one Catalog item can lead to many web pages depending on the provided values of the workflow parameters. For instance, when opening a Power BI report, its date range can be preset from the computed web address using URL parameters. As a result, the BI report will display exactly the data you need.
Files
Finally, you can retrieve just any file with the Catalog. This can be convenient if your organization operates with lots of uniform files — PDF invoices, images, zip-archives, etc., that have been occasionally retrieved. When retrieving a file item from the Catalog, the file will be delivered and saved into the local folder specified by the user. Just like with datasets and web pages, the location and path of the requested file can be calculated dynamically with an EasyMorph workflow depending on provided parameters.
