Exporting data into a database usually takes two steps:
To create a database table use "Database command" transformation. In this transformation pick a database connector, and then provide a new table name. While most database types allow table names contain spaces and letters in mixed upper/lower cases, it is recommended to choose table names that have no spaces and contain either all upper-case letters, or all lower-case.
To facilitate dealing with database data types, in EasyMorph all data types regardless of database type are simplified to just three: Number, Text and Date. EasyMorph automatically detects data types and picks appropriate database types depending on current database connector. You can adjust auto-detected types, if necessary.
Hint: To create a table in particular database schema prepend the table name with the schema name. For instance: myschema.mytable. Note that the schema must already exist.
To export into a database table use the "Export to database" transformation. Under the hood, this transformation employs SQL INSERT to insert batches of 10/100/1000 rows into the target database table. The target table must already exist before using "Export to database" transformation. Prior to exporting data EasyMorph automatically detects data types in the target table. In case of type mistmatch for a particular value it will be exported as NULL or as default value. Therefore it's typically a good idea to ensure that your data is clean and consistent before exporting — EasyMorph has many transformations that can help with this.
Hint: In order to map fields in EasyMorph to fields in existing database table just rename EasyMorph fields so that the new names exactly match corresponding field names in the database. You can do the renaming and export in a derived table, if you prefer to keep working with unchanged names in EasyMorph.
Bulk loading is recommended for large datasets (>1mln rows), when performance may be critical. Typically it requires two steps:
Bulk loading typically has very good performance, and allows fast loading big tables due to database-specific optimizations. Although, it requires learning how to compose necessary commands. See the links below to find documentation on bulk load commands in some popular RDBMSes:
Hint: Insert parameters into SQL commands in curly braces. When sending a command a parameter name in curly braces will be replaced with the parameter's value.
Hint: You can also use "Database command" to trigger stored procedures.
Note that export transformations and "Database command" transformation are not calculated automatically, even when Auto-run is on. They are considered side-effect transformations because they affect external data and systems. Such transformations should be run manually, by pressing "Run project" on the main toolbar, or F5.
A typical workflow for updating a database table is as follows:
As you can see the workflow requires deleting rows before exporting. In EasyMorph there are two ways to delete particular rows in a database table: using a query, or a list of keys.
One way to delete database rows is to use a query in "Delete database rows" transformation. The transformation deletes rows using the filtering conditions of the query. Column selections in the query are ignored. Under the hood, the transformation generates a SQL request DELETE FROM [table] WHERE [condition].
The "Delete database rows" transformation has only one setting — a query.
Hint: Use query preview to see the rows that will be deleted by the transformation.
Another way to delete database rows is using the "Delete matching database rows" transformation. This transformation deletes rows where key fields in the database table match key fields in current EasyMorph table. Under the hood, the transformation creates a temporary table in the databse with keys from the EasyMorph table, then generates a SQL request DELETE FROM [table] WHERE [key] EXISTS IN [temp.table], or similar.
Note that matching works only for key fields of particular types:
This transformation is frequently used together with "Select matching database rows". It's also convenient in the cases when a list of IDs to delete is obtained from a 3rd party data source (e.g. a spreadsheet).