Transient Field Processing
Transient field processing detects changes in any fields marked as transient in a Cache table in order to incrementally update rows in the Data Warehouse that contain the transient fields.
Comparison Process
Transient fields are selected for comparison based on selections made on the Data Warehouse Transient Comparison Configuration form in myAvatar.
The incremental middleware retrieves a list of transient fields for a table in myAvatar, queries the Data Warehouse database for these fields, and creates an SHA1 hash value with the concatenation of the returned values for each row in the table.
For each table identified for the comparison process, an SHA1 hash of the fields identified in the target database will be created in the source database (myAvatar) and compared against the corresponding hash. These hash values, along with a Row ID, are inserted into myAvatar where they are compared with the corresponding table. (The disk consumption for inserting these hash values will be the equivalent of adding a 50-character %String column to each table.)
Transient fields that are too process-intensive to be viable in a hash value comparison will be handled by the background journal process.
If a transient field has been added to a table in the compare process since the last runtime, a snapshot is created of the table in the target database before the compare processes commences. If a difference is found, the ROWID will be added to the SYSTEM.dss_inc_full_load table.
When the comparison process completes, a message is sent to the middleware that indicates the table is now ready to begin the incremental load process. If any tables fail the comparison the process will requery those tables and push the rows found by the comparison query to the Data Warehouse.