In most OBIA projects we create both incremental and full load mappings for each tables in DWH target.Below you can find the difference bw both.
Full Load is the entire data dump load taking place the very first time.In this we give the last extract date as empty so that all the data gets loaded
Full Load is the entire data dump load taking place the very first time.In this we give the last extract date as empty so that all the data gets loaded
Incremental - Where delta
or difference between target and source data is dumped at regular
intervals.Here we give the last extract
date such that only records after this date are loaded.
Note:
Note:
- Full load: $$Last_Extract_Date = either leave it blank or give some old date so that all the records are loaded
- Incremental load:$$Last_Extract_Date = The date from which you need the data (to get latest record).
Why
Incremental?
- Speed. Opting to do a full load on larger datasets will take a great amount of time and other server resources. Ideally all the data loads are performed overnight with the expectation of completing them before users can see the data the next day. The overnight window may not be enough time for the full load to complete.
- Preserving history. When dealing with a OLTP source that is not designed to keep history, a full load will remove history from the destination as well, since full load will remove all the records first, remember! So a full load will not allow you to preserve history in the data warehouse.
Full
Load vs. Incremental Load:
Full Load
|
Incremental Load
|
Truncates
all rows and loads from scratch.
|
New records and updated ones are
loaded
|
Requires
more time.
|
Requires less time.
|
Can
easily be guaranteed
|
Difficult. ETL must check for
new/updated rows.
|
Can be lost.
|
Retained.
|
Thanks for clarifying the difference!
ReplyDeletewhat is an incremental file & full file in ssis
ReplyDelete