TECHLOGBOOKS

Monday, 7 August 2017

EDQ Interview Questions & Answers – 2

  1. What is the main purpose of Lookup and Return?
Lookup and return is one of the main processor used in the EDQ for the data enrichment. This processors takes one or more attributes as input and returns one or more attributes as output as per the reference data definition.
  1. If you multiple files/sources to read the data, how are going bring all data together in one stream?
First of all create snapshot of all the files and add a reader processor for each file and then by using the Merge processor you can bring all the files together.
P.S : All the files has to be in the same format to bring together in merge process/ you can selectively choose few columns from each file in Merge processor
  1. How will you identify and eliminate duplicates in EDQ ?
In order to just identify duplicates we can use Duplicate check processor by passing one or more  attributes on which duplicates needs to be identified.
In order to eliminate/merge these duplicate, we can use Group and merge processor by passing one or more attributes on which duplicates needs to be merged.
  1. What is the difference between Reference data and Look up ?
Reference data is an object which you create explicitly with Data and definition on which columns to refer and return. It holds both data and definition and more static, i.e. data will not change dynamically.
Lookup is something which you can create using stage data and define which columns to lookup and return and data here is dynamic, i.e. every time the Staged data gets refreshed, look up on that staged data works on refreshed data.
  1. After cleansing the data in EDQ , how will you pass the data to the downstream system or external system ?
This can be done in multiple ways, few of the most popular methods are
  1. Export the final cleansed staged data as a file(.txt,.xls etc.. )
  2. Write the cleansed data to the Staging table in a schema outside EDQ, to do so you need to have a data store pointing to that table beforehand.

No comments:

Post a Comment