Raw data may include formats that are not compatible or inconsistent. Hence, transformation and cleaning of the data are necessary in order to include data coming from various origins:
Normalization: All data must be in one standard unit of measurement and date and currency format.
Data Cleaning: This process identifies any duplicate entries, corrects the error present in the data, and derives missing values.
Data Mapping: It refers to field mapping from one source to another to ensure joined data makes sense. For example, 'Name' in one dataset but 'Full Name' in the other.
Data Transformation will be accomplished using SQL or Python with the pandas library or otherwise with help from data wrangling packages such as Apteryx or Data Wrangler.