Unlock the Potential of Programming: Coding for Solutions

Tuesday, March 12, 2019

My Master Thesis

Classifying fraudulent Purchasing Behavior Utilizing Machine learning, ETL and database methods 

Summary 

This thesis investigates the unusual purchasing order behavior. It can say that this thesis is very helpful to solve the main problem which is being faced by those companies which are involved to make the Purchase Order of goods to complete their requirements.

This research will play a great role in the following three main areas to help any company.
  • Prevent the company to order the goods less than their requirements i.e. to avoid losing potential opportunities and to avoid delays in projects.
  • Enable the company to have a stable supply chain so that resources can be allocated accordingly.
  • Prevent the company to order the goods greater than their requirements i.e. to prevent white-collar crimes

It will investigate these above scenarios by using Machine Learning technique. It will select more than one Machine Learning algorithms (KNN, Kernel SVM, Random Forest and Artificial Neural Network) and then trained the algorithms from the past clean behavior of the Supply and Demand history of the goods. This is similar to fraud detection in which unusual spending patterns are suspected followed by the investigation to see if the spending is genuine.

The manufacturer company has the various branches all over the world and for the analytical purpose the headquarter needs updated information with its all branches all over the world that’s why Database systems and ETL(full loading, Incremental loading, and SCD type 2 loading ) are also in action in this paper.

METHODS AND RESULTS

Local database -“Thesis _Pak”

The raw data “PurchaseTable_FullLoad.csv,” was imported to that local database by using the SSIS tool. The sample of PurchaseTable_FullLoad.csv looks like the following.

PurchaseTable_FullLoad.csv

In order to extract this data to the local database, the full loading technique was adopted in the following manners. All the work is done in the SSIS tool.

IF EXISTS (SELECT 1 FROM information_schema.tables where table_name like 'tblPurchases_FullLoad')
    DROP TABLE tblPurchases_FullLoad

-- create a table to hold purchase ledger items
CREATE TABLE tblPurchases_FullLoad(
            [DescriptionOrder] [varchar](255) NULL,
            [Supplier] [varchar](255) NULL,
            [QuanlitiesOrder] [float] NULL,
            [CreatedData] [datetime] NULL,
                [Label] [varchar](255) NULL,
           
)
IF EXISTS (SELECT 1 FROM information_schema.tables where table_name like 'tblPurchases_FullLoad')
    DROP TABLE tblPurchases_FullLoad

-- create a table to hold purchase ledger items
CREATE TABLE tblPurchases_FullLoad(
            [DescriptionOrder] [varchar](255) NULL,
            [Supplier] [varchar](255) NULL,
            [QuanlitiesOrder] [float] NULL,
            [CreatedData] [datetime] NULL,
                [Label] [varchar](255) NULL,
           
)


The process to extract the Purchase data workbook and dumped into a local database


Now, the time to load the data to headquarter (USA) of ABC Company from its local branch by using the SSIS tool


SSIS package set up for Incremental Load: loading new record from the source table to target table

SSIS package set up for Incremental Load: updating a record in target table after the data have been altered in a source table


SSIS package set up for Incremental Load: updating record in target table after the data have been altered in source table


SSIS package set up for Incremental Load: updating and adding the new records simultaneously


The head quarter also adopted the SCD2 technique to load the data from its branch, let say that branch is located in Canada, because it wants to keep the historical data from that branch. In order to get this implementation, the two tables have been created. The staging table ([dbo].[PurchasesStaging_SCD2Load]), which is created in its branch while the “dbo.TargettblPurchases_SCD2” table have been built up in the headquarter. Hence, it can say that “dbo.TargettblPurchases_SCD2” acts like a target table while “[dbo].[PurchasesStaging_SCD2Load])” table have been used to extracting the data during the process of SCD2 implementation.
To keep the eyes on new records and changed records in dbo.TargettblPurchases_SCD2 MERGE statement comes and plays its role. The MERGE statement inserting the new records into the dbo.TargettblPurchases_SCD2 but the records firstly inserted into the [dbo].[PurchasesStaging_SCD2Load and here BINARY_CHECKSUM function is used to detect changes to a row of a table 

We don’t have any records at the staging table, which is located at the local branch (Canada).
Staging table at the local branch (Canada)

Let’s say that the local branch have the following records in csv file 

Records in csv file in the local branch

These records have to move to the staging table (source table located at Canada) by using the SSIS tool as shown in the following figure

Building up staging table

Then the staging table looks like the following 
Staging table at the local branch database


While we do not have any records on the target table as shown in the following figure

to be continued .......................


Sunday, March 10, 2019

Contents of Blog

Contents of Blog

Oracle PLSQL
  1. Oracle PLSQL---Function SALARY_VALID() Scenario
  2. Oracle PLSQL--Creating Procedures RAISE_SALARY()
  3. Oracle PLSQL--Explicit Cursor Attributes
  4. Oracle PLSQL--KINDS OF CURSOR AND EXPLANATION OF EXPLICIT CURSOR
  5. Oracle PLSQL--codes-concepts of FUNCTIONS
  6. Oracle PLSQL--FUNCTIONS TYPES
  7. Oracle PLSQL--Conditional Loop PL/SQL
  8. Oracle PLSQL--Indefinite Loop Pl/sql
  9. Oracle PLSQL--Complex extended user-defined types (decode the code )
  10. Oracle PLSQL--“TYPE … RECORD” --- DECLARE
  11. Oracle PLSQL--TYPE-TABLE Declare PL/SQL ---Scenario: We want to extract SSN of manager and worker from our database.
  12. Oracle PLSQL--LOOP ---PL/SQL
  13. Oracle PLSQL--GOTO ---PL/SQL
  14. Oracle PLSQL--LABELS -----Scenario we want to extract the most busy employee and least busy employee
  15. OraclePLSQL--DML Update & Transaction Control in PL/SQL Scenario: Let suppose we want to transfer the employee who is the least busy and we want to transfer into another task so we write the following program
  16. OraclePLSQL--DATA MANIPULATION IN BEGIN CLAUSE pl/sql We want to extract the best employee name and his bdate,gender,ssn,dept name,hours to do work as well as bonus from these three tables and we want to evaluate whether or not lived in a big city or a small town and also evaluate his/her address
  17. OraclePLSQL--DATA MANIPULATION IN BEGIN CLAUSE BY SQL FUNCTION
  18. OraclePLSQL--uses of Cursors
  19. OraclePLSQL--INTRO OF DECLARE VARIABLES PART#02
  20. OraclePLSQL--INTRO OF DECLARE VARIABLES PART#01
  21. OraclePLSQL--AGGREGATE AND ANALYSIS FUNCTIONS AS WELL AS WINDOW FUNCTION
  22. (ANSI vs. Oracle): Inner, Outer, Self

SQL
  1. MSSQL Inline Table Valued function(ILTVF)
  2. MSSQL UDFs Scalar functions
  3. MSsql USER DEFINED FUNCTION
  4. Group By and Having Clauses