DATA PIPELINE ARCHITECTURE FOR ACADEMIC INFORMATION SYSTEM AT AKADEMI TEKNIK BIAK

Abstract

In development a information system Intergrated, Architecture planning is the first step must be established. The planning of development in a information system is needed in order to a system can be running according to necessity. The data is used for this research, that is internal data of Biak Technical Academy College and external data of Institution of high education service at IV area in Biak Papua. The main goal of this research is design architecture pipelines data of ATB college. The architecture of pipelines is used for carrying resources of big data from one area to the other area in far distance to be efficiency. The method is used for this research, that is Estract Transform Load (ETL). The process of estract data is needed a special supporting library on apache spark in using library spark session. This spark session is established in order to call data of Biak Technical Academy college with csv extension can be run on apache spark. After the process of estract is established, apache spark will read data with csv extension and establish transform data. The process of transform data csv extension will be loaded in to a frame data as a output of processing ETL The result of research is apache spark technology can be easy for writers in design process information system of Biak Technical Academy and to be one of the best solution in processing Estract Load Transform (ETL) data with the big scale and real-time