Full Text - International Journal of Application or Innovation in

Transcription

International Journal of Application or Innovation in Engineering & Management (IJAIEM)
Web Site: www.ijaiem.org Email: [email protected]
Volume 4, Issue 12, December 2015
ISSN 2319 - 4847
Performance Enhancement of Data Warehouse
Using Minimization of Query Processing
Proposal to Improve ROI
Bikramjit Pal1, Rajdeep Chowdhury2, Prasenjit Chatterjee3, Saswata Dasgupta4, Mallika De5
1
Assistant Professor and Head, Department of Computer Application
JIS College of Engineering, Kalyani, Nadia–741235, West Bengal, India
2
Assistant Professor, Department of Computer Application
3
Project Engineer, Wipro Limited, India
4
Student, Department of Computer Science and Engineering
5
Professor, Department of Computer Science and Engineering
Dr. Sudhir Chandra Sur Degree Engineering College, Kolkata–700074, West Bengal, India
ABSTRACT
The proposed work based on Pseudo Mesh Schema signifies that instead of a Centralized Data Warehouse, all the data will be
resided over the geographically distant Work Station or Site, which is known as Local Site in the present work and among all
the local sites, one of the Site will be elected as the coordinator Site, which does not contain any data at all, rather it contains a
database consisting of MAC IDs of all the Decision Makers System which is attached to it. The proposed work also describes
that the Users or the Decision Makers communicate with their coordinator only through a query and the Coordinator decides
whether the request is from an Authenticated User or not, by searching the MAC_IDs stored in its own Database. Once the
request from an Authenticated User is received by the coordinator, the coordinator starts to distribute or replicate the requested
query over the Local Sites and waits for the response from the Local Site. It is to be kept in mind that, as the Dimension Tables
are distributed over the Local Sites, all the Sites must not contain the same data. Therefore, the Site which contains the desire
data of the query, responses first to its coordinator and then the coordinator sends a Stop Signal to all other Sites to stop their
execution, and finally the Coordinator dispatches the result of the query to the Destination User. In this way the proposed work
ensures the performance enhancement of Data Warehouse than the existing Centralized Query Processing Approach.
Keywords: Data warehouse, star schema, snow flake schema, pseudo mesh schema, topology, dimension table,
normalized form.
1.INTRODUCTION
Data warehouse has become an increasingly popular topic for researchers in respect to modern trends of
business organizations. It is a set of incorporated databases designed to support decision-making and problem
solving functions, comprising of highly summarized data. Data warehouses are designed specifically to
facilitate comprehensive reporting and adept analysis. Most of the Data warehouse is invariably built using star
schema and snowflake schema. Star schema comprises of one or more fact table(s) connected with dimension
tables. Centre of the star schema consist of one or more fact table(s) and it points to distinguished dimension
tables. Snowflake schema is an extension of star schema where each point of the star explodes into more points.
The distinction initiates from the fact that in star schema, fact tables are in normalized format and dimension
tables are in un-normalized format, keeping queries simple and furnishing fast response time, whereas, in
snowflake schema, both fact tables and dimension tables are in normalized format, thereby reducing the query
performance, on the basis of existence of more joins.
In star schema, each dimension is represented by a solitary dimension table, whereas in snowflake schema, that
particular dimension table is normalized into multiple lookup tables, each representing a level in the dimension
Page 35
ISSN 2319 - 4847
hierarchy. Both these architectures are well accepted by the industry, ignoring the ambiguity associated with
them. In star schema, a single fact table has many dimension tables associated with it and in snowflake schema;
it further creates smaller dimension tables from original dimension table(s). Communication amongst two or
more dimension tables in both the concepts is only through the fact table.
The proposed concept ensures that it will furnish a suitable mode of communication amongst n number of
dimension tables. Since the proposed work is being carried out on the Pseudo Mesh Schema, a glimpse of
Pseudo Mesh Schema is given here before the discussion about the present work. This Pseudo Mesh Schema
has already been well accepted by the Computer Society of India at their First International Conference on
Intelligent Infrastructure [10]. A Pseudo Mesh Schema is exactly like the Mesh Network Topology. In the
Pseudo Mesh Schema a set of Dimension Tables are connected to one another and all the Fact and Dimension
Tables are in normalized form ensuring the elimination of data redundancy. The number of links can be
precisely calculated by using the formula n (n-1)/2, where n is the number of tables (Fact and Dimension)
present within the structure. The structure is obviously a flexible one, as any increase or decrease of one or
more databases within the structure does not create additional overhead.
2. FLOW CHART OF THE ENTIRE WORK
For easy understanding, a glimpse of the entire work is established through a Flow Chart as given in Figure–1.
Figure–1: A Flow Chart of the Proposed Work
3. QUERY PROCESSING AND RESPONSE IN A CENTRALIZED DATA WAREHOUSE
It is vital to understand that how the query of User requests are being processed in the Centralized Data Warehouse. For
easy understanding, a glimpse of the said System is established through Figure–2.
Page 36
ISSN 2319 - 4847
Figure–2: Query Processing and Response in a Centralized Data Warehouse
The above figure illustrates that in the Centralized Data Warehouse all the data reside within the Central Database
Server; all the users used to send their request in the form of query only to the Central Server and once the query would
penetrate to the Query Processor of the Data Warehouse then the Query Processor would start to execute the query.
Whatever be the searching technique, in order to search the desired result of query from the huge amount of data, the
query response time is increased. Since in the Multi-tasking environment, the Data Warehouse Query Processor does
not process only a single query or request, the probability of increasing the query time is much higher than the response
time for processing the single query. On the other hand, this slow query response time becomes a constraint to the
Corporate Decision Makers for quick decision. In the present work an indispensible remedy has been proposed and
discussed.
4. A NOTION OF THE DISTRIBUTED DATA WAREHOUSE
This section illustrates that how the entire data of the Central Data Warehouse can be distributed over the
geographically dispersed Local Sites and for establishing a communication to their Coordinator, how all the Local Sites
are connected to each other by using a Network Topology (Mesh Topology).
For lucid understanding the idea is established in Figure–3.
Figure–3: Distributed Data Warehouse
5. DISCUSSION ON DATA DISTRIBUTION METHOD
As it have been proposed that data will be distributed over the geographically dispersed Local Sites, an unambiguous
Data Distribution method must be mentioned in this section. In the Data Warehouse, the Fact Tables usually contain
different key fields of the Dimension Tables. The key fields may be a Primary Key or a Foreign Key depending upon
the database design. In the Data Warehouse purview, the Dimension Tables contain much more descriptive type of data
in comparison to Fact Tables, thus the replication of Fact Tables will not be a serious overhead to the System. Therefore
the proposal is that, all the Fact Tables of the Global Data Warehouse will be replicated over the Local Sites and all the
Dimension Tables will be distributed in Round Robin Fashion. Before explaining further on the said data distribution
technique, first it is needed to establish a pictorial representation of Global Data Warehouse where all the data reside in
the same server.
Page 37
ISSN 2319 - 4847
Figure–4: Global Data Warehouse Architecture with Mesh Schema
6. DISTRIBUTION OF DIMENSION TABLES
In the proposed work, all the Dimension Tables of the Mesh Schema are being distributed over the Local Sites by using
Round Robin distribution method. The Figure-4 has shown a Global Data Warehouse with Mesh Schema having one
Fact Table and Six Dimension Tables. It is assumed that the entire data will be distributed over the three Local Sites
(Could also be distributed over the N number of Sites), each site having two Dimension Tables, and Fact Table is being
replicated over the all Local Sites as shown in Figure–5.
Figure–5: Distribution of Dimension Tables
7. FUNCTIONALITIES OF THE COORDINATOR SITE
As stated earlier, among all the Local Sites one of the Sites will be selected as the Coordinator Site and this
Coordinator Site does not contain any data but it contains the MAC_ID of all the Users connected to it and the role of
the Coordinator in this regard is very vital.
The Coordinator Site plays the following roles:
 Identification of Authenticated User
 Replication of Query over the Local Sites
 Dispatch of the result of query to the User
7.1 Identification of Authenticated User
In the web environment, one of the major duties of any resource centre is to detect whether the Data Access request is
coming from an Authenticated User or not. In the present work, the Coordinator Site executes this responsibility very
well with the help of the MAC_ID stored in its own Database.
Page 38
ISSN 2319 - 4847
Required Algorithm for User Identification
Step-1: Get the MAC_ID of the Requested User work station by executing the get_macID ( ) subroutine of the
Coordinator
Step-2: Performs a searching in the Coordinator Database with MAC_ID obtained through the user request/query.
If the obtained MAC_ID is found in the Database, then set True to the Authenticated_User_Flag.
Otherwise set False to the Authenticated_User_Flag.
Step-3: Return the Authenticated_User_Flag.
For easy understanding, the above stated functionalities are shown in Figure–6.
Figure–6: User Identification
7.2 Replication of Query over the Local Sites
After the pointing of Authenticated User, the next immediate responsibility of the Coordinator is to replicate or
distribute the query over the Local Sites and wait for a response from the Sites.
For this step, a well designed algorithm can ensure the communication in efficient way between the Coordinator and
the Local Sites.
Required Algorithm for Communication between the Coordinator and the Local Sites
In this algorithm, all the Local Sites are being treated as Participants and the Coordinator as the Coordinator Site.
Step-1: Coordinator: Send the PREPARE Signal to all the Participants and activate a Timeout.
Step-2: Participants: Wait for the PREPARE Signal. If all the Participants are ready then as an acknowledgement they
send a READY Signal to the Coordinator.
Step-3: Coordinator: Wait for the acknowledgement from all the Participants.
A. If the Acknowledgement is obtained before expiry of the Timeout, then the Coordinator activates a
Timeout and dispatches the User Data Access query to each of the Participant and wait for the
Acknowledgement from the Participants.
B. If the Timeout is not expired and Acknowledgement has also not been obtained from all the Participants,
then the Coordinator sends a Global Reset Command to all the Participants and waits for the
Acknowledgement from the Participants.
C. If the Timeout is expired and the Acknowledgement has not been obtained from the Participants, then the
Coordinator sends a Global Reset Command to all the Participants and waits for the Acknowledgement
from the Participants.
Step-4: Participants: Wait for the command Query Execution or the Global Reset and send an Acknowledgement to
the Coordinator.
A. If the command is Query Execution then the Participants starts to execute the assigned query in its local
database and if the desired result is found, then transmits it to the Coordinator and wait for an
Acknowledgement from the Coordinator.
B. If the command is Global Reset then each of the Participants resets their current state and transmits an
Acknowledgement message to the Coordinator.
Page 39
ISSN 2319 - 4847
Step-5: Coordinator: Wait either for the result of the Query or the Global Reset Acknowledgement message.
A. If the Query result arrives within the given Timeout, then the Coordinator sends an Acknowledgement
message to the Participant who was transmitted the Result of the assigned query and sends a STOP Signal
to all the rest Participants and wait for an Acknowledgement.
B. If the given Timeout expires and the Result of the Query has not arrived, then the Coordinator Resends the
Query to all the Participants and waits for an Acknowledgement.
C. If Acknowledgement is of Global Reset, then the Coordinator resets its present state and records the entire
communication in the Log.
Step-6: Participants: Wait for the Stop or the Re–Execution of the Query.
A. If the Signal is for Stop, then the Participants stop the execution of assigned query and transmit an
Acknowledgement message to the Coordinator.
B. If the Query is reassigned by the Coordinator, then the Participants follow the Step A of the Step 4.
Step-7: Coordinator: Wait for the Acknowledgement message.
A. If the Acknowledgement message is Stop of Query Execution, then the Coordinator resets its present state
and records the entire communication in the Log, otherwise follows the Step A & B of Step 5.
7.3 Dispatch the Result of Query to the User
If the Data against the requested Query exists at any of the Local Site, then the Coordinator gets response from that
Site.
Once the desired query result is obtained by the Coordinator, the Coordinator dispatches the result to the Destination
User. Hence, the proposed work ensures the quick response to the user than the existing approach.
8. CONCLUSION
The proposed work in the present paper ensures better performance of the Data Warehouse by implementing
Query Processing Scheme, but in order to implement or simulate the proposed work it is needed to build
specific software which can be implemented as a future work.
In the proposed work, two algorithms have been stated, namely, one for Identification of Authenticated User
and another for Communication between the Coordinator and Participants. Those algorithms will work as a
backbone for communication in the data warehouse environmental setup. As future work those two algorithms
can be implemented in the form of Software.
During communication between the Coordinator and the Local Sites, different failures can occur. The analysis
of those failures are very vital to implement the possible remedy for those failures and that can be considered as
a very good future work. There is a scope of enhancing the searching operation by implementing an efficient
Heuristic Search.
REFERENCES
[1] Rajdeep Chowdhury, Saurab Dutta, Mallika De, “Towards Design, Analysis and Performance Enhancement of
Data Warehouse by Implementation and Simulation of P2P Technology on Proposed Pseudo Mesh Architecture”,
International Journal of Innovative Research in Science, Engineering and Technology, An ISO 3297:2007
Certified Organization, Volume–3, Special Issue–6, February, 2014, ISSN–23476710 (P), ISSN–23198753 (O),
Journal Impact Factor–1.682, DOI–10.15680/IJIRSET, Pages 178–187.
[2] Rajdeep Chowdhury, Prasenjit Chatterjee, Pubali Mitra, Olive Roy, “Design and Implementation of Security
Mechanism for Data Warehouse Performance Enhancement Using Two Tier User Authentication Techniques”,
International Journal of Innovative Research in Science, Engineering and Technology, An ISO 3297:2007
Certified Organization, Volume–3, Special Issue–6, February, 2014, ISSN–23476710 (P), ISSN–23198753 (O),
Journal Impact Factor–1.682, DOI–10.15680/IJIRSET, Pages 165–172.
[3] Rajdeep Chowdhury, Bikramjit Pal, Mallika De, “Proposed Business Principles Governing Enterprise Data
Warehouse Design: Conceptual Framework with Enhancement of Knowledge Infrastructure via Context Model”,
Research Journal of Science and Technology, Volume–3, Number–4, July–August, 2011, ISSN–09754393 (P),
ISSN–23492988 (O), Pages 212–216.
Page 40
ISSN 2319 - 4847
[4] Rajdeep Chowdhury, Bikramjit Pal, Saikat Ghosh, “Proposed Formula Based on Study of Correlation Between
Hub and Spoke Architecture and Bus Architecture in Data Warehouse Architecture, Based on Distinct
Parameters”, Research Journal of Science and Technology, Volume–3, Number–3, May–June, 2011, ISSN–
09754393 (P), ISSN–23492988 (O), Pages 143–146.
[5] Rajdeep Chowdhury, Bikramjit Pal, “Proposed Hybrid Data Warehouse Architecture Based on Data Model”,
International Journal of Computer Science and Communication, Volume–1, Number–2, July–December, 2010,
ISSN–09737391, Journal Impact Factor–1.85, Pages 211–213.
[6] Rajdeep Chowdhury, Soumya Kanta Dey, Soupayan Datta, Sweta Shaw, “Design and Implementation of Proposed
Drawer Model Based Data Warehouse Architecture Incorporating DNA Translation Cryptographic Algorithm for
Security Enhancement”, Proceedings of International Conference on Contemporary Computing and Informatics,
27th–29th November, 2014, SJCE, Mysore, Pages 55–60 Proceedings: IEEE Part Number–CFP14AWQ-ART,
Proceedings in USB: CFP14AWQ-USB, ISBN–978-1-4799-6628-8, INSPEC Accession Number–14881472, DOI–
10.1109/IC3I.2014.7019812 (Published and Archived in IEEE Digital Xplore, ISBN–978-1-4799-6629-5).
[7] Saurabh, A., K., Nagpal, B., “A Survey on Current Security Strategies in Data Warehouses”, International
Journal of Engineering Science and Technology, 3 (4), (2011), pp. 3484-3488.
[8] Vieira, M., Vieira, J., Madeira, H., “Towards Data Security in Affordable Data Warehouse”, 7th European
Dependable Computing Conference, (2008).
[9] Blanco, C., Medina, E., F., Trujillo, J., Jurjens, J., “Towards the Secure Modeling of OLAP Users‟ Behavior”,
Funded Research Project.
[10] Rajdeep Chowdhury, Bikramjit Pal, Amitava Ghosh, Mallika De, “A Data Warehouse Architectural Design
Using Proposed Pseudo Mesh Schema”, Proceedings of The First International Conference on Intelligent
Infrastructure, CSI–2012, 47th Annual National Convention of Computer Society of India, 01st–02nd December,
2012, Science City Auditorium, Kolkata, ISBN (13)–978-1-25-906170-7, ISBN (10)–1-25-906170-1, Pages 138–
141(Published by Tata McGraw Hill Education Private Limited).
[11] Chowdhury, R., Pal, B., “Proposed Hybrid Data Warehouse Architecture Based on Data Model”, International
Journal of Computer Science and Communication, 1 (2), (2010), pp. 211-213.
[12] KALIDO Dynamic Information Warehouse - A Technical Overview, KALIDO White Paper, (2004).
[13] Chaudhuri, S., Dayal, U., “An Overview of Data Warehousing and OLAP Technology”, ACM SIGMOD
Record, 26 (1), (1997).
[14] Patel, A., Patel, J., M., “Data Modeling Techniques for Data Warehouse”, International Journal of
Multidisciplinary Research, 2 (2), (2012), pp. 240-246.
[15] Farhan, M., S., Marie, M., E., El-Fangary, L., M., Helmy, Y., K., “An Integrated Conceptual Model for
Temporal Data Warehouse Security”, Computer and Information Science, 4 (4), (2011), pp. 46-57.
[16] Eder, J., Koncilia, C., “Changes of Dimension Data in Temporal Data Warehouses”, Proceedings of Third
International Conference, Data Warehousing and Knowledge Discovery, DaWaK „01, Munich, Germany, LNCS,
Springer, (2001), pp. 284-293.
[17] Golfarelli, M., Maio, D., Rizzi, S., “The Dimensional Fact Model: A Conceptual Model for Data Warehouses”,
International Journal of Cooperative Information Systems, 7 (2-3), (1998), pp. 215-247.
[18] Golfarelli, M., Rizzi, S., “A Methodological Framework for Data Warehouse Design”, Proceedings of ACM
First International Workshop on Data Warehousing and OLAP, DOLAP, Washington, (1998), pp. 3-9.
[19] Bernardino, J., Madeira, H., “Data Warehousing and OLAP: Improving Query Performance Using Distributed
Computing.”
[20] Jens Albrecht, Holger Gunzel, Wolfgang Lehner, “An Architecture for Distributed OLAP”, International
Conference on Parallel and Distributed Processing Techniques and Applications PDPTA’98, 1998.
[21] D. Comer, “The ubiquitous B-tree”, ACM Computing Surveys, 11(2):121-137, 1979.
[22] Anindaya Datta, Bongki Moon and Helen Thomas, “A Case for Parallellism in Data Warehousing and OLAP”,
Proceedings of the 9th International Conference on Database and Expert Systems Applications (DEXA’98),
September 1998.
[23] D.J. DeWitt, Jim Gray, “Parallel Database Systems: The future of high performance database systems”, Comm.
of the ACM, 35(6), June 1992, pp.85-98.
[24] C.I. Ezeife, K. Barker, “A Comprehensive Approach to Horizontal Class Fragmentation in a Distributed Object
Based System”, Distributed and Parallel Databases, 1:247-272, 1995.
[25] C.I. Ezeife. “A Partition-Selection Scheme for Warehouse Aggregate Views”, Int. Conf. of Computing and
Information, Manitoba, Canada, June, 1998.
[26] “HP Intelligent Warehouse”, Hewlett Packard white paper, http://www.hp.com, 1997.
[27] Marcus Jurgens and Hans-J. Lenz, “Tree Based Indexes vs. Bitmap Indexes: A Performance Study”, Intl.
Workshop DMDW’99, Heidelberg, Germany, 1999.
[28] The Data Warehouse Toolkit, Ralph Kimball, Ed. J. Wiley & Sons, Inc, 1996.
Page 41
ISSN 2319 - 4847
[29] S.J. Lim and Y.-K. Ng, “A Formal Approach for Horizontal Fragmentation in Distributed Deductive Database
Design”, Int. Conference on Database and Expert Systems Applications (DEXA’96), pp234-243, Zurich,
Switzerland, September 1996.
[30] Hongjun Lu, Beng Chin. Ooi, and Kian Lee Tan, Query Processing in Parallel Relational Database Systems,
IEEE Computer Society, May 1994.
[31] M.E. Meredith and A. Khader, “Divide and Aggregate: Designing Large Warehouses”, Database Programming
and Design, 9(6), June 1996.
[32] M. Ozsu and P. Valduriez. Principles of Distributed Database Systems. Second edition, Prentice-Hall, New
Jersey, 1999.
[33] Arnon Rosenthal and Edward Sciore, View Security as the Basis for Data Warehouse Security, Proceedings of
the International Workshop on Design and Management of Data Warehouse (DMDW’2000), Sweden, June, 2000.
COMMUNICATING AUTHOR(S)
Mr. Bikramjit Pal is currently working as Assistant Prof. and Head in the Department of Computer
Application, JIS College of Engineering, Kalyani, Nadia, West Bengal from October 2006. He has
an overall experience of almost 17 years of teaching post graduate and under graduate students. He
is an MCA from Berhampur University, Berhampur, Orissa of 1998 batch and has obtained B. Sc.
(Hons.) degree in Statistics from B. H. U., Varanasi in 1995. Mr. Pal has been involved in active
research for the last five years and presently pursuing Ph. D. from Department of Engineering and
Technological Studies, University of Kalyani, West Bengal.
Mr. Rajdeep Chowdhury is an Assistant Professor in Department of Computer Application at JIS
College of Engineering, Kalyani, Nadia, West Bengal, India. Mr. Chowdhury is presently pursuing
Ph.D. at Department of Engineering and Technological Studies, University of Kalyani. Mr.
Chowdhury was awarded Masters of Computer Application (MCA) from JIS College of Engineering
under West Bengal University of Technology. Mr. Chowdhury has authored/co-authored Fifty Seven
(57) research articles/papers in refereed National/International Journals and National/International Conferences.
Mr. Chowdhury has worked as Program Committee Member/Reviewer for numerous National/International
Conferences as well as National/International Journals and has been part of numerous Editorial Boards as well. Mr.
Chowdhury was invited as Delegate/Speaker to deliver an Invited Lecture at 3rd International Conference on
Foundations and Frontiers in Computer, Communication and Electrical Engineering, C2E2 2016, held at Supreme
Knowledge Foundation Group of Institutions (SKFGI), West Bengal in January, 2016. Mr. Chowdhury was invited as
Delegate/Speaker for Invited Talk on ‘MCA–Admission Eligibility and Career Opportunities’ in ‘Career Plus Live’
Program at DD Bangla Television Channel in July, 2013, aired Live.
Mr. Chowdhury has been recipient of numerous Accolades, Awards and Honor by Corporate bigwigs and Academia.
Few notable of which being; Winner of Telegraph Special Prize in Science & Engineering Fair 2016, organized by
Birla Industrial and Technological Museum (BITM), National Council of Science Museums, Ministry of Culture,
Government of India; Session Chair at 3rd International Conference on Foundations and Frontiers in Computer,
Communication and Electrical Engineering, C2E2 2016, held at Supreme Knowledge Foundation Group of Institutions
(SKFGI), West Bengal in January, 2016; Included in Marquis Who’s Who in the World 2016–33rd Edition; Included in
the 2000 Outstanding Intellectuals of the 21st Century–9th Edition, by International Biographical Centre, Cambridge,
England; Awarded and Facilitated for coining the IIT Pedagogical Course ‘MEET 2K15’ by IIT Bombay; Awarded
Gold Certificate in Infosys Campus Connect INSPIRE–Faculty Excellence Awards 2013; Winner in Infosys Campus
Connect INSPIRE–Faculty Excellence Awards 2013; Top Practitioner in WIPRO Mission 10X and Archived in
Mission 10X BookRack; Winner of 2nd Prize in 4th All India Inter Engineering College Academic Meet 2013; Winner
of 1st Prize in 3rd All India Inter Engineering College Academic Meet 2012; Winner in Infosys Campus Connect
Faculty Contest Series–INSPIRE 2012; Conferred with Certificate of Recognition for Outstanding Contribution by
Infosys Limited in 2011.
Page 42

Full Text - International Journal of Application or Innovation in

Transcription

Similar documents

HAIRHOUSE WAREHOUSE

Cirrus Tech Core Capabilities THE CRANE

Rooms To Go Distribution Center

Egulf-1d

rampion headquarters

11820 Central Parkway

Facility Rental Packet

1Base Bldg

High storage density in the frozen storage warehouse for

Gekas Ullared - LIT1132-2.cdr