INFS8210

联系我们： 手动添加方式: 微信>添加朋友>企业微信联系人>13262280223 或者 QQ: 1483266981

INFS8210 Business Analytics for Managers INFS8210 Business Analytics for Managers Week 2 Descriptive Analytics: Data Warehousing Chapter 3 – 10th edition | not in 11th edition Alex Richardson 2016-2022 1 INFS8210 Business Analytics for Managers Learning Objectives Understand the basic definitions and concepts of

data warehouses Learn different types of data warehousing

architectures; their comparative advantages and

disadvantages Describe the processes used in developing and

managing data warehouses Explain data warehousing operations … Alex Richardson 2016-2022 2 INFS8210 Business Analytics for Managers Learning Objectives Explain the role of data warehouses in decision

support Explain data integration and the extraction,

transformation, and load (ETL) processes Describe real-time (a.k.a. right-time and/or

active) data warehousing Understand data warehouse administration and

security issues Alex Richardson 2016-2022 3 INFS8210 Business Analytics for Managers Main Data Warehousing Topics (acronym overload) DW definition Characteristics of DW Data Marts

ODS, EDW, Metadata DW Framework DW Architecture & ETL Process DW Development DW Issues Alex Richardson 2016-2022 4 INFS8210 Business Analytics for Managers What is a Data Warehouse A physical repository where relational data are

specially organized to provide enterprise-wide,

cleansed data in a standardized format “The data warehouse is a collection of integrated,

subject-oriented databases designed to support

DSS functions, where each unit of data is non- volatile and relevant to some moment in time”

Alex Richardson 2016-2022 5 INFS8210 Business Analytics for Managers A Historical Perspective to

Data Warehousing 1970s 1980s 1990s 2000s 2010s ü Mainframe computers ü Simple data entry

ü Routine reporting ü Primitive database structures ü Teradata incorporated ü Mini/personal computers (PCs) ü Business applications for PCs ü Distributer DBMS ü Relational DBMS ü Teradata ships commercial DBs ü Business Data Warehouse coined ü Centralized data storage ü Data warehousing was born

ü Inmon, Building the Data Warehouse

ü Kimball, The Data Warehouse Toolkit

ü EDW architecture design ü Exponentially growing data Web data ü Consolidation of DW/BI industry

ü Data warehouse appliances emerged ü Business intelligence popularized ü Data mining and predictive modeling ü Open source software ü SaaS, PaaS, Cloud Computing ü Big Data analytics ü Social media analytics ü Text and Web Analytics ü Hadoop, MapReduce, NoSQL ü In-memory, in-database 6 INFS8210 Business Analytics for Managers Characteristics of DWs Subject oriented Integrated Time-variant (time series) Nonvolatile Summarized Not normalized Metadata Web based, relational/multi-dimensional

Client/server, real-time/right-time/active… Example documentation – Oracle Database Data Warehousing Guide https://docs.oracle.com/en/database/oracle/oracle-database/12.2/dwhsg/introduction-data-warehouse-concepts.html Alex Richardson 2016-2022 7 INFS8210 Business Analytics for Managers Data Mart A departmental small-scale “DW” that stores

only limited/relevant data

– Dependent data mart

A subset that is created directly from a data

warehouse

– Independent data mart A small data warehouse designed for a strategic

business unit or a department

Alex Richardson 2016-2022 8 INFS8210 Business Analytics for Managers Other DW Components Operational data stores (ODS) – A type of database often used as an interim area for a data

warehouse Oper-marts – An operational data mart.

Enterprise data warehouse (EDW) – A data warehouse for the enterprise.

Metadata (Data about data) – In a data warehouse, metadata describe the contents of a data

warehouse and the manner of its acquisition and use

Alex Richardson 2016-2022 9 INFS8210 Business Analytics for Managers A Generic DW Framework Data Sources ERP Legacy POS Other OLTP/wEB External

data Select Transform Extract Integrate Load ETL

Process Enterprise Data warehouse Metadata Replication A

P

I

/

M id d le w a re Data/text

mining Custom built applications OLAP, Dashboard, Web Routine Business Reporting Applications (Visualization) Data mart (Engineering) Data mart (Marketing) Data mart (Finance) Data mart (…) Access No data marts option Alex Richardson 2016-2022 10 INFS8210 Business Analytics for Managers DW Architecture Three-tier architecture 1. Data acquisition software (back-end) 2. The data warehouse that contains the data & software 3. Client (front-end) software that allows users to access and

analyze data from the warehouse Two-tier architecture First two tiers in three-tier architecture is combined into one … sometimes there is only one tier Alex Richardson 2016-2022 11 INFS8210 Business Analytics for Managers DW Architectures Tier 2: Application server Tier 1: Client workstation Tier 3: Database server Tier 1: Client workstation Tier 2: Application & database server 12Alex Richardson 2016-2022 INFS8210 Business Analytics for Managers Data Warehousing Architectures

Issues to consider when deciding which

architecture to use: – Which database management system (DBMS) should be

used

– Will parallel processing and/or partitioning be used

– Will data migration tools be used to load the data

warehouse – What tools will be used to support data retrieval and

analysis

Alex Richardson 2016-2022 13 INFS8210 Business Analytics for Managers A Web-Based DW Architecture Web Server Client (Web browser) Application Server Data warehouse Web pages Internet/ Intranet/ Extranet Alex Richardson 2016-2022 14 INFS8210 Business Analytics for Managers Alternative DW Architectures Source Systems Staging

Area Independent data marts (atomic/summarized data) End user

access and

applications ETL Source Systems Staging

Area End user

access and

applications ETL Dimensionalized data marts

linked by conformed dimensions (atomic/summarized data) Source Systems Staging

Area End user

access and

applications ETL Normalized relational

warehouse (atomic data) Dependent data marts (summarized/some atomic data) (a) Independent Data Marts Architecture (b) Data Mart Bus Architecture with Linked Dimensional Datamarts (c) Hub and Spoke Architecture (Corporate Information Factory) 15Alex Richardson 2016-2022 INFS8210 Business Analytics for Managers Alternative DW Architectures Each architecture has advantages and disadvantages! Which architecture is the best Source Systems Staging

Area Normalized relational

warehouse (atomic/some

summarized data) End user

access and

applications End user

access and

applications Logical/physical integration of

common data elements Existing data warehouses Data marts and legacy systems ETL Data mapping / metadata (d) Centralized Data Warehouse Architecture (e) Federated Architecture 16 INFS8210 Business Analytics for Managers Ten factors that potentially affect the

architecture selection decision 1. Information

interdependence between

organizational units 2. Upper management’s

information needs 3. Urgency of need for a data

warehouse 4. Nature of end-user tasks 5. Constraints on resources

6. Strategic view of the data

warehouse prior to

implementation 7. Compatibility with

existing systems 8. Perceived ability of the

in-house IT staff 9. Technical issues 10. Social/political factors Alex Richardson 2016-2022 17 INFS8210 Business Analytics for Managers Teradata Corp. DW Architecture Alex Richardson 2016-2022 18 INFS8210 Business Analytics for Managers Data Integration and the Extraction,

Transformation, and Load Process ETL = Extract Transform Load Data integration

– Integration that comprises three major processes: data access, data

federation, and change capture.

Enterprise application integration (EAI) – A technology that provides a vehicle for pushing data from source systems

into a data warehouse

Enterprise information integration (EII)

– An evolving tool space that promises real-time data integration from a

variety of sources, such as relational or multidimensional databases, Web

services, etc. Alex Richardson 2016-2022 19 INFS8210 Business Analytics for Managers Data Integration and the Extraction,

Transformation, and Load Process Packaged

application Legacy

system Other internal

applications Transient

data source Extract Transform Cleanse Load Data warehouse Data mart Alex Richardson 2016-2022 20 INFS8210 Business Analytics for Managers ETL (Extract, Transform, Load)

Issues affecting the purchase of an ETL tool – Data transformation tools are expensive – Data transformation tools may have a long learning curve Important criteria in selecting an ETL tool – Ability to read from and write to an unlimited number of data

sources/architectures – Automatic capturing and delivery of metadata – A history of conforming to open standards – An easy-to-use interface for the developer and the functional

user

Alex Richardson 2016-2022 21 INFS8210 Business Analytics for Managers Data Warehouse Development Data warehouse development approaches – Inmon Model: EDW approach (top-down)

– Kimball Model: Data mart approach

(bottom-up) – Which model is best Table 3.3 provides a comparative analysis

between EDW and Data Mart approach – Note the source is 15 years old – focus on differences One alternative is the hosted warehouse Alex Richardson 2016-2022 22 INFS8210 Business Analytics for Managers Alex Richardson 2016-2022 23 INFS8210 Business Analytics for Managers Additional DW Considerations

Hosted Data Warehouses Benefits: – Requires minimal investment in infrastructure – Frees up capacity on in-house systems – Frees up cash flow – Makes powerful solutions affordable – Enables solutions that provide for growth – Offers better quality equipment and software – Provides faster connections – … more in the book Alex Richardson 2016-2022 24 INFS8210 Business Analytics for Managers Representation of Data in DW Dimensional Modeling

– A retrieval-based system that supports high-volume query access Star schema

– The most commonly used and the simplest style of dimensional

modeling – Contain a fact table surrounded by and connected to several

dimension tables Snowflakes schema

– An extension of star schema where the diagram resembles a

snowflake in shape Alex Richardson 2016-2022 25 INFS8210 Business Analytics for Managers Multidimensionality The ability to organize, present, and analyze data by

several dimensions, such as sales by region, by product, by

salesperson, and by time (four dimensions) Multidimensional presentation

– Dimensions: products, salespeople, market segments, business

units, geographical locations, distribution channels, country, or

industry – Measures: money, sales volume, head count, inventory profit,

actual versus forecast – Time: daily, weekly, monthly, quarterly, or yearly Alex Richardson 2016-2022 26 INFS8210 Business Analytics for Managers Star versus Snowflake Schema Fact Table SALES UnitsSold … Dimension TIME Quarter … Dimension PEOPLE Division … Dimension PRODUCT Brand … Dimension GEOGRAPHY Country … Fact Table SALES UnitsSold … Dimension DATE Date … Dimension PEOPLE Division … Dimension PRODUCT LineItem … Dimension STORE LocID … Dimension BRAND Brand … Dimension CATEGORY Category … Dimension LOCATION State … Dimension MONTH M_Name … Dimension QUARTER Q_Name … Star Schema Snowflake Schema Alex Richardson 2016-2022 27 INFS8210 Business Analytics for Managers Analysis of Data in DW OLTP vs. OLAP… OLTP (online transaction processing) – Capturing and storing data from ERP, CRM, POS, … – The main focus is on efficiency of routine tasks OLAP (Online analytical processing) – Converting data into information for decision support – Data cubes, drill-down / rollup, slice & dice, … – Requesting ad hoc reports – Conducting statistical and other analyses

– Developing multimedia-based applications – …more in the book Alex Richardson 2016-2022 28 INFS8210 Business Analytics for Managers OLAP vs. OLTP Alex Richardson 2016-2022 29 INFS8210 Business Analytics for Managers OLAP Operations Slice – a subset of a multidimensional array Dice – a slice on more than two dimensions Drill Down/Up – navigating among levels of data

ranging from the most summarized (up) to the most

detailed (down) Roll Up – computing all of the data relationships for one

or more dimensions

Pivot – used to change the dimensional orientation of a

report or an ad hoc query-page display Alex Richardson 2016-2022 30 INFS8210 Business Analytics for Managers OLAP Slicing

Operations on a

Simple Three- Dimensional

Data Cube 31 Product Ti m e G e o g ra p h y Sales volumes of

a specific Product

on variable Time

and Region

Sales volumes of

a specific Region

on variable Time

and Products Sales volumes of

a specific Time on

variable Region

and Products Cells are filled

with numbers

representing

sales volumes

A 3-dimensional

OLAP cube with

slicing

operations Alex Richardson 2016-2022 INFS8210 Business Analytics for Managers Variations of OLAP

Multidimensional OLAP (MOLAP) OLAP implemented via a specialized multidimensional

database (or data store) that summarizes transactions into

multidimensional views ahead of time

Relational OLAP (ROLAP) The implementation of an OLAP database on top of an

existing relational database

Database OLAP and Web OLAP (DOLAP and

WOLAP); Desktop OLAP,… Alex Richardson 2016-2022 32 INFS8210 Business Analytics for Managers DW Implementation Issues Identification of data sources and governance Data quality planning, data model design ETL tool selection Establishment of service-level agreements Data transport, data conversion Reconciliation process End-user support Political issues … more in the book

Alex Richardson 2016-2022 33 INFS8210 Business Analytics for Managers Successful DW Implementation Things to Avoid Starting with the wrong sponsorship chain Setting expectations that you cannot meet Engaging in politically naive behavior Loading the data warehouse with information just because it

is available Believing that data warehousing database design is the same

as transactional database design Choosing a data warehouse manager who is technology

oriented rather than user oriented … more in the book Alex Richardson 2016-2022 34 INFS8210 Business Analytics for Managers Failure Factors in DW Projects Lack of executive sponsorship Unclear business objectives Cultural issues being ignored – Change management Unrealistic expectations Inappropriate architecture Low data quality / missing information Loading data just because it is available

Alex Richardson 2016-2022 35 INFS8210 Business Analytics for Managers Massive DW and Scalability Scalability – The main issues pertaining to scalability: – The amount of data in the warehouse – How quickly the warehouse is expected to grow – The number of concurrent users – The complexity of user queries

– Good scalability means that queries and other data- access functions will grow linearly with the size of

the warehouse Alex Richardson 2016-2022 36 INFS8210 Business Analytics for Managers Real-Time/Active DW/BI Enabling real-time data updates for real-time

analysis and real-time decision making is

growing rapidly – Push vs. Pull (of data) Concerns about real-time BI – Not all data should be updated continuously – Mismatch of reports generated minutes apart – May be cost prohibitive – May also be infeasible

Alex Richardson 2016-2022 37 INFS8210 Business Analytics for Managers Enterprise Decision Evolution and Data

Warehousing 38 INFS8210 Business Analytics for Managers Real-Time/Active DW at

Teradata 39 INFS8210 Business Analytics for Managers Traditional versus Active DW Alex Richardson 2016-2022 40 INFS8210 Business Analytics for Managers DW Administration and Security Data warehouse administrator (DWA) – DWA should… have the knowledge of high-performance software, hardware and

networking technologies possess solid business knowledge and insight be familiar with the decision-making processes so as to suitably

design/maintain the data warehouse structure possess excellent communications skills Security and privacy is a pressing issue in DW – Safeguarding the most valuable assets

– Government regulations (HIPAA, etc.) – Must be explicitly planned and executed

Alex Richardson 2016-2022 41 INFS8210 Business Analytics for Managers The Future of DW Sourcing… – Web, social media, and Big Data – Open source software – SaaS (software as a service) – Cloud computing Infrastructure… – Columnar (data in columns instead of row-based) – Real-time DW – Data warehouse appliances – Data management practices/technologies – In-database & In-memory processing – New DBMS – Advanced analytics – … Alex Richardson 2016-2022 42 INFS8210 Business Analytics for Managers Questions Alex Richardson

INFS8210最先出现在KJESSAY历史案例。