Data Warehouse, Data Mining and Big Data Basics
Data Warehouse, Data Mining and Big Data Basics. Foundational Computer Awareness explanations, examples and explained practice questions for banking and recruitment exam preparation.
By Akhilesh Saini
Learning objectives
Distinguish operational databases from analytical stores, explain data warehousing and data mining, and recognize the common 'V' vocabulary of big data without advanced algorithms.
A transaction screen and a management report have different needs
A banking application commonly records operational events: opening an account, updating status or saving an authorized transaction. This is operational/transactional processing. A regional manager may ask a different question: How did product usage change across branches over several years? Comparing records from many systems, periods and departments is an analytical task.
An ordinary operational database can certainly answer reports, but a data warehouse is especially designed to collect, integrate and organize data from several sources for reporting, analysis and business intelligence. It is not simply a giant backup or a renamed spreadsheet.
A conventional pipeline might look like:
Operational sources → extract/clean/transform → data warehouse → reports/dashboards/analysis.
Data warehouse and the ETL/ELT idea
ETL means Extract, Transform, Load:
- Extract: obtain relevant records from approved sources.
- Transform: clean, validate, standardize and combine them for analysis.
- Load: place the prepared records into the target store.
ELT means Extract, Load, Transform, with transformations done after loading into the target environment. Both approaches exist; choosing one does not determine whether the source data is accurate.
A warehouse can serve organization-wide questions, historical trends and dashboards. A data mart is commonly a more focused analytical subset or store aimed at a department or business area. Its design may depend on the larger warehouse or another integrated platform.
OLTP vs OLAP: a high-yield exam pair
| Term | Full form | Typical purpose | Simple example |
|---|---|---|---|
| OLTP | Online Transaction Processing | Record routine operational transactions | Record an authorized transaction and update records |
| OLAP | Online Analytical Processing | Summarize and analyze data from different angles | Compare monthly product totals by region |
OLTP usually emphasizes many small, controlled operations. OLAP commonly emphasizes aggregations, exploration and analytical queries. The names describe workload styles, not a guarantee that one type of database can do only one task.
What is data mining?
Data mining is the process of discovering useful patterns, associations or relationships from data using statistical, computational or machine-learning techniques. A fictional analysis may find that unusual combinations of transaction time and behavior are associated with higher review rates. Such a pattern may inform a human-designed rule or a fraud-monitoring model, but it is not proof that an individual acted fraudulently.
Examples of basic mining tasks include classification (assigning cases to categories), clustering (grouping similar cases), association analysis (finding items/events occurring together) and anomaly detection (flagging unusual observations). For this curriculum, identifying what data mining does matters more than studying formulas or algorithms.
Data warehousing ≠ data mining: a warehouse stores/integrates data for analysis; mining investigates data for patterns. A warehouse can be a source for mining, but mining is also possible with other datasets.
What is big data?
Big data describes datasets or data-processing demands whose scale, speed, complexity or variety challenge ordinary techniques. It is a relative engineering and analytical idea, not one mandatory file-size threshold.
The common three Vs are:
| V | Meaning | Simple example |
|---|---|---|
| Volume | Amount of data | Large numbers of events/records |
| Velocity | Speed of generation or processing | Rapid incoming events |
| Variety | Different forms | Structured rows, logs, images, text |
Two additional commonly taught Vs are Veracity (reliability/quality) and Value (usefulness of insights). Textbooks vary in how many Vs they list; the three core Vs are a useful starting point.
Warehouse, lake and other terms
A data lake often accepts raw data of differing structures for later processing and analysis. A data warehouse more often standardizes/integrates data for organized analytical queries. A lakehouse seeks to combine aspects of the two. These descriptions are typical patterns, not strict boundaries; modern systems overlap. No product-specific architecture is required for a Computer Awareness exam.
Worked classification example
A bank-like fictional service collects thousands of event records. Saving each allowed event as it occurs is an operational recording task. Combining eligible records from many departments into an analytical store is data warehousing. Searching that prepared data for a recurring pattern is data mining. Building a dashboard showing trends is business intelligence/reporting. Handling extremely high event volumes and rapid streams may involve big-data technologies. These concepts overlap but are not synonyms.
Common confusions and exam traps
- Data warehouse ≠ backup: analytical integration and disaster recovery are different goals.
- OLTP ≠ OLAP: transaction processing versus analytical processing.
- Data mining does not mean mining cryptocurrencies or merely storing data.
- A discovered pattern does not prove causation.
- Big data is not defined by one universal gigabyte count.
- ETL order differs from ELT; both involve preparing data for use.
- Cloud computing and big data can work together, but neither automatically implies the other.
Quick revision
- Data warehouses integrate data for analysis and reporting.
- ETL means Extract, Transform, Load; ELT changes the order.
- OLTP focuses on operational transactions; OLAP focuses on analysis.
- Data mining discovers patterns; a warehouse organizes analytic source data.
- Big data is commonly described by Volume, Velocity and Variety, with Veracity and Value often added.
Practice questions — CA-08-06
1. A data warehouse is mainly used for:
A. Replacing the computer monitor
B. Integrated analysis and reporting across data sources
C. Keyboard input
D. Routing Wi-Fi packets
2. ETL expands to:
A. Enter, Transfer, Link
B. Extract, Test, Login
C. Extract, Transform, Load
D. Examine, Type, List
3. The full form of OLTP is:
A. Online Transaction Processing
B. Online Table Presentation
C. Offline Transaction Printing
D. Open Transfer Link Protocol
4. OLAP is most closely associated with:
A. Replacing a mouse
B. Starting BIOS
C. Bluetooth pairing
D. Summarizing and analyzing data by different dimensions
5. Which one is an example of data mining?
A. Buying additional hard disks
B. Finding recurring patterns or anomalies in a dataset
C. Only copying a file
D. Renaming an extension
6. In the three core Vs of big data, V for "Velocity" refers to:
A. Photo resolution
B. Screen width
C. Speed of incoming or processed data
D. File extension format
7. Which big-data V describes different forms such as logs, images and relational rows?
A. Variety
B. Volume
C. Veracity only
D. Value only
8. What is a sensible difference between OLTP and OLAP?
A. OLAP can never query tables
B. OLTP only creates charts
C. They are two names for email protocols
D. OLTP emphasizes operational changes; OLAP emphasizes analytical queries
9. Data mining and data warehousing differ because:
A. Both mean the same as backup
B. Warehousing integrates analytical data while mining discovers patterns in data
C. Mining is an Ethernet cable
D. Warehousing always means a spreadsheet
10. What is a data mart?
A. An e-commerce website name only
B. A keyboard memory cell
C. A focused analytical data store or subset for a domain/department
D. A universal SQL password
11. Which statement about big-data size is correct?
A. No single universal file-size threshold defines big data
B. It must always be exactly 1 TB
C. It is always unstructured
D. It is necessarily stored in one spreadsheet
12. A possible data-mining flag for unusual behavior is best treated as:
A. Guaranteed proof of wrongdoing
B. A physical document
C. A replacement for all human judgment
D. An analytical signal that may require careful review
Answers and explanations — CA-08-06
1. B — Integrated analysis and reporting across data sources. Warehouses gather and prepare data from multiple sources for analytical queries and reporting.
2. C — Extract, Transform, Load. ETL extracts data, transforms or prepares it, then loads it into a target system.
3. A — Online Transaction Processing. OLTP denotes workloads for recording many routine operational transactions.
4. D — Summarizing and analyzing data by different dimensions. OLAP is Online Analytical Processing, often used for aggregated comparisons and multi-angle reporting.
5. B — Finding recurring patterns or anomalies in a dataset. Mining looks for meaningful patterns and relationships rather than just retaining records.
6. C — Speed of incoming or processed data. Velocity characterizes how quickly data arrives or must be handled.
7. A — Variety. Variety captures heterogeneous data types and structures.
8. D — OLTP emphasizes operational changes; OLAP emphasizes analytical queries. The names describe typical workload goals, although real systems can support a mixture.
9. B — Warehousing integrates analytical data while mining discovers patterns in data. A warehouse organizes/serves data for analysis; mining derives potentially useful patterns from data.
10. C — A focused analytical data store or subset for a domain/department. A data mart commonly targets analytical needs of one subject or department.
11. A — No single universal file-size threshold defines big data. Big-data challenges depend on scale, velocity, variety and processing needs rather than one fixed size cutoff.
12. D — An analytical signal that may require careful review. A statistical anomaly is not automatically evidence of a crime or a causal explanation.