The metadata repository serves as a central ________ for storing and accessing information related to data lineage.
- Hub
- Repository
- Vault
- Warehouse
The metadata repository acts as a centralized storage and access point for all metadata related to an organization's data assets, including data lineage information. It serves as a repository or database where metadata is collected, managed, and made accessible to users and systems across the organization. By centralizing metadata in a repository, organizations can ensure consistency, accessibility, and integrity of metadata, facilitating effective data management and governance practices.
________ is a data extraction technique that involves reading data from a source system's transaction log.
- Change Data Capture (CDC)
- Delta Load
- Full Load
- Incremental Load
Change Data Capture (CDC) is a data extraction technique that involves reading data from a source system's transaction log to capture changes since the last extraction, enabling incremental updates to the data warehouse.
Apache Hive provides a SQL-like interface called ________ for querying and analyzing data stored in Hadoop.
- H-SQL
- HadoopSQL
- HiveQL
- HiveQL Interface
Apache Hive provides a SQL-like interface called HiveQL for querying and analyzing data stored in Hadoop. This interface simplifies data querying for users familiar with SQL.
In a data warehouse, what is a dimension table?
- A table that contains descriptive attributes
- A table that contains primary keys and foreign keys
- A table that stores metadata about the data warehouse
- A table that stores transactional data
A dimension table in a data warehouse contains descriptive attributes about the data, such as customer demographics or product categories. These tables provide context for the measures stored in fact tables.
Scenario: Your company needs to process large volumes of log data generated by IoT devices in real-time. What factors would you consider when selecting the appropriate pipeline architecture?
- Data freshness, Cost-effectiveness, Programming model flexibility, Data storage format
- Hardware specifications, User interface design, Data encryption, Data compression
- Message delivery guarantees, Operational complexity, Network bandwidth, Data privacy
- Scalability, Fault tolerance, Low latency, Data consistency
When selecting the appropriate pipeline architecture for processing IoT-generated log data in real-time, factors such as scalability, fault tolerance, low latency, and data consistency are crucial. Scalability ensures the system can handle increasing data volumes. Fault tolerance guarantees system reliability even in the face of failures. Low latency ensures timely processing of incoming data streams. Data consistency ensures the accuracy and integrity of processed data across the pipeline.
What does a physical data model include that the other two models (conceptual and logical) do not?
- Business rules and constraints
- Entity-relationship diagrams
- High-level data requirements
- Storage structures and access methods
A physical data model includes storage structures and access methods, specifying how data will be stored and accessed in the underlying database system, which the conceptual and logical models do not.
What role does data stewardship play in a data governance framework?
- Ensuring data compliance with legal regulations
- Managing data access permissions
- Overseeing data quality and consistency
- Representing business interests in data management
Data stewardship involves overseeing data quality and consistency within a data governance framework. Data stewards are responsible for defining and enforcing data standards, resolving data-related issues, and advocating for the proper use and management of data assets across the organization.
The use of ________ can optimize ETL processes by reducing the physical storage required for data.
- Data compression
- Data encryption
- Data normalization
- Data replication
The use of data compression can optimize ETL (Extract, Transform, Load) processes by reducing the physical storage required for data. It involves encoding data in a more compact format, thereby reducing the amount of disk space needed to store it.
What are the key considerations for choosing between batch loading and real-time loading strategies?
- Data complexity vs. storage requirements
- Data freshness vs. processing overhead
- Processing speed vs. data consistency
- Scalability vs. network latency
Choosing between batch loading and real-time loading involves weighing factors such as data freshness versus processing overhead. Batch loading may offer higher throughput but lower data freshness compared to real-time loading.
________ is a method of load balancing where incoming requests are distributed evenly across multiple servers to prevent overload.
- Content-based routing
- Least connections routing
- Round-robin routing
- Sticky session routing
Least connections routing is a load balancing technique that distributes incoming requests across multiple servers based on the current number of active connections. Servers with fewer connections receive more requests, helping to evenly distribute the workload and prevent any single server from becoming overwhelmed. This approach promotes efficient resource utilization and enhances system reliability by preventing overload on individual servers.