Data Science and Analytics - Tech

Notes

•Big Data: Concept and Market Perception Often dismissed as hype promoted by firms like Gartner, Big Data refers to large volumes of data from new sources. While not entirely new, its significance has grown due to increasing data volumes and sources. •Characteristics and Technical Aspects of Big Data Big Data is defined by scale, distribution, diversity, and speed, requiring advanced technologies for storage and analysis. Key features include: - Volume: From 0.9ZB in 2009 to 35ZB in 2020, a 44-fold increase. - Processing Complexity: Data structures evolve rapidly, necessitating real-time analysis. - Variety: 80-90% unstructured data, demanding tools like MPP. •Data Storage, Analysis, and Visualization Effective management involves storing, interpreting, analyzing, predicting, and visualizing data. It integrates sensors, IoT, and other sources to support decision-making through dashboards, monitoring systems, and collaboration platforms. •Evolution of Data Volume Over Time Data has grown from terabytes in the 1990s to exabytes in the 2010s, driven by technological advances and new companies. Early data management used data islands and warehouses, evolving into analy...