Big Data Opportunities and Challenges

Information increases rapidly at a rate of 10x every five years. From 1986 to 2007, the international capacities for technological data storage, computation, processing, and communication were tracked through 60 analogues and digital technologies; in 2007, the capacity for storage in general-purpose computers was 2.9 × 10²⁰ bytes (optimally compressed) and that for communication was 2.0 × 10²¹ bytes. These computers could also accommodate 6.4 × 10¹⁸ instructions per second. However, the computing size of general-purpose computers increases annually at a rate of 58%. In computational sciences, Big Data is a critical issue that requires serious attention. Thus far, the essential landscapes of Big Data have not been unified. Furthermore, Big Data cannot be processed using existing technologies and methods. Therefore, the generation of incalculable data by the fields of science, business, and society is a global problem. With respect to data analytics, for instance, procedures and standard tools have not been designed to search and analyze large datasets. As a result, organizations encounter early challenges in creating, managing, and manipulating large datasets. Systems of data replication have also displayed some security weaknesses with respect to the generation of multiple copies, data governance, and policy. These policies define the data that are stored, analyzed, and accessed. They also determine the relevance of these data. To process unstructured data sources in Big Data projects, concerns regarding the scalability, low latency, and performance of data infrastructures and their data centers must be addressed.

In the IT industry as a whole, the rapid rise of Big Data has generated new issues and challenges with respect to data management and analysis. Five common issues are volume, variety, velocity, value, and complexity according to. In this study, there are additional issues related to data, such as the fast growth of volume, variety, value, management, and security. Each issue represents a serious problem of technical research that requires discussion. Hence, this research proposes a data life cycle that uses the technologies and terminologies of Big Data. Future research directions in this field are determined based on opportunities and several open issues in Big Data domination. Figure 1 groups the critical issues in Big Data into three categories based on the commonality of the challenge.

Figure 1 Challenges in Big Data.