Theory Driven or Process Driven Predictions?

Conclusion

Indeed, BDA seeks to gain insights "born from the data" and entails "disruptive innovations" with implications how research is conducted: instead of inductively proposing theories from small sample data and/or deductively confirming theories based on theory-driven instruments and data collection, we now process given big datasets step by step to generate relational insights and predictions. The core of the shift pertains to the scientific method employed in BDA. In BDA the research can start with processing huge amount of data to reach data-driven discoveries, rather than starting with theory or with small sample data to be interpreted by humans.

Nevertheless, scientific theories have not become obsolete in BDA research. But the shift towards process-driven generation of insights and predictions poses new epistemological challenges that require different theoretical guidance for each step in the data processing, particularly lightweight theory-driven data and parameter selection, systematic big data validity and reliability reflection, and an overall theoretical framework supporting method selection and result interpretation. All of which should be on the agenda of IS research and need to be addressed by existing and/or to be developed IS theories.

Recently, few research studies, have introduced guidelines on how to conduct big data analytics research. However, triggered by the propagating challenges of 'streetlight' research and data monetization, our study differs from the previous papers by focusing on epistemological challenges: having identified the possible epistemological pitfalls in each step of the BDA process, we have introduced a lightweight theory-driven guidance that aims to improve the governance, acceptability, and eventually trustworthiness of BDA.

This work is only another stepping stone towards reflecting and addressing the epistemological challenges associated with BDA and BDA-based predictions. Further research is required with regards to the impact and support of BDA-based predictions in relation to IS theories as well as management theories. In particular, longitudinal studies on the relation of explanations and predictions are needed in order to appropriately contextualize BDA in the history of science.