BUS250: Introduction to Business Intelligence and Analytics

Business intelligence systems can provide organizations with a significant competitive advantage. By using BI systems to improve operational efficiency, make better decisions, enhance the quality of services offered to customers, and reduce risk, organizations can deliver goods and services of higher quality and at a lower price. These advantages will only grow as the amount of data and the complexity of available data increases. BI systems can drive business success through data mining, extraction, and analysis. A critical component of BI is data visualization, which involves presenting visually appealing and easy-to-understand insights. Dashboards, reports, and interactive visualizations help users interpret complex data sets quickly and make informed decisions. Moreover, BI enables self-service analytics, empowering non-technical users to explore data and generate insights independently, reducing reliance on IT departments for data-related tasks.

Read this historical overview of business intelligence. Notice how BI systems evolved with the development of large-scale database systems to support online and real-time transaction processing. As organizations started to accumulate large amounts of data, this naturally led to the desire to analyze that data to provide insights to support decision-making. The rise of specialized data repositories like data warehouses and easy access to external and

This video explains the key components of a BI architecture. Pay particular attention to the needs of the various stakeholders, those who interact with the system. Note how different types of people use the functional components of a BI system. Then, think of a company from your experience or one that you have researched. Who are the different stakeholders (groups associated with this company somehow)? What are their needs, and how do they differ from each other? How does each group interact with the BI system? Compare this with the examples in the video. What similarities and differences do you notice?

Read this brief definition of business intelligence. What are the main reasons organizations use BI systems? Make sure that you understand the definition of business intelligence and how organizations can use it.

Read this paper for an overview and examples of how big data is used in specific areas, such as supply chain management, risk management, and logistics of business in industry. One of the biggest issues for analysts with big data is knowing how to separate the valuable data from that which does not help answer their requirements. Consider the times, even in school, when you cannot find the right information. Sometimes, narrowing search terms can be difficult if you are unfamiliar with the topic. Sometimes, people describe intelligence as 'connecting the dots', but it is rarely simple, like a 'paint-by-numbers' art project. The dots are not just lying around waiting to be connected. More appropriately, it has been described as filtering out the right radio signals from the fray in a huge city. You have to be carefully tuned to your requirements, as these guidelines keep you on track to finding the right data to answer the questions you need to focus on rather than following rabbit holes and finding yourself in the weeds, awash in signals. What are some examples where you have had to make decisions and were concerned about the quality of the data you used to make those decisions? What did you do to 'connect the dots'?

Business intelligence gives organizations the critical information to generate a competitive advantage. Play the following video case study from the distribution and manufacturing industry. As you listen to the case, note how BI technology has provided a competitive advantage in this industry. What is the nature of the competitive environment? In what specific ways do BI systems and competitive advantage? Think of at least three other businesses or industries. How could a BI system add a competitive advantage in those? What are the top attributes of a BI system that you believe contribute to this?

This paper is the result of research surveying executives and includes robust analysis, which offers some insight into their needs. The results in this case study show that current tools were not sufficient, and more information architecture using data warehousing, OLAP tools, and data mining were required to equip them for their information needs and better decision-making. Take note of when you have found yourself without the tools to perform your best analysis. Were you able to obtain the tools and information you needed, or did you have to be creative or

Because data is critical to making business decisions, we will find that a large part of business intelligence is the study of data management systems. This video explains the differences between a database (where we often find our transaction data), a data warehouse (where we organize data at a higher level of aggregation), and a data lake (which can be quite unstructured). We will also need to start thinking about getting data from our database sources to our data warehouse. We call this process data mining and ETL. More on this later in the course. Can you identify examples of data warehouses from your personal experiences or companies you have researched? How was the data warehouse and the data in it used to support the needs of executives? How would you go about deciding if a given data warehouse is effective in meeting executives' needs? How would you score the warehouse?

Watch this video on integrating diverse data sources into Power BI, a common BI tool. Using diverse data sources presents challenges such as ensuring data quality and consistency across varying formats and structures, integrating disparate datasets into a unified format, addressing concerns related to data privacy and security, mitigating biases inherent in the data, managing interoperability issues between different technologies and standards, scaling data processing pipelines to handle large volumes of data, establishing robust data governance practices, and managing the associated costs of infrastructure, tools, and expertise required for handling diverse datasets. As we will see, these can be addressed with a data warehouse.

We will look at various software tools organizations use to implement business intelligence systems throughout the course. As an introduction to these tools, play the video describing Microsoft Power BI, a very popular and widely used system. Think about businesses that you are familiar with. How could you see a tool like this being used in those businesses? What kind of strategic advantage might a tool like this provide? Also, note that a data analysis and presentation tool like this is only as good as the available data. In the next module of the course, we will look in more detail at what data is available to organizations, how that data is organized, and how it can be mined and extracted for use in a BI system.

Watch this video to learn more about KPIs (key performance indicators). They are the few indicators that can determine the health of the whole enterprise. Just like your blood pressure is a simple-to-measure KPI that can give insight into your overall health, the KPIs of an organization determine the overall health of the organization. The challenge is identifying them and then designing our BI system to specifically track them. We have a lot of data we could be distracted by and must carefully focus on the KPIs. Simply throwing all the data we have access to into the BI system is known as the

This article discusses the data sources organizations use to populate their business intelligence systems. In addition to internal data sources, often created due to the operation of business transaction processing systems, organizations are now also using numerous external data sources. Such external data sources may include government data, social media data, research data, and so on. Think about the kind of data that you use in your personal or business life. What data do you require to make decisions? Where do you find this data? How do you process or analyze this data to help you make decisions?

In addition to coming from both internal and external sources, data may be available in many different formats. One broad classification is to divide data into structured and semi-structured data. In general, structured data is data that is capable of being stored in alphanumeric format in relational database processing systems. Semi-structured data is more difficult to process and may include text, images, videos, social media posts, and so forth. Think about the kinds of data you encounter in your personal or business life. How would you classify your data?

Perhaps the most significant issue associated with data is the quality of the data. Data quality often boils down to trust. Do I trust that the external data provider is taking the same steps I would take to ensure that data is truthful and reliable? As the amount of data available increases, the issue of data quality becomes more prominent. The more data we use, the more risk of data quality issues. As you view this video, reflect on what you can do to ensure that the quality of the data you use in your business intelligence systems remains high. Pay attention to the concept of data governance. We will look at this in more detail later in the course.

In addition to data quality, we must also consider the relevance of the data to our particular needs. Determining the relevance of data to a business problem involves several methods, including defining clear business objectives and key performance indicators (KPIs) to guide the analysis, conducting thorough exploratory data analysis (EDA) to understand the characteristics and patterns within the data, leveraging domain expertise and subject matter knowledge to assess the potential impact of the data on solving the business problem, employing statistical techniques and machine learning models to identify correlations and predictive relationships between variables, conducting hypothesis testing to validate assumptions and hypotheses and engaging stakeholders to gather feedback and insights on the usefulness and relevance of the data in addressing the business problem.

Poor data quality can severely affect management decisions, leading to erroneous analyses, flawed insights, and misguided strategies. Inaccurate, incomplete, or inconsistent data can undermine the credibility and reliability of decision-making processes, resulting in suboptimal resource allocation, ineffective risk management, and missed growth opportunities. Moreover, decisions based on poor-quality data may lead to costly errors, damaged reputation, and diminished stakeholder trust. Without reliable data to inform decision-making, organizations risk making ill-informed choices that can negatively impact performance, competitiveness, and long-term success. Play the video, which describes what one organization did to improve the quality of the data they use in BI systems to support decision-making. What are the top 3-5 lessons that you took away from this discussion?

Data integration in business intelligence refers to combining data from different sources or systems within an organization to provide a unified view for analysis and decision-making purposes. This integration typically involves merging data from disparate databases, applications, and platforms into a coherent data repository or warehouse. Data consolidation combines data from various sources, such as transactional databases, ERP systems, CRM platforms, spreadsheets, and external sources like social media or market research data. Data transformation involves converting and standardizing data formats, structures, and semantics to ensure consistency and compatibility across different sources. It may also include data cleansing, normalization, and enrichment to improve data quality. Data synchronization ensures that data across different systems is kept up-to-date and synchronized in real-time or at regular intervals to provide users with timely and accurate information. Data governance involves implementing policies, processes, and controls throughout the integration process to ensure data quality, security, and compliance with regulatory requirements. Data modeling involves designing a logical and efficient data model that represents the integrated data in a way that supports meaningful analysis and reporting. This may involve creating dimensional models for data warehouses or data marts. By combining data from disparate sources and providing a unified view of information, data integration in business intelligence enables organizations to gain valuable insights, make informed decisions, and improve overall operational efficiency and effectiveness.

Business intelligence tools can only be effective if we provide them with the right data. The process of designing a system to do this is data engineering, and the specific processes are referred to as ETL: Extract, Transform, and Load. The idea is conceptually simple, yet identifying all the available data sources and formats and then developing systems to extract and transform this data can get complicated. What would be the top three or four challenges associated with doing this? How would you overcome these challenges? Consider an organization you are familiar with. What types of data does this organization need? How would you begin to approach the process of data engineering to perform an ETL process on that data?

One of the biggest challenges associated with BI systems is the need to integrate multiple and disparate information systems. Think about a BI application. What types of integration issues could arise, especially among different database systems? How could the concepts of systems integration be applied?

Big data is a term that has emerged in the last several years. The most significant implication of big data for business intelligence is that now we need to think about the data feeding our BI systems as coming from potentially anywhere. In the past, the data we used was mostly generated internally and mostly in a very structured format. Now, data may come from anywhere and be in any format. Specialized tools like Hadoop have been developed to help us extract big data for inclusion in our data warehouse. Think about a business that you are familiar with. What types of external big data sources would you want to include in your data warehouse to support BI? Where will you get this data? What format will it be in?

Many relational databases are what we call SQL databases. In other words, we can use Structured Query Language (SQL) to query the database and return results. Because of the many different types of data that we may want to use for business intelligence, NoSQL, which should be read as

This video presents a detailed case analysis of how Big Data is used to enhance BI in the consumer packaged goods industry. Notice the types of questions that big data and the analysis of big data can answer. This analysis can add insights to managerial decisions and improve a firm's competitive advantage. In addition to the situations described here, what are some other examples of how big data analysis can add insight? Also, think about an organization or industry that you are familiar with. What types of KPIs are measured to determine success? What kind of data is gathered to measure those KPIs? How could the data analysis described in the video add insight and lead to improved decision-making in your example?

Read these foundational concepts about databases and how data is stored in databases. Note that when discussing databases in this context, we are almost always talking about relational databases. Relational databases are how organizations store their internal transaction processing data. We refer to systems that process transactions as OLTP: OnLine Transaction Processing systems, recognizing that almost all business transactions are completed online. We noted earlier that firms may seek internal and external sources and that data can be structured or semi-structured. While not a general rule, it is common for internal data to be stored in relational databases and external data in other formats and structures.

This video presents a framework for data quality assessment. Now that we have defined data quality, we must consider how we will assess it and what processes we can implement. Think about an organization you are familiar with, or research an organization. What processes are in place in that organization to assess data quality? How would you evaluate the effectiveness of those processes?

Data governance encompasses the policies, processes, and controls established within an organization to ensure the effective management, quality, security, and compliance of its data assets throughout those assets' lifecycle. It involves defining roles and responsibilities for data management, establishing standards and procedures for data collection, storage, and usage, and implementing data quality assurance, metadata management, and access control mechanisms. Data governance aims to align data management practices with business objectives, mitigate data misuse or loss risks, and foster trust in data integrity and reliability among stakeholders. By enforcing consistent and transparent data practices, data governance enables organizations to maximize the value of their data assets, support regulatory compliance, and drive informed decision-making.

A data warehouse is a specialized repository where we gather data from various systems and sources and store it all in one place. We then extract data from the data warehouse to analyze and support business intelligence systems. Note how data warehouses are organized, how data is moved from many different and diverse sources to the data warehouse, and how a BI system uses data from the warehouse. Can you identify an example of an organization that uses a data warehouse? Do some research and see if you can find one. How is the data warehouse in the example you found organized? What are some of the technologies used, and what types of BI systems does the data warehouse support?

This video describes the main components of a data warehouse architecture, including the ETL components. A data warehouse architecture refers to the structural framework and design principles that govern the organization, storage, retrieval, and analysis of data within a data warehousing environment. It typically encompasses several components and layers that work together to facilitate efficient data management and analytics. Take note of the key components of a data warehouse architecture. If you were on a data warehouse development team, how would you approach the design of a data warehouse?

Watch this video, which discusses some of the issues and processes associated with data warehouse design. What do you think are the key challenges? An effective data warehouse design process could address these challenges. Can you identify companies that have done a good job of data warehouse design? Why were they successful?

There are many different ways that data can be represented in a database. We call these data models. Read this selection, noting each type of data model. You'll want to be able to explain the similarities and differences between each data model. Then, complete the exercise at the end of the lesson.

Read this page and list the models listed in order of decreasing level of abstraction. Pay attention to the summary given on conceptual models. Complete the exercises at the end of the chapter.

The integration of data management and data warehousing involves aligning strategies, technologies, and methods to effectively collect, store, manage, and analyze data assets within an organization. Data management encompasses the practices and policies for ensuring data quality, integrity, security, and compliance throughout its lifecycle. In contrast, data warehousing focuses on creating centralized repositories for storing and organizing data from multiple sources to support reporting, analytics, and decision-making. By integrating data management principles and practices into the design, implementation, and operation of data warehousing solutions, organizations can establish robust frameworks for data governance, metadata management, master data management, and data quality management, enabling them to derive actionable insights, enhance data-driven decision-making, and drive business value effectively. This integration facilitates the seamless flow of high-quality data across the organization, empowering users to access trusted, consistent, and relevant information to support their strategic initiatives and operational processes.

Preparing data for BI analysis is commonly known as data scrubbing or cleaning. Generally, it can be a very long, large, and complex process, but the concepts are fairly straightforward. Watch this video, which describes the data cleaning process in Excel. How could you extend these concepts to a more complex, multi-vendor database environment? What would be some of your first steps if you were in charge of data cleaning for a BI system? What challenges do you expect to encounter?

These modules introduce you to some fundamental concepts of data analysis and interpretation. Human decision-makers face the challenge of interpreting the data they have collected and stored in their data warehouse to impart meaning to the data. Once we understand the meaning of the data and can interpret it properly, we can use this information to make decisions. The first step in data analysis often involves presenting data in forms other than simple lists and tables. In this unit, we will explore some formats – graphs, charts, and diagrams. We will then explore how we can organize these data presentation formats into what we call a business intelligence dashboard. Much like the dashboard on your car, we want to present the most critical information in the most convenient form for the decision-maker and store less useful data in a less visible location.

Exploratory data analysis is the process of taking data and formatting it into some kind of useful context, typically through visualization. This video describes some common ways we can present data for exploratory analysis. As you watch the video, note how the process of transforming data into charts, graphs, and other visual representations aids in understanding the data and its context. Then, identify three or four data items you use regularly, either in your professional or personal life, like your credit card spending. What form of data presentation would you use for this data? How would the presentation of the data in this form allow you to make better decisions?

Watch this in-depth presentation of data analysis and how data analysis can add insights that support business decision-making. Take note of the examples presented and how the data analysis tools were selected based on the needs of the data. Note also the discussion of data analysis tools for unstructured text data at the end of the video. Remember that data can come from inside and outside the organization and can be either structured or semi-structured (or even unstructured). The challenge in designing a Business Intelligence system is identifying the most appropriate analysis tools for a given situation.

Read this article, which describes leaders' unique needs for good information to make decisions. How do the concepts you have studied so far in the course support the needs of executive decision-makers? Identify examples of the types of decisions a strategic decision-maker might make and what data would be required. How can this data be translated into actionable insights?

Read this article, which examines the information needs of managerial decision-making. In decision-making, context knowledge refers to understanding the surrounding circumstances, environment, and constraints that influence the decision, encompassing factors such as background information, stakeholders, risks, and potential consequences. It provides the broader perspective necessary for informed decision-making by considering the context in which choices are made. On the other hand, domain knowledge pertains to expertise and specialized understanding within a particular subject area or field, providing insights into the specific concepts, principles, and best practices relevant to the decision. Domain knowledge enables decision-makers to assess the feasibility, effectiveness, and potential risks associated with different courses of action within their area of expertise. Both context and domain knowledge are crucial in making well-informed decisions that align with desired outcomes and objectives.

As the name implies, data mining is searching through large amounts of data to extract the elements that will be most useful to the BI system. We then store the data extracted by data mining in the data warehouse. Read this article to learn how data mining employs techniques from statistics, pattern recognition, and machine learning to support decision-making. The article further clarifies the other key algorithm families used in data mining, such as predictive modeling and segmentation.

This video presents a more detailed construct or way of thinking about data mining and how it is used to extract information and insights from the data stored in our data warehouse. As you watch, think about the decisions that you must make. Which techniques and concepts would you use to mine the data in your warehouse?

Text is a special kind of data. Much of the world's information is stored not in organized relational databases but rather in structured or unstructured text. Structured text might be a country's legal codes, case law, court rulings, and other formal documents. Unstructured text might take the form of comments about our products or services made by customers on a social media system. Much insight can be gained from textual data, but it can be hard to sort, read, and organize. The emerging techniques of text mining can come into play here. Watch this video for a discussion of the role of text mining and some of the associated text mining techniques. How could you make use of text mining? Identify some examples. Where is the text data located? How will you mine the data? How will you organize the insights gained from the data? How will the insights allow you to make better decisions? What would be an example?

This video describes techniques for exploring text data in more detail. Notice how text data can add insight into business, social, and personal problems. We can extend the rather simple methods used in text analysis into the broader area of Natural Language Processing (NLP). In NLP systems, we rely on the computer to process the text more to add insight. As you watch the video, make sure you can define the various terms used.

The first level of data analysis has to do with visualization. Visualization techniques take the data we have gathered and present it to human analysts and decision-makers. These different visualizations can then be further grouped into reports, business intelligence dashboards, animations, and so on. The objective is to help humans achieve greater insight into the data than would be possible by simply looking at the raw data. This video covers the basic principles of data visualization. Note the characteristics of the various types of visualization. Then, think about the data you use to make decisions in your professional or personal life. How would visualization aid you in understanding what the data is telling you? What type of visualization would you use?

This video expands on the notion of data visualization and discusses some of the characteristics of human beings and human perception that make visualization of data such a powerful technique. As you watch, note how data visualization is influenced by human perception. Also, think about different people that you know. Is there such a thing as an

We already introduced the concept of a BI dashboard. Now, we want to think about some of the techniques and methods we can use to design a BI dashboard to meet an organization's specific needs. This page gives a good overview of some things that go into the design of a BI dashboard. Think about a particular situation, organization, or context you are familiar with. Which of the techniques introduced in this module would you use to design a BI dashboard in your specific case? Why did you select those?

R is one of the most commonly used data presentation tools. Data visualization and presentation is one of the key elements and skills of business intelligence. Human decision-makers are inherently more receptive to information that is presented in a visual format. The developer's challenge is creating mechanisms to display data and information in a form that best facilitates understanding the underlying meaning. We can present data in many different ways, and selecting a specific presentation format can greatly affect the usefulness of the underlying information. All the time and effort we spend designing and implementing our data gathering, data warehousing, and data management systems could be lost if we cannot present the data we have obtained in an understandable format.

Tableau is widely used to create data visualizations. The core product is the Tableau desktop. Play the following video for a comprehensive demonstration of Tableau and a discussion of how it can be used to create effective data visualizations for a BI dashboard. If you want to, try downloading Tableau and experimenting with some of the data formats. How easy or difficult do you find It to use? What types of applications could you envision for software like this in an organization you are familiar with?

These videos discuss a data visualization technique known as a heat map. When we create a two-dimensional data visualization, we call it a heat map. Colors represent the values of the individual cells in the heat map. The color variation may be by hue or intensity. Heatmaps can be very useful for visualizing data in some types of applications. After you have played both videos, identify some possible applications from your experience. Why did you select these data for presentation via a heatmap? How do you think using a heatmap would improve a decision-maker's understanding of the data and lead to better decisions?

Bubble charts are another commonly used technique for presenting data. Watch this video and note how bubble charts are constructed. How do they add value to data? Can you think of any examples from your experience where bubble charts would be the appropriate tool for data presentation?

This video presents a comprehensive overview of many commonly used data visualization techniques. Many methods and techniques are available. Can you think of examples from your experience of using these techniques? How do people generally use these data presentation techniques? Can you identify any common patterns? How might the type of people viewing the data influence the method selected? For example, are engineers different from accountants in how they like to see data presented? Do people from one country or culture prefer different data presentation formats than people from other countries or cultures? Why do you think that is?

Text is a special kind of data, and text mining is a specialized technique to retrieve text data from large, unstructured data repositories such as legal codes. However, once we have mined the text data, we must now think about how we can deduce meaning from the text. This can be a real challenge. We could, of course, simply read all the text, but this could be very time-consuming and requires a great deal of specialized expertise. New techniques are being developed to rapidly retrieve meaning from text-based data to address these challenges. One of these techniques, word clouds, can be particularly useful for quickly gaining a sense of what meaning is conveyed by text. Watch this video to see how word clouds are constructed and how they can be used to understand the meaning in text. What are some types of text data that you rely on? How might a word cloud help?

We can deduce the underlying sentiment or emotion inherent in a piece of text with sentiment analysis. We can use this natural language processing technique to try to understand the author's sentiment, typically from positive to negative. Storyboarding is a technique used in various creative fields to visualize and plan the sequence of events or interactions. Storyboarding involves creating a series of sketches or frames representing key scenes or moments in a narrative. Sentiment analysis techniques could be used to analyze the emotional tone of the story being depicted in the storyboard. For example, sentiment analysis could be used in advertising to analyze customer reviews or social media comments about a product. The insights gained could then be used to inform the creation of a storyboard for a new advertising campaign so that the narrative resonates with the target audience and evokes the desired emotional response.

Until now, we have been discussing data analysis primarily through data presentation, such as presenting data in graphs, charts, and other visuals. We also talked about dashboards and different ways to organize these visual presentations. In this unit, we will look at an entirely new level of data analysis. In addition to viewing and presenting data visually, you can process it using analytic techniques. These techniques could include statistical analysis, forecasting, and various specialized mathematical methods for modeling and analysis. This approach and these techniques are collectively called data science, and practitioners are called data scientists. One of the most common things these data scientists do is create models. Models are a representation of reality, and they allow us to test different scenarios and explore different situations. Financial modeling was one of the earliest uses and is quite well understood. But think about other kinds of models. What models do you use in your work? How do you use them?

Data science is the broad field of applying statistical and quantitative analysis techniques to data. Watch the video for an introduction to the field and some examples of how statistical and quantitative approaches can be used to solve business problems.

Watch this video detailing how data science models can be applied in business intelligence. A data science model is a computational framework or mathematical representation designed to analyze and extract insights from data. These models are constructed using various statistics, machine learning, and artificial intelligence techniques to identify patterns, make predictions, or optimize processes within datasets. Data science models can range from simple linear regression models to complex deep learning architectures, depending on the complexity of the problem being addressed and the nature of the data available. They are employed across diverse domains such as finance, healthcare, marketing, and manufacturing to support decision-making, drive innovation, and enhance understanding of complex phenomena. The effectiveness and accuracy of models in business intelligence can be determined through a comprehensive evaluation process encompassing multiple facets. Initially, the model's performance metrics, such as precision and recall, are indicators of its accuracy in predicting outcomes. Assessing the model's ability to generalize to unseen data through techniques like cross-validation provides insights into its robustness. Beyond numerical metrics, stakeholder feedback and domain experts' assessment contribute to understanding the model's relevance and utility within the business context. Regular monitoring and recalibration of the model based on evolving data and business needs to ensure its continued effectiveness over time.

Read this material on mathematical modeling. Try to complete the exercise at the end of the chapters. Don't worry if you have not yet learned the math necessary to complete the exercises. Rather, notice how many physical systems can be represented by mathematical models. We can then implement these models using programming languages like Python and R. We will discuss those later. Think about the types of systems you interact with, personally or in business situations. How might you use a mathematical model to help you understand this system?

This section gives an example of how the model developed earlier could be extended to account for additional knowledge of the system's structure. Again, if you have not yet been exposed to this particular type of mathematical analysis, simply note that all models can be refined as we gain more knowledge about the system. Think back to the model you identified in the last section. How could you extend this model to make it a better reflection of reality?

Watch the video on predictive analytics, a kind of technology that learns from experience (data) to predict the future. Predictive analytics in business involves using statistical algorithms, machine learning techniques, and data mining to analyze historical data and identify patterns, trends, and relationships that can be used to make predictions about future events or outcomes. By leveraging historical data and relevant variables, predictive analytics enables organizations to forecast customer behavior, anticipate market trends, optimize operations, mitigate risks, and enhance decision-making across various business functions such as marketing, sales, finance, and supply chain management.

As we learned earlier, the primary goal of data mining is to extract valuable knowledge from raw data, uncovering hidden patterns and relationships that can inform decision-making and drive business strategies. Data mining plays a crucial role in transforming raw data into actionable insights in analytic systems. It enables organizations to sift through vast amounts of data to identify meaningful patterns, anomalies, and correlations that might not be apparent through traditional analysis methods. By leveraging data mining techniques such as clustering, classification, regression, association rule mining, and anomaly detection, analytic systems can uncover valuable insights, improve decision-making processes, optimize operations, and gain a competitive edge in the market. Data mining also facilitates predictive modeling, allowing organizations to forecast future trends and outcomes based on historical data, ultimately enhancing strategic planning and resource allocation. Overall, data mining is a powerful tool within analytic systems, enabling organizations to unlock the full potential of their data for informed decision-making and actionable insights. Watch the video and note how data mining was used to support life sciences and drug development analytic models. How could you extend these ideas to a problem that you are familiar with? What kinds of analytic models do you use? How could data mining help you?

Descriptive statistics are a set of techniques used to summarize and describe the key features of a dataset. They provide simple, clear summaries of the characteristics of the data, such as its central tendency, variability, distribution, and shape. Descriptive statistics commonly include mean, median, mode, standard deviation, range, and percentiles. In business intelligence, descriptive statistics serve as a tool for understanding and interpreting data. They provide a concise snapshot of the data, allowing stakeholders to quickly grasp essential aspects of the information. Benefits include data summarization, performance measurement, and benchmarking. What types of descriptive statistics do you use? How do they add insight to your decision-making?

Read this selection for additional detail on descriptive statistics. Complete the exercises and check your answers.

Watch the video for a brief introduction to inferential statistics. Note the difference between descriptive and inferential statistics.

Have you used analysis of competing hypotheses (ACH)? Can you think of a recent decision at work or home in which using a SACH matrix might have helped you decide? Use an ACH matrix to choose between two open positions you have saved on your favorite job announcement app. Does ACH help you keep your bias from the process and make it more objective? Or did it only show you that there is one more attractive position for reasons you had not articulated before you undertook your analysis? Now you know what they are; even if your ACH shoots it down, you may still want it. Even if ACH does not eliminate bias, when applied to personal decisions, it can at least reveal what they are. Acknowledgment is half the battle against irrational decision-making.

Now, we will introduce the R programming language. You can also follow the links and download a copy of R to follow along and complete the examples. R is a powerful and versatile programming language primarily used for statistical computing, data analysis, and graphical visualization. It is widely used in the creation of models in BI applications. R includes a comprehensive set of tools and libraries for handling, manipulating, and analyzing data sets of various sizes and complexities. It also has extensive packages covering areas such as machine learning, time series analysis, and data visualization. These features, combined with a relatively easy-to-use interface that allows non-programmers to rapidly get up to speed, make R a popular choice for developing models in BI systems.

Like R, Python is a programming language that enjoys substantial usage in BI applications. It has a fairly clean syntax and is a great language to learn for professionals. You can start slow and work your way up to some very sophisticated full-stack applications. Anyone on a BI team should have a basic understanding of Python and what it can do.

Watch this video comparing Python and R. People have strong opinions about their favorite tool and which is most appropriate for their situation. Without getting into direct comparisons, you should be aware of these two languages and how they compare. Do you have a preference for R or Python? Why?

A final topic to be aware of in BI systems is the role of mobility. Much effort is being expended to make BI applications more mobile, primarily by porting them to tablets. Watch the video, which gives a brief introduction to a BI application designed for mobile devices. What role do you think mobility would have in applications that you are familiar with? What are the benefits of mobility? Are there any downsides?

In this module, we want to consider some of BI systems' legal and ethical dimensions. In any business setting, we are governed by business law. Read this selection, which covers intellectual property (IP). Complete the exercises in the chapter as you go through it. IP is crucial in business intelligence systems because it protects the unique insights, methodologies, algorithms, and innovations developed through data analysis and processing. Business intelligence systems often involve the creation of proprietary algorithms, data models, visualizations, and reports that provide organizations with a competitive edge and strategic advantages. By securing IP rights through patents, copyrights, or trade secrets, companies can safeguard their investment in developing innovative BI solutions and prevent unauthorized use or replication by competitors. Furthermore, protecting IP encourages investment in research and development efforts.

Modern information systems can raise various legal and ethical issues in addition to those associated with intellectual property. After reading the selection, what are some of the most significant ethical challenges that professionals using and developing BI systems should consider? How do these ethical issues translate into specific laws and regulations? Is the legal structure keeping up with the development of new technology?

Ethics refers to the principles, values, and standards that guide individuals and organizations in distinguishing right from wrong and determining appropriate conduct in various contexts. Ethical standards can vary from person to person and from society to society. Ethical standards generally form the basis for legal standards in many countries. There are many ethical issues in the use of information technology and business intelligence. Many of these have not yet been addressed by legal systems. Thus, understanding the basic principles of ethical thinking is necessary to help IT professionals guide their decision-making. Have you ever been in a situation where you thought that some proposed plan might be unethical? Why did you believe that? What ethical principles were you following?

Privacy engineering involves integrating privacy considerations and protections into the design, development, and implementation of systems, products, and processes from the outset. It encompasses a multidisciplinary approach combining principles from privacy, security, usability, and technology to mitigate risks and protect individuals' privacy rights throughout the entire data processing lifecycle. This includes practices such as data minimization, anonymization, encryption, access controls, and privacy-by-design principles, ensuring that privacy is embedded into the architecture and functionality of systems and products.

Bias in data science refers to systematic inaccuracies or distortions in data collection, analysis, or interpretation that result in unfair, discriminatory, or skewed outcomes. This can manifest in various ways, such as selection bias, where the data sample is not representative of the population; algorithmic bias, where machine learning algorithms perpetuate existing societal biases present in the training data; or confirmation bias, where researchers selectively focus on evidence that confirms their preconceived beliefs. Addressing bias in data science involves recognizing and mitigating these biases to ensure that data-driven insights and decisions are fair, reliable, and unbiased. This requires careful consideration of data sources, assumptions validation, algorithmic fairness evaluation, and ongoing monitoring and adjustment of processes to minimize bias and promote equity in data analysis and decision-making. What are some examples of this that you can identify from your experience?

Organizations need to formalize their commitment to privacy through privacy policies. Privacy policies are legal documents or statements that outline how an organization collects, uses, shares, and protects the personal information of individuals. These policies typically detail what types of data are collected, the purposes for which the data is collected, and individuals' rights regarding their data. Additionally, privacy policies often include information about data retention practices, security measures implemented to safeguard data, and procedures for accessing or updating personal information. Privacy policies are essential for transparency and compliance with privacy regulations, such as GDPR or CCPA, and help establish trust between organizations and individuals by clarifying how personal data is handled. What types of privacy policies do you have in your organization? How are they developed? How effective do you feel they are?

GDPR stands for the General Data Protection Regulation. It is a comprehensive data protection law enacted by the European Union that strengthens and harmonizes data protection regulations across EU member states. The GDPR governs the processing and handling of personal data, providing individuals greater control over their personal information and imposing strict obligations on organizations that collect, store, or process such data. What are some examples that you can think of where the GDPR might affect business operations?

Anonymization is a process through which personal data is made non-personal. When we collect data, we often collect it from sources that include a lot of personally identifiable information that allows us to identify a particular individual. Since there are many laws and regulations relating to the use of personal information, we want to remove the personal information or at least modify it so that it no longer leads back to a particular person. Find some examples of data you might be interested in that would contain data that allows specific people to be identified. How would you process that data to anonymize it?

It is one thing to understand the principles of ethics and the law, and it is another thing to design a business that will internalize them and abide by them. Watch this video, which describes concrete steps to take when building an ethical culture through employee selection and training. Have you seen any of these practices in a business? How could you implement some of these in your current work situation? Identify the steps you would take.

We tend to think that privacy regulations will reduce the utility of the data we have collected. However, enforcing privacy can be a competitive advantage. Why could this be the case? Identify an example from your experience where privacy and the enforcement of regulations could yield a competitive advantage. What are the roles of corporate culture and governance structures?

Ultimately, formal organizational policy will guide organizational managers in how they approach and make decisions. It is, therefore, important that policies be developed and implemented in this area. Are you aware of policies in an organization that you are familiar with? What types of ethical issues in decision-making do the policies focus on? Are there ways you could identify to improve the policies?