Topic outline
-
This final unit brings the process of business intelligence production together. Once analysts make their estimates or assessments, the management team has to determine how to turn the information into action. The organization's health and future depend upon its intelligence. There are various ways to manage the process, from the daily operations of the analytic team to information integration throughout the organization for optimal decision-making at all levels. It is one thing to be a member of an analyst team, but when people notice that you are so well-trained and good at your work, they will find opportunities to challenge your skills, and you will find yourself increasingly placed in project management roles. To be prepared, think of project management as the glue that holds the process together. If a project is poorly managed, it will likely go off the rails. No wonder how well the individual analysts or other team members perform, as there will be no cohesion and no certainty that all requirements are fully met. As you go through this unit, record your thoughts and lessons learned. Then, try to identify 3–5 rules for project management that you can take to your next project, either to implement yourself as the manager or to evaluate your manager against. They can even help you coach and guide your peers.
Completing this unit should take you approximately 10 hours.
-
In addition to ensuring your team is aware of the deliverables planned to meet the requirements, you must always conduct quality control checks. That same project team working with vehicle-borne telecom tech had another tradition. Every Friday, the PM would again bring coffee and breakfast, and each team member would present what they had accomplished that week toward meeting their assigned part(s) of the requirement. This normally took 3–4 hours. The team asked probing questions, made suggestions, and provided additional feedback. Then the PM did the same. If the team member was deemed to have made significant progress, they were free to take the afternoon off. If not, they would stay to work with the PM on what needed to be improved or reconsidered, and the PM would provide coaching and mentoring. In the after-project assessment, no team members reported resenting the PM for not letting them leave early. They appreciated that the PM never took Friday afternoon off as someone always needed coaching, and they knew their skills and the project would only improve on those Friday afternoons.
-
This article revisits the managing project requirements initially discussed in Unit 2. The project team and their clients or other stakeholders must agree on what is expected when the project is complete, or someone will be left dissatisfied. An important aspect of this shared understanding is whether the requirements are expressed as outputs, outcomes, or benefits. This can depend upon the type of project and the perceived needs of the decision-maker that has requested the project. Misunderstanding the basic nature of what the client or stakeholder wants can derail even the most elegantly managed project.
-
Have you ever received coaching or mentoring from a manager? Was it didactic and condescending, or was it valuable? Did it enhance your knowledge or skills? Did it result in crashing or fast-tracking the project? Did the decision-maker consider the project outcomes valuable?
-
A key takeaway from this article is that "the purpose of operations is to keep the organization functioning while the purpose of a project is to meet its goals and conclude. Therefore, operations are ongoing while projects are unique and temporary". Why is this such an important distinction? Projects have a beginning and an end. Operations always hum along in the background while everyone else works on projects. Which category does BI fit into? What does the BI analyst do? List up to 10 activities. Which of these consists of persistent monitoring? Maybe there is an aspect the business decision-makers want to observe daily, weekly, or monthly. This could be production levels, hiring rates, training costs, or anything else. These would be considered operational activities. These are normally almost fully automated via dashboards with little input from the analyst once the program is set to run. There may be some analytic process you add before you submit the regular report, but you are not creating something unique and new. If this is all a firm uses its BI capacity for, it wastes a valuable resource that should be constantly put to work on long- and short-term projects to answer strategic-level questions.
-
In addition to ensuring you have met all the requirements in your TOR, it is important to ensure you anticipate and have answered all the questions your decision-maker may ask as a result of your findings. Otherwise, you will be sent back to the drawing board to find new answers. Anticipating questions and never leaving your decision-maker wondering about anything goes a long way to enhance your credibility. In addition, you must be sure that you have answered not only the questions asked but look at them holistically to try to discern what challenges keep your decision-maker up at night and approach those through the questions asked. This will resonate with your decision-maker and increase your ability to access them.
-
This is a detailed how-to guide to ensuring your requirements are fully captured. It is a useful guidebook with processes you will tweak and adapt to each project plan. Is anything surprising or new to you? You will not use all of these approaches for every project, but having them at hand is useful when planning your next project.
-
-
Even if your team struggled mightily to ensure it had the right questions in the TOR, you might discover a hole in your analysis made by a big unasked question once you are halfway through the project. Be sure you leave your decision-maker with all they need to make clear, effective decisions.
-
In a famous 1964 U.S. Supreme Court case on obscenity and the limits of free speech, Justice Potter Stewart tried to explain "hard-core" pornography, or what is obscene, by saying, "I shall not today attempt further to define the kinds of material I understand to be embraced... [b]ut I know it when I see it"... This failure to effectively categorize and describe in a standardized way has been used since as a common witticism on lazy thinking. We are all often lazy in our thinking when we take for granted what we know.
Think briefly about the phrase "vanilla is better than chocolate". Students have an immediate, passionate response to this statement, but when I ask them to define vanilla, then chocolate, we hear crickets in the classroom. Once they do a little research and learn that vanilla is a key component in the delicious, sweet-tasting chocolate we all love and that chocolate by itself is bitter, they all have to agree, many despite the earlier insistence of personal preference, that without vanilla, there is no chocolate as they know it, thus vanilla truly IS better than chocolate, as it is also delicious on its own.
Consider the phrase "cats are better than dogs". Do you approve or disapprove of this statement? Describe both animals' physical, behavioral, and other characteristics. Is this an appropriate statement like "vanilla is better than chocolate?" Or, do we need to add something? Should we say "cats are better than dogs at X"? or "cats are better than dogs for Y"?Getting the question right is the key to effective analysis.
-
-
-
Longer-term projects tend to require progress reports, particularly because they often accompany invoices. The invoice must show that the requisite progress has been made to justify the invoicee's payment at that stage of the project. Shorter-term projects will only have a final deliverable. Like progress reports, they intend to persuade the decision-maker that your analytic findings are valid and reliable and that they should feel confident basing decisions on them, unless the analyst expresses low confidence in their findings.
-
This is a very basic guide to writing progress reports with some valuable hints.
-
Read the text of the memo. This is from the August 6, 2001, Presidential Daily Briefing, a document produced daily by senior intelligence analysts and presented to the top official of the U.S. government. When it was released in 2004, it was an instant indictment of how President George W. Bush and his national security team had failed to protect the United States from the 9/11 terrorist attacks in New York and Washington, DC. Do you agree? What does the memo tell the president specifically? What actions should he have taken based solely on the information in that memo?
The problems with the memo are legendary. The findings are completely inconclusive and are not actionable at all. They are awash in conjecture and talk about Osama bin Laden's intentions and desires. They do not tell about his capability to act upon those intentions and when, where, or how he will do so. The document also does not provide the top U.S. decision-maker any information about the analysts' confidence level in their analysis. A useful intelligence product will provide all of this and be actionable.
-
No one expects analysts to be the next great novelist, but that does not mean you should not adhere to the gold standards of your craft. Beyond BLUF, concision, accuracy, and organization, you should address several style and formatting considerations in your findings reporting. Avoiding passive voice, for instance, keeps your decision-maker from having to ask "who" did what. Adding some context as relevant, but ensuring your content is concise, is another balance that takes time to learn. Look at these guides for helpful tips on making sure your decision-maker reads what you have provided.
-
This is a nice guide for improving or "tightening up" your writing, ensuring clarity, concision, and directness, as described in the article. This is useful for any information exchange writing. Over the next week, note how often you use passive voice, are too wordy, or use cliché expressions or qualifiers.
-
This was the world's first style manual for intelligence analysts in the civilian world in 2008 and is still one of the best. Without standardized language within your organization, communication among analysts and decision-makers will remain ambiguous. It is important to agree on a confidence scale, so your decision-makers know how confident you are of your findings, the calculation of which will include consideration of source.
-
Kristan Wheaton was a Professor of Intelligence Studies at Mercyhurst University for 16 years until he became the Professor of Strategic Futures at the U.S. Army War College in 2019. His blog posts explore the detailed aspects of producing intelligence estimates and thoughts about the field. His context is typically the national security arena more often than business, but his posts are valuable as they focus on the process, not the product types. This post uses the example of an Iran National Security Estimate (NIE), the most respected intelligence document in the U.S. national security context. He emphasizes the need for a shared lexicon for expressing analytic estimates, particularly careful, standardized use of words of estimative probability (WEPs).
-
What does confidence mean? Are you confident in your abilities as an analyst or project manager? What makes you confident? Your overall confidence in your ability is one kind of confidence. Analytic confidence is something else entirely. It calculates how much faith you have in the estimate(s) you provide to your decision-makers. Confidence can be affected by many factors, which should be included in your evaluation of how seriously your decision-maker should take your findings.
These factors include considering whether you had sufficient time to make your estimate and were working under optimal conditions for evaluating sources and making determinations based on them. It also includes your assessment of source availability and reliability. In addition to developing accurate, concise estimates, agreeing on a confidence scale within your organization is important. Hence, your decision-makers know how confident you are of your findings. This article goes in-depth about the importance of conveying analytic confidence to help your decision-makers take action with confidence themselves.
-
-
If your boss does not like to read long reports, do not give them long reports if you want them to care about your intelligence estimates. Some decision-makers prefer a formal briefing with an opportunity to ask questions about the findings, the sources, the process, etc. Some only care about the findings in response to their requirements. Some like to drop by and get informal, oral project updates. If you insist on communicating with your decision-maker in a way that does not suit their personal preferences, your analysis will have no value to them. This is one of the hardest things to teach, as every decision-maker is unique.
-
This post is about getting to know your decision-maker, not formalizing requirements. You will have to use your intuitive skill to observe how your decision-maker takes in and processes information to communicate in a way that makes them listen. Think of a time when you have had difficulty communicating with a boss. Did you misunderstand this communication gap with something else, such as organizational culture or a bad boss? How did you resolve the gap? Did you quit in a huff or get fired or sidelined? Or did you figure out the communication problem and realize that you were the one who needed to change your style?
-
Using evidence-based business decision-making is derived from the scientific method and the medical field. A highly prolific writer on intelligence, Prof. Steve Marrin, lately of James Madison University in Virginia, has written extensively over the past decade about using medicine as a model for developing the discipline of intelligence. This article offers a simple description of the process and its essential pros and cons.
-
-
-
Richard Hackman pioneered research on how to build optimal analyst teams in the late 1990s and early 2000s. His work remains at the forefront of our understanding of the need for diverse teams to ensure all possible outcomes and pathways are considered and all hypotheses are explored and demolished. His work on the dynamic quality of these teams is today's cannon. Top-performing managers must be ready to manage drama, frustration, and passion but know that these are the ingredients that make their teams passionate and high-performing.
-
Optimal analytic teams are diverse, with varying perspectives and skill sets, with a healthy respect for each member's area and level of competence. Depending upon their competence, they will quickly learn what tasks should naturally fall to which member. Eventually, the team will develop its personality, which is a positive in a highly functional team. The key to developing the right team is to start with the right members.
-
This article discusses the traditional process by which teams build cohesion and working relationships.
-
Optimal analytic teams are diverse, with varying perspectives and skill sets, with a healthy respect for each member's area and level of competence. Depending upon their competence, they will quickly learn what tasks should naturally fall to which member. Even with the best members, teams undergo a growing process, which, if well managed, they form by getting to know each other, the norm, creating healthy work patterns. They perform well, getting positive feedback from the manager and the decision-maker, which makes them stronger for the next project. The storming occurs at any point in the project, sometimes early as they scope out their turf and become aware of others' strengths and weaknesses and sometimes later when project pressures increase. In any case, research shows that when bonds appear to be fraying, the skillful manager will get out of the storm so the team can work out their issues, then re-engages to ensure the team can effectively reconcile and proceed apace. Two analyst teams were working on government research projects with three-month timeframes. To enhance the performing process, the team leaders created innovative competitions between the teams to help form identities for the individual groups. One way was a dance competition between the two teams; another was a scavenger hunt. When the storms came at the end of the first month of the project, they started playing kickball twice a week. They would come to work the following days sore, bruised, and happy to share stories of their legendary triumphs the evening before. These were two of the university's best-performing teams in ten years of contracting with this government agency.
Have you been part of a team that went through all four stages? How rough was the storm? Was the team able to reform afterward? Some observers believe a team that does not go through a storming phase performs poorly. Perhaps the storm is an indication of their level of dedication. What has been your experience?
-
This article discusses three types of leadership styles. What kind of leader do you prefer – democratic, authoritarian, or laissez-faire? What kind of leader will you be when you have an analytic team to manage?
The article also delves into the psychology of teams. Conformity is the enemy of imagination. The need for conformity of thought can devolve into groupthink or a situation when the lowest common denominator of thought prevails. This environment stifles creativity and diversity and causes teams to concentrate on getting a job done, not doing it in the most innovative ways.
-
-
Some managers are good at attracting top talent but fear their ability to manage them effectively. This insecurity can lead to micromanagement and over-organization, which reduces morale in highly functional individuals. Suppose the manager trusts them to have the skills needed to perform in the roles for which they have been selected. In that case, the manager must trust the team to form itself into a structure in which all team members will use their best skills to support the common effort, and they will sort themselves into roles naturally. They will also create work teams that know how to organize and plan for every task, big and small. If the team members are highly capable, the best thing for a manager to do is to offer support and protection and get out of the way, just checking in to make sure the team has everything it needs to succeed.
-
This article effectively describes the tug-of-war in which the team manager must constantly struggle between external and internal forces. Consider the discussion questions about the politics of managing external stakeholders. This is where the manager must serve as a force field, holding back the external forces that desire to place the pressures of their influences and agendas on the team, which may include tight timelines or budgets or competing demands of the manager's immediate boss and that boss' boss. Once you add in the client's internal or external demands, the struggle becomes even more multifaceted, and the manager must become an expert juggler.
-
The article provides useful ideas on how to build successful teams, particularly by paying close attention to team size and composition, providing accountability, and using periodic check-ins to gauge how the team is progressing. It also talks more about team building and small group formation. It is important to lay the groundwork for successful teams at the very outset.
-
-
-
Recruitment of diverse skills is paramount, as discussed earlier, but ensuring these members with varied communication styles and project approaches can be motivated to work as a team is critical. A highly successful manager recounts their formative experience in leading a team as a newly formed analytic team member in a university research setting using graduate students as analysts.
The manager was a stern but popular professor who, on day one, explained to the team that he had met the chief of analyst training for a government research agency at a recent conference. He had extolled the virtues of the program's analyst training program and persuaded the chief to test out the quality of his students in an experiment. The chief had questions he wanted to be answered on future challenges and opportunities related to global telecommunications technologies. He had tasked his government analysts with the requirements, then engaged a second team from a highly respected private research firm to do the same project. The chief agreed to provide funding for a group of student analysts to take on the same challenge for three months over summer vacation.
Unfortunately, the funding came at the last minute, so the professor could only recruit who was available instead of the "dream team" he had imagined, as most students already had summer internships and jobs. He admitted this to the team on day one but told them about their competition. He urged them to "blow the doors" off their competitors and give this project their all. He appointed a manager that was a superb analyst but did not have the skills to lead this team effectively. This manager constantly sent tedious surveys and had one-on-one check-ins, making most of the team aggressively turn against her. Another somewhat more experienced analyst, without management experience, arose from the ranks as a leader from within that helped convey the professor's desires without such intrusions and caused the actual manager to back off and see where the analysts drove the bus with some direction but little interference.
There came the point about halfway through the project when the professor came to the team after reading their latest draft findings with his large head in his hands. He sat down and apologized to them for tasking them with something that was clearly beyond their abilities. He admitted the blame was squarely on his shoulders and hoped they could work through the rest of the project to the best of their abilities. Then he took each of them into his office and told them the unvarnished truth about what he had expected from that analyst and where they were lacking. His approach as a coach and a sad dad inspired them to do such amazing work that the client had to admit their work had far superseded that of the other two projects at the end of the project. The experience led the government agency to contract another dozen projects with the university department over the next seven years until Federal funding crises outside the agency's control ended the program to the chagrin of all the agency leaders. Sometimes humility from a manager can be the most inspiring ingredient to great leadership.
When the older analyst later became the manager of most of the next projects, she had some growing pains learning how to transition from team member to manager and fellow student to faculty advisor. Eventually, she inspired the same level of life-long loyalty and common effort that kept these teams and projects going. The opportunity to be on one of these teams became highly sought after as the optimal summer work experience.
She also instituted a mid-point self-evaluation survey and sat down with each team member to discuss their self- and manager-identified strengths and weaknesses. She effectively coached each one to high achievement. She also placed herself as a "human shield" between the organization's and the team's demands to ensure they would not be distracted by budgetary and other concerns that pressured the project from the outside.
She used project funds with client approval to stock the team office space with all the food and drinks the team voted for, creating the Googleplex model without knowing it. Each team member was appointed a "chore" like in a family structure, such as refreshment inventory manager, morning or afternoon coffee manager, IT specialist, quality control for writing, graphics, briefings, etc. This allowed the work to occur with few distractions and an immediate understanding that each team member was already contributing even before individual project-related products were evaluated.
Team members began to spend their weekends in the project space working longer hours than expected because they enjoyed the companionship (and the air conditioning!). They moved lobby furniture into the presentation room to watch sports together and made the workspace their summer home. This first team is still very close 16 years later, even though all members are separated by geography and profession today.
Have you ever been a member or manager of a team with this sort of cohesion? Have you had the opportunity to replicate this experience as a manager yourself? What impediments to cohesion and effectiveness have you seen? Have you made an effort to overcome them? How has that worked?
-
-
The study in this article provides reasons why BI is not fully implemented in some organizations. The thematic approach to determining why the technological tools are or are not used by managers provides an actionable set of areas where improvements can be made. One aspect that may relate to our discussions on asking the right questions is, "do managers really need to know how to use BI tools?" In many organizations, managers have enough knowledge about specialized data and tools to know how to task their specialists (BI analysts, in this case) to use them to answer their requirements. This can give analysts too much power in organizations with poor systems, whose data quality is questionable, or with authoritarian cultures where analysts fear giving the "wrong" answer. But in a highly effective environment, managers manage, and analysts analyze.
-
-
The average person is very poor at calibrating their ability, particularly regarding decision-making. According to research, men tend to be more likely to overestimate their ability, and women tend to be more likely to underestimate theirs.
-
This brief talk by a champion poker player discusses the importance of analytic reasoning and information in making the best decisions and why we need to leave our intuition at home to rest up for small determinations with little consequence when we have big decisions that require data and analysis. What are the three lessons? When have you followed these lessons? Did they work well or not?
-
This page provides a quick view of how humans perceive the world around them. It points out how "no two people have the same worldview" because how we sense the environment depends upon our unique experiences. For instance, people who have spent time in combat may be averse to loud noises. Thus, this experience skews their initial information-gathering process in a loud, noisy environment. They may be so attuned to the discomfort the noise causes they may have difficulty noticing anything else in a new environment. This will cause them to later categorize that environment as little else than "noisy".
Maybe it was a candle factory, and the colleague they were there with came away with a mainly olfactory perception that it was very smelly, even overwhelmingly so, because they have a perfume allergy and do not like strong smells. They did not even notice the noise. Another colleague, a visual artist, was positively engaged with all the colors used to create the candles. For them, the sound and the smell of the factory barely registered.
This is the problem with going with your gut, especially if you are a boss. Your pre-existing perceptions will color your view of using data to improve decision-making. If your boss comes from an environment where "whatever the boss says goes", they may have little faith in the process or product your data analysis derives for them. They may think they know the answer to their or the client's question, and your findings would only confirm or question their anticipated outcome. A boss that insists on their gut as the final arbiter will only value your analytic findings when they confirm their view.
This wastes time and resources for the organization if it supports a robust BI capability. Suppose your boss has an open mind or experience with data-driven processes effectively supporting decisions. In that case, your life will be easier, and your organization will benefit from your work, rather than only being supported by your boss' predetermined decisions that do not take the data into account.
Sometimes the data takes us to places we do not expect. We must follow it to the end, and if the outcome is surprising, replicate the process to be sure the findings are accurate. If they are, the key will be in communicating them to an equally surprised decision-maker so they can be accepted as valid. When might you have gone with your gut and realized late that your perception took you in the wrong direction? Did you have an opportunity to correct the decision or interaction that resulted from starting with a misplaced perception? Did your realization come from absorbing and processing additional information or data?
-
-
Does your organization have a mentoring program? If not, where and how can you find one? Does this person have the life experience to teach you what you need to learn? Are they approachable and ready to listen when you need them?
-
This article presents a different perspective on learning organizations, more focused on the individual and how the organization best serves them. How can an organization avoid "losing out on its learning abilities when members of the organization leave"? What are the six factors related to time? These relate not only to constraints on learning but also to operational and project activities of organizations writ large. Keep these in mind whenever you plan a new project or for your current projects or operational support roles, and make sure they are considerations for defining your scope. Managing up is something analysts do all the time. This happens when you work with your managers to refine requirements, develop your TOR, and define your scope. It is also a key skill for ensuring your analytic findings find a receptive audience, despite expected results. Have any of these tips helped you to effectively "manage up" in the past? How could you apply some of them in the future to communicate more effectively with your decision-makers?
-
Read this article about personal and organizational learning. Do you believe McCall's statement that "leaders are made, not born, through the trial and error learning that occurs through actual work: adversity, challenge, frustration, and struggle lead to change"? Does your organization provide those kinds of learning opportunities? Or does it punish mistakes? Does it embrace other kinds of learning? Do you agree that "too many organizations focus on learning the wrong things"? Have you had this experience? How do you think an organization can be sure that its learning offerings or plans align with its strategic priorities? Do you have a mentor? If not, where and how can you find one? Does this person have the life experience to teach you what you need to learn? Are they approachable and ready to listen when you need them?
-
-
Suppose your organization has an IT problem without an effective organizational or departmental risk anticipation and mitigation plan. In that case, your tech tools could become unavailable at a critical time in your project execution. You may also discover that your platform is missing key features or add-ons that make your project more difficult or impossible to complete as planned. It is important to run through the collection and analysis in this case and report processes you intend to use to ensure you have the tools to conduct them most efficiently. It is also important to ensure you have tools your personnel can use. Not all data collection, warehousing, exploitation, and reporting tools have the same functionality and ease of use. If your team is not fully acquainted with your system, you may have significant delays and a lack of output. Before committing to the project, these must be worked out with your organization or department.
Problems with data can also relate to a lack of IT tools, but they are more likely to relate to the availability and quality of your data. This can be related to needing a dataset your organization does not have a subscription to or even obtaining data only to discover it is incomplete or inaccurate. Once you have the data you need, it may need more cleaning and normalization than anticipated. It may not relate as directly as you had anticipated to your project's specific requirement(s).
Identifying potential IT and data challenges before you finalize your TOR is important. One purpose is to provide the client (internal or external) with what issues may arise and how to plan for them in advance. This could include innumerable checks of IT systems or a good relationship with IT to be sure they can help you obtain what you need. This may also require adjustment to the budget, the project timeline, and the scope and client deliverable expectations. No matter the potential risks to BI projects, such caveats should be explored and included in the TOR.
-
This article presents a nice overview of risk planning and mitigation implementation common for most organizations and projects. In the case of BI projects, the risk planning process can be similar, but the pitfalls are unique to BI. They are most likely to be related to technology and data.
-
This article revisits managing project requirements. The project team and their clients or other stakeholders must agree on what is expected when the project is complete, or someone will be left dissatisfied. An important aspect of this shared understanding is whether the requirements are expressed as outputs, outcomes, or benefits. This can depend upon the type of project and the perceived needs of the decision-maker that has requested the project. Misunderstanding the basic nature of what the client or stakeholder wants can derail even the most elegantly managed project.
-
-
-
This review video is an excellent way to review what you've learned so far and is presented by one of the professors who created the course.
-
Watch this as you work through the unit and prepare to take the final exam.
-
We also recommend that you review this Study Guide before taking the Unit X Assessment.
-
-
-
Take this assessment to see how well you understood this unit.
- This assessment does not count towards your grade. It is just for practice!
- You will see the correct answers when you submit your answers. Use this to help you study for the final exam!
- You can take this assessment as many times as you want, whenever you want.
-