As the need for data insights becomes more and more important, how do you ensure you’re making the most of your company’s data? In this webinar, Carl Rosseel, Curriculum Manager at DataCamp, will discuss the opportunities and pitfalls of becoming a data-driven organization. Specifically, he will discuss the role business intelligence (and business intelligence tools) play when scaling the production of data insights, and how it fits into the analytics function throughout the organization. Moreover, he will outline the specific opportunities and challenges when scaling the use of business intelligence throughout the organization, and provide solutions for these challenges.

Slides

Key Takeaways:

• Identify the role of business intelligence when scaling data skills throughout the organization

• Learn how to build data-driven teams, and how empowered data and business analysts turn data into actionable insights

• Understand the obstacles and opportunities when scaling business intelligence throughout the organization

Webinar Transcripts

Motivation 

In today’s webinar, I want to discuss how business intelligence tools empower the analysts of tomorrow to democratize data insights throughout different teams and the organization. Breaking down the agenda, we're going to see what data-driven means in a data-driven organization, and what it doesn't. Secondly, we're going to talk about the world of business intelligence and democratizing reporting and data insights. We’ll also discuss the potential obstacles that organizations can come across while going in that direction. Finally, we'll also touch upon the role learning plays in this whole process.

What data-driven is, and what it isn’t

Data volume increases

Let's start first to zoom out and look at how the growing volume of data is pushing organizations to become data-driven. Data ends up being everywhere these days; data volume worldwide is growing exponentially. By 2025, we expect to have almost 170 billion zettabytes of data. Just to put this in perspective, one zettabyte is equal to all the grains of salt on all the world's beaches. It's 10 to the power 21 bytes, or 10 with 21 zeros behind, so this is really a massive amount of data.

Organizations are making active investments in data

Secondly, what we see is that organizations are starting to make major investments in becoming data-driven. What we see, first of all, is that 99% of organizations have made active investments in data science and artificial intelligence initiatives. Secondly, what you see is that almost 68% of organizations now have a Chief Data Officer employed up from 12% in 2012. 

The data science talent crunch 

Thirdly, we also see the growth for data scientists is that it has been growing exponentially by as much as 650% from 2012 to 2018 on LinkedIn. We see the same trend in the future — the job growth for data scientists is expected to be around 3.5 to 4 times higher than the average job profile in the next decades. And finally, we're also seeing more job postings than job searches.

Modernization of data infrastructure

Then also, what we see together with this huge growth in data is that companies are also spending on this. 25% of all software infrastructure in 2019 was spent on data infrastructure, and it’s expected to rise more in the coming years. We also see that the worldwide spending forecast on Public Cloud to modernize data infrastructure is over $300 billion. We expect this to increase by around 20% as well in the coming years.

All of this really led to something else, which is the rise of centers of excellence. This could be an internal analytics team or an external expert hired who really knows the ins and outs of artificial intelligence and machine learning. They can focus on automation, and really, increase business efficiencies in multiple parts. 

Actually, I'm here to tell you today that modernizing data infrastructure and building these centers of excellence only scratches the surface of becoming a data-driven organization. 

What a data-driven organization is

To really explain this, let's have a look at what a data-driven organization actually is. A data-driven organization is an organization that has forged a data culture where everyone understands the value of data, that everyone has the skills, mindset, and ability to work and reason with data to do their best work. So let’s break it down into its parts

• Subject matter experts supplement their experience with a data-driven mindset. This can be a strategist, a manager, a marketing specialist, or basically any type of role that increases and enhances their performance in the role with data. 

• Secondly, it's about functional departments that are strong and loyal customers of the data organization. What we refer to as functional partners are finance, HR, operations, and so forth. Are they using data to optimize their day-to-day business process and make sure they make the right decisions? 

• Finally, the diversity of roles and tools that achieve data-driven outcomes on a daily basis. We'll actually touch a bit deeper on this part later when we go to the different data personas.

What does a data-driven organization look like? It basically means that we are using data in all parts of the organization. So as an example here’s a glimpse of the type of departments you can find in a data-driven organization:

• Supply chain could benefit from stock and warehouse optimization, and forecasting inbound and outbound flows, and optimizing the picking order time. 

• Marketing uses data to forecast customer churn, lifetime value, or do A/B testing to calculate the incremental ROI of a campaign, optimizing all your marketing expenses and making sure that your customers stay with you. 

• Finance uses data to automate workflows, and produce better forecasts for key financial metrics 

• Product uses data to A/B test product features. They use data to optimize every part of the conversion funnel, and they visualize all the metrics the business cares about using product analytics.

• Human resources leverage people analytics for performance management and attrition reduction. 

Conversely, what is not a data-driven organization? An organization that is not data-driven can be seen as such: 

• An organization where a part of the organization focuses on difficult AI and machine learning problems. The phrase “a part of an organization” here is really important. Just because one or two teams in an organization use AI and machine learning, that doesn't mean that your whole organization is data-driven. It is great that a part of the organization does this, and it could be that they probably provide tremendous value. But that doesn't mean that your whole organization is data-driven. 

• An organization that looks at data as a “thing to do” but not a “way to think”. We shouldn't be looking at data as something that we need to do but as a methodology for solving problems. So, this is really a shift in mindset. 

• Finally, an organization that's focused on the most cutting-edge tools and not on skills and culture. Data-driven skills should be embedded throughout your organization and not just in your analytics teams.

The data science hierarchy of needs

To further strengthen this, let's have a look at the data science hierarchy of needs by Monica Rogati. She basically developed this pyramid and framework to say what an organization needs to become data-driven and operationalize AI. In this case, we adapted the pyramid to make it a little bit simpler: 

• At the bottom, we find data infrastructure. This is the key to everything. You can have on-premise servers or you can use a public cloud such as Google, AWS, and so forth. Secondly, you have data quality. 

• Data quality is incredibly important. You need to make sure you can trust the data you have. It is also about how data quality permeates throughout your organization. Can you trust data from different departments, and are you tracking the different health metrics of your data? 

• Thirdly, is Business Reporting. Once infrastructure and quality is in place, you can start creating dashboards. You can segment your data by different product features or markets. You can really visualize your data to see how your business is doing. 

• Then we move on to Analysis and Insights. So this is really when you can go that one level further, and you can start analyzing and deriving insights based on your data that impact business decisions. 

• Then, finally, on top of everything else, comes AI and machine learning. This could be something like fraud detection, supply chain optimization, bidding algorithms in marketing, and so forth. 

Looking at this pyramid, you could also say that 80% of all data-driven and all value from data comes from maybe the bottom layers, and mainly the two layers of business reporting, and analysis and insights are where BI tools play a key role, and they can really play a key role here for your whole organization to become data-driven. 

Business Intelligence helps data-driven organizations democratize data insights

Let's dive deeper into why Business Intelligence tools are so important for democratizing reporting and data insights. Let's start first by defining Business Intelligence. According to CIO Review, Business Intelligence leverages software and services to transform data into actionable insights that inform an organization’s strategic and tactical business decisions. 

That's a great definition, but let's have a deeper look, we can break down Business Intelligence Tools as such:

• Supercharged spreadsheet tools — they're modernized spreadsheet tools; they are more powerful; they can handle a lot of data and it's very useful to create visualizations. 

• You could all say that it is easy to connect them to a variety of different sources. You can connect them to warehouses — like Amazon, Redshift, Google BigQuery, and so on and so forth. And, of course, they also have all the normal and key connectors to CSV, Excel files, and so forth. 

• What's quite special about them is that they are drag and drop tools. There's no coding involved. So you could definitely say they are easy to learn. But that doesn't necessarily mean that they are easy to master. Creating your first visualizations in these tools can be very, very easy. But creating full-blown business dashboards with the correct filters and the visualizations to really display the story is something else entirely and is something that takes time to learn.

Here we see a quick snapshot of the two main business intelligence tools, Tableau, and PowerBI. On the left, you can actually see a map or a folder of different dashboards in Tableau. On the right, you see one specific dashboard that's worked out in Power BI. 

How do BI tools fit into the whole organization, and which data personas will actually be using business intelligence tools? So first, let us go to the eight personas:

• Data consumers and leaders: Data Consumers and Leaders often work in non-technical roles, but they consume data insights and analytics to make data-driven decisions.  

• Business analysts: Business Analysts are responsible for tying data insights to actionable results that increase profitability or efficiency.

• Data analysts: Similar to Business Analysts, Data Analysts are responsible for analyzing data and reporting insights from their analysis

• Data scientists: Data Scientists investigate, extract, and report meaningful insights into the organization’s data. They communicate these insights to non-technical stakeholders and have a good understanding of machine learning workflows and how to tie them back to business applications

• Machine learning scientists: Machine Learning Scientists are responsible for developing machine learning systems at scale. 

• Statisticians: Similar to Data Scientists, Statisticians work on highly rigorous analysis, which involves designing and maintaining experiments such as A/B tests and hypothesis testing.

• Programmers: Programmers are highly technical individuals that work on data teams and work on automating repetitive tasks when accessing and working with an organization’s data. 

• Data engineers. Data Engineers are responsible for getting the right data in the hands of the right people. 

Business Intelligence tools are especially used by Business and Data Analysts. Here’s a detailed breakdown of both their roles:

• A business analyst is someone who ties insights to actionable results to increase profitability and efficiency. Their most commonly used tools are spreadsheets, BI tools, and SQL to extract data from databases. When you look at the intermediate business analyst, you see that they have a deep knowledge of the business domain. This could be finance where they know everything about corporate valuation and forecasting, and so forth. And often, they also have a strong grasp of BI tools to visualize and show and report on the data they are working with. Examples of job titles that fit this persona are business analyst, marketing analyst, or financial analyst. 

• Data Analysts, they're really responsible for generating data insights and reporting them. They kind of work with the same tools, but they also use R and Python as programming languages. Because they have a deep understanding of the data analysis workflow, they know how to create data pipelines and they really work well on importing, manipulating, and cleaning all this data. If we look at the different job titles here, you can see data analyst, operations analyst, marketing analyst, and business analyst again. This is just to show that these job titles can have different meanings in different companies. In some companies, a business analyst might be a lot closer to a data analyst than others. 

Modern analysts are experts in going wide

So what else are analysts? You could say that they are experts in going wide. They have a T-shaped profile, but first and foremost, most of the job of business and data analysts is to do business analysis. 

• They use basic statistics as well as various different analytical techniques. 

• They know a thing or two about databases and programming. 

• They master business intelligence tools

But that's not the only thing they do. They also have the following strengths:

• Project management

• Domain knowledge

• Business case writing, and it's very important that they know how to tell their story and communicate their findings to the organization.

Now we know all about this type of T-shaped analyst. How do they provide value? 

1. You could say that they dig through all the data a company has, and look to generate insights. 

2. They inform data consumers and leaders, and they find new problems or opportunities. 

3. They can find very complex problems or problems you didn't know you have, or new opportunities to expand into, for what you might potentially hire new people or more technical people.

4. Finally, they are often also the bridge between technical and non-technical stakeholders, and they are not statistics or machine learning experts. I want to pause a moment here on the last one. 

All of this is to say that Business Intelligence tools enable analysts to democratize data insights. When analysts use BI tools, it can really spread data throughout an organization.

How BI tools spread data insights

Generate maps

Let's have a look at a few examples of how they can do this. Here, you can see a GIF of a map. This is actually taken from a DataCamp course, where learners can visualize the number of people that are leaving a certain bike station to travel to the city. You can see which stations are busier and where we need to resupply bikes faster. As we said before, you can create easy drag-and-drop visualizations. 

Generate treemaps

You can see an example where we're creating a treemap of different genres of games in Tableau. This was related to a course on DataCamp related to computer games. Here you can see, for example, action in the top left is the most popular genre. 

Create dashboards

Finally, you'll also have dashboards, and you can use visualizations as interactive filters. By clicking on a certain visualization, the other visualizations actually change. All these three visualizations were by Tableau so far.

Rollout security in PowerBI

The first one you see here is the rollout security. You can actually put different filters on how your dashboard looks, depending on who is viewing it. In this case, you can view the dashboard as Lily, who is a salesperson, would see a completely different dashboard because it's adapted to her own sales results. Finally, you can also filter by dates, and it could easily drill down from year to quarter to months, or even on a daily level. 

Dynamic dates

The daily level might look a little bit too busy, but it can still be handy if you want to zoom in on certain weeks. So we didn't focus here on a particular full-blown business dashboard as that was also not the case of today. 

But combining all of these techniques can really help you in creating advanced dashboards that are user-friendly for stakeholders and leaders to use. So doing that, data and business analysts can empower decision-makers with data.

Data democratization can be thwarted by a lack of trust, skills, or literacy in data

What are the potential obstacles towards data democratization? We define three main obstacles here:

• Lack of trust in data

• Lack of data skills

• Imbalance in data literacy.

Lack of data trust

So let's start with the lack of trust in data. We look back at the data pyramid from a Data Science Hierarchy of Needs. What happens when you have a problem with the bottom two layers? Data infrastructure might not be there or data quality is not in place, and you cannot trust the actual data that you are reporting. It's as simple as everything falls apart. 

You don't know what to trust anymore. All your analytics teams will spend way too much time digging into the data, not knowing what's correct anymore, and so forth. And this could be a potentially huge disaster. But we have to say as well here on the lack of trust in data that the future is bright. You see the rise of new categories focused purely on quality, and they really look at data quality just as people look at code of software quality. Secondly, we also have a lot of metadata management tools, and we look at how our database is stored and organized, and who changed what to a certain database, and to track these changes. These are also data governance platforms to help you to manage everything around it, such as security and so forth.

Lack of data skills

Before we dive into the lack of data skills, let's really investigate what data skills actually are. So you could say in the late nineties, the typical tool they used was Excel. Depending on their role and their organization, they also use other technologies, but this was rather rare and the main tool that they basically used was Excel. 

Today, this looks completely different. Some roles might use a lot of R or Python. Other roles might need to use a lot of SQL where others are very heavily using BI tools such as Power BI or Tableau, and then others again they still use Excel as it fits all their needs. Even the data consumers in an organization need to use data — so the lack of data skills is showing across the board. 

And research shows that 60% of people are more likely to feel burnout or are unproductive or unhappy at work because they are overwhelmed by data. Suddenly, they are given a project, and they need to use data to come up with decisions to answer for it, and they don't know how to handle and analyze all these huge amounts of data. And this really leads, on average, to lose them five days on average to time off and procrastination because they feel so overwhelmed. 

For the lack of data skills, the answer is in your people. First of all, hiring in the data-driven age, you need to identify the right personas to hire. We went through the eight different personas before, from data and business analysts to statisticians and data engineers. Finally, you need to create a rubric for data success when hiring. Skills shouldn't be the only thing that you look at, you should look at attitude, willingness to learn, future potential, and so forth. 

Finally, once you have these people, you need to embed analysts to promote scalable insights. This one, you can actually link back to the previous slides. What if you have analysts all embedded in your organization, to help people make the most of data, to help your HR manager, to basically make sure that you are embedded in all parts of the organization to promote these insights. 

This goes hand-in-hand with learning in a data-driven age. Firstly, you have to make sure that you create learning opportunities. Once these learning opportunities are there, it's very important that you promote and reward a psychologically safe learning culture. If, for example, a data analyst is upskilling himself or herself to become a data scientist, he or she should also be rewarded for upskilling. That’s because, if you won’t reward people for learning new things, they will probably leave, and go to another company. So it's really key these days that you make learning part of your organization's priority. 

Imbalance in organizational data literacy

Then, the third and final part that we see as a potential obstacle is data literacy or an imbalance of it. So first, let's understand data literacy. This is the ability to read, understand, and communicate data insights, and most importantly, be critical and reason with data. So, what are the symptoms of low data literacy? 

• The first one is that leaders cannot grasp various analytics concepts. This could be things such as different machine learning concepts, but also how data analysis is done, and how do you use hypothesis testing?

• Secondly, data is not regarded as assets by leaders and stakeholders. It could be that some leaders are stakeholders in general; they trust their gut feeling more than data. 

• And finally, data experts are siloed and are not actively part of the conversation and decision-making process.

Let's see it in action. Let's look at a case here where I’m a stakeholder. I own a shoe store online, and I'm currently active in Belgium, and I want to expand to France. So I ask my analyst for supporting data to do that. I’m going to get one star for this, because you are using data to support a pre-made decision, and your analysts can probably come up with some ways and find some data to support this decision. But it could also be that this analyst is feeling maybe not that appreciated in the organization because he or she might feel that “I am pulling data for decisions that are being taken anyway.”

What about the second case? Analysts presented data to leadership about expanding to France, but also about expanding to Germany. Should we expand to France or Germany? Are there pros and cons of the different markets? What would be even better is if we can create radical candor between us. What we mean by this is that we really care personally about each other, but also challenge each other directly. So why are we not talking about England? Maybe it is better? What is the hypothesis? Why do we want to expand to France or Germany? Is it because we expect higher margins there? Is it because the shoe market in France is in the lift, and we see an opportunity there? So we'll create a full-blown business case. So, data literacy here is key to realize the full value of data. We need to create this balance between your analysts and decision-makers and make them work together.

Learning is an essential prerequisite for democratizing data insights

Let's look at the role learning plays in democratizing data insights. Data will only become more important and so will learning. Data analysts and scientists are ranked as the job roles with the highest increasing demand by the World Economic Forum: Future of Jobs Report. And maybe even more striking, 94% of business leaders expect their employees to pick up new skills on the job, up from 65% in 2018. This is really a huge increase in the number of people that expect people to upskill.

Secondly, the transformational impact of data upskilling is already paying dividends. A report from McKinsey Report that 70% of organizations that invest in upskilling are reporting positive ROI. On the right, you can see from a report from Deloitte, 88% of organizations that invest in upskilling all employees on analytics have exceeded business goals; so they compare this in the report with companies that only upskilled analytics teams and people on analytics basically, and there 61% of organizations exceeded business goals. They actually see huge value in upskilling the whole organization and allowing them to learn how to use data. Here are a few examples of organizations that are already doing this, and they are realizing that upskilling is really the only way. 

You can see the example of Marks & Spencer and Amazon, but I want to zoom in on AT&T on the bottom right. They have a $1 billion 10-year long program that they launched on upskilling 440,000 people. This is really massive amounts that they are investing in upskilling people and making sure they have the right tools and knowledge to work with data. What I really want to stress here is that the answer to becoming data-driven is in your people.

Questions and answers

• Question: I'm going to start with the first one from Carla. What do you think is the most important skill to learn when learning business intelligence tools like Tableau or PowerBI?Answer:  I would say it's a very good question. I think mainly what you want to focus on is general data analysis skills, data manipulation, data visualization, and it's really about investigating the data in multiple ways possible. Have an open mindset, and don't just focus on one problem, and really try to look at data in as many different ways as possible.

• Question:  Do you think data scientists can also use BI tools, or is it just for the analysts like business analysts and data analysts?

• Answer: That's also a great question. I think analysts and scientists should be tool agnostic, and they should definitely be able to work with that tool. But that said, I think it really depends on what your organization or team needs in that specific way. So I wouldn't say it's definitely part of the tools they use, but it's definitely an option for them to use if it was a requirement or was asked from their organization, or simply if it fits their use case really well.

• Question: What do you think are the most popular BI tools to learn today?

• Answer: According to market research, both Tableau and PowerBI are the most popular at the moment, but you have others lurking around the corner. You have Looker which has been acquired by Google. You have Data Studio. There are definitely a few more options on the market, but the main tools being used right now are Tableau or Power BI.

• Question: How about using, for example, Shiny or Dash Python as BI tools alternatives. What do you think of that?

• Answer: Again, I think these have a huge business case. Probably for standardized reports that maybe are very core to the key performance metrics and KPIs of the business, it's a very good use case, and potentially even better, so you can even standardize them. But there are some things they do miss, such as drilling down, having different dynamic filters, looking at data from different use cases. So, I would really say it depends on the whole scenario around it. If you want to create a dashboard on what is the goal of the company and how are the main KPIs doing, and you don't want to add any other filters to really focus on the main metrics that you want to show to leadership, then an R Shiny dashboard might be absolutely perfect. But then other use cases where you want more flexibility in what you're looking at, I can see a lot of value in using a tool like Tableau or PowerBI.

• Question: What is the main difference between a data analyst and a data scientist in your eyes, Carl?

• Answer: I think a data analyst is really someone who should dig through all the data your company has. He or she, depending on their role in the company, should definitely have deep domain knowledge, and also know how to apply statistics or do a T-test and things like that. But I don't necessarily think they should be a machine learning expert or know how to productionize models for example. So I think that that's really where the big difference is. What you could say differently, as well, they don't necessarily have to be the best coders. It's my opinion on data analysts. So, they really need to be able to do a quick analysis, they know where all the data sets are and how to get to the most important points while someone like a data scientist probably in cases focuses on more complex problems. He or she also will need more time to do that and write codes, and helping to research different methods to do that is really important. That, for me, is a really big difference between data scientists and data analysts. I'm not sure if that answers your question, I hope it does. Otherwise, feel free to answer a follow-up question.

• Answer: There are always different ways to get to Rome, right? I think the main part is that you find the right people that are motivated and want to learn it, and then doing that, there are different learning resources out there. Of course, in DataCamp, we already have developed a full SQL curriculum and we are working on expanding our business intelligence content. So, that's definitely a great place to start as well.

• Question: What will be the best way to learn BI tools and SQL, as seemingly more and more organizations require us to use some of these more advanced technologies

• Answer: There are always different ways to get to Rome, right? I think the main part is that you find the right people that are motivated and want to learn it, and then doing that, there are different learning resources out there. Of course, in DataCamp, we already have developed a full SQL curriculum and we are working on expanding our business intelligence content. So, that's definitely a great place to start as well.

• Question: If you learn any one of the tools, Tableau or PowerBI, is it easier to learn the rest as well? Or should you know everything, and adapt based on the company's style?

• Answer: I think you should start by learning one which is on the company’s stack. Personally, for example, I learned Tableau because it was the one that's being used by the company I worked in, and I kind of learned that tool on the job. I really think it's good to get good at one of these tools and understand how to visualize data. Of course, you'll also learn tools specific to Tableau in this case, and knowledge and skills on how to apply that, but I think these are transferable to other business intelligence tools. 

• Question: How is PowerBI different from dashboards, let's say, developed in Excel?

• Answer: I would say the main difference here is how user-friendly it can be. Once you publish them, and they can be used by your stakeholders, that's also how PowerBI is in essence developed, right? So you can publish it, they can look at it, as we saw in an example, you can even give users a different view depending on their role or persona, like sales in different regions. In some sense, it gives a lot more flexibility on how you are showing your data to different people. In contrast with Excel, there's also no need to share files through Dropbox, Google Drive, or e-mails, as there is already a place that is shared for everyone to access. You can also restrict these people from touching the backend; you don't have to start protecting cells in your Excel and really do all this work in Excel to make your Excel as user-friendly as possible. PowerBI is built in a way to be user-friendly and to share data from the start.

• Question: Which programming language is most suitable in business intelligence work and mostly used? To rephrase that question, which programming language is the most used for an analyst role?

• Answer: If we exclude SQL, based on my personal experience, and I think also more recently, Python is becoming a lot more popular and also used a lot for automation tasks. Between Python and R, I would say that in many organizations, Python is indeed more heavily used than R.

• Question: What advice can you give to an organization that is just getting started and wants to scale its data maturity?

• Answer: I would really say here to focus on the skills and the culture to create this mindset where people look at data and data science as a methodology for solving problems. I really think it starts with upskilling at a general level and making sure that culture is there, and that people are rewarded for learning.