AI + Power BI = Wow BI

I’m glad to inform my readers that Microsoft is adding new AI capabilities inside Power BI. And, these are no-code solutions.

Let’s check these 4 new exciting capabilities in detail and in the order of quick wins as per me.

In this post, I will explain uses cases with examples from multiple industries for each of the new capabilities coming in Power BI. This will be followed by a general approach to solve such problems, and then the new AI + Power BI approach to solve such problems.

Capability 1: Key Driver Analysis – or Key Influencer Analysis

Note: Per Microsoft this would be available to all Power BI users.

Suppose you have a dataset of employee attrition which includes details of the employees who are in the company, who left the company along with age, gender, salary, job role, satisfaction, education, years with current manager etc.

Your task is to find factors influencing attrition. Why are employees leaving the company? What segments of employees are leaving?

A general approach for answering such questions would be to use R or Python, fit a model (say using Random Forest algorithm) or use techniques like RFE (Recursive Feature Elimination) to find out top factors affecting our label – Attrition. More details on this general approach and how we did this using R and Power BI is mentioned in detail in our case study here.

With new AI capabilities in Power BI, this would be just a click away. The outcome of the analysis from Power BI would be shown as a kind of “lollipop” chart as shown below.

Image source: Microsoft

Example: When Parental encouragement is true, the probability of a student to plan to attend college increases by 1.8x,

Or, when the employee has spent more than 2 yrs with current manager and his job satisfaction is low then attrition increases by 2.3%

From the screenshot it is not clear how multiple driver analysis can be performed: Ex: When parental encouragement is true and Gender is male – what happens then? 

A contingency matrix would have helped in this case.

Capability 2: Azure Cognitive Capabilities – Sentiment Analysis, Image tagging, object detection in Power BI

Note: Per Microsoft this would be a Power BI Premium capability

You started a campaign on Twitter and would like to analyze your users sentiments – positive/negative.

For a call center company you would like to analyze chat script and identify key items customers are talking about right within your BI reports.

Or, an E-Commerce company would like to detect objects in the images attached with customer reviews, and identify which product/brand is causing negative sentiments or causing pulling “Andon Cord”.

A general approach would be to use Azure Cognitive APIs inside your Power BI report using Power Query (more about this later) using calls such as: Web.Contents(AzureAPICallWithParams).

Another general approach would be to develop and use custom Deep Learning models. A Twitter sentiment analysis (racial vs non-racial tweet) model was developed by us and is hosted in our GitHub repo.

With new AI capabilities in Power BI, this could be just a matter of invoking a function from Power BI ribbon. We do not know yet how this will be invoked by users. But, definitely this will make our BI reports comprehensive and improve decision making.

A snippet of such comprehensive report is attached below.

Image source: Microsoft

When this comes out in preview we will have to see if Microsoft has provided ability to not fire API calls for items already tagged/analyzed – otherwise you will have to pay for every API calls (even for repeats).

Capability 3: Automated Machine Learning models

Note: Per Microsoft this would be a Power BI Premium capability

Imagine in your Power BI report along side Sales Oppty data I provide you a confidence score or probability score against each Oppty data. The Oppty owner can look at this number and decide which Oppty are more likely to be won so he/she can then focus his/her efforts on top highly likely Oppty.

A general approach to add this would be a data scientist developing such models and a developer integrating it inside the Power BI report, and a business analyst consuming the report.

With new AI capability, Microsoft is targeting business analyst so they can build, train, and apply the models right within Power BI service without writing a single line of code. Isn’t that waow?

From the initial screenshots by Microsoft, it looks like this will be part of DataFlows (another new capability, which I will talk about in later posts)

Image source: Microsoft

When this feature is out in preview, we will have to see how easy will it be to do feature engineering – feature selection, normalization, pruning, binning etc. But, this is for sure going to ease out the effort in long term.

Capability 4: Use your existing Azure ML Models in Power BI

Note: Per Microsoft this would be available to all Power BI users.

This capability is more of easing out collaboration between a data scientist and a business analyst.

Typically a data scientist builds models in Azure ML platform and publishes the model as API endpoint. A data analyst or engineer uses that model endpoint to predict outcomes and populate the data inside the BI report. This BI report is then consumed by a business analyst.

In the new AI approach the models developed by data scientist would easily be searching in Power BI, and a new interface would be provided in Power BI to hook to that model and use it in reports.

There are no screenshots for this capability by Microsoft. 

—–

The public preview of these capabilities will be launched towards the end of Nov 2018.

We would evaluate these capabilities and posts about it when they arrive.

What thoughts you have on these capabilities? How are you going to use these capabilities?

Let us know.

Thank you,

Ranbeer Makin

References:

Power BI AI Capability Announcement

Power BI AI Capability Preview Signup Form

Power BI Embedded for Gov clouds

Most of the articles/posts/questions published on community or websites would be on Power BI “Commercial” product. The URL you have seen would be https://app.powerbi.com

This blog post is about “https://app.powerbigov.us” – Power BI URL for US Gov customers. More specifically this blog post is about embedding Power BI assets (Reports, Dashboards, QnA) for US Gov customers.

I would like to highlight certain differences in app registration URL and config URLs when embedding Power BI assets for US Gov clouds.

-> App Registration URL

The app registration URL for commercial cloud and US Gov cloud for Power BI Embedded are different. For:

  1. Commercial Product -> https://dev.powerbi.com/apps
  2. US Gov Product -> https://dev.powerbigov.us/apps

-> When you register your app, the Power BI API name in Azure Portal would look like this:

Commercial Product

commercialproduct

US Gov Product

govproduct

TIP: Based on our experience, when troubleshooting please look out for the API name that you see in Azure Portal. If you are embedding for Gov clouds, the API name should be “Microsoft Power BI Government Community Cloud”. If that is not the name, you would need to register the app on https://dev.powerbigov.us/apps

-> Embedding Config URLs

Commercial Product

commercialproductconfig

US Gov Product

govproductconfig

You would see “resourceUrl”, “apiUrl” and “embedUrlBase” is different in case of US Gov clouds.

That’s it. These are the only changes in configuration and setup required for embedding in Power BI US Gov clouds. Everything else remain the same.

If you are looking for details on embedding with a sample .NET code, please head to our  blog post on baby steps to embed your Power BI report.

Apply the configuration changes as mentioned in this blog post and you would be set to see the Power of embedded analytics for your US Gov clouds.

Questions? Please contact us through our website.

Thank you

Who else wants Heat Stream analysis with Power BI?

Namaste! It’s been a tiring month – working on customer projects, building a product prototype, getting work done by my team, phew! – I’m donning multiple hats. Recently we wrapped up two projects on showing heat streams with Power BI. The projects were challenging, and you know customers will take out the best from you. And, it happened with us as well…

Heat streams could be very useful in analyzing large amount of data sets and analyzing patterns or “heats” over a period of time.

Some use cases of heat streams could be:

  1. Analyze call center calls by weekday and time of the day. The time of the day as X-axis and weekdays as Y-axis with the number of calls as “heats”
  2. Perform clickstream analysis for website clicks
  3. Analyze Patient re-admissions and re-admission types in a hospital over a period of years

Usually, in a heat stream visual, we put the time of the day or date or year on X-axis, a discrete or a continuous value on Y-axis, and fill the visual with a discrete or a continuous value with gradient colors.

The code that we developed used ggplot and geom_raster layers along with various settings for formatting axes. This R code combined with Power BI gave us BI capabilities. The visuals were seamlessly sliced/diced based on the data we selected in Power BI. I’m attaching here screenshots of visuals that we created using R and Power BI.

Our customers were wowed by the output they saw from the data. Remember, if data is the new oil, then insights in the new king. And we do this using interesting and stunning visuals.

Screenshot 1:

HeatStreamVisual1

Screenshot 2:

HeatStreamVisual2
Heat stream Visual 2 using Power BI and R

Note: You need large amount of data to have this kind of output. We can further improvise these visuals to be interactive. This can easily be done using plotly and htmlwidgets library combined with Power BI.

The biggest challenge you will face in plotting such visuals is handling large amount of data points on “x-axis”. You may have to use breaks or cuts to limit the points.

Have you plotted heat streams in Power BI/R? What were the most challenging aspects of your project?

We would love to know.

Thank you

R

Note: Next week we will be starting a series of blog posts on how we secure customers data with data anonymization and masking techniques. There are some incredible techniques that we use which give our customers 100% confidence in data security. 

Do subscribe to our blog posts to not miss our proven data masking techniques and other interesting articles.

Note: There is a custom visual for plotting heat streams in Power BI, but it cannot generate heat streams anything like what we have shown above.

The biggest challenge in applying Machine Learning is….

Machine Learning and AI applications are everywhere. Recently Andrew Ng launched drive.ai – self-driving cars. He aptly said: “The future is here”. But I must say this future arrived pretty quickly!

After talking to our customers and over the period of time we found that there are many challenges before we apply machine learning or AI algorithms. Many of our customers want to see results quickly. We take them through methodical steps to avoid surprises.

Here are few challenges to applying machine learning to solve business problems.

1. Do we know what problems to solve? This is the first question you should ask, and the biggest one. My customers want to train deep learning model on the cloud but when my team asks them deeper questions about what they want to solve, they do not have answers. Or even if they do have answers they are not very clear.

This is where our team’s expertise comes in. We ask questions to our customers to help them understand what problems they want to solve. Questions range from asking about business objectives, what is the current problem, what results are they expecting, what is their vision etc.

2. What data do we have? Machine learning or AI algorithms rely on data. In order to predict future, you need to know past behaviors. In order to know past behavior, you need historic data. In most cases, you have data available. But the question is: is the data relevant? Is it cleaned? If you want to predict a customer’s next purchase you need to have customers historic transactions, demographics details. Another question is, is having data enough?

3. Do you have labeled data? For you to apply ML classification techniques you need to have labeled data. For example, we were working with one of our customers to automatically generate marketing headlines using Deep Learning models, in this problem we need a lot of marketing articles with “good” and “bad” headlines so ML engine knows what is good and bad.

Similarly, in a classic problem of tweet sentiment analysis, you need to label a tweet as positive or negative before ML engine can predict a new tweet’s sentiment.
Who will tag the headlines as “good” or “bad”? Who will put tweet’s sentiment as positive or negative?

4. Do you have trained people? This was the biggest challenge before but now no more. There are many online courses available where one can learn basics and advanced materials. One has to push their limits and learn new materials. I train my team through these courses that are available. Some of them are free while some of them cost as little as $10.

The challenges remain the same customer to customer. They just take different shapes and sizes. We take our customers through methodical steps in solving their business problems. We make hypotheses, test them iteratively, present findings and outcomes, and proceed to next milestone.

It’s always good to take baby steps in Machine Learning and AI Problems.

Please get in touch with us for your data analytics and data science needs.

 

5 step approach to Data Analytics problems

Data Profiling – or simply data discovery. Based on your business objectives and what information you would like to extract we understand your data with a variety of statistical measurements. This could be from simpler methods like mean, median to more complex methods like variance or standard deviation or quartile analysis

Outcome: a sense of data ranges and fields to build next steps in analysis

Find systematic patterns using Regression – we apply regression techniques to identify significant relationships and strength of impact of independent variables to dependent variables.

Outcome: key variables impacting our outcome, and their strength which can help us prioritize in case of conflicts. Ex: living in San Francisco vs having a family is more important factor for buying a car.

Correlation analysis – Looking to see if there are unique relationships between variables that are not immediately obvious. Ex: Is credit score and monthly income correlated? If yes then how does this impact my outcome?

Outcome: set of parameters/variables highly correlated which will impact our business decisions.

Outlier analysis – are the outliers showing us new emerging trends or are they just outliers? We need to check this in our data to capture trends early.

Outcome: set of outliers with their future impact on outcome. What’s the onset of those outlier?

Cohort/cluster analysis – data segmentation. Then combine the outcome of previous steps with coming up with cohorts or segmentation to target with business objectives. This is where we discover consumer preferences, segment our data, and analyze micro data for improving our decision making

Outcome: set of variables and their values which result in targeted cohorts

All in all combined you can see a full picture of your data with results and outcomes expressed as visuals for better understanding of your data.