Real time reporting with Power BI
By 2025, close to 30% of the data will be real time – IDC
We have 6 years before we hit that, but I must say real time data or rather ability to report it real time is key for business success even now.
Imagine a scenario where you could view production plant quality metrics real time on a TV?
A grid operator tracks electricity flows in real time. Metrics like Power, Voltage with trends and power lines plotted on a geo map and viewed LIVE!
Sensory data, campaign performance (as they happen), field sales activity, and many more examples of real time data and real time reporting needs.
Some of the above examples are definitely real time but customers also ask for near real time – as soon as my new data is available I need to view it in my reports and dashboards within couple of mins.
How do we achieve real time and near real time reporting scenarios with Power BI Platform?
There are couple of solutions to achieve this:
- Power BI Scheduled Refresh (near real time, 8 refreshes per day for Pro, 48 refreshes per day for Premium/PPU)
- Power BI Direct Query or Live connection
- Power BI Push data sets
- Streaming data sets in Power BI
Next, we will cover real time and near real time reporting scenarios and discuss possible solutions along with limitations/pros & cons. Each scenario and solution is unique, and you can use any of the proposed solutions based on your needs.
Scenarios:
a) I would like to have my sales dataset refresh every 2 hours or I would like to have my Google Analytics dataset refresh every 1 hour.
When your dataset refresh time gap between two refreshes can be more than 30 mins, you can go ahead with scheduled refreshes. This assumes you have imported the data in a Power BI dataset and you would want to refresh that dataset.
Based on our experiences when you present schedule dataset refresh options to customers they are not taken aback. Scheduled refreshes have limits on the number of refreshes per day but once you schedule them to occur more frequently in business hours, it helps.
With Power BI Pro Licenses you can setup 8 refreshes a day per dataset. So you can setup refresh times to be: 7AM, 8AM, 9AM, 11AM, 2PM, 3PM, 4PM, 5PM
Total 8 refreshes spread out in business hours.
With Power BI Premium/PPU Licenses you can setup 48 refreshes a day per dataset. So you could do refreshes every 30 mins! Power BI Premium is definitely costly (read USD 60K/year) but you get more refreshes, more capacity and enterprise level features (AI and more). Update: March 10, 2021. Power BI Premium per user, a new offering, is priced at $20 per user per month.
Or, you can put the workspace where your dataset resides in an Embedded Capacity (Premium/Non-Premium) and get rid of refresh limits. In this case you will have to use Power BI REST APIs to refresh the dataset. Quite simple to invoke. Read more on this capability here.
You can even use Power Automate and trigger refresh based on events.
More on Power BI Refreshes in this Microsoft post.
b) I would like to have my dataset refresh every 1 min (or LIVE)
Assume you have a data source which can be directly queried from Power BI, you can achieve this scenario.
Sales data stored in a SQL box (Azure or on-prem), you setup direct connection to this data source in Power BI. Your Power BI Report is now directly connected to the data source.
If you are viewing this report on a browser you can hit browser refresh and it would fire query live to the source, fetch latest data and show on your visuals.
You can even automate the browser refresh using a Chrome Plugin. Read more here on how to achieve this.
List of data sources which can be directly queried from Power BI Platform here.
c) I would like to have my production plant quality metrics to be shown live
This assumes the data source where your quality metrics are stored is not direct-queryeable by Power BI.
In this scenario you need to push data from your data source to either a Power BI Push dataset or to a database. This push can be done every 1 min or so. This push can be done by setting up Azure Functions.
Once the quality metrics are pushed to a Push dataset or to a database, you can setup Power BI reports on top of it and view the data live.
Power BI Push datasets have some limitations as mentioned here.
How to setup Push datasets? You can read this external blog post here.
d) I would like to see electricity flows and their Voltage and Power metrics live on my TV
If you need something real and live, something like electricity flows or sensor data, you can setup Streaming datasets in Power BI.
Once you have streaming datasets setup, you can view them through Power BI dashboards. These datasets do not store historical data. They will store limited data in temp cache.
You can open these dashboards in a browser and put them on a TV. No need of browser refresh as the tiles would refresh “live”.
How to setup Streaming datasets? You can watch this YouTube video by Patrick (MSFT) here.
Read more here on Power BI Push and Streaming datasets.
Did we cover all the possible solutions? Ok, let’s check it once more.
Additional Scenario 1: We are a global organization and we would like to view our Google Analytics data every 1 hour. The dataset is shared with our global marketing team spread across every continent.
Solution suggested: In this case instead of going with scheduled refreshes, you can go with pushing GA data to either a Power BI Push datasets or to a database and setting up direct DB connection. Or, if you are an enterprise with Power BI Premium licenses you can set 30 mins refresh intervals.
Additional Scenario 2: I have more than 5 plants where I need to view live data. Should I setup one dashboard with filters?
Solution suggested: You can view data from streaming datasets only in a dashboard. Dashboards do not support filters yet. In your case you can setup 5 dashboards and share these URLs.
Any other solutions you want us to include?
Let us know.
Thanks
Ranbeer Makin
