After a short holiday break, it is time for a new blog post! This time about one of the most exciting functionalities in Power BI that has been released in the past release wave, Power BI Deployment Pipelines!
In this blog I will elaborate on what Power BI Deployment Pipelines are, why it is an important part of your landscape and a comparison to other options you have nowadays. But most important, why you should really care about Deployment Pipelines!
You might have seen the announcement, Power BI Deployment Pipelines have been released in May 2020. It is around for about two months now. On different social channels I have seen a lot of buzz around it already, both positive and negative honestly. Though, I think this is a great step forward!
Back in 2018, I posted a blog about multi-tier architecture and continuous delivery with Power BI. If you are not familiar with a DTAP approach and why this helps you to structure your development processes, I advise you to first read that blog. Personally, I am really excited about Deployment Pipelines! With this functionality, Microsoft starts offering an out-of-the-box functionality that helps you to easier move your Power BI content through you DTAP pipeline.
The positive side
Deployment Pipelines allows you to configure up to three stages in your pipeline. What I really like about the setup is the easy intuitive interface to configure your pipeline, as well as the deployment parameters you can set. It is easy to switch a parameter value as the content moves through the pipeline for example.
The ability to be selective in the content you deploy is a huge win! This allows you to only deploy that one report, or that one change in your dataset, instead of moving all the content in one go.
A second advantage for Deployment Pipelines is the meta data deployment. By moving content to the next stage, a meta data deployment is done. As a result, you are not overwriting your dataset which would have required a full refresh. This helps you in deploying only that one measure or doing quick fixes on your report without the need to directly working on production or create a temporary freeze of content.
Deployment Pipelines will also provide you a clear overview of all your stages in one place. This will be your go-to navigation for each project that you run in a DTAP approach and links to every single workspace in this setup. Permissions for the pipeline are separated from workspace permissions, what is a big advantage if you ask me. For example, you might want to limit the users in the production workspace to the bare minimum, while development and test can include more users.
I want to avoid re-writing the same content as others have done already. If you want to see more about Deployment Pipelines, I advise you to have a look at below video by Adam Saxton about Deployment Pipelines.
The less positive side 😉
Though, I must admit some of the less positive feedback I have seen online is valid and I have to agree with them till some extend. I will highlight a few of them below.
One of the most seen things, is that Deployment Pipelines are only available for Power BI Premium. I think this mainly related to the fact what I described above about meta data deployment. I think that XMLA endpoints are leveraged on the back-end in order to do the meta data deployment. Though, I would love to see Deployment Pipelines coming to Power BI Pro as well, but I doubt if this will happen. At this moment, there is no sign that this is planned or will happen anytime soon.
Maybe you are already familiar with different Premium options you have, like A-SKUs, EM-SKUs and P-SKUs. The requirement for Premium can be a blocker for the adoption of this functionality if you ask me. You can still do that with the development and test stages in Deployment Pipelines. There are some things you should take into account when it comes to the Premium requirement. I advise you to checkout the Microsoft docs about this topic to see if this is a blocker for you.
Another thing that I have seen and is a huge blocker for me personally as well, is the lack of functionality for dataflows in Deployment Pipelines. At the moment of writing (spoiler!) it is not possible to deploy dataflows via Deployment Pipelines. For a clean DTAP approach, you want to have identical environments for each stage in the pipeline. This also includes dataflows.
Versioning is a missing functionality at this moment. In the unfortunate situation that you messed up in your deployment, there is no easy roll-back scenario that can be executed. However, versioning is possible in a slightly different approach by saving your PBIX files on SharePoint or OneDrive, but will make it a bit more difficult to do a easy rollback. First thing that crossed my mind is integration with Azure DevOps, which is in line with other Microsoft products and services.
Finally, an improvement that is maybe not relevant for all of us. If we want to onboard Deployment Pipelines in our existing processes, it would be very welcome to have REST API calls to interact with Deployment Pipelines. The basic thing would be triggering a pipeline to run but adjusting settings or defining what content should be deployed is even more awesome!
What is coming?
Recently, the 2020 release wave 2 plan was shared by Microsoft. In this release plan there are multiple items related to Deployment Pipelines. It is good to see that Microsoft keeps investing in it and further develops the support for DTAP processes.
Below items are coming according to the public release wave plan.
The descriptions for each of these items is still very limited. So, there is not much more to share at this moment. But keep an eye out on the release plan to see future enhancements to the plans!
Even before Deployment Pipelines existed, I wrote a blog about Versioning and CI/CD for Power BI with Azure DevOps. In this blog I describe how you can achieve similar goals. The comparison between the two approaches is worth to look at. I will not repeat all the content that I described in the Azure DevOps blog, but highlight the most interesting differences below.
Working with multiple users
If your scenario requires to work with multiple users, both the Azure DevOps as the Power BI Deployment Pipelines will work for you. In both scenarios you can configure privileges in a sufficient way, separated from the content itself.
For Power BI a Pro license each user is needed to work with Deployment Pipelines + the need for a premium capacity, where Azure DevOps also requires sufficient licensing to get it working. This can be either a user based or service-based license. On this page you will find more about Azure DevOps pricing.
On this specific topic, I have no preference. My advice should be to go for the approach that best fits in your current way of working in the organization.
Incremental deployment and meta data deployment
Here we hit the first difference. As I explained in the previous part of this blog, Deployment Pipelines are working via a meta data deployment. This is a huge win over the current setup with Azure DevOps which overwrites the current dataset and report in the Power BI service during the release.
The DevOps approach is at a disadvantage for this specific topic. Though, there are ways to open-up similar functionality with Azure DevOps by leveraging the XMLA Read/Write endpoints that are available for Power BI Premium nowadays. This is not an out-of-the-box functionality and is a more technical approach and requires more configuration.
Exploring XMLA endpoints via Power BI premium is worth looking at, but a technical matter. Based on the current setup, I would go for the meta data deployment with Deployment pipelines.
The current setup for Deployment Pipelines require Power BI Premium. This creates a higher threshold to prefer Deployment Pipelines over Azure DevOps. Especially since DevOps works for both the Premium and Pro environment.
My preferred option is Azure DevOps in this case. Simply because you can go for one way-of-working for every scenario.
There is no versioning option in Deployment Pipelines at this moment, while Azure DevOps offers this as part of the solution. You might still consider going for Deployment Pipelines. The option you have nowadays for versioning is saving your files on OneDrive or SharePoint, which requires an additional product/service and additional access management. Integration in the platform would be beneficial. Although versioning is not on the roadmap, I can imagine (and hope!) that Microsoft will invest in versioning capabilities for Deployment Pipelines.
Currently, I would opt for DevOps here. If you want to avoid switching again over time and can live with the current limitation of Deployment Pipelines, you might decide otherwise.
Honestly, I think the inability to work with dataflows is one of the biggest blockers for Deployment Pipelines at this moment. At least it is for the clients I work for. Although The DevOps approach also does not offer this functionality but can be added fairly easy. For example, have a look at this blogpost about moving dataflows across workspaces. Since DevOps is fully customizable, you can easily add this step to your process.
The freedom to customize your process by adding dataflows as part of the pipeline is a huge pro for DevOps. I would opt for DevOps in this specific scenario.
Deployment Pipelines in Power BI is definitely a functionality that makes a huge difference in the level of professionalism of your development process. It helps you and your team in better structure the end-to-end process, by a proper test environment and release management.
Though, there is some functionality lacking in Deployment Pipelines. The 2020 release plan wave 2 looks promising with new functionality coming to Deployment Pipelines, but this will not fill all the gaps that are there at this moment. Usually the release plan is just a part of what is coming, so let’s hope for more!
Having that said, let’s conclude the battle based on the functionality that is available at the moment of writing. Personally, I would prefer DevOps over Deployment Pipelines. That is mainly based on the lack of versioning functionality and requirement for Premium in Deployment Pipelines. For all other items, you can think about workarounds.
At the same time, Deployment Pipelines is very promising! Please do not forget to keep an eye on the future development of this functionality. If you get the change, I would recommend you to start playing around with it. I dare to suggest that my personal preference will change over time. In the end it is all about the right fit for your organization and the goals you want to reach!