Update: Move dataflows across workspaces

Over a year ago, I wrote a blog about moving dataflows across workspaces using a PowerShell script. Especially useful if you want to move dataflow logic from your development to test, acceptance or production workspace.

I received a bunch of feedback on this script and run into some issues myself as well lately. It was about time for an update of the script! Below I share the issues that are addressed in this new version and what new additions are added to the script.

Move dataflows across workspaces using PowerShell

Issues addressed and new enhancements

The script leverages the Power BI REST API for imports in a group. This API can be leveraged for publishing pbix files as well as dataflows.

Conflict Handler Mode
The issues I faced pretty early already, is the Conflict Handler Mode. This parameter in the API call defines whether the existing dataflow should be overwritten, ignored or generating an unique name for the dataflow.

After some trial and error with this specific setting, this should now be addressed in the new version of the PowerShell script. By first time deployment, it will publish the dataflows with the name as is. If the script detects that there are existing dataflows with a similar name, it will use the overwrite setting in the conflict handler mode.

Changing connection strings
If you move a dataflow from Development to Production, what is a very likely scenario in a DTAP development approach, it is also likely that your data source changes at the same time. As the previous version of the script simply extracted the model.json from workspace A and uploaded in workspace B, there was no easy way to change the connection string during deployment. This resulted in manual steps after the dataflow was moved from A to B.

As dataflows do not support change of parameter values outside the dataflow yet, another solution had to be found. In the new version of the script, there is a new option which does a simple find & replace on a text string. So, if you specify the connection string for environment A as well as for B, it will do a find and replace during deployment. Below example shows the connection strings that are specified in a variable of the script.

< rest of the script > ...
$replaceItems =@(
[Replacement]::new("https://dlswedemod01.blob.core.windows.net","https://dlswedemop01.blob.core.windows.net")
)
... < rest of the script > 

Move multiple dataflows at a time
Finally, it could result in a lot of work if you want to move a whole bunch of dataflows. The previous version of the script only supported to move one dataflow at a time.

The new version now supports to move multiple dataflows at a time. By specifying another variable in the script, you can easily conduct a list of source workspace, destination workspace and the dataflow name that you want to move during script execution.

< rest of the script > ...
$dataFlowItems =@(
    [DataFlow]::new("Sales Analytics DEV","Sales Analytics PRD","Sales Analytics Actuals"),
    [DataFlow]::new("Sales Analytics DEV","Sales Analytics PRD","Sales Analytics Budgets"),
    [DataFlow]::new("Sales Analytics DEV","Sales Analytics PRD","Sales Analytics Forecasts"),
    [DataFlow]::new("Sales Analytics DEV","Sales Analytics PRD","Sales Analytics Management Structure"),
    [DataFlow]::new("Sales Analytics DEV","Sales Analytics PRD","Sales Analytics Opportunity")
)
... < rest of the script > 

The full script, including description how to use it, is now available in my GitHub library for Power BI Automation. Find the full script there and take advantage of it!

Finally, I want to do a big shout-out to my colleague Ton Swart who contributed to this script to make these changes happen!

Links

2 thoughts on “Update: Move dataflows across workspaces

  1. Pingback: Move dataflows across workspaces with the Power BI REST API – Data – Marc

  2. Pingback: Moving Power BI Dataflows Across Workspaces – Curated SQL

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s