Automatically create function record for Expression.Evaluate in Power BI and Power Query

Some time ago I wrote a blogpost on how to create a function library in Power BI or Power Query (http://www.thebiccountant.com/2017/08/27/how-to-create-and-use-r-function-library-in-power-bi/). There I also presented a way to pull that function code automatically from GitHub.

Problem

In that code I used the function Expression.Evaluate to execute the imported text and create functions from it. The inbuilt functions that I’ve used in that code have to be passed as an environment record at the end of the expression. I’ve used #shared for it, as this returns a record with all native M-functions and is quick and easy to write (if environments are new to you, check out this series: https://ssbi-blog.de/the-environment-concept-in-m-for-power-query-and-power-bi-desktop/ ). But as it turns out, this can cause problems when publishing to the service unfortunately (https://social.technet.microsoft.com/Forums/ie/en-US/208b9365-91e9-4802-b737-de00bf027e2a/alternative-calling-function-with-text-string?forum=powerquery – please leave a vote if you would like to use #shared in the service as well).

Solution

To bypass these problems, you can specify each used function individually (as you can see in Tony McGoverns newest treasure here for example: https://www.tonymcgovern.com/powerquery/calculate-percentile/ ).

But depending on the length and complexity of your function, determining which functions are used and manually write down the record can become a bit laborious and spoil the fun. Therefore, I’ve created a function that does that for you autoMagically:

The formula

For the record

Pass in the query text as the first parameter and you’ll get a text string for the record:

Function record for Expression.Evaluate

For the query

And if you additionally pass in the URL for your function code as well, the full Expression.Evaluate-statement will automatically be created for you:

Full query text for Expression.Evaluate

Be aware, that the results of these functions are text that you have to copy and paste into your function code. You cannot simply reference the query, as this would repeat the problem with #shared.

Enjoy & stay queryious 😉

Permutations table in Power BI or Power Query in Excel

Today my husband came with a special task to me: He had to re-wire an old instrument on board of our sailing boat and “lost” the original matching instructions. There were 4 cables to go into 4 different holes, which left him with 24 possible combinations. So he needed a permutations table with all possible combinations that he could print out and tick each combination that he would try subsequently:

To be honest, he was quicker in creating the permutations table manually in Excel than me in Power Query, as it took me a while to develop an easy enough algorithm. But the technique I came out with at the end is so typical “Power Query”-ish, that I’m going to share it here:

The Video

The Function

Here comes the function code:

Enjoy & stay queryious 😉

Stretching and Compressing Time Series with Power Query and Power BI

The good folks from SumProduct have published a challenge for Excel that I find worth doing in Power Query instead:

The Challenge

Stretching or compressing a forecast so that the proportion of the original series will be maintained:

The Function

This is not so easy mathematically (as the solution-blogpost explains in detail), so the code for the function isn’t short as well:

Read more

RegEx in Power BI and Power Query in Excel with Java Script

From time to time Huang Caiguang sends me some geeky M-code that turns out to be very useful (like this for example). Today it’s utilizing java script code for RegEx (regular expressions): The Web.Page function can execute JS code and you can pass strings from you M-environment to it with simple string concatenation. So escape the js-code and use the ampersand (“&”) to reference the string. As a function it looks like so:

let
fx=(x,y)=>Web.Page(
“<script>
var x=”&x&”;var y=new RegExp(‘”&y&”‘,’g’);var b=x.match(y);document.write(b);
</script>”)[Data]{0}[Children]{0}[Children]{1}[Text]{0}
in fx(“””hello012中国1235″””,”\\d+”)

This code extracts all decimals (“\\d+”) from the string provided as the first argument and concatenates them with a comma. But just try for yourself by pasting the code into the advanced editor.

This doesn’t work in PowerBI service, so if you need it there, you can switch to an R-script that can be used with a personal gateway. But for Excel users, this is the way to go.

Please vote for a native implementation in M here: https://ideas.powerbi.com/forums/265200-power-bi-ideas/suggestions/8892295-regular-expressions-support-in-power-query

Enjoy and stay queryious 😉

Migrate a Power Query or Power BI file to a local SSAS instance

In Visual Studio there is a wizard to migrate an Excel Power Pivot model to a SSAS model. But this will not bring over the M-queries unfortunately. But there is a workaround to achieve this. It requires SQL Server 2017 or higher:

The steps:

  1. Import the Excel file in Power BI Desktop, save and close the pbix-file
  2. Open Azure Analysis service, open the Web Designer and create a new model where you import the pbix
  3. Open that model with Visual Studio (this will actually create a download that holds the VS-file)
  4. Open that file in Visual Studio, load the data, build and change the deployment target from Azure to you local SSAS-database before deploying.

See how it goes:

Warning: There are some limitations for the M-functionalities in SSAS (see here for example: General Overview by Microsoft or Use your own SQL … by Chris Webb), so you might want to give it a thorough test before rolling out. There are missing a lot of data sources currently, like web-queries for example who will hopefully soon be added as well.

This method has been described by Soheil Bakhshi here before: http://biinsight.com/import-power-bi-desktop-model-to-ssas-tabular-2017-using-azure-analysis-services/ 

Enjoy & stay queryious 😉

GuestPost: Newbie to Newbie Learning M-Language as your first Programming Language

Foreword from Imke: “Happy to publish my 2nd guest post here: I met Rafael Knuth in the Technet-forum where it was a joy to see how quickly he was set on fire by the M-language. When he vented the idea about starting a newbie-to-newbie-series where he would share his learning experiences and perspectives as an “Excel-guy”, I was quick enough to engage him for an intro on my site. As it turns out, he is a VERY talented communicator as well, but just see for yourself”:

I’m just a regular corporate marketing guy in his late forties with no formal programming education. However, one day I woke up and decided to teach myself to code. I had no plan whatsoever, and my learning journey was anything but a carefully planned venue. It was rather accidental fumbling & stumbling, accompanied by loads of frustrations, with frequent, prolonged breaks to recover from my failed attempts to teach myself to code.

What makes learning to code so hard?

These are the main obstacles in my views:

1) Lack of time
As a professional in a corporate environment, it’s nearly impossible to put 20 hours a week aside to teach yourself a new skill from scratch, without major sacrifices in other areas of your life.

2) Your brain’s “wrong” wiring
What makes learning so hard is the amount of knowledge you have to unlearn: “Why is my program not doing what I expect?” Because you set the wrong expectations. Rewire your brain.

3) Complexity of the subject
Coding is a hard piece of candy, bluntly speaking. There is good reason why there’s such a dramatic undersupply with good developers.

4) Lack of applicability of your knowledge
So, you did that course on Python at Codecademy. How do you put your newly acquired skills at work? Unless you prove me wrong, my answer is: Not at all.

5) Unrealistic expectations
Become a Data Scientist in a 6 month bootcamp. You will find tons of offerings like that. So, basically what it says, is: “You can be smarter than all those guys who put years and years into studying programming, mathematics, acquiring PHDs – just join our course and you’ll get there in no time.” Good luck with that.

Microsoft M-Language comes to your rescue

Read more