Convert DateTime to ISO 8601 date and time strings in Power Query

Often, when querying APIs it is required to enter date and time filters in ISO 8601 format . Today I show a quick way to convert DateTime to ISO 8601 string, based on an ordinary DateTime field according to the following pattern:

2020-10-11T15:00:00-01:00

This represents the 11th October 3pm in UTC -1 timeszone.

Steps to convert DateTime to ISO 8601

If I enter:

#datetime(2020,10,11,12,0,0)

into the formula bar, it will be converted to :

11/10/2020 12:00:00

Comparing to the desired ISO format the year, month and days are in the wrong order. So using the universal Text.From function will not return the correct result.

Fortunately, there are a couple of xxx.ToText function in Power Query that allow for dedicated formatting parameters in their 2nd arguments. For example, the function DateTime.ToText can actually return the string in the desired format if you pass a format string as the 2nd parameter:

Convert DateTime to ISO 8601

DateTime.ToText with format parameters

The syntax for these format strings can also be found here.

Last step is to add the time-zone string ( & “-01:00”), as I’ve started from a DateTime value only:

DateTime.ToText(DateTime.From(#datetime(2020,10,11,12,0,0)), "yyyy-MM-ddThh:mm:ss") & "-01:00"

Enjoy & stay queryious 😉

Retrieve header fields like response status from Web.Contents in Power BI and Power Query

Many Power Query function not only return their values as advertised in their function documentation, but on top of that a metadata record. This record is like tag that holds additional information about the returned main value (for more details about this, please check out my friend Lars Schreiber’s article about it).

Useful metadata for the Web.Contents function

Today I discovered that the function Web.Contents delivers a really nice record with a couple of useful information. To retrieve header fields, you have to use the Value.Metadata function, like so for example:

Return header fields like response status from Web.Contents

Interesting metadata from the Web.Contents – function

This might help for some advanced web query tasks.

How to use

If you want to use this in production, you’d probably branch out the logic. So first use Web.Contents and keep that result in a column or variable. Then add another column that references it and return the metadata record.
Apply the logic check on it and create a last column where you finally parse the content from the binary that Web.Content has returned.

Enjoy & stay queryious 😉

Split up an existing Power BI report into a Golden Dataset and a thin report

The other day I discovered a neat way to split up an existing Power BI report into a Golden Dataset and a thin report file with very few adjustments to the existing setup. Imagine you have a Power BI report published for some time already in an app with row level security. Now, you want to create other reports from the dataset as well and decide it’s time to create a golden dataset from which multiple other thin reports can also be fed from as well. But ideally you want to keep your published app, that many users are working with already, unchanged.

Concept to split up existing Power BI Report to Golden Dataset

Simply follow these steps in meticulous order and the world is yours 🙂

Steps to split up existing Report to Golden Dataset

Read more

Transform a query into a function in Power Query and Power BI

In my previous blogpost I’ve described a method how to extract a substring that follows a certain pattern from a string. In this post I show how to transform a query into a function that can be applied to many rows of a table.

Video how to transform a query into a function

Please check the video for detailed steps. In there I also show how to modify the code. It shall also detect strings with a sequence of just 8 numbers. In the original query, those had to be followed by a minus sign and another number:

Read more

Extract pattern string and numbers from text using List.Accumulate in Power Query

A typical task when cleaning data is to extract substrings from string that follow a certain pattern. In this post I’m going to describe a method that uses the List.Accumulate function for it. Extract a pattern string.

Task

I have to extract a payroll key from a description field that starts with 8 number, followed by an “-” and another number.

aölsfdk0125-fds  da12345678-0asdf

So I’m after the 12345678-0.

Plan

I plan to approach this by

Read more

Performance tip to speed up slow pivot operations in Power Query and Power BI

Pivot operations in are a very handy feature in  Power Query but they can slow down refresh performance. So with some bittersweet pleasure I can tell that I found a trick to speed them up. The sweetness comes from the fact that the performance improvement is very significant. But the bitterness comes from the fact that I could have used this for almost 4 years now, but was too blind to realize at the time when I first worked with the code.

Trick to speed up a slow pivot table

This might not work everywhere, but for my tests, it worked really well: Don’t use an aggregation function when you want fast pivoting:

slow pivot

Don’t aggregate when you want a fast pivot in Power Query

But if your data isn’t aggregated on the row- & column values already, you’ll get this error message:

Error when the values are not adequately aggregated

So to make this work, you have to aggregate the values on the axis’ values before.

Let’s walk through the steps:

Walkthrough

Read more