Power Query (M) pulling multiple pages

@jraiche wrote:

This PowerQuery/M code automatically grabs multiple pages of dreamfactory JSON recursively. The column will still need to be expanded, etc, as usual.

Improvements welcome, I'm by no means a powerquery expert, but I wanted to get dreamfactory working with PowerBI. While this is tailored to dreamfactory, it should work with any paged api.

let getDFUrl = (url as text, optional previousResult) =>
        let
			Res = Json.Document(Web.Contents(url)),
			offset = 
                            try Res[meta][next]
                            otherwise 0,
			combination = if (previousResult <> null) then
				Table.Combine({
					previousResult,
					Table.FromList(Res[resource], Splitter.SplitByNothing(), null, null, ExtraValues.Error)
				})
			else
				Table.FromList(Res[resource], Splitter.SplitByNothing(), null, null, ExtraValues.Error),
			
			Result = 
                            if (offset <> null) then
                                if (offset > 0) then 
					@getDFUrl(url & "&offset=" & Number.ToText(offset), combination) 
				else 
					combination
                            else
                                "nothing..."
        in Result
in getDFUrl("your_dreamfactory_url_with_filters_and_api_key_and_stuff")

This query seems to have issues in Excel at the moment, but PowerBI is working fine.

Posts: 1

Participants: 1

Read full topic