[go: up one dir, main page]

0% found this document useful (0 votes)
30 views7 pages

(Power BI Data Transformation) (Cheatsheet) - 2

Uploaded by

Yash Palwankar
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
30 views7 pages

(Power BI Data Transformation) (Cheatsheet) - 2

Uploaded by

Yash Palwankar
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 7

# [ Power BI Data Transformation ] [ cheatsheet ]

1. Data Import

● Import data from a CSV file: Csv.Document(File.Contents("C:\data.csv"))


● Import data from an Excel file:
Excel.Workbook(File.Contents("C:\data.xlsx"))
● Import data from a JSON file: Json.Document(File.Contents("C:\data.json"))
● Import data from a SQL Server database: Sql.Database("server",
"database", [Query="SELECT * FROM table"])
● Import data from a web API: Web.Contents("https://api.example.com/data")
● Import data from a folder: Folder.Files("C:\data")
● Import data from a SharePoint list:
SharePoint.Tables("https://contoso.sharepoint.com/sites/mysite",
"ListName")
● Import data from an OData feed:
OData.Feed("https://services.odata.org/V4/Northwind/Northwind.svc")
● Import data from an ODBC source: Odbc.Query("dsn=MyDSN", "SELECT * FROM
table")
● Import data from a text file:
Text.FromBinary(File.Contents("C:\data.txt"))

2. Data Cleansing

● Remove duplicate rows: Table.Distinct(table)


● Remove blank rows: Table.SelectRows(table, each not
List.IsEmpty(Record.FieldValues(_)))
● Replace null values with a specific value: Table.ReplaceValue(table, null,
0, Replacer.ReplaceValue, {"ColumnName"})
● Remove rows with errors: Table.RemoveRowsWithErrors(table)
● Fill down missing values: Table.FillDown(table, {"ColumnName"})
● Fill up missing values: Table.FillUp(table, {"ColumnName"})
● Trim whitespace from text: Table.TransformColumns(table, {{"ColumnName",
Text.Trim, type text}})
● Clean text by removing non-printable characters:
Table.TransformColumns(table, {{"ColumnName", Text.Clean, type text}})
● Capitalize text: Table.TransformColumns(table, {{"ColumnName",
Text.Proper, type text}})

By: Waleed Mousa


● Lowercase text: Table.TransformColumns(table, {{"ColumnName", Text.Lower,
type text}})
● Uppercase text: Table.TransformColumns(table, {{"ColumnName", Text.Upper,
type text}})

3. Data Transformation

● Rename columns: Table.RenameColumns(table, {"OldColumnName",


"NewColumnName"})
● Reorder columns: Table.ReorderColumns(table, {"Column1", "Column2",
"Column3"})
● Remove columns: Table.RemoveColumns(table, {"ColumnName"})
● Filter rows based on a condition: Table.SelectRows(table, each
[ColumnName] > 10)
● Sort rows: Table.Sort(table, {{"ColumnName", Order.Ascending}})
● Group rows and aggregate: Table.Group(table, {"GroupColumnName"},
{{"AggregateColumnName", each List.Sum([ColumnName]), type number}})
● Pivot data: Table.Pivot(table, List.Distinct(table[PivotColumnName]),
"PivotColumnName", "ValueColumnName", List.Sum)
● Unpivot data: Table.UnpivotOtherColumns(table, {"ColumnName"},
"AttributeColumn", "ValueColumn")
● Transpose a table: Table.Transpose(table)
● Split a column by delimiter: Table.SplitColumn(table, "ColumnName",
Splitter.SplitTextByDelimiter(","), {"Column1", "Column2"})
● Merge columns: Table.CombineColumns(table, {"Column1", "Column2"},
Combiner.CombineTextByDelimiter(" "), "NewColumnName")
● Extract text before a delimiter: Table.TransformColumns(table,
{{"ColumnName", each Text.BeforeDelimiter(_, " "), type text}})
● Extract text after a delimiter: Table.TransformColumns(table,
{{"ColumnName", each Text.AfterDelimiter(_, " "), type text}})
● Extract text between delimiters: Table.TransformColumns(table,
{{"ColumnName", each Text.BetweenDelimiters(_, "{", "}"), type text}})
● Replace text: Table.TransformColumns(table, {{"ColumnName", each
Text.Replace(_, "old", "new"), type text}})
● Add a custom column with a formula: Table.AddColumn(table,
"NewColumnName", each [Column1] + [Column2], type number)
● Add an index column: Table.AddIndexColumn(table, "IndexColumn", 1, 1)
● Duplicate a column: Table.DuplicateColumn(table, "ColumnName",
"NewColumnName")
● Merge queries: Table.NestedJoin(table1, {"JoinColumn"}, table2,
{"JoinColumn"}, "NewColumnName", JoinKind.LeftOuter)

By: Waleed Mousa


● Append queries: Table.Combine({table1, table2})

4. Aggregation and Grouping

● Count rows: Table.RowCount(Table)


● Sum a column: List.Sum(Table[ColumnName])
● Average a column: List.Average(Table[ColumnName])
● Find the minimum value in a column: List.Min(Table[ColumnName])
● Find the maximum value in a column: List.Max(Table[ColumnName])
● Group by a column and count rows: Table.Group(Table, {"ColumnName"},
{{"Count", each Table.RowCount(_), type number}})
● Group by a column and sum values: Table.Group(Table, {"ColumnName"},
{{"Sum", each List.Sum([ColumnToSum]), type number}})
● Group by a column and average values: Table.Group(Table, {"ColumnName"},
{{"Average", each List.Average([ColumnToAverage]), type number}})
● Group by a column and find the minimum value: Table.Group(Table,
{"ColumnName"}, {{"Min", each List.Min([ColumnToMin]), type number}})
● Group by a column and find the maximum value: Table.Group(Table,
{"ColumnName"}, {{"Max", each List.Max([ColumnToMax]), type number}})

5. Filtering and Sorting

● Filter rows based on a condition: Table.SelectRows(Table, each


[ColumnName] > 10)
● Filter rows based on multiple conditions: Table.SelectRows(Table, each
[Column1] > 10 and [Column2] = "Value")
● Filter rows based on a list of values: Table.SelectRows(Table, each
List.Contains({"Value1", "Value2", "Value3"}, [ColumnName]))
● Filter rows based on a date range: Table.SelectRows(Table, each [Date] >=
#date(2022, 1, 1) and [Date] <= #date(2022, 12, 31))
● Sort a table by a column in ascending order: Table.Sort(Table,
{{"ColumnName", Order.Ascending}})
● Sort a table by a column in descending order: Table.Sort(Table,
{{"ColumnName", Order.Descending}})
● Sort a table by multiple columns: Table.Sort(Table, {{"Column1",
Order.Ascending}, {"Column2", Order.Descending}})

6. Joins and Merges

● Inner join two tables: Table.Join(Table1, "JoinColumn", Table2,


"JoinColumn", JoinKind.Inner)
● Left outer join two tables: Table.Join(Table1, "JoinColumn", Table2,
"JoinColumn", JoinKind.LeftOuter)
By: Waleed Mousa
● Right outer join two tables: Table.Join(Table1, "JoinColumn", Table2,
"JoinColumn", JoinKind.RightOuter)
● Full outer join two tables: Table.Join(Table1, "JoinColumn", Table2,
"JoinColumn", JoinKind.FullOuter)
● Cross join two tables: Table.CrossJoin(Table1, Table2)
● Merge queries: Table.NestedJoin(Table1, {"Key"}, Table2, {"ForeignKey"},
"NewColumn", JoinKind.LeftOuter)
● Append queries: Table.Combine({Table1, Table2})

7. Date and Time Transformations

● Extract year from a date column: Table.TransformColumns(table,


{{"DateColumn", Date.Year, Int64.Type}})
● Extract month from a date column: Table.TransformColumns(table,
{{"DateColumn", Date.Month, Int64.Type}})
● Extract day from a date column: Table.TransformColumns(table,
{{"DateColumn", Date.Day, Int64.Type}})
● Extract hour from a time column: Table.TransformColumns(table,
{{"TimeColumn", Time.Hour, Int64.Type}})
● Extract minute from a time column: Table.TransformColumns(table,
{{"TimeColumn", Time.Minute, Int64.Type}})
● Extract second from a time column: Table.TransformColumns(table,
{{"TimeColumn", Time.Second, Int64.Type}})
● Extract day of week from a date column: Table.TransformColumns(table,
{{"DateColumn", Date.DayOfWeek, Int64.Type}})
● Extract day of year from a date column: Table.TransformColumns(table,
{{"DateColumn", Date.DayOfYear, Int64.Type}})
● Extract quarter from a date column: Table.TransformColumns(table,
{{"DateColumn", Date.QuarterOfYear, Int64.Type}})
● Extract week of year from a date column: Table.TransformColumns(table,
{{"DateColumn", Date.WeekOfYear, Int64.Type}})
● Add a specific number of days to a date column:
Table.TransformColumns(table, {{"DateColumn", each Date.AddDays(_, 7),
type date}})
● Add a specific number of months to a date column:
Table.TransformColumns(table, {{"DateColumn", each Date.AddMonths(_, 3),
type date}})
● Add a specific number of years to a date column:
Table.TransformColumns(table, {{"DateColumn", each Date.AddYears(_, 1),
type date}})

By: Waleed Mousa


● Calculate the difference between two dates in days:
Table.AddColumn(table, "DaysDiff", each Duration.Days([EndDate] -
[StartDate]), type number)
● Calculate the difference between two times in hours:
Table.AddColumn(table, "HoursDiff", each Duration.Hours([EndTime] -
[StartTime]), type number)

8. Number Transformations

● Round numbers: Table.TransformColumns(table, {{"ColumnName", each


Number.Round(_, 2), type number}})
● Truncate numbers: Table.TransformColumns(table, {{"ColumnName", each
Number.Truncate(_), type number}})
● Ceiling numbers: Table.TransformColumns(table, {{"ColumnName", each
Number.Ceiling(_), type number}})
● Floor numbers: Table.TransformColumns(table, {{"ColumnName", each
Number.Floor(_), type number}})
● Absolute value of numbers: Table.TransformColumns(table, {{"ColumnName",
each Number.Abs(_), type number}})
● Negate numbers: Table.TransformColumns(table, {{"ColumnName", each
Number.Negate(_), type number}})
● Calculate the square root of numbers: Table.TransformColumns(table,
{{"ColumnName", each Number.Sqrt(_), type number}})
● Calculate the logarithm of numbers: Table.TransformColumns(table,
{{"ColumnName", each Number.Log(_), type number}})
● Calculate the exponential of numbers: Table.TransformColumns(table,
{{"ColumnName", each Number.Exp(_), type number}})
● Calculate the modulo of numbers: Table.TransformColumns(table,
{{"ColumnName", each Number.Mod(_, 5), type number}})
● Calculate the factorial of numbers: Table.TransformColumns(table,
{{"ColumnName", each Number.Factorial(_), type number}})

9. Text Transformations

● Concatenate text columns: Table.AddColumn(table, "ConcatenatedColumn",


each [Column1] & " " & [Column2], type text)
● Extract length of text: Table.TransformColumns(table, {{"ColumnName",
each Text.Length(_), type number}})
● Extract first N characters from text: Table.TransformColumns(table,
{{"ColumnName", each Text.Start(_, 5), type text}})

By: Waleed Mousa


● Extract last N characters from text: Table.TransformColumns(table,
{{"ColumnName", each Text.End(_, 5), type text}})
● Reverse text: Table.TransformColumns(table, {{"ColumnName", each
Text.Reverse(_), type text}})
● Pad text with leading characters: Table.TransformColumns(table,
{{"ColumnName", each Text.PadStart(_, 10, "0"), type text}})
● Pad text with trailing characters: Table.TransformColumns(table,
{{"ColumnName", each Text.PadEnd(_, 10, "0"), type text}})
● Remove leading whitespace: Table.TransformColumns(table, {{"ColumnName",
each Text.TrimStart(_), type text}})
● Remove trailing whitespace: Table.TransformColumns(table, {{"ColumnName",
each Text.TrimEnd(_), type text}})
● Remove all whitespace: Table.TransformColumns(table, {{"ColumnName", each
Text.Remove(_, " "), type text}})
● Replace text using a regular expression: Table.TransformColumns(table,
{{"ColumnName", each Text.Replace(_, "[a-z]", "X"), type text}})

10. Conditional Transformations

● Add a conditional column: Table.AddColumn(table, "ConditionalColumn",


each if [Column1] > 10 then "High" else "Low", type text)
● Filter rows based on a complex condition: Table.SelectRows(table, each if
[Column1] > 10 and [Column2] = "A" then true else false)
● Apply a conditional formatting rule: Table.TransformColumns(table,
{{"ColumnName", each if _ > 100 then "Green" else if _ > 50 then "Yellow"
else "Red", type text}})
● Pivot data based on a condition: Table.Pivot(table,
List.Distinct(table[PivotColumn]), "PivotColumn", "ValueColumn", each if
_ = null then 0 else _)
● Group rows based on a condition: Table.Group(table, {"GroupColumn"},
{{"Count", each Table.RowCount(_), type number}, {"Sum", each if
[ConditionColumn] = "A" then List.Sum([ValueColumn]) else 0, type
number}})

11. Advanced Transformations

● Create a custom function: (x) => x * 2


● Apply a custom function to a column: Table.TransformColumns(table,
{{"ColumnName", each MyCustomFunction(_), type number}})

By: Waleed Mousa


● Apply a custom function to multiple columns:
Table.TransformColumns(table, {{"Column1", each MyCustomFunction(_), type
number}, {"Column2", each MyCustomFunction(_), type number}})
● Create a parameterized query: (parameter) => let Source =
Csv.Document(File.Contents("C:\data.csv"), [Delimiter=","]), FilteredRows
= Table.SelectRows(Source, each [ColumnName] = parameter) in FilteredRows
● Invoke a parameterized query: MyParameterizedQuery("ParameterValue")
● Create a function to merge multiple files: (folder) => let Source =
Folder.Files(folder), CombinedData = Table.Combine(Source[Content]) in
CombinedData
● Create a function to unpivot multiple columns: (table, columnsToUnpivot)
=> let UnpivotedTable = Table.UnpivotOtherColumns(table,
columnsToUnpivot, "Attribute", "Value") in UnpivotedTable

12. Error Handling

● Replace errors with a specific value: Table.ReplaceErrorValues(table,


{{"ColumnName", "DefaultValue"}})
● Remove rows with errors: Table.RemoveRowsWithErrors(table)
● Handle errors using a try-otherwise expression:
Table.TransformColumns(table, {{"ColumnName", each try _ otherwise
"Error", type text}})
● Handle missing or null values: Table.TransformColumns(table,
{{"ColumnName", each if _ = null then 0 else _, type number}})

13. Data Profiling and Quality Assessment

● Count the number of rows: Table.RowCount(table)


● Count the number of distinct values in a column:
Table.RowCount(Table.Distinct(table[ColumnName]))
● Find the minimum value in a column: List.Min(table[ColumnName])
● Find the maximum value in a column: List.Max(table[ColumnName])
● Calculate the average of a column: List.Average(table[ColumnName])
● Calculate the sum of a column: List.Sum(table[ColumnName])
● Calculate the standard deviation of a column:
Table.StandardDeviation(table[ColumnName])
● Calculate the variance of a column: Table.Variance(table[ColumnName])
● Calculate the median of a column: Table.Median(table[ColumnName])
● Calculate the mode of a column: Table.Mode(table[ColumnName])
● Calculate the percentile of a column: Table.Percentile(table[ColumnName],
0.9)
By: Waleed Mousa

You might also like