|
53 | 53 | " %} |
54 | 54 |
|
55 | 55 | {% set derived_columns_description = " |
56 | | - derived_columns::dictionary Defines values and datatypes for derived ('added' or 'calculated') columns. The values of this dictionary are the desired column names, |
57 | | - the value is another dictionary with the keys 'value' (holding a column name, a SQL expression, or a static string beginning with '!') and |
58 | | - 'datatype' (holding a valid SQL datatype for the target database). |
| 56 | + derived_columns::dictionary Defines values and datatypes for derived ('added' or 'calculated') columns. The key of each entry is the desired output column name. |
| 57 | + The value is a dictionary that supports the following keys: |
| 58 | + 'value' (required) A column name, a SQL expression, or a static string beginning with '!'. |
| 59 | + 'datatype' (required) A valid SQL datatype for the target database. |
| 60 | + 'src_cols_required' (optional) A column name or list of column names used by the expression. Required when include_source_columns=false. |
| 61 | + 'overwrite_src_cols' (optional, default false) When true, the columns listed in 'src_cols_required' are excluded from the staging CTE |
| 62 | + output, so only the aliased column appears. Useful for renaming columns whose original name cannot be referenced |
| 63 | + downstream (e.g. columns with special characters such as umlauts on Databricks). |
59 | 64 |
|
60 | 65 | Examples: |
61 | 66 | {'conversion_duration': {'value': 'TIMESTAMP_DIFF(conversion_date, created_date, DAY)', Creates three derived columns. The column 'conversion_duration' calculates |
|
64 | 69 | 'datatype': 'STRING'}, The column 'account_name' duplicates an already existing column and gives |
65 | 70 | 'account_name': {'value': 'name', it another name. More derived columns can be added as additional keys of |
66 | 71 | 'datatype': 'String'}} the dictionary. |
| 72 | +
|
| 73 | + {'account_name_clean': {'value': 'Kontonäme', Renames a source column whose name contains a special character. Setting |
| 74 | + 'datatype': 'STRING', 'overwrite_src_cols: true' ensures the original column is dropped from the |
| 75 | + 'src_cols_required': 'Kontonäme', CTE output, so only 'account_name_clean' appears and the special-character |
| 76 | + 'overwrite_src_cols': true}} column name does not propagate downstream. |
67 | 77 | " %} |
68 | 78 |
|
69 | 79 | {% set sequence_description = " |
|
0 commit comments