Skip to content

Extract Nested Data

The Extract Nested Data component flattens nested data into rows. This is generally done by taking nested data in the form of key:value pairs (such as a JSON dictionary) and using those keys as column names.

The flow into this component should include a single variant-type column that is to be unpacked.


Properties

Name = string

A human-readable name for the component.


Include Input Columns = boolean

Choose whether to include input columns.


Columns = data structure

Define the variant's structure. Unique endpoints in this structure will be unpacked into columns. The variant from the input can usually be automatically detected, and its structure will be displayed to allow selection of elements.

To select an element, click its checkbox. To select every element in the variant, click Select all.

To edit an element, click the Actions button, . . . , next to the element, and then click Edit element. To add a new element, click the Actions button next to the VARIANT heading at the top of the structure, and then click Add element. Each element should be assigned a unique Key, a Type, and an Alias.

To delete an element, click the Actions button next to it and click Delete element.

To automatically select every element in the structure, click Autofill.

To remove all elements added to the structure so far, reverting to a blank structure, click Reset.

Click Save when you have finished editing and selecting elements.


Outer Join = boolean

If No (the default), any input rows that can't be expanded, either because they can't be accessed in the path or because they have zero fields or entries, are completely omitted from the output. If Yes, a single row is generated for expansions with zero rows.


Input Alias = string

If two properties have identical names, one will be given this prefix to differentiate them. More than two identically named properties will result in an error. The default is i, and this does not need changing in the vast majority of use cases.


Array Prefix = string

If two array structures have identical names, one will be given this prefix to differentiate them. More than two identically named structures will result in an error. The default is f, and this does not need changing in the vast majority of use cases.


Casting Method = drop-down

Select the casting method:

  • Fail on invalid data (default)
  • Replace all unparseable values with null
  • Replace unparseable dates and timestamps with null

Case Columns Alias Names = drop-down

Set the casing for alias columns names. Settings include Lower, No, or Upper. The default is No.

Name = string

A human-readable name for the component.


Columns = data structure

Define the variant's structure. Unique endpoints in this structure will be unpacked into columns. The variant from the input can usually be automatically detected, and its structure will be displayed to allow selection of elements.

To select an element, click its checkbox. To select every element in the variant, click Select all.

To edit an element, click the Actions button, . . . , next to the element, and then click Edit element. To add a new element, click the Actions button next to the VARIANT heading at the top of the structure, and then click Add element. Each element should be assigned a unique Key, a Type, and an Alias.

To delete an element, click the Actions button next to it and click Delete element.

To automatically select every element in the structure, click Autofill.

To remove all elements added to the structure so far, reverting to a blank structure, click Reset.

Click Save when you have finished editing and selecting elements.


Aliases = column editor

  • Source Column: Select the source column that you wish to provide an alternative name (an alias) for.
  • Target Column: Provide an alternative name for the column.

You can provide aliases for any or all input columns. This property is optional, however, and can be left empty if you don't wish to provide alternative names for any columns.


Include Input Columns = boolean

Choose whether to include input columns.


Outer Join = boolean

If No (the default), any input rows that can't be expanded, either because they can't be accessed in the path or because they have zero fields or entries, are completely omitted from the output. If Yes, a single row is generated for expansions with zero rows.


Snowflake Databricks Amazon Redshift (preview)