SQL
The SQL transformation component lets you write your own SQL SELECT statement. You can use this component without any input, or to transform data that it receives from other components in your pipeline. Designer automatically saves your SQL statement as you work. You can sample data and view variables while writing your SQL statement in the Query field.
To query a table from another component in the pipeline, your SELECT statement must use an expression that references the component by name. This expression has the form $T{<component name>}. For example, to select all from a table you have read with a Table Input component called "Table Input 1", use the following statement:
SELECT * FROM $T{Table Input 1}
If you need to write a non-SELECT SQL statement, the SQL Script orchestration component allows for custom non-SELECT statements where the output can be used as part of the orchestration flow.
Note
This component supports the use of pipeline and project variables. For more information, read Variables.
Use case
This component is extremely versatile and can be used to query and transform data using complex logic. For example, you can use it to:
- Aggregate data from multiple tables in your pipeline.
- Clean and standardize data before loading it into your data warehouse.
- Combine data from different sources using custom join and filter logic.
Properties
Name = string
A human-readable name for the component.
Query = code editor
The SQL query to execute.