A wide array of record and set level transformations drive Centerprise’s data integration capabilities. They can be applied to data sets from any source in order to prepare them for delivery to reporting and analytics systems used in sales, human resources, finance and other key functions across the enterprise. Dataflow pipelines can encompass everything from simple operations that sort tables and transfer them to a destination to complex integration processes that involve multiple joins, aggregations, lookups and custom expressions.
Today, the volume and variety of data moving through these pipelines continues to grow with increasingly large transformations pushed through intermediary staging servers. As a result, performance has become a pressing concern for end-users .
Although Centerprise’s ETL engine deploys smart processing algorithms and a high degree of parallelism to optimize the scalability and speed of large data integration jobs, this three-step process can still act as a bottleneck to efficient data delivery.
In pushdown optimization mode, Centerprise leverages the native computing capacity of powerful RDBMS to perform all necessary operations in a dataflow. This setting allows Centerprise to load data directly into target databases from where it can be enriched and prepared in native SQL code using the intuitive drag-and-drop transformations already available on our platform. This optimized extract-load-transformation (ELT) process is ideally suited to large data sets.
In Astera Centerprise 8.0 we’ve added native SQL support for number of advanced transformations, thus allowing users to reap the full performance benefits of ELT in even the most complex dataflow pipelines. The cleansing, enrichment, and validation features offered in pushdown optimization mode include:
- Join Transformations
- Filter Transformations
- Sort Transformations
- Union Transformations
- Aggregate Transformations
- Distinct Transformations
- Database Table Lookups
- List Lookups
- Sequence Generator Transformations
- Route Transformations
- Switch Transformations
- Various Function Transformations
- Slowly Changing Dimensions
- Dimension Lookups
- DiffProcessor Database Write Strategy
- Over 40 different sources and destinations
For a comprehensive overview of these operations and how they work in-database, download the full document on pushdown supported transformations in Centerprise 8.0.