Syncfusion Data Integration Platform Release Notes

Data Integration Platform

Features and Enhancement

UI changes and other features

  • Improved overall usability and stability of Data Integration Platform.
  • Data provenance view for list of selected processor groups.
  • Provided support to add or modify registry variables from web interface.
  • Provided support to view the execution results of processors workflow.
  • Provided support to empty queue for multiple selected connections.
  • Now supports running multiple versions of the same components.
  • Provided manual document for SSL and Kerberos support for Data Integration Platform.
  • Improved usability in user and policy management.

New or Improved Processors

  • Added UpdateRecord, PartitionRecord, LookupRecord, and PutElasticsearchHttpRecord processors.
  • Provided deep link to components in the flow and share those links with others.
  • New Record oriented abstraction for reading/writing schema aware event streams from CSV, JSON, AVRO, Grok, and plaintext with easy extension for other formats/schemas.
  • QueryRecord processor to execute SQL queries over a stream of records powered by Apache Calcite.
  • ConvertRecord processor to efficiently transform records from a given schema and format into another schema and format.
  • SplitRecord processor to efficiently split huge record bundles into configurable batch sizes for divide and conquer or protect downstream systems.
  • Processors to efficiently stream Records into and out of Apache Kafka in a format and schema aware manner and which automatically handle achieving high throughput and full provenance.
  • Controller Services for plugging into and managing data schemas (Avro Schema Registry, Hortonworks Schema Registry) that integrate nicely into the record readers and writers.
  • Features/improvements related to Change Data Capture (CDC), including CaptureChangeMySQL which reads from the MySQL bin logs, EnforceOrder, and PutDatabaseRecord processors, as well as a “Rollback on Failure” capability of some Put processors.
  • New processors and controller service to support a Wait/Notify pattern enabling conditions in another portion of a flow to signal another portion to continue or execute.
  • For those that like to write new capabilities on the fly using scripting languages you can build your own reporting tasks, record readers, and writers using various scripting languages now and the ExecuteScript processor lets you write in Clojure now too.
  • The JSON Jolt Transform processor now allows Jolt transforms to include NiFi expression language statements and is much faster.
  • New processors to compute and compare content using Fuzzy Hashing powerful for cyber security and other cases.
  • New processors to interact with Google Cloud Platform/Google Cloud Storage and Azure Blob and Table Storage.
  • Added ExtractCCDAAttributes processor to extract information from a Consolidated CDA formatted flow file.
  • New reporting tasks available to push flow status and bulletins out via SiteToSite protocol.
  • There are a 228 processors, 35 controller services, and 10 reporting tasks now available out the box. Check them out!