Qordoba can work with MySQL, MariaDB, and other SQL databases to get data and can send it back using the JDBC connector. This integration can be hosted or remotely accessed from our platform.
Note: We recommend the hosted version
This integration executes the provided SQL select query. The query result will be converted to the Avro format. Streaming is used, so arbitrarily large result sets are supported.
This processor can be scheduled to run on a timer or can be triggered by incoming data.
In the tables below, you can view which parameters you can use to customize your configuration with this particular system:
'SQL select query' The SQL select query to execute. The query can be empty, a constant value, or built from attributes using Expression Language. If this property is specified, it will be used regardless of the content of incoming flowfiles. If this property is empty, the content of the incoming flow file is expected to contain a valid SQL select query, to be issued by the processor to the database. Note that Expression Language is not evaluated for flow file contents. 'Normalize Table/Column Names' Whether to change non-Avro-compatible characters in column names to Avro-compatible characters. For example, colons and periods will be changed to underscores in order to build a valid Avro record. #the default value is false 'Max Wait Time' The maximum amount of time allowed for a running SQL select query, zero means there is no limit. Max time less than 1 second will be equal to zero. #the default value is 0 seconds 'Database Connection Pooling Service' The Controller Service that is used to obtain connection to database
'JDBC Connection Pool' Specifies the JDBC Connection Pool to use in order to convert the JSON message to a SQL statement. The Connection Pool is necessary in order to determine the appropriate database column types. 'Support Fragmented Transactions' If true, when a FlowFile is consumed by this Processor, the Processor will first check the fragment.identifier and fragment.count attributes of that FlowFile. If the fragment.count value is greater than 1, the Processor will not process any FlowFile with that fragment.identifier until all are available; at that point, it will process all FlowFiles with that fragment.identifier as a single transaction, in the order specified by the FlowFiles fragment.index attributes. This Provides atomicity of those SQL statements. If this value is false, these attributes will be ignored and the updates will occur independent of one another. #the default value is true 'Transaction Timeout' If the <Support Fragmented Transactions> property is set to true, specifies how long to wait for all FlowFiles for a particular fragment.identifier attribute to arrive before just transferring all of the FlowFiles with that identifier to the 'failure' relationship 'Batch Size' The preferred number of FlowFiles to put to the database in a single transaction #the default value is 100 'Obtain Generated Keys' If true, any key that is automatically generated by the database will be added to the FlowFile that generated it using the sql.generate.key attribute. This may result in slightly slower performance and is not supported by all databases. #the default value is false