Class AbstractTableSyncTransformStep

java.lang.Object
com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.AbstractTransformStep
com.kingsrook.qqq.backend.core.processes.implementations.tablesync.AbstractTableSyncTransformStep
All Implemented Interfaces:
ProcessSummaryProviderInterface
Direct Known Subclasses:
AbstractRecordSyncToScheduledJobProcess, ScheduledReportSyncToScheduledJobProcess
public abstract class AbstractTableSyncTransformStep extends AbstractTransformStep
This class is for transforming records from a Source table to a Destination table. The Source table has a (unique/primary) key field: sourceTableKeyField, Which is matched against the Destination table's foreign-key: destinationTableForeignKeyField

  • Field Details

  • Constructor Details

    • AbstractTableSyncTransformStep

      public AbstractTableSyncTransformStep()

  • Method Details

    • getProcessSummary

      public ArrayList<ProcessSummaryLineInterface> getProcessSummary(RunBackendStepOutput runBackendStepOutput, boolean isForResultScreen)
      Description copied from interface: ProcessSummaryProviderInterface
      Note - object needs to be serializable, and List isn't... so, use ArrayList?

    • getErrorProcessSummaryLines

      protected ArrayList<ProcessSummaryLineInterface> getErrorProcessSummaryLines(RunBackendStepOutput runBackendStepOutput, boolean isForResultScreen)

    • populateRecordToStore

      public abstract QRecord populateRecordToStore(RunBackendStepInput runBackendStepInput, QRecord destinationRecord, QRecord sourceRecord) throws QException
      Map a record from the source table to the destination table. e.g., put values into the destinationRecord, from the sourceRecord. The destinationRecord will already be constructed, and will actually already be the record being updated, in the case of an update. It'll be empty (newly constructed) for an insert.
      Throws:
      QException

    • getLookupsToPreLoad

      protected List<Pair<String,String>> getLookupsToPreLoad()
      Specify a list of tableName/keyColumnName pairs to run through the preloadRecords method of the recordLookupHelper.

    • getExistingRecordQueryFilter

      protected QQueryFilter getExistingRecordQueryFilter(RunBackendStepInput runBackendStepInput, List<Serializable> sourceKeyList)
      Define the query filter to find existing records. e.g., for determining insert vs. update. Subclasses may override this to customize the behavior, e.g., in case an additional field is needed in the query.

    • getSyncProcessConfig

      protected abstract AbstractTableSyncTransformStep.SyncProcessConfig getSyncProcessConfig()
      Define the config for this process - e.g., what fields invalid input: '&' tables are used.

    • runOnePage

      public void runOnePage(RunBackendStepInput runBackendStepInput, RunBackendStepOutput runBackendStepOutput) throws QException
      Description copied from class: AbstractTransformStep
      Do the transform logic for one page of records
      Specified by:
      runOnePage in class AbstractTransformStep
      Throws:
      QException

    • logResults

      protected void logResults(RunBackendStepInput runBackendStepInput, AbstractTableSyncTransformStep.SyncProcessConfig syncProcessConfig)
      Log results of transformation

    • extractSourceKeyValueFromRecord

      protected Serializable extractSourceKeyValueFromRecord(QRecord sourceRecord, String sourceTableKeyField)
      Given a source record, extract what we'll use as its key from it. Normally this is just its sourceTableKeyField value - but - a subclass may do something more interesting, including, returning a java-record.

    • getExistingRecord

      protected QRecord getExistingRecord(Map<Pair<String,Serializable>,QRecord> existingRecordsByForeignKey, QFieldMetaData destinationForeignKeyField, Serializable sourceKeyValue)

    • getExistingRecordsByForeignKey

      protected Map<Pair<String,Serializable>,QRecord> getExistingRecordsByForeignKey(RunBackendStepInput runBackendStepInput, String destinationTableForeignKeyField, String destinationTableName, List<Serializable> sourceKeyList) throws QException
      Run the existingRecordQueryFilter - to look in the destinationTable for any records that may need an update (rather than an insert). Generally returns a Map, keyed by a Pair of the destinationTableForeignKeyField and the value in that field. But, for more complex use-cases, one can override the buildExistingRecordsMap method, to make different keys (e.g., if there are two possible destinationTableForeignKeyFields).
      Throws:
      QException

    • buildExistingRecordsMap

      protected Map<Pair<String,Serializable>,QRecord> buildExistingRecordsMap(String destinationTableForeignKeyField, List<QRecord> existingRecordList) throws QException
      Overridable point where you can, for example, keys in the existingRecordsMap with different fieldNames from the destinationTable. Note, if you're overriding this method, you'll likely also want invalid input: '&' need to override getExistingRecord.
      Throws:
      QException

    • getQueryJoins

      protected List<QueryJoin> getQueryJoins()

    • getAssociationNamesToInclude

      protected Collection<String> getAssociationNamesToInclude()

    • initializeRecordLookupHelper

      protected void initializeRecordLookupHelper(RunBackendStepInput runBackendStepInput, List<QRecord> sourceRecordList) throws QException
      If needed, init a record lookup helper for this process.
      Throws:
      QException

    • reinitializeRecordLookupHelper

      protected void reinitializeRecordLookupHelper(RunBackendStepInput runBackendStepInput, List<QRecord> sourceRecordList) throws QException
      for pages after the first, possibly load more records in the lookup helper.
      Throws:
      QException

    • addAuditForExecuteStep

      protected void addAuditForExecuteStep(AuditSingleInput auditSingleInput)
      Let the subclass "easily" add an audit to be inserted on the Execute step.