Class StreamedETLWithFrontendProcess

java.lang.Object

com.kingsrook.qqq.backend.core.processes.implementations.etl.streamedwithfrontend.StreamedETLWithFrontendProcess

public class StreamedETLWithFrontendProcess
extends Object

Definition for Streamed ETL process that includes a frontend. This process uses 3 backend steps, and 2 frontend steps, as follows: - preview (backend) - does just a little work (limited # of rows), to give the user a preview of what the final result will be - e.g., some data to seed the review screen - review (frontend) - a review screen - validate (backend) - optionally (per input on review screen), does like the preview step, but on all records from the extract step. - review (frontend) - a second view of the review screen, if the validate step was executed. - execute (backend) - processes all the rows, does all the work. - result (frontend) - a result screen The preview invalid input: '&' execute steps use additional BackendStep codes: - Extract - gets the rows to be processed. Used in preview (but only for a limited number of rows), and execute (without limit) - Transform - do whatever transformation is needed to the rows. Done on preview and execute. Always works with a "page" of records at a time. - Load - store the records into the backend, as appropriate. Always works with a "page" of records at a time. Only called by execute step.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	StreamedETLWithFrontendProcess.Builder

Field Summary

Fields

Modifier and Type	Field	Description
static final String	DEFAULT_PREVIEW_MESSAGE_FOR_DELETE
static final String	DEFAULT_PREVIEW_MESSAGE_FOR_INSERT
static final String	DEFAULT_PREVIEW_MESSAGE_FOR_INSERT_OR_UPDATE
static final String	DEFAULT_PREVIEW_MESSAGE_FOR_UPDATE
static final String	DEFAULT_PREVIEW_MESSAGE_PREFIX
static final String	FIELD_DEFAULT_QUERY_FILTER
static final String	FIELD_DESTINATION_TABLE
static final String	FIELD_DO_FULL_VALIDATION
static final String	FIELD_EXTRACT_CODE
static final String	FIELD_FETCH_HEAVY_FIELDS
static final String	FIELD_INCLUDE_ASSOCIATIONS
static final String	FIELD_LOAD_CODE
static final String	FIELD_PREVIEW_MESSAGE
static final String	FIELD_PROCESS_SUMMARY
static final String	FIELD_RECORD_COUNT
static final String	FIELD_SOURCE_TABLE
static final String	FIELD_SUPPORTS_FULL_VALIDATION
static final String	FIELD_TRANSACTION_LEVEL
static final String	FIELD_TRANSFORM_CODE
static final String	FIELD_VALIDATION_SUMMARY
static final String	STEP_NAME_EXECUTE
static final String	STEP_NAME_PREVIEW
static final String	STEP_NAME_RESULT
static final String	STEP_NAME_REVIEW
static final String	STEP_NAME_VALIDATE
static final String	TRANSACTION_LEVEL_AUTO_COMMIT
static final String	TRANSACTION_LEVEL_PAGE
static final String	TRANSACTION_LEVEL_PROCESS

Constructor Summary

Constructors

Constructor	Description
StreamedETLWithFrontendProcess()

Method Summary

Modifier and Type	Method	Description
static QProcessMetaData	defineProcessMetaData(Class<? extends AbstractExtractStep> extractStepClass, Class<? extends AbstractTransformStep> transformStepClass, Class<? extends AbstractLoadStep> loadStepClass, Map<String,Serializable> defaultFieldValues)
static QProcessMetaData	defineProcessMetaData(String sourceTableName, String destinationTableName, Class<? extends AbstractExtractStep> extractStepClass, Class<? extends AbstractTransformStep> transformStepClass, Class<? extends AbstractLoadStep> loadStepClass)
static StreamedETLWithFrontendProcess.Builder	processMetaDataBuilder()
static void	resetValidationFields(RunBackendStepInput runBackendStepInput, RunBackendStepOutput runBackendStepOutput)	useful for a process step to call upon 'back'

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

STEP_NAME_PREVIEW

public static final String STEP_NAME_PREVIEW

See Also:

• Constant Field Values

STEP_NAME_REVIEW

public static final String STEP_NAME_REVIEW

See Also:

• Constant Field Values

STEP_NAME_VALIDATE

public static final String STEP_NAME_VALIDATE

See Also:

• Constant Field Values

STEP_NAME_EXECUTE

public static final String STEP_NAME_EXECUTE

See Also:

• Constant Field Values

STEP_NAME_RESULT

public static final String STEP_NAME_RESULT

See Also:

• Constant Field Values

FIELD_EXTRACT_CODE

public static final String FIELD_EXTRACT_CODE

See Also:

• Constant Field Values

FIELD_TRANSFORM_CODE

public static final String FIELD_TRANSFORM_CODE

See Also:

• Constant Field Values

FIELD_LOAD_CODE

public static final String FIELD_LOAD_CODE

See Also:

• Constant Field Values

FIELD_SOURCE_TABLE

public static final String FIELD_SOURCE_TABLE

See Also:

• Constant Field Values

FIELD_DESTINATION_TABLE

public static final String FIELD_DESTINATION_TABLE

See Also:

• Constant Field Values

FIELD_RECORD_COUNT

public static final String FIELD_RECORD_COUNT

See Also:

• Constant Field Values

FIELD_DEFAULT_QUERY_FILTER

public static final String FIELD_DEFAULT_QUERY_FILTER

See Also:

• Constant Field Values

FIELD_FETCH_HEAVY_FIELDS

public static final String FIELD_FETCH_HEAVY_FIELDS

See Also:

• Constant Field Values

FIELD_INCLUDE_ASSOCIATIONS

public static final String FIELD_INCLUDE_ASSOCIATIONS

See Also:

• Constant Field Values

FIELD_SUPPORTS_FULL_VALIDATION

public static final String FIELD_SUPPORTS_FULL_VALIDATION

See Also:

• Constant Field Values

FIELD_DO_FULL_VALIDATION

public static final String FIELD_DO_FULL_VALIDATION

See Also:

• Constant Field Values

FIELD_VALIDATION_SUMMARY

public static final String FIELD_VALIDATION_SUMMARY

See Also:

• Constant Field Values

FIELD_PROCESS_SUMMARY

public static final String FIELD_PROCESS_SUMMARY

See Also:

• Constant Field Values

DEFAULT_PREVIEW_MESSAGE_FOR_INSERT

public static final String DEFAULT_PREVIEW_MESSAGE_FOR_INSERT

See Also:

• Constant Field Values

DEFAULT_PREVIEW_MESSAGE_FOR_UPDATE

public static final String DEFAULT_PREVIEW_MESSAGE_FOR_UPDATE

See Also:

• Constant Field Values

DEFAULT_PREVIEW_MESSAGE_FOR_INSERT_OR_UPDATE

public static final String DEFAULT_PREVIEW_MESSAGE_FOR_INSERT_OR_UPDATE

See Also:

• Constant Field Values

DEFAULT_PREVIEW_MESSAGE_FOR_DELETE

public static final String DEFAULT_PREVIEW_MESSAGE_FOR_DELETE

See Also:

• Constant Field Values

DEFAULT_PREVIEW_MESSAGE_PREFIX

public static final String DEFAULT_PREVIEW_MESSAGE_PREFIX

See Also:

• Constant Field Values

FIELD_PREVIEW_MESSAGE

public static final String FIELD_PREVIEW_MESSAGE

See Also:

• Constant Field Values

FIELD_TRANSACTION_LEVEL

public static final String FIELD_TRANSACTION_LEVEL

See Also:

• Constant Field Values

TRANSACTION_LEVEL_AUTO_COMMIT

public static final String TRANSACTION_LEVEL_AUTO_COMMIT

See Also:

• Constant Field Values

TRANSACTION_LEVEL_PAGE

public static final String TRANSACTION_LEVEL_PAGE

See Also:

• Constant Field Values

TRANSACTION_LEVEL_PROCESS

public static final String TRANSACTION_LEVEL_PROCESS

See Also:

• Constant Field Values

Constructor Details

StreamedETLWithFrontendProcess

public StreamedETLWithFrontendProcess()

Method Details

defineProcessMetaData

public static QProcessMetaData defineProcessMetaData(String sourceTableName,
 String destinationTableName,
 Class<? extends AbstractExtractStep> extractStepClass,
 Class<? extends AbstractTransformStep> transformStepClass,
 Class<? extends AbstractLoadStep> loadStepClass)

defineProcessMetaData

public static QProcessMetaData defineProcessMetaData(Class<? extends AbstractExtractStep> extractStepClass,
 Class<? extends AbstractTransformStep> transformStepClass,
 Class<? extends AbstractLoadStep> loadStepClass,
 Map<String,Serializable> defaultFieldValues)

Parameters:

defaultFieldValues - - expected to possibly contain values for the following field names: - FIELD_SOURCE_TABLE - FIELD_DESTINATION_TABLE - FIELD_SUPPORTS_FULL_VALIDATION - FIELD_DEFAULT_QUERY_FILTER - FIELD_DO_FULL_VALIDATION - FIELD_PREVIEW_MESSAGE

resetValidationFields

public static void resetValidationFields(RunBackendStepInput runBackendStepInput,
 RunBackendStepOutput runBackendStepOutput)

useful for a process step to call upon 'back'

processMetaDataBuilder

public static StreamedETLWithFrontendProcess.Builder processMetaDataBuilder()