10000 Three utPLSQL tests fail when they run isolated / optimize suites to be executed · Issue #968 · utPLSQL/utPLSQL · GitHub
[go: up one dir, main page]
Skip to content
Three utPLSQL tests fail when they run isolated / optimize suites to be executed #968
New issue
New issue
Closed
#1001
Closed
Three utPLSQL tests fail when they run isolated / optimize suites to be executed#968
#1001
Labels
bug
@PhilippSalvisberg

Description

@PhilippSalvisberg
PhilippSalvisberg
opened on Jul 5, 2019

Describe the bug

When all tests are run via top suite selection, then all tests succeed. But the following three utPLSQL tests fail when they are run isolated (in a new connection):

  • at "UT3$USER#.TEST_COV_COBERTURA_REPORTER.REPORT_ON_FILE", line 41 ut.expect(l_actual).to_be_like(l_expected);
  • at "UT3$USER#.TEST_COVERAGE_SONAR_REPORTER.REPORT_ON_FILE", line 30 ut.expect(l_actual).to_equal(l_expected);
  • at "UT3$USER#.TEST_COVERALLS_REPORTER.REPORT_ON_FILE", line 36 ut.expect(l_actual).to_equal(l_expected);

Furthermore the execution time differs how tests are passt to ut_run. If just the top suites are passed then less code is executed (before, after, ...). If all tests are listed the the identical number of tests are executed, but the runtime is much slower (due to unnecessary multiple executions of before, after, ...). In this case it's a factor of 2.5. So, it's really significant. However, I consider this an ENHANCEMENT and not a BUG. But it is an optimization worth considering, since I expect that such scenarios happen more often when tools such the SQL Developer Extension are used to run tests chosen tests.

Provide version info


Error starting at line : 1 in command -
sql
Error report -
Unknown Command

19.0.0.0.0
19.0.0


PL/SQL procedure successfully completed.


UT_VERSION                                                  
------------------------------------------------------------
v3.1.8.3142-develop


BANNER                                                                           BANNER_FULL                                                                                                                                                      BANNER_LEGACY                                                                        CON_ID
-------------------------------------------------------------------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------- -------------------------------------------------------------------------------- ----------
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production           Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production                                                                                           Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production                    0
                                                                                 Version 19.2.0.0.0                                                                                                                                                                                                                                          



PARAMETER                      VALUE                                                           
------------------------------ ----------------------------------------------------------------
NLS_LANGUAGE                   ENGLISH                                                         
NLS_TERRITORY                  SWITZERLAND                                                     
NLS_CURRENCY                   SFr.                                                            
NLS_ISO_CURRENCY               SWITZERLAND                                                     
NLS_NUMERIC_CHARACTERS         .'                                                              
NLS_CALENDAR                   GREGORIAN                                                       
NLS_DATE_FORMAT                DD.MM.YYYY HH24:MI:SS                                           
NLS_DATE_LANGUAGE              ENGLISH                                                         
NLS_SORT                       BINARY                                                          
NLS_TIME_FORMAT                HH24:MI:SSXFF                                                   
NLS_TIMESTAMP_FORMAT           DD.MM.RR HH24:MI:SSXFF                                          

PARAMETER                      VALUE                                                           
------------------------------ ----------------------------------------------------------------
NLS_TIME_TZ_FORMAT             HH24:MI:SSXFF TZR                                               
NLS_TIMESTAMP_TZ_FORMAT        DD.MM.RR HH24:MI:SSXFF TZR                                      
NLS_DUAL_CURRENCY              SF                                                              
NLS_COMP                       BINARY                                                          
NLS_LENGTH_SEMANTICS           BYTE                                                            
NLS_NCHAR_CONV_EXCP            FALSE                                                           

17 rows selected. 


PORT_STRING                                                 
------------------------------------------------------------
x86_64/Linux 2.4.xx

Information about client software

irrelevant.

To Reproduce

  1. Connect as ut3_test_helper with the current develop version

  2. Run the following (no failures)

SET SERVEROUTPUT ON SIZE UNLIMITED
EXECUTE ut.run(ut_varchar2_list('UT3_TESTER', 'UT3$USER#'));
  1. Look at the output, should be similar to this (excerpt):
Finished in 61.538512 seconds
876 tests, 0 failed, 0 errored, 5 disabled, 0 warning(s)

  1. Connect as ut3_test_helper (again, fresh new session)

  2. Now we run all tests again, but this time we list all tests explicitly:

SET SERVEROUTPUT ON SIZE UNLIMITED
EXECUTE ut.run(ut_varchar2_list('UT3_TESTER.test_ut_utils.test_trim_list_elements', 'UT3_TESTER.test_ut_utils.trim_list_elemts_null_collect', 'UT3_TESTER.test_ut_utils.trim_list_elemts_empty_collect', 'UT3_TESTER.test_ut_utils.to_string_emptyblob', 'UT3_TESTER.test_ut_utils.to_string_emptyclob', 'UT3_TESTER.test_ut_utils.to_string_nullblob', 'UT3_TESTER.test_ut_utils.to_string_nullclob', 'UT3_TESTER.test_ut_utils.to_string_nulldate', 'UT3_TESTER.test_ut_utils.to_string_nullnumber', 'UT3_TESTER.test_ut_utils.to_string_nulltimestamp', 'UT3_TESTER.test_ut_utils.to_string_nulltimestamp_ltz', 'UT3_TESTER.test_ut_utils.to_string_nulltimestamp_tz', 'UT3_TESTER.test_ut_utils.to_string_nullvarchar2', 'UT3_TESTER.test_ut_utils.to_string_blob', 'UT3_TESTER.test_ut_utils.to_string_clob', 'UT3_TESTER.test_ut_utils.to_string_date', 'UT3_TESTER.test_ut_utils.to_string_timestamp', 'UT3_TESTER.test_ut_utils.to_string_timestamp_ltz', 'UT3_TESTER.test_ut_utils.to_string_timestamp_tz', 'UT3_TESTER.test_ut_utils.to_string_varchar2', 'UT3_TESTER.test_ut_utils.to_string_verybigblob', 'UT3_TESTER.test_ut_utils.to_string_verybigclob', 'UT3_TESTER.test_ut_utils.to_string_verybignumber', 'UT3_TESTER.test_ut_utils.to_string_verybigvarchar2', 'UT3_TESTER.test_ut_utils.to_string_verysmallnumber', 'UT3_TESTER.test_ut_utils.test_filter_list', 'UT3_TESTER.test_ut_utils.filter_list_null_collection', 'UT3_TESTER.test_ut_utils.filter_list_empty_collection', 'UT3_TESTER.test_ut_utils.test_clob_to_table', 'UT3_TESTER.test_ut_utils.test_test_result_to_char', 'UT3_TESTER.test_ut_utils.test_table_to_clob', 'UT3_TESTER.test_ut_utils.test_append_with_multibyte', 'UT3_TESTER.test_ut_utils.test_clob_to_table_multibyte', 'UT3_TESTER.test_ut_utils.test_to_version_split', 'UT3_TESTER.test_ut_utils.replace_multiline_comments', 'UT3_TESTER.test_ut_test.after_each_executed', 'UT3_TESTER.test_ut_test.after_each_proc_name_invalid', 'UT3_TESTER.test_ut_test.after_each_procedure_name_null', 'UT3_TESTER.test_ut_test.application_info_on_execution', 'UT3_TESTER.test_ut_test.before_each_executed', 'UT3_TESTER.test_ut_test.before_each_proc_name_invalid', 'UT3_TESTER.test_ut_test.before_each_proc_name_null', 'UT3_TESTER.test_ut_test.ignore_savepoint_exception', 'UT3_TESTER.test_ut_test.owner_name_invalid', 'UT3_TESTER.test_ut_test.owner_name_null', 'UT3_TESTER.test_ut_test.package_in_invalid_state', 'UT3_TESTER.test_ut_test.package_name_invalid', 'UT3_TESTER.test_ut_test.package_name_null', 'UT3_TESTER.test_ut_test.procedure_name_invalid', 'UT3_TESTER.test_ut_test.procedure_name_null', 'UT3_TESTER.test_ut_test.before_test_executed', 'UT3_TESTER.test_ut_test.before_test_proc_name_invalid', 'UT3_TESTER.test_ut_test.before_test_proc_name_null', 'UT3_TESTER.test_ut_test.after_test_executed', 'UT3_TESTER.test_ut_test.after_test_proce_name_invalid', 'UT3_TESTER.test_ut_test.after_test_proc_name_null', 'UT3_TESTER.test_ut_test.test_output_gathering', 'UT3_TESTER.test_ut_test.disabled_test', 'UT3_TESTER.test_ut_test.aftertest_errors', 'UT3_TESTER.test_ut_test.aftereach_errors', 'UT3_TESTER.test_ut_test.beforetest_errors', 'UT3_TESTER.test_ut_test.beforeeach_errors', 'UT3_TESTER.test_ut_suite.rollback_auto', 'UT3_TESTER.test_ut_suite.rollback_auto_on_failure', 'UT3_TESTER.test_ut_suite.rollback_manual', 'UT3_TESTER.test_ut_suite.rollback_manual_on_failure', 'UT3_TESTER.test_ut_suite.disabled_suite', 'UT3_TESTER.test_ut_suite.beforeall_errors', 'UT3_TESTER.test_ut_suite.aftereall_errors', 'UT3_TESTER.test_ut_suite.package_without_body', 'UT3_TESTER.test_ut_suite.package_with_invalid_body', 'UT3_TESTER.test_ut_suite.trim_transaction_invalidators', 'UT3_TESTER.test_ut_executable.form_name', 'UT3_TESTER.test_ut_executable.exec_schema_package_proc', 'UT3_TESTER.test_ut_executable.exec_package_proc_output', 'UT3_TESTER.test_ut_executable.exec_failing_proc', 'UT3_TESTER.test_suite_manager.test_get_schema_ut_packages', 'UT3_TESTER.test_suite_manager.test_schema_run', 'UT3_TESTER.test_suite_manager.test_top2_by_name', 'UT3_TESTER.test_suite_manager.test_top2_bt_name_cur_user', 'UT3_TESTER.test_suite_manager.test_by_path_to_subsuite', 'UT3_TESTER.test_suite_manager.test_by_path_to_subsuite_cu', 'UT3_TESTER.test_suite_manager.test_subsute_proc_by_path', 'UT3_TESTER.test_suite_manager.test_subsute_proc_by_path_cu', 'UT3_TESTER.test_suite_manager.test_top_pack_by_name', 'UT3_TESTER.test_suite_manager.test_top_pack_by_name_cu', 'UT3_TESTER.test_suite_manager.test_top_pack_by_path', 'UT3_TESTER.test_suite_manager.test_top_pack_by_path_cu', 'UT3_TESTER.test_suite_manager.test_top_pck_proc_by_path', 'UT3_TESTER.test_suite_manager.test_top_pck_proc_by_path_cu', 'UT3_TESTER.test_suite_manager.test_top_pkc_proc_by_name', 'UT3_TESTER.test_suite_manager.test_top_pkc_proc_by_name_cu', 'UT3_TESTER.test_suite_manager.test_top_pkc_nosub_by_name', 'UT3_TESTER.test_suite_manager.test_top_pkc_nosub_by_name_cu', 'UT3_TESTER.test_suite_manager.test_top_subpck_by_path', 'UT3_TESTER.test_suite_manager.test_top_subpck_by_path_cu', 'UT3_TESTER.test_suite_manager.test_search_invalid_pck', 'UT3_TESTER.test_suite_manager.test_search_nonexisting_pck', 'UT3_TESTER.test_suite_manager.test_search_nonexist_sch_pck', 'UT3_TESTER.test_suite_manager.test_desc_with_comma', 'UT3_TESTER.test_suite_manager.test_inv_cache_on_drop', 'UT3_TESTER.test_suite_manager.test_inv_pck_bodies', 'UT3_TESTER.test_suite_manager.test_pck_with_dollar', 'UT3_TESTER.test_suite_manager.test_pck_with_hash', 'UT3_TESTER.test_suite_manager.test_test_with_dollar', 'UT3_TESTER.test_suite_manager.test_test_with_hash', 'UT3_TESTER.test_suite_manager.test_empty_suite_path', 'UT3_TESTER.test_suite_manager.test_pck_with_same_path', 'UT3_TESTER.test_suite_manager.disable_suite_floating_annot', 'UT3_TESTER.test_suite_manager.pck_proc_in_ctx_by_name', 'UT3_TESTER.test_suite_manager.pck_proc_in_ctx_by_path', 'UT3_TESTER.test_suite_manager.add_new_long_test_package', 'UT3_TESTER.test_suite_builder.test_bad_procedure_annotation', 'UT3_TESTER.test_suite_builder.test_bad_package_annotation', 'UT3_TESTER.test_suite_builder.throws_value_empty', 'UT3_TESTER.test_suite_builder.throws_value_invalid', 'UT3_TESTER.test_suite_builder.test_annotation', 'UT3_TESTER.test_suite_builder.test_annot_duplicated', 'UT3_TESTER.test_suite_builder.test_annotation_ordering', 'UT3_TESTER.test_suite_builder.test_tag_annotation', 'UT3_TESTER.test_suite_builder.suite_tag_annotation', 'UT3_TESTER.test_suite_builder.test_tags_annotation', 'UT3_TESTER.test_suite_builder.suite_tags_annotation', 'UT3_TESTER.test_suite_builder.test_2line_tags_annotation', 'UT3_TESTER.test_suite_builder.suite_2line_tags_annotation', 'UT3_TESTER.test_suite_builder.test_empty_tag', 'UT3_TESTER.test_suite_builder.suite_empty_tag', 'UT3_TESTER.test_suite_builder.test_duplicate_tag', 'UT3_TESTER.test_suite_builder.suite_duplicate_tag', 'UT3_TESTER.test_suite_builder.test_empty_tag_between', 'UT3_TESTER.test_suite_builder.suite_empty_tag_between', 'UT3_TESTER.test_suite_builder.test_special_char_tag', 'UT3_TESTER.test_suite_builder.suite_special_char_tag', 'UT3_TESTER.test_suite_builder.suitepath_from_non_empty_path', 'UT3_TESTER.test_suite_builder.suitepath_annot_duplicated', 'UT3_TESTER.test_suite_builder.suitepath_annot_empty', 'UT3_TESTER.test_suite_builder.suitepath_annot_invalid_path', 'UT3_TESTER.test_suite_builder.no_suite_description', 'UT3_TESTER.test_suite_builder.suite_description_from_suite', 'UT3_TESTER.test_suite_builder.suite_annot_duplicated', 'UT3_TESTER.test_suite_builder.rollback_type_valid', 'UT3_TESTER.test_suite_builder.rollback_type_duplicated', 'UT3_TESTER.test_suite_builder.rollback_type_empty', 'UT3_TESTER.test_suite_builder.rollback_type_invalid', 'UT3_TESTER.test_suite_builder.suite_descr_from_displayname', 'UT3_TESTER.test_suite_builder.displayname_annot_duplicated', 'UT3_TESTER.test_suite_builder.displayname_annot_empty', 'UT3_TESTER.test_suite_builder.suite_from_context', 'UT3_TESTER.test_suite_builder.before_after_in_context', 'UT3_TESTER.test_suite_builder.before_after_out_of_context', 'UT3_TESTER.test_suite_builder.context_without_endcontext', 'UT3_TESTER.test_suite_builder.endcontext_without_context', 'UT3_TESTER.test_suite_builder.duplicate_context_name', 'UT3_TESTER.test_suite_builder.before_aftertest_multi', 'UT3_TESTER.test_suite_builder.before_aftertest_twice', 'UT3_TESTER.test_suite_builder.before_aftertest_pkg_proc', 'UT3_TESTER.test_suite_builder.before_aftertest_mixed_syntax', 'UT3_TESTER.test_suite_builder.multiple_before_after', 'UT3_TESTER.test_suite_builder.multiple_standalone_bef_aft', 'UT3_TESTER.test_suite_builder.before_after_on_single_proc', 'UT3_TESTER.test_suite_builder.multiple_mixed_bef_aft', 'UT3_TESTER.test_suite_builder.beforeall_annot_duplicated', 'UT3_TESTER.test_suite_builder.beforeeach_annot_duplicated', 'UT3_TESTER.test_suite_builder.afterall_annot_duplicated', 'UT3_TESTER.test_suite_builder.aftereach_annot_duplicated', 'UT3_TESTER.test_suite_builder.before_after_mixed_with_test', 'UT3_TESTER.test_output_buffer.test_receive', 'UT3_TESTER.test_output_buffer.test_doesnt_send_on_null_text', 'UT3_TESTER.test_output_buffer.test_doesnt_send_on_null_elem', 'UT3_TESTER.test_output_buffer.test_send_line', 'UT3_TESTER.test_output_buffer.test_waiting_for_data', 'UT3_TESTER.test_output_buffer.test_purge_text_buffer', 'UT3_TESTER.test_output_buffer.test_purge_clob_buffer', 'UT3_TESTER.test_file_mapper.default_mappings', 'UT3_TESTER.test_file_mapper.specific_owner', 'UT3_TESTER.test_expectation_processor.who_called_expectation', 'UT3_TESTER.test_expectation_processor.who_called_expectation_0x', 'UT3_TESTER.test_before_after_annotations.beforetest_local_procedure', 'UT3_TESTER.test_before_after_annotations.beforetest_local_proc_with_pkg', 'UT3_TESTER.test_before_after_annotations.beforetest_twice', 'UT3_TESTER.test_before_after_annotations.beforetest_one_ext_procedure', 'UT3_TESTER.test_before_after_annotations.beforetest_multi_ext_procedure', 'UT3_TESTER.test_before_after_annotations.beforetest_missing_procedure', 'UT3_TESTER.test_before_after_annotations.beforetest_one_err_procedure', 'UT3_TESTER.test_annotation_parser.test_proc_comments', 'UT3_TESTER.test_annotation_parser.include_floating_annotations', 'UT3_TESTER.test_annotation_parser.parse_complex_with_functions', 'UT3_TESTER.test_annotation_parser.no_procedure_annotation', 'UT3_TESTER.test_annotation_parser.parse_accessible_by', 'UT3_TESTER.test_annotation_parser.complex_package_declaration', 'UT3_TESTER.test_annotation_parser.complex_text', 'UT3_TESTER.test_annotation_parser.ignore_annotations_in_comments', 'UT3_TESTER.test_annotation_parser.ignore_wrapped_package', 'UT3_TESTER.test_annotation_parser.brackets_in_desc', 'UT3_TESTER.test_annotation_parser.test_space_before_annot_params', 'UT3_TESTER.test_annotation_parser.test_windows_newline', 'UT3_TESTER.test_annotation_parser.test_annot_very_long_name', 'UT3_TESTER.test_annotation_parser.test_upper_annot', 'UT3_TESTER.test_annotation_manager.add_new_package', 'UT3_TESTER.test_annotation_manager.update_modified_package', 'UT3_TESTER.test_annotation_manager.add_new_test_package', 'UT3_TESTER.test_annotation_manager.update_modified_test_package', 'UT3_TESTER.test_annotation_manager.keep_dropped_data_in_cache', 'UT3_TESTER.test_annotation_manager.no_data_for_dropped_object', 'UT3_TESTER.test_annotation_manager.cleanup_dropped_data_in_cache', 'UT3_TESTER.test_annotation_manager.trg_skip_existing_package', 'UT3_TESTER.test_annotation_manager.trg_add_existing_on_compile', 'UT3_TESTER.test_annotation_manager.trg_add_existing_on_refresh', 'UT3_TESTER.test_annotation_manager.trg_not_add_new_package', 'UT3_TESTER.test_annotation_manager.trg_add_new_test_package', 'UT3_TESTER.test_annotation_manager.trg_no_data_for_dropped_object', 'UT3_TESTER.test_annotation_manager.trg_update_modified_package', 'UT3_TESTER.test_annot_throws_exception.throws_same_annotated_except', 'UT3_TESTER.test_annot_throws_exception.throws_one_of_annotated_excpt', 'UT3_TESTER.test_annot_throws_exception.throws_with_leading_zero', 'UT3_TESTER.test_annot_throws_exception.throws_diff_annotated_except', 'UT3_TESTER.test_annot_throws_exception.throws_empty', 'UT3_TESTER.test_annot_throws_exception.bad_paramters_with_except', 'UT3_TESTER.test_annot_throws_exception.bad_paramters_without_except', 'UT3_TESTER.test_annot_throws_exception.one_valid_exception_number', 'UT3_TESTER.test_annot_throws_exception.nothing_thrown', 'UT3_TESTER.test_annot_throws_exception.single_exc_const_pkg', 'UT3_TESTER.test_annot_throws_exception.list_of_exc_constant', 'UT3_TESTER.test_annot_throws_exception.fail_not_match_exc', 'UT3_TESTER.test_annot_throws_exception.mixed_exc_list', 'UT3_TESTER.test_annot_throws_exception.mixed_list_notexi', 'UT3_TESTER.test_annot_throws_exception.named_exc_pragma', 'UT3_TESTER.test_annot_throws_exception.named_exc_ora', 'UT3_TESTER.test_annot_throws_exception.named_exc_ora_dup_ind', 'UT3_TESTER.test_annot_throws_exception.nodata_exc_ora', 'UT3_TESTER.test_annot_throws_exception.defined_varchar_exc', 'UT3_TESTER.test_annot_throws_exception.non_existing_const', 'UT3_TESTER.test_annot_throws_exception.bad_exc_const', 'UT3$USER#.test_tfs_junit_reporter.escapes_special_chars', 'UT3$USER#.test_tfs_junit_reporter.reports_only_failed_or_errored', 'UT3$USER#.test_tfs_junit_reporter.check_classname_suite', 'UT3$USER#.test_tfs_junit_reporter.check_classname_suitepath', 'UT3$USER#.test_tfs_junit_reporter.check_flatten_nested_suites', 'UT3$USER#.test_tfs_junit_reporter.check_nls_number_formatting', 'UT3$USER#.test_tfs_junit_reporter.check_failure_escaped', 'UT3$USER#.test_tfs_junit_reporter.check_encoding_included', 'UT3$USER#.test_teamcity_reporter.report_produces_expected_out', 'UT3$USER#.test_teamcity_reporter.escape_special_chars', 'UT3$USER#.test_teamcity_reporter.trims_long_output', 'UT3$USER#.test_sonar_test_reporter.report_produces_expected_out', 'UT3$USER#.test_sonar_test_reporter.check_encoding_included', 'UT3$USER#.test_sonar_test_reporter.check_failure_escaped', 'UT3$USER#.test_realtime_reporter.xml_report_structure', 'UT3$USER#.test_realtime_reporter.total_number_of_tests', 'UT3$USER#.test_realtime_reporter.pre_run_composite_nodes', 'UT3$USER#.test_realtime_reporter.execution_time_of_run', 'UT3$USER#.test_realtime_reporter.escaped_characters', 'UT3$USER#.test_realtime_reporter.pre_test_nodes', 'UT3$USER#.test_realtime_reporter.post_test_nodes', 'UT3$USER#.test_realtime_reporter.single_failed_message', 'UT3$USER#.test_realtime_reporter.multiple_failed_messages', 'UT3$USER#.test_realtime_reporter.serveroutput_of_test', 'UT3$USER#.test_realtime_reporter.serveroutput_of_testsuite', 'UT3$USER#.test_realtime_reporter.error_stack_of_test', 'UT3$USER#.test_realtime_reporter.error_stack_of_testsuite', 'UT3$USER#.test_realtime_reporter.warnings_of_test', 'UT3$USER#.test_realtime_reporter.warnings_of_testsuite', 'UT3$USER#.test_realtime_reporter.get_description', 'UT3$USER#.test_junit_reporter.escapes_special_chars', 'UT3$USER#.test_junit_reporter.reports_only_failed_or_errored', 'UT3$USER#.test_junit_reporter.reports_xunit_only_fail_or_err', 'UT3$USER#.test_junit_reporter.check_classname_suite', 'UT3$USER#.test_junit_reporter.check_classname_suitepath', 'UT3$USER#.test_junit_reporter.check_nls_number_formatting', 'UT3$USER#.test_junit_reporter.report_test_without_desc', 'UT3$USER#.test_junit_reporter.report_suite_without_desc', 'UT3$USER#.test_junit_reporter.reporort_produces_expected_out', 'UT3$USER#.test_junit_reporter.check_classname_is_populated', 'UT3$USER#.test_junit_reporter.check_failure_escaped', 'UT3$USER#.test_junit_reporter.check_encoding_included', 'UT3$USER#.test_html_extended_reporter.report_on_file', 'UT3$USER#.test_extended_coverage.coverage_for_object', 'UT3$USER#.test_extended_coverage.coverage_for_schema', 'UT3$USER#.test_extended_coverage.coverage_for_file', 'UT3$USER#.test_documentation_reporter.report_produces_expected_out', 'UT3$USER#.test_debug_reporter.includes_event_info', 'UT3$USER#.test_debug_reporter.includes_run_info', 'UT3$USER#.test_coveralls_reporter.report_on_file', 'UT3$USER#.test_coveralls_reporter.report_zero_coverage', 'UT3$USER#.test_coverage_sonar_reporter.report_on_file', 'UT3$USER#.test_coverage_sonar_reporter.check_encoding_included', 'UT3$USER#.test_cov_cobertura_reporter.report_on_file', 'UT3$USER#.test_cov_cobertura_reporter.report_zero_coverage', 'UT3$USER#.test_expect_to_have_count.success_have_count_cursor', 'UT3$USER#.test_expect_to_have_count.fail_have_count_cursor', 'UT3$USER#.test_expect_to_have_count.fail_have_count_cursor_report', 'UT3$USER#.test_expect_to_have_count.success_not_have_count_cursor', 'UT3$USER#.test_expect_to_have_count.fail_not_have_count_cursor', 'UT3$USER#.test_expect_to_have_count.success_have_count_collection', 'UT3$USER#.test_expect_to_have_count.fail_have_count_collection', 'UT3$USER#.test_expect_to_have_count.success_not_have_count_coll', 'UT3$USER#.test_expect_to_have_count.fail_not_have_count_coll', 'UT3$USER#.test_expect_to_have_count.fail_have_count_null_coll', 'UT3$USER#.test_expect_to_have_count.fail_not_have_count_null_coll', 'UT3$USER#.test_expect_to_have_count.fail_have_count_object', 'UT3$USER#.test_expect_to_have_count.fail_have_count_null_object', 'UT3$USER#.test_expect_to_have_count.fail_have_count_number', 'UT3$USER#.test_expect_to_have_count.fail_not_have_count_object', 'UT3$USER#.test_expect_to_have_count.fail_not_have_count_null_obj', 'UT3$USER#.test_expect_to_have_count.fail_not_have_count_number', 'UT3$USER#.test_expect_to_be_true_false.to_be_true_null_boolean', 'UT3$USER#.test_expect_to_be_true_false.to_be_true_success', 'UT3$USER#.test_expect_to_be_true_false.to_be_true_failure', 'UT3$USER#.test_expect_to_be_true_false.to_be_true_bad_type', 'UT3$USER#.test_expect_to_be_true_false.not_to_be_true_null_boolean', 'UT3$USER#.test_expect_to_be_true_false.not_to_be_true_success', 'UT3$USER#.test_expect_to_be_true_false.not_to_be_true_failure', 'UT3$USER#.test_expect_to_be_true_false.not_to_be_true_bad_type', 'UT3$USER#.test_expect_to_be_true_false.to_be_false_null_boolean', 'UT3$USER#.test_expect_to_be_true_false.to_be_false_success', 'UT3$USER#.test_expect_to_be_true_false.to_be_false_failure', 'UT3$USER#.test_expect_to_be_true_false.to_be_false_bad_type', 'UT3$USER#.test_expect_to_be_true_false.not_to_be_false_null_boolean', 'UT3$USER#.test_expect_to_be_true_false.not_to_be_false_success', 'UT3$USER#.test_expect_to_be_true_false.not_to_be_false_failure', 'UT3$USER#.test_expect_to_be_true_false.not_to_be_false_bad_type', 'UT3$USER#.test_expect_to_be_null.null_blob', 'UT3$USER#.test_expect_to_be_null.null_boolean', 'UT3$USER#.test_expect_to_be_null.null_clob', 'UT3$USER#.test_expect_to_be_null.null_date', 'UT3$USER#.test_expect_to_be_null.null_number', 'UT3$USER#.test_expect_to_be_null.null_timestamp', 'UT3$USER#.test_expect_to_be_null.null_timestamp_with_ltz', 'UT3$USER#.test_expect_to_be_null.null_timestamp_with_tz', 'UT3$USER#.test_expect_to_be_null.null_varchar2', 'UT3$USER#.test_expect_to_be_null.null_anydata', 'UT3$USER#.test_expect_to_be_null.uninit_object_in_anydata', 'UT3$USER#.test_expect_to_be_null.uninit_nested_table_in_anydata', 'UT3$USER#.test_expect_to_be_null.uninit_varray_in_anydata', 'UT3$USER#.test_expect_to_be_null.blob_not_null', 'UT3$USER#.test_expect_to_be_null.empty_blob', 'UT3$USER#.test_expect_to_be_null.boolean_not_null', 'UT3$USER#.test_expect_to_be_null.clob_not_null', 'UT3$USER#.test_expect_to_be_null.empty_clob', 'UT3$USER#.test_expect_to_be_null.date_not_null', 'UT3$USER#.test_expect_to_be_null.number_not_null', 'UT3$USER#.test_expect_to_be_null.timestamp_not_null', 'UT3$USER#.test_expect_to_be_null.timestamp_with_ltz_not_null', 'UT3$USER#.test_expect_to_be_null.timestamp_with_tz_not_null', 'UT3$USER#.test_expect_to_be_null.varchar2_not_null', 'UT3$USER#.test_expect_to_be_null.initialized_object', 'UT3$USER#.test_expect_to_be_null.initialized_nested_table', 'UT3$USER#.test_expect_to_be_null.initialized_varray', 'UT3$USER#.test_expect_to_be_not_null.blob_not_null', 'UT3$USER#.test_expect_to_be_not_null.empty_blob', 'UT3$USER#.test_expect_to_be_not_null.boolean_not_null', 'UT3$USER#.test_expect_to_be_not_null.clob_not_null', 'UT3$USER#.test_expect_to_be_not_null.empty_clob', 'UT3$USER#.test_expect_to_be_not_null.date_not_null', 'UT3$USER#.test_expect_to_be_not_null.number_not_null', 'UT3$USER#.test_expect_to_be_not_null.timestamp_not_null', 'UT3$USER#.test_expect_to_be_not_null.timestamp_with_ltz_not_null', 'UT3$USER#.test_expect_to_be_not_null.timestamp_with_tz_not_null', 'UT3$USER#.test_expect_to_be_not_null.varchar2_not_null', 'UT3$USER#.test_expect_to_be_not_null.initialized_object', 'UT3$USER#.test_expect_to_be_not_null.initialized_nested_table', 'UT3$USER#.test_expect_to_be_not_null.initialized_varray', 'UT3$USER#.test_expect_to_be_not_null.null_blob', 'UT3$USER#.test_expect_to_be_not_null.null_boolean', 'UT3$USER#.test_expect_to_be_not_null.null_clob', 'UT3$USER#.test_expect_to_be_not_null.null_date', 'UT3$USER#.test_expect_to_be_not_null.null_number', 'UT3$USER#.test_expect_to_be_not_null.null_timestamp', 'UT3$USER#.test_expect_to_be_not_null.null_timestamp_with_ltz', 'UT3$USER#.test_expect_to_be_not_null.null_timestamp_with_tz', 'UT3$USER#.test_expect_to_be_not_null.null_varchar2', 'UT3$USER#.test_expect_to_be_not_null.null_anydata', 'UT3$USER#.test_expect_to_be_not_null.uninit_object_in_anydata', 'UT3$USER#.test_expect_to_be_not_null.uninit_nested_table_in_anydata', 'UT3$USER#.test_expect_to_be_not_null.uninit_varray_in_anydata', 'UT3$USER#.test_expect_to_be_empty.success_be_empty_cursor', 'UT3$USER#.test_expect_to_be_empty.fail_be_empty_cursor', 'UT3$USER#.test_expect_to_be_empty.fail_be_empty_cursor_report', 'UT3$USER#.test_expect_to_be_empty.success_be_empty_collection', 'UT3$USER#.test_expect_to_be_empty.fail_be_empty_collection', 'UT3$USER#.test_expect_to_be_empty.fail_be_empty_null_collection', 'UT3$USER#.test_expect_to_be_empty.fail_be_empty_object', 'UT3$USER#.test_expect_to_be_empty.fail_be_empty_null_object', 'UT3$USER#.test_expect_to_be_empty.fail_be_empty_number', 'UT3$USER#.test_expect_to_be_empty.success_be_empty_clob', 'UT3$USER#.test_expect_to_be_empty.fail_be_empty_clob', 'UT3$USER#.test_expect_to_be_empty.success_be_empty_blob', 'UT3$USER#.test_expect_to_be_empty.fail_be_empty_blob', 'UT3$USER#.test_expect_to_be_empty.fail_not_be_empty_cursor', 'UT3$USER#.test_expect_to_be_empty.success_not_be_empty_cursor', 'UT3$USER#.test_expect_to_be_empty.success_not_be_empty_coll', 'UT3$USER#.test_expect_to_be_empty.fail_not_be_empty_collection', 'UT3$USER#.test_expect_to_be_empty.fail_not_be_empty_null_coll', 'UT3$USER#.test_expect_to_be_empty.fail_not_be_empty_object', 'UT3$USER#.test_expect_to_be_empty.fail_not_be_empty_null_object', 'UT3$USER#.test_expect_to_be_empty.fail_not_be_empty_number', 'UT3$USER#.test_expect_to_be_empty.fail_not_be_empty_clob', 'UT3$USER#.test_expect_to_be_empty.success_not_be_empty_clob', 'UT3$USER#.test_expect_to_be_empty.fail_not_be_empty_blob', 'UT3$USER#.test_expect_to_be_empty.success_not_be_empty_blob', 'UT3$USER#.test_expect_not_to_be_null.blob_not_null', 'UT3$USER#.test_expect_not_to_be_null.blob_0_length', 'UT3$USER#.test_expect_not_to_be_null.boolean_not_null', 'UT3$USER#.test_expect_not_to_be_null.clob_not_null', 'UT3$USER#.test_expect_not_to_be_null.clob_0_length', 'UT3$USER#.test_expect_not_to_be_null.date_not_null', 'UT3$USER#.test_expect_not_to_be_null.number_not_null', 'UT3$USER#.test_expect_not_to_be_null.timestamp_not_null', 'UT3$USER#.test_expect_not_to_be_null.timestamp_with_ltz_not_null', 'UT3$USER#.test_expect_not_to_be_null.timestamp_with_tz_not_null', 'UT3$USER#.test_expect_not_to_be_null.varchar2_not_null', 'UT3$USER#.test_expect_not_to_be_null.initialized_object', 'UT3$USER#.test_expect_not_to_be_null.initialized_nested_table', 'UT3$USER#.test_expect_not_to_be_null.initialized_varray', 'UT3$USER#.test_expect_not_to_be_null.null_blob', 'UT3$USER#.test_expect_not_to_be_null.null_boolean', 'UT3$USER#.test_expect_not_to_be_null.null_clob', 'UT3$USER#.test_expect_not_to_be_null.null_date', 'UT3$USER#.test_expect_not_to_be_null.null_number', 'UT3$USER#.test_expect_not_to_be_null.null_timestamp', 'UT3$USER#.test_expect_not_to_be_null.null_timestamp_with_ltz', 'UT3$USER#.test_expect_not_to_be_null.null_timestamp_with_tz', 'UT3$USER#.test_expect_not_to_be_null.null_varchar2', 'UT3$USER#.test_expect_not_to_be_null.null_anydata', 'UT3$USER#.test_expect_not_to_be_null.uninit_object_in_anydata', 'UT3$USER#.test_expect_not_to_be_null.uninit_nested_table_in_anydata', 'UT3$USER#.test_expect_not_to_be_null.uninit_varray_in_anydata', 'UT3$USER#.test_matchers.test_be_between2', 'UT3$USER#.test_matchers.test_match', 'UT3$USER#.test_matchers.test_be_like', 'UT3$USER#.test_matchers.test_timestamp_between', 'UT3$USER#.test_matchers.test_timestamp_ltz_between', 'UT3$USER#.test_matchers.test_timestamp_tz_between', 'UT3$USER#.test_expectations_json.success_on_same_data', 'UT3$USER#.test_expectations_json.fail_on_diff_data', 'UT3$USER#.test_expectations_json.null_json_variable', 'UT3$USER#.test_expectations_json.not_null_json_variable', 'UT3$USER#.test_expectations_json.fail_null_json_var', 'UT3$USER#.test_expectations_json.fail_not_null_json_var', 'UT3$USER#.test_expectations_json.empty_json', 'UT3$USER#.test_expectations_json.not_empty_json', 'UT3$USER#.test_expectations_json.fail_empty_json', 'UT3$USER#.test_expectations_json.fail_not_empty_json', 'UT3$USER#.test_expectations_json.to_have_count', 'UT3$USER#.test_expectations_json.fail_to_have_count', 'UT3$USER#.test_expectations_json.not_to_have_count', 'UT3$USER#.test_expectations_json.fail_not_to_have_count', 'UT3$USER#.test_expectations_json.to_have_count_array', 'UT3$USER#.test_expectations_json.to_diff_json_extract_same', 'UT3$USER#.test_expectations_json.to_diff_j
D110
son_extract_diff', 'UT3$USER#.test_expectations_json.long_json_test', 'UT3$USER#.test_expectations_json.json_same_diffrent_ord', 'UT3$USER#.test_expectations_json.long_json_test2', 'UT3$USER#.test_expectations_json.long_json_diff', 'UT3$USER#.test_expectations_json.check_json_objects', 'UT3$USER#.test_expectations_json.check_json_arrays', 'UT3$USER#.test_expectations_cursor.with_temp_table', 'UT3$USER#.test_expectations_cursor.success_on_same_data', 'UT3$USER#.test_expectations_cursor.success_on_same_data_float', 'UT3$USER#.test_expectations_cursor.success_on_empty', 'UT3$USER#.test_expectations_cursor.success_on_both_null', 'UT3$USER#.test_expectations_cursor.success_to_be_null', 'UT3$USER#.test_expectations_cursor.success_not_to_be_not_null', 'UT3$USER#.test_expectations_cursor.success_not_to_be_null', 'UT3$USER#.test_expectations_cursor.success_to_be_not_null', 'UT3$USER#.test_expectations_cursor.success_is_empty', 'UT3$USER#.test_expectations_cursor.success_is_not_empty', 'UT3$USER#.test_expectations_cursor.failure_is_null', 'UT3$USER#.test_expectations_cursor.failure_is_not_null', 'UT3$USER#.test_expectations_cursor.failure_is_empty', 'UT3$USER#.test_expectations_cursor.failure_is_not_empty', 'UT3$USER#.test_expectations_cursor.fail_null_vs_empty', 'UT3$USER#.test_expectations_cursor.fail_on_difference', 'UT3$USER#.test_expectations_cursor.fail_on_expected_missing', 'UT3$USER#.test_expectations_cursor.fail_on_actual_missing', 'UT3$USER#.test_expectations_cursor.fail_on_different_column_name', 'UT3$USER#.test_expectations_cursor.fail_on_different_column_order', 'UT3$USER#.test_expectations_cursor.pass_on_different_column_order', 'UT3$USER#.test_expectations_cursor.pass_on_diff_column_ord_uc', 'UT3$USER#.test_expectations_cursor.fail_on_multi_diff_col_order', 'UT3$USER#.test_expectations_cursor.fail_on_multi_diff_col_ord_uc', 'UT3$USER#.test_expectations_cursor.fail_on_different_row_order', 'UT3$USER#.test_expectations_cursor.include_time_in_date_with_nls', 'UT3$USER#.test_expectations_cursor.uses_default_nls_for_date', 'UT3$USER#.test_expectations_cursor.exclude_columns_as_list', 'UT3$USER#.test_expectations_cursor.exclude_columns_as_csv', 'UT3$USER#.test_expectations_cursor.exclude_columns_as_mixed_list', 'UT3$USER#.test_expectations_cursor.exclude_columns_as_mix_csv_lst', 'UT3$USER#.test_expectations_cursor.exclude_columns_xpath_invalid', 'UT3$USER#.test_expectations_cursor.exclude_columns_xpath', 'UT3$USER#.test_expectations_cursor.exclude_ignores_invalid_column', 'UT3$USER#.test_expectations_cursor.include_columns_as_list', 'UT3$USER#.test_expectations_cursor.include_columns_as_csv', 'UT3$USER#.test_expectations_cursor.include_columns_xpath', 'UT3$USER#.test_expectations_cursor.include_ignores_invalid_column', 'UT3$USER#.test_expectations_cursor.include_exclude_col_csv_xpath', 'UT3$USER#.test_expectations_cursor.include_exclude_columns_list', 'UT3$USER#.test_expectations_cursor.data_diff_on_rows_mismatch', 'UT3$USER#.test_expectations_cursor.char_and_varchar2_col_is_equal', 'UT3$USER#.test_expectations_cursor.column_diff_on_data_type_diff', 'UT3$USER#.test_expectations_cursor.column_diff_on_col_name_diff', 'UT3$USER#.test_expectations_cursor.column_diff_on_col_position', 'UT3$USER#.test_expectations_cursor.column_diff_on_col_pos_unord', 'UT3$USER#.test_expectations_cursor.data_diff_on_col_data_mismatch', 'UT3$USER#.test_expectations_cursor.data_diff_on_20_rows_only', 'UT3$USER#.test_expectations_cursor.column_and_data_diff', 'UT3$USER#.test_expectations_cursor.col_and_data_diff_not_ordered', 'UT3$USER#.test_expectations_cursor.compares_sql_and_plsql_types', 'UT3$USER#.test_expectations_cursor.closes_cursor_after_use', 'UT3$USER#.test_expectations_cursor.closes_cursor_after_use_on_err', 'UT3$USER#.test_expectations_cursor.reports_on_exception_in_cursor', 'UT3$USER#.test_expectations_cursor.exception_when_closed_cursor', 'UT3$USER#.test_expectations_cursor.compares_over_1000_rows', 'UT3$USER#.test_expectations_cursor.deprec_to_equal_excl_varch', 'UT3$USER#.test_expectations_cursor.deprec_to_equal_excl_list', 'UT3$USER#.test_expectations_cursor.deprec_not_to_equal_excl_varch', 'UT3$USER#.test_expectations_cursor.deprec_not_to_equal_excl_list', 'UT3$USER#.test_expectations_cursor.deprec_equal_excl_varch', 'UT3$USER#.test_expectations_cursor.deprec_equal_excl_list', 'UT3$USER#.test_expectations_cursor.col_diff_on_col_name_implicit', 'UT3$USER#.test_expectations_cursor.col_mtch_on_col_name_implicit', 'UT3$USER#.test_expectations_cursor.cursor_unorderd_compr_success', 'UT3$USER#.test_expectations_cursor.cursor_unord_compr_success_uc', 'UT3$USER#.test_expectations_cursor.cursor_unordered_compare_fail', 'UT3$USER#.test_expectations_cursor.cursor_joinby_compare', 'UT3$USER#.test_expectations_cursor.cursor_joinby_compare_uc', 'UT3$USER#.test_expectations_cursor.cursor_joinby_col_not_ord', 'UT3$USER#.test_expectations_cursor.cursor_joinby_compare_twocols', 'UT3$USER#.test_expectations_cursor.cursor_joinby_compare_nokey', 'UT3$USER#.test_expectations_cursor.cur_joinby_comp_twocols_nokey', 'UT3$USER#.test_expectations_cursor.cursor_joinby_compare_exkey', 'UT3$USER#.test_expectations_cursor.cur_joinby_comp_twocols_exkey', 'UT3$USER#.test_expectations_cursor.cursor_joinby_comp_nokey_ex', 'UT3$USER#.test_expectations_cursor.cursor_joinby_comp_nokey_ac', 'UT3$USER#.test_expectations_cursor.cursor_joinby_compare_1000', 'UT3$USER#.test_expectations_cursor.cursor_unorder_compare_1000', 'UT3$USER#.test_expectations_cursor.cursor_joinby_compare_fail', 'UT3$USER#.test_expectations_cursor.cursor_joinby_cmp_twocol_fail', 'UT3$USER#.test_expectations_cursor.cur_joinby_cmp_threcol_fail', 'UT3$USER#.test_expectations_cursor.unord_incl_cols_as_list', 'UT3$USER#.test_expectations_cursor.joinby_incl_cols_as_list', 'UT3$USER#.test_expectations_cursor.unord_excl_cols_as_list', 'UT3$USER#.test_expectations_cursor.joinby_excl_cols_as_list', 'UT3$USER#.test_expectations_cursor.excl_dif_cols_as_list', 'UT3$USER#.test_expectations_cursor.inlc_dif_cols_as_list', 'UT3$USER#.test_expectations_cursor.inlc_exc_dif_cols_as_list', 'UT3$USER#.test_expectations_cursor.compare_obj_typ_col_un', 'UT3$USER#.test_expectations_cursor.compare_obj_typ_col_jb', 'UT3$USER#.test_expectations_cursor.comp_obj_typ_col_un_fail', 'UT3$USER#.test_expectations_cursor.comp_obj_typ_col_jb_fail', 'UT3$USER#.test_expectations_cursor.comp_obj_typ_col_jb_multi', 'UT3$USER#.test_expectations_cursor.comp_obj_typ_col_jb_nokey', 'UT3$USER#.test_expectations_cursor.compare_nest_tab_col_jb', 'UT3$USER#.test_expectations_cursor.compare_nest_tab_col_jb_fail', 'UT3$USER#.test_expectations_cursor.compare_nest_tab_cols_jb', 'UT3$USER#.test_expectations_cursor.compare_nest_tab_cols_jb_fail', 'UT3$USER#.test_expectations_cursor.compare_tabtype_as_cols_jb', 'UT3$USER#.test_expectations_cursor.compare_tabtype_as_cols', 'UT3$USER#.test_expectations_cursor.compare_tabtype_as_cols_coll', 'UT3$USER#.test_expectations_cursor.compare_rec_colltype_as_cols', 'UT3$USER#.test_expectations_cursor.compare_rec_colltype_as_attr', 'UT3$USER#.test_expectations_cursor.compare_collection_in_rec', 'UT3$USER#.test_expectations_cursor.compare_rec_coll_as_cols_fl', 'UT3$USER#.test_expectations_cursor.compare_rec_coll_as_join', 'UT3$USER#.test_expectations_cursor.unordered_fix_764', 'UT3$USER#.test_expectations_cursor.cursor_to_contain', 'UT3$USER#.test_expectations_cursor.cursor_to_contain_fail', 'UT3$USER#.test_expectations_cursor.cursor_to_contain_uc', 'UT3$USER#.test_expectations_cursor.cursor_to_contain_unordered', 'UT3$USER#.test_expectations_cursor.cursor_contain_joinby', 'UT3$USER#.test_expectations_cursor.cursor_contain_joinby_fail', 'UT3$USER#.test_expectations_cursor.to_contain_incl_cols_as_list', 'UT3$USER#.test_expectations_cursor.to_cont_join_incl_cols_as_lst', 'UT3$USER#.test_expectations_cursor.contain_join_excl_cols_as_lst', 'UT3$USER#.test_expectations_cursor.contain_excl_cols_as_list', 'UT3$USER#.test_expectations_cursor.cursor_not_to_contain', 'UT3$USER#.test_expectations_cursor.cursor_not_to_contain_fail', 'UT3$USER#.test_expectations_cursor.cursor_not_to_contain_joinby', 'UT3$USER#.test_expectations_cursor.not_cont_join_incl_cols_as_lst', 'UT3$USER#.test_expectations_cursor.not_cont_join_excl_cols_as_lst', 'UT3$USER#.test_expectations_cursor.to_contain_duplicates', 'UT3$USER#.test_expectations_cursor.to_contain_duplicates_fail', 'UT3$USER#.test_expectations_cursor.udt_messg_format_eq', 'UT3$USER#.test_expectations_cursor.udt_messg_format_empt', 'UT3$USER#.test_expectations_cursor.xml_error_actual', 'UT3$USER#.test_expectations_cursor.xml_error_expected', 'UT3$USER#.test_expectations_cursor.no_length_datatypes', 'UT3$USER#.test_expectations_cursor.colon_part_of_columnname', 'UT3$USER#.test_expectations_cursor.specialchar_part_of_colname', 'UT3$USER#.test_expectations_cursor.nonxmlchar_part_of_colname', 'UT3$USER#.test_expectations_cursor.insginificant_whitespace1', 'UT3$USER#.test_expectations_cursor.insginificant_whitespace2', 'UT3$USER#.test_expectations_cursor.insginificant_whitespace3', 'UT3$USER#.test_expectations_cursor.insginificant_whitespace4', 'UT3$USER#.test_expectations_cursor.insginificant_whitespace5', 'UT3$USER#.test_expectations_cursor.nulltowhitespace', 'UT3$USER#.test_expectations_cursor.number_from_dual', 'UT3$USER#.test_expectations_cursor.compare_number_pckg_type', 'UT3$USER#.test_expectations_cursor.uc_columns_exclude', 'UT3$USER#.test_expectations_cursor.compare_long_column_names', 'UT3$USER#.test_expectation_anydata.fail_on_different_type_null', 'UT3$USER#.test_expectation_anydata.fail_on_different_type', 'UT3$USER#.test_expectation_anydata.fail_on_different_object_data', 'UT3$USER#.test_expectation_anydata.fail_on_one_object_null', 'UT3$USER#.test_expectation_anydata.fail_on_collection_vs_object', 'UT3$USER#.test_expectation_anydata.fail_on_null_vs_empty_coll', 'UT3$USER#.test_expectation_anydata.fail_on_one_collection_null', 'UT3$USER#.test_expectation_anydata.fail_on_one_collection_empty', 'UT3$USER#.test_expectation_anydata.fail_on_different_coll_data', 'UT3$USER#.test_expectation_anydata.success_on_both_anydata_null', 'UT3$USER#.test_expectation_anydata.success_on_both_object_null', 'UT3$USER#.test_expectation_anydata.success_on_both_coll_null', 'UT3$USER#.test_expectation_anydata.success_on_same_coll_data', 'UT3$USER#.test_expectation_anydata.fail_on_coll_different_order', 'UT3$USER#.test_expectation_anydata.success_on_same_object_data', 'UT3$USER#.test_expectation_anydata.exclude_attributes_as_list', 'UT3$USER#.test_expectation_anydata.exclude_attributes_as_csv', 'UT3$USER#.test_expectation_anydata.exclude_attributes_xpath', 'UT3$USER#.test_expectation_anydata.exclude_ignores_invalid_attrib', 'UT3$USER#.test_expectation_anydata.include_attributes_as_list', 'UT3$USER#.test_expectation_anydata.include_attributes_as_csv', 'UT3$USER#.test_expectation_anydata.include_attributes_xpath', 'UT3$USER#.test_expectation_anydata.include_ignores_invalid_attrib', 'UT3$USER#.test_expectation_anydata.include_exclude_attributes_csv', 'UT3$USER#.test_expectation_anydata.include_exclude_attrib_list', 'UT3$USER#.test_expectation_anydata.reports_diff_attribute', 'UT3$USER#.test_expectation_anydata.reports_diff_structure', 'UT3$USER#.test_expectation_anydata.deprec_to_equal_excl_varch', 'UT3$USER#.test_expectation_anydata.deprec_to_equal_excl_list', 'UT3$USER#.test_expectation_anydata.deprec_not_to_equal_excl_varch', 'UT3$USER#.test_expectation_anydata.deprec_not_to_equal_excl_list', 'UT3$USER#.test_expectation_anydata.deprec_equal_excl_varch', 'UT3$USER#.test_expectation_anydata.deprec_equal_excl_list', 'UT3$USER#.test_expectation_anydata.data_diff_on_atr_data_mismatch', 'UT3$USER#.test_expectation_anydata.data_diff_on_20_rows_only', 'UT3$USER#.test_expectation_anydata.collection_include_list', 'UT3$USER#.test_expectation_anydata.collection_exclude_list', 'UT3$USER#.test_expectation_anydata.collection_include_list_fail', 'UT3$USER#.test_expectation_anydata.array_same_data', 'UT3$USER#.test_expectation_anydata.array_diff_data', 'UT3$USER#.test_expectation_anydata.array_is_null', 'UT3$USER#.test_expectation_anydata.array_null_equal_array_null', 'UT3$USER#.test_expectation_anydata.array_null_equal_array_notnull', 'UT3$USER#.test_expectation_anydata.empty_array_have_zero_elem', 'UT3$USER#.test_expectation_anydata.array_empty_equal_array_empty', 'UT3$USER#.test_expectation_anydata.arr_empty_equal_arr_notempty', 'UT3$USER#.test_expectation_anydata.collection_is_null', 'UT3$USER#.test_expectation_anydata.collection_is_empty', 'UT3$USER#.test_expectation_anydata.varray_same_data', 'UT3$USER#.test_expectation_anydata.varray_diff_data', 'UT3$USER#.test_expectation_anydata.varray_is_null', 'UT3$USER#.test_expectation_anydata.varray_null_equal_varray_null', 'UT3$USER#.test_expectation_anydata.varr_null_equal_varr_notnull', 'UT3$USER#.test_expectation_anydata.empty_varray_have_zero_elem', 'UT3$USER#.test_expectation_anydata.varr_empty_equal_varr_empty', 'UT3$USER#.test_expectation_anydata.varr_empty_equal_varr_notempty', 'UT3$USER#.test_expectation_anydata.collection_join_by', 'UT3$USER#.test_expectation_anydata.collection_join_by_fail', 'UT3$USER#.test_expectation_anydata.collection_unordered', 'UT3$USER#.test_expectation_anydata.collection_unordered_fail', 'UT3$USER#.test_expectation_anydata.object_join_by', 'UT3$USER#.test_expectation_anydata.object_unordered', 'UT3$USER#.test_expectation_anydata.collection_to_contain', 'UT3$USER#.test_expectation_anydata.object_to_contain', 'UT3$USER#.test_expectation_anydata.arr_empty_eq_arr_empty_unord', 'UT3$USER#.test_expectation_anydata.arr_empty_nqua_arr_e_unord', 'UT3$USER#.test_expect_to_be_less_than.actual_date_greater', 'UT3$USER#.test_expect_to_be_less_than.actual_number_greater', 'UT3$USER#.test_expect_to_be_less_than.actual_interval_ym_greater', 'UT3$USER#.test_expect_to_be_less_than.actual_interval_ds_greater', 'UT3$USER#.test_expect_to_be_less_than.actual_timestamp_greater', 'UT3$USER#.test_expect_to_be_less_than.actual_timestamp_tz_greater', 'UT3$USER#.test_expect_to_be_less_than.actual_timestamp_ltz_greater', 'UT3$USER#.test_expect_to_be_less_than.actual_date_equal', 'UT3$USER#.test_expect_to_be_less_than.actual_number_equal', 'UT3$USER#.test_expect_to_be_less_than.actual_interval_ym_equal', 'UT3$USER#.test_expect_to_be_less_than.actual_interval_ds_equal', 'UT3$USER#.test_expect_to_be_less_than.actual_timestamp_equal', 'UT3$USER#.test_expect_to_be_less_than.actual_timestamp_tz_equal', 'UT3$USER#.test_expect_to_be_less_than.actual_timestamp_ltz_equal', 'UT3$USER#.test_expect_to_be_less_than.actual_date_less', 'UT3$USER#.test_expect_to_be_less_than.actual_number_less', 'UT3$USER#.test_expect_to_be_less_than.actual_interval_ym_less', 'UT3$USER#.test_expect_to_be_less_than.actual_interval_ds_less', 'UT3$USER#.test_expect_to_be_less_than.actual_timestamp_less', 'UT3$USER#.test_expect_to_be_less_than.actual_timestamp_tz_less', 'UT3$USER#.test_expect_to_be_less_than.actual_timestamp_ltz_less', 'UT3$USER#.test_expect_to_be_less_than.not_actual_date_greater', 'UT3$USER#.test_expect_to_be_less_than.not_actual_number_greater', 'UT3$USER#.test_expect_to_be_less_than.not_actual_interval_ym_greater', 'UT3$USER#.test_expect_to_be_less_than.not_actual_interval_ds_greater', 'UT3$USER#.test_expect_to_be_less_than.not_actual_timestamp_greater', 'UT3$USER#.test_expect_to_be_less_than.not_actual_timestamp_tz_gretr', 'UT3$USER#.test_expect_to_be_less_than.not_actual_timestamp_ltz_gretr', 'UT3$USER#.test_expect_to_be_less_than.actual_clob', 'UT3$USER#.test_equal.equal_fail_on_type_diff', 'UT3$USER#.test_equal.not_equal_fail_on_type_diff', 'UT3$USER#.test_equal.failure_on_data_diff', 'UT3$USER#.test_equal.failure_on_actual_null', 'UT3$USER#.test_equal.failure_on_expected_null', 'UT3$USER#.test_equal.failure_on_both_null_with_parm', 'UT3$USER#.test_equal.failure_on_both_null_with_conf', 'UT3$USER#.test_equal.success_on_equal_data', 'UT3$USER#.test_equal.success_on_both_null', 'UT3$USER#.test_equal.success_on_both_null_with_parm', 'UT3$USER#.test_be_less_or_equal.actual_date_greater', 'UT3$USER#.test_be_less_or_equal.actual_number_greater', 'UT3$USER#.test_be_less_or_equal.actual_interval_ym_greater', 'UT3$USER#.test_be_less_or_equal.actual_interval_ds_greater', 'UT3$USER#.test_be_less_or_equal.actual_timestamp_greater', 'UT3$USER#.test_be_less_or_equal.actual_timestamp_tz_greater', 'UT3$USER#.test_be_less_or_equal.actual_timestamp_ltz_greater', 'UT3$USER#.test_be_less_or_equal.actual_date_equal', 'UT3$USER#.test_be_less_or_equal.actual_number_equal', 'UT3$USER#.test_be_less_or_equal.actual_interval_ym_equal', 'UT3$USER#.test_be_less_or_equal.actual_interval_ds_equal', 'UT3$USER#.test_be_less_or_equal.actual_timestamp_equal', 'UT3$USER#.test_be_less_or_equal.actual_timestamp_tz_equal', 'UT3$USER#.test_be_less_or_equal.actual_timestamp_ltz_equal', 'UT3$USER#.test_be_less_or_equal.actual_date_less', 'UT3$USER#.test_be_less_or_equal.actual_number_less', 'UT3$USER#.test_be_less_or_equal.actual_interval_ym_less', 'UT3$USER#.test_be_less_or_equal.actual_interval_ds_less', 'UT3$USER#.test_be_less_or_equal.actual_timestamp_less', 'UT3$USER#.test_be_less_or_equal.actual_timestamp_tz_less', 'UT3$USER#.test_be_less_or_equal.actual_timestamp_ltz_less', 'UT3$USER#.test_be_less_or_equal.not_actual_date_greater', 'UT3$USER#.test_be_less_or_equal.not_actual_number_greater', 'UT3$USER#.test_be_less_or_equal.not_actual_interval_ym_greater', 'UT3$USER#.test_be_less_or_equal.not_actual_interval_ds_greater', 'UT3$USER#.test_be_less_or_equal.not_actual_timestamp_greater', 'UT3$USER#.test_be_less_or_equal.not_actual_timestamp_tz_gretr', 'UT3$USER#.test_be_less_or_equal.not_actual_timestamp_ltz_gretr', 'UT3$USER#.test_be_less_or_equal.actual_clob', 'UT3$USER#.test_be_greater_than.actual_date_greater', 'UT3$USER#.test_be_greater_than.actual_number_greater', 'UT3$USER#.test_be_greater_than.actual_interval_ym_greater', 'UT3$USER#.test_be_greater_than.actual_interval_ds_greater', 'UT3$USER#.test_be_greater_than.actual_timestamp_greater', 'UT3$USER#.test_be_greater_than.actual_timestamp_tz_greater', 'UT3$USER#.test_be_greater_than.actual_timestamp_ltz_greater', 'UT3$USER#.test_be_greater_than.actual_date_equal', 'UT3$USER#.test_be_greater_than.actual_number_equal', 'UT3$USER#.test_be_greater_than.actual_interval_ym_equal', 'UT3$USER#.test_be_greater_than.actual_interval_ds_equal', 'UT3$USER#.test_be_greater_than.actual_timestamp_equal', 'UT3$USER#.test_be_greater_than.actual_timestamp_tz_equal', 'UT3$USER#.test_be_greater_than.actual_timestamp_ltz_equal', 'UT3$USER#.test_be_greater_than.actual_date_less', 'UT3$USER#.test_be_greater_than.actual_number_less', 'UT3$USER#.test_be_greater_than.actual_interval_ym_less', 'UT3$USER#.test_be_greater_than.actual_interval_ds_less', 'UT3$USER#.test_be_greater_than.actual_timestamp_less', 'UT3$USER#.test_be_greater_than.actual_timestamp_tz_less', 'UT3$USER#.test_be_greater_than.actual_timestamp_ltz_less', 'UT3$USER#.test_be_greater_than.not_actual_date_greater', 'UT3$USER#.test_be_greater_than.not_actual_number_greater', 'UT3$USER#.test_be_greater_than.not_actual_interval_ym_greater', 'UT3$USER#.test_be_greater_than.not_actual_interval_ds_greater', 'UT3$USER#.test_be_greater_than.not_actual_timestamp_greater', 'UT3$USER#.test_be_greater_than.not_actual_timestamp_tz_gretr', 'UT3$USER#.test_be_greater_than.not_actual_timestamp_ltz_gretr', 'UT3$USER#.test_be_greater_than.actual_clob', 'UT3$USER#.test_be_greater_or_equal.actual_date_greater', 'UT3$USER#.test_be_greater_or_equal.actual_number_greater', 'UT3$USER#.test_be_greater_or_equal.actual_interval_ym_greater', 'UT3$USER#.test_be_greater_or_equal.actual_interval_ds_greater', 'UT3$USER#.test_be_greater_or_equal.actual_timestamp_greater', 'UT3$USER#.test_be_greater_or_equal.actual_timestamp_tz_greater', 'UT3$USER#.test_be_greater_or_equal.actual_timestamp_ltz_greater', 'UT3$USER#.test_be_greater_or_equal.actual_date_equal', 'UT3$USER#.test_be_greater_or_equal.actual_number_equal', 'UT3$USER#.test_be_greater_or_equal.actual_interval_ym_equal', 'UT3$USER#.test_be_greater_or_equal.actual_interval_ds_equal', 'UT3$USER#.test_be_greater_or_equal.actual_timestamp_equal', 'UT3$USER#.test_be_greater_or_equal.actual_timestamp_tz_equal', 'UT3$USER#.test_be_greater_or_equal.actual_timestamp_ltz_equal', 'UT3$USER#.test_be_greater_or_equal.actual_date_less', 'UT3$USER#.test_be_greater_or_equal.actual_number_less', 'UT3$USER#.test_be_greater_or_equal.actual_interval_ym_less', 'UT3$USER#.test_be_greater_or_equal.actual_interval_ds_less', 'UT3$USER#.test_be_greater_or_equal.actual_timestamp_less', 'UT3$USER#.test_be_greater_or_equal.actual_timestamp_tz_less', 'UT3$USER#.test_be_greater_or_equal.actual_timestamp_ltz_less', 'UT3$USER#.test_be_greater_or_equal.not_actual_date_greater', 'UT3$USER#.test_be_greater_or_equal.not_actual_number_greater', 'UT3$USER#.test_be_greater_or_equal.not_actual_interval_ym_greater', 'UT3$USER#.test_be_greater_or_equal.not_actual_interval_ds_greater', 'UT3$USER#.test_be_greater_or_equal.not_actual_timestamp_greater', 'UT3$USER#.test_be_greater_or_equal.not_actual_timestamp_tz_gretr', 'UT3$USER#.test_be_greater_or_equal.not_actual_timestamp_ltz_gretr', 'UT3$USER#.test_be_greater_or_equal.actual_clob', 'UT3$USER#.test_ut_runner.version_comp_check_compare', 'UT3$USER#.test_ut_runner.version_comp_check_ignore', 'UT3$USER#.test_ut_runner.version_comp_check_short', 'UT3$USER#.test_ut_runner.version_comp_check_exception', 'UT3$USER#.test_ut_runner.pass_varchar2_name_list', 'UT3$USER#.test_ut_runner.pass_varchar2_name', 'UT3$USER#.test_ut_runner.pass_varchar2_suite_csl', 'UT3$USER#.test_ut_runner.pass_varchar2_test_csl', 'UT3$USER#.test_ut_runner.pass_varch_test_csl_spc', 'UT3$USER#.test_ut_runner.pass_csl_with_srcfile', 'UT3$USER#.test_ut_runner.pass_csl_within_var2list', 'UT3$USER#.test_ut_runner.is_test_true', 'UT3$USER#.test_ut_runner.is_test_false', 'UT3$USER#.test_ut_runner.is_suite_true', 'UT3$USER#.test_ut_runner.is_suite_false', 'UT3$USER#.test_ut_runner.has_suites_true', 'UT3$USER#.test_ut_runner.has_suites_false', 'UT3$USER#.test_ut_runner.keep_an_open_transaction', 'UT3$USER#.test_ut_runner.close_newly_opened_transaction', 'UT3$USER#.test_ut_runner.run_reset_package_body_cache', 'UT3$USER#.test_ut_runner.run_keep_dbms_output_buffer', 'UT3$USER#.test_ut_runner.test_purge_cache_schema_type', 'UT3$USER#.test_ut_runner.test_rebuild_cache_schema_type', 'UT3$USER#.test_ut_runner.test_get_suites_info_notag', 'UT3$USER#.test_ut_runner.test_get_suites_info_tag', 'UT3$USER#.test_ut_runner.test_get_suites_info_twotag', 'UT3$USER#.test_ut_runner.test_get_reporters_list', 'UT3$USER#.test_ut_runner.raises_20213_on_fail_link', 'UT3$USER#.test_ut_run.run_proc_no_params', 'UT3$USER#.test_ut_run.run_proc_specific_reporter', 'UT3$USER#.test_ut_run.run_proc_cov_file_list', 'UT3$USER#.test_ut_run.run_proc_pkg_name', 'UT3$USER#.test_ut_run.run_proc_pkg_name_file_list', 'UT3$USER#.test_ut_run.run_proc_path_list', 'UT3$USER#.test_ut_run.run_proc_path_list_file_list', 'UT3$USER#.test_ut_run.run_proc_null_reporter', 'UT3$USER#.test_ut_run.run_proc_null_path', 'UT3$USER#.test_ut_run.run_proc_null_path_list', 'UT3$USER#.test_ut_run.run_proc_empty_path_list', 'UT3$USER#.test_ut_run.run_proc_warn_on_commit', 'UT3$USER#.test_ut_run.run_proc_fail_child_suites', 'UT3$USER#.test_ut_run.savepoints_on_db_links', 'UT3$USER#.test_ut_run.run_func_no_params', 'UT3$USER#.test_ut_run.run_func_specific_reporter', 'UT3$USER#.test_ut_run.run_func_cov_file_list', 'UT3$USER#.test_ut_run.run_func_pkg_name', 'UT3$USER#.test_ut_run.run_func_pkg_name_file_list', 'UT3$USER#.test_ut_run.run_func_path_list', 'UT3$USER#.test_ut_run.run_func_path_list_file_list', 'UT3$USER#.test_ut_run.run_func_null_reporter', 'UT3$USER#.test_ut_run.run_func_null_path', 'UT3$USER#.test_ut_run.run_func_null_path_list', 'UT3$USER#.test_ut_run.run_func_empty_path_list', 'UT3$USER#.test_ut_run.run_func_cov_file_lst_null_rep', 'UT3$USER#.test_ut_run.run_func_empty_suite', 'UT3$USER#.test_ut_run.raise_in_invalid_state', 'UT3$USER#.test_ut_run.run_in_invalid_state', 'UT3$USER#.test_ut_run.run_and_revalidate_specs', 'UT3$USER#.test_ut_run.run_and_report_warnings', 'UT3$USER#.test_ut_run.run_schema_name_test', 'UT3$USER#.test_ut_run.run_proc_keep_test_data', 'UT3$USER#.test_ut_run.run_proc_keep_test_data_raise', 'UT3$USER#.test_ut_run.run_proc_discard_test_data', 'UT3$USER#.test_ut_run.test_run_by_one_tag', 'UT3$USER#.test_ut_run.suite_run_by_one_tag', 'UT3$USER#.test_ut_run.two_test_run_by_one_tag', 'UT3$USER#.test_ut_run.all_suites_run_by_one_tag', 'UT3$USER#.test_ut_run.two_test_run_by_two_tags', 'UT3$USER#.test_ut_run.suite_with_children_tag', 'UT3$USER#.test_ut_run.suite_with_tag_parent', 'UT3$USER#.test_ut_run.test_nonexists_tag', 'UT3$USER#.test_ut_run.test_duplicate_tag', 'UT3$USER#.test_ut_run.suite_duplicate_tag', 'UT3$USER#.test_ut_run.run_proc_pkg_name_tag', 'UT3$USER#.test_ut_run.run_pkg_name_file_list_tag', 'UT3$USER#.test_ut_run.run_proc_path_list_tag', 'UT3$USER#.test_ut_run.tag_run_func_no_params', 'UT3$USER#.test_ut_run.tag_run_func_pkg_name', 'UT3$USER#.test_ut_run.tag_run_func_path_list', 'UT3$USER#.test_ut_run.run_with_random_order', 'UT3$USER#.test_ut_run.run_and_report_random_ord_seed', 'UT3$USER#.test_ut_run.run_with_random_order_seed', 'UT3$USER#.test_ut_run.ut_version', 'UT3$USER#.test_ut_run.ut_fail'));
  1. this will produce the following result (excerpt):
Failures:
 
  1) report_on_file
      Actual: NULL (clob) was expected to be like: 
          '<?xml version="1.0"?>
          <!DOCTYPE coverage SYSTEM "http://cobertura.sourceforge.net/xml/coverage-04.dtd">
          <coverage line-rate="0" branch-rate="0.0" lines-covered="2" lines-valid="3" branches-covered="0" branches-valid="0" complexity="0" version="1" timestamp="%">
          <sources>
          <source>test/ut3.dummy_coverage.pkb</source>
          </sources>
          <packages>
          <package name="DUMMY_COVERAGE" line-rate="0.0" branch-rate="0.0" complexity="0.0">
          <class name="DUMMY_COVERAGE" filename="test/ut3.dummy_coverage.pkb" line-rate="0.0" branch-rate="0.0" complexity="0.0">
          <lines>
          <line number="4" hits="1" branch="false"/>
          <line number="5" hits="0" branch="false"/>
          <line number="7" hits="1" branch="false"/>
          </lines>
          </class>
          </package>
          </packages>
          </coverage>'
      at "UT3$USER#.TEST_COV_COBERTURA_REPORTER.REPORT_ON_FILE", line 41 ut.expect(l_actual).to_be_like(l_expected);
      
       
  2) report_on_file
      Actual: NULL (clob) was expected to equal: (clob)
          '<?xml version="1.0"?>
          <coverage version="1">
          <file path="test/ut3.dummy_coverage.pkb">
          <lineToCover lineNumber="4" covered="true"/>
          <lineToCover lineNumber="5" covered="false"/>
          <lineToCover lineNumber="7" covered="true"/>
          </file>
          </coverage>'
      at "UT3$USER#.TEST_COVERAGE_SONAR_REPORTER.REPORT_ON_FILE", line 30 ut.expect(l_actual).to_equal(l_expected);
      
       
  3) report_on_file
      Actual: NULL (clob) was expected to equal: (clob)
          '{"source_files":[
          { "name": "test/ut3.dummy_coverage.pkb",
          "coverage": [
          null,
          null,
          null,
          1,
          0,
          null,
          1
          ]
          }
          ]}
           '
      at "UT3$USER#.TEST_COVERALLS_REPORTER.REPORT_ON_FILE", line 36 ut.expect(l_actual).to_equal(l_expected);
      
       
Finished in 157.360601 seconds
876 tests, 3 failed, 0 errored, 5 disabled, 0 warning(s)

Expected behavior

  • the ut.run in step 2 and 5 above should produce the same number of succeeded, failed, errored, disabled tests. (BUG)
  • the ut.run in step 2 and 5 should have similar execution times (now 61 vs. 157 seconds), this means the list of past suitepaths should be analyzed and top level suitepaths applied whenever possible. In this case the call in step 5 should result in an internal call looking the same as in step 2. (ENHANCEMENT)
  • EXECUTE ut.run('UT3$USER#.TEST_COV_COBERTURA_REPORTER.REPORT_ON_FILE'); succeeds in a fresh connection (BUG)
  • EXECUTE ut.run('UT3$USER#.TEST_COVERAGE_SONAR_REPORTER.REPORT_ON_FILE'); succeeds in a fresh connection (BUG)
  • EXECUTE ut.run('UT3$USER#.TEST_COVERALLS_REPORTER.REPORT_ON_FILE'); succeeds in a fresh connection (BUG)

Metadata

Metadata

Assignees

No one assigned

    Labels

    bug

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions
      0