8000 Merge pull request #1079 from utPLSQL/feature/standalone_coverage · utPLSQL/utPLSQL@ad39737 · GitHub
[go: up one dir, main page]
Skip to content

Commit ad39737

Browse files
jgebaljgebal
authored
Merge pull request #1079 from utPLSQL/feature/standalone_coverage
Standalone coverage
2 parents 2956d07 + 955de5c commit ad39737

File tree

65 files changed

+1178
-971
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

65 files changed

+1178
-971
lines changed

‎docs/userguide/coverage.md

Lines changed: 130 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
![version](https://img.shields.io/badge/version-v3.1.11.3455--develop-blue.svg)
22

33
# Coverage
4-
utPLSQL comes with a built-in coverage reporting engine. The code coverage reporting is based on the DBMS_PROFILER package provided with Oracle database.
4+
utPLSQL comes with a built-in coverage reporting engine. The code coverage reporting uses DBMS_PROFILER package provided with Oracle database.
55
Code coverage is gathered for the following source types:
66
* package bodies
77
* type bodies
@@ -11,9 +11,9 @@ Code coverage is gathered for the following source types:
1111

1212
**Note**
1313

14-
> The package and type specifications are explicitly excluded from code coverage analysis. This limitation is introduced to avoid false-negatives. Typically package specifications contain no executable code. The only exception is initialization of global constants and variables in package specification. Since most package specifications are not executable at all, there is no information available on the number of lines covered and those would be reported as 0% covered, which is not desirable.
14+
> The package and type specifications are excluded from code coverage analysis. This limitation is introduced to avoid false-negatives. Typically package specifications contain no executable code. The only exception is initialization of global constants and variables in package specification. Since most package specifications are not executable at all, there is no information available on the number of lines covered and those would be reported as 0% covered, which is not desirable.
1515
16-
To obtain information about code coverage of your unit tests, all you need to do is run your unit tests with one of built-in code coverage reporters.
16+
To obtain information about code coverage for unit tests, run utPLSQL with one of built-in code coverage reporters.
1717
The following code coverage reporters are supplied with utPLSQL:
1818
* `ut_coverage_html_reporter` - generates a HTML coverage report providing summary and detailed information on code coverage. The HTML reporter is based on the open-source [simplecov-html](https://github.com/colszowka/simplecov-html) reporter for Ruby. It includes source code of the code that was covered (if possible)
1919
* `ut_coveralls_reporter` - generates a [Coveralls compatible JSON](https://coveralls.zendesk.com/hc/en-us/articles/201774865-API-Introduction) coverage report providing detailed information on code coverage with line numbers. This coverage report is designed to be consumed by cloud services like [Coveralls](https://coveralls.io)
@@ -33,8 +33,8 @@ If you have `execute` privilege only on the unit tests, but do not have `execute
3333
If the code that is being tested is complied as NATIVE, the code coverage will not be reported as well.
3434

3535
## Running unit tests with coverage
36-
Using the code coverage functionality is as easy as using any other [reporter](reporters.md) for the utPLSQL project. You just run your tests from your preferred SQL tool and save the reporter results to a file.
37-
All you need to do is pass the constructor of the reporter to your `ut.run`
36+
Using the code coverage functionality is as easy as using any other [reporter](reporters.md) for the utPLSQL test-run. You just run your tests from your preferred SQL tool and save the reporter results to a file.
37+
All you need to do is pass the constructor of the reporter to the `ut.run` procedure call.
3838

3939
Example:
4040
```sql
@@ -43,10 +43,10 @@ begin
4343
end;
4444
/
4545
```
46-
Executes all unit tests in the current schema, gathers information about code coverage and outputs the HTML text into DBMS_OUTPUT.
46+
The above command executes all unit tests in the **current schema**, gathers information about code coverage and outputs the HTML report as text into DBMS_OUTPUT.
4747
The `ut_coverage_html_reporter` will produce an interactive HTML report. You can see a sample of code coverage for the utPLSQL project [here](https://utplsql.github.io/utPLSQL-coverage-html/)
4848

49-
The report provides summary information with a list of source code that was expected to be covered.
49+
The report provides summary information with a list of source code that should be covered.
5050

5151
![Coverage Summary page](../images/coverage_html_summary.png)
5252

@@ -57,8 +57,8 @@ The report allow you to navigate to each source file and inspect line by line co
5757

5858
#### Oracle 12.2 extended coverage with profiler and block coverage
5959
Using data collected from profiler and block coverage running parallel we are able to enrich information about coverage.
60-
For every line recorded by profiler if we have a partially covered same line in block coverage we will display that information
61-
presenting line as partially covered, displaying number of block and how many blocks been covered in that line.The feature will be automatically enabled in the Oracle database version 12.2 and higher, for older versions current profiler will be used.
60+
For every line recorded by the profiler if we have a partially covered same line in block coverage we will display that information
61+
presenting line as partially covered, displaying number of block and how many blocks have been covered in that line.The feature will be automatically enabled in the Oracle database version 12.2 and higher, for older versions current profiler will be used.
6262

6363
utPLSQL installation automatically creates tables needed by `dbms_plsql_code_coverage` on databases in versions above 12c Release 1.
6464
Due to security model of `dbms_plsql_code_coverage` package, utPLSQL grants access to those tables and creates synonyms for those tables.
@@ -92,7 +92,8 @@ The default behavior of coverage reporting can be altered using invocation param
9292

9393
### Schema based Coverage
9494

95-
To simply gather coverage for all objects in your current schema execute tests with coverage reporting.
95+
To gather coverage for all objects in the **current schema** execute tests with coverage report as argument.
96+
This is the default reporting option and therefore additional coverage options don't need to be provided.
9697

9798
```sql
9899
exec ut.run(ut_coverage_html_reporter());
@@ -109,7 +110,7 @@ exec ut.run(ut_coverage_html_reporter());
109110
#### Setting coverage schema(s)
110111

111112
By default, coverage is gathered on the schema(s) derived from suite paths provided to execute tests.
112-
This is correct as long as your test packages and tested code share the same schema.
113+
This is a valid approach as long as your test packages and tested code share the same schema.
113114

114115
So when you run:
115116
```sql
@@ -531,8 +532,8 @@ Unit test code is mapped to files in `test_results.xml`
531532

532533
In order to allow deterministic and accurate mapping of database source-code into project files, the project directory and file structure needs to meet certain criteria.
533534
- Source code is kept separate from test code (separate directories)
534-
- Each database (source-code) object is stored in individual file. Package/type specification is kept separate from it's body.
535-
- File name (file path) contains name of database object
535+
- Each database (source-code) object is stored in an individual file. Package/type specification is kept separate from its body.
536+
- File name (file path) contains the name of database object
536537
- Each file-path clearly identifies object type (by file extension)
537538
- Each file contains representation of database object "as is". No extra commands (like `set echo off` `ALTER SESSION SET PLSQL_CCFLAGS = 'debug:TRUE';`) or blank lines are present before `CREATE TYPE`,`CREATE TYPE` etc.
538539
- When project is spanning across multiple database schemes, each file-path clearly and uniformly identifies object owner
@@ -659,3 +660,119 @@ begin
659660
end;
660661
```
661662

663+
## Reporting coverage outside of utPLSQL
664+
665+
utPSLQL allows fo standalone reporting code coverage across multiple database sessions. This functionality enables coverage reporting for external testing tools.
666+
667+
Following API calls enable the standalone coverage reporting.
668+
669+
- `ut_runner.coverage_start( coverage_run_id );` - initiates code coverage within a session
670+
- `ut_runner.coverage_stop();` - stops gathering of code coverage within a session
671+
- `.get_report( ... )` - coverage reporters function producing coverage report as pipelined data-set (to be used in SQL query)
672+
- `.get_report_cursor( ... )` - coverage reporters function producing coverage report as ref-cursor
673+
674+
Example:
675+
```sql
676+
--SESSION 1
677+
-- gather coverage on code using specific coverage_run_id value
678+
declare
679+
l_coverage_run_id raw(32);
680+
begin
681+
l_coverage_run_id := 'A6AA5B7361251CE6E053020011ACA055';
682+
-- l_coverage_run_id := sys_guid;
683+
ut_runner.coverage_start(l_coverage_run_id);
684+
685+
--The code to gather coverage on goes here
686+
687+
ut_runner.coverage_stop();
688+
end;
689+
/
690+
```
691+
692+
```sql
693+
--SESSION 2
694+
-- alternative approach
695+
-- gather coverage on code using specific coverage_run_id value
696+
exec ut_runner.coverage_start('A6AA5B7361251CE6E053020011ACA055');
697+
698+
--The code to gather coverage on goes here
699+
700+
exec ut_runner.coverage_stop();
701+
```
702+
703+
704+
```sql
705+
--SESSION 1 or SESSION2 2 or SESSION 3
706+
-- run after calls in SESSION 1 & 2 are finshed
707+
-- retrieve coverage report in HTML format coverage_run_id value
708+
select *
709+
from table(
710+
ut_coverage_html_reporter().get_report(
711+
a_coverage_options => ut_coverage_options(
712+
coverage_run_id => 'A6AA5B7361251CE6E053020011ACA055'
713+
)
714+
)
715+
);
716+
```
717+
718+
```sql
719+
--SESSION 1 or SESSION2 2 or SESSION 3
720+
-- run after calls in SESSION 1 & 2 are finshed
721+
declare
722+
l_results_cursor sys_refcursor;
723+
begin
724+
l_results_cursor := ut_coverage_html_reporter().get_report_cursor(
725+
a_coverage_options => ut_coverage_options(
726+
coverage_run_id => 'A6AA5B7361251CE6E053020011ACA055'
727+
)
728+
);
729+
--fetch and process the cursor results
730+
close l_results_cursor;
731+
end;
732+
/
733+
```
734+
735+
Specification of parameters for `get_report` and `get_report_cursor`
736+
```sql
737+
function get_report(
738+
a_coverage_options ut_coverage_options,
739+
a_client_character_set varchar2 := null
740+
) return ut_varchar2_rows pipelined
741+
```
742+
743+
```sql
744+
function get_report_cursor(
745+
a_coverage_options ut_coverage_options,
746+
a_client_character_set varchar2 := null
747+
) return sys_refcursor
748+
```
749+
```sql
750+
ut_coverage_options(
751+
coverage_run_id raw,
752+
schema_names ut_varchar2_rows := null,
753+
exclude_objects ut_varchar2_rows := null,
754+
include_objects ut_varchar2_rows := null,
755+
file_mappings ut_file_mappings := null
756+
);
757+
```
758+
759+
The `a_client_character_set` is used to provide character set to the report. Coverage reports in XML and HTML format include this information to assure that HMTL/XML encoding tag is aligned with encoding of the report produced.
760+
Use this parameter to provide encoding of your client application.
761+
762+
The `a_coverage_options` parameter is used to control the scope and formatting of data returned by report.
763+
764+
`ut_coverage_options` object accepts the following arguments
765+
766+
- `coverage_run_id` - identifier of coverage run to generate report for - data-type `RAW(32)`
767+
- `schema_names` - optional - list of schema names to include in coverage report - data-type `UT_VARCHAR2_ROWS`
768+
- `exclude_objects` - optional - list of object names to exclude from report - data-type `UT_VARCHAR2_ROWS`
769+
- `include_objects` - optional - list of object names to gather coverage on - data-type `UT_VARCHAR2_ROWS`
770+
- `file_mappings` - optional - list of schema names to gather coverage on - data-type `UT_FILE_MAPPINGS`
771+
772+
`coverage_run_id` parameter identifies a common coverage run. The valid value type for that parameter is RAW(32).
773+
It is recommended to use `sys_guid()` to generate a common, unique identifier for a specific coverage run.
774+
If the identifier is not unique, previous runs of coverage that used the same `coverage_run_id` will be aggregated to the resulting coverage report.
775+
776+
For details on the meaning of `schema_names`, `exclude_objects`, `include_objects`, `file_mappings` see sections above.
777+
Note that data-types of include/exclude/schema lists are different when calling `ut.run` vs. calling `get_report/get_report_cursor`.
778+

‎examples/RunExpectations.sql

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ set echo off
1111
@@demo_expectations.pck
1212

1313
begin
14-
ut_coverage.coverage_start();
14+
ut_coverage.coverage_start(sys_guid());
1515
ut_coverage.set_develop_mode(true);
1616
ut.run();
1717
ut_coverage.set_develop_mode(false);

‎source/api/ut_runner.pkb

Lines changed: 30 additions & 48 deletions
Original file line numberDiff line numberDiff line change
@@ -20,27 +20,6 @@ create or replace package body ut_runner is
2020
/**
2121
* Private functions
2222
*/
23-
function to_ut_object_list(a_names ut_varchar2_list, a_schema_names ut_varchar2_rows) return ut_object_names is
24-
l_result ut_object_names;
25-
l_object_name ut_object_name;
26-
begin
27-
if a_names is not empty then
28-
l_result := ut_object_names();
29-
for i in 1 .. a_names.count loop
30-
l_object_name := ut_object_name(a_names(i));
31-
if l_object_name.owner is null then
32-
for i in 1 .. cardinality(a_schema_names) loop
33-
l_result.extend;
34-
l_result(l_result.last) := ut_object_name(a_schema_names(i)||'.'||l_object_name.name);
35-
end loop;
36-
else
37-
l_result.extend;
38-
l_result(l_result.last) := l_object_name;
39-
end if;
40-
end loop;
41-
end if;
42-
return l_result;
43-
end;
4423

4524
procedure finish_run(a_run ut_run, a_force_manual_rollback boolean) is
4625
begin
@@ -51,7 +30,6 @@ create or replace package body ut_runner is
5130
ut_compound_data_helper.cleanup_diff();
5231
if not a_force_manual_rollback then
5332
rollback;
54-
ut_utils.cleanup_session_temp_tables;
5533
end if;
5634
end;
5735

@@ -95,9 +73,7 @@ create or replace package body ut_runner is
9573
) is
9674
l_run ut_run;
9775
l_coverage_schema_names ut_varchar2_rows;
98-
l_exclude_object_names ut_object_names := ut_object_names();
99-
l_include_object_names ut_object_names;
100-
l_paths ut_varchar2_list := ut_varchar2_list();
76+
l_paths ut_varchar2_list;
10177
l_random_test_order_seed positive;
10278
l_tags ut_varchar2_rows := ut_varchar2_rows();
10379
begin
@@ -113,55 +89,51 @@ create or replace package body ut_runner is
11389

11490
ut_event_manager.trigger_event(ut_event_manager.gc_initialize);
11591
ut_event_manager.trigger_event(ut_event_manager.gc_debug, ut_run_info());
92+
11693
if a_random_test_order_seed is not null then
11794
l_random_test_order_seed := a_random_test_order_seed;
11895
elsif a_random_test_order then
11996
dbms_random.seed( to_char(systimestamp,'yyyyddmmhh24missffff') );
12097
l_random_test_order_seed := trunc(dbms_random.value(1, 1000000000));
12198
end if;
122-
if a_paths is null or a_paths is empty or a_paths.count = 1 and a_paths(1) is null then
99+
100+
l_paths := ut_utils.filter_list(ut_utils.string_table_to_table(a_paths,','), '.+');
101+
if l_paths is null or l_paths is empty then
123102
l_paths := ut_varchar2_list(sys_context('userenv', 'current_schema'));
124-
else
125-
for i in 1..a_paths.count loop
126-
l_paths := l_paths multiset union ut_utils.string_to_table(a_string => a_paths(i),a_delimiter => ',');
127-
end loop;
128103
end if;
129104

130105
begin
131106
ut_expectation_processor.reset_invalidation_exception();
132107
ut_utils.save_dbms_output_to_cache();
133108

134109
ut_console_reporter_base.set_color_enabled(a_color_console);
110+
135111
if a_coverage_schemes is not empty then
136112
l_coverage_schema_names := ut_utils.convert_collection(a_coverage_schemes);
137113
else
138114
l_coverage_schema_names := ut_suite_manager.get_schema_names(l_paths);
139115
end if;
140116

141-
if a_exclude_objects is not empty then
142-
l_exclude_object_names := to_ut_object_list(a_exclude_objects, l_coverage_schema_names);
143-
end if;
144-
117+
145118
if a_tags is not null then
146-
l_tags := l_tags multiset union distinct ut_utils.convert_collection(
119+
l_tags := l_tags multiset union distinct ut_utils.convert_collection(
147120
ut_utils.trim_list_elements(ut_utils.filter_list(ut_utils.string_to_table(a_tags,','),ut_utils.gc_word_no_space))
148121
);
149122
end if;
150-
l_exclude_object_names := l_exclude_object_names multiset union all ut_suite_manager.get_schema_ut_packages(l_coverage_schema_names);
151-
152-
l_include_object_names := to_ut_object_list(a_include_objects, l_coverage_schema_names);
153123

154124
l_run := ut_run(
155-
null,
156-
l_paths,
157-
l_coverage_schema_names,
158-
l_exclude_object_names,
159-
l_include_object_names,
160-
set(a_source_file_mappings),
161-
set(a_test_file_mappings),
162-
a_client_character_set,
163-
l_random_test_order_seed,
164-
l_tags
125+
a_run_paths => l_paths,
126+
a_coverage_options => ut_coverage_options(
127+
coverage_run_id => sys_guid(),
128+
schema_names => l_coverage_schema_names,
129+
exclude_objects => ut_utils.convert_collection(a_exclude_objects),
130+
include_objects => ut_utils.convert_collection(a_include_objects),
131+
file_mappings => set(a_source_file_mappings)
132+
),
133+
a_test_file_mappings => set(a_test_file_mappings),
134+
a_client_character_set => a_client_character_set,
135+
a_random_test_order_seed => l_random_test_order_seed,
136+
a_run_tags => l_tags
165137
);
166138

167139
ut_suite_manager.configure_execution_by_path(l_paths, l_run.items, l_random_test_order_seed, l_tags);
@@ -296,5 +268,15 @@ create or replace package body ut_runner is
296268
return l_result;
297269
end;
298270

271+
procedure coverage_start(a_coverage_run_id raw) is
272+
begin
273+
ut_coverage.coverage_start(a_coverage_run_id);
274+
end;
275+
276+
procedure coverage_stop is
277+
begin
278+
ut_coverage.coverage_stop;
279+
end;
280+
299281
end ut_runner;
300282
/

‎source/api/ut_runner.pks

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -145,5 +145,9 @@ create or replace package ut_runner authid current_user is
145145
*/
146146
function hash_suite_path(a_path varchar2, a_random_seed positiven) return varchar2;
147147

148+
procedure coverage_start(a_coverage_run_id raw);
149+
150+
procedure coverage_stop;
151+
148152
end ut_runner;
149153
/

0 commit comments

Comments
 (0)
0