SystemVerilog Functional

Coverage
Dr. O. Ait Mohamed
ECE Department
Concordia university
Winter 2021
Objectives
• Explain the different flavors of coverage
• Identify the difference between functional and code coverage

3/25/21 Functional Coverage 2

Coverage Flavors
• Coverage helps you track the verification progress
• Coverage is a simulation metric we use to measure verification
progress and completeness

3/25/21 Functional Coverage 3

Code coverage
• Code coverage measures how much of the design code has been
tested è implementation coverage
• i.e, Code coverage measures how well you have tested features that
were implemented
è It can’t tell what features were never implemented

3/25/21 Functional Coverage 4

Functional Coverage
• Functional coverage (FC) is based on the verification specification
• How well the specification is tested?
• FC is a specification coverage

3/25/21 Functional Coverage 5

Example
• Assume we would like to report that part of the design was never
enabled?
• What is the difference between code and functional coverage?

Code coverage would show 100% coverage of the source code

If the design code is missing, code coverage won’t tell

Functional coverage reports only 50% coverage for this case as one
of the two modes of the project was not tested

3/25/21 Functional Coverage 6

Assertion coverage
• Based on SystemVerilog Assertions (SVA)
• Measures coverage of complex sequences of design behavior (such as
state machine flows)

3/25/21 Functional Coverage 7

Coverage Comparison
• Which coverage measurement is the most important?
• Functional Coverage
• What values were seen in certain key variables?
• è How well did you test the specification?
• Code coverage
• What lines and expressions in the DUT were exercised by testbench?
• è How well did you test the implementation?
• Both are important

Does high functional and

Functional Coverage

Need more FC Check your

high

points bug rate Code coverage mean that

you are done ?
Start of the Missing
project RTL?
low

low high
3/25/21 Code coverage Functional Coverage 8
Summary
• Explain the different flavors of coverage
• Identify the difference between functional and code coverage

3/25/21 Functional Coverage 9

Functional Coverage Strategies
• Explain the importance of the functional coverage
• Discuss the function coverage strategies
• Explain the functional coverage convergence

3/25/21 Functional Coverage 10

Why functional Coverage
• What caused the Pentium FDIV Bug?

3/25/21 Functional Coverage 11

Functional Coverage Strategies
• Gather information, not data
• What coverage would gather a FIFO?
• All combinations of head and tail pointers?
• For a 1K FIFO, this is a million coverage points
• What are important DUT “states”
• Empty, Partial, Full
• Pointer wraparound
• Errors such as underflow/overflow
• (This is not the same as state-machine coverage)
• Once you can describe design states, you should put these in the
verification plan to guide writing tests and measuring coverage
• Include design configuration tool
3/25/21 Functional Coverage 12
Functional Coverage Convergence
• Create initial constrained-random Constrained
testbench random test

• Perform many runs with different Many runs,

seeds different seeds
• Gather functional coverage to
measure progress
• Identify the coverage holes Directed Functional
Add test case Coverage
• Make minimal code changes constraints
• Such adding new constraints
• Run more tests to close in on goals
Identify
• When a case is too difficult to holes
reach, make a directed test
• And keep gathering coverage Minimal Code
modifications

3/25/21 Functional Coverage 13

What Can Coverage Tell You?
• SystemVerilog functional coverage provides:
• Coverage of variables and expressions
• Did simulation generate every possible value on data_bus?
• How many times was data_bus equal to 0?
• …..
• Cross coverage between variables and expressions
• How many times was opcode == DIV and operand_b == 0 ?
• Did simulation ever cause a read with cache_miss operation?
• Coverage of sequences
• How many times did two stack push operations occur in a row, with the almost_full
flag_set?
• …..

3/25/21 Functional Coverage 14

Summary
• Explain the importance of the functional coverage
• Discuss the functional coverage strategies
• Explain the functional coverage convergence

3/25/21 Functional Coverage 15

SystemVerilog Functional
Coverage Flow

3/25/21 Functional Coverage 16

Objectives
• Describe the functional coverage flow
• Visualize the functional Coverage with Questa

3/25/21 Functional Coverage 17

Functional Coverage Flow
• What are the steps to gathering and measuring coverage?
• From your verification plan, decide how to measure if a particular test has met its goal
• Not only functional correctness, but also the range of design states, data values, and timing
• This is covergroup
• For the given design state, what signals and expressions need to be monitored?
• These each form a coverpoint
• For each point, what are the values that
indicate that the goal has been met?
• These are a range of values: specific opcodes
or address ranges
• These are coverage bins
• You may need to look at combinations of
values concurrently
• This is cross coverage
• Instantiate covergroups in your testbench
3/25/21 Functional Coverage 18
Functional Coverage Flow
• What are the steps to gathering and measuring coverage?
• During simulation, when do you read the signal value to gather coverage?
• This is the sample for your covergroup è cg.sample();
• This is when a specific bin is incremented
• At the end of simulation, all the coverage data is written into a coverage database
• These can be merged with data from other passing tests
• Delete coverage results from tests that fail
• You run a reporting tool to make a human readable report from the data
• Reports can be text, HTML, displayed inside an interactive GUI, or in Accelera UCDB standard
• You need to analyze the results by hand to find the coverage holes and plan how to fill them
• Close in on 100% coverage, but rarely achieved
• Often these holes are not reachable by the testbench
• You an define exceptions and reanalyze
• With directed tests, most time is in planning, writing, and debugging each test
• With constrained-random stimulus and coverage-driven testing, you spent more time at a high
level, analyzing the coverage results to plan the next step.

3/25/21 Functional Coverage 19

Demos

3/25/21 Functional Coverage 20

Summary
• Describe the functional coverage flow
• Visualize the functional Coverage with Questa

3/25/21 Functional Coverage 21

SystemVerilog Covergroup
Definition
Objectives
• Define covergroups within modules and classes
• Set coverpoints on signals and expressions

3/25/21 Functional Coverage 23

Functional Coverage
• SystemVerilog provide specific language constructs for functional
coverage
• What are these constructs? And how you adapt them in your code?

3/25/21 Functional Coverage 24

Covering Transactions
• How should you gather coverage on transactions?
• Imagine you are conduction a traffic study and want to measure vehicle flow
• Where would you put the black box to record data about vehicles?
• If you put the box in every car, you need thousands of them
• Instead, put one at each important intersection
• Likewise, when gathering cover data, don’t put covergroups inside transaction
classes
• Typically, a simulation has thousands of transactions that are created, sent, destroyed
• Find “intersections” where transactions are sent and observed and gather data there
• Instantiate a few key covergroups instead of thousands
• Look for critical locations where you can sample both transactions and their context
• A covergroup in a transaction does not know about actions such as “sent” or “received”

3/25/21 Functional Coverage 25

Covergroup Definitions
• A covergroup is a user-defined type, similar to a class definition
• Encapsulates a set of coverage points
• Can be defined anywhere a function can be defined
• Like a class, a covergroup is just a definition and must be constructed
• A covergroup is passive and must be triggered to sample values
module video (…); Class Monitor;
enum {red, green, blue} colors; Transaction t; // bit [3:0] src,dst;
covergroup cg_video; covergroup cg_a; In classes, covergroups
COLORS : coverpoint colors; SRC: coverpoint t.src; are constructed without
endgroup DST: coverpoint t.dst; a handle name
cg_video cg_v; endgroup
initial begin function new();
cg_v = new (); cg_a = new ();
end endfunction
endmodule endclass
In modules and interfaces, covergroups are Class covergroups can only be constructed in
constructed the same way as class object the class constructor
3/25/21 Functional Coverage 26
Coverpoint definition
• A coverpoint is an expression that is covered-SystemVerilog tracks the
values
• Can be a net, variable or expression that is sampled periodically
• Encapsulates a set of coverage points
• Always give the coverpoint a name, using a statement label
• If a name is not specified, the simulator creates a name

logic [3:0] pixel_hue, base_addr, offset;

covergroup cg2;
HUE: coverpoint pixel_hue;
ADDR: coverpoint base_addr + offset;
endgroup

3/25/21 Functional Coverage 27

Summary
• Define covergroups within modules and classes
• Set coverpoint on signals and expressions

3/25/21 Functional Coverage 28

SystemVerilog Sampling and
Conditional Coverage
Objectives
• Explain how to gather coverage data
• Set a condition or guard for disabling coverage

3/25/21 Functional Coverage 30

Sampling
• As we learned, Covergroups are passive constructs that need to be
triggered to sample values
• When are coverage values gathered?
• The coverpoints in a group must be periodically sampled, all together

covergroup cg_values_coverage;
VALUES: coverpoint data {
bins low = { [0:127] };
bins high = { [128:255] };
}
endgroup

covergroup cg_state_coverage;
STATES: coverpoint state;
endgroup

3/25/21 Functional Coverage 31

Sampling
• When are coverage values gathered?
• The coverpoints in a group must be periodically sampled, all together
• On-demand sampling is done by calling the sample() method – Most common
approach
• Automatic sampling times can be defined within the covergroup
• Synchronously on a clock edge or asynchronously on a value change
covergroup cg_data; covergroup cg_a @(posedge clock)
DATA: coverpoint data ; DATA: coverpoint data;
Synchronous sampling
... ….
endgroup endgroup;
cg_data c1 = new();
task send_transaction(…);
…. covergroup cg_b @(state)
c1.sample(); DATA: coverpoint data;
…. Asynchronous sampling
endgroup;

3/25/21 Functional Coverage 32

On-demand sampling
Conditional Coverage
• How can you temporarily stop gathering coverage during reset?
• The iff construct can be used to specify a condition (guard) for when
coverage should be sampled
• Can be specified at several levels of coverage granularity
• On coverpoint definitions (affects all bins in the coverpoint)
• On a bins definition (affects only that bin)
• On cross definitions (affects only that cross coverage)

covergroup cg10
HUE: coverpoint pixel_hue iff (reset_n. == 1); If the guard expression evaluates to false at a
endgroup; sampling point, then no sample is taken

3/25/21 Functional Coverage 33

Summary
• Explain how to gather coverage data
• Set a condition or guard for disabling coverage

3/25/21 Functional Coverage 34

SystemVerilog Cover point
bins
Objectives
• Explain how to gather coverage for a group of variables, a single
variable and a single data value
• Describe the difference between automatic and user defined bins
• Discuss some performance considerations related to coverpoints

3/25/21 Functional Coverage 36

Coverpoints
• Coverpoints define signals or expressions that need to be monitored
or covered.

3/25/21 Functional Coverage 37

Automatic Coverpoint Bins
• How is coverage data sampled and stored?
• Each coverpoint contains one or more coverage bins (counters)
• Bins contain a count of how many times specific values occurred
• If you don’t specify any bins, they are automatically created for each possible
2-state value of the cover, up to a max of 64 bins
• At each sample, the coverpoint expression is sampled, and the corresponding
bin is incremented

3/25/21 Functional Coverage 38

User-defined Coverpoint Bins
• How can you define your own bins?
• Explicit coverpoint bins can be defined with a bins construct
• bins construct : Allows creating a separate bin for each value in the given range list or a single bin
for the entire range of values
• Defines a named counter for a set of values, enclosed in {}
• When the group is sampled, if the coverpoint value matches the set of values, the bin is
incremented
• The values covered by a bin can be:
• A list of one or more values, a fixed range of values, or an open-ended range of values

3/25/21 Functional Coverage 39

Performance
• Defining a large number of bins for a coverpoint can significantly slow
down simulation run time
• You need to BE VERY CAREFUL!
• Bin arrays (such as h[100] or i[]) can represent a large number of bins

• Don’t use multiple bins construct for the default bin è bins c[] = default;

• Multiple bins construct would create a separate counter for each individual
value, which could adversely affect simulation

3/25/21 Functional Coverage 40

Summary
• Explain how to gather cover coverage for a group of variables, a single
variable, and a single data value

• Describe the difference between automatic and user defined bins

• Discuss some performance considerations related to coverpoints

3/25/21 Functional Coverage 41

SystemVerilog Special
Coverage Bins
Objectives
• Specify coverpoint bins for a 4-state values

• Use wildcard bins

• Exclude certain values from the coverage

3/25/21 Functional Coverage 43

Special Coverage Bins
• How to specify coverpoint bins for 4-state values?

• How to exclude certain values from coverage?

3/25/21 Functional Coverage 44

Coverage Bins for 4-State Values and Don’t
Care Bits
• Automatic coverpoint bins are only created for each possible 2-state values of the
coverpoint
• A bins definition can specify bins for 4-state values
• Count how many times a signal was X or Z

• When a bins definition include an X or Z, the bin count should be only be incremented when the
sampled value has an X or Z in the same bit positions
• wildcard bins turns bits set to X,Z or ? Into don’t care values
• Only 2-state values in the don’t care bit positions are counted; X and Z values are excluded from
the coverage

3/25/21 Functional Coverage 45

Excluded and Illegal Coverage Point Values
• How can you flag special treatment for certain values ?
• Values tagged as ignore_bins are excluded from a coverage point
• Excluded values are not counted and have no effect on coverage percentage
• Values tagged as illegal_bins are designated as illegal
• A run-time error is issued if the illegal value occurs
• Illegal values are excluded from coverage goal

3/25/21 Functional Coverage 46

Sample Coverage Report

3/25/21 Functional Coverage 47

Summary
• Specify coverpoint bins for 4-state values

• Use wildcard bins

• Exclude certain values from the coverage

3/25/21 Functional Coverage 48

SystemVerilog Cross
Coverage
Objectives
• Measure the cross coverage between two or more variables

3/25/21 Functional Coverage 50