10000 refactor: Changed the way reasons are being returned from decide APIs by zashraf1985 · Pull Request #279 · optimizely/ruby-sdk · GitHub
[go: up one dir, main page]
Skip to content

refactor: Changed the way reasons are being returned from decide APIs #279

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 17 commits into from
Jan 7, 2021
Merged

refactor: Changed the way reasons are being returned from decide APIs #279

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
c4bdcb2
fixed for bucketer
zashraf1985 Dec 15, 2020
7adb387
fixed get_bucketing_id
zashraf1985 Dec 15, 2020
055ef6f
fixed get_variation_for_feature_rollout
zashraf1985 Dec 15, 2020
5d11886
fixed get_forced_variation
zashraf1985 Dec 16, 2020
7cbf993
fixed a few more methods
zashraf1985 Dec 16, 2020
37bf64d
fixed get_variatopm
zashraf1985 Dec 16, 2020
ce727a9
fixed get_variation_for_feature_experiment
zashraf1985 Dec 16, 2020
63525e6
fixed get_variation_for_feature
zashraf1985 Dec 16, 2020
514a93d
added assertions for reasons in bucketer
zashraf1985 Dec 17, 2020
f13d883
added assertions for reasons for get_variation_for_feature_rollout
zashraf1985 Dec 17, 2020
2c6533a
added assertions for reasons everywhere
zashraf1985 Dec 18, 2020
ef8ddd5
added reasons to another missing method and updated unit tests
zashraf1985 Dec 19, 2020
a71680f
fixed an argument error
zashraf1985 Dec 22, 2020
af181e3
removed an unwanted message fron decide reasons
zashraf1985 Dec 22, 2020
5720f83
removed another unwanted reason
zashraf1985 Dec 22, 2020
dd4add4
removed unnecessary bucketing reason
zashraf1985 Dec 22, 2020
87894b3
fixed a log message
zashraf1985 Dec 22, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
added reasons to another missing method and updated unit tests
  • Loading branch information
@zashraf1985
zashraf1985 committed Dec 19, 2020
commit ef8ddd52db04292ea166a0e9ebc5864ff1fda540
58 changes: 19 additions & 39 deletions lib/optimizely/audience.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,33 +38,24 @@ def user_meets_audience_conditions?(config, experiment, attributes, logger, logg
# This defaults to experiment['key'].
#
# Returns boolean representing if user satisfies audience conditions for the audiences or not.
decide_reasons = []
logging_hash ||= 'EXPERIMENT_AUDIENCE_EVALUATION_LOGS'
logging_key ||= experiment['key']

logs_hash = Object.const_get "Optimizely::Helpers::Constants::#{logging_hash}"

audience_conditions = experiment['audienceConditions'] || experiment['audienceIds']

logger.log(
Logger::DEBUG,
format(
logs_hash['EVALUATING_AUDIENCES_COMBINED'],
logging_key,
audience_conditions
)
)
message = format(logs_hash['EVALUATING_AUDIENCES_COMBINED'], logging_key, audience_conditions)
logger.log(Logger::DEBUG, message)
decide_reasons.push(message)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This message should not be in reasons - no helpful info.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


# Return true if there are no audiences
if audience_conditions.empty?
logger.log(
Logger::INFO,
format(
logs_hash['AUDIENCE_EVALUATION_RESULT_COMBINED'],
logging_key,
'TRUE'
)
)
return true
message = format(logs_hash['AUDIENCE_EVALUATION_RESULT_COMBINED'], logging_key, 'TRUE')
logger.log(Logger::INFO)
decide_reasons.push(message)
return true, decide_reasons
end

attributes ||= {}
Expand All @@ -80,39 +71,28 @@ def user_meets_audience_conditions?(config, experiment, attributes, logger, logg
return nil unless audience

audience_conditions = audience['conditions']
logger.log(
Logger::DEBUG,
format(
logs_hash['EVALUATING_AUDIENCE'],
audience_id,
audience_conditions
)
)
message = format(logs_hash['EVALUATING_AUDIENCE'], audience_id, audience_conditions)
logger.log(Logger::DEBUG, message)
decide_reasons.push(message)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good! I think this message should BE in reasons since it shows "audience_conditions".


audience_conditions = JSON.parse(audience_conditions) if audience_conditions.is_a?(String)
result = ConditionTreeEvaluator.evaluate(audience_conditions, evaluate_custom_attr)
result_str = result.nil? ? 'UNKNOWN' : result.to_s.upcase
logger.log(
Logger::DEBUG,
format(logs_hash['AUDIENCE_EVALUATION_RESULT'], audience_id, result_str)
)
message = format(logs_hash['AUDIENCE_EVALUATION_RESULT'], audience_id, result_str)
logger.log(Logger::DEBUG, message)
decide_reasons.push(message)

result
end

eval_result = ConditionTreeEvaluator.evaluate(audience_conditions, evaluate_audience)

eval_result ||= false

logger.log(
Logger::INFO,
format(
logs_hash['AUDIENCE_EVALUATION_RESULT_COMBINED'],
logging_key,
eval_result.to_s.upcase
)
)
message = format(logs_hash['AUDIENCE_EVALUATION_RESULT_COMBINED'], logging_key, eval_result.to_s.upcase)
logger.log(Logger::INFO, message)
decide_reasons.push(message)

eval_result
[eval_result, decide_reasons]
end
end
end
13 changes: 10 additions & 3 deletions lib/optimizely/decision_service.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,9 @@ def get_variation(project_config, experiment_key, user_id, attributes = nil, dec
end

# Check audience conditions
unless Audience.user_meets_audience_conditions?(project_config, experiment, attributes, @logger)
user_meets_audience_conditions, reasons_received = Audience.user_meets_audience_conditions?(project_config, experiment, attributes, @logger)
decide_reasons.push(*reasons_received)
unless user_meets_audience_conditions
message = "User '#{user_id}' does not meet the conditions to be in experiment '#{experiment_key}'."
@logger.log(Logger::INFO, message)
decide_reasons.push(message)
Expand Down Expand Up @@ -242,8 +244,10 @@ def get_variation_for_feature_rollout(project_config, feature_flag, user_id, att
rollout_rule = rollout_rules[index]
logging_key = index + 1

user_meets_audience_conditions, reasons_received = Audience.user_meets_audience_conditions?(project_config, rollout_rule, attributes, @logger, 'ROLLOUT_AUDIENCE_EVALUATION_LOGS', logging_key)
decide_reasons.push(*reasons_received)
# Check that user meets audience conditions for targeting rule
unless Audience.user_meets_audience_conditions?(project_config, rollout_rule, attributes, @logger, 'ROLLOUT_AUDIENCE_EVALUATION_LOGS', logging_key)
unless user_meets_audience_conditions
message = "User '#{user_id}' does not meet the audience conditions for targeting rule '#{logging_key}'."
@logger.log(Logger::DEBUG, message)
decide_reasons.push(message)
Expand All @@ -266,8 +270,11 @@ def get_variation_for_feature_rollout(project_config, feature_flag, user_id, att
# get last rule which is the everyone else rule
everyone_else_experiment = rollout_rules[number_of_rules]
logging_key = 'Everyone Else'

user_meets_audience_conditions, reasons_received = Audience.user_meets_audience_conditions?(project_config, everyone_else_experiment, attributes, @logger, 'ROLLOUT_AUDIENCE_EVALUATION_LOGS', logging_key)
decide_reasons.push(*reasons_received)
# Check that user meets audience conditions for last rule
unless Audience.user_meets_audience_conditions?(project_config, everyone_else_experiment, attributes, @logger, 'ROLLOUT_AUDIENCE_EVALUATION_LOGS', logging_key)
unless user_meets_audience_conditions
message = "User '#{user_id}' does not meet the audience conditions for targeting rule '#{logging_key}'."
@logger.log(Logger::DEBUG, message)
decide_reasons.push(message)
Expand Down
99 changes: 62 additions & 37 deletions spec/audience_spec.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,27 +34,25 @@
expect(Optimizely::Audience.user_meets_audience_conditions?(config,
experiment,
user_attributes,
spy_logger)).to be true
spy_logger)[0]).to be true

# Audience Ids exist but Audience Conditions is Empty
experiment = config.experiment_key_map['test_experiment']
experiment['audienceIds'] = ['11154']
experiment['audienceConditions'] = []

expect(Optimizely::Audience.user_meets_audience_conditions?(config,
experiment,
user_attributes,
spy_logger)).to be true
user_meets_audience_conditions, reasons = Optimizely::Audience.user_meets_audience_conditions?(config, experiment, user_attributes, spy_logger)
expect(user_meets_audience_conditions).to be true
expect(reasons).to eq(["Evaluating audiences for experiment 'test_experiment': [].", "Audiences for experiment 'test_experiment' collectively evaluated to TRUE."])

# Audience Ids is Empty and Audience Conditions is nil
experiment = config.experiment_key_map['test_experiment']
experiment['audienceIds'] = []
experiment['audienceConditions'] = nil

expect(Optimizely::Audience.user_meets_audience_conditions?(config,
experiment,
user_attributes,
spy_logger)).to be true
user_meets_audience_conditions, reasons = Optimizely::Audience.user_meets_audience_conditions?(config, experiment, user_attributes, spy_logger)
expect(user_meets_audience_conditions).to be true
expect(reasons).to eq(["Evaluating audiences for experiment 'test_experiment': [].", "Audiences for experiment 'test_experiment' collectively evaluated to TRUE."])
end

it 'should pass conditions when audience conditions exist else audienceIds are passed' do
Expand Down Expand Up @@ -85,15 +83,25 @@
allow(Optimizely::CustomAttributeConditionEvaluator).to receive(:new).and_call_original

# attributes set to empty dict
expect(Optimizely::Audience.user_meets_audience_conditions?(config,
experiment,
{},
spy_logger)).to be false
user_meets_audience_conditions, reasons = Optimizely::Audience.user_meets_audience_conditions?(config, experiment, {}, spy_logger)
expect(user_meets_audience_conditions).to be false
expect(reasons).to eq([
"Evaluating audiences for experiment 'test_experiment_with_audience': [\"11154\"].",
"Starting to evaluate audience '11154' with conditions: [\"and\", [\"or\", [\"or\", {\"name\": \"browser_type\", \"type\": \"custom_attribute\", \"value\": \"firefox\"}]]].",
"Audience '11154' evaluated to UNKNOWN.",
"Audiences for experiment 'test_experiment_with_audience' collectively evaluated to FALSE."
])

# attributes set to nil
expect(Optimizely::Audience.user_meets_audience_conditions?(config,
experiment,
nil,
spy_logger)).to be false
user_meets_audience_conditions, reasons = Optimizely::Audience.user_meets_audience_conditions?(config, experiment, nil, spy_logger)
expect(user_meets_audience_conditions).to be false
expect(reasons).to eq([
"Evaluating audiences for experiment 'test_experiment_with_audience': [\"11154\"].",
"Starting to evaluate audience '11154' with conditions: [\"and\", [\"or\", [\"or\", {\"name\": \"browser_type\", \"type\": \"custom_attribute\", \"value\": \"firefox\"}]]].",
"Audience '11154' evaluated to UNKNOWN.",
"Audiences for experiment 'test_experiment_with_audience' collectively evaluated to FALSE."
])

# asserts nil attributes default to empty dict
expect(Optimizely::CustomAttributeConditionEvaluator).to have_received(:new).with({}, spy_logger).twice
end
Expand All @@ -104,10 +112,12 @@
'test_attribute' => 'test_value_1'
}
allow(Optimizely::ConditionTreeEvaluator).to receive(:evaluate).and_return(true)
expect(Optimizely::Audience.user_meets_audience_conditions?(config,
experiment,
user_attributes,
spy_logger)).to be true
user_meets_audience_conditions, reasons = Optimizely::Audience.user_meets_audience_conditions?(config, experiment, user_attributes, spy_logger)
expect(user_meets_audience_conditions).to be true
expect(reasons).to eq([
"Evaluating audiences for experiment 'test_experiment': [].",
"Audiences for experiment 'test_experiment' collectively evaluated to TRUE."
])
end

it 'should return false for user_meets_audience_conditions? when condition tree evaluator returns false or nil' do
Expand All @@ -118,17 +128,22 @@

# condition tree evaluator returns nil
allow(Optimizely::ConditionTreeEvaluator).to receive(:evaluate).and_return(nil)
expect(Optimizely::Audience.user_meets_audience_conditions?(config,
experiment,
user_attributes,
spy_logger)).to be false

user_meets_audience_conditions, reasons = Optimizely::Audience.user_meets_audience_conditions?(config, experiment, user_attributes, spy_logger)
expect(user_meets_audience_conditions).to be false
expect(reasons).to eq([
"Evaluating audiences for experiment 'test_experiment_with_audience': [\"11154\"].",
"Audiences for experiment 'test_experiment_with_audience' collectively evaluated to FALSE."
])

# condition tree evaluator returns false
allow(Optimizely::ConditionTreeEvaluator).to receive(:evaluate).and_return(false)
expect(Optimizely::Audience.user_meets_audience_conditions?(config,
experiment,
user_attributes,
spy_logger)).to be false
user_meets_audience_conditions, reasons = Optimizely::Audience.user_meets_audience_conditions?(config, experiment, user_attributes, spy_logger)
expect(user_meets_audience_conditions).to be false
expect(reasons).to eq([
"Evaluating audiences for experiment 'test_experiment_with_audience': [\"11154\"].",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This debugging message should be removed from reasons.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

"Audiences for experiment 'test_experiment_with_audience' collectively evaluated to FALSE."
])
end

it 'should correctly evaluate audience Ids and call custom attribute evaluator for leaf nodes' do
Expand Down Expand Up @@ -213,10 +228,13 @@
}
experiment['audienceIds'] = %w[11110]

expect(Optimizely::Audience.user_meets_audience_conditions?(config,
experiment,
user_attributes,
spy_logger)).to be false
user_meets_audience_conditions, reasons = Optimizely::Audience.user_meets_audience_conditions?(config, experiment, user_attributes, spy_logger)
expect(user_meets_audience_conditions).to be false
expect(reasons).to eq([
"Evaluating audiences for experiment 'test_experiment_with_audience': [\"11110\"].",
"Audiences for experiment 'test_experiment_with_audience' collectively evaluated to FALSE."
])

expect(spy_logger).to have_received(:log).once.with(
Logger::DEBUG,
"Evaluating audiences for experiment 'test_experiment_with_audience': " + '["11110"].'
Expand All @@ -236,10 +254,17 @@
experiment['audienceIds'] = %w[11154 11155]
experiment['audienceConditions'] = nil

expect(Optimizely::Audience.user_meets_audience_conditions?(config,
experiment,
user_attributes,
spy_logger)).to be false
user_meets_audience_conditions, reasons = Optimizely::Audience.user_meets_audience_conditions?(config, experiment, user_attributes, spy_logger)
expect(user_meets_audience_conditions).to be false
expect(reasons).to eq([
"Evaluating audiences for experiment 'test_experiment_with_audience': [\"11154\", \"11155\"].",
"Starting to evaluate audience '11154' with conditions: [\"and\", [\"or\", [\"or\", {\"name\": \"browser_type\", \"type\": \"custom_attribute\", \"value\": \"firefox\"}]]].",
"Audience '11154' evaluated to UNKNOWN.",
"Starting to evaluate audience '11155' with conditions: [\"and\", [\"or\", [\"or\", {\"name\": \"browser_type\", \"type\": \"custom_attribute\", \"value\": \"chrome\"}]]].",
"Audience '11155' evaluated to UNKNOWN.",
"Audiences for experiment 'test_experiment_with_audience' collectively evaluated to FALSE."
])

expect(spy_logger).to have_received(:log).once.with(
Logger::DEBUG,
"Evaluating audiences for experiment 'test_experiment_with_audience': " + '["11154", "11155"].'
Expand Down
Loading
0