MMAP inconsistent results #79
Description
The UAI 2014 Inference competition does not seem to provide reference solutions for the MMAP task.
Therefore, we are validating our results with those produced by the Merlin solver, which can be found here: https://github.com/radum2275/merlin.
The solutions produced by our tool and those generated by Merlin are inconsistent for several problems. It would be valuable to determine the reason for this discrepancy.
MWE:
using Revise, BenchmarkTools, Test, Artifacts
using TensorInference
import Pkg;
Pkg.ensure_artifact_installed("uai2014", "Artifacts.toml");
problem_sets = dataset_from_artifact("uai2014")["MMAP"]
problems = [
("Segmentation", 12, TreeSA(ntrials=1, niters=2, βs=1:0.1:40)),
("Segmentation", 13, TreeSA(ntrials=1, niters=2, βs=1:0.1:40)), # fails!
#("Segmentation", 14, TreeSA(ntrials=1, niters=2, βs=1:0.1:40)) # fails!
]
for (problem_set_name, id, optimizer) in problems
problem = problem_sets[problem_set_name][id]
model = MMAPModel(
read_model(problem); optimizer,
evidence=read_evidence(problem),
queryvars=read_queryvars(problem)
)
_, solution = most_probable_config(model)
test_result = @test solution == read_solution(problem)
println(test_result)
end