8000 reproducible orca running image tests by antoinerg · Pull Request #3237 · plotly/plotly.js · GitHub
[go: up one dir, main page]
Skip to content

reproducible orca running image tests #3237

New issue
8000

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 35 commits into from
Closed

reproducible orca running image tests #3237

Changes from 1 commit
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
dfcb720
test software rendering on CI
antoinerg Nov 8, 2018
f79008a
add test-image-orca to workflow
antoinerg Nov 8, 2018
b0e4789
do no checkout source for test-image-orca
antoinerg Nov 8, 2018
75a9fcd
do checkout source for test-image-orca
antoinerg Nov 8, 2018
822a7df
baselines from local dev machine
antoinerg Nov 9, 2018
3cb8631
compare test images with baselines
antoinerg Nov 9, 2018
362769d
shuffle list of mocks to even out time of calculation per chunk
antoinerg Nov 9, 2018
eb4cde3
remove previous test-image jobs
antoinerg Nov 9, 2018
6bc3549
shuffle in a deterministic fashion
antoinerg Nov 9, 2018
02abebb
print out level of parallelism
antoinerg Nov 9, 2018
c360099
fix missing whitespace
antoinerg Nov 9, 2018
ca939fd
replace env variable CI with CIRCLECI
antoinerg Nov 9, 2018
a0f3d38
split work across nodes
antoinerg Nov 9, 2018
3aa1244
add parallelism
antoinerg Nov 9, 2018
d6543bf
bump parallelism to 8
antoinerg Nov 9, 2018
a21d0e7
reduce no_output_timeout to detect when orca freezes
antoinerg Nov 9, 2018
d23f8ed
support 3 digit parallelism number
antoinerg Nov 9, 2018
e178fd1
bump parallelism to 12
antoinerg Nov 9, 2018
cb52e30
update a few baselines with reproducible ones
antoinerg Nov 9, 2018
4a44a98
set parallelism back to 8
antoinerg Nov 9, 2018
b248309
retry if orca hangs
antoinerg Nov 12, 2018
b9ea11a
fix executable path
antoinerg Nov 12, 2018
18ba1a3
fix retry logic
antoinerg Nov 12, 2018
dff6773
orca-build-verify.sh accepts mock's name as command line argument
antoinerg Nov 12, 2018
d05fed0
double-check images that fail
antoinerg Nov 12, 2018
5a88f9d
comment runner code
antoinerg Nov 12, 2018
d86fa93
fix image paths
antoinerg Nov 12, 2018
bb0a91c
use plotly.js from build folder
antoinerg Nov 14, 2018
8dfe096
update baseline using current branch's plotly.js
antoinerg Nov 14, 2018
835e588
update another using currently built plotly.js
antoinerg Nov 14, 2018
ba050a0
use orca with MathJax support
antoinerg Nov 14, 2018
799d64a
delete diff images that perfectly match
antoinerg Nov 14, 2018
b3b56c3
update baselines containing MathJax now that it's enabled
antoinerg Nov 14, 2018
9779c46
update baselines after installing missing fonts
antoinerg Nov 14, 2018
70ea751
update baselines with pixelRatio forced to 2
antoinerg Nov 15, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
delete diff images that perfectly match
  • Loading branch information
@antoinerg
antoinerg committed Nov 14, 2018
commit 799d64a624fe5a34c15c422116c2fe6aa97d070f
10 changes: 10 additions & 0 deletions .circleci/orca-build-verify.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,14 @@ function compare()
}
compare $NODE_QUEUE

function delete_perfect_matches()
{
grep -l -R "all: 0" $DIFF_IMAGES/ | gawk '{match($0, /diff\/([^.]*)/, arr); print arr[1]}' > /tmp/succeeded
cat /tmp/succeeded | \
xargs -n1 -I {} rm $DIFF_IMAGES/{}.png $DIFF_IMAGES/{}.txt
}
delete_perfect_matches

CODE=$(grep -R "all: [^0]" $DIFF_IMAGES/ | wc -l)

# If there are different images, retry
Expand All @@ -93,7 +101,9 @@ if [ "$CODE" -ne "0" ]; then
cat /tmp/failed
generate /tmp/failed
compare /tmp/failed
delete_perfect_matches
CODE=$(grep -R "all: [^0]" $DIFF_IMAGES/ | wc -l)
fi

echo "$CODE different images"
exit $CODE
0