Forgot what I said earlier about possible solution (but read). It seems I understood (I am still not sure that found the problem but). I used the color dots instead of the numbers because it was faster.
First step is to mark all the pixels as transparent or filled
Second step is to extract concrete shapes and give them appropriate numbers
Then do the the second step but for holes. There is a problem. You store the data of all the contours in one matrix (I base this assumption at the image from your article). When you start to mark the holes, you don't trait the pixels of six as part of the hole so you end up with 3 different holes in the circle. When you start to mark the holes you need to do it in a bit another way. You need to understand for which shape this hole belongs. When you create the hole you need to check if the pixel doesn't belong to that shape rather then just checking if the pixel is empty...
In the case if we check if something is not red so the blue,pink,black,green and yellow will be treated as one hole in the case when we check if it isn't filled we have 4 different holes: pink,black,blue and yellow. But yellow makes the hole inside green shape which will be also added to the output file.