Rhythm is the most basic skill for people learning to dance. Beginners need practice but also close coaching and constant feedback. However, in most dance classes teachers often find challenging to provide attention to each student. A possible solution to this problem would be to automate the provision of feedback to students by objectively assessing rhythm from their movement data. But how effective would a fully automated approach be compared to dance experts in evaluating dance performance? We conducted a study aimed at exploring this by 'measuring' dance rhythm from accelerometer data streams and contrasting the algorithm results with expert human judgement. We developed RiMoDe, an algorithm that tracks bodily rhythmic skills, and gathered a dataset that includes 282 independent evaluations made by expert dance teachers on 94 dance exercises performed by 7 dance students. Our findings revealed major gaps between a purely algorithmic approach and how experts evaluate dance rhythm. We identified 6 themes that are important when assessing rhythm. We discuss how these themes should be considered and incorporated into future systems aimed at supporting people learning to dance.