Ask a Judge, round 1: Crossing categories?
Posted by Kevin Keller | Posted in Contests & Judging, Uncategorized | Posted on June 14, 2012, 7:01 AM
5
From Stephanie Dingle:
How are members to handle situations when a judge, during evals, openly admits to crossing categories? A Presentation judge spends more time discussing singing or music … a Music judge saying there was a lack of musicality but I scored higher because of energy.
These are a few vague issues I’ve heard myself, but I still question the idea that each category can be judged independently of each other. But the manual does claim this is the process.
Response:
Thanks, Stephanie – you ask a great question and one that is often asked.
Ultimately, we all watch exactly the same performance, just with different filters on how we view and evaluate a performance. One of the distinct changes that was made 20 years ago, when we went from the four-category to three-category system, was that we recognized that there is significant overlap in the different viewpoints – each element influences the other.
At times over the past 20 years, the pendulum has swung a lot, from being able to say anything under the claim of “common ground” to “stay in your category.” Neither was appropriate, and I believe we’ve found a balance that recognizes some natural boundaries and yet recognizes that all components come together – that we can’t talk about our own view without recognizing the influence it has from others.
In nearly every performance, a group will score better if they sing better. So, if what I work on as a Music judge doesn’t help them sing better, I’m not helping them out the most. (My challenge is how do I help them and yet teach them something about music they didn’t know before that can help them?) And I’d say the same thing for the Presentation judge. So we’re all working on singing.
Overlap? It depends on how it is handled. We all expect that the Singing judge will talk about the mechanics of singing (breathing, posture, resonance, phonation, etc.). If a Music or Presentation judge acted as if he were a voice teacher, then that would be a potential issue. Even though many Presentation and Music judges are well-qualified to teach vocal techniques, we would expect them to approach their scoring and their evaluations from their own category’s viewpoint.
For example, a Presentation judge could recognize that part of the singing issue was lack of emotional connection; it was stilted, and perhaps that inhibited the singer from realizing the voice already inside him. The Presentation judge could get the group involved in the emotion and thereby improve the singing. A Music judge could talk about flow, and through that work get them singing better. It works in all the other directions as well, such as when a Singing judge discusses an interpretive choice (primarily the Music category’s domain) that affected a group’s singing quality and blend.
In my experiences as a Music judge, I have often asked “Who are you? What is the emotional story line?” Unlike a Presentation judge (who will help them explore a variety of emotions), I just want their musical delivery to match the emotional story line; and sometimes, what they choose isn’t supported by the arrangement. No other category will be able to provide that feedback. So, a good judge will live in all categories as he attempts to diagnose and help the group improve, but ultimately should provide some guidance that no other category will provide. People learn in different ways, and getting three voices giving you feedback from different perspectives on the same issue is valuable.
I guess I’ll have to re-read the manual to see if it really says that they are independent. If there is a passage or passages, let me know!
Kevin Keller (Society C&J Chairman)
Buy Kamagra Soft
Awesome as always, Mr. Keller!
Although the “Common Ground” concept generally disappeared as a specific, visible component of the judging sometime after the new categories were introduced, it fortunately remains in the sense of the category overlap. As Kevin noted, the fundamental of it all is good singing, and if your group is not in tune, whatever the reason, it detracts from the audience’s perception of the storytelling (Presentation) and the appreciation of the musical themes (Music).
Ask yourself the last time you heard a comedic group that didn’t sing well but executed good comedy and got the heartfelt laugh. How much more effective would the laugh be had the singing been better and not distracting? How much more musical would everything have been had the singing not only been in tune, but A-level ringing, capable of enhancing the impact of the comedy and raising the stature of the group in the eyes and ears of the audience?
That is why the categories overlap — because the best performance, in the eye of the non-judging audience, combines so many elements which need to be done well to achieve optimal performance.
Our categories merely reflect that, as they should.
Great discussion, so glad I rediscovered this site.
This is not a criticism but an observation. It seems that if categories overlapped evenly (picture a beautifully perfect Venn diagram) then the variance within each category would be about the same. However it seems that the largest differences (not counting penalties) are within PRS, followed by MUS, then SNG, with MUS and SNG being closer in variance.
However it also seems that the variance in PRS is almost always on the high end, and large variances are infrequent, which means that I don’t think that PRS is unduly influencing contest results. Except when it should (Senate-Aires for one). So maybe I see large variances in PRS and miss small but significant variances in MUS and SNG.
Does C&J ever study variance? Are there consistent correlations between category rankings and overall rankings or does one category impact results more than the others, and if that is discovered how would it be addressed?
-Steve
Hi Steve – good to hear from you.
Wow, what a question. The simple answer is yes. Meticulously.
We study the numbers for each contest for each judge. At the International level, you have enough to really make the assessments you make. And you would be correct in your assessment (the data is public so it’s easy enough to do your own study).
We’ve discussed this very issue at the SCJC level on several occasions.
We address it by looking at the scoring performance of the individual judge and looking for any sort of scoring issues that might crop up. There are essentially three measures that we look at. “Bias” – which is like it sounds – on average how far away from the panel average are you? Although we will say a judge scored a group low, they might have been riding low the entire contest. Although that doesn’t influence the ranks, it can influence wild cards so it’s important. We look at “range”, which is how consistent a judge is across a variety of scoring ranges. A judge might underscore higher groups and overscore lower groups. This judge has a low range. He has no influence on the results of hte contest. Meanwhile the judge who is punitive to lower groups and overscores higher groups has a large range and has too much influence on the contest. The final statistic is a standard deviation statistic that measures how large the errors are. The smaller the better. Those judges that you’ve encountered (in the past, of course!) that you feel are erratic will have a high stdev, while those who are “right there” will have a low stdev.
Bottomline is every category specialist reviews his judges and looks for any sort of anomolies and trends. Meanwhile, at a society level we examine the overall trends.
That was the not so simple answer…
Kevin Keller
C&J Chairman
Kevin, thanks for the response. I kind of forgot I posted here so am very late in responding. I’ve always wondered about ranges and influence so your response is fantasic; thanks for the info!