More on how you judge stunt

The thread by Peter Germann brought out some good comments on judging and I got a couple of emails about more detail on scoring and how do I set my point values. Now understand this is an art not a science and every good judge has to develop their own style and keep improving it. Well here goes a try at explaining how I approach it.
The three basic elements that I consider are shape, size and position. The scores for these as a group are done in my head and a number comes up without much conscious effort. The brain’s capability to give you that number comes from comparison, study and experience. It is automatic – the more study, experience, practice and confidence the better the number. The more you put everything out of your mind and concentrate only on the maneuver in front of you the better the number.
The principals I use for evaluating. Shape is the most important with size and position being equal but lesser value than shape. Errors in these basics cost a flyer major points:
Shape - is it a good picture of the required geometry - smooth, round/square proper geometry with sharp corners and straight runs where required. Multiple maneuvers should have consistent overlays and intersections.
Size - is it at the required angular size - 0, 45, 90, etc.
Position - is it in the proper position relative to the level flight plane with bottoms at 5’ or “0” degrees and are 90’s directly overhead.

There are other factors that make up the remainder of the point award. The rule book lists brief small errors or deviations from the prescribed flight path that should be deductions these are kinks, wobbles, etc. Errors here are in the 1 or 2 point range and I do a quick deduct for them.

I know Brett doesn’t agree but another factor is Lack of consistency in hitting heights and sizes from maneuver to maneuver. This is not addressed directly but can be derived from the rulebook requirement for doing maneuvers to approximately the same angular dimensions and to judge them on the degree of perfection with which they are performed.
I take this to include level flight or 0 degrees, 45 degrees, etc. This means for example holding at exactly 5’ in level flight is better than wandering from 4’ to 6’ in the tolerance zone. The same goes for 45 degree heights. The reason for this is most expert flyers and top 20 flyers are easily capable of holding to 4’ to 6’ but there are some who can hold exactly 5’ lap after lap and hit 45 degrees exactly maneuver after maneuver. All else being equal these are the better flyers and should be scored accordingly.
This is added as I go through the flight as consistency makes me more generous as the flight progresses.
Giving 40 points to a maneuver would require satisfying all of the above criteria and that does not happen very often.

Another concern in getting experience is to compare your scoring range and point spread against others. This has to come from experience. There is nothing wrong with being a high judge or low judge within reason as long as you are not in a situation where hi/lo scores are being tossed out. Also there is nothing wrong, within reason, in having a narrow point spread or broad point spread as long as you are tracking well on placement. To assist getting a handle on this I have included a page out of the Stunt Book that I used in training judges. The whole Stunt Book is available on the PAMPA web site.
Let’s hear from some others on how you judge.

Regards,
Dave Cook


[photo not recovered: 26662.jpg]

Attachment #1, (jpg file)

Dave,
My little chart that I use for judging gives me a point of reference to relate to the 10-40 point system. Very similiar to yours. The A,B,C,... system is more intutive to me.


[photo not recovered: 26663.jpg]

Attachment #1, (jpg file)

Frank,
Good chart, I like it and I'm sure it will be used.

Dave Cook

I debated and finally decided tha tI have to say something. I know you commented about the "expected"range for scores, I really would resist any predisposition that an INtermidiate pilot will do a manuever within a certain range. (or advanced or expert for that matter) I think it can lead to a train of thought that is dangerous. I fly intermidiate, I think most of the reason is because certain manuevers are bad, not that all my manuevers are equally bad. I just have a tender spot with regards to this. I have seen inter and even beginner pilots make stellar take offs and landings and get 25 for them, whereas an expert had a noticable bobble or two in the approach and gets 32 or 35. It does cause one to wonder.
Please note I AM NOT saying that I feel unfairly judged per se', just that it really can lead to some misconceptions about it. I personally would rather not see that reference on there.
thanks for listening
respectfully

Mark, Absolutely correct! The class be it beginner, advanced, or expert has absolutely no bearing on the quality of the maneuver. All maneuvers are to be judged as to how well they were executed as per the book, not the class. As you said if a beginner makes a 40 point takeoff or landing then that is the score he/she should get. I also believe that if you see a 40 point maneuver then give the contestant 40 points. I know some will argue that an advanced pilot should not get more then 32 points or whatever cause they save them for the experts. They say only experts can get higher points. They are flatowrong and should not be allowed to judge. Peut riod. RJ

Mark & Bob,
Good point and it is exactly why we put the cautionary note on the chart that a beginner could throw a 40 pointer at you at any time. But they will not be doing that all the time or they belong in a higher class. These are average scores and caution judges that they should not prejudges any flyer - this is why we have classes.We also emphasize this in the Stunt Book.

Thanks,
Dave Cook

Sir, Yessir, Sir! I just wanted to be sure that I understood what I thought I read and I am happy I got it right. Bob Whitely

On Peter's F2B site discussing whether they should have a trial that drops the K factor, we've been talking about judging on a curve. I think typically, F2B judges have tried to look at a maneuver as perfect, then deduct points for mistakes. Some have described a rather complicated formula they use to determine their final score for a maneuver. Ted noted that this was, in fact, just that: complicated. He (and I) talked about using a sort of modified curve for judging. (my thought, not Ted's) Look at a maneuver and compare it against both the rule book and other fliers of the day. Where does it fall? A sort of ranking system. Then apply a score based on both references. A lot harder to explain than to do.

It simplifies judging to some extent. I noted that a trained person can (if he has any sort of skills in pattern recognition or shape metrics) can determine minute changes in pattern or path and it isn't too hard to look a a flight and figure out where each maneuver falls in comparison to both the rule book ideal and in comparison to other pilots judged.

The tough part is deciding if a maneuver is worth 32 or 33 points, but if you look at it this way (both from a curve view and against the rule book), it becomes somewhat easier. Not tremendously, but somewhat.

-------------------------------------------------------------------------------------------------------
Some days it seems like a coin toss between George Orwell and Monty Python...

When I judge, I don't remember much about previous flights so I have a problem with doing the curve. When in doubt on 32 vs 33, I will sometimes look at the score I gave on the previous maneuver and decide if this one is better, the same, or worse.

Mark has a good point. But there is a bit more to it. As in when we are faced with a too-small judging cadre during a local contest the CD begins foraging at the bottom of the pool. Where "judges" like me swim in defensive swarms.

It kinda comes down to being a little more careful in what you wish for. Talking a guy into holding a clipboard and pencil for a few rounds doesn't equal stellar judging. Or even competent judging.

While I have since cleaned up my act somewhat I will readily admit that the first two or three times I was coerced into pulling judging duties I did some mental calculations on the way to the circle and so had scores in mind. Yes, well in advance of the first engine start.

Was that fair to the contestants? Of course not! But I did make it clear my JJ (Junior Judge) status right up front.

On the other hand, while some goof like Mark may have driven a mere 6 hours to see his righteous take-off squeak only an Intermediate-quality score from me, I never felt my scores failed in determining the correct ranking of the contestants. Even during the first contest in which I judged.

Does that make my (early) approach correct? Of course not.

Another point: While I do not recall seeing such take place here in the NW, I do recall hearing of situations at local California contests where a single judge was used. Okay, it seems as if I only remember Brett doing this--maybe we can get some additional input here...

Still, isn't it better to get *one* judge who is a known quantity than, say, two of me?

To put a finer point on it, other than the NW Regionals, I would have absolutely no problem flying to a single skilled judge during any NW contest, during any event, including Expert PA.

The list would include: Steve Helmick, Dave Gardner, Mike Conner, John Thompson, Paul Walker, Joan Cox, Chris Cox, Bob Parker.

Apologies to those left off the list--I'm fixin' to eat dinner shortly--but the one-judge concept seems acceptable to me and I would like to hear from others on the subject.

Dan

Then, there's the Judge who only awards "prime numbers"...

Like Randy's example, sometimes I agonize over giving a (say) 32, which was what I "flashed on". Or maybe upgrading a point, or downgrading a point. I think one of our local curmudgeons gives .5 points in these cases. I just default to the original "flashed" number. With 10-40 points, I think whole numbers are good enough, and I'd actually like to see that spelled out in the rulz. I'd really take a dim view of somebody giving 31.4375 points for the OH8, if I was a tabulator, tho I don't see any rule against it. With the FAI 1-10 (which is actually 0-10, the way I see it)...then 10ths are expected...but I'd lobby for 10-100, with no decimals allowed. Decimals are BAD! I deal with 4 digits to the right of the decimal every day, so trust me on this!

I listened to Bob Parker, when he told me that 25 points was the average (aka, "mathematical mean" of 10 - 40 (10 + 40 = 50, 50/2 = 25, for the math challenged). I tend to think of "average" maneuvers to be recognizeable, but not very enjoyable to watch.

Sometimes there is a trick flown that just "sparkles"...I give that 38 to 40 points, and am perfectly willing to give 40 points to any skill class flier, if they deserve it. Sometimes a single entrant in Beginner or Intermediate might get a generous score from me, for encouragement...but I hope it doesn't go to their heads... Steve

Edit: for Spelling

Steve Helmick
Renton, Washington
USA

To start off with, since I have flown in front of Dan and Steve on numerous occasions, ( and likely will again this year!) I am releived that neither of you apparantly has taken my comments as offensive or personal, whew,, would hate to see my scores then huh anyway, on to the point, Dan I agree wholeheartedly, despite my comments, I believe that at the end of the day, the correct person wins so ultimately, the number isn't all that important. UNLESS you are a bottom feeding, praise hungry, crash happy, on the verge of demoralized some times Intermediate. (ok so I really haven't ever been demoralized except maybe Portland last year when I lost my brand new plane to a flippin dandelion )The only time it really bothers me is if I am flying classic or profile against Advanced or expert fliers and THEN get "Intermidiate" scores when I know my manuever was pretty solid. Just to be clear, It is not a common occurance, actually only think I can remember two times when I thought to myself, "self, you got hosed, that was a rockin *fill in th blank** .Ok well maybe when I really smoke a takeoff for my intermidiate flights and get 25 it does get under my skin just a little, but as long as the right person ends up taking home that prestige laden trophy, the fat check and the dancing girls then all is well in the world.

Guys, we really need the flying season to be longer, sigh, Im going stir crazy, maybe its time to take up indoor FF too?

>>Or just move to Tucson. <<

Yea, yea. When I'm down there in March, talk my wife into it.

-------------------------------------------------------------------------------------------------------
Some days it seems like a coin toss between George Orwell and Monty Python...

Re scoring "ranges" for skill classes.

I think I take a more comprehensive opposition to the concept than have others on this thread. Comments that anyone can "throw a 40 pointer" at you is certainly true but, in my estimation, that isn't the issue regarding differing bottom lines (the one that goes on the scoreboard) between skill classes.

It is my feeling that the differences in the bottom line come at least as much as the result of better performance on easier maneuvers than on hard ones as it does on uniform inferiority on all maneuvers. In other words, I think Advanced flyers are perfectly capable of flying takeoffs, round maneuvers and landings as well as anyone (maybe not as consistently, but regularly). Scores on such maneuvers should reflect the same degree of excellence on the score sheet as those done by Expert flyers.

I believe where at least 50% of the difference in the bottom line comes from is the poorer consistent performance on hard maneuvers.

Thus, I would expect Expert flyers to get generally consistent numbers on the entire range of maneuvers with only little difference between hard and easy tricks (and sometimes superior numbers on particularly well done "hard" ones). The result would predictably be a narrow range of maneuver scores.

I would equally expect that the Advanced flyers' inferior bottom lines would reflect much less uniformity and include a much broader range of scores, reflecting some done as well as many expert maneuvers but most "hard" maneuvers scoring notably less.

Ditto for Intermediate flyers; scores ranging from occasional "equal to those of the Advanced (and even, once in a while, Expert) flyers but with even lower scores for the hard ones".

Ultimately, the difference between Experts and Intermediates should be reflected in a widening of the range of scores assigned to the various maneuvers ... NOT from a *lower but equally narrow* "range" of scores as those given to Experts.

What worries me about "training" judges to use narrow ranges of scores based on "assumptions about what those scores 'should' be is that they will literally lock into the "range" regardless of the quality of the maneuver.

I think a better approach to training for cross skill class events would be to emphasize the likely need to "expect" (not limit the judges to, but expect) the range of scores given in lower classes to reflect a wider percentage of the thirty points available. I would prefer that "expected ranges of scores for a given skill class" never be suggested.

Just one man's opinion.

Ted

>I think a better approach to training for cross skill class
>events would be to emphasize the likely need to
>"expect" (not limit the judges to, but expect) the
>range of scores given in lower classes to reflect a wider
>percentage of the thirty points available. I would prefer
>that "expected ranges of scores for a given skill
>class" never be suggested.
>
>Just one man's opinion.
>
>Ted

Ted,

Thank you. This is spot on. I believe there should never be a reference to a "bracket" or "scoring range" for any CLPA class.

I've given forties to Intermediate flyers. I once gave a 29 to a world champion.

The maneuver is what it is. You score it according to that, not according to who is holding the handle.

Randi

First,

Merry Christmas, all... and transfer that for whichever seasonal holiday you prefer, if you wish...!

Second, I think Dave's score range brackets were intended ONLY to reflect the final scores as history, not as prediction. (The difference the news media ignore when they warn of health risks, for instance. If people who have a certain condition or illness also shared some other activity, exposure, or condition, that's history. It, alone, is not necessarily a basis to brand doing a certain thing a factor which increases risk. Financial advisors alway remind that 'past history does not guarantee future results.')

In essence, the whole idea of score ranges associated with Skill Class entered deals with the past track record of each flier. If a guy flying Intermediate scores consistently well above the other INT fliers around him, and the scores are more like thos of the ADV fliers, he should move up. ...Based on current track record.

Ted's points about consistency are excellent, btw.

Ted,
Good comments and I will add some of your thoughts to the next revision of the Stunt Book. I agree that there is no maneuver by maneuver expectation of a certain score range for any class.
The total score is what places a flyer in a class not the individual maneuver score.
In training we always caution judges not to judge to the class - judge the maneuver being flown only to the rulebook.
The chart we put up as an illustration for a discussion on scoring and the words on other pages in the Stunt Book elaborated on the points you made.

Thanks,
Dave Cook

I judge F2b in the U.K. so my comments have nothing to do with the AMA system of which I know nothing.

The FAI rules for judges are quite explicit.

Each manoeuvre in a flight has points removed for errors.

The FAI makes no reference to the ranking of pilots.

Each manoeuvre in a flight should be judged on its merits without reference to the pilot or the model (i.e. how good his/her reputation is or how nice the model looks).

The only thing that counts is the quality of the manoeuvres.

(Answer to an earlier comment.)The only difference between marking out of ten and a hundred is a dot!!!!!

HAPPY LANDINGS

>I judge F2b in the U.K. so my comments have nothing to do
>with the AMA system of which I know nothing.
>
>The FAI rules for judges are quite explicit.
>
>Each manoeuvre in a flight has points removed for errors.
>
>The FAI makes no reference to the ranking of pilots.
>
>Each flight should be judged on its merits without reference
>to the pilot or the model (i.e. how good his/her reputation is
>or how nice the model looks).
>
>The only thing that counts is the quality of the manoeuvres.
>
>(Answer to an earlier comment.)The only difference between
>marking out of ten and a hundred is a dot!!!!!
>
>HAPPY LANDINGS


Jeff,

Please note that my reference was to ranking the maneuvers, not the pilots. Doing so was purposeful for the very reason you mention.

Just as a for instance on how things can easily get misconstrued I encourage you to note that in your post you state the following: "Each flight should be judged on its merits without reference to the pilot or the model ...". I know what you MEANT, but what you SAID was that "flights" should be judged when what actually takes place is that "maneuvers" are judged. Thus pointing out that both reading and writing should be critically evaluated before posting.

Merry Christmas

Ted

TED

Thank's for pointing out this error.

I have edited it to read correctly.

Many thanks and -

HAPPY LANDINGS.

>TED
>
>Thank's for pointing out this error.
>
>I have edited it to read correctly.
>
>Many thanks and -
>
>HAPPY LANDINGS.

Same to you, Jeff. Hope you and yours had a merry Christmas.
Ted

I always enjoy Teds' verbosity with the exception that I usually have to read it twice so as to get everything.

I think he said that as one moves up in the classes the scoring gets tighter as there are fewer mistakes being made. An intermediate flyer could get a 22 and once in awhile a 38 whereas an expert flyer will nearly always score in the low to high 30s' consistantly.

Always score the maneuver not the class or the pilot. RJ

In my adventures watching guys whirl toy aeroplanes around, I almost always see top flyers err to the point that their maneuvers look akin to something that I perform. I frequently award scores in the mid 20s to even the top pilots.

Notice that I said "award"....I don't GIVE anyone anything.

I also believe that a judge should NEVER look at what he has been awarding a flyer, although I know that several do.



See Ya

Peabody

www.gscb.us / www.richpeabody.com

Hello Rich, As you are an F2b flyer you might be interested in this.

If you see judges looking at scores during a competition you might like to remind them of this (which I have copied and pasted directly) from the FAI F2b Judges Guide:-


4B.12. Results Awareness
In order to prevent influence of any kind, no judge should look at tabulated results scores and/or at contestants’ "placing" until after the completion of a contest.
Neither should judges discuss individual
official flights, nor the execution of manoeuvres; nor the marks awarded, nor the tabulated results (placing) or scores, with anyone at all during the whole contest. This includes discussions with the other judges, with any contestant, with any Team Manager, and with all spectators.
The Head Judge should ensure that all members of the judging panel are aware of this requirement and that they all
observe these requirements throughout the contest.

What more is there to say.

HAPPY LANDINGS.

Peabody wrote:

I took Rich's comment to mean previous maneuver(s), not previous flight. In that context, he's right on the mark.

I'm often surprised at who won. The other Judge must be blind as a bat! Steve

Steve Helmick
Renton, Washington
USA

Bob,

One of the interesting aspects of using a point-deduction method is that it actually gets easier the higher up in class you go.

I'd be the first to admit that using point-deduction in Intermediate is a challenge. It's nigh on to impossible in Beginner.

And now to the couch, to sleep off today's prime rib and Yorkshire pudding. Russ and I hope everyone had as good a Christmas as we did.

Hugs to all,

Randi

>One of the interesting aspects of using a point-deduction
>method is that it actually gets easier the higher up in class
>you go.
>
>I'd be the first to admit that using point-deduction in
>Intermediate is a challenge. It's nigh on to impossible in
>Beginner.

There's a big difference between a deduction system, and error counting. I use deductions, but I am firmly convinced that strict error counting can't be made to work as a general rule.

I also think the goal would be to rank all skill levels on the same scale. It's easy to say, anyway, not so easy in practice.

With regards to Dave's original post, and specifically the "scoring range" part - while generally a correct observation about where scores from different classes lie, *has to* be completely out of your head when judging. "Bracket Scoring" is the most dangerous trap you can fall into when judging, just because it's an enticing way to make sure you don't stand out from the crowd.

Unfortunately, if you fall prey to it, you are effectively pre-judging the flight, and also tending to narrow the scoring range. As Ted notes above, what you get with lesser skill classes is a wider scoring range. As the skill level increases, the average maneuver scores on a flight narrow dramatically. What "bracket scoring" does is artificially narrow the maneuver range for the lower classes - which frequently alters the results, since one or two really good (or really bad) maneuvers usually make the difference in the ultimate results.

Brett

Brett, That was my concern when I posted earlier. Not that people neccessarily are intending to, but more that by the mere mention or suggestion of "expected" ranges, it will prejudice the judges potentially to expect a manuever in a certain range. I think a valid comparison could be ,made as well to if I as an intermidiate show up with a full blown Impact, or a ARF flight streak, is the judge influenced by this? (ok maybe I wouuld make a better point talking about advanced instead of Intermidiate) I am just concerned that any suggestion of ~anticipated ranges~ will impact the judging

I think the pattern should be judged by the rulebook and no other way. There should be no distinction about how pretty the plane is or who flies it or what direction it is flown or any other influence outside of what is stated in the rule book and the guidelines for the judges. My concept of the true way to judge the pattern

Good luck on this one Leo. From the discussion you can see already that some judges try to go by the book, and others have their own methods and criteria.

Why should it be so hard to stick to the rule book. That is why it is there. I a contest the thing that should be judged in stunt is appearence and the pattern as provided by the AMA. This is the only flaw I see in the Pampa way of classifying flyers.

>Why should it be so hard to stick to the rule book. That is
>why it is there. I a contest the thing that should be judged
>in stunt is appearence and the pattern as provided by the AMA.
>This is the only flaw I see in the Pampa way of classifying
>flyers.

What flaw are you referring to? Dave's original post, with the exception of the expected "ranges" for classes. is in almost complete agreement with the AMA rule book. Dave is merely describing how you arrive at a particular score given a particular magnitude and type of errors.

The "classes" information it a whopping big exception and having that in you head when judging is absolutely fatal. It's generally reflective of reality - on AVERAGE - but it has absolutely no correlation with what any individual pilot might get on any individual maneuver. Remember that we are not attempting to assign scores for the overall quality of the flight, but for *the quality of each individual maneuver.*. One maneuver score has absolutely nothing to do with the next*What I repeat over and over (about once every 15 minutes or so) during judge's clinics is that anyone might get any score at any time.

Overall, if you use the deduction method (not error counting which is ultimately unworkable) to determine the "quality" factors Dave describes, it's not only the best way to judge stunt, it's probably the *only* reliable method.


Brett

*This is also another minor problem with Dave's original comment. As he points out, I don't agree that one maneuver height affects the other. In fact to me it's clearly not in accordance with the rule book and obviously not in accordance with the concept that one maneuver score doesn't affect another. Dave may be alone in his belief that this is supported by the rules, based on the letters and emails I got after I rebutted the comment in stunt news. It's been beaten to death so there's no point in continuing the argument.

I agree that "error-counting" and subsequent subtraction from "perfect" requires a really experienced judge. There simply isn't enough time to go through a bunch of mental calculations. I think most good judges "flash" on a number immediately following the maneuver, and that's what goes down on paper.

Another point. I think I have always received the score I deserved. And although my flying buddies tend to not criticize my pattern, the judges often give me tips on improving my score. For instance, I was advised that my corners were altogether too violent, and that my 45 deg was more like 35 deg. (that seems to be a very common error ) So, I've kept that in mind.

Floyd in OR

It's hard to read the rules with a fresh eye when you are used to 20,30, 40 or more years of going by gut feel. With the new published rules it would behoove prospective judges to read section 14. Judging Procedures carefully. Particularly sections 14.4,5,6,7, and 8. The judging procedures give a quite clear direction on how to do the task and what the priorities are that is very much at odds with the traditional look and leap to a number style of judging.

Dave, I can't find anything in the rules that would indicate that varying the entry height in the 4-6 ft. range between maneuvers is any kind of error at all. "Degree of perfection with which the maneuvers are performed" specifically says maneuvers, not level flight. Perfection in level flight isn't part of the game according to the rules and it shouldn't factor into a judges decisions either. Until a new line on the score sheet, Overall Impression, is voted in, stuff like this shouldn't be done.