Multi circle contests No Standout

The best flight in the contest will not win in a multi-circle, multi-round contest. that is one thing that stands out in the loooong discussion about the forthcoming world champs. Given the vagaries of scoring, the weather, and flying the winning flight is almost certainly not the best flight of the contest. It appears that anything done to reduce the effects of judging variation and the weather make the best flight winning even less likely. Mulitiple rounds, multiple circles put a huge premium on consistency and wash out the effects of any one standout flight.

Given that we might reasonably want to see the trophy go to the best flight of the day, or the contest, how can that be done??

IMHO, it's just not possible. It's a subjective thing. Humans judging a human performance. It's like figure skating. A flight impression is different from a measurable score of true accuracy.

The rules place great emphasis on the "degree of perfection" of the maneuvers, so I find it curious that at larger contests the best flights go by the boards in favor of consistency. It can't be done and we've done it this way for 30 years aren't reasons not to consider changing, to my mind. Analyzing the problem and trying out different solutions is more productive.

for example:

One possible way to possibly ensure the best flights of the contest win-
run a round. rank the flights in score order. run a second round. rank all the flights from both rounds and throw out the bottom half. This eliminates some of the flyers who consistently did not have a good flight. Then run a third round. After the third round, rank all the flights flown in order and you will have your winners. If one pilot has more than one place(i.e. his scores are 2nd and 4th, say) he would get the higher rank. By assigning the flight order randomly each pilot would face the same chances for the weather and possible trends in scores during a round. For 100 contestants the number of flights 200 to <300 depending on how many flyers are eliminated after round 2. This method could be extended to 4 or more rounds by eliminating any flights not in the top 100 flights after each round.

You can't eliminate the weather or possible drifts in judging, so this system randomizes them at least as well as multiple circles/judges. It also puts the emphasis on flying the best flight possible each time, as opposed to being good, consistent and hoping for a few lucky points on a few maneuvers.

>The rules place great emphasis on the "degree of perfection"
>of the maneuvers, so I find it curious that at larger
>contests the best flights go by the boards in favor of
>consistency. It can't be done and we've done it this way
>for 30 years aren't reasons not to consider changing, to my
>mind. Analyzing the problem and trying out different
>solutions is more productive.

Phil,

I do not think I ever said that a change could not be done. In the last 20 to 30 years, our Nationals Open Champion and the World Champion have been determined by the best two of three finals flights. There have not been very many poeple, in fact the number would probably be zero until you came along, that ever recognized there was a "problem" that those winning flyers might not have had the single highest scoreing flight at their respective events.

You might be interested to know that at our last two Team Trials, there has been a special award that went to the pilot who had the highest scoring flight during the finals round. This is a perpetual trophy, named after Bob Baron and was created by Tom Dixon. A fine tradition has been started. You also might be interested that the individual who has won this award each of these Team Trials was Bill Werwage who was also the number one placing team member each time.

Now, in your example, you are talking about ensuring that the "best flights of the contest win". You wrote "flights". Did you really mean "flight". I thought in your first post, you were concerned that the single best flight might not be flown by the winner of the contest. Now, you are talking about the best "flights" and determining the winner by some sort of an ordinal ranking system applied to two or more rounds of flights. Analysis of several of our own Nationals and Team Trials show that an ordinal ranking systems would not significantly change the outcome of those contests. (The RC pattern people sometimes employ an ordinal ranking system and it seems to be satisfactory to them.) Does this mean that a pilot who posts the #1 scoring flight sometime during the multi-round system you describe still wins even though he might have ranked only 10th and 15th in the other rounds while there was another pilot who consistently scored in the top 3 during those rounds? With a hundred pilots, you suggest that there could be up to 300 flights. That would only take about six days to do. And it assumes that the judges maintain absolute consistency throughout all of those flights. And the luck of the draw and impact of random weather changes become even more of a factor than what the current system is trying to minimize. I might find this exercise more interesting if I understood it better.

Our team trials follow a format somewhat analogous to the World Championships, though we do employ a two-circle qualifiction period and each pilot has four flights during the qualification period. Since the World Championships use a best two of three finals, so do our Team Trials. Until the format of the World Championships changes, we will likely continue to use the same format. Not because it has been that way for 30 years, but because that is the way it is now. As far as our finals at the Nats is concerned, there will still need to be a balance between a qualification process that is fair to the individual pilot, allowing each pilot an "ample" opportunity to demonstrate his skills while at the same time requireing reasonable resources in terms of time and manpower (judges and other officials to run the contest). The reaction to the way the PAMPA has evolved our Nats procedures has been universally favorable. (Though there has been some grumbing that it is too long, too many flights, and too much work to compete there.)

If there has been a "problem" with this process, I am not aware that it has ever been brought to the attention of the PAMPA leadership who is responsible for organizing and running the Nats or the AMA who still has the ultimate authority on how the Nats are administered.

I think somehow you feel that the people who win the big contests where there are multiple rounds only fly consistently and might not be as good as some ace who gets it all together in one flash of brilliance. I think the converse of your scenario is the prevailing situation. The pilots who are winning or in the top two or three positions normally will have the highest scoring flight of the day. Now that is an interesting revelation. The winning flights of the day are normally flown by the best pilots of the day. I do not see that there is a "problem". And I do not see that your "solution" to the "problem" would be more "productive".

Phil,

I dug out the past two PAMPA Nats issues.

At the 02 Nats Paul W. had the highest scoring flight in every round of the Top 5 flyoff. So in that case the highest scoring flight did take the trophy home.

At the 03 Nats David F. won the trophy with one flight that was the third highest flight score and then he another that WAS the highest flight score of the top 5 flyoff. Now the flight that was the third highest was only out by 2 points.

However I bet if you look at the past winners you will see that they are usually the ones with the single highest score of the flyoff and their other flight just supports it.

Look at Baron's 96 win. In the Flyoff early rounds when the winds were down he stomped them and put up a score that was way ahead. However in the later rounds the wheather was not his best friend. However his really high score from earlier put him in the winners circle.

You bring up an interesting point though. In the top 5 if you post the highest flight score for the day by more than say 3 or 4 points there is better than great chance you will walk off with the hardware.

I will look some more but I bet find that the highest flight score gets the trophy.

Interesting and to the point comments Doug. I appreciate it.

How did Bob Baron fly during qualifying in '96? If he had put up his stomper flight then would it have helped him?

I just find it curious that with all the emphasis in the judging placed on trying to do the maneuvers perfectly at big contests the procedures turn 90 degrees and switch to consistency.(very good to be sure flying , but consistant wins). I think it's great that Paul won by winning every round. Excellent flying and consistent to boot.

I'd be also interested in some data showing how often the best individual flight got washed out by getting added to a more mediocre score.

I went back through a bunch (not all) Nationals results from about 1989 - present, specifically "Final Five" Saturday ... what's interesting to note is that the winners tend to not only have the highest individual score, but the most consistent scores as well. Overall, the variation in individual flight scores between flights one through three is consistently lower for the winner; and higher for 2nd through 4th place. (5th place didn't really show any consistent trending one way or the other.) The only significant exception I noticed to this trend was, in fact Bob Baron in 1996 - his first round score, which was significantly higher than his scores for rounds 2-3, was enough to secure the win.

I'll post some hard numbers when i have time to crunch them all out, but the bottom line is that not only have the Nats champions from the past 10-15 years usually put up the highest individual score on Finals Day, but they've also tended to have the most consistent scores as well. Pretty impressive.

Something else I noticed that was interesting is how often the order (1-5) on Top 20 day is noticeably different from the final order on Top 5 day. This probably doesn't mean much - after all, the ultimate goal on Top 20 day is to make the Top 5, and not really to "win". Also, the strategy to flying is probably a little different on Top 20 day without having a throwaway flight (unlike Top 5, which is best 2 out of 3). Still, it's kind of interesting to see the differences.

Mike,

Thanks for taking the time to do this research.

Phil, who began this thread and is a long-time friend of mine, unfortunately sometimes takes the view that all of this Stunt Stuff and suitable analysis of methodology pretty much started about the time he began to fly Stunt himself.

This is not literally true of course, just be being a little harsh, but I have had pretty good luck in assuming that people such as Keith Trostle are indeed capable of meaningful analysis of how we determine our national champions. That he--and the others--don't blah, blah, blah about it once every month or so does not mean such has not taken place numerous times over the years, the very nice result being current practices at the Nationals each year.

Now if only the World Champs this year could be run to such a high standard. Alas...

>The best flight in the contest will not win in a
>multi-circle, multi-round contest. that is one thing that
>stands out in the loooong discussion about the forthcoming
>world champs. Given the vagaries of scoring, the weather,
>and flying the winning flight is almost certainly not the
>best flight of the contest. It appears that anything done
>to reduce the effects of judging variation and the weather
>make the best flight winning even less likely. Mulitiple
>rounds, multiple circles put a huge premium on consistency
>and wash out the effects of any one standout flight.
>
>Given that we might reasonably want to see the trophy go to
>the best flight of the day, or the contest, how can that be
>done??

Phil,

I thought you would understand that the discussion about the multi-circle format involves the qualification process and only the qualification process leading up to a finals round. Even now, the FAI rules provide for a lengthy one-circle qualification period to select a set number of finalists (15) to fly their three flights during the finals round. The scores from the qualification rounds are not counted for the final placings of the finalists in any way.

Indeed, the best flight of the contest could be performed during the qualification rounds, but that would only allow that pilot to possibly proceed to the finals round. That single best flight and its score means nothing other than it placed the pilot in the finals round in a single circle qualification period or in the case of a multi-circle qualification process, improves that pilot's chances for qualifying for the finals round.

It is analogous to the heat races at a track meet. The fastest time of the meet, in fact a world record could be set during a preliminary heat, but that does not guarantee anything for that runner other than an opportunity to run in the next round of heats or the finals. It does not guarantee that that runner is going to win or even place in the final feature race.

The issue you raise has been brought up before regarding the question if the championship trophy should go to the best flight of the day, or the contest. The question has also been raised if the trophy should go to the best flight or to the pilot who flies the best more consistently.

The qualification process at the FAI World Championships and at our Nats has several things in common. There is a weeding out process during a qualification period to determine who gets to fly during the finals round. The qualification scores have no bearing on the placing of those who fly in the finals round. Each pilot in the finals round at our Nats and at the World Championships has three flights. The finals round scores and placings are based on the best two of those three flights by each pilot. It would appear, even to a casual observer, that the winner in this type of competition is the one who can demonstrate a consistently higher caliber of flying than any other, not necessarily the one who scored the single highest flight.This has been the process at our Nationals for nearly 30 years with other variations of a qualification process for another 10 to 15 years before that. The best two of three flights format for the FAI finals round has been used for almost 40 years. I am not aware that any consideration has been given to the single highest flight relative to final two-flight scores and placing in this best two-of-three flight format here in the U.S. for the 30 years PAMPA has run the Nats. Neither was this a consideration during the last three years of deliberations by the FAI F2B rules committee capably led by Peter Germann and Andy Sweetland.

The qualification process narrows the field of pilots for a finals round to fly in more similar weather conditions and to reduce possible judging variabilities. A multi-circle qualification procedure provides more opportunity for any individual pilot to demonstrate his capabilities by reducing the random adverse effects of weather and the luck of the draw.

It would seem that even those who have not participated in a CL Aerobatic competition using a qualification process would be able to grasp the concepts that have been explained by several people in these various SSW threads.

LAST EDITED ON Apr-07-04 AT 00:16 AM (CDT)
 
Keith,

Good reply.

I too have thought many times before that maybe it(trophy) should go to the best one score on the finals day. But that just cant be the right way to do it. Simply because we dont all get to go at the same time. The track meet works because all the runners run the final heat at the same time,they all face the same conditions, and the fastest one runner of the final heat wins. Not the fastest of three runs added togther. In stunt since we work through an order it is only fair to let everyone have a few cracks at it and total up the score. This minimizes weather factors. In 02 I nailed flight one on day one and was running in about 4th place. I flew round two, I always fly in front of judges when I can. At the verticle 8 the winds came so hard I had to fly out my flight and hope for no crash on landing. It was a wild little wind storm on a cool front. Several passed after me and 20 minutes later the weather was top notch again. By Phil's format me and one other would have been booted from the contest by a total act of nature. I think that might leave to many with a sour taste and for some making that 1000 mile trip each year would be a bit tougher to do. Getting in one flight and getting booted is going to drive people away from traveling to get in one flight.

I really like it at our local contests that have two rounds and add both scores together. That makes it really tough, FUN, but tough. I kind of like the no throw away approach. Have they ever thought of adding all three scores on top 5 day and not using the throw away method? Just curious. It sure would make it alot harder.

Phil,

If you look across sports in general there are none that award the one best performance as the champion. You have to be consistant to get there and then consistant to win it all. In the top 5 you could have one killer flight and stink it up two more times is that champ? Maybe so but in most cases the person putting in the highest scores always wins at the Nats level. It usually isnt a really high score one time and then two stinkers to land him in fifth.

This has made me wonder a bit about this. I used to have a real problem with this. One of my flights would just knock em out and the next one was in the dumper. I am going to dig up a few nats PAMPAs and see how it faired on the score board. Phil's method actually makes consistancy even a larger factor! No give aways. I bet the winner was usually the highest scorer anyway...we will see....

PS: ON one flight wind it all...It is to easy for a possible judges "pick" a winner. I am not saying that they would actually do that but there was a WC where judges were looking at the boards and I am sure there have been other instances. Multi flight combined scores will mininmize that factor.

LAST EDITED ON Apr-08-04 AT 01:34 AM (CDT)
 
>I really like it at our local contests that have two rounds
>and add both scores together. That makes it really tough,
>FUN, but tough. I kind of like the no throw away approach.
>Have they ever thought of adding all three scores on top 5
>day and not using the throw away method? Just curious. It
>sure would make it alot harder.

Doug,

Actually, the Nats format does have one day where there are not any throwaway scores and you went through this. It is the semi-finals day where there are 20 pilots, each flying two flights in front of two separate sets of judges. All four flights count. Each pilot is trying to get in the top five for the finals the next day. One error or one flameout or whatever will knock a really good pilot out (if it is possible to to say that a really good pilot can still have a mental lapse or some equipment problem). Some really good pilots have missed the finals because of some minor problem on one of those four flights. That makes it tough on those who so miss the finals.

The VSC events are the same thing. Both flights are counted to determine the placings for OTS and Classic. (Bart Klapinski missed pattern points on one flight in OTS in 2003 and still won the event!)

Correction added:

As Doug Moon states in the next post, my statement that there are four flights by each pilot in the Nats semi-finals round is incorrect. Each of 20 pilots which are selected during the qualifying round flies one flight in front of judges and a second fight in front of another set of judges, both flights scores are counted to determine the placing of the 20 semi-finalists. The top five then move on to the finals round the next day.

I apologize for the error. My thanks to Doug for correcting me on this.

>Actually, the Nats format does have one day where there are
>not any throwaway scores and you went through this. It is
>the semi-finals day where there are 20 pilots, each flying
>two flights in front of two separate sets of judges. All
>four flights count.

Keith,

It is really just one flight in front of each set of judges. So two flights added together no throw aways. Not trying to nit pick but I dont want anyone to miss something in the format should they not know it at this time.

Yeah I went through and man it was tough!

Doug,

Oops! You caught me with one of those senior moments. Thanks for calling me on that.