RA-Aus's Understanding of Safety 🙄

[APenNameAndThatA]

The latest message from RA-Aus contained this gem.

 

"This weekend I was supposed to fly out to William Creek. Something I have been so looking forward to for months. My plane is literally packed, fuelled and ready to go. I’ve been watching the weather for a week or two, watching trends, looking at the entire route, effect of weather on diversion airports etc. On Wednesday I made the call to not go because the weather in Canberra was going to be marginal for my departure on Thursday morning and there are also very strong winds forecast for my departure from William Creek on Sunday. When looking at the trip in a systematic way, consulting aviator friends and colleagues, identifying threats, and weighing up risk vs reward, I’m comfortable that this is the right decision for me."

 

The problems here, that I can identify are as follows.

a) There was no reference to written personal minimums. Personal minimums are written in knots, feet, metres, minutes and octas. "Marginal" and "very strong" are not proper ways of making go/no-go decisions. 

b) He should have checked the weather on Thursday morning, not on Wednesday day. 

c) About weighing up risk and reward, my understanding is that the when the RFDS decide if they can make a flight or not, the pilots are not told if the trip is an emergency or routine. They have a deliberate policy of not considering risk and reward. To be fair, I have different minimums for flying alone and with my family. 

 

Then, there was this gem. 

 

The U.S. Federal Aviation Administration (FAA) defines aeronautical decision making (ADM) as “a systematic approach to the mental process used by aircraft pilots to consistently determine the best course of action in response to a given set of circumstances” (FAA Advisory Circular 60-22). I ask you all to apply a systematic approach to decision making, to weigh up risks and take the time you need to make good decisions so you do come home to your family and friends. There is a wealth of information on Aviation Decision Making including the abovementioned Advisory Circular and this article provided through SKYbrary.aero  by the Flight Safety Foundation. 

 

The link was to here https://skybrary.aero/articles/decision-making-training-oghfa-bn. The article guilty of my pet hate: not being concrete and specific AND not so abstract that it gives some overall insight. Instead, it provides a middling level of abstraction, talking about things, rather than actually saying what they are. 

 

Then, there was this, also a gem. 

 

When I started in this role back in January 2021, it was midway through COVID-19 and La Nina hadn’t yet arrived on the East Coast. The result being that from September 2020 until May 2022 there were no fatal accidents – a record we were very proud of. We’ve now had four accidents where four RAAus members have lost their lives since May.

 

There is no recognition of random variation and regression to the mean. If COVID and La Nina were so obviously the cause of these things, then it would have been possible to predict that there would not have been fatalities during lockdown and that there there would be a spike after May. Nobody made the prediction, for obvious reasons. Sheesh. 

 

Lastly, "preempt" doe not mean what he said it does. To be fair, this is the new usage. 

 

 

 

 

 

 

 

 

[jackc]

Sadly I have to say, the more I hear of RAAus, the less  respectful I have become 🙂  We should all look up to the peak body that should have the best interests of it members at heart.

 I am not so sure it has anymore, from the people I speak to. We have moved so far from AUF it’s almost unbelievable 😞 

[walrus]

This subject is not amenable to hearsay or personal convictions, period.

 

‘’There is an old and rigorous method of determining if there is an increase or decrease in safety levels over time; that method is called statistical process control.  It was developed at least as early as WWI to assure the quality of safety sensitive products such as  explosives and munitions and is based on straight forward and well understood mathematics.

 

‘’What is required is either an estimate or actual activity (eg. a9hours flown, number of flights).  An estimate or actual numbers of accidents and/or incidents over time, in other words a history.

 

‘’From these numbers an average and standard deviation are calculated. We can then apply the normal distribution and calculate the three sigma limits for the distribution of incidents and accidents. This is the computed number within which 99.95%(im getting old) of all statistics will fall. That may be for example, 6 accidents per quarter plus or minus three. It is only after there have been 9 (6 + 3) accidents can we say Statistically that we can bev99% sure that something has changed.

 

‘’Of course there is more to it than this but you get my drift. I’m sure RAA know this and in any case ATSB  can advise them on design and computation of the relevant data..

 

‘’In other words, we don’t know if four fatals are significant or not without mathematical analysis of our accident history; we just do not know.

 

‘’There is therefore no sense in getting knickers in a knot about the overall state of the pilot population just yet.

 

‘Having said that, if poor weather decision making appears to be a common factor in a relatively large number of incidents (statistics again) then it is possible to test that hypothesis (maths again) by developing a questionnaire, taking a sample of RAA, PPL and commercial pilots and testing their skills. The results of such an exercise then allows RAA and CASA, advised by ATSB, to change the syllabus and training on the basis of fact, not guesswork.

 

‘’Having done that - changed the training, you would then watch for a change in quarterly (or annual, or monthly) statistics (mean and standard deviation) to determine if your training was effective.

 

‘’Been there and done it, It ain’t that bloody hard. (at least when you are young).

Edited September 24, 2022 by walrus

[turboplanner]

7 hours ago, walrus said:

This subject is not amenable to hearsay or personal convictions, period.

 

‘’There is an old and rigorous method of determining if there is an increase or decrease in safety levels over time; that method is called statistical process control.  It was developed at least as early as WWI to assure the quality of safety sensitive products such as  explosives and munitions and is based on straight forward and well understood mathematics.

 

‘’What is required is either an estimate or actual activity (eg. a9hours flown, number of flights).  An estimate or actual numbers of accidents and/or incidents over time, in other words a history.

 

‘’From these numbers an average and standard deviation are calculated. We can then apply the normal distribution and calculate the three sigma limits for the distribution of incidents and accidents. This is the computed number within which 99.95%(im getting old) of all statistics will fall. That may be for example, 6 accidents per quarter plus or minus three. It is only after there have been 9 (6 + 3) accidents can we say Statistically that we can bev99% sure that something has changed.

 

‘’Of course there is more to it than this but you get my drift. I’m sure RAA know this and in any case ATSB  can advise them on design and computation of the relevant data..

 

‘’In other words, we don’t know if four fatals are significant or not without mathematical analysis of our accident history; we just do not know.

 

‘’There is therefore no sense in getting knickers in a knot about the overall state of the pilot population just yet.

 

‘Having said that, if poor weather decision making appears to be a common factor in a relatively large number of incidents (statistics again) then it is possible to test that hypothesis (maths again) by developing a questionnaire, taking a sample of RAA, PPL and commercial pilots and testing their skills. The results of such an exercise then allows RAA and CASA, advised by ATSB, to change the syllabus and training on the basis of fact, not guesswork.

 

‘’Having done that - changed the training, you would then watch for a change in quarterly (or annual, or monthly) statistics (mean and standard deviation) to determine if your training was effective.

 

‘’Been there and done it, It ain’t that bloody hard. (at least when you are young).

While you're correct, you're off on a different track, maybe picking up something on another site.

The Aviation Safety Digest did someting like that when people like Macarhthur Job had wartime experience.

Being "sure RAA know this" is a meaure of your GA background, because RAA doesn't have unlimited staff.

However, the simple statistics you mentioned are useful, using 12 months in lieu of per quarter; that works quite well going back over the past 20 years.

Our issue is not all fatals are reported centrally, we seem to pick up more through the local news media than RAA reports, wheras you'd think RAA would get a call from local police on every fatal.

Historically in GA there were 5 or 6 CFIT every winter all shoing a ball of aluminium at the site, and more than half with the comment "The pilot was known to practice scud running and had boasted he could fly on instruments." These days that comment isn't posted.

The numbers will fluctuate, so a blow out without a trend is unremarkable.

[plugga]

I'll play devil's advocate here; what was the purpose of the message, to write a technical piece on decision making, or to provoke discussion or introspection amongst the pilot body?  If the latter, then I believe it achieved its purpose and quibbling over the details amounts to attempting to remove the fly spots from the pepper.

 

The problem with quoting hard numbers is that a percentage of the readership will immediately adopt them as "the rulez" and abdicate personal decision making. Marginal is a concept which pilots recognise in relation to weather and visibility in particular and I read "too strong" as beyond my personal limits.

 

As to the timing of decision, I'm not sure why you'd fault an early call.  There may well have been other factors playing into that decision.  Had the call on Wed been go, then clearly you would re-evaluate on Thurs before depsrture, but if it's "no", the matter ends there.

 

Risk vs reward; the RFDS example is a red herring,  the writer was obviously across the reasons for the flight and I read only a recognition of the fact that the reward side of the equation may subtly influence decision making.

 

So no, it's not a technically perfect piece of writing, but I believe it probably resonates better with most of the readership than a university level technical treatise.

[walrus]

There may also be an ill advised push to write more regulations….

 

 

There was a similar movement in marine circles once to classify vessels according to size and equipment and then to limit their voyaging according to weather and sea state. It failed because of that same problem - reducing the accountability of the skipper.

 

‘’What is good sense in a Tiger Moth is not necessarily safe in a 150 knot RV what’sit and vice versa.

 

Where I live, there WAS a very brief (maybe one and a half hour) window between fronts where said RV rocket could have braved some turbulence and  got away VFR and headed North at altitude to better weather at warp speed but for us Jabiru types this wasn’t possible. We are too slow.

 

‘’It may be that the victim saw that same window….

Edited September 24, 2022 by walrus

[old man emu]

12 hours ago, walrus said:

if poor weather decision making appears to be a common factor in a relatively large number of incidents

In analysing anything, you can't simply use the end result  - in this case events that lead to at least one fatality. You have to go deeper and look at the factors - Operator, Device, Environment operated in. If you replace "poor weather" with any other factor likely to produce an accident then you can see the cause and work on that. If you looked at the cause of all fatal incidents over the past 50 years, I wonder if EFOTO would not come out on top. Then you have to dig into the causes of each EFOTO. 

 

So, harping on about the number of fatalities takes us nowhere. As an example, the mantra for road safety is  "Speed kills". There is no deeper examination of the actual cause of a collision. As a result of applying that, we get speed detection equipment used in places where there is no history of serious or fatal collisions. As a result, the normal practice is to keep an eye out for speed detection equipment, and having located it, pass and speed up.

 

To prove the stupidity of that approach, I drive on a rural highway with a speed limit of 110 kph. The recent rain events have resulted in isolated potholes forming in otherwise good road surfaces. The potholes are deep and are impossible to detect at night. If a car is travelling up to the speed limit, and one of it's front wheel enters one of these potholes the driver could lose control and veer to one side of the road or the other, colliding with a big tree or a big rig, depending which way it veered. Was that incident caused by unlawful speeding?

[turboplanner]

1 hour ago, old man emu said:

If a car is travelling up to the speed limit, and one of it's front wheel enters one of these potholes the driver could lose control and veer to one side of the road or the other, colliding with a big tree or a big rig, depending which way it veered. Was that incident caused by unlawful speeding?

That sums it up. You need to look at the broad cause then drill down to what actually happens, and that can be one of a hundre things which, if they had not been present would not have caused the accident.

All formal motor racing employs speed conventions, but that's only to keep the different sizes all at the same speed; there's no specific speed limit. in this case the pothole triggered it, but the driver may have been driving blind as so many do. 

 

So much of flying is due to poor decision-making particularly with weather, but if there is no Met training, no Met practicals, no Met tests how are you going to defend your duty of care if you're an instructor; you could well be in the mix.

[Yenn]

I agree that poor decision making is the major cause of accidents, not only in flying, but also on the road. The use of personal minima always seems stupid to me. If we set a minimum visibility for ourselves, how do we know when we get to that minimum while flying. It may be OK for the forecasts, but they are not always correct.

What is this about no met training Turbs? Are you saying that RAAus pilots don't get met training. If so something is seriously amiss. I don't know RAAus standards as my training was GA not RAAus and back a few years and from what I see it appears that GA pilots are trained to a lower standard nowadays.

[Garfly]

7 hours ago, plugga said:

I'll play devil's advocate here ... quibbling over the details amounts to attempting to remove the fly spots from the pepper.

Me too. I'm with the devil on this one.

I have my own gripes about RAAus safety-messaging (e.g contradictory attitudes to new EC/traffic awareness tech) but when I read here CEO Matt's chatty newsletter piece being torn apart for teeny weeny dogma deviations, or slip-ups in statistical science, I'm loathe to jump on the bandwagon.

Sheesh, no need to be more pious than the Pope   ;- ) 

 

 

 

 

 

Edited September 25, 2022 by Garfly

[pmccarthy]

2 hours ago, old man emu said:

If a car is travelling up to the speed limit, and one of it's front wheel enters one of these potholes the driver could lose control and veer to one side of the road or the other, colliding with a big tree or a big rig, depending which way it veered. Was that incident caused by unlawful speeding?

We have potholes everywhere at present. I can cruise my Prado at 100kph and it just rolls over them. But if my wife's Corolla Hybrid hit one it would bust the tyre and shatter the rim, the tyres are really low profile. So she has to swerve suddenly instead. What would an investigator conclude?

[jackc]

I am not relying on RAAus to keep me safe, that responsibility is up to ME 🙂  IF I have doubts,  I ask questions to experienced pilots, not just ONE but any I come in contact with.  Between what I learn and examples of other peoples experiences and the studies I do, help me a lot.  

[jackc]

1 minute ago, pmccarthy said:

We have potholes everywhere at present. I can cruise my Prado at 100kph and it just rolls over them. But if my wife's Corolla Hybrid hit one it would bust the tyre and shatter the rim, the tyres are really low profile. So she has to swerve suddenly instead. What would an investigator conclude?

Investigators always look for necks to hang ropes around…… 😞 

[Garfly]

38 minutes ago, jackc said:

I am not relying on RAAus to keep me safe, that responsibility is up to ME 🙂  IF I have doubts,  I ask questions to experienced pilots, not just ONE but any I come in contact with.  Between what I learn and examples of other peoples experiences and the studies I do, help me a lot.  

I agree.  For example, I think that in recent years, I've learned more about 'safe' recreational flying on this forum than anywhere else (especially if we include the various aviation videos we share). Actually, I've always thought it a pity that there's so little cross-over between Rec. Flying and RAAus cultures; they seem totally discrete entities, mutually hostile even, at times.  I guess there are hard-to-avoid reasons for this but still, it's a bit of a shame, and a bit counter-productive.

 

 

 

 

Edited September 25, 2022 by Garfly

[jackc]

Yesterday, I attended a flight planning course that was organised as impromptu thing, we had 15 or so attendees, it was organised by a local CFI/L4.   The course was great for me, but there were a lot side discussions over smoko  and lunch etc.

I was able to buy some CASA publications that another Instructor brought along.

The. conversations were cordial and not curmudgeonly in any way.  Not so sure it would be like that if RAAus were involved….

This is my style for learning new things 🙂 

[facthunter]

The Upper Echelons" no longer have People of vast Aviation experience and it will show. Esoteric concepts may sound fine to some but contain little substance. You can't fly totally by numbers. It's a skill and an art with severe penalties for shortcomings. The Flying environment is unforgiving and YOUR decisions decide how risky it is. You can never know too much about weather when you fly VFR with no anti icing etc. I ALWAYS gave storms a wide berth even with big stuff. Nature has forces that dwarf our meagre craft.. Nev

[turboplanner]

1 hour ago, pmccarthy said:

We have potholes everywhere at present. I can cruise my Prado at 100kph and it just rolls over them. But if my wife's Corolla Hybrid hit one it would bust the tyre and shatter the rim, the tyres are really low profile. So she has to swerve suddenly instead. What would an investigator conclude?

You've certainly touched a raw nerve there; we're short one new tyre, wheel alighnment out and some suspension damage from one of them. Thousands have appeared around Melbourne, thousands on the Gippsland highways. I'm not sure whether they are due to Dept Transport (Vicroads) ceasing maintenance during covid or the new PBS truck regulations where much heavier axle load/combinations are legal. The pot holes and seal pealing are all on the wheel track lines so you are forced to dive towards the side of the centre of the road. The one that took out the tyre was around 350 diameter and 300 mm deep. There is a provision for "environment" in crash investigations so it would most likely fit there.

[turboplanner]

1 hour ago, Yenn said:

I agree that poor decision making is the major cause of accidents, not only in flying, but also on the road. The use of personal minima always seems stupid to me. If we set a minimum visibility for ourselves, how do we know when we get to that minimum while flying. It may be OK for the forecasts, but they are not always correct.

What is this about no met training Turbs? Are you saying that RAAus pilots don't get met training. If so something is seriously amiss. I don't know RAAus standards as my training was GA not RAAus and back a few years and from what I see it appears that GA pilots are trained to a lower standard nowadays.

Ask around and you'll get vague answers like  "I'm careful", "I use Windy, doesn't everyone", and some will have read a book, but so far no one I've spoken has said  they've done a course or been tested. 

[facthunter]

You do a substantial 3 hour exam at ATPL level and are an "accredited MET observer" at that time. . Some of it will relate to stuff that's not directly useful to U/L's but getting a copy of a past exam wouldn't do any harm.. I've never found any ONE book that covered it and northern hemisphere  rotation is opposite the southern around highs and lows. I read a lot of that material and also gliding Met as I had a strong interest in it as a subject. It still doesn't stop you getting in strife if the Forecast is a mile out but It MUST have helped . Nev

[MattP]

20 minutes ago, turboplanner said:

Ask around and you'll get vague answers like  "I'm careful", "I use Windy, doesn't everyone", and some will have read a book, but so far no one I've spoken has said  they've done a course or been tested. 

I learned a lot about weather I didn't know (or forgot) during my instrument training, and the concept of personal minimums took on a new dimension as technically once you're instrument rated you've really got some low minimums mandated by the A/C POH or part91 to stop you legally, it comes down to your own decision making and personal minimums. One thing I still see though (and admittedly my personal minimums are quite conservative) is people launching into the sky VFR on days where I'm either happy to complain about not flying as I've cancelled or just take a quick lap around the circuit a few times. I also remember doing a weather course as part of my PPL, but that was at a large school and they had it as part of the general "course" they ran all students through at the time. 

 

I don't know what schools do now, but as part of my PPL nav's I did do a nearly 5 hour flight interstate and then back which included showing my weather planning / briefing, alternates etc. There were also opportunities to join the instructors on trips away for weekends etc. just to learn about all the things you don't see in the school environment (even basic things like how to tie down the aircraft at a country strip, contacting for prior permission etc. that might not be experienced by those training out of larger class D / C airports).

 

This problem isn't a new one, and despite the calls from people that the fancy new avionics people have or EFB's are causing issues, I'm not sure the data on CFIT's / weather related incidents backs up that hypothesis.

 

Like everything, there are some people who have more risk appetite than others, or who may have been successful scud running in the past who continue to do so. 

 

I had a similar query the other day about RAA met training as I'd come from a GA background and don't know what is taught on the RAA syllabus.

Edited September 25, 2022 by MattP
Spelling

[turboplanner]

6 minutes ago, MattP said:

I had a similar query the other day about RAA met training as I'd come from a GA background and don't know what is taught on the RAA syllabus.

It was RAA pilots I was talking about - recreational aviation as against GA where the training you described is the norm, through the in-house courses etc.

[facthunter]

IF I had to decide I'd suggest the RAAus is much like the treatment of the issue of stalling and spinning in that "just don't go there" is the main thrust.  Met is a big subject and local knowledge comes a lot into it. Afternoon cooling near the Coast can produce low stratus  and/or FOG fast. Low cloud base means the relative humidity is high and slight mixing/cooling may cause more cloud to form. Nev

[MattP]

I can understand not covering it as part of the general RAA BAK type training, but for the X country endorsement, I would have expected to see this type of training / need to demonstrate consideration of weather. Even more important in lighter aircraft without as much inertia. As the aircraft coming onto the RAA register become more capable and the gap between RAA and GA flying blurs more, its going to be important for the RAA to demonstrate that there's not a considerable difference in capability / quality of training from someone who learned at an RAA school vs. a GA school. Ideally numbers vs letters on the side of the plane shouldn't make a difference at all [talking 3 axis here, not lightweight / weight shift etc.]. Given that's the way CASA sees it in terms of logging hours then I'm going to assume they are expecting this is the case already?

 

[turboplanner]

10 minutes ago, facthunter said:

IF I had to decide I'd suggest the RAAus is much like the treatment of the issue of stalling and spinning in that "just don't go there" is the main thrust.  Met is a big subject and local knowledge comes a lot into it. Afternoon cooling near the Coast can produce low stratus  and/or FOG fast. Low cloud base means the relative humidity is high and slight mixing/cooling may cause more cloud to form. Nev

......and increasingly RAA has adopted bigger and faster aircraft designed to enter the "Touring Aircraft" class where you fly from one weather pattern to a second or third, and need to be hot on Met.

Edited September 25, 2022 by turboplanner

[MattP]

Yep - and what concerns me about the marketing of said style of aircraft in such ways is that there is an element out there that will see the fancy glass, sleek bodies etc. and think they are go anywhere, anytime, do anything machines. They aren't. Even a cursory look at some of the speeds on these types shows that the cruise speed vs VA speed gap is significant and VNE is awful close in a lot of instances to cruise, not giving you much room to move should you hit some rough stuff. Even understanding the idea of VA and that sometimes you just might need to slow it down rather than blast through everywhere at warp 5. Add into that turbocharging courtesy of the 915, and the ability to go high, needing an understanding of the differences in TAS vs IAS and which limiting speeds use which. Add in descent planning (not VNAV and AP on the G3X autopilot), and need to be on top of the aircraft and its not that any of this is hard but you can go really quickly from something that putts around the pattern at 60 - 70 knots to a 130+ knot glass speedster quickly. None of this is the fault of the designs here, its more the type of people they're appealing to and (potentially from what we're discussing here) lack of training to prepare them for the jump.