Unfortunately you are mistaken and there is about 60 years worth of medical data to disagree with you.
The basic facts are:
If you are exposed to INHALED asbestos you are at signidficant increased risk of
1: pulmonary fibrosis - scaring and tightening of the lung tissue with decreased capacity of the lungs to absorb oxygen.
This in itself is a NON-malignant disease though may serve as the starting ground for lung cancer ( NOT so much mesothelioma usually but the commoner squamous cell and less common adenocarcinoma type lung cancer.
If you smoke as well the incidence of this type of cancer goes up dramatically. But non smokers with asbestosis have a high incidence of these cancers as well.
2: pleural plaques. - these are thickening of the outer surface of the lungs often cause no direct problems themselves. Often present but not known about until coincidentally picked up on screening X-rays. But they may be the fertile starting ground for mesothelioma. The effect of smoking on the causation or presence is knot fully understood but They are often present in non smokers.
3: Mesothelioma. Probably one of the highest mortality cancers we can get. They arise in outer surface/lining of the lungs and some authorities have suggested they always arise in plaques - plaques don't have to be very thick to be a breeding ground and may not be picked on X-rays. Big and obvious ones are more cause for concerns. They are resistant to pretty much all treatments. The incidence is high in non-smokers with known exposure to asbestos but is higher in smokers.
Overall it's like buying tickets in the lottery.
If you want to win you gotta buy a ticket ( if you want to get mesothelioma you gotta inhale some asbestos)
But you can win with just a single ticket or you can increase your chances and buy lots of tickets.
If you want to really increase your chances even more you buy lots of tickets and you buy those games with the powerball already picked for certain etc. ( ie inhale asbestos and smoke. )
Just as you can buy lots of tickets, buy the high chance games and play a system and still not win, some lucky people can inhale lots of asbestos, smoke, get pleural plaques and still manage to evade the mesothelioma. But just cos one person does it doesn't mean the rest of the world can do the same and hope to get away with it.
And time does not give any indication as to whether you'll get away with it either.
The P50 [years] ( that is the number of years it takes from exposure to mesothelioma to appear in 50% of the population thus exposed ) appears to be about 15 -30 years. which means some people will take even longer and some sooner. So it's always a long wait to say you've dodged the bullet.