Because my best friend works directly for Chuck Norris...
He's been to his ranch, inside his home, met his wife and kids, trained with him, etc. He knows Chuck personally and can call him up if he needs to.
Straight from Chuck's mouth and the mouth of other people (men and women fighters, instructors, students) that were there and witnessed this first hand. They would get together and train, then spar all the time back in the day. Bruce would come thru and train but NEVER spar.
My friend said that from what he gathered, the best way to describe it would be is along the lines of MJ going to a training camp of other NBA players in his prime and getting absolutely destroyed on the court.
The aura was there that he was untouchable/unbeatable, and he was doing nothing to have that aura called into question. Him losing a fight or a sparring match was something Bruce refused to risk. Seeing those guys fight, especially Chuck, was enough to make Bruce play the sidelines