It was very suspicious to be challenged the first day he just arrived and thus understandable for him to think it was someone else. Even so, the reaction he had on didn't fit the situation.
"If someone dares to challenge me, they should have some resolve." What he was saying was actually simple. He would not rush out to accept any challenges. Those that wanted to challenge him should at least be willing to wait until he was also willing to accept the challenge.
In the most simple terms, what he was asking was how he would benefit from challenges. Right! Musk didn't have any interest in satisfying other people's curiosity at his expense. Since that was the case, he ignored the sound, as it should not be targeting him.
"That guy probably is not around! Why doesn't this idiot come back when those he is challenging are present." He said it out loud, but it couldn't be heard by the guy shouting on the outside.