Melania Trump may have more influence over President Trump than people realize ... and Harvey has a story from inside the Oval Office to prove it.

During Thursday's "TMZ Live," Harvey recalled visiting Trump at the White House in February 2017 ... when the president told him Melania had convinced him to delay his controversial Muslim travel ban.

According to Harvey, the meeting went down the day after Trump's first speech to Congress. It was tempered and well received, and Trump said Melania warned him he'd finally earned some goodwill ... and he shouldn't "f*** it up" by immediately imposing the ban.

Trump listened to his wife ... telling Harvey he thought Melania was right and would hold off on the restrictions ... at least for two weeks.

Harvey says the exchange showed Melania had Trump's ear ... at least in that moment.

It's relevant now because Melania reportedly complained about Trump's immigration crackdown after people she knew were deported, with Trump listening and blaming adviser Stephen Miller for going too far.

Sounds like the First Lady isn't afraid to speak her mind ... and Trump just might be listening.