it took me a while to figure out "I'm just saying!" but you can read it as "I am merely expressing my opinion on this matter, kind sir."