your idea is correct, but you are forgetting about implied odds.
what if that person makes the flush, how much more can he factor in that he will make on the turn and/or river after making the flush?
so yea, if you bet the pot, he is getting 2:1 on hitting 4:1, but if he believes you will call a pot size bet on the turn after hitting the flush, then he is actually getting
$1 in the pot.
you bet 1, he has 2:1 to call, he calls 1.
4 in the pot.
he makes the flush, he bets 4 you call 4.
so in actuality...
3/4 times he loses 1 on the call
1/4 times he wins 7 on the call
-1, -1, -1, +7 totals +4 so its a good call.
this is like the cliff notes version of this concept, numbers might be off a bit. for more details and a better example read this page I wrote: http://www.checktheodds.com/impliedodds.html
other factors to consider:
do the players have more chips so that if he hits will he get paid off, how often will he get paid off by you, does he have other outs, can he be up against a bigger flush draw, exposed cards, etc. etc.
Plus and this sounds stupid but it is probably true for many that don't even consider the odds in any way: most people get that 'feeling' of its 'my turn' so they feel destiny is with them. When they are losing they are thinking 'my turn' to get it back and when they are winning they are thinking 'my turn' because I am on a rush