As a writer of fiction, you have a job. It's to create worlds, situations, and characters that your readers will buy.
When a reader picks up a story or book, they have a chip on their shoulder. They're daring you make them believe. Orson Scott Card, in his book Character and Viewpoint , calls it the 'Oh, yeah?' factor. Imagine a big former high school football player (you know the type--the ones with more attitude than athletic talent, which is why they didn't go on to play college or pro football) with his beer belly and his unshaven cheeks. He has a remote in one hand and a light beer in the other, because THAT'S going to help get rid of that gut. He's kind of growling and grunting, he's frowning down at you, because even though he's pretty much a failure at life he's still VERY BIG! He says, "Come on, big shot. Take your best shot!"
And if you don't succeed, he's going to punch you in the nose with that meaty fist of his (translate: toss your story/novel in the trash).
With characters it should be fairly simple to gauge whether his/her actions are believable. You simply ask yourself, 'In this situation would I do the same thing?' If not, it might be a good idea to re-evaluate your characters, simply because it's a difficult job to write about things you know nothing of. If your character is faced with a wild-eyed gunman would he run like a chicken down the street? Cower in a corner? Beg for his life? If those are things YOU would do, then that's a character YOU understand well enough to write about. That's a simplistic example. What you would do in such a situation would depend on many things. Different circumstances would prompt different reactions.
Think back. I'm sure you've had situations in which you THOUGHT about doing something that you wouldn't otherwise do. That counts as 'experience.'
But let's go back to our example. If the LAST thing you would ever do in this situation is to put your finger in the barrel of the gun, how can you believably write about that?
Again, this example is simplistic. It's really less about the action and more about the emotion, less about what your character would DO and more about what your character would FEEL.
Character reactions should be logical. They should fit the character. If your suave, daring, handsome hero screams like a girl at the sight of a mouse, that's illogical.
Character reactions should be consistent. If, in once scene, your character cries like a baby from fear at the prospect of taking the elevator, but in a subsequent scene walks onto an elevator without a hitch, that's inconsistent characterization.
The key to creating believable characters is to make them REAL to YOU. You should know your character so well you'll know exactly what the character will do and what he will feel in any given circumstance.
In this article, Character Creation by Jeff Heisler, the author gives some tips on creating believable characters. He suggests actually interviewing your characters. Good idea. I think I'll go do that right now.
And you should, too.