I see even quite seasoned writers struggling with this, so let's talk about punctuation in dialogue.
**Note: This is for American English. I know other places might do different things, and that's not even getting into other languages and their punctuation/capitalization norms.
Dialogue is often marked in text with what are called "dialogue tags", i.e. those little "he said"s and "she said"s. The important thing to remember is that dialogue tags are part of the sentence. Because they're a part of the sentence, you don't want to use a period in the middle of it, unless there are multiple sentences within the dialogue itself.
For example,
"This is a piece of dialogue," she said.
Notice that "she" is not capitalized and there is only one period, at the end of the entire sentence, not just the statement in quotes. If we move the tag…
She said, "This is a piece of dialogue."
The period that was after "said" has become a comma, and the period is now at the end, still enclosed in the quotation marks.
**Note, you can use a period outside of quotes, but only if you're not quoting a complete thought. So, for instance, if I were talking about the history of the word "history". (Did you know it was sometimes written with as hystorie?)
Now, because questions marks and exclamation points add intonation that cannot be conveyed with a comma, they can be used at the end of dialogue, but make sure your dialogue tag is still not capitalized!
"Is this a question?" he asked.
"It is, indeed, a question!" she cried.
Now, what if you have a character telling a long story, complete with paragraph breaks? You can do that! Place a quotation mark at the beginning of the first line, as normal. When you reach a paragraph break, don't close your quotation. Instead, just put another beginning quotation mark on your next paragraph, until the tale is done.
Franklin's voice began softly. "It was a dark and stormy night. The trees howled in the wind as the rain pattered the windows of the ancient house. The wooden beams over their heads shuddered and creaked.
"But then, a crash echoed through the house!" he yelled.
I'd be remiss if I didn't also say that you probably shouldn't have a character talking for so long uninterrupted by description or other characters interacting that you need to do this more than like, once a manuscript. If that. I think I've done it once in the past five years, and it was a character narrating essentially an extended flashback.
And that's your primer on punctuating dialogue! If you have any questions, drop me a comment or an email. Teaching English is literally my day job, so I can answer all of your burning minutiae questions, haha.
Today's song is Don't Speak by No Doubt.