Something is only properly called "public domain" if the copyright has expired, and there aren't any patents or trademarks applicable either. Many old books are now in the public domain, so it's legal to grab whole paragraphs from them without attribution.
It would still be plagiarism to claim that you wrote it yourself, though. (Not that you're doing this, I'm making a related point for contrast.) Shakespeare's descendants can't sue you if you copy whole monologues in your own play. But you'd still be in deep trouble. Plagiarism isn't a crime against the original author, it's a separate issue from copyright. (It might not be against the law either, I'm not sure. So the police won't come after you, but your reputation would be sunk.)
As I understand it, you can legally do literally anything with public domain content (unless the result is hate speech, libel, or slander, or probably some other things).
I'm not sure if JFK's speeches are literally in the public domain. His estate and heirs may have some legal control over their use.
Morally and ethically, use your own judgement. Making someone look bad by leaving out some important words or sentences might not be cool. Associating a historical figure with something they probably wouldn't support if they were alive today would also not be cool. People are going to recognize JFK from his voice and the words, so you don't really need to worry about making sure the full context is there for people to identify the exact source.
In the general case, even stuff that's legally public domain still morally has ties to the wishes of the creator's heirs. Some artists care more than others about annoying people, or on purpose doing things that the heirs won't like. (Your case sounds like something none of the Kennedy's would would have a problem with.)
In your specific case, I don't think there's anything stopping you from putting together any snippets you want from JFK's speeches, even to the point of making it sound like he said something he didn't (by cutting mid-sentence) unless you got your source material from a news organization that still has a copyright on the material. JFK is so famous and well known that distortions of his speeches would be seen as parodies or something, because people aren't going to suddenly start believing bad things about him based on some high school kid's project that had his voice in it. (I know that's not your intention, just making up examples to show that even this wouldn't be pushing the boundaries.) Now, if making what you were claiming was a factual documentary, then it'd be a problem if you left out some key words. Otherwise hey, if you want JFK's voice saying stuff for your project, you can probably get away with it if aren't ever implying that he ever said what you're having him say. So again, the boundaries are WAY beyond what you're asking about.
