At my age you would think I would have known but my understanding on capacitors has been completely and utterly wrong.
It took a video from this guy Collin Cunningham to teach me what a capacitor really is and how it functions.
In fact, if you are new to electronics in any way, I can thoroughly recommend watching all of his videos. They contain clear and concise information which will leave you with enough understanding to search for the next level of knowledge. Thanks Collin and thanks also to Jeelabs for pointing out these videos.