Yeah - I wasn’t precise. The contradictions are not within the system itself, but between the system and some meta-system. I’m remembering from notes, but as I recall Godel constructed a formula that was not formally demonstrable within an arithmetical system which was demonstrably demonstrable (!?) outside the system. Something like “every integer possesses such-and-such an arithmetical property”, which every integer can be tested for, but which the arithmetic cannot prove. That is, you can come up with formulas that the arithmetic says are not demonstrable, but which are demonstrable outside the arithmetic (which, as far as the arithmetic goes, means nothing, because it involves a statement/formula the arithmetic can’t even contemplate).
If you take a consistent arithmetic on its own terms, it can be consistent, but as soon as you consider it from outside, you will see that the “truths” it comes up with are contingent on remaining within that system, and that some of its truths can be false when considered outside of the system (or without the proviso of “within this system, this formula is true). But Godel wasn’t talking about that - other people do, and tend to vastly overstate what he came up with (like I did).
I believe he also showed that you cannot use an arithmetic to prove its own consistency. You need to go outside the system to prove it, but then you will end up using another system which cannot prove its own consistency, and so on (I don’t think he got into that, though). While that is not a contradiction, it opens up the logical possibility that there are contradictions (because you cannot demonstrate the opposite). But that is not the same as saying that there are in fact contradictions.
I find that I understand it well enough when immersed in it, but that it is very, very difficult to say in plain language because it is very subtle.