This was a great read for me, in terms of writing and content. I very much agree with you (and Wittgenstein) that the flaws discussed should be taken in stride, rather than taken as cause to immediately discredit the system (or to nitpick the terms of the flaw itself).

The idea of a perfect system is Platonic. It assumes we can model the light outside even though we have not directly seen it, that we can grasp the universe with our limited cognition. When theories fail, maybe the theory and the test of the theory both need to find a "deeper foundation" as you put it.