The other problem with that, and I will resort to Dostoyevsky's The Brothers Karamazov for this one, is that this human narrator who happens to live in the world of the story also happens to know things he can't possibly know. The narrator of Dostoyevsky's monumental novel is apparently someone from the village where the events take place, if my memory still serves me right. All is fine up to that point, but how is this person capable of knowing what shade of red such and such character gets in his cheeks at such and such point of the story? Once again, after several hundred pages of intense prose, and extremely sparse, if any, reminders that a character is telling you this story, you forget that the narrator is the voice of someone, and instead, you read it in "God Mode" as I call the disincarnate voice of third person narration ex nihilo.
The good sailor of Frankenstein cannot possibly remember so much about what Frankenstein told him; that is simply beyond his abilities. The breaking of the fourth wall happens when it becomes obvious to the audience that it can no longer hold the willing suspension of disbelief. This cool term was coined by Samuel Taylor Coleridge in 1817, and it basically refers to what you do when you watch a movie: you don't spend the whole movie telling yourself you're looking at actors and at images shot through a camera, instead, you let yourself immersed in the story. You know it's a movie, but that does not matter; it's not an illusion which you're beholding, it's a work of art.
The problem of human narrators who happen to be characters in the story is that when these are given supernatural memories and other extraordinary abilities without justification, is that it kills the suspension of disbelief. It is as when the microphone shows up at the top of the image in a movie. All of a sudden, you're forced to face the fact that this is a movie, filmed by cameras and a crew, one of which perhaps had a little cramp in his arm and dropped the perch a little more than he should have. Not to mention the others who failed to see that obnoxious microphone.