Billy's "going crazy"

At the beginning of Slaughterhouse Five, we identified Billy Pilgrim as the main character, but unlike many other books, Billy is a difficult character to admire,  relate to, and fully understand. His experiences with the Tralfamadorians and his perspectives of time are very unusual. Many events throughout his life have caused him to stand out and act differently than most people. His war comrades were not fond of him, his family thought he needed help and "the doctors agreed: He was going crazy" (pg 100). From my view, the most obvious reason as to why Billy Pilgrim is "going crazy" is because he is time traveling and interacting with the Tralfamadorians. Another factor could be the traumatic experiences that he encountered during the war. However, the doctors don't believe that either of these things caused Billy to go crazy, "they were sure that Billy was going to pieces because his father had thrown him into the deep end of the Y.M.C.A. swimming pool when he was a little boy, and had then taken him to the rim of the Grand Canyon" (pg 100).
These two small events that occurred when Billy was a child, seem very insignificant compared to other events (such as the war and time travel). So, why would the doctors think that his father might be the cause to Billy's "craziness"?  If Billy has endured much more frightening experiences, it seems as though those more powerful events would be the cause to Billy's mental health. Would you define Billy's mental state as crazy?
Or, perhaps it is too "human" to diagnose something. The Tralfamadorians believe that the past, present, and future have always existed and always will exist, so maybe it is not right to say that Billy is "going crazy".