Is a "dark night of the soul" an essential aspect of "spiritual progression" (as many refer to transcendence)?

What has your own experience been as the veil is lifted? Are you addicted to or convinced that struggle is the only method of purification? How much influence does the personality (or karma) have on one's experience in awakening to Reality? Is a higher intellect more inclined to suffer due to "creating" a higher mountain to climb? Do we choose whether to suffer/struggle, and if so, then is recognition of this choice part of the experience?