A few days ago I posted a excerpt from an interview with Rabbi Steinsaltz in which he said that the Jewish response to the question of whether ours was the best of all possible worlds or the worst of all possible worlds is that "We are living in the worst of all possible worlds in which there is still hope."
“But has God placed us in this worst of all possible but hopeful worlds for a reason?”
Rabbi Steinsaltz replied:
One of these is the question which asks about the purpose of Creation.
And the fact is, as one chasidic rebbe said with respect to this very question, there is language in the Midrash to the effect that the Almighty had a teiva, a desire, and if you have a desire you don't ask "why?"
The language of the Midrash is very suggestive at this point because a teiva is something we can't explain.
To answer a question about the "why" of Creation can, philosophically, be proven to be impossible.
But perhaps we can say this much:
So in a way, existence in any other world is not "proof."
I cannot test it by driving it off a cliff, but I can test it on the roughest terrain where I must come to the edge of a cliff and have to stop.
How is a new plane tested?
They put it under nearly impossible conditions, which the plane must withstand.
Otherwise the whole experiment doesn't prove anything.
The same with Creation.
Creation would have been pointless unless it was a Creation under precisely these difficult circumstances.
So I am saying, theologically speaking, that the worst possible world in which there is yet hope is the only world in which Creation makes sense."
--Rabbi Adin Steinsaltz