The Earth Simulator is a balanced, stable, and versatile TeraFLOP
system, yielding high sustained performance across a broad range of
applications.  In this talk we try to asses the Earth Simulator system
design with a view to the design of a Petascale system. In light of
foreseeable and difficult scaling effects, as well as some necessary
compromises, we discuss challenges to the MPI implementer, and to the
MPI standard itself.