I took my four-year-old who is a big fan of the Ice Age series and the pixar films. Ponyo is a wonderfully animated, captivating film but probably better for a slightly-older crowd. At least two times, my son wanted to go home, scared by the expressionless father figure and his use of waves as a his army. A similar aged child in the row in front felt the same way. By the end, my son just asked "is it over now?", clearly ready to hit the road. When we got home, he didn't recap the film to his mother as he does with his favorites.
As an adult, I was fascinated by the animation and the uniqueness of the style (I haven't seen any other Miyazaka films), but I think it may be better for 5 or 6 year olds and up.