Or, if you don't want to replace the <body> tag (for example, if you had <body color="black"> or similar):
Code:
find path/to/dir/ -name *.html -exec sed -i 's/\(<body.*>\)/\1\
<p>Hello, world<\/p>/' {} \;
Note that the line break after "\1\" is intentional, to put the <p>Hello, world</p> on its own line.