Right click the "start button", then click 'menu editor'. Then select where you want it to go, and clck 'add entry', then you need to type in the path to the starters. eg, in my Suse box the path to Writer is: /usr/X11R6/bin/OOo-writer --messages-in-window and the rest are the same, except they substitute writer for math, impress, etc. Call it a name (OpenOffice Writer, eg), give it an icon and go onto the next one. Alterntively, just create one entry for 'soffice' and open the programs from within there.
Just remember that the paths and files are case sensitive and you should be fine.