I have not tested this and don't have the time to do so right now, but try:
VBoxManage modifyvm <vm name or uuid> --uart1 0x3F8 4
VBoxManage modifyvm <vm name or uuid> --uartmode1 /dev/ttyS0
see VBoxManage list vms for name and uuid.
That should set COM1 to irq 4 port base 3F8 (1016 dec) and tie it into ttyS0 of the physical host.
Let us know the results.
-- Edit --
Misread your original post.. My above comments will not allow access from the host OS to the virtual machines serial port. just the physical serial port to the guest OS.
try "server" instead of specifying a device
VBoxManage modifyvm <vm> --uartmode1 server /tmp/foo
That will create a UNIX domain socket file. you can communicate with is using nc -U or by installing socat or using any application that connects to UNIX domain sockets. You will no be able to redirect stdin or out directly to the socket.