Sure you can!
You will have to setup some simple ftp server which I'm sure comes with your distro. This will handle the ftp-access.
Concerning access from only one ip:
It might be that the ftp-server can be set up to just serve requests from one ip. You could also define a user for ftp whose password is not known to anyone but you.
If you want to be sure, you could also configure your firewall to only accept your IP on Port 21 (TCP) which is the ftp-port.
I myself haven't set up a ftp-server yet so no help on that one. I'm pretty sure there is some useful howto at
www.linuxdoc.org.
HTH, Steave.