If a partition only contains data files one can simply copy every file to another partition of another filing system and size.

But if the partition contains /dev/*, symbolic links and other special files, how do you copy everything to another partition of a different filing system and size?

You can exclude the /dev/ and other "special" files. As far as the symbolic links, depending on what kind of file it is pointing to, you can copy the links over, but that's if the files that are being linked to are in within the backup.

Why are you trying to copy /dev/?

Because I want to change filing system (xfs to ext3), resize, defrag and shrink (vmware concept) in one go.

You can't just do that. You need to set all of that up before you copy your files over.

Please answer the question in the title and do not diverge. Not possible to copy /dev/random and /dev/zero from XFS to a blank EXT3?

You can use rsync with the --links flag, which will copy simlinks, though your system is likely to be broken if you are trying to migrate filesystems by simply copying /dev.

You don't. The files in /dev are created at boot.
Besides that, just copying your files from one system to another won't work. You have to keep this in mind:
1. If your system uses UUIDs you have to change them on the new system.
2. You have to reinstall the bootloader.
3. If you don't have a /boot-partition be aware that your bootloader must support XFS.

What if I install the same linux distro to a smaller EXT3 partition, delete everything but /dev in this, and then copy everything but /dev from the old XFS partition using cp? Bootloader supports XFS.

Sorry, I misread your post, I thought you want from ext3 to XFS, not from XFS to ext3.
But the problems are the same:
1. After the copy you have to reinstall the bootloader.
2. If you use UUIDs you have to change your fstab on the new system.

Copying over a new installed system will bring exactly nothing, the new files will be overwritten with the old and you have the same problems as mentioned above.
In reality adapting a copied system is a trivial task, just copy it with rsync to the new partition, and exclude /dev, /sys and /proc. Then create this directories with a simple mkdir on the new partition, adapt your /etc/fstab according to the new UUIDs, chroot into the new system and reinstall the bootloader.
If you search LQ or the web you should find plenty of HowTos regarding the cloning of a system with rsync.

Aren't the problems above eliminated if /etc/fstab, /dev, /sys and /proc are kept as in the fresh install and everything else is copied with rsync or cp? The tasks you mention like installing a bootloader under chroot etc seem risky and definitely not trivial to my level.

It depends on your partitioning. If you use more than one partition, like a separate /home-partition, you have to give it to your new system also as /home-partition. Be sure to not format it accidentally at install time.
By the way, I don't think that copying a system to a newly installed one is more trivial than re-installing the bootloader.

Found a tool that seems to include /dev/* in its copies, it's rsync with the -avH options and it seems to have worked, except it does not copy the master boot record or the system partition's boot record.

How do you install the bootloader then, is there a liveCD for just this task?