Honestly never had luck doing it in that form. I've installed and built the Qt tools on the target, or looked to install the tools on the target via a package installer.
I got Qt installed on Beaglebone Black, Rev C, Debian; via a package and it works fine.
I configured, built, and installed Qt on an Intel ATOM. It took a while and a few iterations to get it working right, but I did eventually get it working. Big ones were to do things like --no-webkit, and no SQL, stuff that I didn't want or need. I tried to cut it back as much as possible, also no demos, not docs, no examples.
Sorry, but never have I gotten it so that I could "build" it on a host and then run it on a target.
Maybe if you have cross compilation tools and build Qt to make static applications it may work, but that might also be very difficult as my configure/build/install path for the ATOM was.
Another suggestion is IMHO that while the designer is helpful, be aware of the code structure you need and be capable of creating your UI from purely code and not a bunch of design patterns. Many Qt examples are purely code, or so I've noticed. I do not use the designer.