I have two drives, /sda for / and /boot and /sdb for /home (encrypted), I was going to install ubuntu alongside fedora using this setup too, I marked /dev/sdb to be used as /home without format, but by my great mistake I skiped the warning and got /dev/sdb formated to be used as /home to ubuntu. Results are that I missed my encrypted drive, I`m trying to find the lost partition with testdisk, going through deep scan, every help needed.

If that drive got formatted, then the LUKS header was almost certainly overwritten. I hope you have good backups, because there is no recovery from that.

On the off chance that testdisk somehow missed the LUKS partition, you can run "hexedit /dev/sdb" and look for the hex string "4c 55 4b 5e ba be" (that's the ASCII characters "LUKS" followed by the hex bytes 0xba and 0xbe) occuring at the start of a sector. If you don't find that within the first few megabytes of the disk (assuming that your /home partition started near the beginning of the disk), then the LUKS header and the encryption key are gone forever.