How much memory does the system have?
It takes memory allocate space for 4,000,000 possible files.
My 8GB system only allocates 815,155 possible files. You very possibly overloaded the system and it can't allocate memory for the process to to be able to open the necessary files.
You really shouldn't need to change the value.
By the way, the modern kernel already uses a rule of thumb to set file-max based on the amount of memory in the system; from fs/file_table.c in the 2.6 kernel:
* One file with associated inode and dcache is very roughly 1K.
* Per default don't use more than 10 of our memory for files.
n = (mempages * (PAGE_SIZE / 1024)) / 10;
files_stat.max_files = max_t(unsigned long, n, NR_FILE);
and files_stat.max_files is the setting of fs.file-max; this ends up being about 100 for every 1MB of ram.
From that estimate, 4,000,000 files would need about 40GB just for the file table (4,000,000/100 =40,000 MB). I would expect the system is hung.
If you can boot into single user mode you should be able to remove/comment out those lines you put in /etc/sysctl.conf to recover.