I am curious how your solution has provided you with your output?
Surely if you have gone to the trouble of editing File1, running your commands and then opening the outputed file to once again re-add the ip addresses, it would
have been quicker to have just manually created the file in the first place?
Maybe you could have tried something like:
grep -vf File2 File1