Can we get some examples of the text involved? If the lines to be matched are identical, then it should be easy. But there may be more work involved if you need to make partial matches, or if there's a chance of oddball characters or the like.
Here's a grep one-liner that should work in the simplest case. You'll have to direct it into a new file, as grep doesn't have in-place editing.
Code:
grep -v -F -f "smallfile.txt" largefile.txt >newfile.txt
-v invert match (print non-matching lines)
-F fixed strings (for efficiency and to avoid regex problems)
-f match strings from filename