Keep it in mind: you'll face that kind of errors when the linker cannot find a reference to a symbol, but find a library containing it.
You need to add the path to the library and, eventually, the pkg-config flag.
In fact if you run
pkg-config --libs libcrypto you get the flags for the libs to add!

To get the respective clfags, instead, you can run
pkg-config --cflags libcrypto (in this case none are required)