As you see, you were absolutely right to think this was a kernel setting: sysctl gives you considerable (and dangerous!) powers over the kernel.
http://access.redhat.com/documentati...c-dir-sys.html
See also the man pages for proc and sysctl.
Oops! Our posts have crossed! Amazon Linux in a cut-down version of Red Hat. This would work in CentOS, but evidently that bit was thought unnecessary by Amazon.
You could try adding
net.ipv4.conf.all.disable_ipv4=1
net.ipv4.conf.default.disable_ipv4=1
to the end of the file /etc/sysctl.conf and see if that works.
On reflection, I'm not sure that you can disable ipv4 and leave ipv6 in place: I think ipv6 may require some ipv4 code, but this is all a bit beyond me. Usually people want to know how to rempve ipv6. Perhaps you could tell us why you want to do this? There might be another solution to whatever your problem is.