More or less, yes. The non-devel rpm will contain the binaries, which may include executables and/or libraries - anything needed to run the software.
The devel may included headers and/or libraries and/or additional files. The contents vary based on the package. These are files needed to write software for the package or incorporating the package.
The easiest was to see is too look at a few packages on your system and compare the contents. For example:
$ rpm -ql bzip2-devel
$ rpm -ql bzip2