mv -D.jpg picture.jpg
results in the same original problem. The mv command interprets "-D.jpg" as an option; not as a filename. It would work if you prefixed the -D with the current directory path ./ or the absolute path. That, however, would basically just work it's way back to a plain rm command.
And, just for a little info, there is a rename command. It's slightly different than what you might expect. You give rename specific text to replace, the text to replace it with, and then a list of files to perform the substitution on. For example
rename jpg jpeg *.jpg
That will rename every file in the current directory that ends with .jpg to end with jpeg. It will also replace jpg with jpeg if found within
the filename itself. In other words, pic01.jpg will be renamed pic01.jpeg. vacation_jpg.jpg would be renamed vacation_jpeg.jpeg
You can read all about it with man rename