The last month in a nutshell can be summarized as follows:
First there were a few exchanges of emails in which my complaint was met with some box-standard replies not acknowledging the issue or willingness to resolve the issue. I then wrote a lengthy email in which I described in detail my reasons why they should have stopped charging me 2 months ago. Afterwards, there was a dead silence from them. Every 2-3 days I would send an email politely demanding a reply. In my last email around the time when I started this thread, I mentioned two things:
- the fact that I've been their customer (on and off) for the last 4 years (they used to have different subscription models) and the way they are going to handle this complaint will obviously influence my decision if I'm going to use their services next season/year.
- I asked them if they would like to state their reasons before I take the matter further.

I don't know what has made them change their mind. Interestingly, they haven't changed the wording on their website so I don't think they really see they were in the wrong. It was probably done to shut me up and count on me renewing subscription next year (which I might do, because otherwise I haven't had any problems with them until last month).

Exactly right. You persisted, you didn't seemingly scream. You stated you points probably in a clear and definitive manner, while also pointing out that you were a historical customer, one who come and goes, and one who would base their inclination to use their service in the future, conditionally. So the "right" person, evaluated that and made a business decision which is to your preferences. And correct, you may use their service again in the future knowing that they've listened to you now. I've experienced similar. And in my cases, as I suspect yours, they benefited by having you subscribe for longer than what you're asking for, in a refund, and likely your waning days where you did still use it and are not requesting refund, you likely were not too active, but you are conceding that you did use their service. I'm sure they can see that too.

Laws in the US are written to protect banks and large corporations. They pay the politicians to do that, and it's extremely effective. Bribing politicians is completely legal, as long as you call it a "campaign contribution".

If PayPal is involved, start there. They seem serious about keeping customers happy, and you have the best chance of a refund there. Going to small claims court isn't a viable option for most people. Using PayPal for online purchases seems to me to be the best route if you don't have a credit card available. Personally, I tend to use PayPal with a credit card as the payment option. I have a small bank account set up to keep PayPal happy, and if that gets looted it won't bankrupt me, but I generally pay with a credit card, not the bank account, just in case.