Thank you for the suggestion. I looked it up and it seems the setting that manages it is "Subscribe by default when I post or reply to.. - Threads". However, with this setting I'd be automatically subscribed not only when I create a thread but also whenever I respond to a thread. Am I correct and if yes, is it possible to separate this setting into two?
If you look further down that page you should see special subscriptions. I think what you want is to set "Replies" to Yes.