If you just get rid of the need for the "draw card" dice symbol, then I think it works.
Just change it to "On your turn, draw two cards, keep one." with no prerequisite for a specific die to be drafted.
My thoughts exactly. It is worth it when you need a card and didn't get the die roll, but as an extra choice (not even an extra card), it isn't close to worth the penalty.
My (less radical) house rule would be that you can draw two cards with it, but you only pay the penalty if you choose to keep the bonus card. Then it would be worth it.