From what I understand of german law, I could be arrested for that! hehe.
Nope, you won't get arrested 'cause youre not wearing/showing signs or symbols of an anticonstitutional and therefore outlawed party or organisation. But be prepared to get lots of hostile reactions (e.g. it's quite likely that you get thrown out of the hotel your staying in - happened recently to a bunch of neo-nazi activists in cologne)