My interpretation of the rules, and the apparent spirit of the game, is that at least one edge of the tile being placed must be adjacent to an existing patch on your board.
The rules start with:
"Patchwork is a form of needlepoint that involves sewing together pieces of fabric into a larger design."
That's introductory flavor text, not a rule.
If you were required to place tiles adjacent to an existing tile, the rule section about placing tiles would certainly say so.