I am assuming that the Blue and Gray goods along with the Sheep and Cows score VPs at end of game. Does the bag numbers on the Orange tiles score or are just used to indicate the good passed out in the Harvest phase?
The numbers on the orange tiles only indicate which harvest phase they apply to. They do not get scored as points.
Also, your assumption about the blue goods and special items is incorrect. The numbers printed next to the sword hilt only signify the strength of a raid or pillage action required to take the tile. They do not count as any points for final scoring.
Goods are technical points worth depending the size of the good. But only if you place them in the -1 areas. For example: wool has the size 2x2. If you put it somewhere in the -1 area you prevent -4 points what's actually the same as gaining 4 points.
If you upgrade green goods to blue it's easier to place these goods on your board because of the rule that green on green tiles is not allowed.