In medieval times, many household containers were made of an orange clay called pygg. When housewives had spare coins, they collected them in their pygg banks. Around the end of the 18^th century, an unknown ceramist  formed a pygg bank into a pig shape.

source: Mistakes that Worked (36-7)