In medieval times, many household containers were made of an orange clay called pygg. When housewives had spare coins, they collected them in their pygg banks. Around the end of the 18th century, an unknown ceramist formed a pygg bank into a pig shape.
source: Mistakes that Worked (36-7)