Eldfjallasafn means "Volcano Museum", and Stykkishólmur has a small but nice one.  They feature the unique collection of professor Haraldur Sigurdsson who has researched volcanoes for over 40 years.