Your biggest problems will be getting a development permit and keeping mold away. The magnitude of both problems will depend upon your location of course.
In most places building codes exist only for 'normal' building techniques. Where I live, you'll need to dish out a bunch of cash for the variance application, architects and engineer fees to create and sign off on the plans, novel plumbing and electrical installations, etc. Not to mention the hassle of installing finishing on un-square everything (walls, floor, etc). And all the trades you bring in to help will probably charge more because it'll take them longer to figure out how to do things.
But of course there are a million reasons to never do something cool like this. If I had some land in the country I'd totally do a straw bale or rammed earth outbuilding. (I think those earth ships looks ridiculous).