Try shopping at specialty markets such as Korean, Japanese, Middle Eastern, Indian etc. Not only do they have some of the best looking and most exotic produce, there prices are amazing when compared to your neighborhood grocery store. All of the Shiitake mushrooms shown above were about $2.32 US. When you buy them dried, which is usually the only way you can get them at the grocery stores, they are about $8.50 for approx 6 mushrooms. BIG SAVINGS!
Shop the sauces and spice sections as well as you can find tons of things to experiment with to add that extra zip to your dishes. Don't know what something is, a quick google of the item or write it in the notes section of your phone and look it up later along with recipes using the item. Have fun on your journey.