For example, if I’m buying Red Oak floors, what’s the difference since the material should, by definition, be the same? Why are the specialty stores so much more expensive?
All floors may look the same to the eye. However, while there is nothing wrong with big box stores, unless you buy a name brand, there is a difference. Also, buying from a mom and pop type operation I find affords you a more personal approach. Plus, that is all they do while a salesperson at a box store may be in floors one day and blinds the next. Again nothing wrong with this, except non box stores are more personable.
Being a specialty store, it may seem a little biased of me, but I’ve worked for a couple big box stores and now I work at Metro Floors. Big box stores make their money by buying at higher quantities from bigger wholesale companies that can provide the amount of flooring they require. Most big box stores don’t have great service in terms of make sure the customer is 100% satisfied. Smaller stores or specialty stores, as said by Fred Allen Builders, are on a more personal level and they are not just selling you the product but selling you their service as well. Yes it could be a little more expensive, but you pay for what you receive. Another thing is that once you establish a relationship with the peopld in that store they might give you discounts. 
It depends on the brand over the place. You can buy Bruce hardwood at Home Depot which installs pretty nicely and has a pretty decent life (prefinished). However, I’ve seen other brands that don’t have 90 degree end cuts or they’re warped, which makes a nice finished product impossible. Stick with decent name brands and you’ll be ok. Do your research before you buy!!