The walls, windows, roof, insulation(or lack of it),doors, wind-resistant wrap and other features. The heat loss/gain of any structure is heavily influenced by the condition of the structure's "envelope".
The building envelope is described in the other post, but it is also the primary heating and cooling "system" that affects indoor environmental quality. The houses we build in North America are not much better than "wooden tents", and in order for both high indoor environmental quality and low energy use, we need to tighten up the building envelope FIRST before designing "active" HVAC systems.
The HVAC designer MUST know how the thermal loads act through the building envelope before applying indoor comfort systems. I've yet to meet many people that understand what "overall" thermal performance is, as opposed to just taking the insulation R-value and centre of glass thermal performance and using those numbers in canned programs.