I sure could use some help.   Last year we purchased a 1970-built California Ranch house in Annapolis MD, with a 40 year old fuel oil boiler, and banged up but mostly functional fin baseboard heaters. R-9 insulation in all walls and cathedral ceilings - whole house is cathedral ceilings except garage.  Tile and brick floor throughout house except bedrooms (carpet).  Huge amounts of original (read: air leaky) mixed casement and double pane fixed windows.  Mold issues, so MUST control humidity.   EXTREMELY shady lot, four foot overhang from roof to exterior wall.   Main floor is about 3500 sq foot with corresponding full basement, basement uninsulated.  Five zones for baseboard radiant heat.  Two separate zones with air conditioners (a 5 ton and a 4 ton), both needing to be replaced.   Currently running in addition to the AC units two big dehumidifiers in basement full time.  Bathrooms do not have vents.   Had blower test done.   Already replaced all the 250 watt bulbs (yes, 250 watts) with CFLs.  Insulating all the copper tubing saved us about 25% on fuel bills immediately.  We actually thought we HAD radiant floor heat until we saw the pipe was uninsulated!

Current summer plans:
1. Foam insulate cathedral ceiling cavity, add rigid insulation on top, replace roof.
2. Insulate cement block basement with rigid foam panels (limited insulation on floor due to ductwork placement but we're hoping for R-5 at least on floor)
3. Air seal through foam/caulk main floor windows and other intrusions.
4. Install ERV
5. Replace heating system (as mild a winter we had and we were spending $1000/month at 62 degrees thermostat. Hellooo space heaters!)
6. Replace AC units soon, possibly this year depending on heating system solution.
7. Future years: add wall insulation beyond original R-9 batt fiberglass insulation.

Questions- and really, any advice on the whole project would be much appreciated.  (I saw earlier posts talking about the caulk guns, etc. so will gratefully mine those for applicability to my project.)
 

-Is radiant heat practical in this size house?  What options should we seriously be considering?   I'm finding each vendor wants to sell what he has and it's tough to find someone who wants to talk to us about real tradeoffs and practicality across multiple options.  
-Most of the main floor has a layer of concrete into which the almost 1/2" thick hexagonal tile was set.  We aren't in a position to replace all the flooring in the house this year.  Is it feasible to convert to radiant heat in multiple stages?
-At the current 5 zone configuration for baseboard heat, it looks like either a fuel oil boiler or running natural gas to the house plus boiler will have a payback of 3 years (natural gas has much higher up front cost but much lower $ per BTU).  Should we look at replacing the old fin units and be done with it? 
-because we are in a position to replace the AC, should we seriously consider geothermal despite its $$?
-If we do something like mini splits in the key rooms (kitchen, living room, 2 bedrooms), then how do we control moisture in the rest of the house for AC?  
-If it matters, the AC ducts in the house are as follows: Zone 1: all bedrooms are in line down one hall, the register and air return in each room is at least 9' up the wall on the interior-most wall of the house.  Zone 2: (living areas) - all ducting runs along the floor with one air return up high near the air handler.  No air returns in the basement (doesn't need it now as the air in basement is super cold and moist).
-Can't afford to replace windows this year (this will need to be done with "exsulation" of walls in future years). I saw some posts regarding possible exterior windows.  Since the windows are a future-year project, should we be considering these? There a couple of walls of windows (mostly fixed-double pane, three sets of sliding doors). Last year we put up plastic on outside, clear shrink-wrap on inside.

Thanks. 

Compare the cost of running the gas line to the house (often pretty pricey) plus modulating-condensing boiler system against running the radiant with either GSHP ("geothermal"), or high-performance air source heat pump with hydronic like the Daikin Altherma sized for the "after" picture of the heat loads after all of your envelope upgrades. The total tonnage after improvements is likely to be half of what your current heat loads are, and every ton of geo costs, even with subsidy, and even the biggest Daikin might not keep up with your current heat load, but likely would for the "after" picture.

Keeping the oil boiler & baseboards in place as backup might be the right way to go until the building improvements catch up.) Even if the heat pump only keeps up at 70% of your current design load it would reduce oil consumption by 90% (replacing it with an electric bill at something like 1/4 the cost of that avoided 90%.) With the 30% tax credits a down-sized GSHP may have lower net upfront cost may be less than bringing in gas and a boiler sized to next-year's load, and the operating cost would be lower. The high-performance ASHP would likely be similar in overall upfront cost to gas, but may still be cheaper to operate (or not you may have to sharpen your pencil on the napkin-math.) If you do a good job on getting the water temps down on the radiant the Altherma could cost about the same as the GSHP system to operate, given your relatively mild outside design temps, but the installed costs relative to GSHP depends a lot on the levels of subsidy.

A new oil boiler would likely be money wasted- oil pricing seems poised to continue to be both more volatile and more expensive per BTU than gas or heat pumps for the forseeable future. You might improve performance from 65% to 88% and cut heating costs by a third, but going to heat pumps or condensing gas would cut it by 3/4 or more.

Air sealing the roof & basement should take precedence over air-sealing the windows, since stack effect is a bigger infiltration driver than wind (but in Annapolis wind driven is still signficant in exposed locations.) Blowing cellulose into wall cavities over your low-density batts will improve overall air tightness, and thermal-bypass stack effect losses significantly, and is a much cheaper retrofit than non-expanding injection-foam solutions. Cellulose isn't perfectly air-tight, but can be a 90-95% solution to the wall-tightness, and when you get around to adding exterior foam you can detail the sheathing or housewrap as a primary air barrier. I'd move this ahead of insulating the basement slab, as it'll be cheaper (likely to even be subsidised), and will have a far greater impact on heating & cooling loads.

It's good to see basement insulation on list. Next to attics it's usually the most accessible and easily achieved BIG reduction in heat load, and far more cost-effective than replacement windows, but it should include air-sealing the band joist to the foundation wall-foam with 2" of closed cell spray foam. In MD going with 2" of EPS against the wall + trapped in place with a 2x4 unfaced-batt insulated wall would bring it up to about R20 performance at relatively low cost. If there's no room for studwalls 2" of fire-rated iso (~R13) is probably a better choice. I did my basement with reclaimed 3" fiber-faced roofing iso held in place with 1x furring on which the gypsum ignition barrier is mounted. In tighter places I had to dado-out channels in the iso to accomodate the furring.

Since you're committing to exterior foam on the roof, you might look into buying a truckload of reclaimed roofing iso (see insulationdepot.com if you don't have local sources) and use it on the basement as well. It runs something like 1/3 of the cost of virgin stock, even in near-perfect shape. Putting even 6" on the roof deck held in place with 2x furring through-screwed to the exising roof deck, and adding a vented 1/2" OSB nailer is surprisingly cheap compared to say, 2" XPS and has triple the R value. But even 3" (~R18-R19) in combination with 5.5" of open cell foam in the 2x6 rafter bays would beat current code-min for MD.

Downloading and modeling the house in-situ using BeOpt (http://beopt.nrel.gov/ ) at it's current and future R values is more useful than standard heat load tools for figuring out both present & future heating/cooling loads, but also in terms of which order to stage your improvements, and calculating the cost-effectiveness of each. (eg: R50 roofs using reclaimed iso pay off pretty quick with oil as your heating fuel, but with virgin-stock iso and GSHP heating it may take awhile.)

Low-E storm windows over clear-glass double-panes will outperform almost any double-pane replacement (but not better-grade triple-panes.) South facing double pane windows are best left as clear-glass and protected from summer sun by awnings or overhangs if they actually get winter sun but all other sides would do well with low-E storms. (If the south side is shaded even in winter, go low-E there too.) But if the plan is to replace them with
The current air conditioning tonnage is ridiculous, (even at current insulation levels), but conforms to the old-schoolers "Lessee, 3600 feet at 400-500 foot a ton comes ta..." type calculations. The "after" picture would likely have peak loads less than half, maybe even less than 1/3 your current 9-ton system(s), given that it's in the shade. It may or may not be worth trying to keep the existing ducts- if they're leaky & uninsulated &/or unbalanced in design it might be cheaper and better to rip them out and start over.

Dana1, thank you for such a comprehensive answer to my admittedly broad questions.   Additional questions for this forum:
-If I stick with a condensing boiler with natural gas, what options do I have to retrofit to radiant heat flooring from fins later?  Some of the fan-blown radiators have seized fans, and I cannot afford to rip out all my flooring this year with such a big pricetag for roof and boiler/heater unit this year.

-re-used panels: Yes, I called a while back to get a sense of what was available, and plan to use this if at all possible.  Pricing was amazing compared to new.   I'd like to price SIP (single sided) as well for a baseline, since that handles both the XPS and the wood decking for the roof - potentially saving us in labor. 

-Any thoughts regarding removing moisture from the house? The less used spaces (basement, spare bedrooms) are the ones previously having mold, so we must handle humidity even if we don't "air condition" for cooling purposes.  With such large overhangs and a shady lot, we don't have to run the air until temps break 90F, which also presents a humidity challenge in wet springs.

-Ceiling -For cost purposes, I was considering cellulose in ceiling (9" or 10" rafters, by the way) and foam for the exterior edges where the wall meets ceiling and the 3' roof overhang area.   

-Each exterior wall has a number of randomly placed fireblocks. I made the assumption that attempting to fill randomly spaced cavities without ripping off either siding or drywall would be a nightmare, so I put this on the back burner.  Any thoughts?   As recommended, in future we are definitely going with an exterior house wrap and some sort of rigid foam on the exterior.  If we replace the windows we will do all of this at the same time.

-Cedar siding on the outside of the house (1970, so it was a two-thickness plank hung vertically).  By any chance does anyone know how to "deconstruct" these for possible re-use?   (I'll post this in the appropriate forum when we get closer to doing this work).

-Are there any suggestions on how to find someone local who can help us wade through all the salesmanship to get to best choices?   I am learning enough to be dangerous but a local trusted advisor would be really helpful.

THANKS.

Dana speaks well to the insulation issues so I will stick to boilers. A good condensing boiler (Viessmann is my pick) can do both baseboard rad and/or floor heat. The difference between the two is the temp that they need to run to get same heat output. Floor heat can be 90-100F and baseboard rads could be up to 200F. A condensing boiler will have its best efficiency at the lower temps. As a temporary fix, baseboard rads are quite cheap which is why they are used so much so adding one or two is not too painful.

On a house that is mostly in the shade using the half-SIP type nailbase panels may not be a great idea, since the shaded roof won't have as much direct solar heating to assist in purging the nailbase OSB of moisture through vapor retardent roofing materials. It would work OK if you used metal roofing mounted on purlins with provisions for cross-venting though.  I've seen deep energy retrofits where an OSB nailer deck was laid directly atop the rigid insulation, with felt + composition shingles directly on top, but in that layup roof leaks would go undetected for years, saturating the iso and rotting the OSB, which is why my preference is to vent the nailer deck.  It could still leak for years, but the damage would be more localized and easier/cheaper to repair. But it's an additional expense, and not required by code, I know.

To be able to use (air permeable) cellulose in the rafter bays, in US climate zone 4A the IRC2012 spedifies a minimum of R15 above the roof deck for dew point control, but given that it's in the shade with lower average deck temps I'd  want to bump that to R20-min.  With 2x10 (9-1/4" nominal depth) the cellulose would give you ~ R32 center-cavity. To hit R50+ for whole-assembly (thermal bridging of the rafters & ridge etc included) you'd want to be putting R25-R30 on the exterior (4-5" of iso.)

When you insulate and air seal the basement, the basement will be warmer (lowering the condensation potential) and there will be less moisture introduced by humid summer air.  Sealing both the walls and slab with a silane/siloxane based masonry sealer is worth it, even if you're adding a poly vapor retarder under the slab-foam (mandatory), to limit ground water moisture migration.  It's still worth setting up a room dehumidifier to drain into a sump or drain, and to limit the RH to 60# for mold-control, but with the sealed insulated basement it's duty cycle will be a fraction of what it would be before taking those measures.  Active ventilation needs to be considered for the whole house now that you're committed to tightening it up and at your summertime dew points an ERV would be preferable to HRV to limit the amount of moisture brought in by ventilation air.

If you intend to cool with a mini-split, the Daikin Quaternity series has the capacity to dehumidify to a programmable RH independently of whether it's cooling or heating, so if the room is already below the cooling setpoint temp it can still dehumidify without it cooling the room.  I don't know of any other GSHP or ASHP systems that have that capability-  it may be worth considering heating & cooling at least one decent sized zone with a Quaternity just for the dehumidification factor.  (They still put out a lot of heat at -4F, and your 99% outside design temp is above +15F, so don't rule it out.  At +15F the efficiency is still pretty good too.)

Retrofitting cellulose into stud bays with randomly placed fire blocking is par for the course- it's done every day.  A dense-packer can usually get there drilling just two holes per bay, one near the very bottom, the other near the top, but if the batts are too sagging or bound-up to snake a dense-packing tube they can still get there with smaller holes drilled every 3' or so.  Even if for some reasong they can't hit 3..2lbs+  density, the improvement on infiltration and thermal bypass aspects is worth it.  It may or may not be easier to dense-pack new-school fiberglass in there, but at 1.8lbs+ density Optima/Spider/L-77 can be about as tight as cellulose (the manufacturers' say tighter than cellulose, but that may only be in the lab.)  Since you'll be stripping the siding at some point but want to salvage what you can, you can ask that the holes all be drilled in a single vertical plank rather than randomly, or (as is commonly done) to pull a plank and drill only the the sheathing (or the inner plank that would be covered.)  Most installers will use pre-fabbed plugs for their drill holes. If they drill through the finish siding you can request cedar plugs  if you care that it matches.  If it's painted/stained siding rather than the weathered-cedar look you'll have some touch-up to do. 

Get some references to local blown cellulose & fiberglass installers and get a few proposals on the table that also include the cathedral ceilings and walls as separate line items.  Doing walls often surprisingly inexpensive- leaving the batts in place (which I would, unless they're dead-easy to pull) the volume of material is pretty tiny- it's all in the labor of popping & reinstalling the siding and the drilling- the actual blowing goes very quickly.  Dense packing the ceilings in netting is likely to be by far the bigger expense (and it is sometimes cheaper to use open cell foam on sloped ceilings- get quotes.)

Don't get caught up in the relative R-values between competing blown fiber products- it's the air-retardency of a higher-density installation that is going to make the true performance difference. (1lbs density installations of fiberglass should not be considered, since it's not sufficiently air retardent.)   Whether R3.4/inch or R4/inch, in a 2x4 studwall with fire-blocking it makes less than R1 difference in whole-wall R after the thermal bridging of the framing is factored in. That difference made up  for with less than 1/4" additional exterior iso.

BTW: Are the facers on the existing low-density batts foil or kraft paper?

Hydronic fan coils with seized fans are kind of a pain to deal with, since you get nearly zero heat out of them until you repair or replace. If you go with low-temp radiator panels now only to abandon them after retrofitting radiant floor it's not money well spent.  Fin-tube baseboard is relatively cheap and you could get by with it, but cheap is not the same as zero-cost.  In general it's usually better to defer heating plant upgrades until after the bulk of the envelope fixes are in, when you have a better idea as to what the final heat load will be, since if you pull the trigger too early the heating plant will be oversized for the final load which makes it  more expensive and less efficient.  But in the face of $4 oil vs. $1 gas it may more to wait if it's going to be 2-5years.  If you upgrade the cooling first and make it a heating/cooling heat pump rather than cooling-only, even if it didn't have capacity for the whole heating load at the coldest hours of winter it would still decimate the oil bill.  (Just the air sealing and insulation you have on the books for this year would likely cut oil use by about 1/3 if you follow through on the wall insulation too.)

Also consider, if you have an R50-ish roof and ~R30ish whole-wall-R ( 3" of exterior iso on the cellulose 2x4 wall would put you well above R25), and