Tim,

Its mud daubers so , well mud - I would soak and blast them with water - a little dish soap might help. Any remaining dirt can be a source for corrosion to start so I would elect to flush them out as best as possible. A little cleverness with a bottle brush or extended gun cleaning rod might help with the spar - if the outboard end is not capped should be doable. Dragging a rag through the spar on a string might help with dislodging the stuff. Bottle brushes which come in different diameters have a wire twist core which can be bent to work around the corners of the hanger bays.

Others might have a favored method but I'd sure try to get the stuff out.