We walked on the canal this afternoon and noticed three separate caves which weren't blocked off. We didn't explore so we have no idea how deep they might go, but you may want to have a look.
The first two were pretty close together. Go to the Dam #5 parking area (it's on Dam #5 road) and walk west on the towpath about 1/3 of mile. You'll see the first cave about halfway up a cliff to the right side of the path, just after the canal dumps into the Potomac for a stretch (this is just when the path starts to curve around these huge rock outcroppings, but before the path turns to all concrete). The second cave opening is a very short distrance further west on the path, and it's right next to the path once it's turned into the elevated concrete section. Like, you could climb right up into it from the concrete.
The third cave is maybe 3/4 mile further west from the elevated concrete section of path, about halfway between the "Two Locks" area and the "Four Locks" section. It's on the other side of the actual canal from the towpath, just a few feet abpve the canal level, in a hillside about 40 feet to the right of the towpath. Since there's no water in the canal you could just climb through a few sticker bushes to reach it. It looks pretty small, about 2'x2', but then I guess you can't judge a good cave by the size of its opening.

There were ruins of an old mill pretty close to it, and they looked very cool too, though they were fenced off to discourage exploration.
Good luck!