Along the Cliffside

This is a brief entry. I still want to develop the last encounter, but I feel like it needs a map and some numbers for monsters. Today wasn't the day for that, so I'd like to consider some other elements that would make this area interesting.
The hillside west of the lake is meant to be too steep to traverse. There are cliffs here, but they don't rise directly out of the water. The cliffs rise several hundred feet up the hillside from the lake. The danger here is falling rocks. The rock face of the cliffs is shale, and the base of the cliffs are steep, shifting piles of loose rock. Movement across the scree would be difficult terrain and require an athletics check each round to avoid falling or shifting the rock. The DC would be high, 25 to cross without incident, missing by greater than 5 would create a rockslide that would do 4d6 damage while moving you 30 feet downhill toward the water.
Attempting to scale the cliff would be difficult and dangerous as well, since the loose rock will not easily accept climbing gear such as picks and pitons. They are as likely to dislodge more rock as hold for passage. DC20 athletics check to climb with disadvantage. Failure means no movement or 2d6 damage from falling rock on a fail of 10 or more.
Stone Giants love these rocky walls, and can easily move up and down them without delay. They live in the higher ridges above the cliffs, and use this as a pathway down to the lake. There is a 1 in 6 chance of encountering 1-4 stone giants for each day on the lake. Often they can be seen wading or bathing in the waters at the base of the rockslide, though sometimes their mood is more playful, and they will stand high on the hillside and throw rocks at the barge as it passes.
The fort on the sandbar that is near the middle of the lake is closer to the cliff face than the swamp. No one feels safe near the swamp, and since it juts out near the middle of the passage, the bargemen have been forced to find a defensible resting spot closer to the cliffs. The giants like to gather at the top of the ridge and toss rocks down at the lights and voices coming up from the campsite.
The good news is that with every trip across the lake, there is more stone available to build up the fortification. At this point, there is a four foot high wall that encloses the area around the dock. They have even begun building a wall out into the water to serve as a breakwater or eventually to enclose the barge and dock within the stone walls. There is no shortage of material in this area.
There is a struggle against the shifting sands of the sandbar. It is difficult to build up since the weight of the stone causes it to settle more quickly. The currents (from the main river and a series of small streams that seem to drain out of the swamp in this region) also cause the sandbars to shift and make permanent building a difficult task.