Hello!
Oh, we never noticed this issue with the official servers as the official map is all surrounded by water and all the cliff areas are accessible.
There is a way to mark parts of the world as restricted area for objects spawn. It's either doesn't work for these pragmium shards or the map creator didn't marked the cliff areas as such. Will check soon. Thanks!
Regards!