I would avoid planting bushy plants right next to the ring, because that is where spooky monsters live. Best to leave a spook-free buffer zone. Plus you don't want branches or shoots extending into the ring, or close enough to serve as an over-the-fence snack bar since you are using the ring for turnout.
Ditto what the others said about checking the toxic plant list. Your state ag department likely has a list of livestock-toxic things that grow where you live.
"I always like the look of a "living fence."
I wonder how hard this is to maintain? http://oldsalemfarm.net/facilities"
This is lovely, but it looks like a Yew or Boxwood shrub...both of which are highly toxic to horses. I'm sure a lovely facility of this class must know this.
I wonder what types of Evergreen shrub you could use to make this look and not worry about poisoning your house if they take a nibble.