I think I'm leaning towards the barn on #3 too. I think overall, it's just going to work out easier to have the barn on the same side of the road as most of the pasture space. #2 would be a good spot for an arena or even as a backup pasture with a run in shed. So even though it's closest to the house, I think I'll end up spending less time/effort making things function if everything was on the same side of the road.
"Farming looks mighty easy when your plow is a pencil, and you're a thousand miles from the corn field." --Dwight D Eisenhower
You want barn close to house when there's bad weather, something you need to do in the house but you're waiting on farrier, vet, etc... you get the picture. Also for power and phone line considerations.
Small paddocks attached to barn stalls. Use parcels #3 and 4 for arena and pasture. If road ever started getting heavy traffic that way you could put a "bridge" over the highway - expensive but safer for the critters/people crossing the road. (The way it is now may not be the way it is 50 years from now).