What area? Consignment tack shops are a good place to start if you have some around. Also some tack stores offer packages for beginner riders which include things like a helmet, gloves, and paddock boots for one price. (I don't know what your level of horse experience is - the helmet is the one item that should NEVER be purchased used.) And do hook up with the parents of slightly older/bigger kids - you can get a steady stream of hand-me-downs, then find an even younger kid to pass them on to next.
Definitely check with your pony club. Ours runs an Outgrown Shop as a fund raiser for the club that is stocked with all sorts of clothing and tack that is donated by members and by the surrounding horse community.
It would help to know what area you are in because there are a few tack shops around here that have a consignment area or offer discounts to pony clubbers.