Honestly I prefer it on a boy, but do some careful scrutiny of the type of community you live in and how long you'll live there. I wouldn't worry too much about his adult interactions with others as an Ashley, it's the early years that lead to loving or hating your name. If he's a self confident adult Ashley, he's not going to give a hoot what the teeny boppers say about his name, or the times when people will assume Ashley is a girl. If he had a troubled childhood, and battled his peers over his name from early on, he's going to enter adulthood a lot less sure of himself, and hating his name. Obviously you can't guarantee he'll never get teased as a child or teen, but I think most of us can get a pretty good feel for our community, large or small. Start reading the birth announcements. Get a feel for what his peers will be named, look around and explore how established gender roles are, etc.
I met a male Lindsey as an adult (after knowing half a dozen female Lindsey's) and he was the epitome of cool. I'm serious, this guy has his picture in the dictionary by the word "cool". I doubt he got teased a day in his life. Self confident, creative, open minded and tolerant, and very well adjusted.
I'm also done to death with the train of thought that boys with effeminate names will struggle all their lives, but girls with masculine names will succeed; girls "stealing" the boys names, and the names then becoming tainted and unusable by boys because :shockgasphorror: people will think he's a girl! People, this is coming from us, not from our children. Yes, if we continue to follow these antiquated beliefs about gender roles, they will continue. If we stop passing on this nonsense, eventually the tide will change. Not for our generation, maybe not even our children's generation. But their children? Yes. They may just live in a much more tolerant world than we do.