I went last year and planning to go again this year, but only because I'll be in the area. It's probably not worth a major trip to get there.
It's about an hour's drive west of SD, so it also gives you an excuse to check out that area or do a side trip to Barahona, Lago Enriquillo, etc.
But it is definitely more than a pile of mangos you can see anywhere else. Besides the mangos, there's a farmer's market/county fair vibe with lots of fresh produce and homeade items for sale. The best part, and why I'm going back, is a mango tasting tent that allows you to try around 25 different varieties of mangos, seeing the difference in flavor and texture, and decide which ones you like best, so when you see different ones in the store you know which ones to buy.