We encountered a similar problem some years ago. Some folk objected to moving the reunion outside of the Memphis/North Mississippi area. Folk claim that the participation will be low if people have to travel. Well, the next reunion was planned for Chicago, and we proved the naysayers wrong. That Chicago reunion was better attended than the previous "home" reunions! We in fact realize that many other family members like the idea of seeing something new and different. Don't let the naysayers be the main voice.
I've personally seen some reunions get the "life" sucked out of them because they were held in the same place year after year. After awhile, folk got bored with them and stop attending.
My vote: Go for it!