Fish vs. food imports on an island

I've recently converted to a vegetarian diet, and been gradually pushing towards a vegan diet. When thinking about the challenges ahead - my family will be traveling to the Caribbean next winter where diet revolves mainly around fish. I'm thinking that in terms of environmental impact, supporting the local livelihood of fishing is superior to supporting the huge fuel costs of flying alternatives in. From an animal rights standpoint though, fish is fish and killing is not justified. Curious how people who have been doing this longer than me approach decisions like this. Thanks!

Edit: thanks for the replies ! Making me see quite clearly that choosing not to eat fish there is the correct path