I think any species of fish/invertebrates we have, including gold fish are invasive species when released into our waters, streams, and lakes.
Even if the fish die it may be a host of other 'things' that may attack our local plants and animals/environment.