Two places will come to mind to any New Haven-ite: Sage, and Leon's. Both are just okay generally American food dining fare.
If you are willing to go a little farther afield but still in the area, I've never eaten on the water in either place, but Milford and Branford are both good nearby areas to explore.
Thank you for writing to The Inn at Longshore to ask about your rehearsal dinner. There is Splash Restaurant in Westport, CT, which is lovely and located on the water in Westport, which is about 25 minutes from New Haven. Another lovely idea, perhaps would be to have the rehearsal dinner prepared for you on a small chartered boat - which would be quite unique and lovely for you.
Splash's telephone number is (203) 341-4666.
Congratulations on your engagement and good luck to you!
The Inn at Longshore
Unfortunately I haven't heard such great things about Leon's and I'm not sure about other waterfront restaurants. We had our rehearsal dinner at Fosters on Orange. The owner, Dave Foster, is wonderful to work with and the food is fantastic. They have a private room you can use. We had a wonderful time and I would highly recommend them if you are not tied to the waterfront.
Dave Foster recently left Foster's as far as I know.
I've been to Sage for brunch and it was nice.
http://www.jenelleswaterfrontcafe.com is more casual but on the water. Friends went to brunch there and liked it.
Click on each tool to see a sample.