I used Ruthies Cakes and was very pleased with the result http://www.ruthiescakes.net/
We got married at the Cherry Valley Lodge in Newark - it was perfect for us because it was all inclusive - we had the ceremony in their outdoor garden, the reception in the banquet hall, and the lodging for all the guests was on site. Compared to other locations we checked out, it was on the less expensive end (still expensive though!).
McClain Creations (mcclaincreations.com) was our baker, and they did a FANTASTIC job. It is a husband/wife team that bake out of their house, so they are extremely reasonably priced. They will make anything you want and it will taste delicious!
How many people are you having? What are you looking for?
The Venue at Smith Brothers! Can be very inexpensive, you can have the ceremony on site, it's BYOB (huge savings) and the Venue itself is amazing, beautiful view of the downtown skyline...people there are great too!!
You can try St. Dominic Service Center. The hall seats 240 and is located just east of downtown. Alice's Piece of Cake is also very good. She may be a little pricey but worth it.
We loved the Old Gahanna Sanctuary, it was inexpensive to rent, you can bring your own booze and any caterer, and the tables and chairs are included.
Click on each tool to see a sample.