I don't think that is really the way to speed it up because when you compile, only the widgets you use are included. I think the thing we should be concerned about is making those widgets faster.
Recently I have taken it upon myself to try and work on a widget and make it faster. Because Gtk is so structured, it becomes bloated very fast. There is an interesting presentation at http://primates.ximian.com/~federico/docs/2005-GNOME-Summit/html/index.html
that you might want to check out...
I know this doesn't really answer your question directly, but maybe it will inspire you or something or not...lol