I managed to figure this out on my own, with the help of http://library.gnome.org/devel/gtk/2.12 ... Files.html
The widget name that follows the widget directive in a resource file has to include the complete "path" of the widget all the way up to the root window, not just the name you give the widget in the code. So if I had set the name of the GtkWindow as "main_window", the following line works
widget "main_window.hello_label" style "labeltest"
or even better, without having to give the GtkWindow a name, you could use
widget "*.hello_label" style "labeltest"
This isn't fully explained in "Foundations of GTK+ Development," and the gtkmm documentation, as usual, doesn't explain anything at all. The above reference seems to be the best explanation of how to use RC resource files.