Thanks for your reply.
So, I have edited my code as following:
GtkWidget *Hbutton = gtk_button_new_with_label("H");
gtk_widget_modify_fg(Hbutton, GTK_STATE_NORMAL, &colorBlue);
gtk_widget_override_background_color(Hbutton, GTK_STATE_NORMAL, &colorRed);
gtk_grid_attach(GTK_GRID(grid), Hbutton, 0, 1, 1, 1);
/* Create second button */
GtkWidget *Hebutton = gtk_button_new_with_label("He");
gtk_widget_override_background_color(Hebutton, GTK_STATE_NORMAL, &colorRed2);
gtk_grid_attach(GTK_GRID(grid), Hebutton, 1, 1, 1, 1);
But, as I had a elaborate discussion with zerohour here
this is probably problem with the shell theme, as override don't solve the problem(buttons are showing color only when not in focus). This is a confirmed problem of the Adwaita theme, as I get the desired effect if I use other GTK+ themes(Like High(/Low)Contrast etc).
So, any advice please?