Well, upon writing this article, it ocurred to me, that the abstract of the article will very likely be shorter than the article itself. That is due to the fact, that I summarized the thougts a while ago in form of a pdf document already. The report be downloaded here.
Later a student, Özlem, did look further in the problems of structure in reinforcement learning and conducted expereiments within the game of tic-tac-toe. The results she acquired can be studied in her project report.