I just found out that gradle will first resolve all plugins and then apply them afterwards so that was not part of my expectation, however it seems reasonable. So the concept will work for dependencies but not for plugins.
So I have to go back to the drawing board and figure what alternatives I have.