It’s not supposed to work that way. Normally gradleApi() behaves the same way as a dependency placed iin Maven “provided” scope. In other words your POM should not explictly declare it as a runtime dependency as Gradle will provide it.
(I originally assumed you are building the plugin with Gradle, but are you building it with Maven?)