Build slowdown after enabling file-system watching

I enabled file-system watching by adding org.gradle.unsafe.watch-fs=true in global gradle.properties.

After that I see the following messages in console:

Could not connect to kotlin daemon. Using fallback strategy. - this happens for every module.

RMI TCP Accept-0: accept loop for ServerSocket[addr=localhost/127.0.0.1,localport=50888] throws
java.net.SocketException: Socket closed
        at java.net.PlainSocketImpl.socketAccept(Native Method)
        at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:409)
        at java.net.ServerSocket.implAccept(ServerSocket.java:560)
        at java.net.ServerSocket.accept(ServerSocket.java:528)
        at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:405)
        at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:377)
        at java.lang.Thread.run(Thread.java:748)

Build succeeded, but it took much longer than usual.

MacOS 10.15.4
Java 1.8.0_232
Gradle 6.5
Kotlin 1.3.72

Only one daemon is running and it was started after changing gradle.properties.