High rate of failure connecting to QWebSocketServer with Connection refused error

I keep getting failure connecting to QWebSocketServer with "Connection refused" socket error on new connections after running the server after sometime randomly and never recover until the server get restarted.

While I haven't manage to reproduce it in controlled test, it seems happen consistently when there are lot of connections. So I try to reproduce it with simple program based on QWebSockets that producing large number of connections in short time.

From my test of 1000 websocket connections creation to sslechoserver, it never able to successfully connected to 1000 connections. Comparing to node based websocket server, it manage to establish all the connections successfully.

While this test don't reproduce the state where the server never recover, the poor performance on handling large number of connections compared to node's websocket server might give a hint on the root cause.

Here are the test program used:
QWebSocketServer based server: sslechoserver example from qtwebsockets source
QWebSockets based client: https://github.com/smhaziq/wstest
node based WebSocket server: https://github.com/smhaziq/wstest/tree/main/test/server/js