There seems to be an issue in macOS Mojave beta where the tileset and other peripheral windows can't be resized. The main app window continues to resize fine, but the peripheral windows lack the "brail" in the bottom right corner; perhaps that relates to the problem?

It isn't clear if this is a Mojave regression to be fixed in FCS (Sept. timeframe), or, if it's the result of a bug fix in Mojave. I haven't experienced any window-resizing issues in other apps in Mojave, so there might be some nonstandard use case that's biting Tiled.