QTextLayout carefully considers soft hyphens to understand where to wrap the line. But their size is considered zero at alignment stage so hyphens brakes out of text boundaries with Qt::AlignJustify.

Example qml file and it's screenshot produced by qmlscene are attached.
Behavior is reproduced even if QTextLayout is used directly, so it's not QTextDocument's bug.