Uploaded image for project: 'Qt Creator'
  1. Qt Creator
  2. QTCREATORBUG-27976

PythonSettings::detectPythonVenvs() stat's /net/bin causing delays when opening a .py file when host has autofs enabled

    XMLWordPrintable

Details

    • Linux/Wayland, Linux/X11, Linux/Other display system
    • 0a2510edcb (qt-creator/qt-creator/8.0) 0a2510edcb (qt-creator/qt-creator/master) 0a2510edcb (qt-creator/qt-creator/qds-3.6) 0a2510edcb (qt-creator/qt-creator/qds-dev) 0a2510edcb (qt-creator/qt-creator/9.0) 0a2510edcb (qt-creator/qt-creator/qds-3.7)

    Description

      Issue observed:

      Multi-second delay when opening a .py file in a non-python project when the host has autofs enabled with magic /net/<host>.

      Diagnosis:

      When a .py file is opened Qt Creator will attempt to detect the python version. It first checks the project and then falls back to calling PythonSettings::detectPythonVenvs(documentPath) to see if the python file being opened is inside a python virtual environment. This function starts in the folder containing the .py file being opened and traverses upwards to identify a virtualenv. This upwards traversal will eventually reach / where it will visit each subdirectory and look for a "bin" subfolder. One of the locations it ends up stat'ing is /net/bin. With autofs enabled this will cause autofs to try to resolve the hostname "bin". Depending on network conditions and resolver configuration this failed lookup can take a few seconds, causing annoying delays when opening python files.

      Steps to reproduce:

      • On Fedora Linux run "dnf install autofs" and "systemctl start autofs". To confirm that the setup for reproducing this issue is correct, simply run "ls /net/bin". It should take a short moment (a few seconds) before it returns with a failure to resolve the hostname "bin".
      • Start Qt Creator on a non-python project so the IDE will call PythonSettings::detectPythonVenvs(documentPath)  when a python file is opened
      • Open a python file outside of a virtualenv

       

      Expected result:

      Python file is opened immediately

       

      Actual result:

      Python file takes a few seconds to open. Note that the resolver may cache lookup misses for a short while too, so after the first python file is opened the next ones may open fast for a little while until the lookup miss is expired and the resolver decides it needs to lookup "bin" again.

        

      Desired fix:

      Don't visit /net subfolder in search of virtualenv. In fact, if you get to / without finding the python virtualenv, it may be time to call it quits. I don't think it is common to create a python virtualenv in /.

       

      The stacktrace below was created by breaking the debugger while Qt Creator was hanging in the stat("/net/bin") call while autofs was trying (and ultimately failing) to automount /net/bin:

      Thread 1 "qtcreator" received signal SIGINT, Interrupt.
statx (fd=-100, path=0x55fffe52d718 "/net/bin", flags=2048, mask=4095, buf=0x7ffd342d9840) at ../sysdeps/unix/sysv/linux/statx.c:28
28	  int ret = INLINE_SYSCALL_CALL (statx, fd, path, flags, mask, buf);
(gdb) bt
#0  statx (fd=-100, path=0x55fffe52d718 "/net/bin", flags=2048, mask=4095, buf=0x7ffd342d9840) at ../sysdeps/unix/sysv/linux/statx.c:28
#1  0x00007f5344f2635e in qt_real_statx (statxBuffer=0x7ffd342d9840, flags=0, pathname=<optimized out>, fd=-100) at io/qfilesystemengine_unix.cpp:315
#2  qt_statx (statxBuffer=0x7ffd342d9840, pathname=<optimized out>) at io/qfilesystemengine_unix.cpp:321
#3  QFileSystemEngine::fillMetaData (entry=..., data=..., what=..., what@entry=...) at io/qfilesystemengine_unix.cpp:966
#4  0x00007f5344ebbd92 in QDirPrivate::exists (this=0x55fff9aa30b0) at io/qdir.cpp:157
#5  0x00007f5344ebead4 in QDir::cd (this=this@entry=0x7ffd342d9a60, dirName=...) at io/qdir.cpp:1025
#6  0x00007f530fd3e9f1 in Python::Internal::PythonSettings::detectPythonVenvs (path=...) at /usr/src/debug/qt-creator-7.0.2-1.fc36.x86_64/src/plugins/python/pythonsettings.cpp:507
#7  0x00007f530fd3ef50 in Python::Internal::detectPython (documentPath=...) at /usr/src/debug/qt-creator-7.0.2-1.fc36.x86_64/src/plugins/python/pythonutils.cpp:68
#8  0x00007f530fd24ae4 in Python::Internal::PyLSConfigureAssistant::documentOpened (document=<optimized out>) at /usr/src/debug/qt-creator-7.0.2-1.fc36.x86_64/src/plugins/python/pythonlanguageclient.cpp:464
#9  0x00007f5344fb5c36 in QtPrivate::QSlotObjectBase::call (a=0x7ffd342d9d40, r=<optimized out>, this=0x55fffae7cd50) at ../../include/QtCore/../../src/corelib/kernel/qobjectdefs_impl.h:398
#10 doActivate<false> (sender=0x55fff99f0d40, signal_index=8, argv=0x7ffd342d9d40) at kernel/qobject.cpp:3886
#11 0x00007f5344fb0c67 in QMetaObject::activate (sender=<optimized out>, m=m@entry=0x7f5321ee5100 <Core::EditorManager::staticMetaObject>, local_signal_index=local_signal_index@entry=5,
    argv=argv@entry=0x7ffd342d9d40) at kernel/qobject.cpp:3946
#12 0x00007f5321c89556 in Core::EditorManager::documentOpened (this=<optimized out>, _t1=<optimized out>)
    at /usr/src/debug/qt-creator-7.0.2-1.fc36.x86_64/redhat-linux-build/src/plugins/coreplugin/Core_autogen/YAYUJISNOO/moc_editormanager.cpp:560
#13 0x00007f5321d0fd69 in Core::Internal::EditorManagerPrivate::addEditor (editor=0x7f532c01a410) at /usr/src/debug/qt-creator-7.0.2-1.fc36.x86_64/src/plugins/coreplugin/editormanager/editormanager.cpp:1486
#14 0x00007f5321cf64b3 in Core::Internal::EditorManagerPrivate::addEditor (editor=0x7f532c01a410) at /usr/src/debug/qt-creator-7.0.2-1.fc36.x86_64/src/plugins/coreplugin/editormanager/editormanager.cpp:1473
#15 Core::Internal::EditorManagerPrivate::openEditor (view=0x55fff9a21360, filePath=..., editorId=..., flags=..., newEditor=0x0)
    at /usr/src/debug/qt-creator-7.0.2-1.fc36.x86_64/src/plugins/coreplugin/editormanager/editormanager.cpp:941
#16 0x00007f5344fb5c36 in QtPrivate::QSlotObjectBase::call (a=0x7ffd342da020, r=<optimized out>, this=0x55fffbe990f0) at ../../include/QtCore/../../src/corelib/kernel/qobjectdefs_impl.h:398
#17 doActivate<false> (sender=0x55fffdf126f0, signal_index=4, argv=0x7ffd342da020) at kernel/qobject.cpp:3886
#18 0x00007f5344fb0c67 in QMetaObject::activate (sender=sender@entry=0x55fffdf126f0, m=m@entry=0x7f534612b260 <QAction::staticMetaObject>, local_signal_index=local_signal_index@entry=1,
    argv=argv@entry=0x7ffd342da020) at kernel/qobject.cpp:3946
#19 0x00007f5345c01266 in QAction::triggered (this=this@entry=0x55fffdf126f0, _t1=<optimized out>) at .moc/moc_qaction.cpp:376
#20 0x00007f5345c03f43 in QAction::activate (this=0x55fffdf126f0, event=<optimized out>) at kernel/qaction.cpp:1161
#21 0x00007f5345d8c9e2 in QMenuPrivate::activateCausedStack (this=this@entry=0x55fff98c34d0, causedStack=..., action=action@entry=0x55fffdf126f0, action_e=action_e@entry=QAction::Trigger, self=self@entry=true)
    at widgets/qmenu.cpp:1384
#22 0x00007f5345d948ec in QMenuPrivate::activateAction (this=0x55fff98c34d0, action=0x55fffdf126f0, action_e=QAction::Trigger, self=<optimized out>) at widgets/qmenu.cpp:1461
#23 0x00007f5345c4a798 in QWidget::event (this=0x55fff98c54f0, event=0x7ffd342da600) at kernel/qwidget.cpp:9033
#24 0x00007f5345c07c82 in QApplicationPrivate::notify_helper (this=this@entry=0x55fff8f11300, receiver=receiver@entry=0x55fff98c54f0, e=e@entry=0x7ffd342da600) at kernel/qapplication.cpp:3637
#25 0x00007f5345c10302 in QApplication::notify (this=<optimized out>, receiver=0x55fff98c54f0, e=<optimized out>) at kernel/qapplication.cpp:3081
#26 0x00007f5344f82658 in QCoreApplication::notifyInternal2 (receiver=0x55fff98c54f0, event=0x7ffd342da600) at kernel/qcoreapplication.cpp:1064
#27 0x00007f5345c0e402 in QApplicationPrivate::sendMouseEvent (receiver=0x55fff98c54f0, event=event@entry=0x7ffd342da600, alienWidget=<optimized out>, nativeWidget=0x55fff98c54f0,
    buttonDown=buttonDown@entry=0x7f5346163330 <qt_button_down>, lastMouseReceiver=..., spontaneous=true, onlyDispatchEnterLeave=false) at kernel/qapplication.cpp:2619
#28 0x00007f5345c63f45 in QWidgetWindow::handleMouseEvent (this=0x55fffd115150, event=0x7ffd342da8b0) at kernel/qwidgetwindow.cpp:580
#29 0x00007f5345c66eb0 in QWidgetWindow::event (this=0x55fffd115150, event=0x7ffd342da8b0) at kernel/qwidgetwindow.cpp:300
#30 0x00007f5345c07c82 in QApplicationPrivate::notify_helper (this=<optimized out>, receiver=0x55fffd115150, e=0x7ffd342da8b0) at kernel/qapplication.cpp:3637
#31 0x00007f5344f82658 in QCoreApplication::notifyInternal2 (receiver=0x55fffd115150, event=0x7ffd342da8b0) at kernel/qcoreapplication.cpp:1064
#32 0x00007f534558b115 in QGuiApplicationPrivate::processMouseEvent (e=0x55fffe0fb6c0) at kernel/qguiapplication.cpp:2275
#33 0x00007f534556af5c in QWindowSystemInterface::sendWindowSystemEvents (flags=flags@entry=...) at kernel/qwindowsysteminterface.cpp:1169
#34 0x00007f5333905aee in xcbSourceDispatch (source=<optimized out>) at qxcbeventdispatcher.cpp:105
#35 0x00007f5343402faf in g_main_dispatch (context=0x7f532c005010) at ../glib/gmain.c:3417
#36 g_main_context_dispatch (context=0x7f532c005010) at ../glib/gmain.c:4135
#37 0x00007f53434582c8 in g_main_context_iterate.constprop.0 (context=context@entry=0x7f532c005010, block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>) at ../glib/gmain.c:4211
#38 0x00007f5343400940 in g_main_context_iteration (context=0x7f532c005010, may_block=1) at ../glib/gmain.c:4276
#39 0x00007f5344fd32fa in QEventDispatcherGlib::processEvents (this=0x55fff8fedc00, flags=...) at kernel/qeventdispatcher_glib.cpp:423
#40 0x00007f5344f810ba in QEventLoop::exec (this=this@entry=0x7ffd342dac40, flags=..., flags@entry=...) at ../../include/QtCore/../../src/corelib/global/qflags.h:69
#41 0x00007f5344f89162 in QCoreApplication::exec () at ../../include/QtCore/../../src/corelib/global/qflags.h:121
#42 0x00007f534557fee0 in QGuiApplication::exec () at kernel/qguiapplication.cpp:1860
#43 0x00007f5345c07bf9 in QApplication::exec () at kernel/qapplication.cpp:2829
#44 0x000055fff8e9942c in main (argc=<optimized out>, argv=<optimized out>) at /usr/src/debug/qt-creator-7.0.2-1.fc36.x86_64/src/app/main.cpp:796
(gdb) quit

      Attachments

        For Gerrit Dashboard: QTCREATORBUG-27976
        # Subject Branch Project Status CR V
        424799,3 Python: skip root on venv detection 8.0 qt-creator/qt-creator Status: MERGED +2 0

        Activity

          People

            davschul David Schulz
            ts Thomas Sondergaard
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes