Uploaded image for project: 'Qt'
  1. Qt
  2. QTBUG-81982

"windeployqt_clean" target not working properly

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Open
    • Priority: P4: Low
    • Resolution: Unresolved
    • Affects Version/s: 5.14.1
    • Fix Version/s: None
    • Labels:
      None
    • Environment:
    • Platform/s:
      Windows

      Description

      0. Overview

      The code snippet from windeployqt.prf in Qt 5.14.1,

      windeployqt_clean.commands = if exist $$WINDEPLOYQT_OUTPUT for /f %i in ($$WINDEPLOYQT_OUTPUT) do $$QMAKE_DEL_FILE %~fi && $$QMAKE_DEL_DIR %~pi
      
      windeployqt_clean.commands = if exist $$WINDEPLOYQT_OUTPUT \
                                      for /f %i in ($$WINDEPLOYQT_OUTPUT) do \
                                          $$QMAKE_DEL_FILE %~fi && $$QMAKE_DEL_DIR %~pi
      

      The first line is OK, but the second and third lines are buggy. In most cases, it does not behave as the developer intended.

      EXPECTED: The target "windeployqt_clean" can clean all deployed files and return no error.

      ACTUAL: The target "windeployqt_clean" returns error.

      The code with bugs fixed,

      windeployqt_clean.commands = if exist $$WINDEPLOYQT_OUTPUT for /f \"usebackq delims=\" %i in ($$WINDEPLOYQT_OUTPUT) do $$QMAKE_DEL_FILE \"%~fi\" && $$QMAKE_DEL_DIR \"%~dpi\" || ver > $$QMAKE_SHELL_NULL_DEVICE
      
      windeployqt_clean.commands = if exist $$WINDEPLOYQT_OUTPUT \
                                      for /f \"usebackq delims=\" %i in ($$WINDEPLOYQT_OUTPUT) do \
                                          $$QMAKE_DEL_FILE \"%~fi\" && $$QMAKE_DEL_DIR \"%~dpi\" || ver > $$QMAKE_SHELL_NULL_DEVICE
      

      1. Issues of the second line

      1.1. The string between parentheses is not path

      1.1.1. Issue

      $$WINDEPLOYQT_OUTPUT will expand into a string with quotes of the path to OUT_PWD\$$TARGET.windeployqt, the list of files.
      If no argument are provided, for /f will treat the quoted string between parentheses as a literal string.

      > for /?
      ...
      FOR /F ["options"] %variable IN (file-set) DO command [command-parameters]
      FOR /F ["options"] %variable IN ("string") DO command [command-parameters]
      FOR /F ["options"] %variable IN ('command') DO command [command-parameters]
      ...
      

      As a result, the for-loop will be executed once and the variable %i will still be $$WINDEPLOYQT_OUTPUT itself, not the path in the list.

      1.1.2. Thinking

      > for /?
      ...
      ...
          or, if usebackq option present:
      
      FOR /F ["options"] %variable IN (file-set) DO command [command-parameters]
      FOR /F ["options"] %variable IN ('string') DO command [command-parameters]
      FOR /F ["options"] %variable IN (`command`) DO command [command-parameters]
      ...
      

      There is no case of a quoted string between parentheses. Through testing, I found that when "usebackq" is added, the quoted string represents the path of a file (It's the case of "file-set", so multiple strings with or without quotes are allowed here. The purpose of the quotes is to prevent the string with spaces be truncated). I verified my conclusion from For /f - Loop through text - Windows CMD - SS64.com .

      Syntax
              FOR /F ["options"] %%parameter IN (filenameset) DO command 
      ...
            usebackq     Use the alternate quoting style:                        
                         - Use double quotes for long file names in "filenameset".
                         - Use single quotes for 'Text string to process'
                         - Use back quotes for `command to process`
      ...
      

      1.1.3. Solution

      We should add the "usebackq" argument.

      1.2. The path was read from $$WINDEPLOYQT_OUTPUT may contain spaces

      1.2.1. Issue

      The contents of $$WINDEPLOYQT_OUTPUT might look like this:

      ...
      C:\b e s\build-project-Debug\Qt5Core.dll
      C:\b e s\build-project-Debug\Qt5Widgets.dll
      C:\b e s\build-project-Debug\translations\qt_ru.qm
      ...
      

      Current for-loop will read them like this:

      ...
      C:\b
      C:\b
      C:\b
      ...
      

      1.2.2. Thinking

      The reason is the "delimiter". See the document:

      > for /?
      ...
              delims=xxx      - specifies a delimiter set.  This replaces the
                                default delimiter set of space and tab.
      ...
      

      From For /f - Loop through text - Windows CMD - SS64.com ,

      Delims
      
      ...
      You can remove all delimiters by using "delims=" this will place everything on the line into the first token.
      

      Read the whole line, this is exactly what we need.

      1.2.3. Solution

      We should add the "delims=" argument.

      2. Issues of the third line

      2.1. There may be spaces in the value of the variables (%i, %~fi and %~pi)

      2.1.1. Issue

      $$WINDEPLOYQT_OUTPUT stores the paths without quotes, and the variables %i, %~fi and %~pi will be replaced by the path strings directly (without quotes). So the program will execute the command like this,
      (Assuming %i=C:\b e s\build-project-Debug\translations\qt_ru.qm)

      del C:\b e s\build-project-Debug\translations\qt_ru.qm && rmdir \b e s\build-project-Debug\translations\
      

      Due to the lack of quotes, the del and rmdir commands did not execute as expected.

      2.1.2. Solution

      Put quotes around variables (Don't forget the backslash).

      2.2. Target windeployqt_clean was stopped due to non-zero error codes

      2.2.1. Issue

      jom returned error codes while doing windeployqt_clean.

      2.2.2. Thinking

      It's a long story. CMD.EXE is hard to use in some cases.

      ...
      The del command does not set the ErrorLevel as long as the given arguments are valid, it even resets the ErrorLevel to 0 in such cases (at least for Windows 7).
      ...
      ...
      rd does not set errorlevel to zero - it leaves errorlevel intact: f.e. if previous operation ends in positive errorlevel and rd finishes successfully it leaves errorlevel unchanged.
      ...
      

      Yes, it is hard to confirm the result through the %ERRORLEVEL% after executing del and rd (rmdir) commands.

      And Now, there is a realistic example.
      Here is the last segment of my $$WINDEPLOYQT_OUTPUT file,

      ...
      C:\b e s\build-project-Debug\translations\qt_uk.qm
      C:\b e s\build-project-Debug\translations\qtbase_uk.qm
      C:\b e s\build-project-Debug\translations\qt_zh_TW.qm
      C:\b e s\build-project-Debug\translations\qtbase_zh_TW.qm
      

      The command $$QMAKE_DEL_FILE \"%~fi\" && $$QMAKE_DEL_DIR \"%~pi\" in the do segment of the for-loop is like this,
      (Merged the solution from Section 2.1.
      Due to QTBUG-81973 The output of "--list" contains non-existent files - Qt Bug Tracker, files with "qtbase" don't exist.
      I recorded the %ERRORLEVEL%, too)

      ...                                                                  %ERRORLEVEL%=0
      del "C:\b e s\build-project-Debug\translations\qt_uk.qm" && ^        %ERRORLEVEL%=0    File has been deleted.
          rmdir "\b e s\build-project-Debug\translations\"                 %ERRORLEVEL%=145  The directory is not empty.
      del "C:\b e s\build-project-Debug\translations\qtbase_uk.qm" && ^    %ERRORLEVEL%=0    Could Not Find the file.
          rmdir "\b e s\build-project-Debug\translations\"                 %ERRORLEVEL%=145  The directory is not empty.
      del "C:\b e s\build-project-Debug\translations\qt_zh_TW.qm" && ^     %ERRORLEVEL%=0    File has been deleted.
          rmdir "\b e s\build-project-Debug\translations\"                 %ERRORLEVEL%=0    The folder "translations\" has been removed.
      del "C:\b e s\build-project-Debug\translations\qtbase_zh_TW.qm" && ^ %ERRORLEVEL%=1    The system cannot find the file specified.
          rmdir "\b e s\build-project-Debug\translations\"                (%ERRORLEVEL%=2)  (The system cannot find the file specified.)  This line will not be executed because the previous failure.
      
      jom: C:\b e s\build-project-Debug\Makefile [windeployqt_clean] Error 1
      The process "C:\Qt\Qt5.14.1\Tools\QtCreator\bin\jom.exe" exited with code 2.
      

      (Note, the operator && has almost no effect like Unix in the above examples because the del command hasn't do any complaining (by setting the %ERRORLEVEL%) at all. So it can be treated as an & operator)

      If QTBUG-81973 The output of "--list" contains non-existent files - Qt Bug Tracker has been fixed, the procedure will become like this,

      ...                                                              %ERRORLEVEL%=0
      del "C:\b e s\build-project-Debug\translations\qt_uk.qm" && ^    %ERRORLEVEL%=0    File has been deleted.
          rmdir "\b e s\build-project-Debug\translations\"             %ERRORLEVEL%=145  The directory is not empty.
      del "C:\b e s\build-project-Debug\translations\qt_zh_TW.qm" && ^ %ERRORLEVEL%=0    File has been deleted.
          rmdir "\b e s\build-project-Debug\translations\"             %ERRORLEVEL%=0    The folder "translations\" has been removed.
      
      The process "C:\Qt\Qt5.14.1\Tools\QtCreator\bin\jom.exe" exited normally.
      

      It works!

      If there are some files which are not from windeployqt in the folder "translations\", in other words they shouldn't be removed by windeployqt_clean, the procedure will become like this,

      ...                                                              %ERRORLEVEL%=0
      del "C:\b e s\build-project-Debug\translations\qt_uk.qm" && ^    %ERRORLEVEL%=0    File has been deleted.
          rmdir "\b e s\build-project-Debug\translations\"             %ERRORLEVEL%=145  The directory is not empty.
      del "C:\b e s\build-project-Debug\translations\qt_zh_TW.qm" && ^ %ERRORLEVEL%=0    File has been deleted.
          rmdir "\b e s\build-project-Debug\translations\"             %ERRORLEVEL%=145  The directory is not empty.
      
      jom: C:\b e s\build-project-Debug\Makefile [windeployqt_clean] Error 145
      The process "C:\Qt\Qt5.14.1\Tools\QtCreator\bin\jom.exe" exited with code 2.
      

      As you can see, the %ERRORLEVEL% from rmdir let the program down.

      I've written a version of windeployqt.prf with complex modification before, which can process files and folders separately, but it can not solve above problem completely.

      2.2.3. Solution

      The simplest solution is to add a conditional operator (||) and a command that sets %ERRORLEVEL% to 0 in any way. Like this,

      $$QMAKE_DEL_FILE \"%~fi\" && $$QMAKE_DEL_DIR \"%~pi\" || ver > $$QMAKE_SHELL_NULL_DEVICE
      

      2.3. Wrong use of substitution of FOR variable references (%~pi vs. %~dpi)

      2.3.1. Issue

      > for /?
      ...
          %~pI        - expands %I to a path only
      ...
          %~dpI       - expands %I to a drive letter and path only
      ...
      

      If the file path in the list from the file OUT_PWD\$$TARGET.windeployqt is different from the location where the toolchain lives (e.g., the C drive), the incomplete path (%~pi) will not point to the real location of the directory to be removed,

      > for %i in (D:\dir\file) do echo "%i" && echo "%~pi" && echo "%~dpi"
      "D:\dir\file"
      "\dir\"
      "D:\dir\"
      

      2.3.2. Solution

      Just use %~dpi instead of %~pi.

        Attachments

          Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

            Activity

              People

              Assignee:
              qtbuildsystem Qt Build System Team
              Reporter:
              mozi_86 Kaijie Liu
              Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:

                  Gerrit Reviews

                  There are no open Gerrit changes