Uploaded image for project: 'Qt Quality Assurance Infrastructure'
  1. Qt Quality Assurance Infrastructure
  2. QTQAINFRA-5766

Provisioning: Cache downloads fail often

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • P3: Somewhat important
    • None
    • unversioned
    • None
    • b3d10251b (dev)

    Description

      Many library downloads from cache (https://ci-files01-hki.ci.qt.io/) fail. TLDR: Some files are missing in cache (e.g. python-3.10.0-amd64.exe), some have non-HTTPS or otherwise incorrect URLs.

       

      I noticed this in my Coin run:

      https://testresults.qt.io/coin/integration/qt/qt5/tasks/web_qt_qt5_1692105391775

      Provisioning

      (1 running - 0/2 completed)

      Error Windows Windows_11_22H2 x86_64 8/15/23, 4:27 PM 1h, 42min, 37s log raw log sw-log downloads path buildkey

      (Error is unrelated)

      Downloading from cached location (\\ci-files01-hki.ci.qt.io\provisioning\windows\7z1604-x64.exe)
      to C:\Windows\Temp\7zip-1604.exe
      
      Cached download failed: Downloading from official location: http://www.7-zip.org/a/7z1604-x64.exe
      
      ...
      
      Downloading from cached location (http://ci-files01-hki.ci.qt.io/input/windows/python-3.10.0-amd64.exe)
      to C:\Windows\temp\python-3.10.0.exe
      
      Cached download failed: Downloading from official location: https://www.python.org/ftp/python/3.10.0/python-3.10.0-amd64.exe
      
      ...
      
      Downloading from cached location (\\ci-files01-hki.ci.qt.io\provisioning\windows\msys2-base-x86_64-20220319.tar.xz)
      to C:\Windows\Temp\msys2-base-x86_64-20220319.tar.xz
      
      Cached download failed: Downloading from official location: http://repo.msys2.org/distrib/x86_64/msys2-base-x86_64-20220319.tar.xz
      
      etc.
       

      There are 8 failed cache downloads in this case, out of 62.

       

      I checked other recent Coin run:

      https://testresults.qt.io/coin/integration/qt/qtbase/tasks/1697706680

      There are cache download fails in Linux, Mac and Windows even when the files are present in ci-files01.

      Done MacOS MacOS_13 x86_64 8/11/23, 7:00 PM 1h, 49min, 35s log raw log sw-log downloads path buildkey

       

      Downloading from primary URL:  http://ci-files01-hki.ci.qt.io/input/mac/macos_10.12_sierra/Postgres-2.5-14.dmg

      -> error 404 -> file was actually not there: https://ci-files01-hki.ci.qt.io/input/mac/macos_10.12_sierra/

      Downloading from primary URL:  http://ci-files01-hki.ci.qt.io/input/ffmpeg/n6.0.tar.gz

      -> 404 -> file not there: https://ci-files01-hki.ci.qt.io/input/ffmpeg/

       

       

      Done Linux Ubuntu_22_04 x86_64 8/12/23, 11:39 AM 1h, 10min, 25s log raw log sw-log downloads path buildkey

       

      Downloading from primary URL:  http://ci-files01-hki.ci.qt.io/input/docker/mimersqlsrv_11.0_amd64-openssl3.deb

      -> 404 -> file not there: https://ci-files01-hki.ci.qt.io/input/docker/

      -> FFmpeg not found, not in cache.

       

       

      Done Windows Windows_11_22H2 x86_64 8/15/23, 1:46 PM 1h, 39min, 56s log raw log sw-log downloads path buildkey

       

      Downloading from cached location (\\ci-files01-hki.ci.qt.io\provisioning\windows\7z2301-x64.exe) to C:\Windows\Temp\7zip-2301.exe
      Cached download failed: Downloading from official location: http://www.7-zip.org/a/7z2301-x64.exe

      The 7zip-2301.exe file is there: https://ci-files01-hki.ci.qt.io/input/windows/

      When you click the file it'll download to your browser:

      https://ci-files01-hki.ci.qt.io/input/windows/7z2301-x64.exe

      The link in script is missing "https" so that might be it. A simple fix might be download the file from URL and place it in the cache, and add/change URL to https if it's not already that.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            eltoivol Elias Toivola
            eltoivol Elias Toivola
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes