Uploaded image for project: 'Qt'
  1. Qt
  2. QTBUG-126259

Encoding odd-sized custom frames causes ASAN crash in sws_rescale

    XMLWordPrintable

Details

    • Bug
    • Resolution: Unresolved
    • P1: Critical
    • None
    • 6.8
    • Multimedia
    • None
    • All
    • Multimedia wk 27-28, Multimedia wk 29-30, Multimedia wk 31-32, Multimedia wk 35-38, Multimedia wk 39-40, Multimedia wk 41-42, Multimedia wk 43-44, Multimedia wk 47-48, Multimedia wk 49-50

    Description

      When encoding custom frames using QVideoFrameInput, and the first frame is even sized, but subsequent frames are odd-sized, encoding crashes during rescaling because sws_rescale only supports even sizes when converting to multiplanar data with down-sampled UV channels. This requires FFmpeg to be built with address sanitizer enabled

      Observed with ASAN enabled in when modifying the test to also verify odd-sized frames

      • tst_QMediaFrameInputsBackend::mediaRecorderWritesVideo_whenInputFrameShrinksOverTime()
      • tst_QMediaFrameInputsBackend::mediaRecorderWritesVideo_whenInputFrameGrowsOverTime

      I build FFmpeg on Windows, and to enable the address sanitizer I use the following command line:

      ../FFmpeg/configure --prefix=/k/dev/ffmpeg/install-asan --toolchain=msvc --disable-doc --enable-debug --enable-network --disable-lzma --enable-pic --disable-vulkan --disable-v4l2-m2m --disable-decoder=truemotion1 --enable-shared --disable-static --disable-optimizations --extra-cflags="-fsanitize=address" --extra-cxxflags="-sanitize=address"  && make install
      

       
      With a 37x37 pixel input, this is what is being reported:

       
      Call stack

      ffmpegmediaplugind.dll!__vcasan::OnAsanReport(const char * description, const char * report, bool __throw) Line 702
      	at D:\a\_work\1\s\src\vctools\asan\vcasan\vcasan.cpp(702)
      ffmpegmediaplugind.dll!__vcasan::ReportCallback(const char * szReport) Line 331
      	at D:\a\_work\1\s\src\vctools\asan\vcasan\vcasan.cpp(331)
      clang_rt.asan_dynamic-x86_64.dll!__asan::ScopedInErrorReport::~ScopedInErrorReport() Line 226
      	at D:\a\_work\1\s\src\vctools\asan\llvm\compiler-rt\lib\asan\asan_report.cpp(226)
      clang_rt.asan_dynamic-x86_64.dll!__asan::ReportGenericError(unsigned __int64 pc, unsigned __int64 bp, unsigned __int64 sp, unsigned __int64 addr, bool is_write, unsigned __int64 access_size, unsigned int exp, bool fatal) Line 538
      	at D:\a\_work\1\s\src\vctools\asan\llvm\compiler-rt\lib\asan\asan_report.cpp(538)
      clang_rt.asan_dynamic-x86_64.dll!__asan_report_load4(unsigned __int64 addr) Line 133
      	at D:\a\_work\1\s\src\vctools\asan\llvm\compiler-rt\lib\asan\asan_rtl.cpp(133)
      [Inline Frame] swscale-8.dll!rgb16_32ToUV_half_c_template(short *) Line 338
      	at K:\dev\ffmpeg\FFmpeg\libswscale\input.c(338)
      swscale-8.dll!bgr32ToUV_half_c(unsigned char * dstU, unsigned char * dstV, const unsigned char * unused0, const unsigned char * src, const unsigned char * dummy, int width, unsigned int * tab, void * opq) Line 393
      	at K:\dev\ffmpeg\FFmpeg\libswscale\input.c(393)
      swscale-8.dll!chr_convert(SwsContext * c, SwsFilterDescriptor * desc, int sliceY, int sliceH) Line 227
      	at K:\dev\ffmpeg\FFmpeg\libswscale\hscale.c(227)
      swscale-8.dll!swscale(SwsContext * c, const unsigned char * * src, int * srcStride, int srcSliceY, int srcSliceH, unsigned char * * dst, int * dstStride, int dstSliceY, int dstSliceH) Line 472
      	at K:\dev\ffmpeg\FFmpeg\libswscale\swscale.c(472)
      swscale-8.dll!scale_internal(SwsContext * c, const unsigned char * const * srcSlice, const int * srcStride, int srcSliceY, int srcSliceH, unsigned char * const * dstSlice, const int * dstStride, int dstSliceY, int dstSliceH) Line 1047
      	at K:\dev\ffmpeg\FFmpeg\libswscale\swscale.c(1047)
      swscale-8.dll!sws_scale(SwsContext * c, const unsigned char * const * srcSlice, const int * srcStride, int srcSliceY, int srcSliceH, unsigned char * const * dst, const int * dstStride) Line 1216
      	at K:\dev\ffmpeg\FFmpeg\libswscale\swscale.c(1216)
      ffmpegmediaplugind.dll!QFFmpeg::`anonymous namespace'::FrameConverter::convert(SwsContext * converter, AVPixelFormat format, const QSize & size) Line 352
      	at K:\qt\qt6\qtmultimedia\src\plugins\multimedia\ffmpeg\recordingengine\qffmpegvideoframeencoder.cpp(352)
      ffmpegmediaplugind.dll!QFFmpeg::VideoFrameEncoder::sendFrame(std::unique_ptr<AVFrame,QFFmpeg::AVDeleter<void (__cdecl*)(AVFrame * *),&av_frame_free>> inputFrame) Line 448
      	at K:\qt\qt6\qtmultimedia\src\plugins\multimedia\ffmpeg\recordingengine\qffmpegvideoframeencoder.cpp(448)
      ffmpegmediaplugind.dll!QFFmpeg::VideoEncoder::processOne() Line 220
      	at K:\qt\qt6\qtmultimedia\src\plugins\multimedia\ffmpeg\recordingengine\qffmpegvideoencoder.cpp(220)
      ffmpegmediaplugind.dll!QFFmpeg::ConsumerThread::run() Line 43
      	at K:\qt\qt6\qtmultimedia\src\plugins\multimedia\ffmpeg\qffmpegthread.cpp(43)
      Qt6Cored.dll!QThreadPrivate::start(void * arg) Line 272
      	at K:\qt\qt6\qtbase\src\corelib\thread\qthread_win.cpp(272)
      clang_rt.asan_dynamic-x86_64.dll!__asan::AsanThread::ThreadStart(unsigned __int64 os_id) Line 307
      	at D:\a\_work\1\s\src\vctools\asan\llvm\compiler-rt\lib\asan\asan_thread.cpp(307)
      
      

       

      Attachments

        Issue Links

          For Gerrit Dashboard: QTBUG-126259
          # Subject Branch Project Status CR V

          Activity

            People

              artemiy Artem Dyomin
              johanseg Jøger Hansegård
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:

                Gerrit Reviews

                  There is 1 open Gerrit change