segfault in QSGRenderThread while playing with huge texts in examples/quickcontrols/texteditor

Steps to reproduce:

• cross-compile for aarch64 and transfer to Raspberry Pi 400
• start examples/quickcontrols/texteditor and keep doing select-all and paste operations to exponentially increase the amount of text in the editor
• if the example runs without a debugger, then after some time it gets OOM killed. According to the system logs, the texteditor application only consumed 176250*4KB which is way less than the available 4GB RAM of the system. Swap space is completely unused.

  [ 1153.676235] QSGRenderThread invoked oom-killer: gfp_mask=0x40cc0(GFP_KERNEL|__GFP_COMP), order=0, oom_score_adj=0
  [ 1153.676263] CPU: 2 PID: 1556 Comm: QSGRenderThread Tainted: G         C         6.1.19-v8+ #1637
  [ 1153.676270] Hardware name: Raspberry Pi 400 Rev 1.1 (DT)
  [ 1153.676274] Call trace:
  [ 1153.676276]  dump_backtrace+0x120/0x130
  [ 1153.676286]  show_stack+0x20/0x30
  [ 1153.676290]  dump_stack_lvl+0x8c/0xb8
  [ 1153.676297]  dump_stack+0x18/0x34
  [ 1153.676301]  dump_header+0x4c/0x21c
  [ 1153.676307]  oom_kill_process+0x2a8/0x2b0
  [ 1153.676314]  out_of_memory+0xf0/0x350
  [ 1153.676319]  __alloc_pages_slowpath.constprop.158+0x7d4/0xbc0
  [ 1153.676325]  __alloc_pages+0x2a8/0x318
  [ 1153.676328]  new_slab+0x2f4/0x3b0
  [ 1153.676333]  ___slab_alloc+0x444/0xad0
  [ 1153.676337]  __slab_alloc.isra.95+0x6c/0xa8
  [ 1153.676341]  __kmem_cache_alloc_node+0x3cc/0x3d8
  [ 1153.676345]  kmalloc_trace+0x4c/0x128
  [ 1153.676351]  v3d_create_object+0x34/0x70 [v3d]
  [ 1153.676376]  __drm_gem_shmem_create+0x40/0x1c8 [drm_shmem_helper]
  [ 1153.676387]  drm_gem_shmem_create+0x1c/0x28 [drm_shmem_helper]
  [ 1153.676396]  v3d_bo_create+0x20/0x60 [v3d]
  [ 1153.676410]  v3d_create_bo_ioctl+0x40/0x118 [v3d]
  [ 1153.676423]  drm_ioctl_kernel+0xc8/0x180 [drm]
  [ 1153.676553]  drm_ioctl+0x210/0x418 [drm]
  [ 1153.676621]  __arm64_sys_ioctl+0xb0/0xf0
  [ 1153.676627]  invoke_syscall+0x4c/0x110
  [ 1153.676634]  el0_svc_common.constprop.3+0xfc/0x120
  [ 1153.676639]  do_el0_svc+0x34/0xd0
  [ 1153.676645]  el0_svc+0x30/0x88
  [ 1153.676649]  el0t_64_sync_handler+0x98/0xc0
  [ 1153.676655]  el0t_64_sync+0x18c/0x190
  [ 1153.676661] Mem-Info:
  [ 1153.676667] active_anon:228502 inactive_anon:4487 isolated_anon:0
                  active_file:0 inactive_file:60 isolated_file:0
                  unevictable:563880 dirty:0 writeback:0
                  slab_reclaimable:8183 slab_unreclaimable:14167
                  mapped:5503 shmem:569898 pagetables:3119
                  sec_pagetables:0 bounce:0
                  kernel_misc_reclaimable:0
                  free:122014 free_pcp:63 free_cma:115152
  [ 1153.676678] Node 0 active_anon:914008kB inactive_anon:17948kB active_file:0kB inactive_file:240kB unevictable:2255520kB isolated(anon):0kB isolated(file):0kB mapped:22012kB dirty:0kB writeback:0kB shmem:2279592kB writeback_tmp:0kB kernel_stack:4368kB pagetables:12476kB sec_pagetables:0kB all_unreclaimable? no
  [ 1153.676689] DMA free:475296kB boost:0kB min:3560kB low:4448kB high:5336kB reserved_highatomic:0KB active_anon:23332kB inactive_anon:3884kB active_file:0kB inactive_file:212kB unevictable:287404kB writepending:0kB present:917504kB managed:833168kB mlocked:0kB bounce:0kB free_pcp:252kB local_pcp:0kB free_cma:460608kB
  [ 1153.676700] lowmem_reserve[]: 0 2930 2930 2930
  [ 1153.676720] DMA32 free:12760kB boost:0kB min:12820kB low:16024kB high:19228kB reserved_highatomic:0KB active_anon:890676kB inactive_anon:14064kB active_file:220kB inactive_file:0kB unevictable:1968116kB writepending:0kB present:3080192kB managed:3001132kB mlocked:16kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
  [ 1153.676729] lowmem_reserve[]: 0 0 0 0
  [ 1153.676747] DMA: 18*4kB (EC) 115*8kB (MEC) 118*16kB (UEC) 91*32kB (MC) 73*64kB (UMEC) 33*128kB (EC) 16*256kB (UMEC) 6*512kB (MC) 7*1024kB (UMEC) 4*2048kB (MEC) 107*4096kB (C) = 475488kB
  [ 1153.676828] DMA32: 657*4kB (ME) 450*8kB (ME) 255*16kB (UM) 88*32kB (UME) 1*64kB (U) 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 13188kB
  [ 1153.676887] 570041 total pagecache pages
  [ 1153.676890] 0 pages in swap cache
  [ 1153.676894] Free swap  = 102396kB
  [ 1153.676898] Total swap = 102396kB
  [ 1153.676902] 999424 pages RAM
  [ 1153.676905] 0 pages HighMem/MovableOnly
  [ 1153.676909] 40849 pages reserved
  [ 1153.676913] 131072 pages cma reserved
  [...process.table...]
  [ 1153.677395] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/,task=texteditorexamp,pid=1551,uid=1003
  [ 1153.677441] Out of memory: Killed process 1551 (texteditorexamp) total-vm:3328052kB, anon-rss:704992kB, file-rss:0kB, shmem-rss:8kB, UID:1003 pgtables:5936kB oom_score_adj:0
  

• Trying to catch the issue with a debugger, I don't get an OOM kill but the following stacktrace:

  Thread 5 "QSGRenderThread" received signal SIGSEGV, Segmentation fault.
  [Switching to Thread 0x7fdbe940c0 (LWP 1681)]
  0x0000007fe2e5f260 in ?? () from /usr/lib/aarch64-linux-gnu/dri/vc4_dri.so
  (gdb) bt
  #0  0x0000007fe2e5f260 in ?? () from /usr/lib/aarch64-linux-gnu/dri/vc4_dri.so
  #1  0x0000007fe2e6ab70 in ?? () from /usr/lib/aarch64-linux-gnu/dri/vc4_dri.so
  #2  0x0000007fe2e6636c in ?? () from /usr/lib/aarch64-linux-gnu/dri/vc4_dri.so
  #3  0x0000007fe2e73bac in ?? () from /usr/lib/aarch64-linux-gnu/dri/vc4_dri.so
  #4  0x0000007fe244e2b0 in ?? () from /usr/lib/aarch64-linux-gnu/dri/vc4_dri.so
  #5  0x0000007fe26b9a04 in ?? () from /usr/lib/aarch64-linux-gnu/dri/vc4_dri.so
  #6  0x0000007fe26b9da8 in ?? () from /usr/lib/aarch64-linux-gnu/dri/vc4_dri.so
  #7  0x0000007ff64a03c8 in QOpenGLFunctions::glDrawElements (this=<optimized out>, mode=0, count=0, type=0, indices=0x7fd40008d0)
      at qtbase/include/QtGui/../../../../qt5-git/qtbase/src/gui/opengl/qopenglfunctions.h:720
  #8  QRhiGles2::executeCommandBuffer (this=this@entry=0x7fd4001230, cb=cb@entry=0x7fd407d8b8)
      at /home/builder/qt5-git/qtbase/src/gui/rhi/qrhigles2.cpp:3051
  #9  0x0000007ff64a2a88 in QRhiGles2::endFrame (this=0x7fd4001230, swapChain=0x7fd407d370, flags=...)
      at /home/builder/qt5-git/qtbase/src/gui/rhi/qrhigles2.cpp:1991
  #10 0x0000007ff63627b4 in QRhi::endFrame (this=0x7fd4001210, swapChain=0x7fd407d370, flags=...)
      at /home/builder/qt5-git/qtbase/src/gui/rhi/qrhi.cpp:7597
  #11 0x0000007ff76ab9f0 in QSGRenderThread::syncAndRender (this=this@entry=0x55556e3950)
      at /home/builder/qt5-git/qtdeclarative/src/quick/scenegraph/qsgthreadedrenderloop.cpp:734
  #12 0x0000007ff76ac5b8 in QSGRenderThread::run (this=0x55556e3950)
      at /home/builder/qt5-git/qtdeclarative/src/quick/scenegraph/qsgthreadedrenderloop.cpp:929
  #13 0x0000007ff5c8eaf4 in QThreadPrivate::start(void*)::$_0::operator()() const (this=<optimized out>)
      at /home/builder/qt5-git/qtbase/src/corelib/thread/qthread_unix.cpp:324
  #14 (anonymous namespace)::terminate_on_exception<QThreadPrivate::start(void*)::$_0>(QThreadPrivate::start(void*)::$_0&&) (t=...)
      at /home/builder/qt5-git/qtbase/src/corelib/thread/qthread_unix.cpp:260
  #15 QThreadPrivate::start (arg=0x55556e3950) at /home/builder/qt5-git/qtbase/src/corelib/thread/qthread_unix.cpp:283
  #16 0x0000007ff57b4648 in start_thread (arg=0x7fdbe939c0) at pthread_create.c:477
  #17 0x0000007ff5463c1c in thread_start () at ../sysdeps/unix/sysv/linux/aarch64/clone.S:78
  

Thanks manordheim for causing the crash.