media: vb2: check memory model for VIDIOC_CREATE_BUFS

vb2_core_create_bufs did not check if the memory model for newly added
buffers is the same as for already existing buffers. It should return an
error if they aren't the same.

Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Reported-by: syzbot+e1fb118a2ebb88031d21@syzkaller.appspotmail.com
Cc: <stable@vger.kernel.org>      # for v4.16 and up
Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>

drivers/media/common/videobuf2/videobuf2-core.c

@@ -812,6 +812,9 @@ int vb2_core_create_bufs(struct vb2_queue *q, enum vb2_memory memory,
 		memset(q->alloc_devs, 0, sizeof(q->alloc_devs));
 		q->memory = memory;
 		q->waiting_for_buffers = !q->is_output;
+	} else if (q->memory != memory) {
+		dprintk(1, "memory model mismatch\n");
+		return -EINVAL;
 	}
 
 	num_buffers = min(*count, VB2_MAX_FRAME - q->num_buffers);