|
@@ -6,205 +6,6 @@
|
|
<sumit dot semwal at ti dot com>
|
|
<sumit dot semwal at ti dot com>
|
|
|
|
|
|
|
|
|
|
-Kernel cpu access to a dma-buf buffer object
|
|
|
|
---------------------------------------------
|
|
|
|
-
|
|
|
|
-The motivation to allow cpu access from the kernel to a dma-buf object from the
|
|
|
|
-importers side are:
|
|
|
|
-- fallback operations, e.g. if the devices is connected to a usb bus and the
|
|
|
|
- kernel needs to shuffle the data around first before sending it away.
|
|
|
|
-- full transparency for existing users on the importer side, i.e. userspace
|
|
|
|
- should not notice the difference between a normal object from that subsystem
|
|
|
|
- and an imported one backed by a dma-buf. This is really important for drm
|
|
|
|
- opengl drivers that expect to still use all the existing upload/download
|
|
|
|
- paths.
|
|
|
|
-
|
|
|
|
-Access to a dma_buf from the kernel context involves three steps:
|
|
|
|
-
|
|
|
|
-1. Prepare access, which invalidate any necessary caches and make the object
|
|
|
|
- available for cpu access.
|
|
|
|
-2. Access the object page-by-page with the dma_buf map apis
|
|
|
|
-3. Finish access, which will flush any necessary cpu caches and free reserved
|
|
|
|
- resources.
|
|
|
|
-
|
|
|
|
-1. Prepare access
|
|
|
|
-
|
|
|
|
- Before an importer can access a dma_buf object with the cpu from the kernel
|
|
|
|
- context, it needs to notify the exporter of the access that is about to
|
|
|
|
- happen.
|
|
|
|
-
|
|
|
|
- Interface:
|
|
|
|
- int dma_buf_begin_cpu_access(struct dma_buf *dmabuf,
|
|
|
|
- enum dma_data_direction direction)
|
|
|
|
-
|
|
|
|
- This allows the exporter to ensure that the memory is actually available for
|
|
|
|
- cpu access - the exporter might need to allocate or swap-in and pin the
|
|
|
|
- backing storage. The exporter also needs to ensure that cpu access is
|
|
|
|
- coherent for the access direction. The direction can be used by the exporter
|
|
|
|
- to optimize the cache flushing, i.e. access with a different direction (read
|
|
|
|
- instead of write) might return stale or even bogus data (e.g. when the
|
|
|
|
- exporter needs to copy the data to temporary storage).
|
|
|
|
-
|
|
|
|
- This step might fail, e.g. in oom conditions.
|
|
|
|
-
|
|
|
|
-2. Accessing the buffer
|
|
|
|
-
|
|
|
|
- To support dma_buf objects residing in highmem cpu access is page-based using
|
|
|
|
- an api similar to kmap. Accessing a dma_buf is done in aligned chunks of
|
|
|
|
- PAGE_SIZE size. Before accessing a chunk it needs to be mapped, which returns
|
|
|
|
- a pointer in kernel virtual address space. Afterwards the chunk needs to be
|
|
|
|
- unmapped again. There is no limit on how often a given chunk can be mapped
|
|
|
|
- and unmapped, i.e. the importer does not need to call begin_cpu_access again
|
|
|
|
- before mapping the same chunk again.
|
|
|
|
-
|
|
|
|
- Interfaces:
|
|
|
|
- void *dma_buf_kmap(struct dma_buf *, unsigned long);
|
|
|
|
- void dma_buf_kunmap(struct dma_buf *, unsigned long, void *);
|
|
|
|
-
|
|
|
|
- There are also atomic variants of these interfaces. Like for kmap they
|
|
|
|
- facilitate non-blocking fast-paths. Neither the importer nor the exporter (in
|
|
|
|
- the callback) is allowed to block when using these.
|
|
|
|
-
|
|
|
|
- Interfaces:
|
|
|
|
- void *dma_buf_kmap_atomic(struct dma_buf *, unsigned long);
|
|
|
|
- void dma_buf_kunmap_atomic(struct dma_buf *, unsigned long, void *);
|
|
|
|
-
|
|
|
|
- For importers all the restrictions of using kmap apply, like the limited
|
|
|
|
- supply of kmap_atomic slots. Hence an importer shall only hold onto at most 2
|
|
|
|
- atomic dma_buf kmaps at the same time (in any given process context).
|
|
|
|
-
|
|
|
|
- dma_buf kmap calls outside of the range specified in begin_cpu_access are
|
|
|
|
- undefined. If the range is not PAGE_SIZE aligned, kmap needs to succeed on
|
|
|
|
- the partial chunks at the beginning and end but may return stale or bogus
|
|
|
|
- data outside of the range (in these partial chunks).
|
|
|
|
-
|
|
|
|
- Note that these calls need to always succeed. The exporter needs to complete
|
|
|
|
- any preparations that might fail in begin_cpu_access.
|
|
|
|
-
|
|
|
|
- For some cases the overhead of kmap can be too high, a vmap interface
|
|
|
|
- is introduced. This interface should be used very carefully, as vmalloc
|
|
|
|
- space is a limited resources on many architectures.
|
|
|
|
-
|
|
|
|
- Interfaces:
|
|
|
|
- void *dma_buf_vmap(struct dma_buf *dmabuf)
|
|
|
|
- void dma_buf_vunmap(struct dma_buf *dmabuf, void *vaddr)
|
|
|
|
-
|
|
|
|
- The vmap call can fail if there is no vmap support in the exporter, or if it
|
|
|
|
- runs out of vmalloc space. Fallback to kmap should be implemented. Note that
|
|
|
|
- the dma-buf layer keeps a reference count for all vmap access and calls down
|
|
|
|
- into the exporter's vmap function only when no vmapping exists, and only
|
|
|
|
- unmaps it once. Protection against concurrent vmap/vunmap calls is provided
|
|
|
|
- by taking the dma_buf->lock mutex.
|
|
|
|
-
|
|
|
|
-3. Finish access
|
|
|
|
-
|
|
|
|
- When the importer is done accessing the CPU, it needs to announce this to
|
|
|
|
- the exporter (to facilitate cache flushing and unpinning of any pinned
|
|
|
|
- resources). The result of any dma_buf kmap calls after end_cpu_access is
|
|
|
|
- undefined.
|
|
|
|
-
|
|
|
|
- Interface:
|
|
|
|
- void dma_buf_end_cpu_access(struct dma_buf *dma_buf,
|
|
|
|
- enum dma_data_direction dir);
|
|
|
|
-
|
|
|
|
-
|
|
|
|
-Direct Userspace Access/mmap Support
|
|
|
|
-------------------------------------
|
|
|
|
-
|
|
|
|
-Being able to mmap an export dma-buf buffer object has 2 main use-cases:
|
|
|
|
-- CPU fallback processing in a pipeline and
|
|
|
|
-- supporting existing mmap interfaces in importers.
|
|
|
|
-
|
|
|
|
-1. CPU fallback processing in a pipeline
|
|
|
|
-
|
|
|
|
- In many processing pipelines it is sometimes required that the cpu can access
|
|
|
|
- the data in a dma-buf (e.g. for thumbnail creation, snapshots, ...). To avoid
|
|
|
|
- the need to handle this specially in userspace frameworks for buffer sharing
|
|
|
|
- it's ideal if the dma_buf fd itself can be used to access the backing storage
|
|
|
|
- from userspace using mmap.
|
|
|
|
-
|
|
|
|
- Furthermore Android's ION framework already supports this (and is otherwise
|
|
|
|
- rather similar to dma-buf from a userspace consumer side with using fds as
|
|
|
|
- handles, too). So it's beneficial to support this in a similar fashion on
|
|
|
|
- dma-buf to have a good transition path for existing Android userspace.
|
|
|
|
-
|
|
|
|
- No special interfaces, userspace simply calls mmap on the dma-buf fd, making
|
|
|
|
- sure that the cache synchronization ioctl (DMA_BUF_IOCTL_SYNC) is *always*
|
|
|
|
- used when the access happens. Note that DMA_BUF_IOCTL_SYNC can fail with
|
|
|
|
- -EAGAIN or -EINTR, in which case it must be restarted.
|
|
|
|
-
|
|
|
|
- Some systems might need some sort of cache coherency management e.g. when
|
|
|
|
- CPU and GPU domains are being accessed through dma-buf at the same time. To
|
|
|
|
- circumvent this problem there are begin/end coherency markers, that forward
|
|
|
|
- directly to existing dma-buf device drivers vfunc hooks. Userspace can make
|
|
|
|
- use of those markers through the DMA_BUF_IOCTL_SYNC ioctl. The sequence
|
|
|
|
- would be used like following:
|
|
|
|
- - mmap dma-buf fd
|
|
|
|
- - for each drawing/upload cycle in CPU 1. SYNC_START ioctl, 2. read/write
|
|
|
|
- to mmap area 3. SYNC_END ioctl. This can be repeated as often as you
|
|
|
|
- want (with the new data being consumed by the GPU or say scanout device)
|
|
|
|
- - munmap once you don't need the buffer any more
|
|
|
|
-
|
|
|
|
- For correctness and optimal performance, it is always required to use
|
|
|
|
- SYNC_START and SYNC_END before and after, respectively, when accessing the
|
|
|
|
- mapped address. Userspace cannot rely on coherent access, even when there
|
|
|
|
- are systems where it just works without calling these ioctls.
|
|
|
|
-
|
|
|
|
-2. Supporting existing mmap interfaces in importers
|
|
|
|
-
|
|
|
|
- Similar to the motivation for kernel cpu access it is again important that
|
|
|
|
- the userspace code of a given importing subsystem can use the same interfaces
|
|
|
|
- with a imported dma-buf buffer object as with a native buffer object. This is
|
|
|
|
- especially important for drm where the userspace part of contemporary OpenGL,
|
|
|
|
- X, and other drivers is huge, and reworking them to use a different way to
|
|
|
|
- mmap a buffer rather invasive.
|
|
|
|
-
|
|
|
|
- The assumption in the current dma-buf interfaces is that redirecting the
|
|
|
|
- initial mmap is all that's needed. A survey of some of the existing
|
|
|
|
- subsystems shows that no driver seems to do any nefarious thing like syncing
|
|
|
|
- up with outstanding asynchronous processing on the device or allocating
|
|
|
|
- special resources at fault time. So hopefully this is good enough, since
|
|
|
|
- adding interfaces to intercept pagefaults and allow pte shootdowns would
|
|
|
|
- increase the complexity quite a bit.
|
|
|
|
-
|
|
|
|
- Interface:
|
|
|
|
- int dma_buf_mmap(struct dma_buf *, struct vm_area_struct *,
|
|
|
|
- unsigned long);
|
|
|
|
-
|
|
|
|
- If the importing subsystem simply provides a special-purpose mmap call to set
|
|
|
|
- up a mapping in userspace, calling do_mmap with dma_buf->file will equally
|
|
|
|
- achieve that for a dma-buf object.
|
|
|
|
-
|
|
|
|
-3. Implementation notes for exporters
|
|
|
|
-
|
|
|
|
- Because dma-buf buffers have invariant size over their lifetime, the dma-buf
|
|
|
|
- core checks whether a vma is too large and rejects such mappings. The
|
|
|
|
- exporter hence does not need to duplicate this check.
|
|
|
|
-
|
|
|
|
- Because existing importing subsystems might presume coherent mappings for
|
|
|
|
- userspace, the exporter needs to set up a coherent mapping. If that's not
|
|
|
|
- possible, it needs to fake coherency by manually shooting down ptes when
|
|
|
|
- leaving the cpu domain and flushing caches at fault time. Note that all the
|
|
|
|
- dma_buf files share the same anon inode, hence the exporter needs to replace
|
|
|
|
- the dma_buf file stored in vma->vm_file with it's own if pte shootdown is
|
|
|
|
- required. This is because the kernel uses the underlying inode's address_space
|
|
|
|
- for vma tracking (and hence pte tracking at shootdown time with
|
|
|
|
- unmap_mapping_range).
|
|
|
|
-
|
|
|
|
- If the above shootdown dance turns out to be too expensive in certain
|
|
|
|
- scenarios, we can extend dma-buf with a more explicit cache tracking scheme
|
|
|
|
- for userspace mappings. But the current assumption is that using mmap is
|
|
|
|
- always a slower path, so some inefficiencies should be acceptable.
|
|
|
|
-
|
|
|
|
- Exporters that shoot down mappings (for any reasons) shall not do any
|
|
|
|
- synchronization at fault time with outstanding device operations.
|
|
|
|
- Synchronization is an orthogonal issue to sharing the backing storage of a
|
|
|
|
- buffer and hence should not be handled by dma-buf itself. This is explicitly
|
|
|
|
- mentioned here because many people seem to want something like this, but if
|
|
|
|
- different exporters handle this differently, buffer sharing can fail in
|
|
|
|
- interesting ways depending upong the exporter (if userspace starts depending
|
|
|
|
- upon this implicit synchronization).
|
|
|
|
-
|
|
|
|
Other Interfaces Exposed to Userspace on the dma-buf FD
|
|
Other Interfaces Exposed to Userspace on the dma-buf FD
|
|
------------------------------------------------------
|
|
------------------------------------------------------
|
|
|
|
|
|
@@ -240,20 +41,6 @@ Miscellaneous notes
|
|
the exporting driver to create a dmabuf fd must provide a way to let
|
|
the exporting driver to create a dmabuf fd must provide a way to let
|
|
userspace control setting of O_CLOEXEC flag passed in to dma_buf_fd().
|
|
userspace control setting of O_CLOEXEC flag passed in to dma_buf_fd().
|
|
|
|
|
|
-- If an exporter needs to manually flush caches and hence needs to fake
|
|
|
|
- coherency for mmap support, it needs to be able to zap all the ptes pointing
|
|
|
|
- at the backing storage. Now linux mm needs a struct address_space associated
|
|
|
|
- with the struct file stored in vma->vm_file to do that with the function
|
|
|
|
- unmap_mapping_range. But the dma_buf framework only backs every dma_buf fd
|
|
|
|
- with the anon_file struct file, i.e. all dma_bufs share the same file.
|
|
|
|
-
|
|
|
|
- Hence exporters need to setup their own file (and address_space) association
|
|
|
|
- by setting vma->vm_file and adjusting vma->vm_pgoff in the dma_buf mmap
|
|
|
|
- callback. In the specific case of a gem driver the exporter could use the
|
|
|
|
- shmem file already provided by gem (and set vm_pgoff = 0). Exporters can then
|
|
|
|
- zap ptes by unmapping the corresponding range of the struct address_space
|
|
|
|
- associated with their own file.
|
|
|
|
-
|
|
|
|
References:
|
|
References:
|
|
[1] struct dma_buf_ops in include/linux/dma-buf.h
|
|
[1] struct dma_buf_ops in include/linux/dma-buf.h
|
|
[2] All interfaces mentioned above defined in include/linux/dma-buf.h
|
|
[2] All interfaces mentioned above defined in include/linux/dma-buf.h
|