LL userspace: Re-enable ipc msg send pos updates#10796
Draft
jsarha wants to merge 118 commits into
Draft
Conversation
If CONFIG_SOF_BOOT_TEST_STANDALONE is set, ipc_init() is terminated early. This ensures SOF will not start to generate or respond to IPC messages that could potentially interfere with standalone test cases (some of which send and receive IPCs). The current implementation leaves the component list uninitialized and this can cause trouble to standalone tests that want to utilzie common IPC code to build messages. Fix this problem by executing more of ipc_init() also in the standalone mode. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
In user-space LL builds (CONFIG_SOF_USERSPACE_LL), irq_local_disable() is a privileged operation. In ipc_comp_free(), use sys_mutex_lock/unlock on the per-component list_mutex to protect buffer list iteration instead. This follows the same locking pattern as PPL_LOCK/PPL_UNLOCK used in pipeline_disconnect(). Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Add a built option to make sof_heap_allo/free available as system calls to user-space. Add a test case for the functions that runs in a user-space thread. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Separate the state for LL scheduler memory into kernel and user accessible resources. The pointer to the LL heap must be accessible from user-space, so that user space can allocate memory and pass the heap pointer as argument. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Turn the pdata->sem into a dynamic object in userspace LL builds. Keep statically allocated semaphore for kernel LL builds. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Replace the dynamically-allocated k_sem (via k_object_alloc) in zephyr_ll_pdata with an embedded sys_sem. The dynamic kernel semaphore could not be freed from user-space threads, causing kernel object leaks that exhausted the kernel object pool during long test runs. sys_sem uses k_futex in user mode and lives in user memory, so no kernel object allocation or cleanup is needed. Add POSIX no-op stubs for sys_sem to maintain testbench build compatibility. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Move the user-accessible heap pointer to system user partition defined in userspace_helper.h. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Add an overlay for Intel 'ptl' target that allows to build SOF with all audio pipeline code running in Zephyr user-sapce. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
A problem exists with DSP panics due to illegal instruction hit in user-space if cold store execution is enabled. Disable it for now until rootcause is found. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Set CONFIG_SOF_ZEPHYR_LL_USER_HEAP_SIZE to 124kiB in the LL user-space overlay. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Allow a non-null pointer at the end of the DMA transfer block list, if and only if it points to the first entry in the block list. The SOF DAI module sets the DMA transfers blocks like this and this change is required to use DAI module from user-space. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
The platform data descriptions need to be accessible to all system threads. These are e.g. used when setting up host/DAI copiers and they need platform DMA properties. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Use separate heaps depending whether host copier is run in user or kernel space. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Make sure we use the same heap to allocate DMA SG buffers as we use for other component resources. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Force to use user-space LL heap in host_common_new() if CONFIG_SOF_USERSPACE_LL is set. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Ensure component heap is correctly passed whenever memory is allocated in the component. This allows to run the component both in kernel and user space. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Use a module context instead of heap to allocate objects in chain-dma module. This is required to support vregion use. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Convert all memory allocations to use the sof_heap_alloc() interface and pass the dai_data specific heap object. This makes dai-zephyr code compatible with use from user-space, but does not effect kernel space use. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
The dai_get()/dai_put() provide a helper to access DAI devices. When used in user-space, the wrapper struct should be created in user-space memory. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
The spinlock used to protect access to DAI properties can be converted to a mutex as this is only accessed from IPC and LL threads and both are normal Zephyr threads. As an additional benefit, use of mutex allows to run the dai-zephyr module in user-space. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Modify code to allocate DAI properties object on stack and use dai_get_properties_copy(). This is required when DAI code is run in user-space and a syscall is needed to talk to the DAI driver. It's not possible to return a pointer to kernel memory, so instead data needs to be copied to caller stack. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
- replace dynamically-allocated k_mutex *lock with embedded sof_umutex - remove k_object_alloc/k_thread_access_grant boilerplate - remove k_object_free leak workaround in dai_common_free() Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Use a module context instead of heap to allocate objects in dai-zephyr module. This is required to support vregion use. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
No need to copy the trace context object to buffer object, when audio pipelines are running in user-space. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
To make ipc4_create_buffer() usable from user-space, do not call cpu_get_id() but instead rely on "src->ipc_config.core". If code is run in kernel space, check the core matching current active core, but skip the check if run in user-space (as check is privileged). Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Make ipc_comp_connect() safe to run in user-space with limited functionality. The core id queries are privileged, so limit connections to cases where pipeline is running on 0. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Move handling of SOF_IPC4_MOD_CONFIG_GET/GET to user-space IPC thread when SOF is built with CONFIG_SOF_USERSPACE_LL. Handling these IPC messages require calls to module interface get/set_attribute() methods, which must be run in user-space. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Move handling of SOF_IPC4_MOD_BIND/UNBIND to user-space IPC thread when SOF is built with CONFIG_SOF_USERSPACE_LL. Handling these IPC messages require calls to module interface bind/unbind() methods, which must be run in user-space. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Move handling of SOF_IPC4_IPC4_MOD_INIT_INSTANCE partially to user-space IPC thread when SOF is built with CONFIG_SOF_USERSPACE_LL. Module init involves multiple privileged actions, so part of module init handling is still done in kernel space, but the module specific initialization is run in user-space. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Move handling of SOF_IPC4_GLB_SET_PIPELINE_STATE to user-space IPC thread when SOF is built with CONFIG_SOF_USERSPACE_LL. The IPC compound message helpers in IPC common.h are turned into system calls, allowing the user-space IPC thread to synchronize with kernel IPC thread that still does the low-level IPC messaging. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Move handling of SOF_IPC4_MOD_LARGE_CONFIG_GET/SET to user-space IPC thread when SOF is built with CONFIG_SOF_USERSPACE_LL. Handling of these IPCs requires calling to module specific code via the module get/set_large_config() method, so this code needs to run in user-space context if audio pipelines are running in user-space. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Make ipc_msg_send() a Zephyr system call so audio processing modules running in user-space LL threads can queue IPC messages (e.g. position updates, notifications) back to the host. The change takes effect only if CONFIG_SOF_USERSPACE_LL=y. Follows the same pattern used for ipc_msg_reply(): a dedicated header with __syscall declaration, z_impl/z_vrfy split, and syscall header registration in CMakeLists.txt. The verifier validates that the msg struct is writable (the implementation touches the list linkage) and that the data buffer, when provided, is readable up to msg->tx_size bytes. Signed-off-by: Jyri Sarha <jyri.sarha@linux.intel.com> (cherry picked from commit b25c15c)
Add an optional heap parameter to ipc_msg_w_ext_init() and ipc_msg_init() so callers can direct allocations to a specific heap. When the heap argument is NULL the existing rzalloc() path is used; when non-NULL, sof_heap_alloc()/sof_heap_free() are used instead. This allows IPC messages to be allocated from userspace-accessible heaps. For audio module contexts, introduce mod_ipc_msg_w_ext_init() and mod_ipc_msg_init() in generic.h. These use mod_zalloc()/ mod_free() for allocations that are automatically tracked and freed with the module lifecycle. ipc_msg_w_ext_init() is moved from a static inline in msg.h to a non-inline function in ipc-common.c due to the additional sof_heap_alloc dependency. Update all existing callers: - Module context callers (cadence, tdfb, sound_dose) use the new mod_ipc_msg_*() variants. - host-zephyr.c uses hd->heap, pipeline-graph.c uses the heap parameter from pipeline_new(). - Remaining kernel-context callers pass NULL for the default heap. Signed-off-by: Jyri Sarha <jyri.sarha@linux.intel.com> (cherry picked from commit 11a17d5)
Move user-facing notification functions (send_copier_gateway_xrun_notif_msg, send_gateway_xrun_notif_msg, send_mixer_underrun_notif_msg, send_process_data_error_notif_msg) to a new notification-user.c file so they can run in userspace. The send_resource_notif() function, which depends on the kernel-side notification pool and IPC message infrastructure, is converted to a Zephyr syscall. The implementation is renamed to z_impl_send_resource_notif() and remains in notification.c alongside is_notif_filtered_out() and ipc4_update_notification_mask(). The send_resource_notif() is converted to a system call only if CONFIG_SOF_USERSPACE_LL=y, without it the behaviour is same as befofe. A z_vrfy_send_resource_notif() handler is added to validate the user-provided data buffer and other parameters before forwarding to the kernel implementation. Signed-off-by: Jyri Sarha <jyri.sarha@linux.intel.com> (cherry picked from commit e385f10)
A module context object needs to be allocated and stored in IPC context so that memory allocations can be done in IPC handlers when IPC handling is running in user-space. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Make vregion_alloc(), vregion_alloc_coherent(), vregion_alloc_align(), vregion_alloc_coherent_align(), and vregion_free() available as Zephyr system calls for user-space threads. Add K_SYSCALL_MEMORY_WRITE verification to all syscall handlers to validate the calling thread has access to the vregion's managed memory area. Add CONFIG_SOF_USERSPACE_INTERFACE_VREGION Kconfig option to control the feature. It is auto-selected by SOF_USERSPACE_LL when SOF_VREGIONS is enabled. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Add a new test to userspace_ll set that tests more of the functionality needed to run full audio pipelines in user-space. The test creates a pipeline with two components (IPC4 host and DAI copiers), does pipeline prepare, one copy cycle in prepared state and tears down the pipeline. One user-space thread is created to manage the pipelines. This would be equivalent to user IPC handler thread. Another user-space thread is created for the LL scheduler. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Place the pipeline position lookup table in the sysuser memory partition and replace k_spinlock with a dynamically allocated k_mutex when CONFIG_SOF_USERSPACE_LL is enabled. Spinlocks disable interrupts which is a privileged operation unavailable from user-mode threads. The mutex pointer is stored in a separate APP_SYSUSER_BSS variable outside the SHARED_DATA struct so Zephyr's kernel object tracking can recognize it for syscall verification. Move pipeline_posn_init() from task_main_start() to primary_core_init() before platform_init(), so the mutex is allocated before ipc_user_init() grants thread access to it. In pipeline_posn_get(), bypass the sof_get() kernel singleton and access the shared structure directly when running in user-space. Grant the ipc_user_init thread access to the pipeline position mutex via new pipeline_posn_grant_access() helper. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
If SOF is built with CONFIG_SOF_USERSPACE_LL, the IPC user handled will require access to coldrodata sections to initialize audio modules. This logic is not required for LLEXT modules, which have existing code to add access to coldrodata (and other sections). This commit is needed for builds where LLEXT is not used. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
This is a set of temporary changes to audio code to remove calls to privileged interfaces that are not mandatory to run simple audio tests. These need proper solutions to be able to run all use-cases in user LL version. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Move the mod_ipc_msg_w_ext_init() implementation from the inline definition in generic.h to generic.c, leaving only the declaration in the header. This reduces header bloat and allows the function to be exported as a proper symbol. Signed-off-by: Jyri Sarha <jyri.sarha@linux.intel.com>
Make mod_ipc_msg_w_ext_init() a properly tracked module resource. Add MOD_RES_IPC_MSG to the resource type enum and an ipc_msg pointer to the module_resource union. The IPC message and its tx_data are now allocated directly from the module's vregion or heap (not via mod_zalloc) to avoid creating separate MOD_RES_HEAP entries that would cause double-free on module unload. The free_contents() handler for MOD_RES_IPC_MSG mirrors ipc_msg_free(): it takes the IPC spinlock, removes the message from the IPC queue list, then frees tx_data and the message struct. Signed-off-by: Jyri Sarha <jyri.sarha@linux.intel.com>
Add a processing_module pointer to struct host_data. Set it in copier_host_create() before calling host_common_new() so the host component can use the module's tracked allocation API. In host_common_new(), use mod_ipc_msg_init() when hd->mod is set, falling back to ipc_msg_init() for the non-module-adapter path. In host_common_free(), use mod_free() to release the tracked message when hd->mod is set, falling back to ipc_msg_free() otherwise. Signed-off-by: Jyri Sarha <jyri.sarha@linux.intel.com>
Promote mod_free_all() to a proper Zephyr syscall, matching the pattern already used by mod_free(). The underlying mod_res_free() callback requires privileged mode for resource cleanup (IPC spinlocks, vregion/heap frees, etc.), so the function must execute in supervisor context. Rename the implementation to z_impl_mod_free_all(), add the z_vrfy_mod_free_all() verification wrapper, and move the declaration into the __syscall / z_impl block in generic.h. Signed-off-by: Jyri Sarha <jyri.sarha@linux.intel.com>
The IPC message is allocated via mod_ipc_msg_init() which tracks it as a MOD_RES_IPC_MSG resource. Free it with mod_free() instead of ipc_msg_free() so the resource tracking is properly cleaned up. Signed-off-by: Jyri Sarha <jyri.sarha@linux.intel.com>
Refactor ds_vamsg() to separate the text formatting from the debug stream record sending. The new ds_send_text_record() takes a pre-formatted text buffer and length, and is declared as a Zephyr syscall when CONFIG_USERSPACE is enabled. ds_vamsg() now formats the message into a local buffer and calls ds_send_text_record(), which can be invoked from both user and supervisor contexts. Add the z_vrfy wrapper with K_SYSCALL_MEMORY_READ validation and register the syscall header in CMakeLists.txt. Define DS_TEXT_MSG_MAX_LEN (128) in the shared user header to keep the buffer size consistent between user and kernel sides. Signed-off-by: Jyri Sarha <jyri.sarha@linux.intel.com>
Enable CONFIG_HOST_DMA_IPC_POSITION_UPDATES in the PTL low-latency userspace overlay to allow host position notifications via IPC when running LL pipelines in user mode. Signed-off-by: Jyri Sarha <jyri.sarha@linux.intel.com>
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The relevant commit in this PR are on top of #10558. The relevat commits are be0b40b and anything above it.
This PR finally enables DMA position updates that was disabled at early phase of LL userspace work.
These commits more or less complete the work I started #10725 . The only loose end I can think of right now is the notification currently sent by the Cadence compressed audio module. That will now surely crash, but I'll fix that once I have the latest version from @ujfalusi .
This is now a separate PR, but it would probably make sense to merge this together with #10725 to mainline as a single PR, when the time is right.