14.

As illustrated, the system 10 may include one or more output devices 15 and 16, each of which may be any electronic device that includes or may be communicatively coupled to at least one speaker and may be configured to output sound by driving the speaker. For instance, as illustrated, the output device 15 is a wireless headset (e.g., in-ear headphones or earbuds) that are designed to be positioned on (or in) a user's ears, and are designed to output sound into the user's ear canal. In some aspects, the earphone may be a sealing type that has a flexible ear tip that serves to acoustically seal off the entrance of the user's ear canal from an ambient environment by blocking or occluding in the ear canal. In this case, the headset may include two earphones, a left earphone for the user's left ear and a right earphone for the user's right ear. In this case, each earphone may be configured to output at least one audio channel of media content (e.g., the right earphone outputting a right audio channel and the left earphone outputting a left audio channel of a two-channel input of a stereophonic recording, such as a musical work). In another aspect, the output device may be any electronic device that includes at least one speaker and is arranged to be worn by the user and arranged to output sound by driving the speaker with an audio signal. As another example, the output device may be any type of headset, such as an over-the-ear (or on-the-ear) headset that at least partially covers the user's ears and is arranged to direct sound into the ears of the user.

In one aspect, the output device 15 may be any type of device that may be worn by a user and produce sound directed into the user's ears, such as a headset. In another aspect, the output device may be any type of electronic device that may be worn by a user, such as smart glasses. In one aspect, the device may include one or more “extra-aural” speakers, which may be arranged to output sound into the ambient environment rather than (directly) into the user's ears. In which case, the output device may be configured to use the extra-aural speakers to produce one or more beam patterns, each of which may include at least a portion of audio content in order to produce spatially selective sound output. Such beam patterns may be directed to locations within the environment, such as a location of the user's ears.

As illustrated, the output device 16 includes one or more loudspeakers. In particular, the output device 16 includes five loudspeakers that are arranged in a 5.1 surround sound loudspeaker arrangement. In one aspect, the output device 16 may be any electronic device that includes at least one loudspeaker that is arranged to output (or project) sound into an ambient environment. Examples may include a stand-alone speaker, a smart speaker, a home theater system, or an infotainment system that is integrated within a vehicle.

In one aspect, the playback device 14 may be configured to spatially render audio content to produce one or more output audio signals (or speaker drivers), with which the playback device may use to drive one or more speakers of the playback device 14, the output device 15, and/or the output device 16. For instance, upon producing the output audio signals, the playback device 14 may transmit the signals to the output device 15 for playback.

As described herein, the system 10 may be configured to perform spatial audio processing operations to produce an augmented ambisonics format. For instance, one or more devices of the system may perform at least some of these operations, such as the playback device 14. In another aspect, either of the output devices may perform at least some of the operations described herein. In which case, the playback device may be an optional device, whereby an output device, such as output device 15, may receive audio content, augment the audio content (e.g., upmix the audio content into a higher-order ambisonics), store the augmented audio content, and/or spatially render the augmented audio content through one or more speakers.

In some aspects, the playback device 14 and the output device 15 (or device 16) may be distinct (separate) electronic devices, as shown herein. In another aspect, the playback device may be a part of (or integrated with) an output device. For example, as described herein, at least some of the components of the playback device (such as a controller, memory, etc.) may be part of the output device 14, and/or at least some of the components of the output device, such as one or more speakers may be part of the playback device. In this case, each of the devices may be communicatively coupled via traces that are a part of one or more printed circuit boards (PCBs) within the devices.

FIG. 2 is a block diagram of a playback device of the system that produces an augmented ambisonics format from a lesser order ambisonics according to one aspect. The playback device 14 includes an audio file 17, a desired HOA format 18, and a controller 20. In one aspect, the audio file 17 and the desired HOA format 18 may be a part of or stored within memory of (e.g., the controller 20 of the) playback device 14. In one aspect, the elements may be a part of one or more other electronic devices, such as the audio file being a part of (e.g., stored in memory of) the media content device 12. In which case, the playback device may stream the audio file 17, via the network 13, from the media device 12. In another aspect, the controller 20 may be a part of another device, such as the output device 15. In which case, the operations described herein may be performed by an output device, and therefore the playback device 14 may be an optional device of the system 10.

The controller 20 may be a special-purpose processor such as an application-specific integrated circuit (ASIC), a general-purpose microprocessor, a field-programmable gate array (FPGA), a digital signal controller, or a set of hardware logic structures (e.g., filters, arithmetic logic units, and dedicated state machines). The controller 20 may be configured to perform audio signal processing operations, such as spatial audio processing operations and/or networking operations. More about the operations performed by the controller 20 is described herein.

In one aspect, the audio file 17 may include any type of audio content, such as a musical composition. The audio file may include an ambisonics audio recording as one or more channels that may be formatted in B-format or higher in one of numerous higher-order ambisonics formatting conventions, for example ACN, SID, Furse-Malham or others and different normalization schemes such as N3D, SN3d, N2D, SN2D, maxN or others, which can result in additional loss. The audio file may include a FOA or HOA representation of a sound field that includes several audio signals (or channels). In some aspects, the audio file 17 may be produced (e.g., in a recording studio) to include audio content as an ambisonics recording. In another aspect, the audio file may be a recording of one or more microphones (not shown) of the system 10. In which case, microphones that may be a part of one or more devices of the system 10 may capture sound of the ambient environment, which may be stored in an ambisonics format.

The desired HOA format 18 includes an indication of an order of ambisonics, which may be greater than a FOA. For instance, the desired format may be a 3^rdorder ambisonics or a 5^thorder ambisonics. The format may be “desired” such that it may be user-defined. In which case, the format may be received through an input device, such as a tablet computer with a touch-sensitive display screen. In another aspect, the desired format may be predefined in a controlled setting, such as a laboratory.

The controller 20 has several operational blocks for performing audio spatial processing to produce an augmented ambisonics format of (a piece of) audio content. As shown, the controller includes a virtual loudspeaker layout 21, a gain estimator 22, a sound field analyzer 23, a (spatial audio) renderer 24, a filter estimator 25, a HOA encoder 26, a storage 27, a (optional) renderer 28, and a (optional) desired speaker layout 19. In one aspect, the controller may have more or less operational blocks. For example, the controller may include one or more gain estimators, each of which may be configured to estimate one or more gains that may be applied to the audio content. In another aspect, the controller may not include the renderer 28 and the desired speaker layout 19 since both of these blocks are optional. A description of the operational blocks is as follows.

The controller 20 may be configured to receive the audio file 17, which includes “P” audio signals 71 of audio content. As described herein, the audio content may be in a spherical audio format, such as a FOA audio format that includes a FOA representation of a sound field as several the several audio signals 71. In the case of FOA, the P=4. In one aspect, the controller may receive the audio file based on user input. For example, a user may request (e.g., via one or more user input devices, such as a touchscreen) a media software application being executed by the controller 20 to stream audio content (e.g., from the media content device 12). In which case, the controller 20 may receive the audio content as a FOA representation via the network 13. Alternatively, the controller 20 may retrieve the audio file 17 from memory, which may be internal (or a part of the playback device 14) or of an external device. In another aspect, the audio file 17 may be of another order of ambisonics, such as a 2^ndorder. The controller 20 may be configured to determine a HOA format to which the received audio signal is to be upmixed as the desired HOA format 18 of the audio signal. In one aspect, the controller 20 may retrieve the desired HOA format 18 from memory, and/or the desired format may be received based on user input.

The virtual loudspeaker layout 21 may be configured to determine a layout of a virtual loudspeaker array based on the HOA format 18. In particular, the determined layout of the virtual loudspeaker array may include an even distribution of the virtual loudspeaker array on a surface of a sphere centered around a virtual listening position, where the arrangement and/or number of loudspeakers of the virtual loudspeaker array may be based on the desired HOA format 18. For example, the virtual layout may be a t-design of virtual loudspeakers, where t may be a parameter that may be based on the desired HOA format. In particular, the virtual layout 21 may include different t-designs for different orders of HOA. For instance, the parameter may be a function of the order of ambisonics, such that t≥2N+1, where N is the order of ambisonics. In one aspect, the parameter, t, may indicate the number of points along the sphere that may represent locations of virtual loudspeakers around the sphere, the distribution or arrangement of loudspeakers on the sphere, and/or the shape of the sphere may be based on the (e.g., order of the) desired HOA format. For instance, as the parameter increases, the number of loudspeakers on the sphere may increase as the order of the HOA format increases. As an example, when the order of ambisonics is N=2, the virtual layout may be a 5-design that may include an icosahedron with twenty faces (as equilateral triangles) with twelve points, each point representing a location of a virtual loudspeaker. In one aspect, to determine the layout 21 of the virtual array the controller 20 may perform a table lookup into a data structure that associates HOA formats with one or more spherical t-designs of virtual loudspeaker arrays.

The gain estimator 22 may be configured to determine one or more vector-base amplitude panning (VBAP) gains for spatially rendering the audio content of the audio signals 71 based on the desired HOA format 18. In particular, the estimator may determine one or more VBAP gains based on the layout of the virtual loudspeaker array determined by the layout 21. VBAP may use a triangulation of three loudspeakers to produce 3D sound, as a virtual sound source inside an area of the loudspeakers. To cause the 3D sound VBAP produces a gain vector (e.g., three gains, one for each loudspeaker) that may be applied as loudspeaker gains to one or more input audio signals. In the present case, the estimator 22 may determine one or more gain vectors for the virtual loudspeakers array. For instance, the estimator may determine one or more gain vectors for each group of virtual loudspeakers that make up vertices of each face of the sphere of the t-design. In which case, the estimator may determine gain vectors for one or more virtual sound sources within at least some of the faces of the sphere, with virtual loudspeakers at their vertices. In some aspects, the gain estimator 22 may determine one or more VBAP gains for each virtual loudspeaker in the virtual loudspeaker layout. In one aspect, to determine the VBAP gains 72, the estimator may perform a table lookup into a data structure that associates VBAP gains with virtual loudspeaker layouts.

The renderer 24 may be configured to receive the (audio signals 71 of the) audio content of the audio file 17 and the VBAP gains 72, and produce several virtual loudspeaker audio signals 74 as spatially rendered audio signals by spatially rendering the signals 71 according to the layout of the virtual loudspeaker array (from the layout 21). The renderer 24 may spatially render (e.g., linearly render) the audio signals 71 using the VBAP gains. In one aspect, the renderer may linearly render the audio signals by applying (e.g., multiplying) the VBAP gains 72 to at least some of the audio signals 71 to produce the virtual loudspeaker signals 74. In one aspect, the renderer may produce N virtual audio signals 74, at least one signal for each virtual loudspeaker of the virtual loudspeaker array determined by the layout 21 (based on the desired HOA format 18), where the virtual signals 74 may include at least a portion of the sound field of the original audio signals 71. In one aspect, to spatially render the sound field of the audio signals 71, the renderer may determine one or more virtual sound sources within the sound field that may be associated with (produced by) one or more virtual loudspeakers of the virtual loudspeaker array in order for the array to produce the sound field, and then select VBAP gains 72 associated with the virtual loudspeakers of the virtual loudspeaker array, which may then be applied to one or more of the audio signals associated with the virtual loudspeakers. In one aspect, the renderer may use any spatial rendering method to render the audio signals 71 to the virtual loudspeaker array associated with the desired HOA format.

Thus, the controller 20 may be configured to determine the gains 72 from the desired HOA format (order) 18 to match a corresponding virtual loudspeaker setup (arrangement). The virtual positions of the virtual loudspeakers may be derived based on the gains, and once gains have been determined the renderer may determine virtual sound sources in between the virtual loudspeakers by assigning one or more gains to the virtual loudspeakers.

As described thus far, the controller 20 may be configured to transform the audio content into the virtual loudspeaker audio signals 74 that may be an intermediate format associated with the uniform virtual loudspeaker array. In one aspect, if the spatially rendered audio signals were used to drive loudspeakers of the array, the sound produced by be very correlated and therefore less desirable. In which case, to improve the sound the controller may be configured to enhance the audio content by performing a parametric analysis to produce filters (e.g., sharpening filters) to be applied to at least some of the virtual signals before re-encoding the signals into a higher order of ambisonics. As a result, the controller may be configured to upmix the original ambisonics format into a higher order that with a higher spatial resolution. The parametric analysis may be as follows.

The sound field analyzer 23 may be configured to receive the audio signals 71 and perform a sound field analysis upon the signals to determine (produce) one or more (spatial) parameters 73 associated with (e.g., one or more sound sources of) the sound field of the (e.g., FOA data of the) audio content of the audio file 17. In one aspect, the analysis may be performed in the time-frequency domain. In which case, the controller may be configured to transform the audio signals 71, which may be in the time-domain, into the time-frequency signals. Time-frequency signals may include frequency components of the audio signals with respect to (or as a function of) time. The analyzer 23 may determine parameters of at least some time-frequency signals of the sound field that quantify one or more properties of the sound field depending on frequency and time. For example, the analyzer 23 may determine a DoA associated with one or more sound sources of the sound field based on an acoustic analysis of at least some of the time-frequency signals, such as being based on cross-correlation between two or more signals and/or acoustic intensity. The analyzer 23 may determine other parameters that may indicate spatial characteristics of one or more sounds of the sound field, such as inter-channel level differences (ICLD), inter-channel time differences (ICTD), and/or inter-channel coherences (ICC). As another example, the analyzer 23 may determine a direct-to-ambience ratio of sound of the sound field by identifying one or more directional components, which may be identified based on a strong correlation between two or more signals, whereas the ambience may be determined based on sound that is fully or partially uncorrelated with the directional component. Other parameters may include diffuseness of the sound field and reverberance of the sound field. In one aspect, the analyzer 23 may use any method to determine any type of parameter that may provide a quantitative properties of the sound field of the audio signals 71 in the time-frequency domain. For instance, the analyzer may estimate DoA of one or more sound sources using multiple signal classification analysis. The analyzer may use (e.g., non-linear) machine learning based methods for parameter estimation.

The filter estimator 25 may be configured to receive the parameters 73 produced by the analyzer 23 and one or more of the audio signals 74 (which may be transformed from the time-domain into the time-frequency domain), and may be configured to estimate (or determine) one or more adaptive filters 75 based on the parameters 73 and/or at least some of the audio signals 74. The filters 75 may include sharpening filters that may provide spatial enhancement of a spatial rendering of the audio content. For example, when applied to one or more of the virtual loudspeaker audio signals 74, the sharpening filters may enhance direction components of one or more signals. In which case, the filters may enhance sound (as perceived by a listener) of one or more sound sources within the sound field. In one aspect, the filters 75 may be non-linear and/or linear filters. The sharpening filters may be any type of audio filter, such as high-pass filters, low-pass filters, band-pass filters, etc. In another aspect, the filters may be signal-dependent. In particular, the adaptive filters may include time-frequency adaptive weights, which may be adaptive based on changes to the audio signal(s) 74. In one aspect, the filters produced by the estimator 25 may be based on the desired HOA format 18 in which the virtual signals 74 are to be encoded. The estimator 25 may produce one or more filters 75 for each of the audio signals for the virtual loudspeaker array. In which case, the estimator 25 may adjust the number and/or type of filters produced based on changes to the virtual loudspeaker array. For instance, if the desired HOA format changes, the filter estimator 25 may adjust the number of filters 75 produced (e.g., based on changes to the number of ambisonics signals associated with the changed format). In another aspect, the adaptive filters may be produced through any method using at least one of the audio signals 74 and/or at least one parameter 73.

The controller 20 may be configured to produce filtered audio signals 76 by applying (e.g., multiplying) the filters 75 to one or more of the virtual loudspeaker audio signals 74. In one aspect, the controller may filter the signals 74 using one or more filters in order to improve (enhance) the spatial resolution of the audio content, as described herein. The HOA encoder 26 may be configured to receive the filtered signals 76 and produce R audio signals 77 in the desired HOA format 18 by encoding the signals 76 according to the desired format 18. For instance, the encoder 26 may apply an encoding matrix upon at least some of the N filtered signals 76 to produce the audio signals 77. In one aspect, the HOA format of the audio signals 77 may be a higher order ambisonics of the audio format of the audio signals 71. In which case, the number of R signals of the audio signals 77 may be greater than the number of P signals of the audio signals 71. For instance, when the audio file 17 is a FOA, P=4, and when the HOA encoder 26 upmixes the audio content into a 2^ndorder ambisonics, R=9.

In one aspect, the HOA encoder 26 may encode the signals 76 in the time-frequency domain. In which case, the controller may apply an inverse time-frequency transformation upon the audio signals 77 to transform the signals into the time-domain. In another aspect, the controller may transform the signals 76 into the time-domain before the HOA encoder 26 re-encodes the audio content into the desired HOA format 18. The controller 20 may be configured to store the audio signals 77 in the desired HOA format in storage 27, which may be a storage device (e.g., memory) of the controller 20. In another aspect, the storage 27 may be a part of another electronic device.

In one aspect, the controller may optionally spatially render the encoded HOA data through one or more output devices. For instance, the renderer 28 may receive the audio signals 77 in the encoded HOA format, and may produce one or more rendered (or driver) signals 78 by spatially rendering at least some of the audio signals 77 based on (according to) a speaker layout 19 of an output device, such as the output device 15.

In one aspect, the speaker layout 19 may include an indication of arrangement of speakers of one or more output devices. For example, with respect to the output device 16 that includes five loudspeakers, the speaker layout 19 may indicate the number of loudspeakers and/or the placement of the loudspeakers with respect to each other (and/or with respect to a reference point within the environment, such as a listening position). With respect to the output device 15, the speaker layout 19 may indicate that the speakers are of a headset. The controller 20 may be configured to determine the speaker layout 19 of an output device that may be communicatively coupled to the playback device 14. In another aspect, the controller 20 may determine the layout of an output device that is to (or is) playing back the audio content. As described herein, the speaker layout 19 may be stored in memory of the playback device. In which case, the speaker layout may be provided by an output device through which audio content is being (or to be) played back. For example, the output device 16 may provide the speaker layout to the playback device 14, via a wireless data connection. In another aspect, the speaker layout 19 may be determined through the use of one or more sensors of the system 10, such as a camera. In which case, the camera may capture an image of an output device 16 and may determine the layout of the loudspeaker(s) of the device based on image recognition.

In one aspect, the renderer 28 may perform non-parametric spatial audio rendering upon one or more of the audio signals 77 to produce one or more driver signals. In the case of a headset (e.g., output device 15), the renderer 28 may produce two driver signals. In one aspect, the renderer 28 may apply one or more spatial filters, such as head-related transfer functions (HRTFs) upon the spatially rendered signals. Continuing with the previous example, when the speaker layout 19 indicates a headset, the renderer may perform linear spatial rendering upon the ambisonics audio signals 77 to produce two rendered signals (a left signal and a right signal), and may apply the HRTFs to produce one or more binaural audio signals as the one or more output audio signals 78. In another aspect, the renderer 28 may perform any type of spatially rendering technique to produce spatially rendered audio signals 78 from the audio signals 77 based on the speaker layout 19.

In one aspect, the renderer 28 may adjust rendering based on head tracking data from one or more (head tracking) sensors (not shown) of the system 10. For example, the output device 15 may include one or more head-tracking sensors, which may monitor head movements of the user, and may provide those movements to the renderer 28, which may adjust the spatial rendering accordingly.

FIGS. 3 and 5 are flowcharts of processes 30 and 50, respectively for performing one or more audio signal processing operations for producing an augmented audio format from another audio format. In one aspect, the processes may be performed by one or more devices of the system 10, as illustrated in FIG. 1. For instance, at least some of the operations of one or more of these processes may be performed by (e.g., the controller 20 of) the playback device 14. As a result, at least some of the operations described herein may be with reference to FIGS. 1 and 2. In another aspect, at least some of the operations may be performed by another device, such as the output device 15 and/or a remote server communicatively coupled to the playback device 14 and/or the output device 15.

Turning to FIG. 3, this figure is a flowchart of one aspect of a process performed by the system 10 to produce an augmented ambisonics format that is an upmixed format of an original ambisonics format according to one aspect. The process 30 begins with the controller 20 receiving audio content in a FOA format that includes a first group of audio signals (at block 31). For instance, the controller 20 may receive the audio signals 71 audio file 17 that includes FOA audio content. The controller 20 produces several spatially rendered audio signals by spatially rendering the audio signals 71 according to a layout of a virtual loudspeaker array (at block 32). For instance, the controller may determine a desired HOA format (e.g., based on user input) for the received FOA data, and may determine the (e.g., t-design) layout based on the desired format. The controller may determine one or more VBAP gains 72 and may apply the gains to the received audio content to produce the spatially rendered audio signals 74.

The controller 20 determines one or more filters 75 by performing a parametric analysis upon at least one of the received audio signals 71 (at block 33). The controller 20 filters at least one of the spatially rendered audio signals (e.g., signals 74) using the one or more filters (at block 34). The controller produces a second group of audio signals (e.g., signals 77) that are in a HOA format based on the spatially rendered audio signals (at block 35). As described herein, the controller 20 may apply the filters 75 to the audio signals 74 that are spatially rendered to produce the filtered audio signals 76, where the controller may encode the audio signals into the HOA format. As described herein the HOA format may be a higher order ambisonics than the FOA of the received audio content. The controller 20 stores the second group of audio signals (e.g., encoded audio signals 77) in a storage device in the desired HOA format, such as storage 27 (at block 36).

In addition, or in lieu of, storing the encoded audio signals, the controller may spatially render the encoded signals for playback through one or more playback devices. The controller 20 may determine a speaker layout (at block 37). For instance, the controller may determine whether an output device, such as the headset 15 is communicatively coupled with the playback device. In which case, the controller may determine its speaker layout (e.g., by performing a table lookup into a data structure that associates speaker layouts with one or more device characteristics, such as a unique identifier of the output device (e.g., a model/serial number)). The controller spatially renders the second group of audio signal according to the speaker layout (of an output device) to produce one or more speaker drivers (at block 38). The controller 20 causes one or more speakers (of the output device) to playback the one or more speaker drivers (at block 39). For instance, the playback device may transmit the driver signals to the output device for playback.

In another aspect, the playback device may spatially render the audio content for playback by one or more speakers that may be a part of the playback device. In which case, the controller 20 may determine a speaker layout of the playback device, and may use the driver signals to drive its speakers for playback.

As described herein, the controller 20 may perform at least some of the operations of process 30. The controller 20 may produce the second group of audio signals in the HOA format, store, and/or spatially render the signals. In another aspect, one or more electronic devices of the system 10 may perform these operations. For example, the media content device 12 may be configured to perform at least some of these operations to produce the audio content in the higher order ambisonics format and to store the audio content, whereas the playback device 14 (and/or output device 15) may spatially render the audio content. In which case, the playback device 14 may be configured to receive the audio content in the HOA format, and may spatially render the audio content for playback, as described herein.

FIG. 4 is a block diagram of a playback device of the system that produces an augmented ambisonics format from one or more microphone signals according to one aspect. The block diagram is similar to the block diagram of FIG. 2, except that the playback device 14 includes a microphone array 41 with one or more microphones 40. In this case, the controller 20 may receive S microphone signals 42, each microphone signal includes sound of an ambient environment captured by a respective microphone 40 of the microphone array 41. In which case, the microphone signals 42 may include a sound field that has one or more sounds of the ambient environment. In one aspect, the microphone array 41 may be a part of or integrated with (e.g., a housing of) the playback device 14. In another aspect, the microphone array 41 may be a part of one or more electronic devices that are located within the ambient environment that may be communicatively coupled with the controller 20. As an example, the microphone array 41 may be a part of the output device 15 that may wirelessly couple to the playback device 14.

In one aspect, the controller 20 may perform one or more operations described herein (e.g., with respect to FIGS. 2 and 3) to produce an augmented ambisonics format from one or more of the S microphone signals 42 captured by the microphone array 41. In which case, the renderer 24 may receive the microphone signals 42 and VBAP gain(s) 72 produced by the estimator based on a desired HOA format, and may spatially render the microphone signals into several virtual loudspeaker audio signals 74 according to a layout of the virtual loudspeakers and using at least some of the VBAP gains. In particular, the sound field analyzer 23 may determine one or more parameters 73 associated with the ambient sound captured by the microphones by performing the parametric analysis upon one or more of the microphone signals 42, as described herein. The renderer 24 may produce spatially rendered audio signals (e.g., signals 74) by spatially rendering the microphone signals to a virtual loudspeaker array, and may produce filtered signals 76 by filtering the spatially rendered signals based on one or more of the parameters. The HOA encoder 26 may encode the filtered signals into HOA signals, as described herein.

As a result, the produced HOA format by the HOA encoder 26 may be an upmix from the microphone signals 42, whereby the produced R HOA signals 77 may include more audio signals than the S microphone signals. Thus, the HOA audio signals 77 may include a sound field that includes one or more sounds of the ambient environment that were captured by the microphone array 41, and is an upmix from the S microphone signals.

FIG. 5 is a flowchart of another aspect of the process 50 performed by the system 10 to produce an augmented ambisonics format of audio content according to another aspect. The process 50 begins with the controller 20 receiving a first group of audio signals (at block 51). For instance, the audio signals may be ambisonics signals that may be a part of an audio file and/or may include one or more microphone signals that includes audio content of an ambient environment. In one aspect, the ambisonics signals may include any type of audio content, such as sound of an XR environment. The controller 20 produces several spatially rendered audio signals by spatially rendering the first group of audio signals according to a layout of several virtual loudspeakers (at block 52). As described herein, the controller may spatially render the signals based on a desired HOA format into which the received signals are to be encoded. The controller 20 produces several filtered audio signals by filtering at least one of the spatially rendered audio signals based on a sound-field analysis of the first group of audio signals (at block 53). The controller may produce filters based on the sound-field analysis in order to sharpen or enhance the spatial resolution of the sound field captured within the received audio signals. The controller 20 encodes the several filtered audio signals into a second group of audio signals in a HOA format, where the second group of audio signals is an upmix of the first group of audio signals (at block 54).

FIG. 6 shows a block diagram of hardware of an audio processing system 90, in one aspect, which may be used with or be a part of any of the aspects described herein (e.g., system 10, which may include the media content device 12, playback device 14, and/or output device 15 or 16). This audio processing system 90 can represent a general-purpose computer system or a special purpose computer system. Note that while FIG. 6 illustrates the various components of an audio processing system that may be incorporated into one or more of the devices described herein, it is merely one example of a particular implementation and is merely to illustrate the types of components that may be present in the audio processing system. FIG. 6 is not intended to represent any particular architecture or manner of interconnecting the components as such details are not germane to the aspects herein. It will also be appreciated that other types of audio processing systems that have fewer components than shown or more components than shown in FIG. 6 can also be used. Accordingly, the processes described herein are not limited to use with the hardware and software of FIG. 6.

As shown in FIG. 6, the audio processing system (or system) 90 (for example, a laptop computer, a desktop computer, a mobile phone, a smart phone, a tablet computer, a smart speaker, a head mounted display (HMD), a headphone set, or an infotainment system for an automobile or other vehicle) includes one or more buses 98 that serve to interconnect the various components of the system. One or more processors 97 are coupled to bus 98 as is known in the art. The processor(s) may be microprocessors or special purpose processors, system on chip (SOC), a central processing unit, a graphics processing unit, a processor created through an Application Specific Integrated Circuit (ASIC), or combinations thereof. Memory 96 can include Read Only Memory (ROM), volatile memory, and non-volatile memory, or combinations thereof, coupled to the bus using techniques known in the art. Camera 91, microphone(s) 92, speaker(s) 93, and display(s) 94 may be coupled to the bus.

Memory 96 can be connected to the bus and can include DRAM, a hard disk drive or a flash memory or a magnetic optical drive or magnetic memory or an optical drive or other types of memory systems that maintain data even after power is removed from the system. In one aspect, the processor 97 retrieves computer program instructions stored in a machine-readable storage medium (memory) and executes those instructions to perform operations described herein.