Defining the Android Audio Architecture

February 22, 2023

Blog

Android OS has captured a significant portion of today’s devices including smartphones, TV, and Tab. According to Statista, the Android mobile operating system (OS), created by Google, is the most widely used smartphone OS now, dominating the mobile OS market with a market share of close to 71.8 percent.

Android is widely adopted across all hardware platforms because it’s an open-source software code and a well-defined framework structure. Linux Kernel is the heart of the Android OS as it provides an abstraction between hardware and the various android components. Audio Processing requires a lot of complex operations and intensive algorithms to produce a high-fidelity audio output. Android audio architecture involves a lot of complex software that has to take care of many aspects such as supporting audio hardware like speakers, earpieces, headsets, HDMI, mic, Bluetooth SCO, A2DP, etc., and also different software apps like phone calls, media player recorder, VIOP application, and SIP application. This article highlights the high-level functioning of the different components in the Android audio architecture.

Overview of Android Audio Architecture

The Android audio architecture outlines how audio functionality is implemented and provides links to the pertinent source code.

Android Audio Architecture

Android audio APPs and JAVA framework

The Android framework consists of a well-defined set of Java APIs that enables the developers to write Android apps with ease. The Android audio app code uses this set of different multimedia (android.media) framework APIs. Internally, these APIs call corresponding Java Native Interface (JNI) glue software code to access the native components that further interact with the lower layers to play audio on the hardware.

Binder

Inter Process Communication (IPC) is an important mechanism for all the different Android components to exchange information. Android-specific IPC named “Binder” is used by the Android process to invoke a routine in another Android process and exchange the data through arguments. Upper layers use the Binder APIs to communicate with native audio services and lower hardware layers.

Audio manager

Audio Manager is a native Android service, which is the core component of the entire audio subsystem. From the Android 7 Nougat release instead of being a part of the native media service, it runs as a separate Android process in the user space. It runs three threads for different services, which are audio policy, audio flinger, and audio to carry out its functionality.

Audio policy service

Primarily the policy decision includes, opening a new input/output audio stream, and re-routing the audio to the different audio devices once changed or selected through UI and stream volume management. Audio Policy Service handles all the actions needed for the execution of these policy decisions. This service also keeps a track of user events and the current system state to notify the audio flinger service.

Audio flinger service

Audio Flinger Service invokes the audio hardware abstraction layer (HAL) APIs to interact with audio hardware.  It provides an accessible interface for the upper layers. The policy decisions and strategies defined by Audio Policy Service are executed by this service. It is also responsible for communicating with the audio device and maintaining its state. It deals with the mixture of multiple audio streams (notification and music), volume scaling, audio stream writes, and providing the media player with information like written audio frames.

Audio service

Audio is a new Android C API introduced in the Android 8 Oreo release. It is designed for high-performance audio applications that require low latency. Android audio apps invoke audio APIs for reading and writing data to audio streams. Audio service handles these low-latency audio APIs. There are two modes in audio APIs, viz “Exclusive mode” which allows client application code to write directly into a memory mapped buffer that is shared with the ALSA driver and “Shared mode” which mixes all the shared streams and follows the audio HAL path.

Audio HAL

A Hardware Abstraction Layer (HAL) is a layer in the Android stack that provides a standard access interface to the native audio services running in the upper layer. Audio HAL bridges the upper layers to lower-level audio driver implementations. A vendor has to implement Audio HAL interfaces with the specific audio driver code. Audio HAL runs in user space as a separate Android process to execute the functionality needed by the upper layer.

The main functionality offered by audio HAL is listed below.

• Managing the opening and closing of audio input/output streams, which includes setting the stream parameters like audio channels, sampling rates, and audio formats. It has to also handle the different state of the media players which includes play, pause, resume, and standby, and provide the feedback path data for written audio frames and timestamps on hardware.
• Loading the specific set of libraries that needs special audio processing and decoding, thereby sending the raw data to the audio hardware.

Audio Device Interface

Android audio architecture supports different types of audio devices, that includes Bluetooth devices, TV tuner systems, HMDI connections, and I2S speakers. Audio HAL is responsible for sending the audio packets from the upper layer to these devices. Once a particular audio device is selected or plugged into the Android hardware, then the audio policy manager executes a set of policies defined and associated with the audio device and all the layers are configured to support the audio data to flow through it. Audio HAL particularly loads the interface code that interacts with the audio device and does read write operations over it, this interface code also carries out inter HAL IPCs for audio devices connected to the other HAL, for example, Bluetooth audio devices which require a Bluetooth HAL to exchange data.

Kernel driver

Audio HAL implementation has code that interacts with the audio drivers. Android kernel uses Advanced Linux Sound Architecture (ALSA) subsystem, or the vendor can introduce a custom driver which is HAL agnostic. Tiny ALSA is a small library to interface with ALSA in the Linux kernel which is used by audio HAL to interact with the different sound cards.

Android stack is huge software code that needs to be built for different hardware platforms. It is very difficult to understand and modify the lower Android layers as it requires a lot of debugging skills. Hardware level implementations change with every vendor and it needs to be mapped accurately with the interfaces defined for the upper layers. Many undocumented parts of the code base are more complex and vendor specific, and it uses intricate IPC mechanisms, and switches between programming languages.

Softnautics enables multimedia companies to design and develop connected multimedia solutions. We help our customers with different customizations and integrations required in the lower layers of Android, which mostly include native codes in user and kernel space. We also assist clients in adding audio based wireless devices and integrating it in the Android audio subsystem. We have expertise in understanding different layers in Android architecture for customization and new designs around it. We have hands-on experience in designing high performance media applications, architect complete video pipelines, audio/video codecs development, applications porting, and more.

Read our success stories related to multimedia engineering to know more about our services.