STM32 iPod/iPhone Accessories Library

General Presentation
15th

November Draft 0.2

iPod application - Examples

How does it work? 1/2

Licensing plan:
Customer must be Apple MFi licensee* to get:
Authentication Coprocessor (ACP) mandatory 30-pin male connectors Testing/certification tools and softwares A customer notice must be signed and returned to division The customer must also be registered on the audio SharePoint by local ST Sales and marketing people. After completion of these last 2 steps, the STM32 iPod/iPhone Accessories software can be delivered to the customer. More details will be provided shortly by the division.

STMicroelectronics provides Apple firmware protocol iPod Accessory Protocol (iAP)

*MFi: Made For iPod program click here
3

How does it work? 2/2

iPod 30-pin connector can provide audio data in different formats:
Digital audio:
USB host core USB Audio class for PCM audio data USB HID class for control and information data

Analog audio:
UART communication for control and information Direct output of analog audio signals on the 30-pin connector

iPod application features

iPod Accessory Protocol provided by ST:
Full compatibility * through iPod/iPhone/iPad families IDPS** authentication 2.0B Digital audio (USB 2.0 FS) Analog audio (UART) Automatic switch from USB to UART communication to support older iPod

* : Fully compatibility : the API/library is based on Apple_Release_2010-9. Full Testing, characterization is on-going **IDPS: Identify Device Preferences and Settings mandatory for any new iPod accessory

Firmware summary
Cortex-M3 CPU 71.88 MHz

64 Kbyte SRAM

256 Kbyte Flash

Volume/ Ch Mixer

DMA

iAP Stack
Loudness Control Audio Class* HID Class* FS USB Host *
XTAL oscillators 32 kHz + 3-25 MHz

5 bands - Equalizer

IS
PLL block

Audio DAC

Amp

HMI Control & Display

Speakers / Headset

USB

UART

I2C

SPI

14.7456 MHz quartz

iPod/iPhone 30-pin connector

Authentication Coprocessor

Touch Screen QVGA LCD

XXX

FW/API from ST Licensed FW from HCC embedded

XXX *

iPod demoboard overview

Connectivity STM32F105 and iPod

USB Host Connector

STM32F105 Connectivity Line

USB Host

iPod/iPhone 30-pin connector

USART

I2C I2S
DAC

Authentication Coprocessor

I2C

Suitable STM32F105 products: -STM32F105RCT6: 256K/64pins -STM32F105RBT6: 128K/64pins -STM32F105VCT6: 256K/100pins -STM32F105VBT6: 128K/100pins

Audio stream

Control data
8

Amplifier

iPod docking demoboard

6
Key Features:

1. Extension connector: straightforward connection to STM3210C eval board via CN1 and CN2 2. iPod/iPhone Authentication Coprocessor 2.0B class 6 (ACP) 3. iPod/IPhone connector 4. Hardware jumpers setting
I2C address selection for ACP module Accessory Identify : UART or USB Power select: iPod/iPhone board power source from iPod or STM3210C eval board

5. Video extension connector 6. Analog Audio

Input analog: Microphone Audio Pre-amplifier module Jack connector (analog audio output from iPod)

STM32 iPod/iPhone control Ref Design (1/2)

Media Player WAV, MP3, *WMA
*MP3 Encoder *MP3 Mixer Media Browser

Cursor (Multi-Input) Joystick & Touch

100

Audio Player
Tags Display MP3
Title : Chariots Of fire Artist : Vangelis Album :STMicroelectronics Genre : Instrumental Year : 1981

80 60 40 20

Volume Control Using Software Mixer

Audio Utilities Loudness 5 bands Equalizer

Progressive Progress Bar

0:25

0:30

0 Mute

Mute Control

Home Repeat

*Karaoke
Game

iPod Control Info & Credits

Play Pause

Browser Fast Forward/Backward

Stop

Cursor (Multi-Input) Joystick & Touch

100

iPod Player
Title : Chariots Of fire Artist : Vangelis Album :STMicroelectronics Genre : Instrumental

80 60 40 20

Volume Control Using Software Mixer

Audio Utilities
5 Bands Equalizer 5dB 2.5dB 0dB 2.5dB 5dB 1KHz 5dB 2.5dB 0dB 2.5dB 5dB 3KHz 5dB 2.5dB 0dB 2.5dB 5dB 6KHz 5dB 2.5dB 0dB 2.5dB 5dB 12KHz Loudness Control Max Gain 5dB 2.5dB 0dB 2.5dB 5dB 18KHz Min Gain Quit

Song Info

Run-time Utilities While playing

Progressive Progress Bar

0:25

0:30

R Play

Mute Mute Control Home

Stop Next/Previous Fast Pause Track Repeat Forward/Backward

Either Loudness Control Or 4 to 5 bands Equalizers

* Unavailable feature

STM32 iPod/iPhone control Ref Design (2/2)

Cursor (Multi-Input) Joystick & Touch
100

Audio Player
Tags Display MP3
Title : Chariots Of fire Artist : Vangelis Album :STMicroelectronics Genre : Instrumental Year : 1981

80 60 40 20

Volume Control Using Software Mixer

Progressive Progress Bar

0:25

0:30

0 Mute

R Play Pause Browser Fast Forward/Backward Stop

Mute Control

Home Repeat

iPod control is using digital audio (USB communication) This demo is NOT using any OS, but based on interrupts priorities MP3/WAV decoding, EQ, volume control, are all done by software within the STM32

11

iPod firmware mechanism - USB data

HCC USB audio class driver

Priority

Audio post processing Transfer complete

SOF period

Mem to I2S transfer DMA transfer DMA transfer DMA transfer DMA transfer DMA transfer USB high priority tasks

IOE

Main loop

Background tasks: (iAP stack and GUI)

Volume regulation with equalizer or Loudness control when enabled.

N x SOF Period
Asynchronous input events (Joystick & touch screen)
12

iPod demo firmware memory footprint

Total Demonstration requirements: Flash: 171.21 Kbytes. Read Only data: 62.82 Kbytes. Code size: 108.39 Kbytes. RAM: 63.68 Kbytes. Heap: 28 Kbytes. Stack: 4Kbytes. Read Write Data: 31.68 Kbytes. MP3 Decoder resources: Flash: 22Kbytes Code: 15 Kbytes. RO Data: 7 Kbytes. RAM (from Heap): 12.45 Kbytes. Equalizer resources: Flash: 1.64 Kbytes Code: 1.52 Kbytes. RO Data: 0.12 Kbytes. RAM : 0.29 Kbytes. Loudness control resources: Flash: 3.16 Kbytes. Code: 1.94 Kbytes. RO Data: 1.22 Kbytes. RAM : 0.61 Kbytes. (scratch : 0.5 Kbytes, persistent 0.11 Kbytes) Mixer resources: Flash: 0.57 Kbytes. Code: 0.57 Kbytes. RO Data: 0 Kbytes. RAM : 16 bytes.

HCC stack resources: Flash: 37.34 Kbytes. Code: 37.16 Kbytes. RO Data: 0.18 Kbytes. RAM : 28.9 Kbytes. GUI: Flash: 19.90 Kbytes. Code: 13.09 Kbytes. RO Data: 6.81 Kbytes. RAM : 5.15 Kbytes. (4 Kbytes heap, 1 Kbytes Stack).

Connectivity Demo: Flash: 73.208 Kbytes. Code: 25.75 Kbytes. RO Data: 47.45 Kbytes. 8 Pictures (48 x 48 16bits). 1 picture (80 x 36 16bits). Zero filled buffer: 4 Kbytes. RAM: 2.03 Kbytes.

IAR C-standard library: FW HAL Drivers: Flash: 1.55 Kbytes. Flash: 9.75 Kbytes. Code: 1.55 Kbytes. Code: 9.71 Kbytes. RO Data: 2 bytes. RO Data: 38 bytes. RAM: 68 bytes. RAM: 60 bytes

iPod firmware only: Flash: 30Kbytes RAM: 45Kbytes

13

Back-up Technical details

iPod player(1/8)
iPod Player controls iPod internal player engine
Authenticate with the iPod. Synchronize screen contents with iPod Player (progress bar, total time, current time, song info). Sends Play, Pause, Stop, Fast forward, Fast Backward, Next track, Previous track and Repeat commands. Retrieve PCM data stream from iPod. Regulate output stream (Volume, audio post processing,...).

15

iPod player(2/8)
USB audio streaming
Two methods were implemented to handle the synchronization of data transfer from iPod to on board DAC(refer to next slides) apply additional filters (Equalizers or loudness control) and volume control

Monitor buffer indexes

Yes

Stream synchronization

Indexes too far/near

No

apply additional filters (Equalizers or loudness control) and volume control

Audio data processing

100 x frame length Circular Buffer

Write data to circular buffer

Transfer PCM to external DAC via I2S peripheral

Audio stream rendering

iPod player(3/8) :Stream synchronization

Consumer period < Producer period
USB transfer I2S transfer
1 2 1 3 2 3 4 4 5 5

Under run condition

I2S needs buffer 10 which is not ready

6 6

7 7

8 8

9 9

10 10 11

11

Accumulative error

Consumer period > Producer period

USB transfer I2S transfer

Over run condition

USB prepared buffer 10 while I2S is starting to transfer buffer 9

2 1

5 3 4

6 5

7 6

9 7

10 8

11 9

Two methods were implemented to handle the synchronization of data transfer from iPod to on board DAC.

iPod player(4/8) :Stream synchronization

Method 1 : adjust USB Start of frame (SOF) period to cancel errors and keep USB audio data streaming synchronized with I2S transfer. (default method)

This method is enabled by default, to have High end Audio Quality 1:1 ratio ( lossless samples)

iPod player(5/8) :Stream synchronization

- Method 2 : a software method to synchronize I2S and isochronous transfer. When needed add one sample at the end of the buffer or remove one sample from the end of the buffer to tune I2S transfer period. Case of adding samples
Buffer with N x samples New sample (left and right) to be added at the end of buffer

/2 /2 Buffer with N+1 x samples + ++ + Left Right

iPod player(6/8) :Stream synchronization

Case of removing samples

One Sample (left and right) to be removed at the end of buffer

N-3 N-2 N-7 N-6 N-5 N-4 N-1

N-8

W
/4

W
/4 Left
N-6

N-5

Right

iPod player(7/8)
HCC USB audio class driver Audio post processing Priority Transfer complete SOF period

Mem to I2S transfer DMA transfer DMA transfer DMA transfer DMA transfer DMA transfer USB high priority tasks IOE Main loop Background tasks: (iAP stack and GUI)
Volume regulation with equalizer or Loudness control when enabled.

N x SOF Period

Asynchronous input events (Joystick & touch screen)

21

iPod player(8/8)
USB tasks scheduling avoid mutex dead locks and concurrency on the USB bus. Priority
USB High priority tasks call HCC Audio task & HCC transfer task

USB high priority tasks USB data out

Time
HID report are sent only after higher priority tasks execution

Preemption time Blocked time Running time

Known limitation(1/5)
iPod tracks Tags: Only the Title, Artist, Album and Genre tags are extracted.
For Demo purposes.

iPod control application supports only iPod Accessory Protocol small packets.
For Demo purposes.

Audio quality may be not good with on board speaker, its due to the on board speaker quality. For a better experience please use a headphone.

23

Known limitation(2/5)
iPod control track information (stream length, Title, Artiste, Album and Genre) are retrieved when iPod sends new chapter or new track event notification. iPod control streaming : supported sampling rate are 48KHz, 44.1 KHz and 32KHz as in Apple specifications. Next/Previous track buttons allow navigation within the selected data base records.
24

Known limitation(3/5)
When MP3 player demo detect Bit rate variable (VBR) it will reset:
Time progress to 0:00 Stream Time length to 0:00 Then "VBR Detected." Message will be displayed.

Media browser: If the mass storage device contains a big number of files some of them may not be displayed.
This is related to the heap space allowed for GUI items and files/folders list. items and list are created dynamically (dynamic memory allocation).

25

Known limitation(4/5)
Maximum FIFO size set to 512 bytes: a received packet larger than 512 bytes will be improperly handled. Supported lingoes and minimal versions
Lingoes General Lingo Simple remote lingo Display remote lingo Digital audio lingo Minimal lingo version 1.02 1.00 1.02 1.02[1]

1. iPod is not switched to extended mode (please refers to iPod Accessory protocol specification for more info)

26

Known limitation(5/5)
Firmware design switch iPod to USB configuration 2 (composite USB device):
iPod mass storage feature is unavailable when connected to the STM32 docking station.

I2S and USB clock synchronization:

Synchronization method which adjust USB start of frame (SOF) packet period produce better audio quality, however this regulation method produce time lag in SOF period from standard 1ms period > 0.05% due to our I2S drift at 44.1KHz

27