Wednesday, October 5, 2016

Amlogic S905 SoC: bypassing the (not so) Secure Boot to dump the BootROM

The Amlogic S905 System-On-Chip is an ARM processor designed for video applications. It's widely used in Android/Kodi media boxes. The SoC implements the TrustZone security extensions to run a Trusted Execution Environment (TEE) that enables DRM & other security features :

Amlogic S905 System Block Diagram

The SoC contains a Secure Boot mechanism to authenticate the TEE image before loading it in TrustZone. And the first link of this Secure Boot chain is the BootROM code, stored directly in the chip.

This articles describes how to extract the BootROM code from this SoC in the Android-based Inphic Spot i7 device.

Technical documentation

Amlogic released a public version of the S905 datasheet thanks to Hardkernel. However, it's heavily redacted, and most parts regarding the Secure Boot or the TrustZone have been removed. But we can still find a lot of technical information in GPL source code packages released by Amlogic & OEMs.
For example, we can find a potential address for the BootROM code:
#define ROMBOOT_START   0xD9040000
#define ROM_SIZE        (64*1024)

Root access over the UART

We start by connecting the serial port (or UART) because this interface could provide a quick & easy access to debug messages & serial console on bootloaders and Linux kernel.
Identifying the serial port on this board is quite simple since there is a port header with labels for the pinout:
UART on Inphic Spot i7 board

We connect an USB to UART adapter to this port. Once the Linux kernel boot process is finished, we have directly access to a root shell.
We can start to explore the (Non-Secure side of the) system. For example, we can dump the partitions :
root@p200:/ # ls -l /dev/block/platform/d0074000.emmc/                      
lrwxrwxrwx root     root              2015-01-01 00:00 boot -> /dev/block/boot
lrwxrwxrwx root     root              2015-01-01 00:00 bootloader -> /dev/block/bootloader
drwxr-xr-x root     root              2015-01-01 00:00 by-num
lrwxrwxrwx root     root              2015-01-01 00:00 cache -> /dev/block/cache
lrwxrwxrwx root     root              2015-01-01 00:00 crypt -> /dev/block/crypt
lrwxrwxrwx root     root              2015-01-01 00:00 data -> /dev/block/data
lrwxrwxrwx root     root              2015-01-01 00:00 env -> /dev/block/env
lrwxrwxrwx root     root              2015-01-01 00:00 instaboot -> /dev/block/instaboot
lrwxrwxrwx root     root              2015-01-01 00:00 logo -> /dev/block/logo
lrwxrwxrwx root     root              2015-01-01 00:00 misc -> /dev/block/misc
lrwxrwxrwx root     root              2015-01-01 00:00 mmcblk0 -> /dev/block/mmcblk0
lrwxrwxrwx root     root              2015-01-01 00:00 mmcblk0boot0 -> /dev/block/mmcblk0boot0
lrwxrwxrwx root     root              2015-01-01 00:00 mmcblk0boot1 -> /dev/block/mmcblk0boot1
lrwxrwxrwx root     root              2015-01-01 00:00 mmcblk0rpmb -> /dev/block/mmcblk0rpmb
lrwxrwxrwx root     root              2015-01-01 00:00 recovery -> /dev/block/recovery
lrwxrwxrwx root     root              2015-01-01 00:00 reserved -> /dev/block/reserved
lrwxrwxrwx root     root              2015-01-01 00:00 rsv -> /dev/block/rsv
lrwxrwxrwx root     root              2015-01-01 00:00 system -> /dev/block/system
lrwxrwxrwx root     root              2015-01-01 00:00 tee -> /dev/block/tee
While the tee partition (Trusted Execution Environment) turns out to be empty, the bootloader partition contains several bootloaders. But not the BootROM because it's stored in the SoC, not the flash.

(Fail at) Reading the BootROM

Since we have root permissions and a potential memory address for the BootROM, we can try to read it directly. The provided Android ROM contains a handy debugfs interface to peek & poke physical memory from user-land:
root@p200:/ # echo "d0070000" >/sys/kernel/debug/aml_reg/paddr              
root@p200:/ # cat /sys/kernel/debug/aml_reg/paddr                            
[0xd0070000] = 0x1000254
This aml_reg driver uses the ioremap kernel function to set up an appropriate kernel page-table mapping for the requested address.

However, if we try to read the hypothetical BootROM area:
root@p200:/ # echo "d9040000" >/sys/kernel/debug/aml_reg/paddr
root@p200:/ # cat /sys/kernel/debug/aml_reg/paddr                            
[  376.546491@0] Unhandled fault: synchronous external abort (0x96000010) at 0xffffff80001aa000
[  376.549396@0] Internal error: : 96000010 [#1] PREEMPT SMP
[  376.554712@0] Modules linked in: dwc_otg dhd(O) aml_thermal(O) mali(O) aml_nftl_dev(PO)
The kernel crashes. So either the BootROM address is wrong or this memory area is set as secure.
Since we don't have other candidates for the BootROM address, let's say the BootROM area is not accessible from the Non-Secure World.

Enter the Secure World

In theory, the Secure Boot chain prevents loading unauthorized code in the Secure World.
A quick inspection of debug logs from the UART during the early phases of boot indicates that the bootloaders are based on the ARM Trusted Firmware (ATF) reference implementation.
ARM Trusted Firmware Design
We will now explore some ways to get access to Secure World.

U-Boot bootloader

Using the console over UART, we can interrupt the U-Boot boot sequence to access to the prompt. From here we can run arbitrary U-boot commands:
Hit any key to stop autoboot: 0
?       - alias for 'help'
aml_sysrecovery- Burning with amlogic format package from partition sysrecovery
amlmmc  - AMLMMC sub system
amlnf   - aml nand sub-system
amlnf_test- AMLPHYNAND sub-system
autoping- do auto ping test
autoscr - run script from memory
However the U-Boot bootloader (named BL33 in the ATF design) runs in Non-Secure mode as we can see in boot logs from the UART console:
INFO:    BL3-1: Preparing for EL3 exit to normal world
INFO:    BL3-1: Next image address = 0x1000000
INFO:    BL3-1: Next image spsr = 0x3c9

U-Boot 2015.01-ga9e9562-dirty (May 06 2016 - 03:36:02)
So at this point we are already locked out of the Secure World. Next.

SMC interface

Secure & Non-Secure Worlds can communicate through the ARM Secure Monitor Call (SMC). When a core executes the SMC instruction, it switches to Secure Monitor mode (exception level EL3).
In the ATF design, the code that runs in EL3 is named the Boot Loader stage 3-1 (BL31). We can find this image in the bootloader partition we've dumped previously. This code is highly critical for TrustZone security so we should explore it.

The open-source ATF code base in the BL31 image facilitates the analysis by reverse engineering, since we can quickly recover the ATF code structure.
Here is the list of registered services that handle SMC interrupts from Normal World:

Registered services in BL31 image

The sip_svc service is interesting because it contains several custom functions developed by Amlogic:

List of handlers in the SIP service
At first glance, functions hdcp22_sec_read_reghdcp22_sec_write_reg look promising because they are read & write primitives for the secure memory. However, they strictly restrict access to specific memory ranges:
Function hdcp22_sec_read_reg decompiled
A quick (incomplete) analysis of other functions didn't reveal any trivial flaw in parameters sanitization (arbitrary read/write bugs). Some of them are quite complex, especially the cryptographic functions, so they have not been inspected at all.

We may be able to trigger Secure World memory corruption from the Normal World if we find bugs in one of these functions, and then achieve privilege escalation to Secure World. However that would require some expert skills to actually exploit them. So let's explore another attack vector.

Bypass the Secure Boot chain

Another solution to get access to the Secure World is to break/bypass/fool/kindly ask the Secure Boot chain at one of its stage. A common attack surface of a Secure Boot chain is the loading, parsing and authentication steps of the next stage.

We don't have access to BL1 code (yet!) since it's stored in the SoC. But we have the BL2 image from the bootloader partition we have dumped previously. So we will analyze the mechanism used by BL2 to parse and authenticate the BL31 image in the hope of finding interesting flaws.

Here start the lengthy process of reverse engineering a binary without any syscall and very few strings to guide our efforts. Fortunately, the BL2 image is quite small: ~40KB. And we have some ideas to save time:

Like BL31, the BL2 image follows the ATF code logic, so reverse engineering efforts are a bit simplified: we can quickly spot main functions & structures defined in the ATF code base.

Another RE trick is to identify the memory-mapped devices accessed by functions to deduce their role.
Several address ranges of these memory areas can be found in the SoC datasheet and the GPL source code. For example, we can expect that cryptographic functions access memory registers dedicated to the hardware cryptographic engine.

Finally, we don't want to spend time reversing open source code, especially cryptographic code because the task is quite hard. And since it's also complex from the developer perspective, we can expect they used a library, which may be open source. So we looked for similarities in function prototypes, call sequence & initialization of context structures between BL2 code and few potential OSS libraries. In our case, we quickly figured out that the cryptographic code comes from the OSS PolarSSL/mbed TLS project.

Analyzing BL2 authentication routine

Once BL2 has loaded the BL3 image from the NAND, the header is parsed. We don't have any information on the header structure yet (not present in the ATF code), but we can notice it starts with a constant magic value "@AML". This helps to quickly locate the parsing code in the BL2 binary.
We combine "guessing" (i.e. looking at the BL31 header in a hex editor) and reverse engineering of the BL2 parsing code to figure out some members of the header structure:
struct aml_img_header {
  unsigned char magic[4];// "@AML"
  uint32_t total_len;
  uint8_t header_len;
  uint8_t unk_x9;
  uint8_t unk_xA;
  uint8_t unk_xB;
  uint32_t unk_xC;
  uint32_t sig_type;
  uint32_t sig_offset;
  uint32_t sig_size;
  uint32_t data_offset;
  uint32_t unk_x20;
  uint32_t cert_offset;
  uint32_t cert_size;
  uint32_t data_len;
  uint32_t unk_x30;
  uint32_t code_offset;
  uint32_t code_len;
  uint32_t unk_x3C;
} aml_img_header_t;
The header indicates that the image is split in 4 parts:
  • header : always 64 bytes
  • signature : RSA-1024, RSA-2048, RSA-4096 or SHA-256
  • cert : x509 certificate
  • code: payload
On our target device, the signature type of the BL31 image (sig_type in the header) is SHA-256. This is intriguing because a SHA-256 hash alone is not enough to provide authentication.
The following pseudocode is the simplified algorithm of the authentication routine in BL2 :
int auth_image(aml_img_header_t *img){
  validate_header(img); // checks on magic value & header length
  hash = hash_sha256(img);// hash whole image except signature
  if(img->sig_type == RSA) {
    return check_rsa_signature(img, hash)
    return memcmp(hash, (char*)img + (img->sig_offset));
We can confirm that the SHA-256 option will only hash the loaded image and compare the result against the precomputed hash in the same image. We could have imagined a more complex solution like a HMAC, but actually in this case only the integrity is checked, there is no authentication.

And even if the loaded image is signed with RSA, we can still switch the signature type to SHA-256 and regenerate the correct hash.
This issue could have been avoided if the signature type was enforced by an eFuse.

This means we can easily modify the BL31 image, the most privileged code in TrustZone.

Customizing BL31 image

In a previous section, we described the SMC function hdcp22_sec_read_reg that can read restricted ranges of secure memory from Normal World.
Time to practice our NOPing-fu to get rid of these limitations and thus obtain full access to secure memory.

Modified hdcp22_sec_read_reg function

We also need to extend the page tables because the BootROM memory area is not mapped. The MMU initialization is implemented in the ATF code base, so again this is easy to spot and analyze in the BL31 binary.
By default, the following memory regions are mapped:
Original list of memory regions mapped in BL31

We extend the size of one of them to cover the BootROM region:
Modified list of memory regions mapped in BL31

The new size of mapped region 0xD9000000 is 0x80000 so it includes the BootROM area 0xD9040000-0xD9050000.
We're almost done with BL31 modifications: we still need to update the SHA-256 hash.

aml_bootloader_tool: parse and rehash Amlogic bootloaders

This tool can parse and regenerate the SHA-256 of bootloaders contained in the bootloader partition. The source code is on GitHub.
Each bootloader is identified by an UUID, they are defined in the ATF source code. In our case, the BL31 image is entry #2:
$ ./aml_bootloader_tool ./dump/bootloader.img H 2        aa640001
fip_toc_header.serial_number:        12345678
fip_toc_header.flags:        0
fip_toc_entry.uuid:        47D4086D4CFE98469B952950CBBD5A00
fip_toc_entry.offset_address:    14000 (absolute: 0x20000)
fip_toc_entry.size:        0x11130
fip_toc_entry.flags:        0x0
magic[@0x0]:        @AML
total_len[@0x4]:    0x11130
header_len[@0x8]:    0x40
unk_xC[@0xC]:        0x5eec9094
sig_type[@0x10]:    0x0
sig_offset[@0x14]:    0x40
sig_len[@0x18]:        0x20
data_offset[@0x1c]:    0x60
unk_x20[@0x20]:        0x0
cert_offset[@0x24]:    0x60
cert_len:        0x0
data_len:        0x110d0
unk_x30[@0x30]:        0x0
code_offset[@0x34]:    0x60
code_len[@0x38]:    0x110d0
unk_x3C[@0x3C]:        0x0
signature:        263BEFAFC5A051C550D31791EC1212576BE65DB8AD365074560F0BABC076D3CA
computed_sha256:    35AD6B284EE2D6B5672DD0958592028D5BF455A6DCD1EB086D8336FB86533853
The hash of the BL31 image has been updated, we can now reflash the dump on the device:
dd if=./bootloader.img of=/dev/block/bootloader
Finally, we reboot the device to load our customized BL31 image in TrustZone.

Dumping the BootROM

The SMC system call can only be invoked from EL1 and above. So we create a simple kernel module that will perform SMC calls to our modified function hdcp22_sec_read_reg in EL3.
This quick 'n dirty hack is based on the Amlogic debugfs driver reg_access. The source code is on GitHub.
Once loaded, to initiate a SMC call, we write arguments to the file /sys/kernel/debug/aml_smc/smc. The first argument is the ID of the called SMC function (0x82000018 in the case of hdcp22_sec_read_reg). The second argument (for this specific SMC ID) is the read memory address. The result DWORD is directly printed in kernel logs (we said dirty).
# insmod ./smc_access.ko
# echo 82000018 D9040000 > /sys/kernel/debug/aml_smc/smc
[  219.092948@0] smc_access: SMC call 82000018 returns: aa1f03e0
The result aa1f03e0 is promising, it corresponds to the ARM instruction: MOV  X0, XZR
To automate the extraction of the entire BootROM memory region, we create a simple script:
$ seq -f %1.f 0xD9040000 0x4 0xD9050000 | xargs printf "echo \"82000018 %x\" > 
echo "82000018 d9040000" > /sys/kernel/debug/aml_smc/smc
echo "82000018 d9040004" > /sys/kernel/debug/aml_smc/smc
echo "82000018 d9040008" > /sys/kernel/debug/aml_smc/smc
echo "82000018 d904fff8" > /sys/kernel/debug/aml_smc/smc
echo "82000018 d904fffc" > /sys/kernel/debug/aml_smc/smc
And finally, we concate all these DWORDS into a file bootrom.bin
$ ls -l ./bootrom.bin
-rw-r--r-- 1 user user 65537 juil.  8 12:43 ./bootrom.bin
$ sha1sum bootrom.bin
bff0c7fb88b4f03e732dc7a4ce504d748d0d47dd  bootrom.bin
$ strings bootrom.bin |tail -22
auth failed, reboot...
gcc version 4.8


The S905 SoC provides hardware features to support Secure Boot, however OEMs can still choose to enable it or not. But even when Secure Boot is enforced, a flaw in the current version of Amlogic's BL2 allows to bypass it. So Trusted Execution Environment cannot be trusted. The good news is BL2 can be patched, unlike BootROM.

I would like to thank @Karnalzi for the help!

Disclosure Timeline

2016-08-08 : Flaw discovered
2016-08-08 : Amlogic & some affected OEMs contacted by email to find a security PoC
2016-08-10 : OEM #1 replies that "Amlogic doesn't provide any direct contact."
2016-08-20 : Second attempt to contact Amlogic by email
2016-09-05 : Bug report shared with Amlogic
2016-09-13 : Status update requested
2016-09-25 : Status update requested
2016-10-05 : Public disclosure

Saturday, February 20, 2016

PowerLine (PLC) support in OpenWrt for D-Link DHP-1565

D-Link 1565 is one of the few routers which integrates a PLC (Power line Communication) chipset (in this case QCA AR7400). Unfortunately, OpenWrt does not provide support for this feature yet.

This post presents configuration steps to enable PLC support in OpenWrt for this device.

Hardware configuration

By digging into D-Link GPL source code released for this device, especially function proprietary_s17_init() in file DHP1565A1_1.01b13_FOSS/src/AthSDK/platform/PLC/drivers/ethernet/phys/athrs17_phy.c, we notice the port 6 of internal switch AR8327  is related to PLC:

This port 6 is configured as a RGMII interface to communicate with PLC chipset. The following patch reproduces the same configuration in OpenWrt:
To build a custom firmware image, please see OpenWrt build system wiki page.
To flash an OpenWrt image, please see OpenWrt flashing wiki page. Don't forget to choose the factory image if you're still running the OEM firmware. If you are already using an OpenWrt firmware, you can use the sysupgrade image.

Network configuration

Once our customized OpenWrt image is flashed & booted, we are already able to see PLC bootloader probes with the swconfig command:
# swconfig dev switch0 show
Port 6:
mib: Port 6 MIB counters
RxBroad     : 2282
Rx64Byte    : 2282
RxGoodByte  : 146048
Filtered    : 2152
enable_eee: ???
pvid: 0
link: port:6 link:up speed:1000baseT full-duplex txflow rxflow
The pvid field indicates that the primary VLAN identifier of port 6 is 0. In file /etc/config/network, we add the port 6 to VLAN identifier 2, which is dedicated to WAN interface in default configuration:
config switch_vlan
option device 'switch0'
option vlan '2'
option ports '0t 5 6'
Warning: configuring PLC port into WAN interface could be a security issue: any host on WAN side can access to PLC chipset. To prevent this risk, you might need to set up a different VLAN configuration.

System configuration


Since the PLC chipset is flashless on this board, PLC firmware needs to be loaded at each boot. We can use Qualcomm Atheros Open Powerline Toolkit to manage & configure this PLC chipset.

We can cross-compile open-plc-utils with the OpenWrt toolchain.
First, we setup the following environment variables:
Then, make & make install commands are enough to build it.
Finally, we copy these freshly built binaries to the device in /overlay/upper/open-plc-utils/.

To do a quick test, we can try the Request Information command:
# /overlay/upper/open-plc-utils/amptool -i eth0.2 -Iar
eth0.2 00:B0:52:00:00:01 Request Version Information
eth0.2 00:B0:52:00:00:01 AR7400 BootLoader
eth0.2 00:B0:52:00:00:01 Fetch Device Attributes
eth0.2 00:B0:52:00:00:01 Device Identity
The line "AR7400 Bootloader" indicates that PLC chipset is ready to load a firmware.

PLC firmware

The PLC firmware blob can be found in the original firmware image. Using Binwalk, we extract the content of the /plc directory from the original rootfs, and copy it to /overlay/upper/plc/.
The NVM file is the firmware blob, and PIB files are configuration files: ceb stands for Europe, and na for North America.

The ampboot tool allows to load the firmware:
# /open-plc-utils/ampboot -i eth0.2 -P /plc/ -N /plc/plc.nvm
eth0.2 00:B0:52:00:00:01 Write /plc/plc.nvm (0) (00000040:17256)
eth0.2 00:B0:52:00:00:01 Start /plc/plc.nvm (0) (000000C0)
eth0.2 00:B0:52:00:00:01 Write /plc/ (0) (00200000:16352)
eth0.2 00:B0:52:00:00:01 Write /plc/plc.nvm (3) (00341194:423788)
eth0.2 00:B0:52:00:00:01 Start /plc/plc.nvm (3) (00341A88)
eth0.2 00:11:22:33:44:55 INT7400-MAC-5-2-5203-01-913-20110713-FINAL-B is running
By default, the HomeplugAV key is used. You can get more information with the amptool command used earlier. This can also be used to discover/join/leave a network, modify the NMK key, etc...

init.d script

To load the PLC firmware at each boot, create the following init script /etc/rc.d/S25amphost :
#!/bin/sh /etc/rc.common
#start right after network
start_service() {
procd_set_param command /open-plc-utils/amphost -i eth0.2 -P /plc/ -N /plc/plc.nvm
procd_set_param respawn
procd_add_reload_trigger "amphost"
Then, to enable this script at startup:
/etc/init.d/amphost enable
Et voilà !

Sunday, February 1, 2015

Analysis of Nexus 5 Monitor mode

This article will first describe how to locate the Monitor mode code in Nexus 5 firmware (hammerhead-ktu84p-factory-35ea0277, bootloader-hammerhead-hhz11k : c32f8bec310c659c1296739b00c6a8ac). Then, we will try to understand what it does (its functionalities). Finally, you will have to find bugs by yourself because I didn't find far !

Note: Terms (Non-)Secure world & (Non-)Secure state are used as synonyms. Term Normal world is also used as synonym of Non-Secure world.

I. Quick introduction to ARM Security Extensions

"The Security Extensions define two security states: Secure state and Non-secure state. All instruction execution takes place either in Secure state or in Non-secure state.[...] The Security Extensions also define an additional processor mode, Monitor mode, that provides a bridge between software running in Non-secure state and software running in Secure state."
"The Secure Monitor Call exception is implemented only as part of the Security Extensions. The Secure Monitor Call instruction, SMC , requests a Secure Monitor function, causing the processor to enter Monitor mode."
"When an exception is taken, processor execution is forced to an address that corresponds to the type of exception. This address is called the exception vector for that exception. A set of exception vectors comprises eight consecutive word-aligned memory addresses, starting at an exception base address. These eight vectors form a vector table."
-- ARM Architecture Reference Manual ARMv7-A

II. OpenSource TrustZone examples

Trusted Execution Environment (TEE) is the "small" secure kernel executed in Secure state. The Monitor code is part of the TEE code.
To get an idea of how the Monitor code works, we can take a look at two TrustZone examples:
After studying these code samples, we can clearly distinguish two parts in Monitor code:
Monitor mode initialization: called once, at TEE initialization time.
In this code, we can notice two specific instructions :
  • Monitor Vector Base Address Register (MVBAR) setup: MVBAR contains the Monitor vector table address. Both samples use the same instructions to setup MVBAR :
  •     MCR     p15, 0, $RX,c12,c0, 1
    where $RX is a pointer to the monitor mode's vector table.

  • SP register setup: the Monitor mode stack address is set into SP register. This register is banked, which means this value will be automatically restored next time the processor enters in Monitor mode.
Exception vectors: called when an exception is taken to Monitor mode.
Both samples implement a simple Secure Monitor Call (SMC) handler that switches between the normal and secure worlds when a SMC call is made. As SMC handler is an entry point to the Secure state, it would be interesting to analyze it in Nexus 5 firmware.

III. Extracting Nexus 5 firmware

We know that the Monitor code may be embedded into the TEE image. In the case of Nexus 5, this image can be extracted from stock ROM.
Once downloaded, we use a small tool to unpack bootloader-hammerhead-hhz11k.img file. One of extracted files is an ELF ARM binary named "tz".

IV. Nexus 5 Monitor mode code

To analyze the Nexus 5 TrustZone binary, we can use IDA Demo 6.6.
Given that setting up MVBAR is very specific to the monitor mode's initialization code, we use it to locate the Monitor mode's initialization code in Nexus 5 TrustZone binary.
Using IDA regex search in code disassembly, we look for the instruction used to write MVBAR :
MCR[[:space:]]+p15, 0, [^,]+,c12,c0, 1
This search returns only 3 occurrences, and one of them also sets the SP register. These instructions are expected to be found in Monitor mode initialization code.

IV.1. Monitor mode initialization function

Here's the disassembly of the Monitor mode initialization code :
LOAD:FE80DB4C init_monitor LOAD:FE80DB4C MSR CPSR_c, #0xD6 ; switch to Monitor mode LOAD:FE80DB50 LDR R0, =monitor_vector_table ; load monitor vector table ptr into R0 LOAD:FE80DB54 MCR p15, 0, R0,c12,c0, 1 ; write R0 to MVBAR LOAD:FE80DB58 BL sub_FE80DB88 ; initialize Non-Secure world LOAD:FE80DB5C LDR SP, =0xFE82B700 LOAD:FE80DB60 MRC p15, 0, R0,c0,c0, 5 ; write MPIDR value to R0 LOAD:FE80DB64 AND R0, R0, #0xFF ; keep Affinity level 0 : current virtual CPU id LOAD:FE80DB68 MOV R1, #0x200 LOAD:FE80DB6C MUL R1, R1, R0 ; compute stack offset for current vCPU LOAD:FE80DB70 SUB SP, SP, R1 ; setup Monitor stack register SP LOAD:FE80DB74 MOV R0, #0b100 LOAD:FE80DB78 MCR p15, 0, R0,c1,c1, 0 ; set FIQ flag in SCR register LOAD:FE80DB7C ISB SY ; flush the pipeline in the processor LOAD:FE80DB80 MSR CPSR_c, #0xD3 ; switch to Supervisor mode LOAD:FE80DB84 BX LR LOAD:FE80DB84 ; End of function init_monitor

We will now proceed to a detailed analysis of each step.
IV.1.A Switch to Monitor mode
MSR instruction moves an immediate value (here 0xD6) to a Special register (here CPSR_c).
LOAD:FE80DB4C MSR CPSR_c, #0xD6 ; switch to Monitor mode
The Current Program Status Register (CPSR) holds processor status and control information. CPSR with "_c" suffix enables writing of bits<0:7> of CPSR (ARM Ref. B9.3.11). This bitfield controls the processor mode and exception masks.
We can use a simple IDAPython script to replace the immediate value 0xD6 with symbols documented in ARM Ref. (B1-1148) :
Thus, the instruction becomes:

This instruction switches the processor to Monitor mode. It also sets CPSR.F and CPSR.I bits to mask FIQ and IRQ exceptions, meaning they cannot be taken.
IV.1.B Setup MVBAR
The Move to Coprocessor from ARM core register instruction (MCR) passes the value of an ARM core register (here R0) to a coprocessor (here CP15).
LOAD:FE80DB50 LDR R0, =monitor_vector_table ; load monitor vector table ptr into R0 LOAD:FE80DB54 MCR p15, 0, R0,c12,c0, 1 ; write R0 to MVBAR

CP15 c12 register is present on an ARMv7-A implementation that includes Security Extensions. This instruction writes R0 value to MVBAR. R0 contains a pointer to Monitor vector table. We will describe this table later.
IV.1.C Initialize Non-Secure world
The function sub_FE80DB88 is called to initialize the Non-Secure world context:
LOAD:FE80DB88 sub_FE80DB88 LOAD:FE80DB88 MRC p15, 0, R1,c1,c0, 0 ; read Secure SCTLR LOAD:FE80DB8C MOV R0, #SCR_NS OR SCR_FW OR SCR_AW ; #0x31
LOAD:FE80DB90 MCR p15, 0, R0,c1,c1, 0 ; switch to Non-Secure (NS) state LOAD:FE80DB94 ISB SY LOAD:FE80DB98 MCR p15, 0, R1,c1,c0, 0 ; write Secure SCTLR value to NS SCTLR LOAD:FE80DB9C MOV R0, #0 LOAD:FE80DBA0 MCR p15, 2, R0,c0,c0, 0 ; clear CSSELR LOAD:FE80DBA4 MCR p15, 0, R0,c2,c0, 0 ; clear TTBR0 LOAD:FE80DBA8 MCR p15, 0, R0,c2,c0, 1 ; clear TTBR1 LOAD:FE80DBAC MCR p15, 0, R0,c2,c0, 2 ; clear TTBCR LOAD:FE80DBB0 MCR p15, 0, R0,c3,c0, 0 ; clear DACR LOAD:FE80DBB4 MCR p15, 0, R0,c5,c0, 0 ; clear DFSR LOAD:FE80DBB8 MCR p15, 0, R0,c5,c0, 1 ; clear IFSR LOAD:FE80DBBC MCR p15, 0, R0,c5,c1, 0 ; clear ADFSR LOAD:FE80DBC0 MCR p15, 0, R0,c5,c1, 1 ; clear AIFSR LOAD:FE80DBC4 MCR p15, 0, R0,c6,c0, 0 ; clear DFAR LOAD:FE80DBC8 MCR p15, 0, R0,c6,c0, 2 ; clear IFAR LOAD:FE80DBCC MCR p15, 0, R0,c7,c4, 0 ; clear PAR LOAD:FE80DBD0 MCR p15, 0, R0,c10,c2, 0 ; clear PRRR LOAD:FE80DBD4 MCR p15, 0, R0,c10,c2, 1 ; clear NMRR LOAD:FE80DBD8 MCR p15, 0, R0,c10,c4, 0 ; clear "MMUDMTR" ? LOAD:FE80DBDC MCR p15, 0, R0,c10,c4, 1 ; clear "MMUDCPR" ? LOAD:FE80DBE0 LDR R1, =dword_FE82B8CC ; load Non-Secure VBAR ptr to R1 LOAD:FE80DBE4 LDR R0, [R1] LOAD:FE80DBE8 MCR p15, 0, R0,c12,c0, 0 ; write Non-Secure VBAR LOAD:FE80DBEC MOV R0, #0 LOAD:FE80DBF0 STR R0, [R1] ; clear Non-Secure VBAR ptr LOAD:FE80DBF4 MCR p15, 0, R0,c13,c0, 0 ; clear FCSEIDR LOAD:FE80DBF8 MCR p15, 0, R0,c13,c0, 1 ; clear CONTEXTIDR LOAD:FE80DBFC MCR p15, 0, R0,c13,c0, 2 ; clear TPIDRURW LOAD:FE80DC00 MCR p15, 0, R0,c13,c0, 3 ; clear TPIDRURO LOAD:FE80DC04 MCR p15, 0, R0,c13,c0, 4 ; clear TPIDRPRW LOAD:FE80DC08 MOV R0, #SCR_FW OR SCR_AW ; #0x30
LOAD:FE80DC0C MCR p15, 0, R0,c1,c1, 0 ; switch back to Secure state LOAD:FE80DC10 ISB SY LOAD:FE80DC14 BX LR LOAD:FE80DC14 ; End of function sub_FE80DB88
First, the security state is switched to Non-Secure. Then, the coprocessor registers banked in both security states (ARM Ref. Banked system control registers) are initialized to zero. Finally, the security state is switched back to Secure.
IV.1.D Setup SP register
On ARMv7-A, Multiprocessor Affinity Register (MPIDR) holds the processor identification information. In this register, bits<0:7> are the affinity level 0 (Aff0). This number represents the current CPU id. Here, this id is used to compute the stack address of current CPU, which is then stored into SP register. The stack size for each CPU is 0x200 bytes.
LOAD:FE80DB5C LDR SP, =0xFE82B700 LOAD:FE80DB60 MRC p15, 0, R0,c0,c0, 5 ; write MPIDR value to R0 LOAD:FE80DB64 AND R0, R0, #0xFF ; keep Affinity level 0 : current virtual CPU id LOAD:FE80DB68 MOV R1, #0x200 LOAD:FE80DB6C MUL R1, R1, R0 ; compute stack offset for current vCPU LOAD:FE80DB70 SUB SP, SP, R1 ; setup Monitor stack register SP

IV.1.E Route FIQ exceptions to Monitor mode
CP15 c1 register is present on an ARMv7-A implementation that includes Security Extensions. This instruction sets bit<2> (0x4) in Secure Configuration Register (SCR), which means FIQ exceptions are now taken to Monitor mode.
LOAD:FE80DB74 MOV R0, #0b100 ; SCR.FIQ LOAD:FE80DB78 MCR p15, 0, R0,c1,c1, 0 ; set FIQ flag in SCR register LOAD:FE80DB7C ISB SY ; flush the pipeline in the processor

We can also notice that bit<0> (SCR.NS : Non-Secure) is not set, meaning current execution state is Secure.
IV.1.F Switch back to Supervisor mode
This instruction switches the processor to Supervisor mode, and sets FIQ & IRQ mask bits.

Monitor mode setup is now complete. Monitor code can then be entered through its exception vector table.

IV.2. Monitor Exception Vector Table

The Monitor exception vector table defines exception vectors to handle exceptions taken to Monitor Mode.
Its structure is described in ARM Ref. (B1-1167) :
The vector table entries
Thanks to the Monitor initialization code, we know the address of Nexus 5's Monitor exception vector table:
LOAD:FE80CEE0 monitor_vector_table LOAD:FE80CEE0 B dead_loop ; not used LOAD:FE80CEE4 ; --------------------------------------------------------------------------- LOAD:FE80CEE4 B dead_loop ; not used LOAD:FE80CEE8 ; --------------------------------------------------------------------------- LOAD:FE80CEE8 B smc_handler ; Secure Monitor Call LOAD:FE80CEEC ; --------------------------------------------------------------------------- LOAD:FE80CEEC B dead_loop ; Prefetch Abort LOAD:FE80CEF0 ; --------------------------------------------------------------------------- LOAD:FE80CEF0 B dead_loop ; Data Abort LOAD:FE80CEF4 ; --------------------------------------------------------------------------- LOAD:FE80CEF4 B dead_loop ; not used LOAD:FE80CEF8 ; --------------------------------------------------------------------------- LOAD:FE80CEF8 B sub_FE80CF24 ; IRQ interrupt LOAD:FE80CEFC ; --------------------------------------------------------------------------- LOAD:FE80CEFC B sub_FE80CFB4 ; FIQ interrupt LOAD:FE80CEFC ; End of function monitor_vector_table

We can see that 3 exception handlers are configured: SMC, FIQ, IRQ. Others are dead loops.

IV.3. Secure Monitor Call handler function

HLOS (non-Secure state) can call the TrustZone API (Secure state) using the SMC instruction to trigger a Secure Monitor Call exception. This exception is taken to the Monitor mode, which switches the processor to Secure Supervisor mode to proceed the call. When called TrustZone function returns, a second SMC exception is triggered, so the processor enters Monitor mode again. Finally, the Monitor mode returns results to the calling function (Non-Secure state).
The Monitor mode acts as a bridge between Non-Secure state and Secure state. It's designed to handle calls initiated from the Non-Secure state only.

The exception vector dedicated to SMC exceptions is a pointer to a function at offset 0x08 in Monitor Exception Vector Table.
In this function, which will be named SMC handler, the very first instruction checks if an exception occurred in Secure or Non-Secure state (When the processor is in Monitor mode, the processor is in Secure state regardless of the value of the SCR.NS bit).
LOAD:FE80D028 smc_handler LOAD:FE80D028 LOAD:FE80D028 varg_r0 = -0x10 LOAD:FE80D028 varg_r1 = -0xC LOAD:FE80D028 varg_r2 = -8 LOAD:FE80D028 varg_r3 = -4 LOAD:FE80D028 LOAD:FE80D028 STMFD SP!, {R0-R3} LOAD:FE80D02C MRC p15, 0, R0,c1,c1, 0 ; read SCR register LOAD:FE80D030 TST R0, #1 ; test SCR.NS bit LOAD:FE80D034 BEQ loc_FE80D210 ; jump if SCR.NS==0

When an exception is taken to the Monitor mode, CPSR.{A,I, F} bits are set to 1, meaning Abort, IRQ and FIQ exceptions can no longer be taken.

IV.3.A. Call to Secure World

If SCR.NS bit is set, it means the Non-Secure world wants to call the Secure world. We will now analyze the operations performed by the SMC handler until the exception return to the Secure world.
IV.3.A.a Setup current security state
This first step configures the Secure Configuration Register (SCR). Bits<1:3> (SCR.IRQ || SCR.FIQ || SCR.EA) are set to route IRQ, FIQ, and External Abort exceptions to Monitor mode. But the Non-Secure bit<0> is not set. So, this core will still be in the Secure state if it exits Monitor mode.
LOAD:FE80D038 MOV R0, #SCR_IRQ OR SCR_FIQ OR SCR_EA ; 0b1110 LOAD:FE80D03C MCR p15, 0, R0,c1,c1, 0 ; write SCR with SCR.NS==0 LOAD:FE80D040 ISB SY ; Instruction Synchronization Barrier LOAD:FE80D040 ; flushes the pipeline in the processor
IV.3.A.b Monitor calls
On a HLOS like Android, SMC exceptions are triggered by the Secure Channel Manager (SCM), implemented in Linux kernel.
A quick look at its source code tells us {R0-R3} registers hold arguments of SMC calls. We also learn that R0 is a bitfield that can be defined by the following macro:
#define SCM_ATOMIC(svc, cmd, n) (((((svc) << 10)|((cmd) & 0x3ff)) << 12) | \
(n & 0xf))
With svc the service identifier, cmd the command identifier, and n the argument count of the SMC call.

In SMC handler, R0 value is first shifted right by 12. Based on the SCM_ATOMIC macro definition, resulting R0 value represents a service identifier svc and a command identifier cmd defined as ((svc) << 10)|((cmd) & 0x3ff).
Then R0 value is tested against several immediate values. For each case, a specific function is called if values match.

As Linux kernel itself initiates a lot of SMC calls, we explore Linux sources to enumerate service and command identifiers passed to SMC calls. Thereby, we will get more information on corresponding functions without reversing them.
Immediate valueService ID (imm>>10)Command ID (imm&0x3ff)Function description
0x402SCM_SVC_BOOTSCM_CMD_TERMINATE_PCPut current core in low power state
0xC05SCM_SVC_UTILCACHE_BUFFER_DUMP_COMMAND_IDDump the L1 and L2 caches on panic
0x404SCM_SVC_BOOT4Dummy function, returns to Non-Secure world
0x1401SCM_SVC_IOSCM_IO_READDummy function, returns to Non-Secure world
0x1402SCM_SVC_IOSCM_IO_WRITEDummy function, returns to Non-Secure world
0x3404SCM_SVC_DCVSDCVS_CMD_EVENTHandle some Dynamic Clock and Voltage Scaling (DCVS) See also event definitions
0x1403SCM_SVC_IOTZ_RESET_IDRelated to GPU power management
0x1404SCM_SVC_IOTZ_UPDATE_IDRelated to GPU power management
0x2401SCM_SVC_PWRSCM_IO_DISABLE_PMIC_ARBITER"Force the SPMI PMIC arbiter to shutdown so that no more SPMI transactions are sent from the MSM to the PMIC."
All these functions have the same epilogue:
LOAD:FE80D738 MOV R3, #SCR_NS OR SCR_FIQ OR SCR_AW ; 0b100101 LOAD:FE80D73C MCR p15, 0, R3,c1,c1, 0 ; write SCR : switch to Non-Secure state LOAD:FE80D740 ISB SY LOAD:FE80D744 MOV R3, #0 ; clear R3 to avoid leak LOAD:FE80D748 MOVS PC, LR ; restore Non-Secure PC & CPSR from LR_mon & SPSR_mon

These instructions switch the processor to Non-Secure state and restore PC & CPSR to perform an exception return.

So SMC calls associated with these specific command/service IDs are kind of "Monitor calls", entirely handled in Monitor mode.

But if R0 value does not match these IDs, the execution continues in Monitor mode.
IV.3.A.c TrustZone lock
If the call has not been handled yet, Monitor code tries to acquire a lock to ensure that only one core at a time enters in TrustZone.
First, current CPU id is retrieved from MPIDR. Then, this value is incremented (because 0 means not locked) and used as lock value.
LOAD:FE80D0E0 LDR R1, =tz_lock LOAD:FE80D0E4 MRC p15, 0, R2,c0,c0, 5 ; read MPIDR register LOAD:FE80D0E8 AND R2, R2, #0xFF ; extract Aff0 from MPIDR LOAD:FE80D0EC ADD R2, R2, #1 LOAD:FE80D0F0 LOAD:FE80D0F0 loc_FE80D0F0 ; CODE XREF: smc_handler+D8j LOAD:FE80D0F0 LDREX R0, [R1] ; read current tz_lock value LOAD:FE80D0F4 CMP R0, #0 ; test if TrustZone is locked LOAD:FE80D0F8 STREXEQ R0, R2, [R1] ; if not locked, try to lock TrustZone LOAD:FE80D0FC CMPEQ R0, #0 ; test if TrustZone is now locked LOAD:FE80D100 BNE loc_FE80D0F0 ; retry if TrustZone is still not locked LOAD:FE80D104 DMB SY ; Data Memory Barrier acts as a memory barrier

Then, it tries to acquire the TrustZone lock. This implementation is very similar to the example provided in ARM Ref. (D7.3.1 Acquiring a lock).
It relies on synchronization primitives (LDREX/STREX) to support exclusive accesses to memory shared between cores.
Once the lock is acquired, the current core is the only one running in TrustZone, and the execution can continue.
IV.3.A.d Pre-exception status
LR_mon and SPSR_mon are both banked registers. Their values are generated by the exception entry. LR_mon contains the return address in Non-Secure world (right after the SMC instruction). The purpose of SPSR_mon is to record the pre-exception value of the CPSR.
LOAD:FE80D108 LDR R0, =NS_core_status ; secure area to store Non-Secure (NS) status LOAD:FE80D10C MOV R1, LR ; read NS return address (LR_mon) LOAD:FE80D110 MRS R2, SPSR ; read NS CPSR (SPSR_mon) LOAD:FE80D114 STMIA R0, {R1,R2} ; write LR_mon & SPSR_mon

These two registers are saved in Secure memory to be restored later on exception return.
IV.3.A.e IRQ interruption flag
Then a DWORD at a static address is unconditionally cleared:
LOAD:FE80D118 LDR R1, =tz_irq_interrupted LOAD:FE80D11C MOV R0, #0 LOAD:FE80D120 STR R0, [R1] ; clear tz_irq_interrupted value

By looking at cross-references, we notice this DWORD is set to 1 in the IRQ handler of Monitor mode. But in both handlers (SMC & IRQ), when an exception returns to the Non-Secure world, the returned value (in R0) is set to 1 if this DWORD is not null.

Futhermore, we can have a look at how SCM interprets the value returned by a SMC call:

do {
    asm volatile(
        __asmeq("%0", "r0")
        __asmeq("%1", "r0")
        __asmeq("%2", "r1")
        __asmeq("%3", "r2")
        ".arch_extension sec\n"
        "smc #0 @ switch to secure world\n"
        : "=r" (r0)
        : "r" (r0), "r" (r1), "r" (r2)
        : "r3");
} while (r0 == SCM_INTERRUPTED);

SCM will reiterate each SMC call while the returned value is 1.

We can deduce that this DWORD indicates if the exception return is due to an IRQ interrupt. TrustZone Whitepaper (3.3.3 Secure interrupts) says ARM recommends the use of IRQ as a Normal world interrupt source. That's why IRQ interrupts are handled in the Normal world.
IV.3.A.f Configure Secure world MMU
Next block of instructions modifies the translation table of Secure MMU (ARM Ref. B3.1 About the VMSA) if two conditions are met:
LOAD:FE80D124 MRC p15, 0, R0,c0,c0, 5 ; read MPIDR register LOAD:FE80D128 AND R0, R0, #0xFF ; extract Aff0 from MPIDR LOAD:FE80D12C CMP R0, #0 LOAD:FE80D130 BNE loc_FE80D164 ; jump if current core != CPU0 LOAD:FE80D134 LDR R0, =tz_ext_elf_loaded ; read external ELF status LOAD:FE80D138 LDR R0, [R0] LOAD:FE80D13C CMP R0, #0 LOAD:FE80D140 BEQ loc_FE80D164 ; jump if no external ELF loaded LOAD:FE80D144 LDR R0, =tz_ext_elf_ttbr0 ; read TTBR0 ptr for external ELF LOAD:FE80D148 LDR R0, [R0] LOAD:FE80D14C DSB SY LOAD:FE80D150 MCR p15, 0, R0,c2,c0, 0 ; write new TTBR0 LOAD:FE80D154 ISB SY LOAD:FE80D158 MCR p15, 0, R0,c8,c7, 0 ; flush TLBs LOAD:FE80D15C DSB SY LOAD:FE80D160 ISB SY

First, it checks if the current core is CPU0.
Then, it checks if a DWORD is not null. By looking at cross-references, we notice that this DWORD is modified in SCM handler of QSEOS_LOAD_EXTERNAL_ELF_COMMAND call (not part of the Monitor code). This SCM call is made by qseecom_load_external_elf() function in the QSEECOM Linux driver. This function allows the HLOS to load an external ELF binary into the Secure World. We can remark that this function first ensures to run on CPU0.
static int qseecom_load_external_elf(struct qseecom_dev_handle *data, void __user *argp)
/* SCM_CALL tied to Core0 */
mask = CPU_MASK_CPU0;
set_cpu_ret = set_cpus_allowed_ptr(current, &mask);
You can also refer to TrustZone Whitepaper to learn more about "Secure World processor affinity" on multiprocessor systems.

Finally, if those checks are successful, the Translation Table Base Register 0 (TTBR0) is modified, and data & instruction TLBs are both flushed. TTBR0 holds the physical address of the first-level translation table used by the Secure MMU to perform table translation walks.

This block of instructions will configure the MMU to create a dedicated address space in the Secure World if an external ELF is loaded on CPU0.
IV.3.A.g Context switching
Before switching to Secure World, Normal World context is saved into Secure memory (TrustZone Whitepaper, 5.3.1 Context switching). It includes :
  • General purpose registers (R0-R12)
  • Banked registers SPSR, SP and LR of each mode IRQ, SVC, ABT, UND.
  • Banked registers SPSR, R8, R9, R10, R11, R12, SP and LR of FIQ mode.
LOAD:FE80D168 MOV LR, SP ; save Monitor stack address LOAD:FE80D16C LDR SP, =NS_core_context ; secure area to store Non-Secure context LOAD:FE80D170 STMFD SP, {R0-LR}^ LOAD:FE80D174 MOV R4, SP LOAD:FE80D178 CPS #CPSR_MODE_IRQ ; switch to IRQ mode LOAD:FE80D17C MRS R12, SPSR ; read SPSR_irq LOAD:FE80D180 STMIA R4!, {R12-LR} LOAD:FE80D184 CPS #CPSR_MODE_SVC ; switch to Supervisor mode LOAD:FE80D188 MRS R12, SPSR ; read SPSR_svc LOAD:FE80D18C STMIA R4!, {R12-LR} LOAD:FE80D190 CPS #CPSR_MODE_ABT ; switch to Abort mode LOAD:FE80D194 MRS R12, SPSR ; read SPSR_abt LOAD:FE80D198 STMIA R4!, {R12-LR} LOAD:FE80D19C CPS #CPSR_MODE_UND ; switch to Undefined mode LOAD:FE80D1A0 MRS R12, SPSR ; read SPSR_und LOAD:FE80D1A4 STMIA R4!, {R12-LR} LOAD:FE80D1A8 CPS #CPSR_MODE_FIQ ; switch to FIQ mode LOAD:FE80D1AC MRS R7, SPSR ; read SPSR_fiq LOAD:FE80D1B0 STMIA R4, {R7-LR} LOAD:FE80D1B4 CPS #CPSR_MODE_MON ; switch back to Monitor mode

Because the current security state is Secure (SCR.NS == 0), CPS instructions can be used to switch to each mode before finally switching back to Monitor mode. MRS instruction reads a Special Register (like SPSR) and writes it to a general purpose register.
Later, this saved context will be restored when the processor switches back to the Normal World.

Then, Secure World context is restored from a previous context switch (Secure to Normal World).
LOAD:FE80D1B8 LDR SP, =S_core_context ; secure area where previous Secure context is stored LOAD:FE80D1BC MOV R1, SP LOAD:FE80D1C0 CPS #CPSR_MODE_IRQ ; switch to IRQ mode LOAD:FE80D1C4 LDMIA R1!, {R12-LR} LOAD:FE80D1C8 MSR SPSR_cxsf, R12 ; write SPSR_irq LOAD:FE80D1CC CPS #CPSR_MODE_SVC ; switch to Supervisor mode LOAD:FE80D1D0 LDMIA R1!, {R12-LR} LOAD:FE80D1D4 MSR SPSR_cxsf, R12 ; write SPSR_svc LOAD:FE80D1D8 CPS #CPSR_MODE_ABT ; switch to Abort mode LOAD:FE80D1DC LDMIA R1!, {R12-LR} LOAD:FE80D1E0 MSR SPSR_cxsf, R12 ; write SPSR_abt LOAD:FE80D1E4 CPS #CPSR_MODE_UND ; switch to Undefined mode LOAD:FE80D1E8 LDMIA R1!, {R12-LR} LOAD:FE80D1EC MSR SPSR_cxsf, R12 ; write SPSR_und LOAD:FE80D1F0 CPS #CPSR_MODE_FIQ ; switch to FIQ mode LOAD:FE80D1F4 LDMIA R1, {R7-LR} LOAD:FE80D1F8 MSR SPSR_cxsf, R7 ; write SPSR_fiq LOAD:FE80D1FC CPS #CPSR_MODE_MON ; switch back to Monitor mode LOAD:FE80D200 LDMEA SP, {R0-LR}^
IV.3.A.f Exception return to Secure world
Finally, the Monitor stack address is restored, and a Return From Exception (RFE) instruction loads the LR and the CPSR of interrupted Secure World from a specific address in Secure memory.
LOAD:FE80D204 MOV SP, LR ; restore Monitor stack address LOAD:FE80D208 LDR LR, =S_core_status ; ptr to previously-saved Secure LR & CPSR LOAD:FE80D20C RFEIA LR ; Return From Exception to Secure World

IV.3.B Return to Non-Secure World

In the case where SCR.NS is not set, the Secure world returns results to calling function in Non-Secure world.
A lot of operations here are similar to those previously described in the "Call to Secure World" section.
IV.3.B.a Pre-exception status
First, LR_mon & SPSR_mon registers are saved in Secure memory to be restored next time the TrustZone is entered. LR_mon contains the return address in Secure world (right after the SMC instruction). The purpose of SPSR_mon is to record the pre-exception value of the CPSR.
LOAD:FE80D210 LDR R0, =S_core_status ; secure area to store Secure status LOAD:FE80D214 MOV R1, LR ; read Secure return address (LR_mon) LOAD:FE80D218 MRS R2, SPSR ; read Secure CPSR (SPSR_mon) LOAD:FE80D21C STMIA R0, {R1,R2} ; write LR_mon & SPSR_mon
IV.3.B.b Context switching
Then, the Secure World context is saved, and the Normal World context is restored from a previous context switch (Normal to Secure World).
LOAD:FE80D224 MOV LR, SP ; save Monitor stack address LOAD:FE80D228 LDR SP, =S_core_context ; secure area to store Secure context LOAD:FE80D22C STMFD SP, {R0-LR}^ LOAD:FE80D230 MOV R4, SP LOAD:FE80D234 CPS #CPSR_MODE_IRQ ; switch to IRQ mode LOAD:FE80D238 MRS R12, SPSR ; read SPSR_irq LOAD:FE80D23C STMIA R4!, {R12-LR} LOAD:FE80D240 CPS #CPSR_MODE_SVC ; switch to Supervisor mode LOAD:FE80D244 MRS R12, SPSR ; read SPSR_svc LOAD:FE80D248 STMIA R4!, {R12-LR} LOAD:FE80D24C CPS #CPSR_MODE_ABT ; switch to Abort mode LOAD:FE80D250 MRS R12, SPSR ; read SPSR_abt LOAD:FE80D254 STMIA R4!, {R12-LR} LOAD:FE80D258 CPS #CPSR_MODE_UND ; switch to Undefined mode LOAD:FE80D25C MRS R12, SPSR ; read SPSR_und LOAD:FE80D260 STMIA R4!, {R12-LR} LOAD:FE80D264 CPS #CPSR_MODE_FIQ ; switch to FIQ mode LOAD:FE80D268 MRS R7, SPSR ; read SPSR_fiq LOAD:FE80D26C STMIA R4, {R7-LR} LOAD:FE80D270 CPS #CPSR_MODE_MON ; switch back to Monitor mode LOAD:FE80D274 SUB SP, SP, #0x94 ; NS_core_context = SP (S_core_context) - 0x94 LOAD:FE80D278 MOV R1, SP ; secure area where previous Non-Secure context is stored LOAD:FE80D27C CPS #CPSR_MODE_IRQ ; switch to IRQ mode LOAD:FE80D280 LDMIA R1!, {R12-LR} LOAD:FE80D284 MSR SPSR_cxsf, R12 ; write SPSR_irq LOAD:FE80D288 CPS #CPSR_MODE_SVC ; switch to Supervisor mode LOAD:FE80D28C LDMIA R1!, {R12-LR} LOAD:FE80D290 MSR SPSR_cxsf, R12 ; write SPSR_svc LOAD:FE80D294 CPS #CPSR_MODE_ABT ; switch to Abort mode LOAD:FE80D298 LDMIA R1!, {R12-LR} LOAD:FE80D29C MSR SPSR_cxsf, R12 ; write SPSR_abt LOAD:FE80D2A0 CPS #CPSR_MODE_UND ; switch to Undefined mode LOAD:FE80D2A4 LDMIA R1!, {R12-LR} LOAD:FE80D2A8 MSR SPSR_cxsf, R12 ; write SPSR_und LOAD:FE80D2AC CPS #CPSR_MODE_FIQ ; switch to FIQ mode LOAD:FE80D2B0 LDMIA R1, {R7-LR} LOAD:FE80D2B4 MSR SPSR_cxsf, R7 ; write SPSR_fiq LOAD:FE80D2B8 CPS #CPSR_MODE_MON ; switch back to Monitor mode LOAD:FE80D2BC LDMEA SP, {R0-LR}^ LOAD:FE80D2C0 MOV SP, LR ; restore Monitor stack address
IV.3.B.c IRQ interrupt flag
Next instructions check the DWORD value which indicates that an IRQ interrupt occurred. If this flag is set, the return value is set to 1 in R0.
LOAD:FE80D2C4 LDR R3, =tz_irq_interrupted LOAD:FE80D2C8 LDR R2, [R3] LOAD:FE80D2CC CMP R2, #0 ; if an IRQ interrupt occurred LOAD:FE80D2D0 MOVNE R0, #1 ; then set return value to 1

This may seem pointless in the context of the SMC handler. But actually this part of code is also used by the IRQ handler to return to the Normal World.
IV.3.B.d Non-secure CPSR & LR
Then CPSR and LR from previously interrupted Non-Secure state are written to SPSR_mon and LR_mon.
LOAD:FE80D2D4 LDR LR, =NS_core_status ; ptr to previously-saved Non-Secure LR LOAD:FE80D2D8 LDR LR, [LR] ; restore Non-Secure return address LOAD:FE80D2DC LDR R3, =NS_core_status.SPSR ; ptr to previously-saved Non-Secure CPSR LOAD:FE80D2E0 LDR R3, [R3] LOAD:FE80D2E4 BIC R3, R3, #CPSR_MASK_FIQ ; clear CPSR.F: FIQ exceptions not masked LOAD:FE80D2E8 MSR SPSR_cxsf, R3 ; write SPSR LOAD:FE80D2EC DMB SY
They will be used later for the exception return.
IV.3.B.e TrustZone lock
After that, tz_lock DWORD is cleared to indicate that this core is no longer running in TrustZone.
LOAD:FE80D2F0 LDR R3, =tz_lock LOAD:FE80D2F4 MOV R2, #0 LOAD:FE80D2F8 STR R2, [R3] ; clear tz_lock LOAD:FE80D2FC DMB SY
IV.3.B.f Exception return to Non-Secure world
The MCR instruction writes to the SCR register to modify the configuration of the current security state:
LOAD:FE80D300 MOV R2, #0 ; clear R2 to avoid leak LOAD:FE80D304 MOV R3, #0 LOAD:FE80D308 MOV R3, #SCR_NS OR SCR_FIQ OR SCR_AW ; 0b100101 LOAD:FE80D30C MCR p15, 0, R3,c1,c1, 0 ; write SCR : switch to Non-Secure state LOAD:FE80D310 ISB SY LOAD:FE80D314 MOV R3, #0 ; clear R3 to avoid leak LOAD:FE80D318 MOVS PC, LR ; restore Non-Secure PC & CPSR LOAD:FE80D318 ; End of function smc_handler

The Security state is switched to Non-Secure (SCR_NS). FIQ interrupts are taken to the Monitor Mode (SCR_FIQ), and the CPSR.A bit can be modified in any security state (SCR_AW), so the Non-Secure world can mask Abort exceptions.
Finally, the exception return is made with a MOVS instruction which branches to the return address in Normal World, and also copies SPSR_mon to CPSR.


We have analyzed a part of Monitor code which allows to switch processor security state through SMC exceptions. We've learnt that some SMC exceptions are fully handled by Monitor code, while others are routed to TrustZone code in Secure Supervisor mode. The latter can be executed by only one core at a time. We have also found that an external ELF can be loaded and executed in TrustZone with a dedicated Secure memory space.
However, this analysis is not complete since IRQ & FIQ handlers have not been studied.
I would like to thank Adrien & Diane for their help.