Reducing AOSP kernel image size

1

I am trying to build an AOSP kernel, but I can't make the resulting boot.img small enough to flash and boot it. It contains the compressed kernel with appended dtbs and the initramfs. My intention is to fuzz-test drivers using Syzkaller. I'm currently using a Pixel XL (marlin). I use kernel branch android-msm-marlin-3.18-pie-qpr3 (tried related ones as well) and I'm trying various gcc-based toolchains. For AOSP I picked Pie (tag android-9.0.0_r46, build PQ3A.190801.002). However, please note that this is a general question, it's not fuzzing-, kernel- or device-specific.

My requirements are:

  • Keep vendor-introduced drivers (because they are what I want to fuzz)

  • Enable KASAN and KCOV (and their dependencies)

  • Enable CONFIG_DEBUG_INFO

  • Keep printk and the like

Things I tried and ideas I have:

  • Tuning kernel config - This helps to some extend, but it's just not enough. I'm already using for example CONFIG_CC_OPTIMIZE_FOR_SIZE=y, CONFIG_CORE_SMALL=y, CONFIG_NET_SMALL=y, CONFIG_KASAN_OUTLINE=y, CONFIG_TRIM_UNUSED_KSYMS=y, CONFIG_SLOB=y, ...

  • Compress kernel (and/or initramfs) with xz or lzma - The idea seems promising. I noticed the kernel build system for arm64 neither supports xz nor lzma out of the box (not sure why?), so I patched it myself to get it building. The build succeeds and the resulting Image.xzkern-dtb or Image.lzma-dtb seems okay, however when attempting to boot it (serial debugging logs):

[5710] partition_enable_wp: group 0 not defined
[5720] DTB offset is incorrect, kernel image does not have appended DTB
[5720] Device info 0x00000131/00010001/0x00010001/0, pmic 0x20009/0x455013/0x0/0x0
[5740] ERROR: Appended Device Tree Blob not found
[5740] panic (frame 0x83a86848):
...

I verified that the DTBs are present and the kernel is compressed as intended. My question here: Is it possible at all? Does the Android bootloader even support xz/lzma? I hardly found any info about that online. Is there any way to use xz or lzma compressed kernels to boot Android?

  • Resize the boot partition - It seems difficult but possible, not sure if worth attempting. Can you recommend any resources or tools for that?

  • Removing unnecessary features from initramfs

  • Link-Time Optimization - I could not get it to work with the AOSP kernel, no matter what gcc version I use. Any tips?

  • Building features as modules instead of built-in - The build won't succeed with any config I try, even when building only single features as module (with different compiler or linker errors depending on the config). Do you have any guides or tips?

I also followed the tutorial on building a Pixel kernel with KASAN and KCOV on the AOSP homepage (link). I had some problems with it, but eventually (using Linaro GCC 5.5.0, with some config changes and without CONFIG_DEBUG_INFO) I could get it building and booting. Nice as a proof-of-concept but not what I need, since I had to skip some required features.

I'm pretty much stuck and already spent some weeks on that problem. Are there any other options? I'd be happy to get it booting in any way. I'll happily provide any details and logs, but felt like it's already a very long post. Thank you a lot in advance!

android
linux
linux-kernel
android-source
asked on Stack Overflow Apr 22, 2020 by onetyone • edited Apr 22, 2020 by onetyone

1 Answer

1

I found a solution working for me. It's a close call though, probably not enough size reduction for different situations. Anyways, it might help others.

I'm using pigz instead of the default gzip tool, which offers an even higher compression level with the -11 switch. Increasing the number of optimization iterations with -I 45 further improved it (default is 15). This squeezes out the last bit that I need: The resulting Image.gz-dtb is about 2.5% smaller than before.

These are the changes I made to the kernel build system (assuming pigz will be in your PATH at build time):

diff --git a/arch/arm64/boot/Makefile b/arch/arm64/boot/Makefile
index b7cf2a498c19..7dfd438e997b 100644
--- a/arch/arm64/boot/Makefile
+++ b/arch/arm64/boot/Makefile
@@ -30,7 +30,8 @@ $(obj)/Image: vmlinux FORCE
        $(call if_changed,objcopy)

 $(obj)/Image.gz: $(obj)/Image FORCE                                    
-       $(call if_changed,gzip)                  
+#      $(call if_changed,gzip)
+       $(call if_changed,pigz)

 $(obj)/Image.lz4: $(obj)/Image FORCE  
        $(call if_changed,lz4)
diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib                      
index c2b437eb23d5..73d4581f7531 100644
--- a/scripts/Makefile.lib
+++ b/scripts/Makefile.lib
@@ -272,6 +272,11 @@ quiet_cmd_gzip = GZIP    $@
 cmd_gzip = (cat $(filter-out FORCE,$^) | gzip -n -f -9 > $@) || \
        (rm -f $@ ; false)

+
+quiet_cmd_pigz = PIGZ    $@
+cmd_pigz = (cat $(filter-out FORCE,$^) | pigz -n -f -11 -I 45 > $@) || \
+       (rm -f $@ ; false)
+
 # DTC
 # ---------------------------------------------------------------------------

It flashes and boots fine, even boots in a reasonable time, as opposed to what I had before.

answered on Stack Overflow May 14, 2020 by onetyone • edited May 15, 2020 by onetyone

User contributions licensed under CC BY-SA 3.0