HowTo use Eclipse with CDT to develop and cross-compile(for ARM) Linux kernel module.

A small HowTo (and reminder for myself) on how to use Eclipse (Xilinx SDK) to develop, cross-compile and upload Linux kernel modules for Zynq (ARM-based) embedded board using Xilinx SDK and Xilinx Embedded Linux aka Petalinux. But most steps are universal, you just have to setup cross-compiler and possibly some missing packages, like sshpass and scp.

This HowTo based on:
HowTo use the CDT to navigate Linux kernel source
Configuring Eclipse for Linux Kernel module development

  1. Linux kernel sources is a huge collection of files and Eclipse indexer needs lots of memory. So, start Xilinx SDK with extra memory for indexer:

    xsdk -vmargs -Xms1024m -Xmx2048m -XX:+UseParallelGC

  2. Add new C Project and configure it:
    1. Give it a name AXIDMATest.
    2. Check Use default location.
    3. Set Project Type to Makefile Project > Empty Project.
    4. Select Xilinx ARM GNU/Linux Toolchain for Toolchain.
    5. Click Finish.
  3. In C/C++ General properties for AXIDMAProject:
    1. Check Enable project specific settings.
    2. Select Indexer on left menu:
      1. Check Enable project specific settings.
      2. Uncheck Index source files not included in the build and check Index all header variants.
    3. Select Preprocessor Include Paths, Macros etc. on left menu:
      1. Select GNU C in Languages
      2. Select CDT User Settings Entries for Settings Entries.
      3. Click Add... button on right.
      4. Select Preprocessor Macros File in top-left drop-down menu.
      5. Select File System Path in top-right left drop-down menu.
      6. Set File to /home/d9/Projects/ZedBoardPetalinux/build/linux/kernel/xlnx-4.0/include/generated/autoconf.h.
      7. Now click on Providers tab and check CDT GCC Built-in Compiler Settings.
      8. Uncheck Use global provider shared between projects and add -nostdinc to Command to get compiler specs:.
    4. Now select Paths and Symbols on left menu:
      1. Select Includes tab.
      2. Select GNU C on left languages list.
      3. Click Add... button on right.
      4. Check Add to all configurations and click File system... button.
      5. Select /opt/Petalinux/petalinux-v2015.4-final/components/linux-kernel/xlnx-4.0/arch/arm/include and click OK.
      6. Repeat prev. steps and add /opt/Petalinux/petalinux-v2015.4-final/components/linux-kernel/xlnx-4.0/include.
      7. Now we have to set some symbols, so select #Symbols tab.
      8. Select GNU C on left languages list.
      9. Click Add... button on right.
      10. Type name __KERNEL__ and value 1, check Add to all configurations and click OK.
      11. You may need other symbols too, like DEBUG and CONFIG_OF.
      12. Now we must exclude staff we don't need, so select Source Location tab.
      13. Click on your project and click Link Folder... button on right.
      14. Check Link to folder in the file system and click Browse... button.
      15. Browse to /opt/Petalinux/petalinux-v2015.4-final/components/linux-kernel/xlnx-4.0 and click OK.
      16. Next expand project, select Filter and click on Edit Filter...button.
        Here we can add folders which we want to exclude from the indexing. All paths are relative to project folder. What to exclude is up to you, but the minimum will be all architectures except ARM.
      17. Apply changes and click OK to close all dialogs.
  4. We can exclude kernel source folders from indexer other way too:
    1. Expand AXIDMATest in Project Explorer tab.
    2. Expand xlnx-4.0, select source folders you think you don't need to be indexed and right click on them.
    3. Select Resource Configuration and click on Exclude from Build....
    4. Select project build configurations in which you want to exclude selected sources. In this case you must have the only Default, so select it and click Ok.
    5. Verify that previously selected folders and sources now greyed out and icons "crossed".
    6. If you want to enable them back - do same steps and uncheck related "build configuration" in Resource configuration.
  5. Add sources to project:
    1. Add new Sources Folder and name it sources.
    2. Copy into it axidmatest.c from "/opt/Petalinux/petalinux-v2015.4-final/components/linux-kernel/xlnx-4.0/drivers/dma/xilinx"
    3. Open axidmatest.c and make sure Indexer did it job (you don't have indexer markers complaining about syntax errors).
  6. Now we need a Makefile:
    1. Add new File to the project folder and name it Makefile.
    2. Code below represents Makefile in my case. You have to modify at least path to your configured kernel folder:
      ARCH:=arm
      CROSS_COMPILE:=arm-xilinx-linux-gnueabi-
      KDIR_DI := ~/Projects/ZedBoardPetalinux/build/linux/kernel/xlnx-4.0/
      PWD  := $(shell pwd)
      
      obj-m += sources/axidmatest.o
      
      all:
      	make -C $(KDIR_DI) M=$(PWD) ARCH=$(ARCH) CROSS_COMPILE=$(CROSS_COMPILE) SUBDIRS=$(PWD/sources) modules
      
      clean:
      	make -C $(KDIR_DI) M=$(PWD) clean
      
  7. Finally we are ready to compile our project. Run Build Project, expand sources folder and and verify that it now includes axidmatest.ko.
  8. To upload module and insmod it to remote system I write a small script, which can be used from Eclipse:
    1. In a AXIDMAProject create folder utils.
    2. In utils folder create file upload_rmmod_insmod.sh.
    3. Put something like code below. You obviously must install sshpass and maybe some other missing utils and modify names, passwords, addresses and etc.
      #!/bin/bash
      
      REMOTE_IP="172.21.0.10"
      FILENAME="axidmatest"
      
      sshpass -p 'xxxx' scp $FILENAME.ko root@$REMOTE_IP:~
      sshpass -p 'xxxx' ssh root@$REMOTE_IP "/sbin/rmmod $FILENAME"
      sshpass -p 'xxxx' ssh root@$REMOTE_IP "/sbin/insmod $FILENAME.ko"
      
    4. Set Execute permissions for this script in console or by right clicking on it and selecting Properties and checking permission boxes.
    5. In Run -> External Tools -> External Tools Configurations... add New launch configuration for Program.
      For some reason it is filtered by default in XSDK 2015.4. So, uncheck Filter Configuration Types in menu on top.
    6. In Main tab:
      1. Type name: AXIDMATest
      2. For Location: select ${workspace_loc:/AXIDMATest/utils/upload_insmod.sh}
      3. For Working Directory: select ${workspace_loc:/AXIDMATest/sources}
    7. In Build tab:
      1. Check Build before launch box.
      2. Select Specific projects radio-button.
      3. Click Projects... button to select projects to build.
      4. Check AXIDMATest box and click OK.
    8. In Common tab:
      1. Check Allocate console (necessary for input) box.
      2. Check Launch in background box.
    9. Click Apply and Close button.
    10. Now we can just run this "External Tool" after making changes to source and it will compile project, upload module to remote system, overwrite existin file, rmmod previous and insmod new version of the module. Output related to script execution will be captured in Eclipse console and module output will be posted on remote system stdout.
    11. axidmatest requires an entrance in DTS in order to work and without it insmod and rmmod are silent. So, to make it print something to stdout add pr_info() to the module __init() and __exit():
      static int __init axidma_init(void)
      {
      	pr_info("axidmatest.ko axidma_init() called.\n");
      	return platform_driver_register(&xilinx_axidmatest_driver);
      
      }
      late_initcall(axidma_init);
      
      static void __exit axidma_exit(void)
      {
      	pr_info("axidmatest.ko axidma_exit() called.\n");
      	platform_driver_unregister(&xilinx_axidmatest_driver);
      }
      module_exit(axidma_exit)
      
      MODULE_AUTHOR("Xilinx, Inc.");
      MODULE_DESCRIPTION("Xilinx AXI DMA Test Client");
      MODULE_LICENSE("GPL v2");
      

HDMI on ZedBoard with Petalinux update.

I finally figure it out why Analog Devices reference design create/generated in Vivado 2014 by script(obviously updated to use new IP's, otherwise it didn't assemble 'Block Design' at all) didn't work. Reason is changes in Xilinx Concat IP, which used in reference design to concatenate interrupt signals from VDMA and I2C IP blocks to Zynq's F2S interrupt bus.
So, now in 2014.1, we got version 2.0 of it and it preserve the order of input signals on the output. Which means we must either change inputs order or change interrupt numbers in DTS.

So, for AD reference design generated in Vivado2014.1 interrupts are:

  • AXI_IIC_MAIN - #56.
  • AXI_VDMA_0 - #55.
  • AXI_IIC_FMC - #59.

zynq16_001

HDMI on ZedBoard with Petalinux.

This is step-by-step tutorial on how to build reference design for Analog Devices ADV7511 HDMI encoder used on ZedBoard with PetaLinux 2013.10. It will be mostly based on AD HDL reference design http://wiki.analog.com/resources/fpga/xilinx/kc705/adv7511 and AD Linux drivers wiki page http://wiki.analog.com/resources/tools-software/linux-drivers/platforms/zynq and Xilinx PetaLinux documentation http://www.wiki.xilinx.com/PetaLinux.

As of today, 25 May 2014, to create HDL design for ADV7511 from scratch, we have to use Vivado 2013.4, even though Vivado 2014.1 is already available. The reason is some changes in a Xilinx IP's (which I didn't had a chance to figure out yet) prevent HDL design from build/work properly.

  1. First step is to download HDL libraries and projects from AnalogDevices repositories on a github: https://github.com/analogdevicesinc/hdl. You can clone it or download a ZIP. I will download a ZIP and extract 'hdl-master' in my Projects/FPGA/ folder on Windows7 machine.
  2. Second step is to build a few Analog Devices IP required to create ZedBoard HDMI design. Run Xilinx Vivado 2013.4, open a TCL console, change directories and 'source' a .tcl scripts. For example, to build AXI_CLKGEN IP:

    cd c:/Projects/FPGA/hdl-master/library/axi_clkgen
    source ./axi_clkgen_ip.tcl

    After script finish, close created project and build the next. For ZedBoard we have build the next IP's:

    • hdl-master/library/axi_clkgen
    • hdl-master/library/axi_hdmi_tx
    • hdl-master/library/axi_i2s_adi
    • hdl-master/library/axi_spdif_tx
    • hdl-master/library/util_i2c_mixer
  3. After we done with all required IP's, we can build ADV7511 reference design for ZedBoard. In a Tcl Console change directory to ADV7511 and run 'system_project' script.

    cd c:/Projects/FPGA/hdl-master/projects/adv7511/zed/
    source ./system_project.tcl

    Script will create block design, run synthesis and implementation, generate bitstream and even export software to SDK(without opening it). This was the case on my system - everything went smoothly. We are done with Vivado and can close it.

    We have to create HDL in Vivado 2013.4, but later we can import created project into Vivado 2014.1 and update it to use latest Xilinx IP's.

  4. Let's build a FSBL. We need very typical Zynq first stage boot loader and I covered creation of it before, so now just a short description:
    • Run XSDK.
    • Create new 'Hardware Platform Specification' project (I named it 'ZedBoard-HDMI-HW') and specify HW created in a previous step.
    • Create Application project (named 'ZedBoard-HDMI-FSBL') using our new 'Hardware Platform' and select to create new BSP for it. Don't forget to use 'Zynq FSBL' template. Build it if this not done automatically.
  5. Next step is to create PetaLinux BSP. This is also very typical PetaLinux BSP, just don't forget to change 'Configuration' to reflect ZedBoard configuration and name it 'ZedBoard-HDMI-petalinux_bsp'.

    zynq15_002

    We are done with Xilinx SDK. You can close it.

  6. Next step is to create PetaLinux project and set 'hardware description'. I will call it 'ZedBoard-HDMI' Petalinux project:

    petalinux-create -t project -n ZedBoard-HDMI
    cd ~/Projects/ZedBoard-HDMI-petalinux_bsp/
    petalinux-config --get-hw-description -p ../ZedBoard-HDMI/
    cd ~/Projects/ZedBoard-HDMI/
    rm -r hw-description

  7. Now, as of today, ADV7511 Linux driver not in a mainstream kernel. So, we need to get Kernel from Analog Devices repository with appropriate patches. Current version is 3.14.0. Let's clone it, and checkout 'xcomm_zynq' branch.

    cd ~/Projects/
    git clone https://github.com/analogdevicesinc/linux.git analogdevices-kernel
    cd analogdevices-kernel/
    git checkout xcomm_zynq

  8. Create necessary directories and copy 'xcomm_zynq' branch to our PetaLinux project directory.

    cd ~/Projects/
    mkdir ~/Projects/ZedBoard-HDMI/components
    mkdir ~/Projects/ZedBoard-HDMI/components/linux-kernel
    cp -a analogdevices-kernel ~/Projects/ZedBoard-HDMI/components/linux-kernel/

  9. Run 'petalinux-config' and change kernel to 'analogdevices-kernel' and system boot device to 'SD card'.
  10. cd ZedBoard-HDMI
    petalinux-config

  11. Next we need to configure Linux kernel for PetaLinux and we need to enable all options required by ADV7511. AnalogDevices kernel support special configuration option 'zynq_xcomm_adv7511_defconfig', but we cannot run it with PetaLinux. So, we have to pre-configure kernel separately ('make ARCH=arm zynq_xcomm_adv7511_defconfig') and just copy resulted config into 'ZedBoard-HDMI/subsystems/linux/configs/kernel'. So, I did it and also copied it into PetaLinux Kernel configs directory '/opt/petalinux-v2013.10-final/etc/template/project/template-zynq/subsystems/linux/configs/kernel'. So, I can later reuse it. Also notice that kernel default config file have dot in the front and PetaLinux files don't.
    Anyway, here is link to my resulted kernel config file: https://blog.idv-tech.com/wp-content/uploads/2014/05/config_hdmi_3_14.config
  12. We also, have to modify 'devices tree' generated by PetaLinux for our project. AnalogDecices Linux kernel have template for ZedBoard which you can find in 'arch/arm/boot/dts/zynq-zed-adv7511.dts', so we basically have to copy missing devices from AD into our tree.
    Link to my resulted DTS file for ZedBoard: https://blog.idv-tech.com/wp-content/uploads/2014/05/adv7511_dts.config.
  13. We are basically done. At this point you my want to modify PetaLinux project, for example, include Qt5 library and test app to check frame buffer device later. I covered this topics in my previous post, so I wont repeat it here.

    Build Petalinux project, create BOOT.BIN and copy it together with Linux image file 'image.ub' on SD card:

    petalinux-build
    petalinux-package --boot --fsbl ../ZedBoard-HDMI-FSBL/Release/ZedBoard-HDMI-FSBL.elf --fpga ../ZedBoard-HDMI-HW/system_top.bit --uboot --force -o images/linux/BOOT.BIN

  14. Insert SD card into slot of ZedBoard and turn it on. During boot kernel should detect ADV7511(hdmi) and ADAU1761(sound) devices and create '/dev/fb0' device.So, below partial bootlog from my ZedBoard:
  15. ...
    [drm] Initialized drm 1.1.0 20060810
    /analogdevices-kernel/drivers/gpu/drm/adi_axi_hdmi/axi_hdmi_drv.c:axi_hdmi_platform_probe[176]
    platform 70e00000.axi_hdmi: Driver axi-hdmi requests probe deferral
    ...
    adv7511-hdmi-snd adv7511_hdmi_snd.2: adv7511 <-> 75c00000.axi-spdif-tx mapping ok
    ...
    zed-adau1761-snd zed_sound.3: adau-hifi <-> 77600000.axi-i2s mapping ok
    ...
    Console: switching to colour frame buffer device 180x56
    axi-hdmi 70e00000.axi_hdmi: fb0:  frame buffer device
    axi-hdmi 70e00000.axi_hdmi: registered panic notifier
    [drm] Initialized axi_hdmi_drm 1.0.0 20120930 on minor 0
    /analogdevices-kernel/drivers/rtc/hctosys.c: unable to open rtc device (rtc0)
    ALSA device list:
      #0: HDMI monitor
      #1: ZED ADAU1761
    Freeing unused kernel memory: 23356K (c062b000 - c1cfa000)
    INIT: version 2.88 booting
    Starting Bootlog daemon: bootlogd.
    ...
    
     _____       _           _      _
    | ___ \     | |         | |    (_)
    | |_/ / ___ | |_   __ _ | |     _  _ __   _   _ __  __
    |  __/ / _ \| __| / _` || |    | || '_ \ | | | |\ \/ /
    | |   |  __/| |_ | (_| || |____| || | | || |_| | >  <
    \_|    \___| \__| \__,_|\_____/|_||_| |_| \__,_|/_/\_\
    
    PetaLinux v2013.10 (Yocto 1.4) ZedBoard ttyPS0
    
    ZedBoard login: root
    Password:
    login[923]: root login  on `ttyPS0'
    
    root@ZedBoard:~# ls /dev/fb0
    /dev/fb0
    root@ZedBoard:~# uname -a
    Linux ZedBoard 3.14.0-g681a2d8-dirty #2 SMP PREEMPT Sun May 25 22:46:28 EDT 2014 armv7l GNU/Linux
    root@ZedBoard:~#
    
  16. This is basically it - once you have a framebuffer device you can start using it. So I ran my Qt5 test app and it worked. We obviously don't have any hardware acceleration with this HDL design, but we got basic FB device and HDMI output. Congratulations!