HowTo use Eclipse with CDT to develop and cross-compile(for ARM) Linux kernel module.

A small HowTo (and reminder for myself) on how to use Eclipse (Xilinx SDK) to develop, cross-compile and upload Linux kernel modules for Zynq (ARM-based) embedded board using Xilinx SDK and Xilinx Embedded Linux aka Petalinux. But most steps are universal, you just have to setup cross-compiler and possibly some missing packages, like sshpass and scp.

This HowTo based on:
HowTo use the CDT to navigate Linux kernel source
Configuring Eclipse for Linux Kernel module development

  1. Linux kernel sources is a huge collection of files and Eclipse indexer needs lots of memory. So, start Xilinx SDK with extra memory for indexer:

    xsdk -vmargs -Xms1024m -Xmx2048m -XX:+UseParallelGC

  2. Add new C Project and configure it:
    1. Give it a name AXIDMATest.
    2. Check Use default location.
    3. Set Project Type to Makefile Project > Empty Project.
    4. Select Xilinx ARM GNU/Linux Toolchain for Toolchain.
    5. Click Finish.
  3. In C/C++ General properties for AXIDMAProject:
    1. Check Enable project specific settings.
    2. Select Indexer on left menu:
      1. Check Enable project specific settings.
      2. Uncheck Index source files not included in the build and check Index all header variants.
    3. Select Preprocessor Include Paths, Macros etc. on left menu:
      1. Select GNU C in Languages
      2. Select CDT User Settings Entries for Settings Entries.
      3. Click Add... button on right.
      4. Select Preprocessor Macros File in top-left drop-down menu.
      5. Select File System Path in top-right left drop-down menu.
      6. Set File to /home/d9/Projects/ZedBoardPetalinux/build/linux/kernel/xlnx-4.0/include/generated/autoconf.h.
      7. Now click on Providers tab and check CDT GCC Built-in Compiler Settings.
      8. Uncheck Use global provider shared between projects and add -nostdinc to Command to get compiler specs:.
    4. Now select Paths and Symbols on left menu:
      1. Select Includes tab.
      2. Select GNU C on left languages list.
      3. Click Add... button on right.
      4. Check Add to all configurations and click File system... button.
      5. Select /opt/Petalinux/petalinux-v2015.4-final/components/linux-kernel/xlnx-4.0/arch/arm/include and click OK.
      6. Repeat prev. steps and add /opt/Petalinux/petalinux-v2015.4-final/components/linux-kernel/xlnx-4.0/include.
      7. Now we have to set some symbols, so select #Symbols tab.
      8. Select GNU C on left languages list.
      9. Click Add... button on right.
      10. Type name __KERNEL__ and value 1, check Add to all configurations and click OK.
      11. You may need other symbols too, like DEBUG and CONFIG_OF.
      12. Now we must exclude staff we don't need, so select Source Location tab.
      13. Click on your project and click Link Folder... button on right.
      14. Check Link to folder in the file system and click Browse... button.
      15. Browse to /opt/Petalinux/petalinux-v2015.4-final/components/linux-kernel/xlnx-4.0 and click OK.
      16. Next expand project, select Filter and click on Edit Filter...button.
        Here we can add folders which we want to exclude from the indexing. All paths are relative to project folder. What to exclude is up to you, but the minimum will be all architectures except ARM.
      17. Apply changes and click OK to close all dialogs.
  4. We can exclude kernel source folders from indexer other way too:
    1. Expand AXIDMATest in Project Explorer tab.
    2. Expand xlnx-4.0, select source folders you think you don't need to be indexed and right click on them.
    3. Select Resource Configuration and click on Exclude from Build....
    4. Select project build configurations in which you want to exclude selected sources. In this case you must have the only Default, so select it and click Ok.
    5. Verify that previously selected folders and sources now greyed out and icons "crossed".
    6. If you want to enable them back - do same steps and uncheck related "build configuration" in Resource configuration.
  5. Add sources to project:
    1. Add new Sources Folder and name it sources.
    2. Copy into it axidmatest.c from "/opt/Petalinux/petalinux-v2015.4-final/components/linux-kernel/xlnx-4.0/drivers/dma/xilinx"
    3. Open axidmatest.c and make sure Indexer did it job (you don't have indexer markers complaining about syntax errors).
  6. Now we need a Makefile:
    1. Add new File to the project folder and name it Makefile.
    2. Code below represents Makefile in my case. You have to modify at least path to your configured kernel folder:
      ARCH:=arm
      CROSS_COMPILE:=arm-xilinx-linux-gnueabi-
      KDIR_DI := ~/Projects/ZedBoardPetalinux/build/linux/kernel/xlnx-4.0/
      PWD  := $(shell pwd)
      
      obj-m += sources/axidmatest.o
      
      all:
      	make -C $(KDIR_DI) M=$(PWD) ARCH=$(ARCH) CROSS_COMPILE=$(CROSS_COMPILE) SUBDIRS=$(PWD/sources) modules
      
      clean:
      	make -C $(KDIR_DI) M=$(PWD) clean
      
  7. Finally we are ready to compile our project. Run Build Project, expand sources folder and and verify that it now includes axidmatest.ko.
  8. To upload module and insmod it to remote system I write a small script, which can be used from Eclipse:
    1. In a AXIDMAProject create folder utils.
    2. In utils folder create file upload_rmmod_insmod.sh.
    3. Put something like code below. You obviously must install sshpass and maybe some other missing utils and modify names, passwords, addresses and etc.
      #!/bin/bash
      
      REMOTE_IP="172.21.0.10"
      FILENAME="axidmatest"
      
      sshpass -p 'xxxx' scp $FILENAME.ko root@$REMOTE_IP:~
      sshpass -p 'xxxx' ssh root@$REMOTE_IP "/sbin/rmmod $FILENAME"
      sshpass -p 'xxxx' ssh root@$REMOTE_IP "/sbin/insmod $FILENAME.ko"
      
    4. Set Execute permissions for this script in console or by right clicking on it and selecting Properties and checking permission boxes.
    5. In Run -> External Tools -> External Tools Configurations... add New launch configuration for Program.
      For some reason it is filtered by default in XSDK 2015.4. So, uncheck Filter Configuration Types in menu on top.
    6. In Main tab:
      1. Type name: AXIDMATest
      2. For Location: select ${workspace_loc:/AXIDMATest/utils/upload_insmod.sh}
      3. For Working Directory: select ${workspace_loc:/AXIDMATest/sources}
    7. In Build tab:
      1. Check Build before launch box.
      2. Select Specific projects radio-button.
      3. Click Projects... button to select projects to build.
      4. Check AXIDMATest box and click OK.
    8. In Common tab:
      1. Check Allocate console (necessary for input) box.
      2. Check Launch in background box.
    9. Click Apply and Close button.
    10. Now we can just run this "External Tool" after making changes to source and it will compile project, upload module to remote system, overwrite existin file, rmmod previous and insmod new version of the module. Output related to script execution will be captured in Eclipse console and module output will be posted on remote system stdout.
    11. axidmatest requires an entrance in DTS in order to work and without it insmod and rmmod are silent. So, to make it print something to stdout add pr_info() to the module __init() and __exit():
      static int __init axidma_init(void)
      {
      	pr_info("axidmatest.ko axidma_init() called.\n");
      	return platform_driver_register(&xilinx_axidmatest_driver);
      
      }
      late_initcall(axidma_init);
      
      static void __exit axidma_exit(void)
      {
      	pr_info("axidmatest.ko axidma_exit() called.\n");
      	platform_driver_unregister(&xilinx_axidmatest_driver);
      }
      module_exit(axidma_exit)
      
      MODULE_AUTHOR("Xilinx, Inc.");
      MODULE_DESCRIPTION("Xilinx AXI DMA Test Client");
      MODULE_LICENSE("GPL v2");
      

Xilinx Petalinux 2015.4 released today.

Today Xilinx released new Petalinux 2015.4 which is synchronized with Xilnx Vivado and SDK 2015.4.
So, now I can finally switch to a latest Vivado and play with Vivado HLS which now included to all Vivado editions including free of cost WebPack.

This release based on Kernel version 4.0 (Xilinx repo tag xilinx-v2015.4), U-boot 2015.07 and GCC 4.8.3 with experimental support of C++11 (GCC 5.1 and later support release C++11, before 5.1 support was experimental). It also includes Vivado DDR patch and support Zynq Ultrascale+ (the only released ZU9).

Unfortunately, Xilinx very slow in adopting latest Eclipse and CDT for it's SDK. Such, current CDT version is 8.8.0 and XSDK still uses 8.3.0 which is a few years old. Very, very conservative.

Time for some hacking :-)

Vivado 2015.2 released today.

Today Xilinx released an update to its Vivado. New version number is 2015.2 and biggest news to me is that it now supports Ubuntu Linux 14.04 TLS(64-bit). It also have number of bugs fixed, but not much of new features - just a support for a few new devices and added CDC report.

Also, just like with 2015.1 where is no Petalinux update. At least yet.

Configure and build Qt5, tslib and evtest for ARM (Xilinx Zynq).

Just a short recipe on how to compile 'tslib' and use it with Qt5 on Xilinx Petalinux.

At the moment I'm using Ubuntu 14.04 64bit as a host machine, MicroZed 7020 as a target, Xilinx Vivado version 2014.2 and Petalinux vesion 2014.2.
My Vivado tools installed to default path '/opt/Xilinx' and I will install Qt5.3.2 and tslib to /opt/Qt/ and /opt/tslib directories.
Also, I'm using 'Project' folder in my home directory for Vivado and Petalinux projects. Now, after I set the scene, let's actually build it.

  1. Clone tslib into our Project folder:
  2. cd ~/Projects/
    git clone https://github.com/kergoth/tslib.git tslib
    cd ~/Projects/tslib

  3. To configure and build 'tslib' we have to setup a few enviroment variables: CROSS_COMPILE, CC, CXX and we have to source Vivado settings64.sh. So, I will create and use a small bash script:
  4. #!/bin/bash

    export CROSS_COMPILE=arm-xilinx-linux-gnueabi-
    source /opt/Xilinx/Vivado/2014.2/settings64.sh

    export CC=$(which arm-xilinx-linux-gnueabi-gcc)
    export CXX=$(which arm-xilinx-linux-gnueabi-g++)

    ./autogen.sh
    ./configure --host=arm-xilinx-linux-gnueabi --prefix /opt/tslib

    make
    sudo make install

  5. Next 'evtest'. This is a small, but very helpful utility when you trying to figure out what is wrong with your touchscreen or touchscreen controller.
  6. cd ~/Projects/
    git clone git://anongit.freedesktop.org/evtest evtest_util
    cd ~/Projects/evtest_util

    #!/bin/bash

    export CROSS_COMPILE=arm-xilinx-linux-gnueabi-
    source /opt/Xilinx/Vivado/2014.2/settings64.sh

    ./autogen.sh
    ./configure --host=arm-xilinx-linux-gnueabi prefix=/opt/evtest/
    make
    sudo make install

  7. Tslib generates 'ts.conf' file, which you can locate in /opt/tslib/etc/. We have to uncomment the module_raw for our touch controller. In my case it 'input'.
  8. Now when we got tslib, we can configure Qt5. I will build Qt5 from scratch using opensource Qt Everywhere Sources. So, lets download and unpack Qt5 sources.
  9. cd ~/Projects/
    wget http://download.qt-project.org/official_releases/qt/5.3/5.3.2/single/qt-everywhere-opensource-src-5.3.2.tar.gz
    tar -zxvf qt-everywhere-opensource-src-5.3.2.tar.gz

  10. Now, before we can configure Qt5 we must create 'mkspecs' for our Xilinx Zynq. So, create new device folder 'linux-arm-xilinx-zynq-g++' and two files in it. 'qplatformdefs' contains just an include, but second file 'qmake.conf' is quite important. This is the place where we set your CFLAGS/CXXFLAGS, some ENV variables which will be used by default by QMAKE and later by Qt5 libs.

    So, if you need to set something differently, this is the time!

  11. cd ~/Projects/qt-everywhere-opensource-src-5.3.2/qtbase/mkspecs/devices/
    create linux-arm-xilinx-zynq-g++

    'qmake.conf':

    #
    # qmake configuration for linux-g++ using arm-xilinx-g++ compiler
    #
    
    MAKEFILE_GENERATOR      = UNIX
    CONFIG                 += incremental gdb_dwarf_index
    QMAKE_INCREMENTAL_STYLE = sublib
    
    include(../../common/linux.conf)
    include(../../common/gcc-base-unix.conf)
    include(../../common/g++-unix.conf)
    
    load(device_config)
    
    QT_QPA_DEFAULT_PLATFORM = linuxfb
    
    # modifications to g++.conf
    QMAKE_CC                = $${CROSS_COMPILE}gcc
    QMAKE_CXX               = $${CROSS_COMPILE}g++
    QMAKE_LINK              = $${QMAKE_CXX}
    QMAKE_LINK_SHLIB        = $${QMAKE_CXX}
    
    # modifications to linux.conf
    QMAKE_AR                = $${CROSS_COMPILE}ar cqs
    QMAKE_OBJCOPY           = $${CROSS_COMPILE}objcopy
    QMAKE_NM                = $${CROSS_COMPILE}nm -P
    QMAKE_STRIP             = $${CROSS_COMPILE}strip
    
    QMAKE_CFLAGS           += -I$$[QT_SYSROOT]/include -DZYNQ
    QMAKE_CXXFLAGS         += -Wno-psabi -I$$[QT_SYSROOT]/include -DZYNQ
    QMAKE_LFLAGS           += -L$$[QT_SYSROOT]/lib
    
    QMAKE_CFLAGS           += -march=armv7-a -mtune=cortex-a9 -mcpu=cortex-a9 -mfpu=neon -pipe -fomit-frame-pointer
    QMAKE_CXXFLAGS         += $$QMAKE_CFLAGS
    
    deviceSanityCheckCompiler()
    
    load(qt_config)
    

    'qplatformdefs.h':

    /****************************************************************************
    **
    ** Copyright (C) 2013 Digia Plc and/or its subsidiary(-ies).
    ** Contact: http://www.qt-project.org/legal
    **
    ** This file is part of the qmake spec of the Qt Toolkit.
    **
    ** $QT_BEGIN_LICENSE:LGPL$
    ** Commercial License Usage
    ** Licensees holding valid commercial Qt licenses may use this file in
    ** accordance with the commercial license agreement provided with the
    ** Software or, alternatively, in accordance with the terms contained in
    ** a written agreement between you and Digia.  For licensing terms and
    ** conditions see http://qt.digia.com/licensing.  For further information
    ** use the contact form at http://qt.digia.com/contact-us.
    **
    ** GNU Lesser General Public License Usage
    ** Alternatively, this file may be used under the terms of the GNU Lesser
    ** General Public License version 2.1 as published by the Free Software
    ** Foundation and appearing in the file LICENSE.LGPL included in the
    ** packaging of this file.  Please review the following information to
    ** ensure the GNU Lesser General Public License version 2.1 requirements
    ** will be met: http://www.gnu.org/licenses/old-licenses/lgpl-2.1.html.
    **
    ** In addition, as a special exception, Digia gives you certain additional
    ** rights.  These rights are described in the Digia Qt LGPL Exception
    ** version 1.1, included in the file LGPL_EXCEPTION.txt in this package.
    **
    ** GNU General Public License Usage
    ** Alternatively, this file may be used under the terms of the GNU
    ** General Public License version 3.0 as published by the Free Software
    ** Foundation and appearing in the file LICENSE.GPL included in the
    ** packaging of this file.  Please review the following information to
    ** ensure the GNU General Public License version 3.0 requirements will be
    ** met: http://www.gnu.org/copyleft/gpl.html.
    **
    **
    ** $QT_END_LICENSE$
    **
    ****************************************************************************/
    
    #include "../../linux-g++/qplatformdefs.h"
    
  12. Next step is to configure Qt5 and below configuration I'm using. Again, this is critical step and if you have change something (compile with OpenGl support for example), you have to do it now. Also, if you want to reconfigure Qt, you have to clean using 'gmake clean' before running configuration script again.
  13. cd ~/Projects/qt-everywhere-opensource-src-5.3.2/
    ./build_qt5_3_2.sh
    gmake
    sudo gmake install

    'build_qt5_3_2.sh':

    #!/bin/bash
    
    export CROSS_COMPILE=arm-xilinx-linux-gnueabi-
    source /opt/Xilinx/Vivado/2014.2/settings64.sh
    
    read -p "Run 'confclean'? (y/n) "
    if [ "$REPLY" == "y" ]; then
            gmake clean
    fi 
    
    ./configure -prefix /opt/Qt/5.3.2 \
    	-device linux-arm-xilinx-zynq-g++ \
    	-device-option CROSS_COMPILE=arm-xilinx-linux-gnueabi- \
    	-release \
    	-confirm-license \
    	-opensource \
    	-optimized-qmake \
    	-no-qml-debug \
    	-qt-zlib \
    	-qt-libpng \
    	-qt-libjpeg \
    	-qt-freetype \
    	-qt-harfbuzz \
    	-qt-pcre \
    	-no-xcb \
    	-qt-xkbcommon \
    	-no-opengl \
    	-no-pch \
    	-verbose \
    	-no-kms \
    	-no-eglfs \
    	-no-icu \
    	-no-iconv \
    	-skip qtwebkit \
    	-tslib \
    	-no-gcc-sysroot \
    	-nomake tools \
    	-no-compile-examples \
    	-I /opt/tslib/include \
    	-L /opt/tslib/lib
    
    read -p "Run 'gmake'? (y/n) "
    if [ "$REPLY" == "y" ]; then
    	cd ~/Projects/$QtSrcName
    	gmake
    fi
    
    read -p "Run 'gmake install'? (y/n) "
    if [ "$REPLY" == "y" ]; then
    	cd ~/Projects/$QtSrcName
    	sudo gmake install
    fi
    
  14. After a few minutes(ha-ha) build will finish and we can build a couple of Qt examples for testing purposes. I created a small script to build each example individually. All I need to do is just copy it in project folder and run it. Examples I usually use is a 'Mainwindow' and a 'Pathstroke'.
  15. 'build_qt5_app.sh'

    #!/bin/bash
    
    export CROSS_COMPILE=arm-xilinx-linux-gnueabi-
    source /opt/Xilinx/Vivado/2014.3/settings64.sh
    
    export QTDIR=/opt/Qt/5.3.2
    export PATH=$QTDIR/bin:$PATH
    export LD_LIBRARY_PATH=/$QTDIR/lib:$LD_LIBRARY_PATH
    
    qmake
    
    make
    
    sudo make install
    
  16. Now, to install Qt5 libs and apps we will create Petalinux 'component'.
  17. cd ~/Projects/$PetalinuxProjectName
    petalinux-create -t libs -n qt-5.3.2 --enable
    cd components/libs/qt-5.3.2/
    rm libqt*
    cp -Pr /opt/Qt/5.3.2/lib .
    cp -Pr /opt/Qt/5.3.2/plugins/ ./lib/
    mkdir bin
    cp /opt/Qt/5.3.2/examples/widgets/painting/pathstroke/pathstroke bin/pathstroke
    cp /opt/Qt/5.3.2/examples/widgets/mainwindows/mainwindow/mainwindow bin/mainwindow

  18. Now, we have to make a few changes in our new 'component' in a Petalinux project( ~/Projects/$PetalinuxProjectName/components/libs/qt-5.3.2/): create 'Makefile' and some file with settings for Qt applications. Tet's call it 'profile.qt-5.3.2'.
  19. 'profile.qt-5.3.2':

    export QT_PLUGIN_PATH=/usr/lib/plugins
    export QT_QPA_FONTDIR=/usr/lib/fonts
    export QT_QPA_PLATFORM_PLUGIN_PATH=/usr/lib/plugins/platforms
    export QT_QPA_PLATFORM=linuxfb
    export QT_QPA_GENERIC_PLUGINS=tslib:/dev/input/event0
    

    Makefile(Don't forget to change username):

    ifndef PETALINUX
    $(error "Error: PETALINUX environment variable not set.  Change to the root of your PetaLinux install, and source the settings.sh file")
    endif
    
    include libs.common.mk
    
    LIB=libqt_5_3_2
    
    all: build install
    
    .PHONY: build
    build:
    
    install:
    	#Install libraries and fonts to the rootfs.
    
    	mkdir -p $(TARGETDIR)/usr/lib
    	USER=d9
    	GROUP=d9
    	rsync -rav ./bin/* $(TARGETDIR)/usr/bin/
    	rsync -rav ./lib/* $(TARGETDIR)/usr/lib/
    
    	#Install the script to ensure the font directory is properly specified.
    	mkdir -p $(TARGETDIR)/etc/profile.d
    	cp profile.qt-5.3.2 $(TARGETDIR)/etc/profile.d/profile.qt-5.3.2
    
    clean:
    
  20. We also need to add 'tslib' to our buildroot. So, I will create another Petalinux 'component', create and copy tslib configuration files:
  21. cd ~/Projects/$PetalinuxProjectName
    petalinux-create -t libs -n tslib --enable
    cd components/libs/tslib/
    rm libtslib*
    rm README
    cp -Pr /opt/tslib/bin .
    cp -Pr /opt/tslib/lib .
    cp -Pr /opt/etc/ts.conf .
    cp -Pr Makefile .
    cp -Pr profile.tslib .
    cp -Pr pointercal .

    My 'profile.tslib'. You may need to change it:

    export TSLIB_TSEVENTTYPE='INPUT'
    export TSLIB_CALIBFILE='/etc/pointercal'
    export TSLIB_CONFFILE='/etc/ts.conf'
    export TSLIB_CONSOLEDEVICE='none'
    export TSLIB_FBDEVICE='/dev/fb0'
    export TSLIB_PLUGINDIR='/usr/lib/ts'
    export TSLIB_TSDEVICE='/dev/input/event0'
    

    'pointercal' for my 800x600 touchscreen:

    94 -13605 53617952 -10567 205 40161292 65536 800 600
    

    'Makefile' (Don't forget to change username):

    ifndef PETALINUX
    $(error "Error: PETALINUX environment variable not set.  Change to the root of your PetaLinux install, and source the settings.sh file")
    endif
    
    include libs.common.mk
    
    LIB=tslib
    
    all: build install
    
    .PHONY: build
    build:
    
    install:
    	#Install libraries and fonts to the rootfs.
    
    	mkdir -p $(TARGETDIR)/usr/lib
    	USER=d9
    	GROUP=d9
    	rsync -rav ./bin/* $(TARGETDIR)/usr/bin/
    	rsync -rav ./lib/* $(TARGETDIR)/usr/lib/
    
    	cp ts.conf $(TARGETDIR)/etc/ts.conf
    	cp profile.tslib $(TARGETDIR)/etc/profile.d/tslib
    	cp pointercal $(TARGETDIR)/etc/pointercal
    
    clean:
    
  22. Qt is a C++ library and like any other C++ application or library it depend on standard c++ library. So, if you didn't include it in your rootfs yet you have to do it now.

    petalinux-config -c rootfs

    Then go to 'Filesystem Packages' -> 'Base' -> 'External-xilinx-toolchain' -> Enable 'libstdc++6'.

  23. This is basically it. Now we have rebuild our Petalinux project and start using it. Just a few notes:
    - After we add Qt5 libraries, our Linux image file will grow is size, so you may need to change U-boot settings to accomodate it.
    - If you need to recalibrate touchscreen - use 'ts_calibrate' utility.
    - If you want to keep it - you must save changes in '/etc/pointercal' file.
    - If you got not only touchscreen, but also mouse and/or keyboard you have to start your application with additional parameters: '-plugin EvdevMouse' '-plugin EvdevKeyboard'.

Some material on Linux Device Drivers from Xilinx.

John Linn posted some interesting material on Linux Device Drivers. It covers basic Linux driver topics in introduction Sessions 1 and 2, UIO drivers in Session 3 and DMA drivers in kernel mode in Session 4. I found all sessions very interesting, but especially on DMA since it really hard to find any kind of information on that topic, besides what is included in Kernel/Documentation folder.

drivers-sessions1-2-public.pdf
drivers-session3-uio-4public.pdf
drivers-session4-dma-4public.pdf