Ubidetach error cannot detach

Unsorted Block Images (UBI)

Unsorted block images (UBI) is an erase block management layer for flash memory devices such as raw NAND devices. UBI serves two purposes:

• tracking NAND flash bad blocks
• providing wear leveling

UBI was written specifically for UBIFS so that the filesystem does not have to deal with these issues. UBIFS is a successor to JFFS2 and a competitor to LogFS.

References:

• ​http://en.wikipedia.org/wiki/UBIFS#Unsorted_Block_Images
• ​http://www.linux-mtd.infradead.org/faq/ubi.html
• ​http://www.linux-mtd.infradead.org/faq/ubifs.html

FLASH chip geometry

Because UBI takes care of bad block tracking and wear leveling, it needs to know some details about the geometry of the raw FLASH device when creating an image.

The geometry data can be found in the datasheet for the specific flash device used. Gateworks uses a variety of NAND Flash devices that fall within two different unique geometries we refer to as ‘normal’ and ‘large’ which references the erase block size:

• The ‘Page Size’, ‘Block Size’, and max blocks are specified by the NAND datasheet. The Logical Erase Block size can be calculated as: (Block Size) — 2 * (Page Size)

The geometry can also be determined at runtime in U-Boot using nand info and Linux using sysfs:

• Cypress S34ML16G202 2048MiB FLASH:
  • U-Boot:

    Ventana > nand info
    
    Device 0: nand0, sector size 128 KiB
      Page size       2048 b
      OOB size         128 b
      Erase size    131072 b
      subpagesize     2048 b
      options     0x40000200
      bbt options 0x    8000
    

    • U-Boot above shows a 2048 byte page size and a 128KiB erase block size which dictates a 124KiB logical erase block size
  • Linux (once device is attached via ‘ubiattach /dev/ubi_ctrl -m 2’ or via ‘root=ubi ubi=2’ kernel cmdline params:

    # cat /sys/devices/virtual/ubi/ubi0/min_io_size 
    2048
    # cat /sys/devices/virtual/ubi/ubi0/eraseblock_size 
    126976
    # cat /sys/devices/virtual/ubi/ubi0/total_eraseblocks 
    16248
    

    • above shows a 2048 byte page size, a 124KiB logical erase block size (which dictates a 128KiB physical erase block size) and a max of 16248 total erase blocks available in this ubi for a total usable space of 16248*124KiB still available for use by ubi volumes
• Micron MT29F16G08 2048MiB FLASH:
  • U-Boot:

    Ventana > nand info
    
    Device 0: nand0, sector size 256 KiB
      Page size       4096 b
      OOB size         224 b
      Erase size    262144 b
      subpagesize     4096 b
      options     0x40000200
      bbt options 0x       0
    

    • Note the 4096 byte page size and 256KiB erase block size which dictates a 248KiB logical erase block size
  • Linux (once device is attached via ‘ubiattach /dev/ubi_ctrl -m 2’ or via ‘root=ubi ubi=2’ kernel cmdline params:

    # cat /sys/devices/virtual/ubi/ubi0/min_io_size 
    4096
    # cat /sys/devices/virtual/ubi/ubi0/eraseblock_size 
    253952
    # cat /sys/devices/virtual/ubi/ubi0/total_eraseblocks 
    8124
    

    • above shows a 4096 byte page size, a 248KiB logical erase block size (which dictates a 256KiB physical erase block size) and a max of 8124 total erase blocks available in this ubi for a total usable space of 253952*248KiB still available for use by ubi volumes
• Micron MT29F2G08 256MiB FLASH:
  • U-Boot:

    Ventana > nand info
    
    Device 0: nand0, sector size 128 KiB
      Page size       2048 b
      OOB size          64 b
      Erase size    131072 b
      subpagesize     2048 b
      options     0x40000200
      bbt options 0x    8000
    

    • U-Boot aqbove shows a 2048 byte page size and a 128KiB erase block size which dictates a 124KiB logical erase block size
  • Linux (once device is attached via ‘ubiattach /dev/ubi_ctrl -m 2’ or via ‘root=ubi ubi=2’ kernel cmdline params:

    # cat /sys/devices/virtual/ubi/ubi0/min_io_size 
    2048
    # cat /sys/devices/virtual/ubi/ubi0/eraseblock_size 
    126976
    # cat /sys/devices/virtual/ubi/ubi0/total_eraseblocks 
    1912
    

    • above shows a 2048 byte page size, a 124KiB logical erase block size (which dictates a 128KiB physical erase block size) and a max of 126976 total erase blocks available in this ubi for a total usable space of 126976*124KiB still available for use by ubi volumes

mkfs.ubifs important arguments:

• -r Build file system from directory DIR
• -F The -F option causes mkfs.ubifs to set a special flag in the superblock, which triggers a «free space fixup» procedure in the kernel the very first time the filesystem is mounted. This fixup procedure involves finding all empty pages in the UBIFS file system and re-erasing them. This ensures that NAND pages which contain all 0xFF data get fully erased, which removes any problematic non-0xFF data from their OOB areas.
• -m Minimum I/O unit size (this is the Page Size)
• -e Logical erase block size (UBI requires 2 minimum I/O units out of each Physical Erase Block (PEB) for overhead thus this is the ‘Block Size’ minus 2x the Page Size
• -c Maximum logical erase block count (Number of Blocks; use the largest of the chips in the same geometry in order to create a ubifs that works on all the above NAND devices)
• -x Compression type — «lzo», «favor_lzo», «zlib» or «none» (default: «lzo»)
• -o Output to FILENAME
• -h Help, provides full list of options

ubinize important arguments:

• -o Output file
• -p Size of the physical eraseblock of the flash in bytes, KiB, or MiB
• -m Minimum input/output unit size of the flash in bytes (this is the Page Size)

UBI/UBIFS use on Gateworks Ventana

UBI is used on the Gateworks Ventana for raw NAND storage. Other Gateworks Product families used NOR FLASH and eMMC.

The Gateworks Ventana default U-Boot environment scripts make a few assumptions about booting from NAND flash:

• the mtdparts env var partitions the NAND device into sections for uboot (16MB as this accounts for the SPL, u-boot.img and redundancy for bad blocks), u-boot env (1MB), rootfs (the remainder of the device)
• the ‘rootfs’ partition (defined by the mtdparts env var) contains a UBI device/image
• the UBI device/image in the rootfs partition contains a volume also called ‘rootfs’
• an optional bootscript may exist in boot/6x_bootscript-ventana (the image env var) and if found will be sourced
• a flattened device tree blob (dtb) may exist in boot/ (named by the automatic env vars fdt_file, fdt_file1, or fdt_file2)
• a kernel may exist in boot/uImage

The above assumptions are not requirements — the U-Boot env scripts can be changed to suit your needs.

The mtd-utils Ubuntu package provides the ubninize and mkfs.ubifs utilities you need to work with ubi images:

sudo apt install mtd-utils

Creating a UBI/UBIFS using existing rootfs, kernel, and device-tree blobs

To create a UBI image suitable for booting on a standard Gateworks Ventana bootloader (taking into account the assumptions above) you can do the following on a Linux system provided you have a root filesystem (myrootfs.tar.gz), kernel (uImage), and device-tree dtb files (imx6*-gw*.dtb).

‘normal’ geometry FLASH devices (2048 byte page size)

To create a UBI image suitable for raw NAND FLASH devices with a 2048 byte page size:

1. create a temporary directory for our rootfs and untar your filesystem to it

   mkdir tmp
   sudo tar --numeric-owner -xvf myrootfs.tar.gz -C tmp
   

of files that you have in your rootfs such as root owned /dev files

1. copy your kernel files (and u-boot bootscript if you have one):

   cd tmp/
   wget http://dev.gateworks.com/ventana/images/linux-ventana.tar.xz
   tar -xvf linux-ventana.tar.xz --keep-directory-symlink
   depmod $(ls /lib/modules/) # create module dependencies
   rm linux-ventana.tar.xz
   cd ..
   

2. create a ubifs image for the 2048 byte page size ‘normal’ device geometry:

   mkfs.ubifs -F -m 2048 -e 124KiB -c 16248 -x zlib -o root.ubifs -d tmp
   

• the -c parameter (max-peb-count) of 16248 assures that the filesystem can be sized up to 16248*124KiB or 1.9GiB provided there is remaining space in the ubi
• the -F parameter sets the free-space-fixup flag causing the filesystem to be resized on mount not to exceed the max-peb-count argument passed with the -c parameter (assuming the volume is dynamic and has blocks remaining)

1. use ubinize to create a ubi image (which can be written to a suitable raw NAND device in U-Boot using nand write or in Linux via ubiformat):

   cat <<EOF > ubinize.cfg
   [ubifs]
   mode=ubi
   image=root.ubifs
   vol_id=0
   vol_type=dynamic
   vol_name=rootfs
   vol_flags=autoresize
   EOF
   # create ubi suitable for 2048byte page size and 128KiB block size devices
   ubinize -m 2048 -p 128KiB -o image.ubi ubinize.cfg
   

‘large’ geometry FLASH devices (4096 byte page size)

To create a UBI image suitable for raw NAND FLASH devices with a 4096 byte page size:

1. create a temporary directory for our rootfs and untar your filesystem to it

   mkdir tmp
   sudo tar -xvf myrootfs.tar.gz -C tmp
   

2. copy your kernel files (and u-boot bootscript if you have one):

   cd tmp/
   wget http://dev.gateworks.com/ventana/images/linux-ventana.tar.xz
   tar -xvf linux-ventana.tar.xz --keep-directory-symlink
   depmod $(ls /lib/modules/) # create module dependencies
   rm linux-ventana.tar.xz
   cd ..
   

3. create ubifs images for the 4096 byte page size ‘large’ device geometry:

   sudo mkfs.ubifs -F -m 4096 -e 248KiB -c 8124 -o root.ubifs -d tmp
   

   • the -c parameter (max-peb-count) of 8124 assures that the filesystem can be sized up to 8124*248KiB or 1.9GiB provided there is remaining space in the ubi
   • the -F parameter sets the free-space-fixup flag causing the filesystem to be resized on mount not to exceed the max-peb-count argument passed with the -c parameter (assuming the volume is dynamic and has blocks remaining)
4. use ubinize to create ubi image (which can be written to a suitable raw NAND device in U-Boot using nand write}} or in Linux via {{{ubiformat:

   cat <<EOF > ubinize.cfg
   [ubifs]
   mode=ubi
   image=root.ubifs #generate this image with mkfs.ubifs
   vol_id=0
   vol_type=dynamic
   vol_name=rootfs
   vol_flags=autoresize
   EOF
   # create ubi suitable for 4096byte page size and 256KiB block size devices
   ubinize -m 4096 -p 256KiB -o image.ubi ubinize.cfg
   

Bad blocks

NAND flashes have a random amount of initial bad blocks. This means that different devices may have slightly different volume sizes (especially if the UBI auto-resize feature is used).

Working with a UBI in Linux OS (extracting or manipulating contents)

Linux has support for ubi:

• mounting a ubi
• creating/renaming/removing ubi volumes
• reading/writing ubi volumes

Working with UBI and UBIFS volumes on Linux require use of the following:

• tools (from mtd-utils package):
  • ubiattach
  • ubidetach
  • ubiformat
  • ubirename
  • ubiupdatevol
  • ubinfo
  • ubimkvol
  • ubirmvol
• kernel support:
  • mtd (the MTD layer)
  • mtdblock (the block layer over MTD)
  • ubi (ubi support)
  • ubifs (ubi filesystem support)
  • nandsim (if you want to simulate a NAND device on a development host)

When using the above tools the MTD partition must not be mounted by your Linux OS meaning you either have to:

• use the nandsim kernel module to simulate a NAND device
• boot Linux target with NAND storage using another boot media (ramdisk, usb, mmc, sata, etc)

Note that the creation of ubifs and ubi images is covered in the section above. This section is all about extracting and / or manipulating the contents of a ubi volume.

References:

• ​http://free-electrons.com/blog/creating-flashing-ubi-ubifs-images/
• ​http://elinux.org/UBIFS
• ​http://www.linux-mtd.infradead.org/faq/ubifs.html

On a (NAND-less) Linux Development Host using nandsim

Often you want to work with UBI and UBIFS images on a Linux Development Host which has no NAND itself. While it is not currently possible to mount a UBI or UBIFS as a loopback device (commonly used for other types of block devices) you can use the nandsim kernel module to simulate a NAND device on a NAND-less Linux development host.

To create a simulated NAND device you need to know the first four bytes returned by the READ_ID flash command, which identifies the characteristics of the device. These values can be found in the datasheet of the FLASH devices. As Gateworks boards support two flash sizes, we will show both examples.

To create a virtual NAND device on a NAND-less Linux development system:

1. load nandsim module with parameters for specific flash device simulating:
   • 256MiB MT29F2G08 FLASH

     sudo rmmod nandsim
     sudo modprobe nandsim
       first_id_byte=0x2c 
       second_id_byte=0xda 
       third_id_byte=0x90 
       fourth_id_byte=0x95
     

   • 2GiB Cyrpess S34ML16G202 FLASH

     sudo rmmod nandsim
     sudo modprobe nandsim 
       first_id_byte=0x01 
       second_id_byte=0xd5 
       third_id_byte=0xd2 
       fourth_id_byte=0x95
     

   • 2GiB Micron MT29F16G08 FLASH:

     sudo rmmod nandsim
     sudo modprobe nandsim 
       first_id_byte=0x2c 
       second_id_byte=0xd5 
       third_id_byte=0xd1 
       fourth_id_byte=0xa6
     

   • the above create a /dev/mtd0 device on your development host
   • by default nandsim will use RAM backed storage which makes for much faster read/write access, however if you are cramped for RAM space and simulating a large device (ie 2GB) you can add a cache_file=/tmp/nandimage param to nandsim to tell it to use a file-backed storage instead. You can remove the file once you rmmod nandsim.
2. populate NAND with an existing ubi:

   sudo modprobe mtdblock
   sudo dd if=image.ubi of=/dev/mtdblock0 bs=2048
   

   • the mtdblock module creates a /dev/mtdblock0 device on your development host
   • Note that instead of the dd command above you could also have used ubiformat /dev/mtd0 to wipe and format a new ubi but this section is documenting how to extract and alter contents of an existing ubi.
3. attach the UBI

   sudo modprobe ubi
   sudo ubiattach /dev/ubi_ctrl -m0 -O2048
   sudo ubinfo -a # optionally show info about the UBI
   

4. mount the ubifs to host

   sudo modprobe ubifs
   sudo mkdir /mnt/ubi # create a mountpoint
   sudo mount -t ubifs ubi0 /mnt/ubi
   ls /mnt/ubi # optionally list contents of mounted filesystem
   

Once you are done working with the files in /mnt/ubi simply unmount the device:

If you have altered the filesystem and wish to extract the altered UBI you can detach and use dd to copy it to a file:

sudo ubidetach /dev/ubi_ctrl -m0
sudo dd if=/dev/mtdblock0 of=image-updated.ubi bs=2048

And when you are done working with the simulated NAND device completely, remove the nandsim module to free up your system memory:

References:

• ​http://www.linux-mtd.infradead.org/faq/nand.html#L_nand_nandsim

On a target board with NAND FLASH

To extract or manipulate the contents of a UBI within the NAND Flash of a Linux system, Linux must be booted from a secondary medium without the NAND FLASH partition being mounted.

The following examples assume that UBI has been configured to use the 3rd (zero based) MTD FLASH partition (ubi.mtd=2 kernel command line) as we typically use the mtdparts=nand:16m(uboot),1m(env),-(rootfs) env variable in U-Boot to dynamically modify the Linux Device-Tree to partition the NAND flash device into 3 partitions: bootloader, bootloader-env, and rootfs. Thus the bootloader is /dev/mtd0, the bootloader env is /dev/mtd1, and the rootfs is /dev/mtd2.

Examples:

• Mounting UBI NAND FLASH volume (from /dev/mtd2):

  ubidetach /dev/ubi_ctrl -m 2 # Make sure /dev/mtd2 is not currently attached
  ubiattach /dev/ubi_ctrl -m 2 # attach /dev/mtd2 as /dev/ubi0
  mount -t ubifs ubi0:rootfs /mnt/ubi #mount 'rootfs' volume from /dev/ubi0 to /mnt/ubi
  

• Updating NAND FLASH using linux target with NAND(/dev/mtd2) with a UBI image:

  ubidetach /dev/ubi_ctrl -m 2 # Make sure /dev/mtd2 is not currently attached
  sync
  ubiformat /dev/mtd2 -y -f image.ubi
  

• Updating NAND FLASH partition (/dev/mtd2) with a UBIFS image

  ubidetach /dev/ubi_ctrl -m 2 # Make sure /dev/mtd2 is not currently attached
  ubiformat /dev/mtd2 -y # format (will wipe contents!)
  ubiattach /dev/ubi_ctrl -m 2 # attach /dev/mtd2 as /dev/ubi0
  ubimkvol /dev/ubi0 -N rootfs -m # create a volume on /dev/ubi0 named 'rootfs'
  ubiupdatevol /dev/ubi0_0 image.ubifs
  

• Updating NAND FLASH partition (/dev/mtd2) with a tarball

  ubidetach /dev/ubi_ctrl -m 2 # Make sure /dev/mtd2 is not currently attached
  ubiformat /dev/mtd2 -y # format (will wipe contents!)
  ubiattach /dev/ubi_ctrl -m 2 # attach /dev/mtd2 as /dev/ubi0
  ubimkvol /dev/ubi0 -N rootfs -m # create a volume on /dev/ubi0 named 'rootfs'
  mkdir /mnt/ubi # create a mount-point
  mount -t ubifs ubi0:rootfs /mnt/ubi # mount 'rootfs' volume from /dev/ubi0 to /mnt/ubi
  tar xvf rootfs.tar.gz -C /mnt/ubi # extract tarball
  umount /mnt/ubi # unmount filesystem
  

Troubleshooting:

The output beneath indicates that /dev/mtd2 is not attached and you may proceed with manipulating the NAND flash.

root@OpenWrt:/# ubidetach /dev/ubi_ctrl -m 2
ubidetach: error!: cannot detach mtd2
           error 16 (Resource busy)

Working with UBI in U-boot Bootloader

The U-boot bootloader has support for ubi and ubifs. Before working with a ubi image you need to ‘attach’ an MTD partition, then you can mount and work with ubifs filesystem images. Note that you can remove/create ubi volumes and you can ls and load from ubifs filesystem images but you cant write to ubifs filesystems:

• determining MTD partition containing ubi’s:

  Ventana > mtd
  
  device nand0 <nand>, # parts = 3
   #: name                size            offset          mask_flags
   0: uboot               0x01000000      0x00000000      0
   1: env                 0x00100000      0x01000000      0
   2: rootfs              0x7ef00000      0x01100000      0
  
  active partition: nand0,0 - (uboot) 0x01000000 @ 0x00000000
  
  defaults:
  mtdids  : nand0=nand
  mtdparts: mtdparts=nand:16m(uboot),1m(env),-(rootfs)
  

  • The above shows there is one RAW NAND device (nand0) and it is partitioned into 3 sections: uboot (from 0x00000000 to 0x01000000), uboot env (from 0x01000000 to 0x01100000) and rootfs (from 0x01100000 to 0x7ef00000). The ‘uboot’ and ‘env’ are part of the boot firmware, so the section named ‘rootfs’ is the section that can contain ubi’s
  • Note that this partitioning is solely based on the ‘mtdids’ and ‘mtdparts’ U-Boot env variables and can be changed if desired. Note also that these variables are used to send information into the kernel so that the kernel creates these sections as specific MTD devices
• attaching a ubi image to an MTD partition:

  Ventana > ubi part rootfs
  ubi0: default fastmap pool size: 256
  ubi0: default fastmap WL pool size: 128
  ubi0: attaching mtd1
  ubi0: scanning is finished
  ubi0: attached mtd1 (name "mtd=2", size 2031 MiB)
  ubi0: PEB size: 131072 bytes (128 KiB), LEB size: 126976 bytes
  ubi0: min./max. I/O unit sizes: 2048/2048, sub-page size 2048
  ubi0: VID header offset: 2048 (aligned 2048), data offset: 4096
  ubi0: good PEBs: 16248, bad PEBs: 0, corrupted PEBs: 0
  ubi0: user volume: 3, internal volumes: 1, max. volumes count: 128
  ubi0: max/mean erase counter: 2/0, WL threshold: 4096, image sequence number: 1592951068
  ubi0: available PEBs: 0, total reserved PEBs: 16248, PEBs reserved for bad PEB handling: 320
  

  • The above ‘attaches’ a ubi image to an MTD partition (equivalent to ubiattach in Linux) so that you can mount ubi volumes. Note that this operation can take several seconds as it must scan the entire ubi image to construct a bad block table (unless fastmap is used). Once attached you can use ubifsmount and you will need to detach via ubi detach before attaching another or when done using it.
• showing ubi volumes that exist in a ubi image:

  Ventana > ubi info layout
  Volume information dump:
          vol_id          0
          reserved_pebs   124
          alignment       1
          data_pad        0
          vol_type        3
          name_len        4
          usable_leb_size 126976
          used_ebs        124
          used_bytes      15745024
          last_eb_bytes   126976
          corrupted       0
          upd_marker      0
          name            boot
  Volume information dump:
          vol_id          1
          reserved_pebs   210
          alignment       1
          data_pad        0
          vol_type        3
          name_len        6
          usable_leb_size 126976
          used_ebs        210
          used_bytes      26664960
          last_eb_bytes   126976
          corrupted       0
          upd_marker      0
          name            rootfs
  Volume information dump:
          vol_id          2
          reserved_pebs   15586
          alignment       1
          data_pad        0
          vol_type        3
          name_len        11
          usable_leb_size 126976
          used_ebs        15586
          used_bytes      1979047936
          last_eb_bytes   126976
          corrupted       0
          upd_marker      0
          name            rootfs_data
  Volume information dump:
          vol_id          2147479551
          reserved_pebs   2
          alignment       1
          data_pad        0
          vol_type        3
          name_len        13
          usable_leb_size 126976
          used_ebs        2
          used_bytes      253952
          last_eb_bytes   2
          corrupted       0
          upd_marker      0
          name            layout volume
  

  • The above shows there are 3 ubi volumes in the attached ubi image named ‘boot’, ‘rootfs’, ‘rootfs_data’. Note the last dump is the volume layout (think of it as a directory table)
  • Note the contents of the various ubi volumes is unknown by the ubi layer
• once a ubi volume is mounted ‘if’ it is a ubifs you can use ubifsmount, ubifsload, ubifsls, ubifsumount to operate on its contents:

  Ventana > ubifsmount ubi0:boot
  Ventana > ubifsls
  <LNK>          31  Tue Jun 23 23:51:55 2020  imx6q-gw53xx.dtb
              38158  Tue Jun 23 23:47:33 2020  gateworks-imx6-imx6q-gw5400-a.dtb
              41991  Tue Jun 23 23:47:30 2020  gateworks-imx6-imx6dl-gw53xx.dtb
              43296  Tue Jun 23 23:47:33 2020  gateworks-imx6-imx6q-gw53xx.dtb
  <LNK>          32  Tue Jun 23 23:51:55 2020  imx6dl-gw53xx.dtb
  ...
            3275024  Tue Jun 23 23:51:55 2020  gateworks-imx6-uImage
  <LNK>          21  Tue Jun 23 23:51:55 2020  uImage
  Ventana > ubifsload $loadaddr uImage
  Loading file 'uImage' to addr 0x12000000...
  Done
  Ventana > ubifsumount
  Unmounting UBIFS volume boot!
  

  • Again you can only use ubifs commands on ubi volumes that contain ubifs filesystems
  • Be sure to unmount your volume when done working with it
• removing ubi volumes

  Ventana > ubi remove rootfs_data
  Remove UBI volume rootfs_data (id 2)
  Ventana > ubi remove rootfs
  Remove UBI volume rootfs (id 1)
  

• creating ubi volumes

  Ventana > ubi create rootfs a00000 static
  Creating static volume rootfs of size 10485760
  

• writing ubi volumes

  Ventana > tftp $loadaddr ventana/test_large.ubifs
  Using FEC device
  TFTP from server 192.168.1.146; our IP address is 192.168.1.1
  Filename 'ventana/test_large.ubifs'.
  Load address: 0x12000000
  Loading: #################################################################
           #################################################################
           #################################################################
           #################################################################
           #################################################################
           #################################################################
           #################################################################
           #################################################################
           #################################################################
           #################################################################
           #################################################################
           ############
           8.1 MiB/s
  done
  Bytes transferred = 10665984 (a2c000 hex)
  Ventana > ubi remove rootfs
  Remove UBI volume rootfs (id 1)
  Ventana > ubi create rootfs $filesize static                      
  Creating static volume rootfs of size 10665984
  Ventana > ubi write $loadaddr rootfs $filesize
  10665984 bytes written to volume rootfs
  

Updating NAND FLASH with a UBI image

Video:

• ​Getting started with Screen and flashing with TFTP

If your ubi image is on a tftp server as image.ubi you can use the following to update the ‘rootfs’ NAND FLASH partition in U-Boot:

Preferred method using the nand_update script will perform the update for you using the image name specified in the image_rootfs env variable.
Run the following commands in the bootloader:

setenv ipaddr 192.168.1.1
setenv serverip 192.168.1.254
setenv image_rootfs image.ubi; 
run nand_update

It may also be necessary to set a gateway or netmask IP address. This can be done using the following commands:

setenv gatewayip <ipaddress>
setenv netmask <ipaddress>

The more manual method without using the nand_update script, if your ubi image is on a tftp server as image.ubi you can use the following to update the ‘rootfs’ NAND FLASH partition in U-Boot (update the ip addresses as appropriate for your network environment):

setenv ipaddr 192.168.1.1
setenv serverip 192.168.1.254
tftpboot ${loadaddr} image.ubi && 
  nand erase.part rootfs && 
  nand write ${loadaddr} rootfs ${filesize}

If your ubi image is on the first partition of a FAT formated MMC called image.ubi you can do the following:

mmc rescan
fatload mmc 0:1 ${loadaddr} image.ubi && 
  nand erase.part rootfs && 
  nand write ${loadaddr} rootfs ${filesize}

If your ubi image is on the first partition of a ext2/3/4 formated MMC called image.ubi you can do the following:

mmc rescan
ext2load mmc 0:1 ${loadaddr} image.ubi && 
  nand erase.part rootfs && 
  nand write ${loadaddr} rootfs ${filesize}

If your ubi image is on the first partition of a FAT formatted USB stick called image.ubi you can do the following:

usb start
usb dev 0
fatload usb 0:1 ${loadaddr} image.ubi && 
  nand erase.part rootfs && 
  nand write ${loadaddr} rootfs ${filesize}

Note that operating on the ubi device as described here destroys the wear-leveling information contained in the UBI therefore it is recommended to work at the ubi volume layer to replace ubifs volumes as described below

Updating NAND FLASH with a ubifs image

When you want to preserve RAW nand wear-leveling you must operate at the ubifs layer (filesystem layer vs ubi layer) so that wear leveling stored in the ubi is maintained.

To update the contents of the rootfs volume with a ubifs in U-Boot:

ubi part rootfs # select the 'rootfs' partition (from mtdparts env var) for ubi
ubi info layout # list volumes
ubi remove rootfs # remove rootfs volume
ubi create rootfs a00000 static # create a 10MB volume named rootfs
tftpboot ${loadaddr} ventana/root.ubifs && ubi write ${loadaddr} rootfs ${filesize}
ubifsmount ubi0:rootfs # mount rootfs volume from ubi
ubifsls /
ubifsumount

Operating on a NAND device at the ubi volume layer like this preserves the wear-leveling information for your flash device and is therefore recommended, especially if you are doing this often.

Use the information in the section above for examples on how to load data in uboot from various sources such as network, mmc, usb.

Updating UBI with low RAM devices

For some devices, the size of the NAND flash exceeds the size of RAM. In this scenario the instructions from the prior section would fail as you would run out of RAM space before you could finish retrieving the file to program the NAND flash with.

In order to flash a UBI in this situation within U-Boot you must either split the UBI image into chunks and transfer/flash each chunk, or you can use U-Boot to create a UBI volume and split the UBIFS and transfer/flash each chunk of the UBIFS. Flashing a UBI has the disadvantage of erasing all the NAND wear leveling information the UBI has been tracking and won’t support the special handling of bad blocks. Therefore it is recommended that you use the split UBIFS method which preserves wear leveling info and works with bad blocks.

Split UBIFS flashing

The following example shows how you can take a UBIFS and split it into chunks that fit into low memory devices and flash them via U-Boot. The example uses tftp to transfer but you can easily replace that instruction with one that loads the chunks from removable storage:

1. First you must split your ubifs image on your Linux development host and note the total size of the pre-split image which you will need during flashing. This example assumes a ~580MB ubifs that we will split into 200MB chunks to fit within a 256MB DRAM system:

   $ ls -l focal-ventana_normal.ubifs
   -rw-rw-r-- 1 gateworks user 581042176 Sep  9 14:56 focal-ventana_normal.ubifs
   $ split -d -b 200M focal-ventana_normal.ubifs focal-ventana_normal.ubifs.part
   $ ls -l focal-ventana_normal.ubifs*
   -rw-rw-r-- 1 gateworks user 581042176 Sep  9 14:56 focal-ventana_normal.ubifs
   -rw-rw-r-- 1 gateworks user 209715200 Sep  9 15:19
   focal-ventana_normal.ubifs.part00
   -rw-rw-r-- 1 gateworks user 209715200 Sep  9 15:19
   focal-ventana_normal.ubifs.part01
   -rw-rw-r-- 1 gateworks user 161611776 Sep  9 15:20
   focal-ventana_normal.ubifs.part02
   $ printf "0x%xn" 581042176
   0x22a20000
   

2. Now boot your target board and break into U-Boot. Use the mtd command to determine the MTD partition name that you wish to use for UBI. This is dictated by the mtdids and mtdparts env variables. The gateworks bootloader defaults the name of the MTD partition used for UBI to ‘rootfs’ as shown below:

   Ventana > mtd
   
   device nand0 <nand>, # parts = 3
    #: name                size            offset          mask_flags
    0: uboot               0x01000000      0x00000000      0
    1: env                 0x00100000      0x01000000      0
    2: rootfs              0x7ef00000      0x01100000      0
   
   active partition: nand0,0 - (uboot) 0x01000000 @ 0x00000000
   ...
   

3. Use the ubi command to attach the rootfs MTD partition to the ubi layer. The rootfs name here must agree with the partition name from the MTD partition as described above… this is ‘not’ necessarily the name of your UBI ‘volume’ used below:

   Ventana > ubi part rootfs
   

4. Use the ubi info layout command to inspect your UBI volumes. You will always have one volume named layout volume which holds the UBI volume information.

   Ventana > ubi info layout
   Volume information dump:
           vol_id          0
           reserved_pebs   15920
           alignment       1
           data_pad        0
           vol_type        3
           name_len        6
           usable_leb_size 126976
           used_ebs        15920
           used_bytes      2021457920
           last_eb_bytes   126976
           corrupted       0
           upd_marker      0
           name            rootfs
   Volume information dump:
           vol_id          2147479551
           reserved_pebs   2
           alignment       1
           data_pad        0
           vol_type        3
           name_len        13
           usable_leb_size 126976
           used_ebs        2
           used_bytes      253952
           last_eb_bytes   2
           corrupted       0
           upd_marker      0
           name            layout volume
   

   • The above shows that there is one data volume named ‘rootfs’
5. remove the existing volume (if you are operating on a blank UBI and only have layout volume skip this step):

   Ventana > ubi remove rootfs
   Remove UBI volume rootfs (id 0)
   

6. Create the new volume. Note that the volume name you use here is the name you would pass to the kernel for mounting. We typically call this ‘rootfs’ (which should not be confused with the MTD partition name).

   Ventana > ubi create rootfs
   No size specified -> Using max size (2021457920)
   Creating dynamic volume rootfs of size 2021457920
   

7. Load your first chunk:

   Ventana > tftpboot $loadaddr ventana/focal-ventana_normal.ubifs.part00
   ...
   done
   Bytes transferred = 209715200 (c800000 hex)
   

   • You could alternatively use the load command to load from removable storage such as microSD or USB Mass storage but you will need to set the $filesize variable appropriately.
8. Write your first chunk — this first chunk you need to provide the total size of the pre-split UBIFS image as the last argument in hex:

   Ventana > ubi write.part $loadaddr rootfs $filesize 0x22a20000
   209715200 bytes written to volume rootfs
   

   • Note the last argument 0x22a20000 which is the total size of the pre-split ubifs image in hex.
9. Continue the transfer and flash of your additional chunks, this time leaving off the last argument:

   Ventana > tftpboot $loadaddr ventana/focal-ventana_normal.ubifs.part01
   ...
   done
   Bytes transferred = 209715200 (c800000 hex)
   Ventana > ubi write.part $loadaddr rootfs $filesize
   209715200 bytes written to volume rootfs
   Ventana > tftpboot $loadaddr ventana/focal-ventana_normal.ubifs.part02
   ...
   done
   Bytes transferred = 161611776 (9a20000 hex)
   Ventana > ubi write.part $loadaddr rootfs $filesize
   161611776 bytes written to volume rootfs
   

Split UBI flashing

Gateworks has created an installable U-Boot script that will apply UBI files containing a single partition image. This can be particularly useful for updating the firmware of a running target.

This method does not work if you have bad blocks and also erases previous wear-leveling data that may be tracked by the previous UBI image. The split UBIFS method above is recommended instead.

To update using a split UBI:

1. Split your UBI

In some situations the RAM of your device will be smaller than the image file required to update your system, therefore it is necessary to split the image file and apply the resulting compressed parts in a piece wise update. The below example uses the split command to split the input file into parts that are reasonably smaller than the RAM size of the board which will be consuming the update.

FILE=xenial-large.ubi
# Split file every 200M with suffix ".part" and decimal increment
split -d -b 200M ${FILE} ${FILE}.part

# Result will be multiple files such as xenial-large.ubi.partXX that are <= 200M and
# which adhere to the naming convention required by the script (ends with .partXX in sequential order)

2. Install the U-Boot script

On the target machine, source the ​nand_split_update.scr U-Boot script attached to this page (or copy its contents with an editor). For example via tftp:

setenv serverip 192.168.1.100
tftpboot nand_split_update.scr
source $loadaddr

3. Configure your U-Boot environment

From the script’s usage:

The following environment variables need to be set in order for this script to run:

    splitfile    - full path file name prefix of the split files (eg file.ubi for file.ubi.part00)

An example configuration:

setenv splitfile xenial-large.ubi          # ubi file from example above

4. Run the added script with run nand_split_update

UBI Fastmap

UBI Fastmap is on optional feature which stores the physical to logical eraseblock relations in a checkpoint (called fastmap) to reduce the initialization time of UBI. Without UBI Fastmap, the init time of UBI is proportional to the number of physical erase blocks on the FLASH device and with Fastmap enabled the scan time is limited to a fixed number of blocks.

While enabling UBI Fastmap could save several seconds in Linux init as well as several seconds in U-Boot ubi attach (depending on number of logical flash blocks) the downside is that you wear out the first 64blocks of your NAND faster. Therefore the Gateworks kernels and bootscripts do not enable this by default.

If you wish to enable this feature because boot time is more critical than NAND lifetime you can do so by:

• enable MTD_UBI_FASTMAP in U-Boot (to use the UBI Fastmap if it was created by the kernel)
• enable MTD_UBI_FASTMAP in Linux kernel
• pass ‘ubi.fm_autoconvert=1’ via kernel params to instruct the kernel to create a UBI Fastmap if it does not yet exist and use it if it does exist
• Note that enabling MTD_UBI_FASTMAP in the kernel/uboot won’t enable it unless ‘ubi.fm_autoconvert=1’ was used on the kenrel

alouker.d:

Доброго дня!
В соответствии с инструкцией http://www.zao-zeo.ru/dokuwiki/doku.php/orion28
пытаюсь записать новую КФС на Орион-28.
На шаге
  ubidetach /dev/ubi_ctrl -m 1[/b]
получаю ошибку
  ubidetach: error!: cannot detach mtd1
    error 16 (Device or resource busy)

Подскажите пожалуйста как мне правильно записать новую КФС на Орион-28.
Может есть какие нибудь альтернативные варианты, типа как загрузка с USB и запись при помощи MfgTool.exe, как для Тион-Про28, или иные способы?

Danila:

Добрый день.
Проверю инструкцию, напишу результат.

Другой способ записи — вскрыть корпус(он не опломбирован), найти разъем Х8 (USB), записать через mfgtool, как для ТионПро28.
Для записи понадобится планка USB и кабель USB AF-AF.

Внимательно подключаться к разъему Х8. Если перепутать полярность — порт сгорит. Первый контакт помечен белой чертой на плате.

Regressor:

А вот у меня не выходит. Попытка обновить систему на свежескомпиленую привела к «error!: cannot detach mtd1» после чего орион28 не загружается и зажигает 2 красных лампы. Далее согласно инструкции перемычка на j3 и шнур usb af-af. После включения устройство ROM Recovery компьютер видит и ставит драйвер hid. mfgtool загружать обновление не желает и выдает ошибку:

— Код: —1 — Panel A Start processing Singlechip NAND <LIST/>.
Panel A Start <CMD/> type=»boot» body=»Recovery» file=»updater.sb» timeout=»60″ onError=»» text=»Booting update firmware.».
Panel A: Operation stopped by user.,1 — Panel A Finished <CMD/> type=»boot» body=»Recovery» file=»updater.sb» timeout=»60″ onError=»» text=»Booting update firmware.» SUCCESS code=0.
Panel A Start <CMD/> type=»boot» body=»Recovery» file=»updater_ivt.sb» timeout=»60″ onError=»» text=»Booting update firmware.».
1 — Panel A DoBoot() — Failed to load updater_ivt.sb to Recovery mode device. (err=2)
1 — Panel A Finished <CMD/> type=»boot» body=»Recovery» file=»updater_ivt.sb» timeout=»60″ onError=»» text=»Booting update firmware.» FAIL code=2.

— Конец кода —

Подозреваю, что с updater_ivt.sb и updater.sb что-то не так — наверное нужны для ориона28 какие-то специальные ?

Danila:

Нужно записывать через USB 1.1. Через 2.0 не получится. Ограничьте скорость в bios или отключите драйвер расширенного хост-контроллера USB для нужного порта.

Если не поможет — нужно смотреть логи из отладочного порта.

UBI FAQ and HOWTO

Table of contents

1. How do I enable UBI?
2. How do I attach an MTD device?
3. How do I create/delete UBI volumes?
4. How do I run JFFS2 on top of an UBI volume?
5. Can I run ext2 on top of UBI?
6. Can I run squashfs on top of UBI?
7. Do I have to format my empty flash before running UBI on top of it?
8. How do I erase flash and preserve erase counters?
9. How do I create UBI images?
10. How do I find out min. I/O unit size, sub-page size, and LEB size?
11. How do I flash UBI images and preserve erase counters?
12. Can UBI logical eraseblocks be written randomly?
13. Why UBI does not use OOB area of NAND flashes?
14. Is UBI tolerant of power failures?
15. What happens when the PEBs reserved for bad block handling run out?
16. May UBI be used on MLC flash?
17. Why does ubiattach on a freshly formatted device fail with «Invalid argument»?
18. What is a sub-page?
19. I get «ubi_io_write: error -5 while writing 512 bytes to PEB 5:512»
20. I get «no VID header found at PEB 7923, only 0xFF bytes»
21. I get: «ubi_io_read: error -74 (ECC error) while reading 126976 bytes from PEB 47:4096, read 126976 bytes»
22. How do I force UBI to ignore sub-pages?
23. How do I implement UBI flasher?
24. What does the «ubi_bgt0d» thread do?
25. How do I speed up UBI initialization
26. Why a dynamic volume is faster to access than a static volume of the
    same size?
27. How do I debug UBI?
28. How do I send an UBI bug report?

How do I enable UBI?

In the Linux configuration menu, go to «Device Drivers» ->
«Memory Technology Devices (MTD)» -> «UBI — Unsorted block images«,
and mark the «Enable UBI» check-box. UBI may be either compiled into the
kernel or be built as a kernel module.

How do I attach an MTD device?

Each MTD device has a name and a number, which you may find out by
examining the /proc/mtd file. The preferable way to attach
MTD devices is to attach them by name, not by number, because MTD device
numbers may change if you change the layout of your flash, while the
names will supposedly stay the same.

If UBI is compiled as a kernel module, it is enough to specify the MTD
device name or number to attach in the module arguments, e.g.

$ modprobe ubi mtd=rootfs
$ modprobe ubi mtd=0

loads the UBI kernel module and attaches MTD device named «rootfs» or with
number 3 (mtd3). And

$ modprobe ubi mtd=config mtd=rootfs
$ modprobe ubi mtd=3 mtd=5

command loads UBI kernel module and attaches MTD devices named «config»
and «rootfs», or mtd3 and mtd5.

If UBI is compiled into the kernel, the mtd device to attach may be
specified in the ubi.mtd=kernel boot parameter, e.g.,

ubi.mtd=rootfs
ubi.mtd=0

command makes UBI attach MTD device named «rootfs» or mtd3 when
the kernel is booting. And

ubi.mtd=config ubi.mtd=rootfs
ubi.mtd=3 ubi.mtd=5

command makes UBI attach MTD devices named «config» and «rootfs», or
mtd3 and mtd5.

And finally, MTD devices may be attached or detached at any time with the
ubiattach and ubidetach
utilities; For example,

$ ubiattach /dev/ubi_ctrl -m 3

attaches mtd3. But these utilities will work with kernel
versions starting from version 2.6.25. And someone should
update them and teach to accept MTD device names, not only MTD device
numbers.

How do I create/delete UBI volumes?

Use the ubimkvol and ubirmvol
utilities. For example, the below command
creates an 128MiB volume on UBI device 0:

$ ubimkvol /dev/ubi0 -N rootfs -s 128MiB

and the following command removes it:

$ ubirmvol /dev/ubi0 -n 0

For additional information, use ubimkvol -h and
ubirmvol -h.

How do I run JFFS2 on top of an UBI volume?

There is an additional driver called gluebi which can emulate
fake MTD devices for each UBI volume and JFFS2 can be used with these emulated
MTD devices. Enable gluebi in the UBI configuration menu (the
«MTD devices emulation driver (gluebi)» option).

Note, pre-2.6.31 kernels did not have a separate
gluebi driver and it was built into the UBI driver. The
corresponding UBI configuration menu option was
«Emulate MTD devices«

Can I run ext2 on top of UBI?

UBI is not a block device emulation layer, it is not an FTL. Neither ext2
nor other «traditional» file systems can be run on top of an UBI device.
Please, read the big red note
and overview documentation sections to
realize why.

However, given UBI takes care of many flash complexities, it provides a
bad-block-free block device access on top of UBI volumes. This feature is useful to
mount read-only file systems.

Can I run squashfs on top of UBI?

Yes. UBI allows to create a read-only block device on top of a UBI volume
which is suitable for read-only, block-oriented file systems, such as squashfs.
See the UBI block device section for more details.

Do I have to format my empty flash before running UBI on top of it?

Ideally, yes, but not necessarily.

Roughly speaking, «UBI-formatted MTD device» has each PEB erased and
containing a valid EC header.
However, it is not always possible because many a lot of existing software is
not UBI-aware and cannot preserve erase counters. Often all it can do is to wipe
out the flash. This is why UBI can to deal with empty flash perfectly well — it
just automatically formats it.

So the answer is no, you do not have to. For example, if you wipe out the
flash and try to attach it to UBI — it will work. UBI will just automatically
format the flash. The formatting process takes time, though.

But keep in mind that every time you erase the flash, you lose erase
counters, so you lose wearing information. Doing this over and over again may
wear out some eraseblocks. This is especially dangerous on MLC NAND flashes
which have very low eraseblock life-cycle. Try to use UBI-aware utilities and
flashing programs.

How do I erase flash and preserve erase counters?

Use the ubiformat
utility. Example:

$ ubiformat /dev/mtd0
ubiformat: mtd0 (NAND), size 536870912 bytes (512.0 MiB), 131072 eraseblocks of
131072 bytes (128.0 KiB), min. I/O size 2048 bytes
libscan: scanning eraseblock 4095 -- 100 % complete
ubiformat: 4094 eraseblocks have valid erase counter, mean value is 104
ubiformat: bad eraseblocks: 13, 666
ubiformat: formatting eraseblock 4095 -- 100 % complete

Note! If you use an old kernel, which is older than 2.6.30 and where
MTD does not support sysfs, you have to pass «-s» argument to
ubiformat.

This section has some hints
for those who implement a custom UBI flasher.

How do I create UBI images?

UBI images may be created using the
ubinize utility. This
utility takes the configuration file on input and generates an UBI image on
output. The input configuration file describes all UBI volumes which the
resulting UBI image has to contain. The configuration file has the
ini-file syntax. Here is an example:

$ cat config.ini
[configuration-data-volume]
mode=ubi
image=config_data.img
vol_id=0
vol_size=512KiB
vol_type=static
vol_name=configuration

[rootfs-volume]
mode=ubi
image=rootfs.img
vol_id=1
vol_size=220MiB
vol_type=dynamic
vol_name=rootfs
vol_flags=autoresize

$ ./ubinize -o ubi.img -p 128KiB -m 512 -s 256 config.ini

The config.ini file tells UBIFS to create 2 volumes:

• static configuration volume of 512KiB in size, assign it ID 0 and
  name «configuration»; the contents of the volume should be taken from
  the config_data.img file;
• dynamic root file-system volume of 220MiB in size, assign it ID 1
  and name «rootfs»; the contents of the volume should be taken from
  the rootfs.img file; this volume should also have
  «auto-resize» flag which means the size of this volume will be amended
  when UBI runs for the first time; namely, UBI will make this volume
  larger by giving available eraseblocks; this may be very useful in case
  of NAND flash (see here for
  more details).

So in the above example, ubinize basically reads 3 input
files:

• The config.ini file which describes how many volumes
  should the resulting ubi.img file contain, their sizes,
  names, and so on; it also refers the files containing the data which
  should be put to the volumes; note, if the volume is supposed to be
  empty, just do not specify the image file;
• the config_data.img image file for the first
  volume;
• the rootfs.img image file for the second volume.

Users often wonder why ubinize needs a configuration file. The
answer is that one UBI image may contain many UBI volumes with different
characteristics and it is difficult to invent a nice command-line interface for
specifying all those characteristics. Thus a configuration file is used. But
feel free sending a patch which teaches ubinize working without
the configuration file if there is only one volume.

Note, UBI reserves physical flash space for volumes. Namely, UBI
reserves a physical eraseblock for each logical eraseblock. The size if LEB
130560 bytes in our example (found out by running ubinize
with -v option), which means that the configuration volume will
have 5 LEBs ([512 * 1024] / 130560 rounded up) and the root file-system
volume will have at least 1767 LEBs. This means that the MTD device the
ubi.img is going to be flashed to has to have at least 1772
physical eraseblocks, which is about 221MiB. But because of the UBI overhead
(see this section), the MTD device
actually has to be at least 225MiB in size. Of course it may be larger,
in which case the «rootfs» volume will be re-sized and take the rest of the
flash space (because of the auto-resize flag).

The implications of the above paragraph are important. The
vol_size option effectively represents the minimum size of the
flash where the volume will be installed. If you are working with multiple
devices (i.e. you are producing an image to be flashed on various devices,
even when ‘identical’), the amount of usable flash will vary because
some devices have more bad blocks than others. Excluding the
vol_size option will cause vol_size to be automatically
calculated based on the size of the input image, and this will produce
maximum robustness in the face of varying numbers of bad blocks on target
devices. You can combine this with the autoresize functionality so that the
maximum amount of free space is made available upon first mount.

Also, the config_data.img and rootfs.img input
files do not have to be 512KiB and 220MiB respectively, but may be smaller if
they contain less data. In this case the resulting ubi.img file
will also be smaller than 221MiB. All the ubinize utility does is
it takes the image files, splits them to LEB-sized chunks, forms PEB data by
adding UBI headers to these LEB
chunks, and writes the result to the output file. It also writes the
volume table (2 physical
eraseblocks). Thus, ubi.img file size will be small if the input
volume images are small. And ubinize does not do any further
padding.

Please, use the Use the
ubiformat utility to
flash the resulting UBI image. Also, you can find detailed description of how
UBI-aware flashing programs should work in
this section.

How do I find out min. I/O unit size, sub-page size, and LEB size?

If you run kernel version 2.6.30 or later, the easiest
way to find out all these parameters is to run the
mtdinfo
tool, which reports them. Of course, the tool has to be run on the
target.

Otherwise, first find out the physical eraseblock (PEB) size from
the flash manual, or from the /proc/mtd file on the target.

Refer this and
this sections for some hints about
what the min. I/O unit and sub-page sizes may be for you.

The LEB size is defined by:

• physical eraseblock size;
• minimum input/output unit size;
• sub-page size;

Please, read this section for
information about how LEB size is calculated (it is denoted by O
there).

How do I flash UBI images and preserve erase counters?

Use the ubiformat
utility. Example:

$ ubiformat /dev/mtd0 -f ubi.img
ubiformat: mtd0 (NAND), size 536870912 bytes (512.0 MiB), 131072 eraseblocks of 131072 bytes (128.0 KiB), min. I/O size 2048 bytes
libscan: scanning eraseblock 4095 -- 100 % complete
ubiformat: 4094 eraseblocks have valid erase counter, mean value is 105
ubiformat: bad eraseblocks: 13, 666
ubiformat: flashing eraseblock 50 -- 100 % complete
ubiformat: formatting eraseblock 4095 -- 100 % complete

Note, this section has
describes how UBI-aware flasher program has to work.

Important!: even if you do not care about erasecounters, do not use
dd for flashing UBI images to NAND flashes, because
dd does handle bad eraseblocks.

Important!: even if you do not care about erasecounters, do not use
nandwrite for flashing UBI images containing UBIFS file-system to
NAND flashes, read here for
more information.

Can UBI logical eraseblocks be written randomly?

No, the flash chip restrictions have to be taken into account. This is
because UBI logical eraseblocks (LEB) are mapped to physical eraseblocks (PEB),
and an LEB write operation is essentially a write to the corresponding PEB
plus a small offset. The important flash restrictions are:

1. all writes have to be aligned to the
   min. I/O unit size and be
   multiple of min. I/O unit size;
2. sometimes it is prohibited to write more then once to the same PEB
   offset (e.g., in case of NAND flash);
3. many NAND flashes (specifically, MLC NAND flashes) require NAND
   pages to be written sequentially from the beginning of the physical
   eraseblock, to the end of the physical eraseblock; for example, it is
   prohibited to first write to offset 2048, then to offset 0; once offset
   2048 has been written to, it is possible to write only to further
   offsets.

Even if the flash chip is devoid of the last restriction, UBI anyway
requires logical eraseblocks to be written sequentially from the beginning
to the end. This is because UBI calculates data CRC when moving logical
eraseblocks to other physical eraseblocks (see
here for more information),
so a write an offset which is less than the furthest written data offset causes
CRC errors;

Why UBI does not use OOB area of NAND flashes?

Because many flashes (e.g., NOR) do not have OOB and UBI was designed to be
generic. Also, modern MLC NAND flashes use whole OOB area for the ECC
checksum, so there is no room for application data.

But of course, things could be optimized for SLC NAND flashes if UBI used
the space available in the OOB area. This is not implemented, but one could
probably do this.

Is UBI tolerant of power failures?

Yes, UBI is designed to be tolerant of power failures and unclean
reboots.

What happens when the PEBs reserved for bad block handling run out?

By default, about 2% of the whole chip size (20/1024 PEB) are reserved for
bad blocks handling.
If the number of blocks that turn bad exceeds that allocation, an error
message will be printed and UBI will switch to read-only mode.

Note: If at attach time, there’s already more bad blocks than reserved PEBs,
UBI will stay in read-write mode. The switching to read-only mode only occurs
when a new bad block appears.

May UBI be used on MLC flash?

Yes, it may, as long as the flash is supported by the MTD layer. UBI does
not use OOB and it requires data to be written sequentially (see
here). UBI guarantees that the difference
between maximum and minimum erase-counters is withing certain threshold,
which is 4096 by default. Since MLC flashes have quite low eraseblock
life-cycle (about 1000-10000, unlike 100000-1000000 for SLC NAND and NOR
flashes), the threshold has to be set to a lower value (e.g., 256). This may be
done via the Linux kernel configuration menu.

Note, unlike UBI, JFFS2 uses random wear-leveling algorithm, which is in
fact not completely random, because JFFS2 makes it more probable to
garbage collect eraseblocks with more dirty data. This means that JFFS2 is not
really appropriate for MLC flashes. However, it is possible to use JFFS2
file-system on top of UBI (see this
section) to improve wear-leveling.

Why does ubiattach on a freshly formatted device fail with «Invalid argument»?

On NAND devices that support sub-page accesses, ubiformat
may choose a different location for the
VID header to the kernel UBI driver.
This can result in the following error when attaching to a UBI device:

$ ubiformat /dev/mtd0
ubiformat: mtd0 (NAND), size 260046848 bytes (248.0 MiB), 131072 eraseblocks of 131072 bytes (128.0 KiB), min. I/O size 2048 bytes
[...]
$ ubiattach /dev/ubi_ctrl -m 0
ubiattach: error!: cannot attach mtd0
           error 22 (Invalid argument)

and in dmesg you will see:

UBI error: validate_ec_hdr: bad VID header offset 2048, expected 512
UBI error: validate_ec_hdr: bad EC header
UBI error: ubi_io_read_ec_hdr: validation failed for PEB 0

This happens because ubiformat assumes the flash does not
support sub-pages, because the kernel does not expose sub-page information
to user-space (which should be fixed when sysfs support is added to MTD).
However, the kernel UBI driver assumes sub-pages are supported and sub-page
size is 512 bytes in our example. To fix this, you should override the default
sub-page size that ubiformat uses to what the kernel expects using
the -s option of ubiformat. For example, if you see
the error above in dmesg, you can tell ubiformat to assume
512-byte sub-page by executing:

$ ubiformat /dev/mtd0 -s 512

Or you may pass «-O 512» would have the same effect as
«-s 512» — the VID header would be put at offset 512.

Alternately, you may wish to actually attach to the UBI device by forcing
VID header offset to be 2048 bytes. In other words, you may ask UBI to avoid
using sub-pages. This is not recommended since this will require more storage
overhead, but may be useful if your NAND driver incorrectly reports that it can
handle sub-page accesses when it should not. To do this with
ubiattach, use:

$ ubiattach /dev/ubi_ctrl -m 0 -O 2048

or on the kernel command-line, pass:

ubi.mtd=0,2048

What is a sub-page?

Please, refer to this section.

I get «ubi_io_write: error -5 while writing 512 bytes to PEB 5:512»

If you have a 2048 bytes per NAND page device, and have
CONFIG_MTD_NAND_VERIFY_WRITE enabled in your kernel, you will need
to turn it off. The code did not currently (as of 2.6.26) perform verification
of sub-page writes correctly. As UBI is one of the few users of sub-page
writes, not much else seemed to be affected by this bug. In Linux 3.7,
CONFIG_MTD_NAND_VERIFY_WRITE was removed:

commit 657f28f8811c92724db10d18bbbec70d540147d6
Date:   Tue Aug 14 22:38:45 2012 -0400

    mtd: kill MTD_NAND_VERIFY_WRITE

    "Both UBI and JFFS2 are able to read verify what they wrote already.
    There are also MTD tests which do this verification. So I think there
    is no reason to keep this in the NAND layer, let alone wasting RAM in
    the driver to support this feature. Besides, it does not work for sub-pages
    and many drivers have it broken. It hurts more than it provides benefits."

    So kill MTD_NAND_VERIFY_WRITE entirely.

I get «no VID header found at PEB 7923, only 0xFF bytes»

The messages mean that UBI could not find
VID header in the eraseblock, but
the header supposed to be there. This probably means some corruption.

However, if you have UBI «build» debugging messages enabled,
(CONFIG_MTD_UBI_DEBUG_MSG_BLD=y), you may see a lot of these
messages and they are harmless. They are just debugging messages in this
case.

I see this UBI error: «ubi_io_read: error -74 (ECC error) while reading 126976 bytes from PEB 47:4096, read 126976 bytes»

The -74 error code is -EBADMSG and means an ECC
error. In other words, UBI tried to read some data from the flash, but the
flash driver found that there is an uncorrectable ECC error, and returned
-EBADMSG.

There may be many reasons for this. It may be because your NAND driver is
buggy, or you HW is buggy. We recommend you to validate the driver using the
MTD tests.

The other possibility is that you failed to flash your UBI/UBIFS image
properly. Try to erase your flash, then attach it to UBI/UBIFS without writing
any image, and check if you still have these errors.

If you do not have errors when you mount empty flash, it is probably indeed
related to how you flash the UBI/UBIFS images. One typical problem is related
to ECC calculation algorithm — read
here for more information. Make sure
that you use ubiformat, or make sure
either that your flashing program skips 0xFF properly (see
here) or that your UBIFS image
was generated with the «free space fixup» flag set (see
here).

Another possibility is that your flash reports that it supports
sub-pages, but does not actually support
them properly.

How do I force UBI to ignore sub-pages?»

If your NAND flash supports
sub-pages, UBI will use them. But
sometimes you may want to ask UBI to ignore sub-pages, for example if you have
an UBI image which was made for a similar NAND flash, but without sub-pages, or
if you just want to disable sub-pages for testing purposes. You may do this
by forcing UBI to read/write the VID header from/to the beginning of the second
page, instead of the second sub-page (which is the default behavior). And UBI
uses sub-pages only for VID headers, so this is enough.

If you attach MTD devices using the
ubiattach tool — just
use the --vid-hdr-offset option and specify the second page
offset. For example, if you have 2048-byte NAND pages, use
--vid-hdr-offset 2048. All the below examples also assume
2048-byte NAND pages.

If you do not use ubiattach and instead specify the MTD device
to attach using kernel boot parameters (see
here), then you may put VID header offset
after the MTD device name or number. For example:

$ ubi.mtd=rootfs,2048
$ ubi.mtd=3,2048

will attach MTD device named «rootfs» (or MTD device number 3) and force
UBI to read/write VID headers from/to offset 2048, which is the second NAND
page in the eraseblock.

Similarly, if you use module load parameters, try

$ modprobe ubi mtd=rootfs,2048
$ modprobe ubi mtd=3,2048

And of course, if you have an MTD partition which is already formatted
so that sub-pages are used, which means all VID headers sit at offsets 512
(second sub-page of a 2048-byte NAND page), you cannot force UBI to stop using
sub-pages (it will fail). You may do this only if you are attaching an empty
MTD partition, or if the flash is already formatted so that all VID headers
are at offset 2048. This means that if you use or
ubinize, you should
avoid using the -s parameter.

How do I implement UBI flasher?

Please, read here

What does the «ubi_bgt0d» thread do?

The UBI back-ground thread is a per-UBI device thread which has
«ubi_bgtXd» name, where «X» is the UBI device number. For
example, «ubi_bgt0d» is a background thread corresponding to UBI
device 0.

The UBI background thread is doing background physical eraseblock erasure.
This is an important optimization which greatly improves UBI I/O throughput
(applications do not have to wait for erasure completion). For example,
UBI unmap operation schedules physical
eraseblocks for erasure.

The background thread also
tortures faulty physical
eraseblocks.

The UBI background thread also moves data from more worn-out physical
eraseblocks to less worn out, i.e., performs wear-leveling. It also moves data
from physical eraseblocks which have bit-flips. See the
UBI overview section for some more
information.

Note, UBI may work without the background thread, so the thread is just an
optimization, although a very important one.

How do I speed up UBI initialization

As this section explains, UBI
has scalability limitations imposed by its design, so there is no much you
can do about it but without re-designing. Here are some ideas

• When UBI attaches an MTD device, it reads a lot of data from the
  media. If you manage to improve read speed on the driver level, you
  will also speed up UBI initialization.
• The MTD layer supplies upper layers (including UBI) with information
  about bad PEBs. It keeps so-called bad block table in RAM, which is
  usually 1 bit per PEB. When the driver initializes, it has to build
  this table by scanning whole flash media, which normally includes
  reading OOB area of 1st NAND page of each PEB. This takes time and may
  be improved by using on-flash BBT (bad block table). In this case the
  bad PEB map is stored on flash and MTD does not have to do any
  scanning. See the NAND_USE_FLASH_BBT constant in the Linux
  source codes. But note, bad PEB scanning is usually minor comparing to
  the UBI scan time, so on-flash BBT is not probably going to give
  much.
• UBI scan time linearly depends on the number of PEB to read, which
  means that you may speed it up by merging two or more PEB and treating
  them as one composite PEB. Then from UBI POV PEB size will become
  larger, and the amount of PEBs to Scan will become smaller. The best
  way to do this is probably to teach UBI merging PEBs. It does not have
  to merge adjacent PEB. Instead, UBI can store the paired PEB number in
  the EC header. If one of the PEBs in the pair becomes bad, the pair may
  be «re-constructed» by moving the good PEB somewhere else and
  substituting the bad PEB by a good one from the pool of reserved
  PEBs.
• Design and implement UBI2. You may teach UBI to store erase
  counters and mapping information in a separate internal volume, instead
  of storing it in each PEB. Then you could implement the «anchor» idea
  to quickly find that internal volume (see
  here). This would solve slow UBI
  attach problem, but not linear memory consumption problem. Later we
  could have UBI3 which could solve more issues. This plan was suggested

  here. Indeed, it is easier to solve one problem at a time.

Why a dynamic volume is faster to access than a static volume of the same size?

Static UBI volumes were originally designed to store blobs of data like
configuration files. Static volumes provide CRC-32 protection for the stored
data, and static volumes know how much data they store. E.g., if you have a
static volume /dev/ubi0_1 of 254KiB in size consisting of 2
127KiB LEBs, and you store a 200KiB file there, and you read whole
/dev/ubi0_1 , you’ll get exactly 200KiB. If
/dev/ubi0_1 was a dynamic volume, you’d read 254KiB. So static
volumes make it easier to store blobs of data.

Thus, static volumes are slower to access (comparing to dynamic
volumes) because UBI has to check the data CRC-32 checksum. And note, it is not
a good idea to use static volumes with R/O UBIFS because UBIFS also protects
all the information it stores in the UBI volume with CRC-32 checksums, and
using static volumes with UBIFS would only slow down UBIFS mount speed.

How do I debug UBI?

Use fake MTD device

When debugging UBI one doesn’t have to use a real embedded platform with real
flash. In many cases, it is easier to use a PC with an MTD device emulator and
run UBI on top of this emulated MTD device. In fact, this is how most of the
UBI development was done.

There are 3 MTD device emulators in Linux kernel available:

• mtdram which simulates NOR flash in RAM;
• nandsim which simulates NAND flash in RAM;
• block2mtd which simulates NOR flash on top of a block
  device;

For example, to get a 32MiB fake NOR flash, run

$ modprobe mtdram total_size=32768

or to get a 64MiB fake NAND flash, run

$ modprobe nandsim second_id_byte=0x36

See here for more information about
the NAND simulator.

To ensure that you have fake MTD devices, run «cat /proc/mtd«.
It should print something like

dev:    size   erasesize  name
mtd0: 02000000 00020000 "mtdram test device"
mtd1: 04000000 00004000 "NAND simulator partition"

The fake MTD devices may further be attached to UBI (see
here).

Debugging messages

Sometimes it is necessary to make UBI print about what it is doing. You may
enable various UBI debugging messages using the «dynamic debug» Linux kernel
infrastructure. Please, refer to the corresponding
UBIFS section for more information. Below
is a similar table for UBI.

Extra self-checks

UBI contains various internal self-check functions which are often
very useful for debugging or testing. Please, refer to the corresponding
UBIFS self-checks
section for more information, because UBI extra self-checks are very
similar, just a bit simpler. Here is a similar table for UBI.

Power-cut recovery testing

UBI suppors power-cut emulation for testing which emulates power-cuts after
a random number of writes. When a power-cut is emulated, UBI switches to
read-only mode and disallows any further write to the UBI volume, thus
emulating a power cut. The main idea of this mode is to emulate power cuts in
interesting places, e.g. when writing the vid header.

Use the tst_emulate_power_cut debugfs file to set flags for
when to enable power-cut emulation. Possible flags for power-cut testing
can be seen in the table below.

Use the files tst_emulate_power_cut_min and
tst_emulate_power_cut_max to set the minimum and maximum number
of successful writes, respectively, before a power-cut is emulated.

Note that UBIFS has its own power-cut emulation functions. Please, refere
to the corresponding
Power-cut recovery testing
section.

How do I send an UBI bug report?

Basically the same way as an
UBIFS bug report

.

Chapter 1 UBI File System Production

• Pre-preparation tool source code

  • Make a normal root file system rootfs
  • Production tool
  1. Cross compilation tool: arm-linux-gcc-4.4.3-20100728.tar.gz
  2. Download zlib-1.2.5.tar.gz :http://www.zlib.net/
  3. Download lzo-2.06.tar.gz :http://www.oberhumer.com/opensource/lzo/download/
  4. Download e2fsprogs-1.41.14.tar.gz:http://e2fsprogs.sourceforge.net/
  5. Download mtd-utils-1.4.6.tar.bz2:http://debian.mirror.inra.fr/debian/pool/main/m/mtd-utils/

• Platform tool

  1. Linux platform: VMware Workstation Pro + Ubuntu12
  2. Target platform: imx.6
  3. Compilation toolchain: linux-ARM cross toolchain

• Precautions
  2. When making the file system, the file system and related files should be placed on Ubuntu.Home directory ‘~’Cannot be placed in a shared directory;
  3. Complete file system creation with shell commands

1.1 Compiling the UBI tool commands used on the development board

• Partial compilation parameters:

  • –host: compile the host
  • –target: target platform
  • –prefix: installation directory
  • CC: Cross tool chain path

• Directory location of related files

  • Root file system rootfs: /home/filesystem/armLinux/rootfs
  • Toolchain: /home/filesystem/FriendlyARM
  • Tool source code: /home/filesystem/mtd

(1) Compile and install e2fsprogs

tar xzvf e2fsprogs-1.41.14.tar.gz
cd e2fsprogs-1.41.14
./configure --host=i686-linux --target=arm-linux CC=/home/filesystem/FriendlyARM/toolschain/4.4.3/bin/arm-linux-gcc --prefix=/home/filesystem/FriendlyARM/toolschain/4.4.3/ arm-none-linux-gnueabi
make
 Sudo make install-libs (just need uuid library, so you don't need to install it completely)
cd ..

Result: You can see that the uuid/uuid.h file is installed in include, libuuid.a is installed in the lib directory in the toolchain directory /home/filesystem/FriendlyARM/toolschain/4.4.3/arm-none-linux-gnueabi.

(2) Compile and install zlib

tar xzvf zlib-1.2.5.tar.gz
cd zlib-1.2.5
./configure --host=i686-linux --target=arm-linux CC=/home/filesystem/FriendlyARM/toolschain/4.4.3/bin/arm-linux-gcc --prefix=/home/filesystem/FriendlyARM/toolschain/4.4.3/ arm-none-linux-gnueabi
make
sudo make install
cd ..

Result: Both zconf.h and libz.a are installed into the toolchain directories include and lib.

(3) Compile and install lzo

ar xzvf lzo-2.06.tar.gz
cd lzo-2.06
./configure --host=i686-linux --target=arm-linux --prefix=/home/filesystem/FriendlyARM/toolschain/4.4.3/arm-none-linux-gnueabi CC=/home/filesystem/FriendlyARM/toolschain/4.4.3/ bin/arm-linux-gcc
make
sudo make install
cd ..

Result: liblzo2.a has been copied to the lib directory of the toolchain.

(4) Compile and install mtd-utils

tar xzvf mtd-utils-1.4.6.tar.gz
cd mtd-utils-1.4.6
export CFLAGS="-static -O2 -g"
export CROSS=/home/filesystem/FriendlyARM/toolschain/4.4.3/bin/arm-linux-
 Export DESTDIR=/home/install (This statement does not work, the generated file is in the home directory under the current file) 
make WITHOUT_XATTR=1
sudo make install

Installation error:

LD  ftl_check
CC  mkfs.jffs2.o
 Mkfs.jffs2.c:71:21: Fatal error: sys/acl.h: There is no compilation of that file or directory compilation.
 Make: ***[/home/filesystem/FriendlyARM/ArmLinux/source/mtd-utils-v1.4.8-d37fcc0/mkfs.jffs2.o] Error 1.

solution:

sudo apt-get install uuid-dev libacl1-dev liblzo2-dev

Or don’t worry about this error, the tools we need have been generated. In the ubi-utils subdirectory, we need the ubiformat, ubiattach, ubimkvol and other files we need (please make sure that it is cross-compiled), and add the required tool copy to the root file system of the target board:

cd ./home/filesystem/FriendlyARM/toolschain/4.4.3/bin/arm-linux
cp * /home/filesystem/armLinux/rootfs/usr/sbin
cd ..
cp share/man/man1  /home/filesystem/armLinux/rootfs/usr/share/man/man1

1.2 Compile and make UBI image tool command (ubuntu)

1.2.1 Installing the UBI Compilation Dependency Tool Library

Create the home/filesystem/mtd/install directory in ubuntu and copy the three source code packages zlib-1.2.5.tar.gz, tar xvf lzo-2.06.tar.gz, e2fsprogs-1.42.tar.gz to ~ Under the /mtd directory.

(1) zlib

tar xvf zlib-1.2.5.tar.gz
cd zlib-1.2.5/
./configure --prefix=~/mtd/install
make
make install
cd ..

(2) lzo

tar xvf lzo-2.06.tar.gz
cd lzo-2.06/
./configure  --prefix=~/mtd/install
make
make install
cd ..

(3) e2fsprogs

tar xvf e2fsprogs-1.42.tar.gz
cd e2fsprogs-1.42/
./configure --prefix=~/mtd/install
make
make install
cd lib/uuid/
make install
cd ../../../

1.2.2 Compiling MTD Utilite

Modify the makfile file before compiling mtd-utils:

$ vi Makefile
PREFIX = ~/mtd/install
ZLIBCPPFLAGS = -I$(PREFIX)/include
LZOCPPFLAGS = -I$(PREFIX)/include
ZLIBLDFLAGS = -L$(PREFIX)/lib
LZOLDFLAGS = -L$(PREFIX)/lib
LDFLAGS += $(ZLIBLDFLAGS) $(LZOLDFLAGS)
CFLAGS ?= -O2 -g $(ZLIBCPPFLAGS) $(LZOCPPFLAGS)
$ vi common.mk
	#PREFIX=/usr

Then compile according to the following instructions:

$ WITHOUT_XATTR=1 make
make install DESTDIR=~/mtd/install
cd ..

The tools used to make the ubi file system are in the following directory:

~/mtd/install/home/<username>/mtd/install/sbin/mkfs.ubifs
~/mtd/install/home/<username>/mtd/install/sbin/ubinize

1.3 UBI file system image creation

The UBI system’s characteristic parameters must match the NAND FLASH partition I/O unit space size, page size, block logical size, block physical size, total block count and other parameter characteristics. So these parameters must be found and calculated before making the UBI image.

(1) NAND FLASH characteristics

We can get the features of the NAND FLASH manual, or you can get the print information from the kernel. When the top cover is normally activated, the terminal print information can be observed, which can occupy the mtd4 with the root file system as the MTD partition. The NAND FLASH characteristics are as follows:

GPMI NAND driver registered.(IMX)
UBI: attaching mtd4 to ubi0
UBI: physical eraseblock size:   262144 bytes (256 KiB)
UBI: logical eraseblock size:    253952 bytes
UBI: smallest flash I/O unit:    4096
UBI: VID header offset:          4096 (aligned 4096)
UBI: data offset:                8192
UBI: max. sequence number:       11

UBI: MTD device size:            480 MiB
UBI: number of good PEBs:        1920
UBI: number of bad PEBs:         0
UBI: number of corrupted PEBs: 0
UBI: max. allowed volumes:       128

It can be concluded that the mtd4 partition size is 480MiB, the I/O unit size is 4096Bytes, the block physical size is 256KiB, the block logical size is 253952Bytes, and the total number of blocks is 1920, so that the UBI file system parameter values ​​can be calculated:
(2) UBI image creation

mkfs.ubifs -F -r /home/filesystem/armLinux/rootfs/ -m 4096 -e 253952 -c 1919 -o ubifs.img
ubinize -o ubi.img -m 4096 -p 256KiB -s 4096 ubinize.cfg

The contents of ubinize.cfg are as follows:
[ubifs]

mode=ubi
image=ubifs.img
vol_id=0
vol_size=464MiB
vol_type=dynamic
vol_alignment=1
vol_name=ubifs0
vol_flags=autoresize

According to the NAND FLASH characteristics used, the following parameters are calculated here as follows:
-m: 4096
-s: 4096
-e: 253952 = 64 (the number of pages is generally fixed)4096 — 2
-c: 1919 = 1920 — 1
UBI file system space size: 464MiB = 1919253952/1024/1024

Parameter introduction:
-F: Enables the automatic filling of the white space «white-space-fixup». This must be enabled by uboot or ubiformat/nandwrite tool, whichever method is best. on.
-r: Make UBI source file system directory
-m: The minimum read/write unit of NAND FLASH, usually page size
-e:LEB size, for AM335x NAND driver, block size-2x(page size)
-c: The maximum number of blocks occupied by the file system, generally less than or equal to block count -1. The file system can access the most 256KiB*1919=464M space on the volume.

Chapter 2 A9 UBI File System Upgrade

2.1 u-boot mode programming

The UBIFS file system image with the volume label created by the ubinize command can be directly written to the NAND FLASH using the nand flash programming command. The specific commands are:

tftp 0x10800000 ubi.img
nand erase 0x2000000 0x1E000000
nand write 0x10800000 0x2000000 0x92c0000

0x10800000: Downloaded memory address
0x2000000: file system start address in A9
0x1E000000: A9 NAND FLASH partition size (480MiB)
0x8d00000: the size of ubi.img, according to its actual size

Result: The programming was successful, the development board was restarted, and the different mirrors were successfully replaced. After many tests, it was successful.

2.2 Mounting a USB flash drive

Pre-preparation: Copy the following tool commands and images to the U disk directory sbin/armtool/ (optional).

• UBI file system image: ubi.img

• Produced tool

   ubiformat
  （flash_erase）
  （nandwrite）
   reboot
  

2.2.1 ubiformat tool to burn UBI image

(1) Direct execution of the command

./ubiformat -q /dev/mtd4 -f ubi.img 

Result: Failed, /dev/mtd4 busy

ubiformat: error!: please, first detach mtd4 (/dev/mtd4) from ubi0
$./ubidetach –p /dev/mtd4
ubidetach: error!: cannot detach "/dev/mtd4"
           error 16 (Device or resource busy)

Result: Failed, error, command execution was unsuccessful.

(2) Uninstall the program and mount the A9 file system before upgrading online.

According to the first error ubiformat: error!: please, first detach mtd4 (/dev/mtd4) from ubi0, so first remove the ubi0 association and the file system program, that is, kill the application that occupies the file system, umount mtd4 Partition (the mtd4 partition mounts the UBI file system).

Killall WorkStationApp //kill the application that occupies the file system		
sleep 3

 Cd /mnt/udisk/sda1/sbin/armtool/ //This is just the directory where my U disk puts the image and command

 Umount /mnt/rwfs/ //umount mounted file system
umount /tmp/
umount /dev/shm/
umount /dev/pts/
umount /sys/
umount /
sleep 1

 ./ubiformat -s 4096 /dev/mtd4 -f ./ubi.img //Erase and burn UBI file system image
sleep 2

./reboot

Prevent burning failure script

=====================================================
#!/bin/sh

echo "## start update ubi image ###"

killall WorkStationApp
if [ $? -ne 0 ]
then
    echo "kill WorkStationApp failed!"
    exit 1
fi

sleep 5
cd /mnt/udisk/sda1/updateA9FileSystem/

umount /mnt/rwfs/
if test $? -ne 0
then
    echo "1 failed!"
    ./reboot
fi

umount /tmp/
if test $? -ne 0
then
    echo "2 failed!"
    ./reboot
fi

umount /dev/shm/
if test $? -ne 0
then
    echo "3 filed!"
    ./reboot
fi

umount /dev/pts/
if test $? -ne 0
then
    echo "4 failed!"
    ./reboot
fi

umount /sys/
if test $? -ne 0
then
    echo "5 failed!"
    ./reboot
fi

umount /
if test $? -ne 0
then
    echo "6 failed!"
    ./reboot
fi

sleep 1
./ubiformat -s 4096 /dev/mtd4 -f ./ubi.img

sleep 1

./reboot

==========================================================================
Result: The programming is successful, restart the development board to run normally, and the replacement of different images is also successful. After many trials, there is basically no problem.

2.2.2 flah_erase&nandwrite tool to burn UBI images

killall WorkStationApp
sleep 3

umount /sys/
umount /

sleep 1
cd /mnt/udisk/sda1/sbin/armtool/

 ./flash_erse /dev/mtd4 0x0 0x0 //Erase mtd4 entire partition
./nandwrite –a /dev/mtd4 ./ubi.img 

sleep 2
./reboot

Result: The programming is successful, and the development board is restarted. After many tests, there is basically no problem.
Description: You need to continue testing and adjust the parameters of the execution command.

Chapter 3 problems encountered

(1) UBI programming error problem

The upgrade of the UBI file system was unsuccessful, basically due to two reasons:
First, the UBI file system does not match the NAND FLASH parameters. Corresponding to the UBI image production in section 1.3.
Second, the UBI file system image is not completely written or written incorrectly. In fact, this is because the file system in A9 is occupied by other programs in the system, and the programming fails to write correctly. The reason for the entry.

The ubi volume (volume) was not created: the reason one

UBI error: ubi_read_volume_table: the layout volume was not found
UBI error: ubi_init: cannot attach mtd4

Verification error: the reason one

UBI error: ubi_io_read: error -74 (ECC error) while reading 253952 bytes from PEB 3:8192, read 253952 bytes
UBI error: ubi_io_read: error -74 (ECC error) while reading 253952 bytes from PEB 3:8192, read 253952 bytes

Error reading ubifs node: the reason may be one or two

UBIFS error (pid 1): ubifs_read_node: bad node type (255 but expected 0)
UBIFS error (pid 1): ubifs_read_node:bad node at LEB 552:57664,LEB mapping status1
UBIFS error (pid 1): ubifs_iget: failed to read inode 6066, error -22
UBIFS error (pid 1): ubifs_lookup: dead directory entry 'utmp', error -22

Mounting ubifs failed: the reason may be one or two

UBI error: vtbl_check: volume table check failed: record 0, error 9
UBI error: ubi_init: cannot attach mtd4
No filesystem could mount root, tried:  ubifs
Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)

(2) Other issues

When the command is executed, the MTD device reports that the resource is busy/linked incorrectly.

ubiformat: error!: please, first detach mtd4 (/dev/mtd4) from ubi0
$./ubidetach –p /dev/mtd4
ubidetach: error!: cannot detach "/dev/mtd4"
           error 16 (Device or resource busy)

solve:
1, kill application
2, umount the MTD partition mount operation:
(1)mount view the mounted file system

[email protected] /WorkStationApp$ mount
rootfs on / type rootfs (rw)
ubi0:rootfs on / type ubifs (rw,relatime)
proc on /proc type proc (rw,relatime)
sys on /sys type sysfs (rw,relatime)
tmpfs on /dev type tmpfs (rw,relatime,mode=755)
/dev/sda1 on /mnt/udisk/sda1 type vfat (rw,relatime,fmask=0022,dmask=0022,codepage=cp936,iocharset=utf8,shortname=mixed,errors=remount-ro)
devpts on /dev/pts type devpts (rw,relatime,mode=600)
shm on /dev/shm type tmpfs (rw,relatime)
rwfs on /mnt/rwfs type tmpfs (rw,relatime,size=512k)
tmpfs on /tmp type tmpfs (rw,relatime,size=65536k)

(2) Kill the application, unmount it

killall WorkStationApp
sleep 3

Umount /mnt/rwfs/ //umount mounted file system
umount /tmp/
umount /dev/shm/
umount /dev/pts/
umount /sys/
umount /

 Command execution 

Chapter 4 Reference Blog

@stangri
I am now seeing this in system log in WRT3200ACM, when I navigate to AdvancedReboot page:

Sat Nov 16 20:24:08 2019 daemon.err uhttpd[4410]: umount: can't unmount /alt/firmware: No such file or directory
Sat Nov 16 20:24:08 2019 daemon.err uhttpd[4410]: umount: can't unmount /alt/overlay: No such file or directory
Sat Nov 16 20:24:08 2019 daemon.err uhttpd[4410]: umount: can't unmount /alt/rom: No such file or directory
Sat Nov 16 20:24:08 2019 daemon.err uhttpd[4410]: ubiblock: error!: cannot open UBI volume "/dev/ubi1_0"
Sat Nov 16 20:24:08 2019 daemon.err uhttpd[4410]:           error 16 (Resource busy)
Sat Nov 16 20:24:08 2019 daemon.err uhttpd[4410]: ubidetach: error!: cannot detach mtd8
Sat Nov 16 20:24:08 2019 daemon.err uhttpd[4410]:            error 19 (No such device)
Sat Nov 16 20:24:08 2019 kern.err kernel: [12241.616455] ubi1 error: ubi_open_volume: cannot open device 1, volume 0, error -16
Sat Nov 16 20:24:08 2019 daemon.err uhttpd[4410]: ubidetach: error!: cannot detach mtd8
Sat Nov 16 20:24:08 2019 daemon.err uhttpd[4410]:            error 19 (No such device)
Sat Nov 16 20:24:08 2019 kern.notice kernel: [12241.634746] ubi2: attaching mtd8
Sat Nov 16 20:24:09 2019 kern.notice kernel: [12242.103562] ubi2: scanning is finished
Sat Nov 16 20:24:09 2019 kern.notice kernel: [12242.111799] ubi2: attached mtd8 (name "rootfs2", size 74 MiB)
Sat Nov 16 20:24:09 2019 kern.notice kernel: [12242.117569] ubi2: PEB size: 131072 bytes (128 KiB), LEB size: 126976 bytes
Sat Nov 16 20:24:09 2019 kern.notice kernel: [12242.124487] ubi2: min./max. I/O unit sizes: 2048/2048, sub-page size 2048
Sat Nov 16 20:24:09 2019 kern.notice kernel: [12242.131314] ubi2: VID header offset: 2048 (aligned 2048), data offset: 4096
Sat Nov 16 20:24:09 2019 kern.notice kernel: [12242.138310] ubi2: good PEBs: 592, bad PEBs: 0, corrupted PEBs: 0
Sat Nov 16 20:24:09 2019 kern.notice kernel: [12242.144341] ubi2: user volume: 2, internal volumes: 1, max. volumes count: 128
Sat Nov 16 20:24:09 2019 kern.notice kernel: [12242.151606] ubi2: max/mean erase counter: 6/3, WL threshold: 4096, image sequence number: 24238052
Sat Nov 16 20:24:09 2019 kern.notice kernel: [12242.160603] ubi2: available PEBs: 0, total reserved PEBs: 592, PEBs reserved for bad PEB handling: 40
Sat Nov 16 20:24:09 2019 kern.notice kernel: [12242.169873] ubi2: background thread "ubi_bgt2d" started, PID 7332
Sat Nov 16 20:24:09 2019 kern.info kernel: [12242.179194] block ubiblock2_0: created from ubi2:0(rootfs)
Sat Nov 16 20:24:09 2019 daemon.err uhttpd[4410]: mount: mounting /dev/ubi1_2 on /alt/overlay failed: Invalid argument
Sat Nov 16 20:24:09 2019 daemon.err uhttpd[4410]: mount: mounting overlay on /alt/firmware failed: No such file or directory
Sat Nov 16 20:24:09 2019 kern.err kernel: [12242.198244] overlayfs: failed to resolve '/alt/overlay/upper': -2
Sat Nov 16 20:24:09 2019 daemon.err uhttpd[4410]: umount: can't unmount /alt/firmware: Invalid argument
Sat Nov 16 20:24:09 2019 daemon.err uhttpd[4410]: umount: can't unmount /alt/overlay: Invalid argument
Sat Nov 16 20:24:09 2019 daemon.err uhttpd[4410]: ubiblock: error!: cannot open UBI volume "/dev/ubi1_0"
Sat Nov 16 20:24:09 2019 daemon.err uhttpd[4410]:           error 16 (Resource busy)
Sat Nov 16 20:24:09 2019 kern.err kernel: [12242.264269] ubi1 error: ubi_open_volume: cannot open device 1, volume 0, error -16
Sat Nov 16 20:24:09 2019 kern.info kernel: [12242.350881] block ubiblock2_0: released
Sat Nov 16 20:24:09 2019 kern.notice kernel: [12242.354773] ubi2: detaching mtd8
Sat Nov 16 20:24:09 2019 kern.notice kernel: [12242.358502] ubi2: mtd8 is detached

LuCI Master (git-19.320.34348-1df1421) / OpenWrt SNAPSHOT r11526-f84d6d2f3a

Code:

Ruckus Wireless Bootloader: 10.1.14T225 (Nov 15 2018 - 04:59:18 -0800)

Booted from partition 2
DRAM:  Validate Shmoo parameters stored in flash ..... OK

ICX7150-12 (POE), PVT1
SYS CPLD VER: 0x4 Released Ver: 0xa

device 0 offset 0x0, size 0xc0000
Enter 'b' to stop at boot monitor:
device 0 offset 0x0, size 0xc0000
bootdelay: ===
Booting image from Primary

NAND read: device 0 offset 0x0, size 0x2000000
 33554432 bytes read: OK
## Loading kernel from FIT Image at 70000200 ...
   Using 'conf@1' configuration
   Trying 'kernel@2' kernel subimage
     Description:  Ruckus Linux MN VER=08.0.80e
     Type:         Kernel Image
     Compression:  lzma compressed
     Data Start:   0x70213ff0
     Data Size:    2184164 Bytes = 2.1 MiB
     Architecture: ARM
     OS:           Linux
     Load Address: 0x61008000
     Entry Point:  0x61008000
     Hash algo:    crc32
     Hash value:   29a83c27
## Loading kernel from FIT Image at 70000200 ...
   Using 'conf@1' configuration
   Trying 'kernel@2' kernel subimage
     Description:  Ruckus Linux MN VER=08.0.80e
     Type:         Kernel Image
     Compression:  lzma compressed
     Data Start:   0x70213ff0
     Data Size:    2184164 Bytes = 2.1 MiB
     Architecture: ARM
     OS:           Linux
     Load Address: 0x61008000
     Entry Point:  0x61008000
     Hash algo:    crc32
     Hash value:   29a83c27
   Verifying Hash Integrity ... crc32+ OK
## Loading ramdisk from FIT Image at 70000200 ...
   Using 'conf@1' configuration
   Trying 'ramdisk@1' ramdisk subimage
     Description:  Ramdisk -rootfs
     Type:         RAMDisk Image
     Compression:  lzma compressed
     Data Start:   0x7042fd88
     Data Size:    25430519 Bytes = 24.3 MiB
     Architecture: ARM
     OS:           Linux
     Load Address: 0x00000000
     Entry Point:  0x00000000
     Hash algo:    crc32
     Hash value:   960fb2cd
   Verifying Hash Integrity ... crc32+ OK
## Loading fdt from FIT Image at 70000200 ...
   Using 'conf@1' configuration
   Trying 'fdt@3' fdt subimage
     Description:  Flattened Device Tree blob
     Type:         Flat Device Tree
     Compression:  uncompressed
     Data Start:   0x7042db30
     Data Size:    8607 Bytes = 8.4 KiB
     Architecture: ARM
     Hash algo:    crc32
     Hash value:   6fc9a37b
   Verifying Hash Integrity ... crc32+ OK
   Loading fdt from 0x7042db30 to 0x65008000
   Booting using the fdt blob at 0x65008000
   Uncompressing Kernel Image ... OK
   Using Device Tree in place at 65008000, end 6500d19e

Starting kernel ...

booting with 4.4 kernel
NAND Type: Micron NAND 2GiB (x 1)
PLATFORM MN
dd cmd: UBI
ubi0 error: ubi_io_read: error -74 (ECC error) while reading 1040384 bytes from PEB 841:8192, read 1040384 bytes
UBIFS error (ubi0:0 pid 565): ubifs_recover_leb: corruption -3
UBIFS error (ubi0:0 pid 565): ubifs_check_node: bad node length 263218
UBIFS error (ubi0:0 pid 565): ubifs_check_node: bad node at LEB 1134:884736
    magic          0x6101831
    crc            0x87f8e43e
    node_type      1 (data node)
    group_type     128 (unknown)
    sqnum          1378685144455
    len            263218
    key            (4194457, xentry, 0x00205e)
    size           335548416
    compr_typ      1344
    data size      263170
    data:
    00000000: d6 c1 c7 35 d4 a1 e1 f2 c1 d0 ef 02 7e 99 b9 63 4f 17 43 76 22 8a 8d f7 1e d1 49 19 d3 0b a3 60
    [trimmed]
    00025fa0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
Unable to handle kernel paging request at virtual address f0ca1000
pgd = d0d14000
[f0ca1000] *pgd=86155811, *pte=00000000, *ppte=00000000
Internal error: Oops - BUG: 7 [#1] PREEMPT SMP ARM
Modules linked in:
CPU: 0 PID: 565 Comm: mount Not tainted 4.4.0 #1
Hardware name: BRCM XGS iProc
task: e51c2600 ti: d0c2c000 task.ti: d0c2c000
PC is at hex_dump_to_buffer+0x158/0x4d0
LR is at 0xd0c2db9a
pc : [<c0225670>]    lr : [<d0c2db9a>]    psr: 20000213
sp : d0c2db00  ip : 00000034  fp : d0c2db4c
r10: 00000020  r9 : c0432298  r8 : 00000020
r7 : f0ca0fff  r6 : 00000010  r5 : 00000066  r4 : 00000083
r3 : 00000066  r2 : 00000030  r1 : 00000032  r0 : 00000031
Flags: nzCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
Control: 12c5387d  Table: 71d14059  DAC: 00000051
Process mount (pid: 565, stack limit = 0xd0c2c190)
Stack: (0xd0c2db00 to 0xd0c2e000)
db00: c0066134 c0065a4c 00000001 f0ca0ff0 00000061 00000000 00000020 00000020
[trimmed]
dfe0: b6e2d071 beab4a18 00051149 b6e2d078 80000230 beab4f18 00000000 00000000
Backtrace:
[<c0225518>] (hex_dump_to_buffer) from [<c0225a8c>] (print_hex_dump+0xa4/0x118)
[<c02259e8>] (print_hex_dump) from [<c01ca444>] (ubifs_dump_node+0x97c/0xbe0)
[<c01c9ac8>] (ubifs_dump_node) from [<c01a8224>] (ubifs_check_node+0x1c0/0x308)
[<c01a8064>] (ubifs_check_node) from [<c01af8e4>] (ubifs_scan_a_node+0xd0/0x1f4)
[<c01af814>] (ubifs_scan_a_node) from [<c01c1f80>] (ubifs_recover_leb+0x578/0x7e0)
[<c01c1a08>] (ubifs_recover_leb) from [<c01b11f0>] (ubifs_replay_journal+0xdd0/0x1540)
[<c01b0420>] (ubifs_replay_journal) from [<c01a5adc>] (ubifs_mount+0x1190/0x192c)
[<c01a494c>] (ubifs_mount) from [<c00d9bdc>] (mount_fs+0x14/0x40)
[<c00d9bc8>] (mount_fs) from [<c00f1bf4>] (vfs_kern_mount+0x50/0xf8)
[<c00f1ba4>] (vfs_kern_mount) from [<c00f4cc4>] (do_mount+0x1a8/0xc40)
[<c00f4b1c>] (do_mount) from [<c00f5acc>] (SyS_mount+0x7c/0xa8)
[<c00f5a50>] (SyS_mount) from [<c000a4a0>] (ret_fast_syscall+0x0/0x3c)
Code: 02a0cee5 cf00000a 010054e1 d100003a (0130f7e5)
[BrcdSoftlockup]: BUG: soft lockup - CPU#0 stuck for 524s! [mount:565]
---[ end trace a6e623cfc8e5011c ]---
note: mount[565] exited with preempt_count 1
Segmentation fault

Mounting Config partition failed, non-recoverable file system corruption
Reformatting the flash, please download config and keys again ...

ubidetach: error!: cannot detach mtd2
           error 16 (Device or resource busy)
ubiformat: error!: please, first detach mtd2 (/dev/mtd2) from ubi0
Formatting Doneubi0 error: ubi_attach_mtd_dev: mtd2 is already attached to ubi0

ubiattach: error!: cannot attach mtd2
           error 17 (File exists)
ubimkvol: error!: UBI device does not have free logical erasebloubi1: attaching mtd3
cks
!!!ERROR:ubimkvol failed for config partition
ubi1: scanning is finished
ubi1: attached mtd3 (name "resources", size 512 MiB)
ubi1: PEB size: 1048576 bytes (1024 KiB), LEB size: 1040384 bytes
ubi1: min./max. I/O unit sizes: 4096/4096, sub-page size 4096
ubi1: VID header offset: 4096 (aligned 4096), data offset: 8192
ubi1: good PEBs: 504, bad PEBs: 8, corrupted PEBs: 0
ubi1: user volume: 1, internal volumes: 1, max. volumes count: 128
ubi1: max/mean erase counter: 1/0, WL threshold: 4096, image sequence number: 677597375
ubi1: available PEBs: 0, total reserved PEBs: 504, PEBs reserved for bad PEB handling: 32
ubi1: background thread "ubi_bgt1d" started, PID 582
UBIFS (ubi1:0): background thread "ubifs_bgt1_0" started, PID 586
UBIFS (ubi1:0): recovery needed
UBIFS (ubi1:0): recovery completed
UBIFS (ubi1:0): UBIFS: mounted UBI device 1, volume 0, name "resources"
UBIFS (ubi1:0): LEB size: 1040384 bytes (1016 KiB), min./max. I/O unit sizes: 4096 bytes/4096 bytes
UBIFS (ubi1:0): FS size: 476495872 bytes (454 MiB, 458 LEBs), journal size 23928832 bytes (22 MiB, 23 LEBs)
UBIFS (ubi1:0): reserved for root: 4952683 bytes (4836 KiB)
UBIFS (ubi1:0): media format: w4/r0 (latest is w4/r0), UUID 8068C335-3096-4AA2-983E-4659B278DEBA, small LPT model
UBIFS (ubi1:0): full atime support is enabled.
Resource partition is mounted!!
Restore is Done
Kernel crashdump is disabled.
Backup is fine
dma_mem_base: 0x96000000, dma_mem_len: 0x8000000, warm_mem_base: 0x9f500000, warm_mem_len: 0xb00000
Creating TUN device
Starting TPM Infra
Group tss and User tss created
kernel.hostname = localhost
hostname set to localhost
Starting TCSD Daemon
TCSD Up and Running
Enabling time stamp.
Starting the FastIron.
USER=root
HOME=/
GCOV_PREFIX=/fast_iron
ethaddr=78:a6:e1:2e:48:29?
TERM=vt102
PATH=/sbin:/usr/sbin:/bin:/usr/bin
crashkernel=64M@0x70000000
ip=192.168.0.210:192.168.0.14::255.255.255.0:MN:eth0:off
SHELL=/bin/sh
PWD=/
i2c-0    i2c          Broadcom iProc I2C adapter          I2C adapter    2
i2c-1    i2c          Broadcom iProc I2C adapter          I2C adapter    2
!!!WARNING: Unable to read Filesystem information!!!
Total no. of blocks in FS = 0
Total no. of free blocks = 0
FIPS Disabled:PORT NOT DISABLED
platform type 90
OS>
Loaded Image SPR08080e from Pri Built on (UTC): Tue Apr  9 10:25:33 2019
Enabling Console Logging
flash_file=0.
Upgrade from pre 8.0.80 image. Dual mode to VLAN config upgrade needed

Hotplugger Daemon Initialized ..
External USB disk is not mounted or plugged in.
set_board_level: gi_board_type = 116[  628.362222] linux-kernel-bde: map phys range 0x3200000-0x3240000 to 0xb0d09000-0xb0d49000
[  628.544467] linux-kernel-bde: map phys range 0x88100000-0x90100000 to 0xa8c00000-0xb0c00000
DMA pool size: 134217728
PHY Reset De-asserted

sw_pp_sdk_init:137 ToR init

SOC unit 0 attached to PCI device BCM56160_B0
Firmware version from File: 2.1.1
Pre Parsing Config Data ...


Done with Pre-allocating nexthop id's for Unicast & Openflow