Showing posts with label Guide. Show all posts
Showing posts with label Guide. Show all posts

Saturday, 22 February 2014

How To: Fix Wi-Fi connecting issues on the HTC One running Android 4.4.2

It turned out that HTC One users have lately some problems with connecting their devices to the Wi-Fi routers after the Android 4.4.2 update. Here we present the iconography solution of this problem.

  1. Open Settings panel and click on the "Wi-Fi" tile
  2. Tap on the "3 dots" menu in the upper right corner
  3. From the list select "Wi-Fi Direct"


  4. You will now see that your device is called "Android_ed2c". Tap on the "3 dots" menu in the upper right corner
  5. From the list select "Rename device"
  6. Replace the "Android_ed2c" with  the "HTC One" text and hit "OK" button


  7. Done!
Have any questions or comments? Feel free to share! Also, if you like this article, please use media sharing buttons (Twitter, G+, Facebook) below this post!

Sunday, 16 February 2014

How to: Generate a kernel log after random reboot

Once in a while, a software bug in the kernel will cause a random reboot, so in order to help kernel developers to fix and troubleshoot the reboots, a kernel log need to be submitted to the developer for further analysis and hopefully lead to a bug fix.

Most Android kernels have "RAM Consoles" to save the necessary kernel logs immediately after reboot in the RAM.  The users can then retrieve this RAM log on a subsequent reboot to submit to kernel developers. Here is a quick tour on how to do that.


[ Using ADB ]
  1. adb shell
  2. su
  3. cat /proc/last_kmsg > /sdcard/last_kmsg.txt
  4. exit
  5. exit
  6. adb pull /sdcard/last_kmsg.txt
File last_kmsg.txt will be located in the same location as adb.exe executable.


[ Using android terminal app ]
  1. su
  2. cat /proc/last_kmsg > /sdcard/last_kmsg.txt
  3. exit
  4. exit
  5. adb pull /sdcard/last_kmsg.txt
File last_kmsg.txt will be located on your SD-card.


[ Using Root Explorer / ES Explorer with Root ]
  1. go to /proc folder
  2. copy last_kmsg to /sdcard/
  3. rename last_kmsg to last_kmsg.txt
File last_kmsg.txt will be located on your SD-card.

The best method to share the last_kmsg.txt content is to upload it to pastebin.com and send a link to the developer.

Source: faux123 (Google +)

Have any questions or comments? Feel free to share! Also, if you like this article, please use media sharing buttons (Twitter, G+, Facebook) below this post!

Wednesday, 18 December 2013

How to: Use "adb sideload" on your Android device


Probably every Android power-user at least once in his life used ADB - Android Debug Bridge. It is a versatile command line tool that lets you communicate with an emulator instance or connected Android-powered device. Using ADB shell commands gives you additional control over your device and sometimes it can save your device from being bricked (example: How to: copy ROM zip file to the freshly wiped device). You can find some more basic information about ADB here.

Since Android Jelly Bean there has been a new ADB mode available in the AOSP recovery, incorporated by the Android developer community into custom recoveries too. It is called "ADB sideload" and most of you probably have heard about it already. This is an alternate method to the one I wrote about here - How to: copy ROM zip file to the freshly wiped device. The main difference is that ADB sideload works only with recoveries based on Jelly Bean source or newer. I believe that ADB sideload was created to simplify the process of flashing/restoring Android update.zip packages.

Keep in mind that while using ADB sideload, the regular ADB shell won't work. To be able to use SIDELOAD mode make sure you're running latest ADB drivers from the Android SDK (Platform-tools). Here are the simple steps you need to follow to flash update.zip package using adb sideload mode (based on stock Android recovery):
  1. Place the ZIP package you want to install in the same location where you keep ADB drivers - adb.exe, AdbWinApi.dll and AdbWinUsbApi.dll (usually it's SDK\platform-tools)
  2. Make sure you have USB debugging enabled in Settings > Development on your device
  3. Make sure your phone drivers are installed on the PC you're going to use
  4. Boot your device in recovery mode (Android logo with a exclamation mark) and connect your device to PC
  5. Hold down "power" button first, followed quickly by "volume up" button. You should now see the recovery menu
  6. Use the volume up/down keys to select "apply update from ADB," then press power to select it
  7. Open a command prompt on the PC (cmd.exe), type and confirm with ENTER:
  8. cd /d <adb.exe location> (for example: cd /d c:\SDK\platform-tools) or you can open your SDK/platform-tools folder, then press SHIFT button and the right-click mouse button and choose “Open command prompt here
  9. adb sideload <filename>.zip (for example: adb sideload update.zip)
  10. The zip package will begin installing. When it's done, select "reboot system now."
How is that different from the alternative method? You don't have to manually create the proper folders structure, push the file and later install if from inside the recovery menu. The result is basically the same, because ADB sideload is also transferring the zip file into the device internal memory and later it automatically begins the installation procedure. However, it works only with recoveries based on Android Jelly Bean source.

Have any questions or comments? Feel free to share! Also, if you like this article, please use media sharing buttons (Twitter, G+, Facebook) below this post!

Sunday, 8 December 2013

How to: Manually install an OTA update on your HTC device

Sometimes it turns out that your device for some reason can't find the most recent OTA (Over The Air) update. However, there is a solution to apply OTA update manually, without being forced to wait for the update being downloaded and installed by your device.

Facts you need to know before continuing:
  1. This method has nothing to do with the "adb sideload" feature available on Android devices since Jelly Bean. It should work on every Android HTC device.
  2. Read about different software versions for the HTC devices here. Keep in mind that the OTA update must match the software version on your device. Each original OTA update.zip package has 2 different software versions in its name - the version of the software that must be currently installed on your device and the version of the software that your device will run after the update. For example:
OTA_M7_UL_JB43_SENSE55_MR_HTC_Europe_3.62.401.1-2.24.401.8_release_338160gxmo5sd9337kadux.zip

As you can see, 2.24.401.8 is the software version you must currently have installed to be able to run the OTA and 3.62.401.1 is the version of the software expected after the update. You can't flash an OTA update on a different software version to that expected by the OTA update for two main reasons:
  1. Formal reason - the OTA update.zip package checks if your system is running the expected software version before it starts to install the update:
    assert(file_getprop("/system/build.prop", "ro.build.fingerprint") == "htc/htc_europe/m7:4.2.2/JDQ39/235216.8:user/release-keys");
    This must match the following build fingerprint from the build.prop. If the versions don't match, the installation of the OTA package will be cancelled.
    ro.build.fingerprint=htc/htc_europe/m7:4.2.2/JDQ39/235216.8:user/release-keys
  2. Technical reason - OTA update.zip packages usually contain patches for the target files. This way OTA update doesn't have to be 1GB size, even if it is supposed to update the whole OS to the newer version. Patch files have an *.apk.p extension (f.g. Camera.apk.p) and each patch file must match the target file (f.g. Camera.apk). To make it possible, OTA update.zip package checks the MD5 checksum of each file on the system partition on your device:
    assert(apply_patch_check("/system/app/Camera.apk",
    "b3b79e40bad0b1bec5fb949ac5f8662f91bee714",
    "5a5bf63e3d7c9de52a7d8d7a292e158263783844"));
    And finally, if the MD5 is correct it applies the patch. All these commands are stored in META-INF\com\google\android\updater-script of each OTA update.zip package. If the MD5 is incorrect, or at least one file is missing, the installation of the OTA package will be cancelled.
    assert(apply_patch("/system/app/Camera.apk", "-",
    b3b79e40bad0b1bec5fb949ac5f8662f91bee714, 4229394, 
    5a5bf63e3d7c9de52a7d8d7a292e158263783844,
    package_extract_file("patch/system/app/Camera.apk.p")));

Now you are ready to manually apply the OTA update on your device. The method you'll use is based on Android stock recovery.
  1. Make sure you have the stock (original) recovery flashed on your device
  2. Make sure you're running a completely unmodified operating system. Additional files like Superuser.apk or su binary are allowed, you just can't have any system files changed or removed (due to the MD5 check)
  3. Make sure you have a reasonable amount of battery charge remaining
  4. Make sure the OTA update you're about to install matches the software version on your device
  5. Copy the OTA update.zip package to your device (internal storage preferred)
  6. Turn OFF the device (make sure fastboot mode is disabled in settings)
  7. Hold your volume down and power keys until the bootloader starts up
  8. Using the volume keys, navigate down to RECOVERY and press power
  9. You are now in stock recovery mode. You should see nothing more than a (sometimes spinning) Android with the red exclamation mark above it (an empty black screen is also possible). Wait for a few seconds
  10. First hold volume up, then (with volume up held down) press power button to enter the main recovery menu. This keys combination may vary from device to device. Try different keys combination if the above one doesn't work (might be all three keys all together too)
  11. Navigate down to "apply from phone storage" text and press power to confirm
  12. Navigate to the location where you copied the OTA update.zip package and press power to confirm
  13. Wait (sometimes you might need to follow further instructions on the screen) until the update process is finished
  14. Once system is booted you should see a confirmation about the successfully flashed update.
Have any questions or comments? Feel free to share! Also, if you like this article, please use media sharing buttons (Twitter, G+, Facebook) below this post!

HTC ROM Update Utility (RUU) variants explanation

Owners of HTC devices might be a little confused when it comes to understanding the current method of numbering used by HTC to mark their software updates. Here you can find a short explanation of this matter.

First of all, we all know that HTC sells their devices all over the world. Every carrier and region of the world has some more or less different "local settings". By "local settings" I mean things like: emergency call numbers, languages, time-zone settings, network settings, carriers branding, GPS settings and more. Because of that each software for even the same device must be different.

Secondly, even the same device might use different hardware related to the region of the world. For example, HTC One is available in at least 6 different hardware configurations. One of the main difference is modem version. GSM version of the HTC One uses MDM8215 chip and LTE variant has MDM9215. Verizon (US) is known to have MDM9615. So that's already 3 different versions of the same device.

Every HTC update has the following numbering structure:

a.bb.ccc.d

a - the first number of the software. Normally it doesn't increase very often, but when it does, it usually means newer Android version.
bb - this number is also a software number. Increasing bb number means some more or less major update based on the same Android version, but sometimes it might even bring newer version of the HTC Sense.
ccc (also cc or cccc) - this is the number of the region/carrier. It has nothing to do with the software itself being newer or older. More details below.
d (sometimes dd) - the third part of the software version (the fourth one in the RUU name). Usually indicates some minor update with some hot fixes.


Example: 3.20.401.3

So basically, when you are interested in the software version, you should look at a.bb.d (3.20.ccc.3) because ccc (401) means only the region/carrier variant of the software.

Here is the list I was able to gather of possible RUU variants:

a.bb.61.d - Orange UK (United Kingdom)
a.bb.65.d - Orange CH (Switzerland)
a.bb.69.d - Orange PL (Poland)
a.bb.73.d - Orange FR (France)
a.bb.75.d - Orange ES (Spain)
a.bb.110.d - T-Mobile UK (United Kingdom)
a.bb.111.d - T-Mobile DE (Germany)
a.bb.112.d - T-Mobile AT (Austria)
a.bb.113.d - T-Mobile CZ (Czech Republic)
a.bb.118.d - T-Mobile PL (Poland)
a.bb.161.d - Vodafone UK (United Kingdom)
a.bb.163.d - Vodafone FR (France)
a.bb.165.d - Vodafone IT (Italy)
a.bb.166.d - Vodafone CH-DE (Switzerland - Germany)
a.bb.168.d - Vodafone GR (Greece)
a.bb.169.d - Vodafone AT (Austria)
a.bb.206.d - O2 UK (United Kingdom)
a.bb.207.d - O2 DE (Germany)
a.bb.401.d - World Wide English (WWE)
a.bb.415.d - Arabic (UAE/Middle East)
a.bb.468.d - Turkey
a.bb.497.d - Morocco
a.bb.498.d - WWE Brightpoint US (United States)
a.bb.502.d - Cingular US (United States)
a.bb.531.d - T-Mobile US (United States)
a.bb.631.d - Rogers Canada (WWE)
a.bb.651.d - Sprint US (United States)
a.bb.661.d - Telus WWE (Canada)
a.bb.666.d - Bell Canada
a.bb.707.d - Asia (WWE)
a.bb.708.d - Asia HK (Hong Kong)
a.bb.709.d - Asia TW (Taiwan)
a.bb.720.d - Asia (India)
a.bb.728.d - Virgin AU (Australia)
a.bb.751.d - Chunghwa Telecom (Taiwan)
a.bb.771.d - Hutchison 3G UK (United Kingdom)
a.bb.777.d - ?
a.bb.841.d - Telsra WWE (World Wide English)
a.bb.862.d - Voda-Hutch AU (Australia)
a.bb.901.d - TIM IT (Italy)
a.bb.911.d - SK Telecom (South Korea)
a.bb.980.d - Optus AU (Australia)
a.bb.999.d - HTC Test RUU
a.bb.1400.d - China
a.bb.1401.d - ?
a.bb.1403.d - HTC China CMCC (China Mobile Communications Corporation)
a.bb.1540.d - HTC One U.S. Developer Edition
a.bb.1600.d - ?
a.bb.1700.d - HTC One Google Edition

The list is quite long, and I'm sure there are still some some missing. If you know more variants, please let me know in the comments section.

The 401 and the 707 devices are usually the first ones that HTC prepare updates for. Carrier branded software is usually months after the WWE updates.

Finally, keep in mind that each software version is connected with the CID (Carrier ID) and MID (Model ID) of your device. That means you can't use whichever version you want, unless you have S-OFF on your device. Each ROM Update Utility.exe package will check your device details (bootloader version, MID and CID) before starting the update process.

Have any questions or comments? Feel free to share! Also, if you like this article, please use media sharing buttons (Twitter, G+, Facebook) below this post!

How to: Copy ROM zip file to the freshly wiped device


Have you ever been in a situation where you mistakenly (or not) wiped the content of the internal storage (including system partition and virtual SD cardon your phone or tablet and unfortunately it doesn't support external storage? Now you can't flash desired custom ROM because you removed it and the device can't be booted due to wiped system partition. Don't worry - it's very easy to fix!

Here are the steps you need to follow to copy the correct rom.zip package to your internal storage:
  1. Download this mini-sdk package and extract it to c:\mini-sdk
  2. Rename the custom ROM package you want to flash to "rom.zip" and copy it to c:\mini-sdk
  3. Connect your device to the PC
  4. Boot your device in recovery mode
  5. Open a command prompt on the PC (cmd.exe), type and confirm each command with ENTER:
  6. cd /d c:\mini-sdk
  7. adb shell
  8. mount /data (error might mean that /data partition is already mounted)
  9. on Android 4.2 and above: mkdir -p /data/media/0
  10. on Android 4.1.2 and below: mkdir -p /data/media
  11. exit
  12. on Android 4.2 and above: adb push rom.zip /data/media/0
  13. on Android 4.1.2 and below: adb push rom.zip /data/media
  14. Wait for a few minutes until the file is copied. It will looks like command shell is not responding, but that means it's still copying. Wait patiently until you'll see blinking cursor again.
  15. Depends on your custom recovery, use the navigation keys to flash the custom rom.zip package in a typical way you used to flash custom ROMs. It should be on your internal storage now. Done!
Have any questions or comments? Feel free to share! Also, if you like this article, please use media sharing buttons (Twitter, G+, Facebook) below this post!

Monday, 21 October 2013

Android terminology explanation and why ROOT isn't a crime


This Android terminology dictionary will be growing in time. If you have any suggestions for terms to be included here, please leave a proper comment below this article. The purpose of this dictionary is to explain basic and fundamental Android terminology to a regular "noob" Android user. It is supposed to be written in language that everyone can understand.

Related articles:

ROOT (Superuser) - special user account used for system administration with the highest possible access permissions. The root/superuser account on Linux/Android OS has the same purpose as the Administrator account on MS Windows. For some reason however the term "root" has a very pejorative meaning in the Android world. Many users believe that "root" (or "rooting") is a forbidden fruit that might brick their Android device, void its warranty, make it unstable, unsecured or even explode in some situations.

What is the truth? Very simple - just go to the Control Panel on your PC/notebook and in User Accounts check the type of your account. Is it "Administrator"? Ouch! You're rooted! I really hope you can live with that. Anyway, it seems that having account with an Administrator rights on the Windows powered machine is nothing unusual. What you need to realize, is that having Android powered device without root access is like using the "Guest" account on your Windows computer.

So what will root access give you? Well, it might give you everything, but might give you nothing too. Basically it all depends on what the user would like to do with his device. Root access by itself doesn't change anything. Just like Administrator account on Windows, you can use its capabilities and change almost everything on your device or you might just keep using the OS as it is.

To add root access to Android device you need 2 files - Superuser.apk and su binary. The su binaries people commonly use are modified to latch onto a companion app, which prompts the user when an app requires root access.


ROM (proper definition) - Read Only Memory. Actually it has nothing to do with what most people think. What does an internet definition say about it? "Read-Only memory is a class of storage medium used in computers and other electronic devices. Data stored in ROM cannot be modified, or can be modified only slowly or with difficulty, so it is mainly used to distribute firmware".


ROM (wrong definition) - most people call Android installation package a "ROM" or a "custom ROM". What we call a "ROM" is just a content of /data, /system and boot partitions. Their technical name is "update package". Flashing this package is nothing other than just replacing particular files on device partitions. Where do these files come from?
  1. Developers can compile builds from Android Open Source Project (AOSP) or similar open source projects like Android Open Kang Project (AOKP) or CyanogenMod (CM).
  2. They can modify closed source vendor images (HTC, Samsung, Sony, LG) by extracting factory system partitions, images (i.e. system.img) or dumping the system partition directly from the device.
What does a typical "custom ROM" looks like? It's a .zip package with files organized in a specific order - applications, framework, binaries, libraries and other files required for proper system work. Imagine a .zip package with "Windows, Users and Program Files" folders inside. That, if we massively simplify it, is what an Android "custom ROM" package is made of.

"Custom ROMs" usually contain root access, so flashing a "custom ROM" results in rooting the device, BUT rooting the device does not always mean flashing a "custom ROM" because you can add root access on a stock system too by adding the su (superuser) binary.

The best name for Android "custom ROM" would be a “modified system zip package”. Why .zip? Because update.zip packages were originally designed for over the air (OTA) updates. So both OTA update and custom "system zip packages" have more less the same construction - system files with the proper update-binary/updater-script (inside META-INF), compressed into ZIP archive. The only method to install (or "flash") ZIP package is via Android recovery.


Boot - The boot.img file is commonly mistaken for the Linux kernel that powers Android. It is actually the image containing the kernel and a ramdisk. The kernel is the fundamental layer of the operating system that controls the input and output of data in the system, it mounts the ramdisk during the boot sequence.


RUU - ROM Update Utility used by HTC to flash/update firmware on HTC device. Again - not very fortunate naming. In fact RUU does contain some "low level" firmware like bootloader, radio, touch panel drivers but it also contains entire EXT3/4 partitions like system or data, which surely can't be considered "Read Only Memory".


Firmware - a combination of persistent memory, program code and data stored in it. "Firmware" is actually a better term than "ROM", but many people are used to calling "Firmware" everything else that comes in the RUU except from system.img. The content of so called "firmware.zip" (called that by HTC itself) was described in this article.


S-OFF/S-ON - S-OFF stands for "Security OFF" and S-ON for "Security ON". It's a term specific to HTC devices (and refers to digital signature checking on the bootloader "hboot"). Retail devices always come with SHIP S-ON locked bootloaders. Hboot can also be found in an engineering version (ENG as opposed to SHIP), but it's not easy to get such a device.


Bootloader - a computer program that loads an operating system and has all the parameters for how to boot the OS. Bootloader is located on a separated partition on your device and is responsible for a low level security on your device. Extended bootloader commands (usually executed via fastboot protocol) can basically manage the whole device, including possibility to gather all device sensitive details (like IMEI, software version, serial number and more), flashing a new operating system or recovery, updating bootloader and other parts of the firmware or even formatting EXT partitions. This is why some vendors (like HTC) lock the bootloaders on their devices to avoid giving end-users too much control over the device.


Fastboot - diagnostic protocol used primarily to modify the flash filesystem via a USB connection from host computer. After enabling the protocol on the device itself (entering "fastboot" mode from inside the bootloader), it will accept a specific set of commands sent to it via USB using a command line, for example "fastboot flash boot boot.img" or "fastboot erase cache".


Android Recovery - a separate partition on your device responsible for installing original OTA updates (.zip) and performing some basic operations on partitions (wiping, formatting, mounting). In some way Android recovery is a sort of mini-OS because it has its own kernel (so it boots independently from the main OS) and on custom recoveries (like 4EXT Recovery, TWRP Recovery) you can run simple applications like AROMA installer or AROMA Manager. Custom recoveries also provide touch support and many advanced features like partition conversion (EXT3--->EXT4), Nandroid backup, possibility to use themes, different wipe and format options, permissions fix and more.

Android Recovery can be based on open source or closed source code. The Android kernel inside the recovery must be always based on open source code.


CID/MID - CID (Carrier ID) and MID (Model ID) numbers are specific tags used by HTC to match a particular device with proper software updates. Many Android devices need different settings, hardware drivers and applications depends on the region of the world or carrier. One device can be available in a few different variants, for example HTC One has 6 different models IDs (MIDs) - PN0711000 (Taiwan), PN0714000 (Austraila, Hong-Kong, Singapore), PN0710000 (Europe), PN0712000 (United States and Canada), PN0771000 (China) and PN0720000 (U.S. Sprint). To check your current MID/CID number you need to use fastboot protocol and type "fastboot getvar all" command or respectively "fastboot getvar mid" / "fastboot getvar cid" if you don't want to see all variables output.


NANDroid backup - an 1:1 copy of the /system, /data, /cache and boot partitions. NANDroid backup can be created and restored using custom Android recoveries like CWM, EXT4 or TWRP. The form of NANDroid backup vary from different custom recovery - could be .tar, .zip, .img or other. NANDroid backup is saved on your SD-card (either virtual or physical) and can be easily transferred to the PC. It's highly recommended to do a regular backup, but to be even more secured, remember about having backup of the backup too.



Article contributors:
  • Shen Ye
  • Want to became an article contributor? Write a comment with an explained term in the the comments section.

Have any questions or comments? Feel free to share! Also, if you like this article, please use media sharing buttons (Twitter, G+, Facebook) below this post!

Wednesday, 16 October 2013

How to: Fix a corrupted DATA partition on the HTC One


Since the HTC One was released, I've had a few users asking me to help them fix their broken device. They couldn't boot their system or access anything on the DATA partition from inside custom recovery. Why this happens so often on the HTC One I have no idea...

Here is a short guide explaining how to fix this problem without sending the device for repair under warranty.

Short Explanation

This method can be used to fix any other partition too, just remember about changing the mount point number - from "mmcblk0p37" to the relevant one. For example, the SYSTEM on the HTC One is "mmcblk0p35". In some rare cases the mkfs.ext4 attributes (like dir_nlink or resize_inode) might change too! To get a list of all eMMC mount points on HTC devices use the command cat /proc/emmc. Never format RAW partitions! This method can be used only to format EXT4 partitions. Not following the guide properly might end in permanent damage to your device. I do not take any responsibility for that.

Some people might also find this helpful in case of need to remove the entire contents of the virtual SD-card. As long as the virtual SD-card is not a separate partition, (i.e. it is emulated space on the DATA partition [/data/media]) it can't be formatted - only wiped clean. I wrote an in-depth explanation of how the virtual SD-card works on Android here - Virtual SD card on Android.

How Do I Recognise The Problem?

When trying to mount the DATA partition from inside the custom recovery using the mount /data command you will most likely see this output: mount: mounting /dev/block/mmcblk0p37 on /data failed: Invalid argument

To be 100% sure you can check filesystem structure with the following command: /sbin/e2fsck -n -f /dev/block/mmcblk0p37

Repair Process

Note: the content of your DATA partition will be lost, including the contents of your virtual SD-card! I also assume you already have custom recovery flashed and you are a Windows user - LINUX people please amend as needed.
  1. Download this mini-sdk package and extract it to c:\mini-sdk
  2. Download this mkfs.ext4 binary and put it into c:\mini-sdk
  3. Connect your device to the PC
  4. Boot your device in recovery mode
  5. Open a command prompt on the PC (cmd.exe), type and confirm each command with ENTER:
  6. cd /d c:\mini-sdk
  7. adb push mkfs.ext4 /tmp
  8. adb shell
  9. chmod 777 /tmp/mkfs.ext4
  10. /tmp/mkfs.ext4 -b 4096 -O ^huge_file,^dir_nlink,^ext_attr,^resize_inode,^extra_isize -m 0 /dev/block/mmcblk0p37
Image of the whole command in one line to avoid mistakes:

Now your DATA partition will be formatted. It is not the same as "wiping", which is only removing all or just some files from particular partition. Formatting means that the entire partition will be re-created with the above attributes. You should see the following output:

mke2fs 1.41.12 (17-May-2010)
Filesystem label=  
OS type: Linux 
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
1703936 inodes, 6815744 blocks
0 blocks (0.00%) reserved for the super user
First data block=0
208 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768,98304,163840,229376,294912,819200,884736,1605632,2654208,4096000
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done
This filesystem will be automatically checked every 25 mounts or 180 days, whichever comes first. 
Use tune2fs -c or -i to override.
Your DATA partition is now completely empty, so we need to copy our particular ROM zip file there:
  1. Copy ROM zip file to c:\mini-sdk and rename it to "rom.zip"
  2. In the same command prompt window type and confirm each command with ENTER:
  3. mount /data
  4. mkdir -p /data/media/0
  5. exit
  6. adb push rom.zip /data/media/0
Copying rom.zip to the device will take a few minutes. Wait until it's done. The output should be: xxxx KB/s (xxxxxxxxxx bytes in xxx.xxxs)Now you can flash rom.zip from inside custom recovery. Done!

Have any questions or comments? Feel free to share! Also, if you like this article, please use media sharing buttons (Twitter, G+, Facebook) below this post!

Saturday, 31 August 2013

The Android ION Memory Manager

Lately there's been quite a bit of discussions about Android "ION". What exactly is ION? Is it just some fancy name or is there more to it?

Let's talk about some history of Android first.

Since the very beginning, vendors of Android devices like HTC, Samsung or Motorola all use different System on a Chip (SoC) solutions from Qualcomm (MSM/Snapdragon), Nvidia (Tegra) and TI (OMAP). Each SoC has its own kernel drivers for managing memory buffers (chunks of scratchpad memory) used by Graphic Processing Unit (GPU), Audio processing, and Camera Stills and Video processing.

Every vendor had their own version of memory management, such as PMEM for Qualcomm, NVMAP for Nvidia and CMEM for TI - private memory not shared with anyone else. Each Android graphics, audio and camera libraries had to be customized to work with each of the SoC's own flavour of memory management, which makes it a nightmare for the Android Maintainers to maintain the fragmentation and compatibility issues abound. However, this was the case for all pre-Ice Cream Sandwich OS like Froyo, Gingerbread or even Honeycomb.

For Android 4.0 (aka Ice Cream Sandwich), Google was finally fed up with the private memory manager structure and decreed that all newer devices with Android 4.0 native should use the new, so called "ION" memory manager.

So what is exactly the Android ION?

In a simple words, Android ION removes ARM specific dependencies. The ION memory manager provides a common structure for how memory will be managed and used by GPU, Audio and Camera drivers. Common functions are:

  • memory allocation / de-allocation
  • Direct Memory Access Pools
  • user-space (Android libraries) memory passing to/from kernel space

With these common functions and structures defined, kernel drivers from each SoC manufacturer needed to rewrite their drivers to be compatible with Ice Cream Sandwich. Once the drivers adopted to the new common structure, the graphics, audio and camera libraries can now be more generic and could care less about the nitty-gritty details of how different SoC vendors' drivers worked.

It was painful at first, but it was a necessary move for Google to impose to all the SoC vendors. Now looking back, this new ION manager enabled manufactures and third party Android projects (like Cyanogen-mod) to quickly bring up newer Android releases for various devices and also reduce the "hidden" Android fragmentation.

If you want to take a look at the code of the ION memory manager, please visit faux123 github - MSM ION

I hope you enjoyed my first Kernel GeekTalk series... more to come soon!

Have any questions or comments? Feel free to share! Also, if you like this article, please use the media sharing buttons (Twitter, G+, Facebook) under this post!

Friday, 14 June 2013

Do we really need S-OFF?

Lately there has been a lot of confusion about if we - HTC users - really need S-OFF on our devices. I think it's time to make this case as clear as possible, and clear up any remaining doubts.

First of all, S-OFF stands for "Security OFF" and S-ON for "Security ON". It's a term specific to HTC devices (and refers to digital signature checking on the bootloader "hboot"). Retail devices always come with SHIP S-ON locked bootloaders. Hboot can also be found in an engineering version (ENG as opposed to SHIP), but it's not easy to get such a device.

You should also know the "fastboot" term: it is a diagnostic protocol used primarily to modify the flash filesystem via a USB connection from host computer. After enabling the protocol on the device itself (entering "fastboot" mode from inside the bootloader), it will accept a specific set of commands sent to it via USB using a command line, for example "fastboot flash boot boot.img" or "fastboot erase cache".


What's the main difference between S-ON and S-OFF from the end-user point of view?


With S-OFF you can:

  • Flash in fastboot original parts of the firmware like: Trust Zone (tz.img), Resource Power Manager (rpm.img), Advanced Digital Signal Processor (adsp.img), bootloader (hboot.img), Radio Config Data (rcdata.img), Splash Screen and others, very often device specific firmware like Consumer IR (cir.img) for the television remote controller in HTC One.
  • Flash in fastboot custom parts of the firmware above, however I've never seen in my life anyone compiling custom rpm.img or tz.img. I've seen custom bootloaders and Splash Screens only. You can also flash modified radio.img but there is rarely anyone out there who does this.
  • Use more advanced fastboot commands, for example you can change the CID (Carrier ID) of your device or even MID (Model ID). And this one is the most important one in the context of this article.
  • Reset the Tampered flag, so your device does not show up as "Relocked" if you relock your bootloader.

For about 2 years you have been able to unlock bootloaders of selected HTC devices on the htcdev.com webpage. Unlocking your bootloader results in an "UNLOCKED" message in the bootloader screen, and allows you to use some of the fastboot commands. For example system, boot and recovery partitions are no longer locked and you can flash a custom boot or recovery onto your device. This doesn't mean S-OFF, but it does give you some more control over your device.

Sometimes there are differences specific to the SoC ("System on a Chip") of each device. Both HTC One X and One X+ (nVidia Tegra 3) have locked out the capability to flash the boot partition from inside recovery, even if your bootloader is unlocked. It is possible to flash the boot partition only via the "fastboot flash boot <boot image name>" command. On the newest HTC smartphone - HTC One (Qualcomm Snapdragon S600) you can use either fastboot or adb shell (dd if=/... of=/...) to write the boot partition.

Do we really need S-OFF?

No, we don't. So what do we need? Because we surely need something. But to understand what we need, it's important to realize where the problem is first.

First of all, comparing HTC devices with Nexus devices is a pointless activity. Never do that. Why? Because they are all S-ON (they call it Secure Boot), and updates for Nexus devices contain the following (based on my experience with Samsung Galaxy Nexus):
  1. bootloader.img
  2. recovery.img
  3. GSM radio
  4. CDMA radio (in case of CDMA device)
That's all. On Nexus device you can flash the original bootloader or radio using the "package_extract_file" command in the updater-script. When HTC releases a major update, however, you will get:
  1. adsp.img
  2. cir.img
  3. dzdata_16g.hdr
  4. dzdata_16g.img
  5. dzdata_32g.hdr
  6. dzdata_32g.img
  7. dzdata_64g.hdr
  8. dzdata_64g.img
  9. bootloader.img
  10. radio.img
  11. recovery.img
  12. rpm.img
  13. sbl1-1.img
  14. sbl1-2.img
  15. sbl1-3.img
  16. sbl2.img
  17. sbl3.img
  18. tp.img
  19. tz.img
  20. more...
See the difference? This firmware images (if updated) are stored inside firmware.zip inside the OTA update. And without S-OFF you can manually update (using fastboot commands or command shell) only recovery, boot, system and sometimes radio. Other partitions are locked and you can't update firmware images other way then only with signed firmware.zip.

Content of HTC OTA update

However, very often, flashing only the content of the system and boot partitions is not enough to have the device fully working. For example, in the HTC One X it was necessary to use the new bootloader together with the official HTC Jelly Bean update, otherwise your device wouldn't boot with an older bootloader. This is why flashing a custom ROM for an HTC device is nowhere the same as flashing a custom ROM on a Nexus device. Apart from having the latest system files, you need to have the latest firmware.zip package flashed as well.

Because HTC sells their devices to different carriers around the world, they need to accept some requirements. For example carrier branding. Because of carrier branding, HTC has more than one version of the RUU (ROM Update Utility) for each device. To indicate the difference between the branded and un-branded versions of the same device, HTC used so called "CID" numbers. 

To find out your current CID number (together with some other useful info) you can use the "fastboot getvar all" command. Also, keep in mind that every OTA update checks CID/MID numbers before it will start to patch your system:




                         ifelse( is_ship_bootloader(getprop("ro.bootloader")) == "t" ,
                         assert(check_cid(getprop("ro.cid"), "00000000" , "11111111" ,
                         "22222222" , "33333333" , "44444444" , "55555555" , "66666666" ,
                         "77777777" , "88888888" , "99999999" , "HTC__001" , "HTC__E11" ,
                         "HTC__102" , "HTC__203" , "HTC__405" , "HTC__Y13" , "HTC__304" ,
                         "HTC__032" , "HTC__A07" , "HTC__J15" , "HTC__016") == "t"););
                         ifelse( is_ship_bootloader(getprop("ro.bootloader")) == "t" ,
                         assert(check_mid("full", "PN0710000") == "t");,
                         assert(check_mid("simple", "PN0710000") == "t"););

Obviously "check_cid" includes also SuperCIDs (00000000, 11111111, ...).
Content of android-info.txt
It's all in updater-script, so it can be easily edited anyway. But the real problem is different. As mentioned already, every OTA update contains firmware.zip - package with bootloader, radio, touch panel drivers, trust zone and other parts of important firmware. It also contains the "android-info.txt" file, where CIDs/MIDs are listed, so your S-ON bootloader won't let you flash an original firmware.zip if your CID number is not listed there. Yes, I'm not talking here about custom radio, bootloader or anything custom at all. Original, untouched firmware.zip from an OTA update can't be flashed onto the device if the CID number doesn't match. Is it a problem? Yes, this is the real problem we're dealing here with. Not S-ON/S-OFF, but CID restrictions and an inability to change the CID number.

How this can be resolved? "android-info.txt" is a plain text file, so it can be edited easily. If your CID number is not on the list, just add one more line with your CID. However, as long as your device is S-ON, you won't be able to flash it, because every firmware.zip is signed with a special key. Once firmware.zip is modified, the signature is broken and the bootloader will reject the request to update it. But there is a different method: you can change the CID number on your device with a fastboot command "fastboot oem writecid <cid number>". The best CID number to use is one of the WWE CIDs (for instance HTC__001). But wait - you can't use this particular fastboot command without S-OFF.

Is this problem a real one, or just some sort of users ill-informed craving? It's very real, because without the ability to flash firmware.zip from a WWE OTA update, every user from any carrier or different world region is forced to wait months to receive OTA updates customized to his CID. Everyone can de-brand his device easily by flashing a stock system image, but it won't be enough: because firmware.zip with corresponding parts of the firmware is needed at the same time. This isn't about the OTA itself, it's about the firmware.zip inside that OTA update.

Dangers:

So what are the dangers of obtaining S-OFF on your device? Some of the partitions in the device are extremely sensitive and can result in your device being bricked if they are even slightly corrupted. With S-OFF you can access all of these partitions and the slightest corruption during transfer (whether that be a power spike or you jiggled the cable slightly) can result in a bricked device as it does not check for signatures.

Here’s an example which has almost happened to me once on an S-OFF device: I was flashing a boot.img via fastboot, the command is: "fastboot flash boot boot.img". However I had made a small but significant typo: "fastboot flash hboot boot.img", simply by mis-hitting the B key; this command would be rejected by a device with S-ON as it is a protected partition, but would be accepted on a device with S-OFF. If I had pressed enter without checking the command, my device would have turned into a paperweight in seconds.
One of the most popular protected partitions the hacking community enjoy flashing is the radio partition. This is also a partition where the slightest corruption will cause your phone to brick. The FCC guidelines state the the radio must be booted with a separate processor (I guess to decrease the risk of it being tampered with), so what happens in a phone when it turns on is: radio is booted via a dedicated processor by the first stage loader, initialising the radio hardware (Wifi, Data, Bluetooth, etc.). Radio successfully boots and initiates the first stage loader to use the main CPU to load the second stage loader into RAM (also known as the SPL). Depending on the boot operation, it will either initiate the system or recovery. So without a functioning radio, the main CPU will not kick on and boot the phone.

Some other facts:
  1. You don't need S-OFF to root your device.
  2. You don't need S-OFF to be able to run Titanium Backup or other applications that requires root access. You just need root privileges for that.
  3. You don't need S-OFF to flash custom recovery image onto your device.
To summarize:

We don't need S-OFF, but we do need the ability to edit the CID number on the device (let's say at least on officially UNLOCKED devices), or the firmware.zip packages inside an OTA update should not be signed, so that "android-info.txt" can be easily edited, or the CID restrictions from android-info.txt should be removed (MID is enough to ensure that the right firmware gets to the right devices).

Something to re-think?

Even if we don't need S-OFF I'm quite worried about the policies of mobile companies and carriers. Their philosophy is "the more you are locked down, the more you are protected". That means Police should not fight with criminals, but everyone should just lock down their doors, windows and stay at home instead. It's far easier and cheaper to lock down mobile devices and not allow root access rather then improving the security in other areas.

Can you imagine that you just bought a brand new notebook for $3000 and:
  • you can login only as a Guest (no Administrator account available by default),
  • you can't change your operating system,
  • you can't use applications that requires Administrator privileges,
  • you can't browse freely the content of your hard drive.
You would say "Where the hell is my freedom?!" Here comes the answer from your notebook manufacturer - "For your own security, you don't have any freedom". Sounds like a George Orwell story to me.

I want the same freedom on my phone that I have on my PC.


This article was written in a cooperation with Shen Ye


Have any questions or comments? Feel free to share! Also, if you like this article, please use media sharing buttons (Twitter, G+, Facebook) down this post!


PS. I want thank to Tom Kelsall, my HTC Elevate companion for his help in a proper grammar redaction of the review! Thanks Tom!