Let's Get Technical

GoogleCL Docs Upload Fixed for Gdata 2.0.15 and 2.0.16

2012-02-02T00:48:00.001-08:00

Months ago, we fixed Google Docs Uploads for GoogleCL folks who use python-gdata 2.0.12 through 2.0.14. Today, we tackled the problem for users of python-gdata 2.0.15 and 2.0.16.

A user sent in a patch that fixed the MIMETYPES error we were getting with 2.0.15 and 2.0.16. It looks like python-gdata moved away from declaring MIMETYPES where we were calling them from, so now we name them explicitly in googlecl/src/googlecl/docs/client.py.

That was a great start, but then we started getting a DocsEntry error and uploads still wouldn't work, so we delved further. Here's the error we got after fixing the MIMETYPES problem.

AttributeError: 'module' object has no attribute 'DocsEntry'

One of our earlier searches brought us to the same error posted on python-gdata's buglist.

Then, we found this blog post, by Demon Lord, where they had success replacing the string "gdata.docs.data.DocsEntry" with the string "gdata.data.GDEntry". We tried that throughout the file googlecl/src/googlecl/docs/client.py and sure enough, that fixed it!

We tested 2.0.0 through 2.0.4 to make sure that didn't break anything for those users and then tested 2.0.12 through 2.0.16 to make sure it worked on all the versions that should be able to upload docs. It does so it's now been committed.

Car Troubleshooting

2012-01-31T18:46:00.000-08:00

I'm going to use this blog the way I talk about using it: as a place for keeping all my technical stuff. Today, that includes cars.

Car 1 (the favorite) leaks oil like crazy. However, upon closer inspection tonight, I have some evidence to support my hypothesis that it only leaks when it's on (possibly only while she's in park--that was harder to test by myself). I checked the oil today, after it had been off for at least a week, and the oil looked alright. I would normally add oil, but my husband wouldn't. I didn't add any. Then, I turned the car on to check the transmission fluid levels (also has a leak, but a much slower one). Transmission levels appear fine. Turned the car off and decided, just for the hell of it, to check the oil. It was on the "L" bar! Sure enough, there was a big puddle of oil under the car, too. I watched the oil and couldn't see the drips at all. Then, I turned the car on and it dripped once a second! After the car was off, it dripped 9 times the following minute. Hopefully it's the gasket and hopefully it's not a tricky gasket to replace. . . The transmission was easy enough -- drain transmission, take off pan, put on new gasket, put pan back on. Hopefully the oil pan is the same!

MIMETYPES fix gives AttributeError: no 'DocsEntry'

2012-01-31T15:43:00.000-08:00

The same gentleman who helped make it possible to fix the docs upload bug also suggested the start of a fix for the MIMETYPES bug we get with python-gdata versions 2.0.15 and 2.0.16 with the fix (with or without this fix, we can't upload docs via python-gdata 2.0.15 or 2.0.16, so the fix was still worth publishing).

The error I get now is:
AttributeError: 'module' object has no attribute 'DocsEntry'

A quick grep of the whole googlecl tree for the term "DocsEntry" shows me that it only exists in gdata versions 2.0.11 and 2.0.14. . . So why does GoogleCL call for it in the trunk/src/googlecl/docs/client.py file? The call for it seems to work with python-gdata 2.0.12 -- I can upload documents of txt and pdf types without a complaint about DocsEntry.

Edit: Resolved this issue and blogged about it here.

Testing GoogleCL Part 3

2011-11-25T14:34:00.001-08:00

Apparently, you don't need to backslash the apostrophes in posts, though. Interesting.

Testing GoogleCL

2011-11-25T14:30:00.001-08:00

If you can see this, then it worked again. Apparently, you need to put a backslash before a \! character, but then it shows the backslash, too. Also, as with the \! character in the post, I had to put a backslash before the apostrophe in my blog's title name (Let\'s Get Technical)

Testing GoogleCL

2011-11-25T14:19:00.001-08:00

If you can see this, then it worked\!

Python Gdata

2011-11-20T01:04:00.001-08:00

Short version: To upload Google Docs with a service that utilizes python-gdata, use a version between 2.0.12 and 2.0.14. Keep reading for more details.

Credentiality and I spent a few hours on this tonight. Here's a brief history and a beautiful table to show you what to expect.

An SSL error occurs in a few platforms using gdata when you try to upload files to Google Docs and it doesn't appear to be limited to python-gdata. It may relate to this. Credentiality and I work on GoogleCL sometimes, so we have all the python-gdata versions from 2.0.0 through 2.0.15 and test code with all of them, anyway. Here are some observations and what has worked for us so far.

In May 2, 2011, Google began to allow uploading "all file types" for non-premium users. Now, if you add a resumable uploader fix to a specific GoogleCL client.py file* and you use python-gdata 2.0.10 or 2.0.12 through 2.0.14, GoogleCL should upload a PDF to Google Docs successfully.

Without the resumable uploader fix, python-gdata 2.0.10 or 2.0.12 through 2.0.14 gives us this 403 error:

403, <errors xmlns='http://schemas.google.com/g/2005'><error><domain>GData</domain><code>ServiceForbiddenException</code><internalReason>Files must be uploaded using the resumable upload mechanism.</internalReason></error></errors>

When we use python-gdata 2.0.5 through 2.0.9 or 2.0.11, we get this 403.4 SSL error:

403, <errors xmlns='http://schemas.google.com/g/2005'><error><domain>GData</domain><code>ServiceForbiddenException</code><internalReason>403.4 SSL required</internalReason></error></errors>

The python-gdata versions before 2.0.5 can't upload PDFs to Google Docs, anyway. We get the following error for those:

{'status': 415, 'body': 'Content-Type application/pdf is not a valid input type.', 'reason': 'Unsupported Media Type'}

python-gdata 2.0.15 doesn't work with GoogleCL to upload to Google Docs, either. Currently, we get this error, which seems to be a GoogleCL error related to some change in gdata from 2.0.14 to 2.0.15:

from gdata.docs.data import MIMETYPES
ImportError: cannot import name MIMETYPES

*In the directory: googlecl/src/googlecl/docs/ .

Hacking Gedit

2011-07-10T20:37:00.000-07:00

If I have time this summer, I'd like to help fix bugs in gedit. I LOVE gedit and I want to contribute. Here's a heads up for those of you installing gedit from the git repository instead of from the tarball which, I believe, is what they recommend. . . but it's certainly what I prefer, anyway.

I edited their README to include install directions for people checking it out from git. We'll see if/when they take the patch. They have a good reputation for accepting help.

Simple install procedure for gedit if you checked out the source code from git:

(you may have done this already) download the most recent version of gedit
% git clone git://git.gnome.org/gedit

change to the toplevel directory
% cd gedit

generate the `configure' script
% ./autogen.sh

run the `configure' script
% ./configure

build gedit
% make

[ Become root if necessary ]

install gedit
% make install

Chem 101

2011-06-30T19:45:00.000-07:00

My study partner and I find our solubility chart a bit muddled for quick equation balancing. Here's our answer to this problem.

Technical Skills Update

2011-02-28T20:46:00.000-08:00

This quarter, I'm taking a Shell Programming class. I keep finding things that would have been useful to know *before* I started hacking the Linux kernel and AOSP. It's an interesting class, though. Better late than never, right?

gedit

2011-02-28T20:44:00.000-08:00

I love that I can highlight a block of code and comment or un-comment it all with two clicks of the mouse! Saves so much time!

Reboot-Persisting All-User-Access USB Serial Port on Froyo

2010-09-07T10:21:00.000-07:00

I turned the USB into a serial port. Then, I noticed it wasn't usable without superuser permissions. Recently, I got some great information from a former Android coder and have now have a permanent fix that persists on reboot with Android 2.2 (Froyo).

First, I edit init.mahimahi.c
(mahimahi is the board for Nexus One (aka HTC Passion), if you're working on another HTC device, you'll need to edit the right file for your device).

$ gedit path/to/device/htc/passion/init.mahimahi.c

I put the following at lines 64 and 65.

#mwk: Enable Serial Port for all users
chmod 0777 /dev/ttyMSM0

I run lunch.

~/mydroid$ . build/envsetup.sh
~/mydroid$ lunch

I choose option 4 (full_passion)

I plug in a Nexus One and extract the device info.

~/mydroid$ cd device/htc/passion
~/mydroid/device/htc/passion$ ./extract-files.sh

I re-make the AOSP.

~/mydroid/device/htc/passion$ cd ../../..
~/mydroid$ make

I get the phone in bootloader mode (the fast way).

$ adb reboot-bootloader

I flash the device. (On this website, step 14 didn't work for me, so I now run the command this way.)

~/mydroid/out/host/linux-x86/bin$ sudo ./fastboot -p passion -w flashall

On the phone, I enable applications from unknown sources.

Settings--> Applications --> Unknown sources --> checked

Also, on the phone, I enable USB Debugging, Stay Awake, and Allow Mock Locations.

Settings--> Applications --> Development --> checked all three

Using Newly-Compiled AOSP OS WITH My Hacked Kernel

2010-08-30T17:03:00.000-07:00

(The rooting part—steps 3 & 4— should not be necessary.)

1. Compile Android 2.2 v 1.1
2. Flash complete Android 2.2 v 1.1
3. Download UniversalAndroot-1.6.2-beta.apk from computer to phone

$ adb install /path/to/Universal-Androot-1.6.2-beta.apk

4. (On the phone.) Run Universal Androot for Android 2.2 from listed apps
5. Get phone into bootloader mode: (Back on computer)

$ adb reboot-bootloader

6. Split the Android 2.2 v1.1 "boot.img" file into "boot.img-kernel" and "boot.img-ramdisk.gz".

$ ./split_bootimg.pl boot.img
Page size: 2048 (0x00000800)
Kernel size: 2206592 (0x0021ab80)
Ramdisk size: 167182 (0x00028d0e)
Second size: 0 (0x00000000)
Board name:
Command line: no_console_suspend=1 msmsdcc_sdioirq=1 wire.search_count=5
Writing boot.img-kernel ... complete.
Writing boot.img-ramdisk.gz ... complete.

6. Rename "zImage" (newly-compiled kernel) as "boot.img-kernel"

$ mv path/to/zImage other/path/to/boot.img-kernel

7. Zip up "boot.img-kernel" (formerly zImage) with "boot.img-ramdisk.gz"

./mkbootimg --kernel boot.img-kernel --ramdisk boot.img-ramdisk.gz -o my-boot.img --cmdline 'no_console_suspend=1 msmsdcc_sdioirq=1 wire.search_count=5' --base 0x20000000

8. Flash phone with new "my-boot.img" file

$ sudo ./fastboot flash boot my-boot.img

Compiling the Android OS: Take 3 -- It Works!

2010-08-27T12:48:00.000-07:00

Last night, I followed the directions here:
http://forum.xda-developers.com/showthread.php?t=724240

I got to ~/mydroid/$ make and then let it compile last night. This morning, there was an error which is posted on this site: http://www.androidiani.com/forum/144139-post15.html

Basically, it was "libgps.so, not found"

The following blog post has great instructions and a fix for this error, so I followed his directions from that point on. Thank you, Al Sutton!
http://blog.alsutton.com/2010/06/29/building-aosp-for-the-nexus-one/

I flashed and rebooted the phone and it worked!

1. From the top of the source tree go to out/target/product/passion/
2. Run “sudo fastboot flash system system.img”
3. Then run “sudo fastboot flash boot boot.img”
4. Then run “sudo fastboot flash recovery recovery.img”
5. Then run “sudo fastboot flash userdata userdata.img”
6. Then run “sudo fastboot reboot”

The phone's display looks right again and it responds to the buttons again! :D I have a working compiled OS to work with now!

Compiling the Android OS: Take 2

2010-08-27T01:32:00.000-07:00

Turns out, it's a bit more complicated than I previously thought. I ran "make clean" and then followed the instructions I've posted below. I've just run "make" again (without the -j3). I'm letting it compile for the night and tomorrow morning I will hopefully be able to finish the next steps and flash the phone successfully this time!

Thanks to lpasq at xda developers for posting the code I needed!
http://forum.xda-developers.com/showthread.php?t=724240

#clean out prior compiled files
make installclean
make clean

#update AOSP froyo source files
repo init -u git://android.git.kernel.org/platform/manifest.git -b froyo
cd ~
repo sync

#setup compiling environment and build
cd ~
. build/envsetup.sh
lunch

#USB connect N1 and initiate ADB
cd ~
adb devices

#ADB pull needed device files
cd ~
cd device
cd htc
cd passion
./extract-files.sh

#build the Passion *.img's
cd ~
make -j3

#view compiled folder's new boot.img/system.img/userdata.img
cd ~
cd out
cd target
cd product
cd passion
echo 'see flashable files:'
ls *.img

#create update.zip
cd ~
make -j3 otapackage

A Note on the Enabled Serial Port for Nexus One

2010-08-26T15:02:00.000-07:00

It worked! Then, I flashed it to the phone and it persisted on reboot!

However, our apps could not access the newly-enabled serial port without superuser permissions. We fixed that this way.

$ ./adb shell
$ su
# chmod 777 /dev/ttyMSM0

Then, after rebooting the phone, the permissions for /dev/ttyMSM0 went back to normal (or were re-written on boot up).

For a phone running Cyanogen 6, I ran:

$ adb shell
$ su
# mount -o remount,rw /dev/block/mtdblock3 /system

I then pushed a file named 99foo to the phone's /etc/init.d/ directory (/etc/ is a symlink to /system/etc). The complete contents of 99foo:

#!/system/bin/sh
#makes USB-serial available to all users
chmod 777 /dev/ttyMSM0

I changed the permissions for 99foo so that they matched the permissions of the rest of the numbered files in this folder. I saw the existing permissions for 99foo as well as the rest of the numbered files by running this command:

# cd /etc/init.d
# ls -l

Then, I used chmod to add and take permissions as necessary. Then, I rebooted the phone. Now, the serial port is accessible to all users all the time!

For the phone running Android 2.2 with our hacked kernel, I will either edit the init.rc to allow access to /dev/ttyMSM0 for all users or, more likely, I will edit the init.rc file to run another file that I will write. Since I cannot edit init.rc while it's on the phone, I used this command (from my comPuter) to copy the init.rc to a location on my computer:

$ adb pull /init.rc /home/myname/init.rc

I will edit it. Then, I'll put it back with this command.

$ adb push /home/myname/init.rc /init.rc

I'm using this page for help.
http://www.androidenea.com/2009/08/init-process-and-initrc.html

I won't get to this very soon, as I have a more pressing matter to resolve. I'll blog more about it after I make some more progress.

Compiling the Android OS

2010-08-26T14:21:00.000-07:00

I followed the directions on this site:
http://source.android.com/source/download.html

When I ran "make" from the "mydroid" folder, I kept getting this error:
error: gnu/stubs-64.h: No such file or directory

I'm using a 32-bit system, so I don't have a stubs-64.h. The installation guide says 64 bit systems are untested and they recommend a 32-bit system, so that seems like a strange error. Until, we did some looking around online and found out that the command they give on the installation guide points to something other than the stable froyo release (top of trunk, which I guess may be gingerbread or some updated version of froyo?).

Then, I ran this (almost the same, but I specify which version of Android OS to reference):

$ repo init -u git://android.git.kernel.org/platform/manifest.git -b android-2.2_r1.1
$ make

It compiled completely this time and now it's time to flash it to a phone!

Enabling the Serial Port on a Nexus One

2010-08-17T23:40:00.002-07:00

The Nexus One USB port works as a serial port when there's no power coming into the connector. But the stock kernel sets it up as a console. To use it as a general-purpose serial port, you have to use a kernel with CONFIG_USB_ANDROID_ACM enabled and CONFIG_MSM_SERIAL_DEBUGGER disabled.

If you want to skip installing the NDK, downloading and compiling the kernel, you can try this precompiled zImage.

We did this on a rooted Nexus One with an unlocked bootloader running Cyanogen CM6. It's probably not necessary to be running Cyanogen for this to work.

Download NDK from here:
http://developer.android.com/sdk/ndk/index.html

Install NDK using instructions from the website.

Make sure arm-eabi-gcc is in your path. It's in android-ndk-r4b/build/prebuilt/linux-x86/arm-eabi-4.4.0/bin.

(export PATH=$PATH:/path/to/android-ndk-r4b/build/prebuilt/linux-x86/arm-eabi-4.4.0/bin to add it)

Make sure your machine has git: $ sudo apt-get install git-core

$ git clone git://android.git.kernel.org/kernel/msm.git
$ git branch -a
$ git checkout -b android-msm-2.6.32 origin/android-msm-2.6.32
$ cd msm
$ ARCH=arm CROSS_COMPILE=arm-eabi- make mahimahi_defconfig
$ ARCH=arm CROSS_COMPILE=arm-eabi- make menuconfig
Exit the menu and say "yes" when it asks if you want to save.

$ gedit .config
Turn on CONFIG_USB_ANDROID_ACM (uncomment it and change "is not set" for "=y").
Turn off CONFIG_MSM_SERIAL_DEBUGGER (comment it out)
Turn off CONFIG_MSM_FIQ_SUPPORT
$ ARCH=arm CROSS_COMPILE=arm-eabi- make

The kernel image should compile and is found in arch/arm/boot/zImage

Try kernel without permanently storing it on the phone. Download fastboot from here:
http://developer.htc.com/adp.html

Reboot into fastboot mode. Power up the Nexus One while holding the trackball. Plug in the USB cable and the fastboot screen should say "fastboot usb".

$ sudo ./fastboot boot /path/to/msm/arch/arm/boot/zImage

If you get this message, then you probably forgot to run as root.

sm/arch/arm/boot/zImage
creating boot image...
creating boot image - 2232320 bytes
< waiting for device >

Once the phone boots up, use adb shell to ls /dev/tty* and if you see dev/ttyMSM0, it worked. (We haven't tried actually using the port yet).

When you reboot you'll be back into your old kernel. Haven't flashed this for real yet.

Check out the xda-developers.com thread linked below for more info on actually using the serial port.

The following sites were helpful:
http://androidboss.com/custom-linux-kernel-for-nexus-one/
http://forum.xda-developers.com/archive/index.php/t-625434.html

Hello World!

2010-08-17T23:40:00.001-07:00

I have a technical blog.

The end.