About Culture – overheard on HN

“But culture is not a set of specific abstract principles. It’s the set of mutual values, expectations, conventions, and methods of communication that define the way people within your organization interact with each other and work together to achieve the company’s goals.

You guide the development of your culture not by fiat and ideology, but by carefully structuring the organization in a way conducive to healthy growth, by selecting the people with the right mindset to put in positions of responsibility, and most importantly, by setting the example in the way that you act and make decisions.

If you proclaim your company officially vegetarian, here are the effects I see to culture, as understood in this way:

* Alienate members on your staff who are not vegetarian.

* Blur the distinction between personal preferences and business strategy.

* Establish a precedent that the founders’s personal opinions on issues wholly orthogonal to the business are still valid grounds for policy-making (imagine the consequences of future managers emulating this).

In the long term, you’re probably moving toward a culture with an unhealthy amount of internal contention, and a staff that may not feel fully invested in the company’s goals.”

https://news.ycombinator.com/item?id=1713486

Hacking the SD960/IXUS110 for scuba photography (using RAW)

The SD960 is the cheapest camera that I can buy with a decent scuba housing – the WP DC32. However, the reason why this camera is dissed is due to its inability to shoot RAW, which is near essential for underwater photography of any kind.

My current trip to the underwater wrecks of Sri Lanka did not get good pics because of this problem. However, there is a solution – the CHDK project (Canon Hack Development Kit) for the SD960 (forum). The best part is that it doesnt impact your camera firmware at all – every time your camera boots, CHDK is temporarily loaded into memory.

The way to hack the SD960IS is fairly simple – first identify your current firmware. I used the ACID utility to do that. Mine was 101d. Incidentally, ACID downloaded the firmware for me, but you can download it yourself here. Unzip the firware zip somewhere. Inside there somewhere would be four files/folders:

• DISKBOOT.BIN
• PS.FI2
• vers.req
• CHDK/ (folder)

Prepare your SD card:

• Make sure your SD card is NOT locked (yes – I wasted many a hour because I forgot this)
• Partition your SD card to have two partitions (note: I used gparted to do this, use any tool that you want). A 16 mb FAT16 partition and the rest as a FAT32 partition. The larger partition is where your pictures will be stored.
• Make sure the 16 mb partition has “BOOT” flag set. In Gparted, you need to right click -> “manage flags” -> check “boot”
• Copy the four files/folders above to BOTH partitions.

Start your camera by pressing the play button not the on/off switch. In your play menu, at the very bottom, there will be a new option called “firmware update”. Start the firmware update. CHDK should now be temporarily loaded into camera memory. You should see a brief flash of the CHDK splash screen. In any case, press “play” and then “menu” button. You should now see a different menu.

Go down to “Miscellaneous Stuff -> Make card bootable”. If this does not work for you, then the linux command that does is

 echo -n BOOTDISK | sudo dd bs=1 count=8 seek=64 of=/dev/sdb1 (where sdb1 is your 16 mb partition)

Now, switch off your camera, take the card out and click the SD card pin to the lock position (yes lock position. CHDK will ignore it and let you click photos without a problem. This is needed for the camera to load CHDK on every start).

Now your SD card will boot CHDK every time the camera is switched on – if you get a new SD card, you’ll have to redo the steps above, since the camera is unaffected by any of this.

Enable RAW mode – this is what we have been waiting for ! Press “play” and “menu” and enter “RAW Options”. Just check the “Shoot in RAW” option and you will generate a RAW picture everytime. This is stored in a different folder called “100Canon“. The RAW format used is DNG which is the universally accepted RAW format.

The very first thing that you should do is press the shutter and get into photo shooting mode. Now pull up the CHDK menu (Play + Menu). Go to “RAW Options” and go to “Create badpixel.bin“. The camera will automatically take a couple of pictures for you. This eliminates bad pixels for you when you are shooting RAW.

Enjoy!

Why Ubuntu should use Android Surfaceflinger instead of investing in Mir and pulling a “Bada” move

The Ubuntu foundation recently released a new spec for a new display server called Mir. This comes on the heels of a similar discovery by Phoronix that Ubuntu touch was using SurfaceFlinger under the covers as well as an announcement that Unity will be switching to Qt/QML for its next version.

I think investing in Mir is a poor move and Ubuntu seems to be all over the place. Most top end device SoC manufacturers will only release Android compatible binary drivers for a long time to come. Even if they go Linux, they will most likely stick to xorg compatible drivers (because the big enterprise Linux gorilla – Redhat – is still invested in Xorg).

If one has already made significant investment in porting Ubuntu userspace to Android+Surfaceflinger (on touch) why would you NOT use that work and port it over to the desktop. There is significant opportunity for innovation on the Android stack and though Google would need a little pushing to allow Ubuntu as a special tech member, it can still be done.

And here’s the most important reason of all – yesterday, I had to attend an important GoToMeeting. I used my Ubuntu desktop and clicked the link only to be faced with a “unsupported platform message”. I dont know why, but I decided to try it on my Android mobile device. I installed the GoToMeeting app – clicked on the email and in less than 2 minutes from the time I got the Linux-unsupported message, I was connected to my meeting.

Android already has a huge, huge, HUGE ecosystem that works frikking well – why would you not want to leverage that ? Ubuntu is trying to pull an Opera Browser or a Bada here with its Mir announcement – and we all know how that panned out.

Why the PS4 is going to be dead in the water

The PS4 was just announced – without an actual console or pricing. But that’s not the worst part – the worst part, the part that is going to kill it is this: no backwards compatibility with PS3 games except through streaming.

The PS4 just signed its own death warrant – there is no way every house in the PS3′s demographic is wired for that kind of bandwidth. Especially in Asia.

I dont even understand why the Sony execs would take a step like this – Especially with the Steam box/”Piston” and the SHIELD device, the console makers are already threatened.

And now they eliminate the single biggest reason to stick with the Playstation platform – sunken costs of game titles.

I personally cant wait for Valve’s Piston – especially if it is going to be Android based.

Sending email using Amazon SES with attachments from php

This script is crazy – there is no known sane example on how to do this, so I had spend an inordinate amount of time hacking together this script. Basically, you need to replay the SMTP email composition to Amazon SES.

This code snippet should help everyone who is trying to put together a way to send email with attachments using SES from PHP.

Easiest way to disable alt-tab grouping in Unity (Ubuntu 12.04)

The Unity shell enables grouping of windows in alt-tab. This is seriously irritating and counterproductive to a lot of us.

I see constant questions on how to disable this and most of it involved installing some tool and checking/unchecking boxes in sequence.

Well, this is an automated script to enable the same – you can even use this if you are doing automated deployment.

Using BCM43142 (Vostro 2420, XPS, etc.) in Linux Mint or Ubuntu

So the BCM43142 chipset is supported by Dell, but I dont think they have merged their changes upstream. The official deb available from Broadcom is outdated.

To use the newer one, download the deb from here – http://jas.gemnetworks.com/wireless-bcm43142/.
Also run “sudo apt-get install linux-headers-generic build-essential dkms” and then install the deb package and reboot.

Wifi should work fine – I tested with a Vostro 2420 on Linux Mint 14 (IMHO much better than Ubuntu)

The fastest way to transfer files on a One X/S, Nexus 7 or Samsung S3, why MTP sucks… and a note to Google engineers

The most recent iterations of the top-end Android devices have start using MTP as the mode to transfer files on their devices. Previously, it was possible to mount the internal (or a microSD card, if available) as a USB drive on your computer – through something called MSC mode. Google’s engineers made the conscious decision to drop that mode. I do believe the choice of dropping MSC to be a valid one (read here for Dan Morrill’s interview as to why).

I am completely unsupportive of the choice of MTP as the alternative. Google really screwed up on this one.

MTP is horrible – it has no notion of timestamps, it is super horrible on performance and it is really screwy to implement. There have been multiple implementations on linux – mtpfs, jmtpfs, go-mtpfs (this one was built by a Google engineer, tired of the poor performing alternatives). Most of them use the core libmtp, which by the way doesnt play too well with One X.

So.. what’s the alternative ? Well, here you cant but admire Apple’s engineering choices going back a decade. I’ll try to explain it in brief, but you can read about it in much more detail by googling for AFC – Apple File Connection protocol and the usbmux service.

Essentially what Apple did was the following:

1. run a secure jailed service on the iPhone called afcd – which acts like a server
2. build a desktop service called usbmuxd which connects to the iPhone over USB and creates a TCP tunnel. This way, iTunes and other applications talk to the iPhone using the robust TCP protocol. (note: there is open source python code which allows you see how this is done.)

Now, it is true that not many applications other than iTunes can talk to the iPhone. But that is a very conscious, deliberate businessdecision. It has nothing to do with the superiority of this solution, which as we shall see shortly can be done easily on Android.

If we look in the android world, most of the pieces are in place – however, they have not been made available to you easily. For example, there is a piece of software called ADB, provided by Google, which allows you to connect to your phone and do a whole bunch of nifty things. This includes extremely fast file transfer over ssh if you want.

All you need to do is the following (this method should work perfectly on Linux as well as Mac/OSX. There is no reason why this should not work on Windows):

• run a ssh server on  your android device – I chose SSHDroid, but feel free to choose anything you like. Take note of the username, password and port number. Let us say the port number was 2222. Start the ssh server.
• download and install android SDK on your machine. Get to the stage where running “adb devices” shows your phone. You can google for this, but it is not too hard. You need to make sure USB debugging is enabled on your phone.
• create a port forward between your desktop and the android phone using USB bridge (see what I did there ?) using adb forward tcp:2222  tcp:2222 . This essentially causes any SSH connections on your desktop’s localhost and port number 2222 to be forwarded over USB to the android device.
• Use a nice GUI tool like filezilla to open a SFTP connection to localhost:2222 . Use the same username and password that you had set when you installed the ssh server on your android phone.
• Transfer files. I got upto a 1mbps transfer rate using my old crappy laptop.

My request to Google engineers building Android 5.0 is this – drop this MTP crap. Use a tried and tested protocol like ssh and TCP as the media link between an Android device and a desktop. What is missing is the server part. I would prefer not to install a ssh server from the app store. Instead, build a secure, chroot-jailed ssh server inside the Android core – a smart (only comes up when USB is connected), power efficient and very performant service. Things like sync over ssh are already solved problems. In fact, you can also mount a ssh connection as a drive in your file manager.

Drop the MTP crap – it doesnt work very well and it doesnt make sense against the alternatives.

Why are Java frameworks so convoluted and complex.

Take a look at Java’s Spring – it has layers upon layers of proxys, abstractions, factories, etc. For example, there is a class in Spring that is described as “Convenient proxy factory bean superclass for proxy factory beans that create only singletons.”

I have a simple question: is the root to all of this infrastructure problems (proxys, abstractions, etc.), the fact that Java introduces a fear of changing state (i.e. the existence of the “private” keyword) ? Since Java guards private variables so effectively and proselytizes a fear of state, the natural evolution of the community to build these layers.

Clojure is not a good comparison, since it avoids fear using immutable state. Ruby/Python is also not a good comparison since, well, it does not respect the privacy of state in any manner. The frameworks that have naturally evolved there, reflect the philosophy of need-to-keep-things-simple (which is the only way to work in a world where state cant be abstracted out).

Perhaps if you ask someone to build a Java framework WITHOUT using the “private” keyword, maybe it will lead to a whole different revolution

Justification for “release fast – release frequently” from a queuing theory point of view

I was looking at the long respin cycles taking place in some of our projects and I came to the conclusion that we are not factoring in WIP (work-in-progress) inventory during product design. In more simple terms, it means “release fast – release often”. This philosophy is merely an implementation of queuing theory applied to fields like lean manufacturing and product design. What happens is that, classically, all work is thought of in terms of %age utilization of resources (in our case people). However, what we should be looking and analyzing instead is product design queues.

In any queue based system (like manufacturing for example), the philosophy of identifying has long since shifted from increasing utilization to reducing spare inventory. In software, spare inventory is usually work-in-progress. Practically, what happens is that when you have a long laundry list of items pending for deployment and which go together, it hits all the symptoms of getting-rid-of-spare-inventory.

What I propose is that we move to a system of much more frequent deployments. Before the product management team gets all happy (!), let me mention that it comes with its tradeoffs – reduced resource utilization is the most frequent. For example, if one looks at some of our sprints, we can see that we had very broad-impacting releases  (typically accompanied by a bunch of seller mailers, communiques, full QA cycles, etc.) – in short a lot of dependencies, which translates into long queues. What you will also see is that I deliberately reduced WIP inventory during that period, which is why you will see reduced resource utilization. However, it allowed us to focus and maintain quality in our releases.This means that product management’s metrics should factor “cycle time” as one of the metrics – time taken to conceive a product to putting it into production – rather than focus solely on resource utilization as the only metric. The important thing to remember is that more often than not, cycle-time will be inversely proportional to resource utilization. This is not an axiom but a mathematical fact born out of queuing theory.