Debugging in *nix

Debugging as it is, is a very tedious and painful job. And it becomes even more intimidating when it needs to be done on Linux/Unix. Read on for a good introduction to techniques for debugging on *nix.


This article by Guido Socher focuses on Linux and is titled Why does this not work!? How to find and fix faults in Linux applications.
This articles touches on properly setting up and using logging, using strace to trace the execution path of the program and using gdb & core files.


This article by Frank Schoep focuses on Linux and is titled GDB GNU Debugger Intro.
This article gives a very good introduction on using gdb and understanding core files.

Brother MFC 240c on Ubuntu 9.04

My Dell Vostro 1000 now runs Ubuntu 9.04 (Jaunty Jackalope). When the time came for me to print & scan a few things using my multi-function device Brother MFC 240c - I realized that I had missed to set it up. Now Google came to my rescue and here's how I did it. It required the follwing three steps:

1. Setup the Printer,
2. Setup the Scanner and
3. Setup the MFC Front Panel buttons Tool - this is optional

Printer Setup
Open Synaptic Package Manager and search for & install the packages along with any dependencies that it may show:

1. brother-cups-wrapper-bh7
2. brother-lpr-drivers-bh7.

This should setup your printer!


Scanner Setup
Go to Brother site and follow the instructions based on the way you connect your printer to your computer. Mostly it should be through a USB connection to your laptop/desktop and for that, use this link.

Here is a list of steps to Install the Scanner:

1. Download the appropriate scanner driver from this link. Note that you need to download the appropriate (32 or 64 bit) brcan2 deb package
2. Connect MFC to computer and switch it on
3. Open a terminal and cd to the folder where the drivers have been downloaded
4. Install the driver using the command:
   sudo dpkg -i --force-all brscan2-0.2.4-4.i386.deb
   
5. Grant permissions to the scanner using the command:
   sudo vi /lib/udev/rules.d/50-udev-default.rules
   
6. Search for the line containing libusb device nodes
   
7. Change MODE="0664" to MODE="0666"
8. Restart your computer and the scanner should work now.
9. Use XSane or any of your favorite scanning app to scan something & verify that everything is good

MFC front Panel buttons Tool Setup
If you want to use the buttons on the Scanner panel to scan, then you need to install the scan-key-tool as follows:

1. Download the scanner driver from this link. Note that you need to download the appropriate (32 or 64 bit) scan-key-tool deb package
2. Connect MFC to computer and switch it on
3. Open a terminal and cd to the folder where the drivers have been downloaded
4. Install the driver using the command:
   sudo dpkg -i --force-all brscan-skey-0.2.1-3.i386.deb
   
5. Run the brscan-skey tool using the command:
   brscan-skey
   
6. Confirm that it is running, using the command: brscan-skey -l
   
7. Press the scan button, select user, select destination, press START on the MFC
8. You should see the output on the console similar to the following:
   
   # scan from SCANNER(brother2:net1;dev0) to /root/brscan/brscan.sP7525
   scanimage: rounded value of br-x from 215.9 to 215.88
   scanimage: rounded value of br-y from 355.6 to 355.567
   
   

Note:

1. Ensure that you enter the correct file names in the dpkg commands. I have given the ones that I used and currently see on the Brother website
2. I have not installed the brscan-skey as I don't (yet) intend to use the MFC panel to scan!
   

Check out the specs & reviews of Brother MFC-240C on Amazon:

Ubuntu @ Home

I am finally running Jaunty Jackalope Ubuntu 9.04 at Home on my Dell Vostro 1000 laptop and am really very happy to be rid of the terrible virus / malware ;) called Windows !


Even though I was the founder member of the Calcutta LUG ILUG-Cal, I had been very reluctant to move fully to Linux. My dear friend, ILUG-Cal co-founder and current Coordinator Indra prodded me relentlessly to move towards using more linux, but to no avail.

WinXp started to drag on my laptop even though it was provided with 2 gigs of RAM and an AMD Athlon x2 64 processor. Apart from this, viruses kept coming (and going courtesy of AVG / CA). So I decided to format the HDD and do a fresh install of Windows in the hope that my machine would become fast & responsive. But God had something else planned for me as I was unable to install WinXp even after multiple attempts on sleep deprived nights. This is not the first time I was applying this strategy to speed up my machine so I was baffled on my inability to get it up & running with WinXp.

It was during this frustration that I looked towards Ubuntu as my saviour and have not looked back since! I asked my friend to download the latest installable desktop version of Ubuntu and give it to me on a flash drive. Made the USB a boot option and booted through the Ubuntu on the flash drive and installation was a breeze.

Everything worked out of the box and I did not need to struggle with anything at all - at the least not yet! Synaptics Package Manager is very good and I was able to install all the utilities I required through this. The Update Manager does a good job of keeping my installation up-to-date.

Recently I needed to use Remote Desktop to log-on to a Windows server and the included Terminal Server Client did a decent job of solving that. Included and pre-installed OpenOffice is also very good and can open up all Windows Office files. I was easily able to download, install & use Firefox, Picasa, FileZilla & VLC. Found an app called Barry for backing up the data from my BlackBerry. Was also able to get my Brother MFC to work perfectly - more on that in another post later.

Some of the things that I did not like are the unavailability of my favorite text editor EditPlus on Ubuntu. Ubuntu version of Picasa has also some quirks and is not as good as expected. A good application manager for BlackBerry is something that is needed urgently. Also, OpenOffice was not able to play some of the macros / animated GIFs embedded in one of the Word Docs that I had recevied. But hey, only very few small complaints compared to the huge list of positives - thats quite impressive for software that is free & open source!

I think everyone should check it out and make it their OS of choice!

Breakfast for a buck in NYC!

Oldbridge Gourmet Deli on 373 Lexington Ave at E 41st St (very close to the Grand Central Terminal & the Grand Hyatt of NY) has a great dollar deal for breakfast.
Update: This deal is dead as the price is now $1.75 - still not bad ...

In the morning till about 10.30 am, they have a buttered bagel and a small coffee for only a Dollar !
The good part is that you can choose from a variety of bagels and flavored coffees instead of the regular ones for the same price. I saw exotic coffee flavors like Banana Cream, Chocolate Raspberry, French Vanilla, etc.
And just because the prices are so low, it doesn't mean that they skimp on the quality. The bagels are of very good quality and had a proper amount of butter. The coffees are very good too and the best part is that you make your own coffee according to your taste. So you can as well mix multiple flavors and add the amount & type of sugar & milk to suit your specific taste.
You can take out if in a hurry, but if you have time, then you eat it in the spacious dining area on the second floor - so no skimping there either!
Two thumbs up to this place for providing such a good quality breakfast for so cheap.
Update: The deli was closed for some time and now has reopened under a new management who have stopped the great deal.
This deal is now dead as the price is now $1.75 - which is still not bad. They now also have an option for a large coffee instead of a small one for $2.25. However, the options for flavored coffees are lesser compared to before, but still present!

Convert Fixed width file to Delimited in Unix

Needed to convert a Convert Fixed width file to Delimited recently. There are many solutions, but the simplest & best solutions involved using the tools available in Unix / Linux. Here are a few possible ways to do it on Unix / Linux.



Input File: fixed.txt
The column widths are 2, 4, 6 & 3

AABBBBCCCCCCDDD
EEFFFFGGGGGGHHH
IIJJJJKKKKKKLLL
MMNNNNOOOOOOPPP
QQRRRRSSSSSSTTT
UUVVVVWWWWWWXXX
YYZZZZAAAAAABBB

Output File: delim.csv

AA,BBBB,CCCCCC,DDD
EE,FFFF,GGGGGG,HHH
II,JJJJ,KKKKKK,LLL
MM,NNNN,OOOOOO,PPP
QQ,RRRR,SSSSSS,TTT
UU,VVVV,WWWWWW,XXX
YY,ZZZZ,AAAAAA,BBB

Using GAWK on Linux

gawk < fixed.txt 'BEGIN{ FIELDWIDTHS = "2 4 6 3"; OFS=",";}{print $1,$2,$3,$4}' > delim.csv

This solution defines field widths & field separator and then uses gawk to split each line. The only down side is that GAWK is available only on Linux or GNU based systems.


Using SED on *nix

sed -e 's/./&,/2' -e 's/./&,/7' -e 's/./&,/14' fixed.txt > delim.csv

This solution uses sed - a stream editor to substitute commas inline.
The -e is to tell sed to execute the next command line argument as a sed program and you can have multiple such commands.
The s/./ looks to substitute any character.
The &, denotes the matched string followed by a comma that we need.
The /2 tells sed to do this substitution for only the first 2 such occurrences.
So in effect, a comma is put in after the first 2 characters are matched. Note that since this is substituting inline, the insertion point for the next comma is at position 2 + 1 + 4 = 7!

Here is a link to a very good list of handy sed one-liners on linuxhowtos.org.


Using AWK on Unix

1. Create a file conversion_rules

{
 a[1] = 2;
 a[2] = 4;
 a[3] = 6;
 a[4] = 3;
 }
 {
        o = "";
        r = 1;
        for (i=1;i<length(a);i++)
        { o = o  substr($0,r,a[i]) ","; r = r + a[i]; }
        o = o  substr($0,r,a[length(a)])
        print o;
 }

2. Run the command:

awk -f conversion_rules fixed.txt > delim.csv

This solution uses plain awk to loop through each line and add comma after the defined column length. An awk program file is created to define the column lengths and the looping & comma insertion logic. This is fed to an awk instance along with the input file and the output is redirected to the required file.

NY Hop-On Hop-Off Tour 15% Discount at GrayLine

A great way to see New York is by taking the famous Hop-On Hop-Off Tour. And if you can get a good discount on the ticket prices, that is like icing on the cake!


New York Gray Line Hop-On Hop-Off Tours has an automatic 15% discount off most (most probably ALL) of their tours - I did not see this advertised anywhere. I checked all their Best Value Tours and all the tours listed on their home page and the discount is automatically applied to these tours.

This is a very good value for money and it is 15% off of their already reduced prices if you buy online.

Personally I have taken their tours twice in as many years and have thoroughly enjoyed it always. I prefer to take their Best Value All Loops Tour (48 hours) which includes 4 separate double decker tour loops that allow you to enjoy the best of Manhattan and Brooklyn with 48 HOURS of unlimited hop-on, hop-off opportunities at over 50 stops! The best part is that the tickets are valid for 48 hours by the clock.

Thanks to my friend Dipankar for pointing out this great deal!

Walkthrough of Market Caps during Crisis

NYTimes came up with a very interesting & unique way to show how the Finance giants fared in terms of Market Cap during the financial crisis. The auto-running chart shows the market caps of the 29 biggest financial firms starting from October 2007 when the market was at it's peak, and moving up to Sept 11, 2009.

It points out a very interesting fact:
JPMorgan Chase and Wells Fargo are the only two firms to be slightly bigger than what they were during the market's peak in October 2007!

Check out this unique chart on the NYTimes website here.

New Risk Models for Wall Street

The race is on to make newer and better Quant models that factor in the human behavior and psychology.


The existing risk models factored in the default risk of counterparties and the financial instruments. They surely did not factor in the market hysteria / panic and the human behavioral aspect into the models. Generally at the time of such panic, the markets as people may be madly rushing in to sell off the stock or even just freeze and hold back on all transactions. This leads to the newly coined type of risk - "Liquidity Risk".

Financial Markets are being looked at as being similar to Online Communities or Social Networks in terms of a group or collective behavior. So the Risk models are being opened up to allow for more variables and factors affecting it. Researchers at Cornell, MIT, etc., are working on newer risk models using these ideas.

Read the full article on the NYTimes website here.

Google FastFlip launched in labs

Google Labs has launched FastFlip - it's a very good concept.


In this web application, Google combines the good qualities from both Print & Net and provides the user with the ability to quickly "flip" through
online pages just like one would do with a newspaper or a magazine.

What Google does is that it takes images of the websites and shows it to the users in a magazine-like format where the users can scan through the articles and click to go to the actual website if they are interested in reading the full article. For more details on the exact methodology that Google uses to show the articles can be seen here.

The "Right" way to Code

In the days before Google, I had formulated the CTP theory which I proposed as the "Right" or better way to Code for Junior Developers. With the advent of Google, this theory transformed to the GCTP theory.


CTP:
Before Google came up, junior programmers often used to copy & reuse existing code which was written either by them or someone else they knew or they had somehow got hold of the relevant code.
This caused a lot of problem when I was managing a team of such developers, so I formulated the CTP Theory.

The first thing I did was to encourage them to use their imagination & creativity to solve the problem themselves. If required, they could look-up the code base that most of the consulting companies already have to gain insight into ways the problem could be solved, and then come up with their own solution.
However, if there was a time crunch and/or it was felt that reinventing the wheel provided no benefit, then CTP should be applied.

C : Copy - find & copy the portion of code / routine you want
T : Transform - transform/change the code to suit your needs - this step was missed most often
P : Paste - paste the changed code in the right place

If CTP was used diligently, it provided greater success to the project


GCTP:
However, with the arrival of Google, everyone started to just google the solution and so I had to come up with the next version of the theory which I named GCTP.

Again, the first thing I did was to encourage them to use their imagination & creativity to solve the problem themselves.
However, if there was a time crunch and/or it was felt that reinventing the wheel was of no benefit, then GCTP should be applied.

G : Google the problem - google the problem
C : Copy - find & copy the portion of code / routine you want from the google results
T : Transform - transform/change the code to suit your needs - this step was missed most often
P : Paste - paste the changed code in the right place

If GCTP is used diligently, it will provide greater success to the project and quicker coding maturity to the junior developers with the least amount of hand-holding/supervision.

Note that the emphasis on both these methods is first to try to use your own brain. But in case these need to be used, then the emphasis in on T - Transform as that is where I found the junior developers making most of the mistakes most probably because either they did not understand the real problem or the way the solution had been coded.