Barrio Malawi - open source

Keeping Many Debian Machines Updated

jon — Mon, 11 Sep 2006 04:58:11 -0400

There are an increasing number of Ubuntu Linux machines on the Mzuzu University campus network (DHCP, squid, iptables, email, web, and some desktops). As you all know, keeping both Windows and Linux machines up-to-date with security updates can be very bandwidth intensive. Because our 256 kb/s connection is used heavily during the day we are trying to schedule all security updates to run once at night and then have all of our client machines connect to the local repository during the day for their security updates so as not to use bandwidth while students and professors are trying to connect online.

Over the past month, we have been comparing various Ubuntu local security update repository solutions: apt-proxy and apt-cacher. Our result: apt-cacher wins hands down. We found apt-proxy to be quite buggy and not to perform very well when multiple machines connected to the security update server at the same time. We have had no problems with apt-cacher. It was easy to install and configure. Most importantly it has greatly reduced the amount of bandwidth needed to keep our campus computers up-to-date with the latest Linux security updates.

Our next task is to find a way to do something similar for Windows machines. Currently each computer on campus is connecting to the windowsupdate.microsft.com website and downloading security updates individually. This is one of the heaviest uses of bandwidth on our network. I have found Windows Server Update Services . Its BETA version, but I think it might do what we need it to.

Any other ideas are most welcome here in Malawi.

Recovering Deleted files with Linux and Sleuthkit

jon — Wed, 17 May 2006 03:39:17 -0400

Ever accidentally delete a file you really needed from the recycle bin? Ever format a disk and install a new operating system only to find later that you forgot to save off some really important files first? The former recently happened to us here at the Univeristy in Mzuzu... we thought doom was upon us until we stumbled upon a open (free to change, use, and distribute) software utility called Sleuthkit (http://www.sleuthkit.org).

When we realized our disaster, we started searching Google for "data recovery", "recover my files", etc. Listings appeared for companies charging hundreds of dollars for data recovery services and software. We tried some of the trial versions of the commercial software, but nothing was able to detect our files. Finally one of us here in the office in Mzuzu stumbled upon Sleuthkit, an amazing utility that was able to recover out deleted files from a formatted disk and didn't cost us a penny!

For all those out there that do not feel comfortable removing hard disks from computers, or if all of this talk of Ubuntu and software is boring you at this stage, let me leave you with just one parting thought - if you are using a computer and discover that a file you really need has somehow been "permanently" deleted the steps you should take are:

1. Immediately stop using the computer and shut it down. When your precious file was "deleted" from the disk, it wasn't actually deleted at all. In fact all the computer did was make a note to it self that the space occupied by the file is now free to be over written. If your file has not yet been over written by new files, then recovery might possible with Sleuthkit.

2. Next, contact all the nerds you know. Tell them what has happened and tell them that you have heard of about tools like "Ubuntu", "Linux", and "Sleuthkit" that can be used to fix the problem. If the nerd's eyes light up when you mention "Ubuntu" or "Linux" you have found the right person to help.

The following are the steps we used to recover our files from the formatted disk here at the university in Mzuzu:

Remove the hard disk from the computer it was in when the files were deleted (if the disk with deleted files in on a laptop do the following: boot the laptop with a liveCD such as knoppix, mount a network drive using smbmount or nfs, use dd to write an image of the disk to the network drive, and skip to step #8 below)
Attach the disk with the "deleted" files to a computer that is running Ubuntu Linux as its operating system (I imagine your Could use a Ubuntu LiveCD here )
*** Note this computer will need to have free disk space larger than the entire size of the disk with deleted files ****
Boot up the Ubuntu computer
use System->Administration->Disks or run the command "df -h" from the command line to determine which system device number your disk being identified by. We used df -h and based on the size of our disks found that the one with the deleted files was referenced as /dev/hdb
*** Only once you are positive that you have the correct system identity of your disk should you proceed ***
Create a disk image of the disk "sudo dd if=/dev/YOUR_DISK_ID of=/home/YOUR_HOME_FOLDER/recovery_disk_image.img"
When the image of the disk with deleted files is ready, power down Ubuntu and remove the disk with the deleted files. Store it in a safe place. But do not plug it back into another computer just yet.
Turn the Ubuntu computer back on
Add the universe software repoitories to your apt sources list (see The Ubuntu Getting Started Guidi on Google for more info)
use apt or synaptic package manager to to install sleuthkit and autopsy
open a command line prompt and run "sudo autopsy"
open a web browser and go to the page http://localhost:9999/autopsy
A graphical interface appears Choose "New case" and follow the questions that the wizard asks you. When prompted for the location of the disk image enter "/home/YOUR_HOME_FOLDER/formatted_disk.img"
After sluethkit analyzes the disk image it will display a listing of the disk's partitions. Choose the partition where you deleted file was saved and click Analyze -> File Analysis
Along the left hand side of the screen you will see a search box where you can search for the deleted file by name. Enter the file name and click Search.
A list of the files matching your search text will appear. Browse the list of files and click on the hyperlink name of the file that you would like to recover
Once the file finishes loading click the Export hyperlink and wait for your web browser to prompt you to save the file
When prompted save the file
Test to make sure your “deleted” file opens correctly on another computer. Transfer the file to another computer using a CD, USB flash drive, floppy disk or the internet

Repeat steps 14 -18 to recover more files that were deleted. Using these steps we were able to recover an entire folder of files from a disk that had been formatted and reinstalled with a new version of Windows XP. Hope it works for you as well as it worked for us

VSAT Community Networks

jon — Fri, 12 May 2006 17:15:47 -0400

I just read that the internet is growing faster in Africa than anywhere else in the world. Most of us here in Africa are connecting via satellite or VSAT connections which are a fairly expensive way to transmit data (though costs are rapidly dropping). While I am finding sufficient knowledge on how to purchase VSATs, I have found very little discussion or collaboration online concerning how best to maintain VSAT internet connections.

Just yesterday I received an email from a friend here in Malawi describing the experience of another friend who “successfully” installed a VSAT connection for their NGO. The NGO found that the the installers from the company they hired to install the VSAT “haven't got a clue [how to install a VSAT]. It took them over 10 days to get it running - basically they installed it and it didn't work and they couldn't work out why.” After asking help from a local internet cafe owner the NGO found “the VSAT company had supplied the wrong kind of network cable (not a cross over one) - a fairly basic mistake and they haven't apologised or anything - so we're not really very impressed.”

Poor quality technical support seems to be a common complaint of those organizations installing VSATs. Often the price for poor technical support in VSAT service contracts is very high. Organizations are usually asked to pay travel, room and board expenses for foreign technical support consultants in addition to paying fees for fixing connection problems. In the case of the University here in Mzuzu, we have found that maintaining the connection can be challenging because the network infrastructure was designed by outside consultants. Very few of the skills needed to keep the VSAT running optimally were transferred in documentation or training to the University IT staff.

Here in Mzuzu there is some consensus that organizations using VSATs could themselves improve technical support services while reducing service contract costs by collaborating online. We are interested in creating an online community to help other organizations around the world sustain and manage their VSAT internet connections. This would include: tools/guides to help organizations estimate and budget for usage costs, a tutorial collection to help build staff capacity for maintaining the VSAT and reduce dependence on outside consultants, a collection of success stories and best practices/uses for internet (giving professors blogs, etc), an online forum for technical questions to be answered by other members of VSAT community.

The other phase of our work here involves designing open source software to control, allocate and optimize network bandwidth on the VSAT (as I discussed before). I truly believe that this is the this is the other piece of the puzzle that could really fuel of fire of internet development here in Africa. User friendly software that turns old PCs into network appliances for allocating bandwidth would empower community groups, NGOs, churches, universities, and cities and allow them to collectively purchase VSAT equipment and distrubute and manage their bandwidth according each organizations contribution towards the connection cost.

I have found that the NoCat (http://www.nocat.net) project is pretty much what we need to begin allocating bandwidth based upon user logins. It is a great community networking product, but the problem is, the project stopped development sometime in 2004. Currently assessing the state of the NoCat code to see if it suits our needs and can be built upon. We are also looking into rolling out our own system.

Here is a more technical details of the community network system we are trying to create:

Place a Linux authentication gateway to filter all network traffic just before it is sent out the VSAT to the rest of the world
The authentication gateway should optionally allow unlimited local network traffic (email, file and print sharing, web browsing)
When a user tries to send data out to the world through the VSAT, the authentication gateway captures the request and the user is asked to login
After logging in network bandwidth limits are set on the gateway on a per user basis according to which network privileges the user has been granted. These are settings controlled by the network administrator.
The user is shown how much network time they have remaining in their account, how much bandwidth has been allocated to them etc.
The user is redirected to the website the originally requested before the capture. A small javascipt frame in the web browser continually checks in with the authentication gateway so that the gateway knows the user is still using the network.
Once logged in the user can send any type of network requests that are permitted by the network administrator through the VSAT
To end the VSAT network session the user closes their web browser
The authentication gateway notices that the user has not checked in for some time and closes network access to the VSAT for the user.

NoCat does most of this already, but is lacking an accounting system that would permit easy creation of network access limits for groups of users. Now to figure out if we can add it to the system... I love nerding.

Professor Saints and the nutty open source textbook

jon — Sun, 23 Apr 2006 20:44:26 -0400

I must apologize for the lack of content on the blog recently. Things have been busy in good ways... very good ways. I have been asked to teach the first ever introductory to computer science course here at Mzuzu University Department of Mathematics! Fortunately the course is only meeting twice a week, so there is plenty of time to piece together lessons and lectures. So far, the students seem to be enjoying themselves. They are very excited about having the chance to learn to program computers. Our only problem so far is that the number of computers on campus comes no where close to matching the students enthusiasm for the subject.

Agreeing to teach at the University has also opened some doors for my research. The other professors seem much more comfortable talking to another lecturer. When I was considered to be only a researcher, it seemed like the faculty here wasn't sure where how to classify me or how they could really work with me.

For our class text book we are using a very neat book called “How to Think Like a Computer Scientist” (http://www.thinkpython.com). I read it this past summer and quickly found it to be one of the best introductions to computer science that I have ever seen. The examples are clear, and topics are presented in a way that gets students writing fairly complex programs in a short amount of time. In addition to being a very high quality text book, the book is also published under an open source license which means the authors have given rights to everyone to download, print, edit, rewrite, or reproduce the text book legally in any way they choose.

The book is has three main authors – a professor from University of Chicago, a high school teacher, and a professional programmer – and claims to have received contributions from over one hundred readers who have proof-read and improved confusing sections of the book. I know that some might ask why on earth the three authors have agreed to let their work be redistributed for free without requiring any sort of monetary compensation from the users of the textbook. You are welcome to ask the authors themselves, but I think the answer might be that by opening their book – making it available for free - the authors have, in fact, benefited greatly economically. Hundreds of people have proof read and contributed to their textbook and the authors have not had to pay any of them a cent! Could you imagine paying an army of one hundred people to edit and improve a book you had written? Also, by opening their book, the authors have built names for themselves. Without waiting for a publisher's approval, these authors have published their book and watched it quickly gain in popularity in a genre where there are hundreds of competing textbooks and authors.

I took me a while to explain the concept of an open source textbook to the faculty here at the University. In Malawi, they are used to receiving “free” textbooks. Loads of old, hand-me-down text books arrive from universities in the USA and Europe each year for “free”. These textbooks are “free” because they are old and no longer wanted. My challenge here was to explain that this open source text book is “free” for a different reason. “How to Think Like a Computer Scientist” is free because the authors have decided that the most productive way they could produce a high quality, cutting edge textbook is to give it away for free and encourage students and teachers around the world to improve the text. The textbook is “free”, not because its not longer good enough for the universities of the USA and Europe... but “free”, because the authors have successfully adopted the proven open source development model (which has brought numerous high quality software products to the market) for literature.

My students, like all university students around the world, are pretty short on cash. The high quality and low cost of this text book made it the ideal textbook for our introductory computer science course here in Malawi. We have printed 5 copies ($4.00 each in our library) for a class set for the library, and the students are currently saving electronic copies of the book to their floppy disks that they carry with them. The administration here seems to also really like the idea of the open textbook. Next year, when Mzuzu University wants to get its hands on one of the latest and greatest computer science text books the University won't be dependent upon the charity of foreign universities or publishers. The updated version of “How to Think like a Computer Scientist” will be there “free” and ready for our students to download and begin reading.

As I think more about the textbook, the way in which it was produced, the impact that it is having here already... I realized more and more... open source, as model for production, clearly stands to have a enormous impact developing world.

Yet I still believe that most of us, having grown up in a copyright society, have a completely wrong impression of what open source (copyleft) development really is all about. Open source production is not some trendy new fashioned form of communism... there are real economic advantages and efficiencies at play here. The way “How to Think Like a Computer Scientist” was able to “hire” hundreds of editors for almost no money is just one example.

My current (though humble) career is another. Most of you know that I spend quite a bit of my free time and earn the majority of my income by working on open source software projects. Yes, its true, I earn money by giving my software away for “free”. But, I want people to know that I give it away, not because I am a good samaritan... I give it away because it is the best way for me to make money. I am an efficient programmer. I am a not such an efficient marketer or a lawyer. If I tried to market my software or enforce copyrights upon the code I have written, I believe that I would be wasting my time, money, and energy. I would rather spend my resources writing more creative software. By distributing my quality code for free I encourage people to use it. The more people that use it, the more likely one of two things will happen: one of the users later be willing to pay me to improve the code I have written, or some other programmer will add functionality to the code then share the changes back with me. Whether the a new client pays me to write new code, or a programmer (that I don't pay) writes a new feature that I can then sell to my clients... either way, I gain economically.

We should be clear that producers (like me) aren't the only ones that benefit in the open source production model. Buyers (my clients) can benefit as well. Usually when I am considered for hire on a software project, I tell the prospective clients that I will only take the job if I am allowed to give away the code that I write during the project for free to the open source community. Clients are always nervous about the idea at first. But there are strong economic arguments they must consider as to why it is to their benefit to pay me to give away my code for free. Sounds crazy I know... but take a client that recently hired me for a small $1000 USD dollar development project in Washington DC as an example. For that price I was able to develop some new functionality for their website. The problem was that I would soon be moving away to Malawi and no longer be able to help them maintain the new functionality. After I completed my work, we “open sourced” the code I wrote. We contributed the code to one of the major open source software projects called Drupal (software to help people manage websites). Shortly afterward, I began receiving request from around the world from other people who were interested in using it in their client's websites. Whether these people sold the code I wrote to their clients or not, it didn't matter. We let everyone who asked have the code. To every one I sent the code, I included a note said I could not promise to maintain the code after I moved away. They would have to use/sell/study the code at their own risk (this is common in open source production). Just two months before I left for Malawi, one of the “free lunch” users contacted me with the news that they were interested in taking over development of the project. They would to continue to update the code and add even more new features to it (not so “free lunch” after all).

Just last month I noticed that a new version of the code I wrote was released for everyone to use for free. There is now a team of three programmers working to maintain the code. The code is higher quality and includes even more new features than the original code i wrote. My clients in Washington DC are now free to use this new and improved code. So for the small price of their $1000 original investment my clients also now have a team of three programmers working to help maintain this new functionality for them that they do not have to pay. They get all of this because they were willing to take the risk open sourcing production of product that, in the end, proved to be valuable to other people.

And so, now I have to admit, that I really do not like the first chapter of the book “How to Think Like a computer Scientist”. I am thinking of rewriting the first chapter of the book using material from the first lecture I gave for class here entitled “How to think like a Computer”. If I do write the text, I will give it away for free to the authors to use in the next edition of the textbook. Why do I want to write a chapter for the book for “free”? Yes, there is part of me that just wants to see a better version of the book next year... but also I know that it would benefit me and my resume greatly to be able to say that I was a contributing author to one of the best computer science text books out there. If I do write a better first chapter both the book's readers, and myself stand to benefit. This is how open source production works.