A Better UI (CLI) for git

I recently read 3 UI books: Design of Everyday Things, About Face, and Don’t Make Me Think.

The books did make me think. A lot. A lot about a tool I use everyday…
git (http://git-scm.com).

Git is sweet tool. But, the UI could use some love.

The UI can be confusing and inconsistent for newbies. In fact, even after three years of using git, there are commands I still find confusing (do I want git revert or git reset… and does it need –hard, HARD, –cached, HEAD, –, or  something else?).

An interface should never be a by-product of the implementation of the system. This is where the Git UI goes wrong.

The command git reset is a good example. I use git reset to undo things. But, its called “reset” and has a bunch of strange parameters which have to do with making writes to the git index. In order to undo changes properly, the reset command requires the user to have a clear understanding of the guts of the git index. This should not be the case for a common command, like undo.

# Why not replace things like: git reset HEAD -- file with:
> git unstage

If we make all the “resetting” and funny “HARD — HEAD” parameters happen behind the scenes, then we make the user happier. A lot happier.

Here are a few other small irks I have with the git UI:

  • I have some files I want to stage. Do I want git add ., git add -u, or git add -A? Which does what again? Confused.
  • Oh geez. I just staged something by accident. How do I unstage it? Its git reset? Or is it git revert? Do I do a HARD, –, HEAD, FILE, or something else? Brain damage.
  • I want to see my changes. git diff? git diff HEAD? or git diff –cached? why not git diff CACHED? Grrrr….
  • git status is cluttered with commands telling me how to undo things “use git reset HEAD <file> to unstage”. If the commands were easy enough to remember in the first place, we would not need to clutter the status output with reminders. And, what does “changed but not updated mean”? I did update my file! Do you mean “changed but not staged”? Lastly, I would like to humbly request a few fewer “#” characters. Just a few. Please.

Here is my shot at re-creating a few parts of the Git UI. My hope is that all of these commands become part of git one day and that git becomes easier for new users and forgetful intermediates like me.

Staging files

# Stage all changes, and file deletions. Leave unknown files alone.
# Instead of git add -u, just type:
> git stage

# Stage just one file change or deletion.
# Instead of git add FILE and git rm FILE, just type:
> git stage FILE

Unstaging Files

Ooops. I staged a changed that I do not want to commit.

# Remove one file from the stage.
# Instead of git reset HEAD -- file, just type:
> git unstage FILE

# unstage all the files you have in the stage right now.
# Instead of git reset HEAD, just type:
> git unstage

Undoing changes I have made:

Ok. I tried something. It does not work. Just get me back to the last commit.

# Undo all the changes I made to working directory.
# Automatically save the user's work in a stash, then
# Instead of git stash; git reset --hard HEAD, just type:
> git retreat
All files changed back to their state at the last commit.
Your work has been saved in stash@{1}. 

# Just undo the changes I made to one file.
# Automatically save a stash of the user's work first
# Rather than git checkout FILE (side note: why is it not git reset FILE? or is it?)
# Instead, just type:
> git retreat FILE
The file was changed back to the it's state at last commit.
Your work has been saved in stash@{1}.

Deleting Files

Let’s get rid of git rm. Lets not even tell newbies about it. Just delete files you want from your project, then:

# smart enough to stage files you have removed as well!!!!
> git stage
> git stage FILE

Status

More readable. Less # signs. Less instructions.

> git status

Staged for commit:
    M daemon.c
    N test.txt

Unstaged Changes (will NOT be committed):
    M hello.c
    ? new.c

Nice and clean. No reminders about how to undo things… you already know that: git undo. And, no # signs!!

Automatic Setup

The first time I run git from a new computer I am most often greeted with an error. Not a very friendly way to great new users. The message says that I have not set my email and user name.

If this error occurs, shouldn’t git just ask me for this information and proceed with the command I typed? Yep, it should.

# my first ever commit with GIT! Don't greet me with an error.
> git commit -m "yippeee git!"

Howdy! So that git can give you credit for changes you make
    Please enter your name? Jon Saints
    Please enter your email? email@gmail.com
Thank you! Committing...

This replaces the more confusing error message that git shows users now:

Error 123: Look! Another stupid user is _trying_ to learn git. 

You should have known to type this first:
> git config --global user.name "Jon Saints"
> git config --global user.email "gmail@gmail.com"

DUFUS!! Just give up now. Seriously.

Switching branches

# I switch branches with the checkout command. Its ok, but what if we had:
> git switch BRANCH

Showing things I have changed

This is one of the most important parts of git. And, it is confusing.

# What is the output? Can you remember which is which?
# git diff
# git diff HEAD
# git diff --cached or git diff CACHED?

# Instead we can... 

# Show all changes from the last commit
> git diff
# or
> git diff HEAD

# Show difference between the stage and my code
> git diff STAGE

# Difference between my stage and last commit
> git diff STAGE HEAD

# Difference between two branches or revisions
> git diff branch1 branch2
> git diff AD34E BCDE543

Deleting a branch

# Let's be consistent with git remote command. Instead of git branch -d
> git branch rm BRANCH

Where to from here?

I am going to make a small python wrapper to try out these commands. It will be called gum. And, it will just write git commands for you.

gum = Git User (interface) Makeover.

“gum unstage” here we come! The code is on github: https://github.com/saintsjd/gum

Comments welcome.

Automated Deployment of PHP Applications using git

Heroku is sweet hosting platform which offers a pure git deployment workflow. I love using it to deploy my Rails projects. I miss using when I deploy a PHP application.

Lets fix that…. Here is a quick heroku-ish git deployment workflow for your PHP applications. We will be using WordPress for this example. The same steps can be adapted for Drupal or any other PHP application.

Our goal is to development and test new features, themes, and plugins for WordPress locally on our development machine. When changes are working properly, we want to deploy them to our live site with one command:

git push origin master

#1: Set up password-less login to your webserver with SSH keys

To run deployments we will need to access our live webserver without a password. Follow the steps  in Setting up ssh public keys to login to webserver without a password.

For each user or machine from which you want to allow deploys you will need to install its SSH key using the steps above.

#2: Setup Code Repository on the Webserver

I connect to the webserver via SSH. My home directory is /home/saintsjd.

I have a /home/saintsjd/www folder for my website and PHP files. I also have a /home/saintsjd/code folder for my git repositories.

cd into /home/saintsjd/code and create a folder called blog.git

cd /home/saintsjd/code
mkdir blog.git

Next, initialize a bare git repository

cd blog.git
git init --bare blog.git

This gives me a git repository to store my code at /home/saintsjd/code/blog.git. I then disconnect from the SSH connection to the webserver.

#3: Setup Development Box

I have php, mysql, and apache running locally on my laptop. I create my WordPress themes, plugins, and updates on my laptop. After I test them, I deploy them to the live webserver for the world to see.

My local website folder is /home/jon/Website. To get started I clone my git repo locally:

git clone ssh://saintsjd@saintsjd.com/home/saintsjd/code/blog.git ./blogdev

This give me:

/home/jon/Website/blogdev

Now i can start filling in code!

Into the /home/jon/Website/blogdev/ folder I copy in all the files for WordPress and run the installer.

Next, I need to add a gitignore file to the project. I use the one here. https://github.com/github/gitignore/blob/master/Wordpress.gitignore. I copy the contents of that file into a .gitignore file in /home/jon/Website/blogdev/.gitignore. I then:

git add .
git commit -m "inital commit"

To send my changes to the server I type:

git push origin master

#4: Setup Live Webserver

Now I install WordPress on my server. I SSH to my server. There my website directory is /home/saintsjd/www. I need to remove this folder so I backup everything in this directory.  Then I remove it and recreate it using git clone. I type:

rm -rf /home/saintsjd/www
git clone /home/saintsjd/code/blog.git /home/saintsjd/www

I then browse to my website http://www.saintsjd.com and run the wordpress installer so the database gets installed on the server.

IMPORTANT: Protect your .git repository from prying eyes

I don’t really want others reading my source code from the git repository via http://www.saintsjd.com/.git/

I prevent this by running:

chmod -R og-rx /home/saintsjd/www/.git

I also add a .htaccess file to the .git folder containing:

Deny from all

#5: Automated Deployment Script

Now that wordpress is installed on both my local machine and my live webserver, I want to be able to deploy any changes I make on my local copy with one git push command. I can do this using a Git hook.

I SSH to my webserver and create a file /home/saintsjd/code/blog.git/hooks/post-receive

touch /home/saintsjd/code/blog.git/hooks/post-receive

I make the file executable

chmod u+x /home/saintsjd/code/blog.git/hooks/post-receive

Then I edit the file

nano -w /home/saintsjd/code/blog.git/hooks/post-receive

and place this code in it (adjusting the config variables to match your server settings)

#!/bin/bash
#CONFIG
LIVE="/home/saintsjd/www"

read oldrev newrev refname
if [ $refname = "refs/heads/master" ]; then  
  echo "===== DEPLOYING TO LIVE SITE ====="  
  unset GIT_DIR  
  cd $LIVE  
  git pull origin master  
  echo "===== DONE ====="
fi

#6 Let the (heroku-like) Fun Begin: Automated Deployment for WordPress

Now that everything is setup. I am free to develop new themes, install new plugins and update on my local development box. Once everything is working properly locally commit and push. The post-receive hook automatically deploys the code changes to my live webserver for me.

Here are my steps:

  1. Make changes to my local site’s code: new theme, new plug-in, etc
  2. test, test and test so more!
  3. When I am happy I commit to the local git repository:
    git add .
    git commit -m "new theme css for header"
  4. I push and deploy to the server using one command:
    git push origin master
  5. After the post-receive hook moves the new code to the live website. I open http://www.saintsjd.com/wp-admin on the live server to trigger and database updates that need to run

That’s it… heroku-ish git deployment for WordPress websites.

Continuous deployment for WordPress using git and fabric

——

UPDATE: This post is out of date. Please see the new and improved version of this page Automated Deployment of WordPress using git

——-

Continuous Deployment is a strategy you’ll want to implement for any successful web project. According to Eric Reis, “It’s a process whereby all code that is written for an application is immediately deployed into production. The result is a dramatic lowering of cycle time and freeing up of individual initiative. It has enabled companies I’ve worked with to deploy new code to production as often as fifty times every day.”

This article describes one method for configuring a continuous deployment infrastructure using git, fabric, and SSH. Its fairly lightweight and should work with basic hosting systems.

Our goal here is to create new features locally on a development machine, commit happy changes to a code repository, and finally deploy the changes to a live web server with ease using server side scripts. It makes deploying new changes to a live wordpress website so easy you’ll want to do it 50 times a day!

Install Deployment Tools on your Dev Machine:

I use fabric for all of my non-rails applications (PHP, Python, etc). Fabric allows you to execute command on remote servers by running a local script on your machine.

Its available at http://docs.fabfile.org/

To install on my local Ubuntu Linux development machine I just:

sudo apt-get install fabric

Set up SSH keys

To run deployments we will need to access our remote SSH server without a password. Follow the steps  in Setting up ssh public keys to login to SSH servers without a password.

For each user or machine from which you want to allow deploys you will need to install its SSH key using the steps above.

Use git to Setup Code Repositories

I use two separate git repositories: one for my wordpress website and another to track changes to my fabric deployment scripts. I store my repositories on my web server in the folder

/home/jon/code/saintsjd.com
/home/jon/code/saintsjd.com/www.git (is my wordpress website git repo)
/home/jon/code/saintsjd.com/deploy.git (is my repo for deployment scripts)

So I cd into /home/jon/code/saintsjd then:

git init --bare www.git
git init --bare deploy.git

Setup a Local Development Space

I have php, mysql, and apache running locally on my development machine. My local website folder is /home/jon/Website. There I create a folder /home/jon/Website/saintsjd.com. I then clone my git repos locally:

git clone ssh://jon@saintsjd.com/home/jon/code/saintsjd.com/deploy.git ./deploy
git clone ssh://jon@saintsjd.com/home/jon/code/saintsjd.com/www.git ./www

This give me:

/home/jon/Website/saintsjd.com/www
/home/jon/Website/saintsjd.com/deploy

Now i can start filling in code!

Install WordPress in Dev Space

Into the /home/jon/Website/saintsjd.com/www/ folder I copy in all the files for wordpress and run the wordpress installer. After WP is installed properly on my local machine I initialize a git repository to start tracking my code.

First, I need to add a gitignore file to the project. I use the one here. https://github.com/github/gitignore/blob/master/Wordpress.gitignore. I copy the contents of that file into a .gitignore file in /home/jon/Website/saintsjd.com/www/.gitignore. I then:

git add .
git commit -m "inital commit"

To send my changes to the server I type:

git push origin master

Install WordPress on the server

Now I install wordpress on my server. I ssh to my server. There my website directory is /home/jon/public_html. I backup everything in this directory then I remove this directory and recreate it using git clone. I type:

rm -rf public_html
git clone /home/jon/code/saintsjd.com/www.git public_html

I then run the wordpress installer.

Writing the deploy scripts

With wordpress installed locally and on my server. I am ready to write a fabric deploy script to easily synchronize changes I make locally to my live version of wordpress on the web server.

On my development machine I:

cd /home/jon/Website/saintsjd.com/deploy

There I create fabfile.py and fill the file with the code below:

from fabric.api import *
env.hosts = ['jon@saintsjd.com']
WEBSITE_PATH = "/home/jon/public_html"

def deploy():
    with cd(WEBSITE_PATH):
        run('git pull')

You will need to change the env.hosts value to use your ssh login for your server. You will also need to change WEBSITE_PATH to reflect the path to your website on the server.

Now when ever we want to deploy changes to the live website we can just

  1. commit our changes with git and push
  2. go to the command line and
    fab deploy

If you want to (I do), you can get fancy and make a backup before each deployment. This is great in case something goes wrong. My fabfile.py with backups looks like:

from fabric.api import *

env.hosts = ['jon@saintsjd.com']
LIVE = "/home/jon/www/saintsjd.com"
BACKUPS = "/home/jon/backups/saintsjd.com"
MYSQL_USER = ""
MYSQL_PASSWORD = ""
MYSQL_DATABASE = ""

def deploy():
    backup()
    with cd(LIVE):
        run('git pull origin master')

def backup():
    run('mysqldump -u %s -p %s --add-drop-table --password=%s  > %s/saintsjd.com-database-current.mysql' % (MYSQL_USER, MYSQL_DATABASE, MYSQL_PASSWORD, BACKUPS) )
    run('/usr/bin/nice tar -czf %s/saintsjd-`date +%%Y%%m%%d%%H%%M%%S`.tar.gz %s/saintsjd.com-database-current.mysql %s' % (BACKUPS,BACKUPS,LIVE) )

With fabfile.py written and working I add it to my git repo and push to the server for safe keeping:

git commit -am “initial commit”
git push origin master

Protect your .git folder on the live website using .htaccess

I don’t really want others reading my git repository via http://www.saintsjd.com/.git/ I prevent this by running:

chmod -R og-rx /home/public_html/.git

I also add a .htaccess file to the .git folder containing:

Deny from all

Continuous deployment of WordPress using git and fabric

My continuous deployment for the website is all set. Now I

  1. make changes to my local site’s code: new theme, new plug-in, etc
  2. test, test test
  3. when I am happy I commit to the local git repository
  4. I push to the server
  5. I then change directory into my ../deploy folder and run
    fab deploy

    on the command line to send the change to the live website

  6. finally I open wp-admin on the live server to trigger and database updates that need to run

What is a bare git repository?

What is the difference between a repository created using the “git init” command and the “git init –bare” command?

Repositories created with “git init” command contain 2 things: the .git folder repository (or code history) and also real life working copies of your source code files. You can think of this type of repository as a working directory. Its a folder with code history stored in .git folder and all the source files as well. You can work in this directory changing source files and save your changes using the “git add” and “git commit” commands.

Bare repositories contain only the .git folder and no working copies of your source files. If you “cd” into a bare repository you find only the .git folder and nothing else. A bare repository strictly contains the version history of your code.

Why use one or the other?

Well, a working repository is for just that… working. Its where you will actually edit files, add files, delete files to create your project saving your changes to the local .git repo along the way. If you are starting a project in a folder which will contain your code as well as a git repo for tracking changes use “git init”. Also if you “git clone” a repository from some one else you will be given a working repository with the .git folder and copies of the working files.

A bare repository is for… sharing. Bare repositories are shared repos where developers can send their local changes from their working copies of projects to the world or to the rest of their team. By using the “git push” command, you are able to send changes in your local working copies of a project to a centralized shared bare repository. Other developers can then “git pull” to receive the changes that you made. If you are collaborating with a team of developers or you need to work on a project from multiple computers , then a bare git repository is what you need in between to coordinate the distributed development.

To summarize: the working directory created with “git init” or “git clone” is my local copy of a project. Its where I add my changes to a project’s code files and test. After I am happy I “git add” and “git commit”. Then I “git push” to a bare repository, usually on another server, so that other developers can access my changes. When I want to update my local working copy I “git pull” to receive the changes that other developers have made.

Deploying a Drupal 7 Website Using git

——

UPDATE: This post is out of date. Please see the new and improved version of this page Automated Deployment of WordPress using git

——-

Distributed revision control systems are an excellent way to manage the source code of web applications. Currently I use git. In the past, I have used bzr and hg. All three great tools.

Git is now my favorite for a number of reasons.

While the tools are great at managing code and facilitating distributed development, it did take a while for me to find an efficient way to deploy my version controlled code to a live web server. The following is an outline of a simple workflow for developing and deploying web applications using distributed revision control systems. You could apply these methods to either bzr, hg, or git.

I keep development copies of the websites I manage for clients on my local machine. I develop and create new features locally on branched copies of the source code. When I complete and test a feature I commit the changes to the central code repository and deploy it to the live server using a few different methods.

Creating your code repository

A code repository is a place to save your code. Its important to not that the repository itself is not a functioning website. It is a file often on a completely different server than the live website and the development boxes. There are two options for creating a code repository. On is github.com. I use GitHub to store my open source projects. For projects with closed source code I store my repositories on server which allows ssh connections in a folder that is not accessible via the web. You can pay for a github account which allows you to create private repositories.

Creating an repository on Github.com
Follow the step here: http://help.github.com/

Creating a repository on your own SSH server
I will create my repository in a non-webaccessible directory on my live server. so… on my live server I will have:

/home/jon/public_html (my live website folder)
/home/jon/code (a private folder not accessible from the web to store my source code)
  1. Log into the server via SSH
  2. my home directory is /home/jon
  3. I create a directory /home/jon/code for all my repositories /home/jon/code
  4. cd into the code directory and type:
git init --bare drupaltest.git

Setting Up Local Development Website
Change directories on your local development computer into a web accessible directory. On my machine the local folder /home/jon/Website is accessible at http://localhost/jon/. So I open the terminal and run:

cd /home/jon/Website

Then:

git clone jon@saintsjd.com:code/drupaltest.git drupaltest

This command clones the repository to my local machine and creates a localfolder called drupaltest. I can now start adding files to it. Lets unzip a fresh copy of Drupal 7 and copy all the files into the drupaltest folder. I then browse to http://localhost/jon/drupaltest/ and I run the Drupal installer.

With drupal installed we are ready to start committing our code to our git repo.

We first need to create a .gitignore file in the root of our web application /home/jon/Website/drupaltest/.gitignore. This file tells git to ignore certain files. We want to keep our settings.php file out of the repository and our sites/default/files folder out of the repository because the settings and files on our development computer will be different from the settings.php and files on the live server. My .gitignore contains:

sites/default/files
sites/default/private
sites/default/settings.php

With /home/jon/Website/drupaltest/.gitignore created, I run:

git add .
git commit -m "initial commit"

I then send my code up to the repository on my server. Remember the repository is just a place to keep my code. It is not a live functioning website. We will create that live site later. To push the code to my respository I run

git push origin master

And I know the command was successful when I see something like:

Counting objects: 1099, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (1087/1087), done.
Writing objects: 100% (1099/1099), 2.81 MiB | 324 KiB/s, done.
Total 1099 (delta 119), reused 0 (delta 0)
To jon@saintsjd.com:code/drupaltest.git
* [new branch]      master -> master

Creating a Live Website from our Repository

The command in this section only need to be run once when you first setup your website. Like the install we ran locally we now need to run an Drupal install on our server to setup a separate database for live data. I ssh to my server and change directories into my public_html folder.

My code repo is stored on the same server as my live website but is in a folder that is not accessible from the web. Remember that i have:

/home/jon/public_html (my live website folder)
/home/jon/code (a private folder not accessible from the web to store my source code)

I SSH to the server and run:

cd public_html
git clone /home/jon/code/drupaltest.git drupaltest

If your repository is on github or on another server you will need to adjust the “/home/jon/code/drupaltest.git” path in the command above.

Then I:

cd drupaltest
ls

I see all my files listed. Now I browse to my website http://www.saintsjd.com/drupaltest/ and run the drupal installer.

With a working site at http://www.saintsjd.com/drupaltest/ we are ready to begin making our changes to the website and deploying the changes to the live site using git. The steps that follow are the steps you will repeat for each new feature that you want to add to your website.

Deploying using git

There are many automated ways of deploying code using git. I recommend Fabric and Heroku for Rails apps. I will show only one basic manual deployment method here.

Our workflow will be:

  1. Develop and test a new features for our website on our local development box
  2. Commit and push the new code to the repository
  3. Pull the tested code from the repository to the live website

We repeat this workflow each time we add a new feature to our website.

Lets say I want to add the pathauto module to my drupaltest website. I download the module from http://drupal.org/project/pathauto and the token module http://drupal.org/project/token, unzip, and copy the modules to my sites/all/modules folder on my local development machine.

I browse to http://localhost/jon/drupaltest/#overlay=%3Fq%3Dadmin%252Fmodules and enable the module. I configure it and test that it actually works.

Once I am happy that the module works well. I use git to commit the new code to my repository.

git add . (stages our new code)
git commit -m “adding pathauto module” (commit our new code to the local repo)
git push origin master (send the code to our origin repo on the server via ssh)

I can then SSH to my live server and cd into the public_html/drupaltest/ folder. Synchronizing my new code changes is as easy as:

git pull

After I pull in the new code I need to enable and configure the module on the live site going to:

http://www.saintsjd.com/drupaltest/#overlay=admin/modules

Repeat the steps in the “Deploying with git” section each time you want to add a new feature to your website.

Important! Protect your .git folder on the live website using .htaccess

I don’t really want others reading my git repository on my live site by browsing to http://saintsjd.com/drupaltest/.git. I prevent this by running:

chmod -R og-rx /home/public_html/drupaltest/.git

I also add a .htaccess file to the .git folder containing:

Deny from all