Tuesday, 14 October 2008

Git: on rebasing

(This is a follow-up to How remotes work).

We've seen how git manages to merge your local changes when you pull from a remote repository.

This approach has a small downside, aesthetically: that is, the creation of a large number of merge commits, making the history more difficult to read. Wouldn't it be nice if git offered you the possibility to simply re-apply your local changes on top of what you just pulled?

Rejoice, because that's what the git rebase command is for.

git rebase origin/master will take the local commits (that are reachable from the master head, but not from origin/master), remove them from the commit tree, and re-apply them on top of origin/master, before moving the master head to the top of the new line of commits it just created. That way, the history is kept linear:

You can afterwards just push your new commits.

Important warning: git rebase changes your commits. Because their place in the tree will be different, their SHA1 will be different as well; and the old ones will disappear. For that reason, you must not manipulate commits with rebase if you have already published them in a shared repository from which someone else might have fetched.

Rebasing is a powerful tool that will enable you to manipulate your branches, moving lines of commits from one location to another. The git-rebase man page has more examples.

Friday, 10 October 2008

Git: how remotes work

One of the difficult things for a git beginner to understand is how remote branches work.

Basically, as git is a distributed version control system, every developer has a full and independent repository. So, how can you pass changes around?

In the examples below, we'll consider a remote repository, that we'll call origin, and a local one (that we'll call local). The remote repository has one branch, called master, that has been cloned as origin/master on the local repository. Moreover, the local repository has one local branch, called master as well (but it doesn't need to be), which is set up to track changes that happen on origin/master. Note that, as origin/master is a remote branch, it cannot be checked out -- only master can.

The fetch operation (command git fetch) copies the latest commits from the master on origin to origin/master, and updates the HEAD of the origin/master branch:

git fetch origin
The circles on the schema (click for a larger version) represent commits, and the arrows are the parent->child relationship between commits. The labels indicate the various HEADs (or branches). It is to be noted that a branch is nothing more than a label following the HEAD of a series of commits.

At the end of this operation, origin/master matches the master branch on the origin, but master on the local repository is still behind. We need to use the git merge command to make master point at the same commit than origin/master:
git merge origin/master
This kind of merge is called a fast-forward because no actual merging of changes is involved. No new commit is created; we have just moved a HEAD forward in history. And that's fast.

Now, what happens if you committed a change on your master on local? Nothing changes for the fetch; the two new commits are still created from origin's master so origin/master matches it exactly:
git fetch origin
However, on local, master and origin/master have bifurcated. To reunite them, you'll need to use git merge, that will create another commit, and make master point to it:
git merge origin/master
The new commit (in orange) is a merge commit: it has two parents. (If conflicts happens, git will ask you to resolve them.)

Ah, but now, your master has two more commits that the origin's master. And you surely want to share your changes with your fellow developers. That's where the git push command is useful:
git push origin

git push will start by copying your two commits to the origin, and ask it to update its master to point at the same location than yours. At the end of the operation, both commit trees should match exactly. Note that git push will refuse to push if your origin/master is not up to date.

The "origin" argument to git fetch and git push is optional; git will use all remotes if you don't specify one.

Finally, a note: as fetch and merge are often done together, a git command combines both: git pull. It's smarter than the addition of the two commands, because it knows how to look in your git config what remote branch is actually tracked by your current local branch, and merge from there -- so you don't even need to type the name of origin/master for the merge.

Next time, we'll speak about rebasing.

Monday, 6 October 2008

Git tip: rewind master, keep head in a branch

Imagine that you just committed something on your master branch, and suddendly realize that you'll have to work a bit more on it. Wouldn't it be great to have committed this last patch on a branch instead?

Git allows you to do this. You can rewind the master by one patch, while retaining the current HEAD in a new branch.

First, create a new branch (let's call it newbranch) that points at your HEAD:

git branch newbranch
Then, rewind the master by one commit:
git reset --hard HEAD^
That's all. You just moved your last commit on its own branch. If you want to continue working on top of it, you just have to
git checkout newbranch
and start hacking.
(Now, this is a very entry-level git tip, but those kind of examples, that demonstrate what makes git different from centralized version control systems, have certainly a good pedagogical value. More to come when I have tuits.)

Tuesday, 23 September 2008

Back from the Italian Perl Workshop

I'm back from Pisa, where I attended the Italian Perl Workshop 2008 last week (kindly invited by the organizers).

This was a very enjoyable conference. Several talks were in English, but most of them were in Italian: however, it was still easy for me to understand what was going on, partly because Italian is not really far from French, partly because of my already acquired familiarity with this language, and finally because good slides always help. (I'm sure I would have had a lot more trouble understanding presentations in Dutch or in Spanish, for example.)

I presented two talks; the first one, on Thursday, was a bit improvised: it was for the first part a plea in favour of writing code that stylistically will avoid clobbering the history in a version control system; for a second part, a live demonstration of a tool I wrote for consulting the history of a git repository from within vim. (More on that tool later.) My second talk, on Friday, was a presentation of the main new features in perl 5.10 -- basically an updated and translated version of the talk I gave at the French Perl Workshop 2007 in Lyon. The fact that I had a cold did not help my voice and my throat was aching after both presentations. I'm not sure I want to hear the audio recordings that were made.

We stayed the whole week-end in Pisa afterwards, visiting the city. Have I mentioned how much I enjoy Italy? Each time I go there I wish I could stay longer; and if I had to choose a country to live in besides France, that would certainly be Italy. (Moreover, they have the best food in the world. And the wines are not bad either.)

I come back with a to-do list that includes starting using Devel::NYTProf, which was presented by Tim Bunce; and looking at Matt Trout's Devel::Declare craziness. It seems that I have joined #moose on the perl IRC network, too. Those things just happen after conferences. And that's why you should go to conferences: they build the community much more than mailing lists and IRC channels. A great thanks to all conference organizers!

I didn't take many pictures of the IPW, but they're on flickr already. More sightseeing shots will follow.

Thursday, 5 June 2008

Fidena and Furiani

In the year of the consulship of Marcus Licinius and Lucius Calpurnius, the losses of a great war were matched by an unexpected disaster, no sooner begun than ended. One Atilius, of the freedman class, having undertaken to build an amphitheatre at Fidena for the exhibition of a show of gladiators, failed to lay a solid foundation to frame the wooden superstructure with beams of sufficient strength; for he had neither an abundance of wealth, nor zeal for public popularity, but he had simply sought the work for sordid gain. Thither flocked all who loved such sights and who during the reign of Tiberius had been wholly debarred from such amusements; men and women of every age crowding to the place because it was near Rome. And so the calamity was all the more fatal. The building was densely crowded; then came a violent shock, as it fell inwards or spread outwards, precipitating and burying an immense multitude which was intently gazing on the show or standing round. Those who were crushed to death in the first moment of the accident had at least under such dreadful circumstances the advantage of escaping torture. More to be pitied were they who with limbs torn from them still retained life, while they recognised their wives and children by seeing them during the day and by hearing in the night their screams and groans. Soon all the neighbours in their excitement at the report were bewailing brothers, kinsmen or parents. Even those whose friends or relatives were away from home for quite a different reason, still trembled for them, and as it was not yet known who had been destroyed by the crash, suspense made the alarm more widespread.

As soon as they began to remove the debris, there was a rush to see the lifeless forms and much embracing and kissing. Often a dispute would arise, when some distorted face, bearing however a general resemblance of form and age, had baffled their efforts at recognition. Fifty thousand persons were maimed or destroyed in this disaster.

-- Tacitus, Annals, Book IV

This happened 20 centuries ago, under the reign of Tiberius.

On the 5th May 1992, part of the biggest football stadium in Corsica (in the village of Furiani) collapsed during a major match. 15 people died, 2000 were wounded. That's a lot less than the fifty thousand mentioned by Tacitus. However, it should be noted that the Furiani catastrophe and the much older one in Fidena share the exact same causes: greed from the entrepreneurs, combined with extreme affluence to a sportive event, and lousy security measures.

Ah, progress.

Friday, 30 May 2008

Port forwarding with ssh

Let's just say that I've a box with an sshd listening on port 22, but I want it to accept as well connections on port, for example, 443. There might be many many ways to do it, but one way is to open an ssh tunnel, by running this as root on the said box:

ssh -g -N -f -L 443:localhost:22 user@localhost

-f forks, so the tunnel stays open in the background as long as the box is up. -g allows connections from the outside world. -N is there to avoid executing a command (just forward the port). And -L gives the tunnel specification.

Tuesday, 27 May 2008

Process substitution in bash

I like the process substitution syntax in bash. Since I don't expect everyone to already use it at lengths, a small note on it might be welcomed.

Process substitution connects the input or the output of a process list to a pipe whose other end is connected to the rest of your command line. For example, this will list your files in reverse order:

sort --reverse <(ls)

You'd said, I could have used a good old pipe, as in ls | sort --reverse. Fair enough, but sometimes you want more than one input, or output. That's where process substitution comes to the rescue.

I find this particularly useful with diff(1). I like diff. Don't you like diff? (GNU diff, I mean, the one with -u for unified diffs. It don't mean a thing if it ain't got that switch.) I read unified diffs every morning at breakfast. (I swear it's true.)

So let's see, I want to compare the output of two commands. Let's say I want to compare two perl optrees. (A perl optree is the intermediate form in which perl compiles your program before executing them.) To get a good, but concise, overview of a perl optree, dumped on the standard error stream, my favourite command is:
perl -MO=Concise foo.pl

(I think I'll make myself an alias for that one day.)

Let's say my almost similar programs are in files foo.pl and bar.pl. To get a diff of their optrees, the command will then be:
diff -u <(perl -MO=Concise foo.pl 2>&1) <(perl -MO=Concise bar.pl 2>&1)

(Note that I redirected stderr to stdout.) Et voilà.

And if that's too much typing, there's always history substitution:
perl -MO=Concise foo.pl 2>&1
diff -u <(!!) <(!!:s/foo/bar/)

Monday, 26 May 2008

On Naples and its garbage

The situation of garbage disposal in Naples and the monopoly that Camorra (the Napolitan Mafia) acquired upon it is an application of the formula:

mafia = capitalism - state

What happened there? A couple of decades ago, the Camorra began to invest in garbage disposal companies. Garbage disposal and its treatment are quite expensive, especially when the laws require not to dump dangerous chemicals anywhere in the fields or the rivers. But if you don't care about polluting an entire country, garbage disposal, at the market prices, is very lucrative, because it allows for enormous margins. Using intimidation and corruption to wipe out local competition, and selling its services to industrials to cheaply dispose of their waste, (thus gaining approval from the powerful industry bosses), the Camorra has now transformed the Campania in a wasteland where people are a lot more likely to develop cancers than in other parts of Italy.

Fair competition is part of capitalism, but without an effective judicial power to prevent violence against competitors, the game is biased. Moreover, without a government to edict and guarantee safeguards, protecting its citizens from whatever consequences unrestrained capitalism might have on the environment or on the society, ecological and social catastrophes are bound to happen -- at the very profit of the bad ones.

My formula of course applies as well to the Sicilian Mafia, born at the end of the XIXth century around Palermo; at that time, it took advantage of a very good economical situation in Sicily and of a default of government, Italy being a young kingdom, and Sicily just out of feudalism.

Thus, every time I read on the internets anarcho-capitalism, I mentally translate it into Mafia rule waiting to happen. I'm sure the Honoured Society would be happy to see a libertarian elected in the White House, even if this libertarian is the most honest man on earth. (The Mafia doesn't have political opinions -- it helps whoever will help it.)

(As an aside, Tony Soprano comes from a Napolitan family, and officially works in garbage disposal. The writers were documented...)

Friday, 9 May 2008

Why People Are Passionate About Perl

brian d foy asks why people are passionate about Perl. So, brian, here are a pumpking's answers to your questions.

The person who introduced me to Perl showed me that I could throw away all those fragile sed and awk scripts and get the job done much quickier. So that gets us to the second question:

I first starting using Perl to process very large data files. Or at least they were considered very large at the time: less than 2 Go, I think. But I quickly grasped the thing and began to wrote small scripts to automate various things on my computer (and others').

I kept using Perl because it was fun. And it still is.

I can't stop thinking about Perl because... they won't let me! If I stop thinking about Perl, I know they're out there to get me, lurking in the dark corners of the intarweb, with their bug reports, their suggestions and their patches.

I'm still using Perl because, well, basically, because I'm paid for it. But on the other hand, I chose a job where I was getting paid to use Perl. I could have chosen a Java, a C, or (my goodness) a PHP job. But with a proper Perl job you have the CPAN at hand, ready to be used. So you can concentrate with the problem solving and the business side of your applications, because almost all the libraries you'll need are already there.

I get other people to use Perl by hitting them on the head with a -- no wait. Actually that's the most interesting question. I get other people to use Perl by trying to make Perl a less annoying language, in my own scale.

I also program in C, but mostly when I have to patch perl itself -- the only place where you can't use Perl.

Tuesday, 25 March 2008

IRC thought of the day

On freenode, every IRC channel is the zoo of another one.

La maison des girafes

Thursday, 20 March 2008

Google charts on Flickr

Looks like the Google Chart API now does maps. I like maps.

I figured that I could map some of my flickr tags to a map, just to see where I've taken my photos. Here are some of my location-related tags, and the current number of photos tagged with:

(total for France tags)404

It's then almost trivial to come up with an image URL from google:

Neat, huh?

But where to go from there? Obviously, it would be a good idea to tag photos with the two-letter ISO country name, maybe as "country:FR" or "country:NL" (not sure what a good convention would be); then, the associated number of photos could be retrieved effortlessly via the Flickr API. Alternatively, if one wants to construct a nice mashup, having a DB full of city names or/and using geotags would be a nice idea too. Tuits anyone?

Wednesday, 19 March 2008

CPAN reports colors

search.cpan.org has, on each distribution page, a link to CPAN testers reports, and, thanks to Slaven Rezic, to matrices that neatly summarize the report statuses per distribution or per author (here's an example for Safe 2.15).

Unfortunately, the default color scheme isn't really readable for color-blind people. However, Slaven used a less known feature of CSS to provide an alternate colouring scheme. In Firefox, when viewing the matrix page, select View > Page Style > High Contrast to get the alternate CSS, et voilà, the colours are now accessible. Thanks Slaven!

Tuesday, 18 March 2008

Harry Connick Jr, the pianist

Keyboard, stripes I recently discovered the pianist side of Harry Connick Jr, who is best known as a crooner. The influence of Thelonious Monk is remarkably high. Broken rhythms, weird block chords, minimalist improvisations: we are in presence of one of the only disciples of The Sphere. He even grabs full codas from Monk. For evidence, listen to One Last Pitch, a trio recording.

And what's remarkable, is that it seems that this influence has been completely overlooked by everybody. Connick is a much, much better pianist than singer. And his piano style is certainly not what you'd expect from a white crooner with the look of an Hollywood playboy.

Well, I guess that it's just another confirmation of the old saying, all jazz musicians are underestimated.

Monday, 21 January 2008

vim and Perl 5.10

If you use vim and want it to highlight correctly the new keywords in Perl 5.10, you can just drop the following syntax plugin script as ~/.vim/syntax/perl.vim :

so $VIMRUNTIME/syntax/perl.vim
syn keyword perlStatementStorage state
syn keyword perlStatementFiledesc say
if exists("perl_fold") && exists("perl_fold_blocks")
syn match perlConditional "\<given\>"
syn match perlConditional "\<when\>"
syn match perlConditional "\<default\>"
syn match perlRepeat "\<break\>"
syn keyword perlConditional given when default
syn keyword perlRepeat break
if exists("perl_fold")
syn match perlControl "\<BEGIN\|CHECK\|INIT\|END\|UNITCHECK\>" contained
syn keyword perlControl BEGIN END CHECK INIT UNITCHECK

An extension, not implemented here, could be to add a new highlight class for the new regexp verbs (MARK, SKIP, COMMIT, etc.)

Wednesday, 16 January 2008

Disk usage graphical presentation

I found out by accident about this GNOME tool, the "Disk Usage Analyzer". It has a surprisingly good UI, displaying subdirectories as concentric circular arcs. It makes visually obvious the spots where all the place is wasted. Or spent.

As an illustration, here's a screenshot of it displaying the disk usage taken by a fresh checkout of Perl 5's sources.

I wouldn't have thought that Encode was taking so much space. (This is, of course, due to all the files that describes the various encodings recognized by this module.)