Tuesday, September 23, 2008

More Git Madness

Today, I spent some time looking at better ways to organize the Portage tree in git, and I'm interested in getting feedback on what I've done.

Please take a look at my new portage-new git repository. This new repository contains both the main gentoo.org tree in the "master" branch, and the funtoo.org tree in the "funtoo.org" branch. This seems to be a much better way to organize things, for the following reasons:

  1. It's space-efficient - the trees are over 99% similar, and now a single clone operation grabs both.
  2. There is a unified history - you can easily see the differences between the trees by typing "git diff master funtoo.org".
  3. The GitHub Network Graph now shows how the gentoo.org and funtoo.org tree relate to one another, which is useful. In the funtoo.org tree, you can see where I'm pulling from.
  4. It allows people to easily switch between both trees with a simple "git checkout" command.

If you want to test out portage-new and see how the branches work, please consult my updated wiki documentation but clone "portage-new" rather than "portage". I have the repo name as "portage" in the wiki docs because I'm already anticipating making this tree the official one in a few days.

I think this is probably the repository model to use for Portage git development. If someone wants to use this tree as the basis for their own development, they can clone the tree and create a foobar.org branch that contains their changes. This will allow them to benefit from the multiple-branch model and facilitate easier integration and diffs with upstream.

Barring any major complaints, in a few days I am probably going to delete my two existing portage git repositories and rename portage-new to portage, and it will become the official one.

Let me know what you think.

12 comments:

Marlon said...

Hi Daniel,

I use your funtoo amd64 stage3 to build a stage4 with some modifications, like gcc-4.3.1,last glibc and hashstyle ldflags. Just a few packages do not compile with gcc-4.3.1, unmask the unstable resolve the question.

Soon as possible, I will upload the stage4 for anyone. And of course write he steps to build and install this stage4.

Thank you,

Marlon

Daniel Robbins said...

Sounds cool, Marlon. Soon I hope to share some tools that will help you build up custom stages.

Regards,

Daniel

Iain said...

Welcome to Planet Larry :)

Matthew said...

logical way to go

zrr said...

I already switched from "emerge --sync" to "git pull" to update my portage every day ;)! This is good stuff. Yesterday I installed the latest Xorg (1.5.0) from your Portage and it all works very well.

In any case I believe it is best you just have _one_ portage tree and let somebody else run another portage tree, i.e. a more normal one or a more "yellow one" or whatever. Linus Torvalds is also only responsible for one Kernel. The mm-Kernel is done by somebody different. When I pull from you I want to be sure that I only have one option. The crazy Daniel-Robbins-Option!

Best
Zeno

Daniel Robbins said...

I forgot to add point #5 - it's much easier to generate patches for upstream. Since we are all going to be sharing changesets at some point, the easier we can make this process the better.

Zeno - glad things are working well for you. If I understand you correctly, you like having only one branch so you don't get the wrong branch. I think I can address this problem by making a patch for emerge --sync so it will do all the git magic behind the scenes to ensure you are in the proper branch. For collaboration (sharing patches) I think this model is the way to go.

zrr said...

Not a problem, I can handle "git checkout". I prefer to use "git pull" to "emerge sync". Lets see how it goes.

Best
Zeno

Wubdidu said...

Hi Daniel,

I threw out my portage tree and checked out the funtoo.org branch of portage-new to /usr/portage.

git pull owns emerge --sync :)

But I ran into one little problem... :/
update-eix doesn't work anymore... it can't find any ebuilds inside /usr/portage, only the ones inside my layman overlays...
It even throws out numerous errors about missing eclasses when I move your portage tree somewhere else and set PORTDIR...
emerge -s is a real pain :/

Apart from that everything works great!

Eduardo said...

Hello Daniel, I'm making a new install of gentoo with both your stage 1 and portage via git, thanks a lot for this, i really apreciate your efforts with the gentoo comunity.
I'm looking foward for the emerge --sync patch, it will make the things more simple for people that doesn't know how to use git!
Thanks again!

Shanoah said...

As far as eix goes, I ran into the same issue. Go into ~/.eixrc, add the line
"PORTDIR_CACHE_METHOD=parse|ebuild*", and rerun "update-eix". It'll take forever, but eix will see everything afterwards.

Anonymous said...

Shanoah, I'm only using PORDIR_CACHE_METHOD=parse in /etc/eixrc. It update really fast and work fine.

phess said...

I'm only using PORDIR_CACHE_METHOD=parse in /etc/eixrc. It update really fast and work fine.


If you're using the Funtoo tree as an overlay like I am, you should put:

OVERLAY_CACHE_METHOD=parse instead of PORTDIR_CACHE_METHOD=parse