Fernando J. Pereda’s blag

October 6, 2008

Generated versions of PMS

Filed under: blag — Tags: , , , , , , — Fernando J. Pereda @ 12:13 pm

Yesterday I linked some PDF versions of PMS in its ‘home page’. This makes PMS more accessible to those that can’t or won’t install a proper TeX system since reading the LaTeX sources is a PITA.

I’ll generate and link versions approved by the Gentoo Council by checking out their signed tags and those versions that the PMS editors deem important. I’ll also link to current HEAD, but this won’t be automated so it might lag a bit if I’m extremely busy.

Also, for those that can’t be bothered reading technical documentation aimed at people implementing a package manager and want to know what’s new in EAPI2, Ciaran McCreesh has published a series of blag posts explaining the new features and whence they came. Make sure to take a look at What’s in EAPI 2?

— ferdy

June 11, 2008

How I migrated Paludis to Git

Filed under: blag — Tags: , , — Fernando J. Pereda @ 7:41 pm

Paludis has been using (and it currently still uses) Subversion to manage its source. We’ve been using git-svn for some months now and recently ciaranm agreed to fully migrate to Git.

To migrate the repository I used my old git-svn clone. This made some stuff a bit trickier, but it was both faster and nicer with pioto‘s server. Things that had to be done:

  • Remove ChangeLog and ChangeLog.old.bz2
  • Remove metadata added by git-svn
  • Rewrite authors and emails since I didn’t use an authors-file
  • Remove empty commits (these are commits that only touched ChangeLog or ChangeLog.old.bz2)

This looks like a good task for git-filter-branch. I probably could have done everything in one go, but I decided to do it one at a time.

Since filter-branch is mostly IO-bound, we’ll try to speed it up as much as possible:

$ sudo mount -t tmpfs -o size=100M none paludis/.git-rewrite
$ myrefs="0.4 0.6 0.8 0.20 0.24 0.26 ....."

The first task is easy:

$ git filter-branch -f --tree-filter 'git update-index --remove ChangeLog' $myrefs
$ git filter-branch -f --tree-filter 'git update-index --remove ChangeLog.old.bz2' $myrefs

To remove metadata created by git-svn I came up with:

tac | sed -n -e '1d' -e '/[^[:blank:]]/,$p' | tac

but dleverton came up with this perl one-liner, and I used it instead:

perl -ne 'print @blanks, $last and undef @blanks if defined $last; if (m/\S/) { $last = $_ } else { undef $last; push(@blanks, $_) }'

I put it in a file and ran:

git filter-branch -f --msg-filter ~/munge-commit-message $myrefs

Changing authors needed a script like the following (with proper mail-addresses):

case ${GIT_AUTHOR_NAME} in
        ciaranm)   n="Ciaran McCreesh"      ; m="foo@bar.com" ;;
        spb)       n="Stephen P. Bennett"   ; m="foo@bar.com" ;;
        halcyon)   n="Mark Loeser"          ; m="foo@bar.com" ;;
        allanonjl) n="John N. Laliberte"    ; m="foo@bar.com" ;;
        steev)     n="Stephen Klimaszewski" ; m="foo@bar.com" ;;
        kugelfang) n="Danny van Dyk"        ; m="foo@bar.com" ;;
        ferdy)     n="Fernando J. Pereda"   ; m="foo@bar.com" ;;
        arachnist) n="Robert S. Gerus"      ; m="foo@bar.com" ;;
        drizzt)    n="Timothy Redaelli"     ; m="foo@bar.com" ;;
        djm)       n="David Morgan"         ; m="foo@bar.com" ;;
        pioto)     n="Mike Kelly"           ; m="foo@bar.com" ;;
        piotr)     n="Piotr Rak"            ; m="foo@bar.com" ;;
        rbrown)    n="Richard Brown"        ; m="foo@bar.com" ;;
        baptux)    n="Baptiste Daroussin"   ; m="foo@bar.com" ;;
        eroyf)     n="Alexander Færøy"      ; m="foo@bar.com" ;;
        compnerd)  n="Saleem Abdulrasool"   ; m="foo@bar.com" ;;
        omp)       n="David Shakaryan"      ; m="foo@bar.com" ;;
        dleverton) n="David Leverton"       ; m="foo@bar.com" ;;
        peper)     n="Piotr Jaroszyński"    ; m="foo@bar.com" ;;
        dev-zero)  n="Tiziano Müller"       ; m="foo@bar.com" ;;
        zlin)      n="Bo Ørsted Andresen"   ; m="foo@bar.com" ;;
        buildtest) n="Nightly Buildtest"    ; m="foo@bar.com" ;;
        flameeyes) n="Diego Pettenò"        ; m="foo@bar.com" ;;
        iluxa)     n="Ilya Volynets"        ; m="foo@bar.com" ;;
        dercorny)  n="Stefan Cornelius"     ; m="foo@bar.com" ;;


git commit-tree "$@"

and ran:

$ git filter-branch -f --commit-filter ~/rewrite-authors $myrefs

Removing empty commits requires a bit more foo:

        while [[ -n $1 ]] ; do
                map "$1"

our_parent_tree=$(map $3)

if [[ -z ${our_parent_tree} ]] || [[ -n $(git diff-tree ${our_tree} ${our_parent_tree}:) ]] ; then
        git commit-tree "$@"
        skip_commit "$@"

This one could have just tested whether the current tree is the same as our parent’s tree (that is, no changes were made by this commit):

[[ ${our_tree} == $(git rev-parse $(map $3):) ]]

But it wouldn’t have made a big difference and I noticed it while filter branch was already running something like:

$ git filter-branch -f --commit-filter '. ~/empty-commits.bash' $myrefs

There’s still stuff to do like tags and adding scratch and probably converting the overlay; but the big thing is done. I think that history is stable already, that is, I won’t have to rewrite it again.

It is sitting in my home in bach and will be published soon.

Update: Re-tagging every paludis was the last step. I thought it was going to be cumbersome and boring, however, git makes this kind of stuff pretty easy. Since ciaranm should sign the tags himself, I did:

$ git log --pretty=oneline origin/releases |
> sed -n -e '/^\([0-9a-f]\{40\}\) Tag\( release\)\? \(.*\)/s--\3|\1|Tag release \3-p' \
> > ~/paludis-git-tags

After some hand editing of the file, creating the tags can be done with something like:

$ while read name msg head ; do
> git tag -m "${msg}" ${name} ${head} ;
> done < paludis-git-tags

To checkout an exact version (assuming ~/git/paludis is your repo, doesn’t have to be a local repo):

$ cd somewhere
$ git archive --format=tar --remote=~/git/paludis --prefix=paludis- 0.4.0 0.4.0 | tar xf -

— ferdy

May 3, 2008

On cooperating and paludis vulnerability

Filed under: blag — Tags: , , , — Fernando J. Pereda @ 11:31 am

Note: This was already published in my old blag but planet and wordpress insist on publishing it again. Sorry.

A serious security issue in paludis was brought to my attention recently, and I feel I should make you all aware. Apparently someone, with root access to a machine, can gain root access by installing or editing paludis config files.

For those interested, this is how it happened (times are GMT+1):

22:34 <@ferdy> bonsaikitten: can you give me any details regarding that
 security bug in paludis?
22:35 <+bonsaikitten> ferdy: it's so obvious you should have found it already
22:37 <@ferdy> bonsaikitten: I should, but I probably haven't
22:37 <+bonsaikitten> ferdy: well, as I am a moron I'm unable to coherently explain :)
22:37 <@ferdy> bonsaikitten: I mean, depends on whether we are talking about
a real security issue or about something we should document to avoid people
shooting themselves in the foot
22:39 <@ferdy> bonsaikitten: is that all you are going to tell me?
22:39 <+bonsaikitten> ferdy: come on, it's obvious. You're supposed to be smart ...
22:39 * bonsaikitten is not in a mood to explain
22:40 <@ferdy> bonsaikitten: you aren't really talking about the paludisbuild issue, are you?
22:41 <+bonsaikitten> mmh no, that's a different one
22:41 <@ferdy> k
22:41 <@ferdy> bonsaikitten: what are we talking about?
22:42 <@ferdy> bonsaikitten: you don't need to explain it... just say, in general 
terms, what the issue is
22:50 <@ferdy> bonsaikitten: so? care to give any useful hint?
22:50 <+bonsaikitten> ferdy: doesn't happen in portage compatibility mode
22:51 <+bonsaikitten> but I blame the vodka, hard to explain when *burp* *giggle*
22:52 <@ferdy> bonsaikitten: what's the impact?
22:53 <+bonsaikitten> ferdy: depends on how annoying the other person is
22:54 <+bonsaikitten> ferdy: worst case random file modification
22:58 <@ferdy> bonsaikitten: and we already agreed that we aren't talking about
the paludisbuild issue, right?
22:59 <@ferdy> bonsaikitten: if we aren't, I'll need more hints....
23:05 <@ferdy> bonsaikitten: can I get an attack vector?
23:05 <@ferdy> that shouldn't need lots of explaining... I can figure out that
part myself
23:19 <@ferdy> bonsaikitten: have you got that attack vector for me?
23:24 <+bonsaikitten> ferdy: look at configuration files, maybe you notice that
there's some exquisit code execution possible there
23:29 <@ferdy> bonsaikitten: you mean those config files that only root can
edit? I must be missing something here
23:29 <+bonsaikitten> ferdy: you are :)
23:29 <+bonsaikitten> not much, and it's basically the same flaw bashrc is
for portage
23:29 <+bonsaikitten> only that bashrc is config_protect'ed ...
23:30 <@ferdy> bonsaikitten: but for a package to clover those files, it must be
in a repo root added, right?
23:31 <+bonsaikitten> someone in the package mangler group, but yes
23:35 <@ferdy> bonsaikitten: but if you can change those files in the first place,
why clover them by adding a malicious repo with a malicious package that changes
those files?
23:35 <+bonsaikitten> ferdy: because it's very subtle
23:36 <@ferdy> moreover, if you can already do that, why not just make the
package install whatever backdoor you want?
23:37 <@ferdy> I mean, it is subtle, but why would anyone go the 'convoluted'
route? he is already able to edit those files (since he had to add that repo)
23:38 <+bonsaikitten> 'cause only paludis is affected and you will find it very
hard to trace
23:38 <+bonsaikitten> that makes it so tempting ...
23:40 <+bonsaikitten> just don't be surprised if it suddenly unmerges itself :)
23:41 <@ferdy> yeah... well...
23:41 <@ferdy> bonsaikitten: mind if I disclose this vulnerability in
23:42 <+bonsaikitten> go ahead
23:42 <@ferdy> ta
23:42 <+bonsaikitten> 't is even on the features page of the package mangler :)

This is a good lesson to learn today:

If you can edit files owned by root in a machine, you can get root access to that machine.

So the bottom line is: There is no vulnerability, if you can mangle paludis config files, you are already root so you don’t need to edit a file to run any command you want. Another lesson one can learn by reading that log is how to be really cooperative.

Ah, and before someone with a need to use cheap psychology asks, the intention of this blag post is to stop the FUD.

— ferdy

Blog at WordPress.com.