Ideas for dm-crypt support

Cardoe wrote this in the wee hours:

In an attempt to improve cross-distro support, I have been considering for some time now reworking the way the current dm-crypt setup in Gentoo works and making it work more like Debian, Ubuntu, Red Hat, Fedora, and CentOS do. Effectively the syntax they use is that of fstab, however its in a new file called /etc/crypttab. This file is formated in the following style:

/dev/VolGroup00/data    crypt-data    none           luks # Password style LUKS
/dev/VolGroup00/tmp     crypt-tmp     /dev/random    tmp  # sets up /tmp for each boot having a fresh key
/dev/VolGroup00/swap    crypt-swap    /dev/random    swap # sets up swap for each boot having a fresh key
/dev/VolGroup00/other   crypt-other   /dev/sda       luks # uses /dev/sda as the key source

The partitions above would all be referenced by /dev/mapper/crypt-NAME. As you can probably guess the above configuration is not an exhaustive iteration of all the configuration possibilities. I am merely trying to investigate if this is something that people would be interested in before I code it. I’ve had this idea kicking around in my head for over 3 months now (in fact I e-mailed the other base-system maintainers about it at the start of March) but I’m terrible at keeping up with my blog. In fact, I’ve still got some parts of my original council platform from LAST year’s election still sitting there as draft posts.

But back to dm-crypt. For more information, look at the Red Hat crypttab(5) man page. While it says there are no options for LUKS, they have added automatically added checking for LUKS and using that instead of the pre-LUKS crypt bits.  That being said, the Debian based version apppears to map bettter for us since it has a lot more options, Debian based crypto crypttab(5). Let me know what you think.

GLEP 54 Mullings

Cardoe wrote this mid-morning:

So since GLEP 54 is at the fore front of everyone’s minds I figured I’d weigh in on it. The two main issues I’ve wanted to see addressed with GLEP 54 are:

  • a consistent method for exposing the revision ID of the sources that was used. This would most likely have to be exposed via a hook that the underlying scm functions (eclass) would provide
  • Information how this can be integrated with Portage (will Zac code this or will be something we’ll have to find someone to code?). Additionally information/API how we could expose the revision (i.e. equery list pkg)

While I know my approach is very Portage centric there are two reasons for this. Paludis already supports this stuff and Portage is the predominate package manager in use by Gentoo users today. While many would simply chalk it up to me being another anti-Paludis developer it’s far from the case. To potentially the surprise of some, I am a paludis user, however I am not a paludis user on my Gentoo commit machine. I feel that since Portage is our predominate package manager in use by users, it’s the package manager I should be using to ensure my ebuilds work. While 99% of the time a test with paludis is good enough, there are always situations where paludis and Portage have different behaviors and it needs to be caught.

While I think that everyone involved in GLEP 54 has the best interest in mind. I see a package that I have maintained for years being used as the example package in the back and forth debates on the topic and to be honest I wish it would stop. I’ve seen a lot of assumptions made about MythTV and back and forth comments about what would be the best for the package. So, I will state the needs of Gentoo’s MythTV packages here once and for all.

MythTV upstream has stated time and time again that they prefer people to use SVN pulls of the code instead of tarballs since tarballs get stale and they’re constantly committing necessary updates without rolling a new tarball. For a long while it worked to apply patches on top of the tarball but this is no longer a viable solution since there’s a lot of changes that actually happen to binary blobs which diff can’t catch. Upstream wants to be able to find out exactly what SVN revision a user is currently using and direct the user to use a different revision very easily. MythTV has a “stable” branch (which isn’t necessarily any more or less stable then their trunk branch) which most distros use but hardly any MythTV developers use and their trunk branch where all their commits and activity takes place. This is what most developers use and they’ll be quick to tell you that it’s more solid then the “stable” branch, until one day when its not and it eats all your programming and preferences and babies. So to remedy this, beandog, myself and now joining in tanderson have implemented functionality to handle the following example ebuilds:

  • mythtv-0.21_p19800
  • mythtv-0.22_alpha20500
  • mythtv-0.22_beta21400

In the above examples, the _p is evaluted and based on that we know that we’re on a stable branch. It then grabs the version, which is 0.21 and converts it to “release-0-21-fixes”, which is the branch name, then we take 19800 as the SVN revision to checkout. For the alpha version the following happens, the _alpha is evaluted to mean “trunk” branch, then 20500 is checked out from trunk. The fictional beta version is supported for the special cases when the developers branch the trunk version for an impending release but haven’t moved it to the normal place yet. Unfortunately over the years they’ve used different places to store these impending releases each time so I just tweak the eclass everytime for this.

There’s two features which I’m hoping will be handled nicely.

  • Have a nice way to fetch the source code outside of src_unpack. How I’ve handled this is managing to get upstream to enable tarballs (well they turned on zip only, not tarballs…) for specific revisions using a specific URL. This will appear in the tree shortly. The downside of this is that we’re having upstream’s server build a tarball of each revision and every revision will require over a 30mb download.
  • Have an easy way to provide a “live” ebuild for trunk. The current GLEP 54 proposal handles this with -scm, however as I stated above. It doesn’t address the storage of the SVN revision which is a must.

UPDATE

I would like to point out to everyone that is sending me mailing list comments that what’s presented to the council is NOT a mailing list comment but in fact the GLEP. The council is not taking up “GLEP 54 + message ID xxxx on gentoo-yyy ML”. The council is being asked simply about GLEP 54 and I am stating what I feel are the shortcomings of the GLEP. If you have an idea that you are suggesting that would bridge the shortfall in GLEP 54, grab the source to the GLEP and add your ideas and re-submit that as the GLEP to be approved.

Inactivty Update

Cardoe wrote this in the late afternoon:

Re:  Gentoo Inactivity Thanks for the suggestions everyone. I ended up using a post board along with MSI’s documentation and it kept claiming the CPU was bad. But when I hit the CMOS reset button (this MB has a button instead of a header), it would boot exactly ONCE. If you did a soft or hard reset, it would claim the CPU was bad again. So I took my old desktop CPU that I had laying around (it wasn’t in a protective case so its questionable…) popped it in with the same results. One of the DIMM slots also appeared bad because when I could get it to boot, when the RAM was in that slot I got a memory check error. Given the fact that the board was over 2 years old, but under 3, it still fell within MSI’s warrenty period. However it was only covered under a parts warrenty and not a labor so I decided to purchase a new motherboard. I ended up on a GeForce 8200 based board since this will hopefully lower my power consumption (since I can remove my discrete card) and allow me to use NVIDIA’s VDPAU for video playback. The board I chose was the MSI K9N2GM-FIH because of its price and HDMI connector.

With regard to Google for my e-mail on my domain, Google support took a day to answer my e-mail but they were prompt in fixing the issue for me. Apparently it was a data inconsistency in the sign up process which caused one piece to think I was already signed up when I hadn’t completed all the sign up pages. After the quick fix it’s been working great and sorting my e-mail happily. Though I am pretty disappointed in Google’s spam filter since a lot more spam mails are getting through then my previous dspam setup (about 25 a day which is up from 2 a day).

I did however have my old trusty Netgear XXX-614 model go out on me and ended up purchasing an ASUS WL-520GU and flashing DD-WRT on it. I haven’t had much time to tinker with DD-WRT, however I’m hoping to get the USB port working to drive my Samsung ML-2010 and maybe get asterisk running on it to drive my VOIP phones at home. Currently I’m running Eko’s v24 TNG svn11218 on it. I still have to setup Static DHCP and port forwarding so I can ssh into my Gentoo machines remotely.

Gentoo Inactivity

Cardoe wrote this mid-afternoon:

So as I had previously mentioned, I’ve relocated to another part of the US and started a new job. Due to the relocation I had some down time, simply because, well I had no internet and my machines were in boxes. After the relocation, my in-laws came to visit which resulted in less free time to devote to Gentoo. As soon as they left I had an interesting issue crop up. My MythTV box, which also serves as my IMAP machine (it downloads all my @gentoo.org, @cardoe.com, @php.net and @freedesktop.org mail and combines it into one IMAP account with de-spam-ing and mail sorting) wasn’t responding to pings. I turned on the TV connected to the machine and it claimed there was no signal, the odd part though the machine does have a LCD on the front controlled by LCDproc & mythlcdserver, which kept the proper time. After attempting to reboot the machine (it’s connected via a wireless bridge ATM since I haven’t wired my new place up yet) it never came back up. So far I’ve taken it apart down to just the MB, CPU, RAM and video card (tried 2 different video cards) and I can’t get the machine to post. Unfortunately I don’t have any PC speakers available so I can’t see if there’s a BIOS error code being beeped out. But this is why I haven’t been replying to e-mails or even reading them.

I did attempt to sign up for Google Apps for Domains today for cardoe.com, with the intent of forwarding everything there. However Google claims that I already have a Google Apps for Domains account and no password I’ve ever used in my life works.

Until I get some of these issues resolved it looks like I’m still out of commission for a few more days.

Gentopia has left the building

Cardoe wrote this just before lunchtime:

The long and arduous trek that the Gentopia Project has seen has finally come to an end. From it’s inception the project faced stiff push back from Gentoo and it’s developer base. In fact, the project was originally named “Project Utopia” mirroring Robert Love’s and Joe Shaw’s vision of a Linux desktop that simply worked. But the Gentoo Top Level Managers at the time deemed it not Gentoo-y enough and the project was quickly reborn as “Gentopia”.

The project quickly grew with developers becoming involved with many of the new technologies and software products coming from the fairly unexplored upstream Freedesktop.org project. The simple interfaces and actions our machine did for us automatically that we loved so much we distrusted by many Gentoo developers who liked the control and know how over their machines. Many developers vowed to never install HAL on their systems for as long as they lived. With GNOME and KDE adopting HAL support and even requiring it, the tide slowly began to change. People expected “Project Utopia” style support from Gentoo and they had it. The project began to wane once people grew acustomed to these features appearing and the integration with their systems improved.

Recently we’ve seen a freedesktop herd rise up in Gentoo which is a collaboration between the various desktop environment herds which is a good thing for users. From the get go I have wanted this herd to take over the Gentopia herd and project and fold it into it’s wing since it includes many freedesktop.org projects, the time has finally come for this to happen. As a result, the Gentopia overlay, web page and e-mail alias are going away and being replaced by the freedesktop herd’s resources. The transition should be seamless for everyone except users of the overlay. For layman users, please run layman -d gentopia and that will remove the overlay from your system.

For the freedesktop herd members, I look forward to them picking up the banner and continuing to provide us with a solid desktop experience that just works.

Big Change

Cardoe wrote this in the late afternoon:

I know I said I was going to work to be more active in the blogosphere with regard to the Gentoo Council however I’ve had a few things come up which has detracted from that. Firstly, with the holidays I had my family and my wife’s family together which ended up being a good time but tiring. Then I spent the week after New Years traveling. And to wrap it up the big change, I’ve accepted a job as a Linux developer in Huntsville, Alabama and I start before the end of the month so I’ve got a lot of moving ahead of me.

I’ll do my best to be responsive via e-mail however my commit activity will probably be pretty low for this coming month.

With regard to Council happenings for the month of December (and the first meeting of January). Nothing was submitted to the council for discussion for the last week of December or the first week of January so those meetings were canceled respectively. The first meeting in December brought about a few items which were discussed and ultimately approved. Instead summing them up I’ll provide direct links to the approved proposals:

The Gentoo Council

Cardoe wrote this in the early afternoon:

It’s been a long time since I’ve written on my blog but I figured that since I am now a member of the Gentoo Council, it’s time I start writing again. I’ll start with a little summary of the Council and what’s been going on lately.

The Gentoo Council is a group of elected Gentoo Developers that are elected on a yearly basis by the developer body as a whole for the purpose of deciding on global issues and policies which affect the Gentoo Linux Distro as a whole or part. The Gentoo Council serves as the technical oversight to the the entire project. We are charged with representing the will of the developer body, while maintaining the best interest for Gentoo and it’s user base. In effect, the Gentoo Council derives its authority from the developer body, this is what differentiates it from the Gentoo Foundation, which handles the financial side of Gentoo.

Gentoo Council meetings are bi-monthly for 1-hour sessions. These sessions are always held publicly on IRC on Freenode in #gentoo-council at 2000 UTC on the 2nd and 4th Thursday’s of the month (with the exception of major holidays). We welcome all interested parties to come join us.

The Council’s agenda items are set by user and/or developer requests on the gentoo-dev and gentoo-council mailing lists. If there are no agenda items set this way or not enough items to occupy the entire 1-hour allocated meeting time, then opened bugzilla entries assigned to the Council are discussed.

Currently the Gentoo Council has one open seat and at the last meeting the Council voted 5 to 1 to open up nominations for 1 week and hold elections for 1 week in place of taking the next person in line from the previous election.

The Council is also taking up issues such as inactive architectures, deploying as-needed by default for the user base, and a scm marking for ebuilds. Recently decided items include approving PMS to be a draft specification for EAPI=0, approving EAPI=2 for use in the tree, and how to handle developers that have been removed from the Gentoo project.

OpenRC hits the tree

Cardoe wrote this mid-afternoon:

OpenRC 0.2 has hit the tree and is our goal for unmasking to ~arch. It’s currently ready for prime time on your ~arch machines. If you’d like to give it a test simply create a /etc/portage/package.unmask/openrc file and fill it with:

>=sys-apps/openrc-0.2
=sys-apps/baselayout-2.0.0*

As always, any issues should be reported to Gentoo’s Bugzilla and assigned to base-system@gentoo.org

One gotcha is that util-linux’s crypto-loop initscript appears to have been silently updated for OpenRC support. You’ll need to make sure you’re on a 2.13 or higher version and potentially re-emerge it.

OpenRC & baselayout-2

Cardoe wrote this in the wee hours:

I know everyone’s wondering when this will hit the tree and I have an answer for you. I’m actively working with the necessary people to make this happen. OpenRC 0.1 was my target originally but I was away the weekend after it came out, then Roy announced an ABI break due to issues with his use of realloc. (Bad Roy! Bad realloc! Though I was not one of the complainers.) So basically Gentoo is going to wait for OpenRC 0.2 and then bring it into the tree initially masked until we can sync everything together and then unleash it to ~arch shortly there after.

The associated bugs for this are #212696 and #213988.

/lib & /bin consistency

Cardoe wrote this late at night:

One of the things I’ve occasionally bumped into on a system that’s gone south and required some recovery is that tools that exist in /bin and /sbin rely on something in /usr, which is often not mounted at the time.

I was thinking of possibly adding this as a Portage QA check. Basically, anything that is installed into /lib, /lib32, /lib64, /bin, and /sbin get passed through scanelf -Ln or ldd to determine if anything would be outside the scope of those directories and it could be fixed.

Also, if any of the people that build binary packages and then do some tests against them want to add this as a test to their system, I’m sure people would be appreciative.