Home_greyopenFATE - openSUSE feature tracking > #305945
Dashboard | Search | Sign up | Login

Please login or register to be able to edit or vote this feature.

Get rid of "symsets" in kernel package dependencies

Feature state

openSUSE-11.2
Done

Description

Currently, the kernel rpm packages group the exported kernel and module symbols in find-provides and provide these "symsets". It has been done this way to reduce the amount of data the solver has to process. It has a number of problems though:

  • Moving modules around or adding symbols changes the symset checksums, so the rpm build needs symsets of the previous kernel(s) to provide both old and new checksums
  • changing or removing one unused or unimportant symbol means that each kmp that uses any other symbol from the same symset needs to be rebuilt
Now that we have a much better solver, it's time to get rid of this and provide just the individual symbols. I created a test project in the build service: home:michal-m:kill-symsets. which rebuilds all 11.1 kmps against a normal kernel and against a kernel with symsets turned off (this kernel is built in home:michal-m:kill-symsets:changes). The size difference is rather low, total size of all the rpms on i586 and x86_64 increased by about 1.3M, installed size decreased by 2.6M (because of the removed /boot/symsets* files). Size of yum metadata increased by 1.4M.

What needs to be done is to make sure that the solver can really handle this fine (-> Michael). I'm waiting for the test repos to appear on download.o.o.

Relations

Testcase

All kernel and kmp packages have ksym(...) provides/requires instead of kernel(...). KMPs must install without dependency errors.

Discussion


icons/user_comment.png M. M. wrote: (9 years ago)

Added attachment table of size changes (sizes.txt)

icons/user_comment.png M. M. wrote: (9 years ago)

I tried installing packages from http://download.opensuse.org/repositories/home:/michal-m:/kill-symsets/ and I didn't notice any prerformance issues with zypper, so I hope there won't be any problems with 4k additional rpm provides.

I pushed this to master, let's hope it will be possible to do this change also in SLE11-SP1.

icons/user_comment.png A. G. wrote: (9 years ago)

Thanks -- hopefully you were not too fast with this: the issue to watch out for here is not the rpm package size. Rather, the real issues are:

  • The additional size of the repository metadata,
  • The speed of the resolver for complex tasks like system updates.
  • The time it will take rpm to resolve dependencies during package installation and updates. This may affect all packages, not just the kernels and kmps. (Our resolver sure performs much better than rpm itself.)

Did you try out these things as well? Thanks!

(Also see Feature #303641, which is a duplicate of this feature.)

icons/user_comment.png M. M. wrote: (9 years ago)

The yum repodata changed by about 1.5MB, see the attachment. I didn't really test yet how rpm itself copes with the change.

icons/user_comment.png A. J. wrote: (8 years ago)

Michael, could you comment on this, please?

icons/user_comment.png M. S. wrote: (8 years ago)

How much bigger is primary.xml.gz?

I don't think our solver will have any problems, but I don't know about rpm. Please do some tests with the rpms, i.e. installung multiple kernel versions and flavors, updating kernels...

icons/user_comment.png M. M. wrote: (8 years ago)

I copied matching kernel flavors from SLE11_BRANCH and master kotd to two directories and ran createrepo in each of them. The difference is:

$ ls -1sh */repodata/primary.xml.gz 
1,4M factory/repodata/primary.xml.gz
60K sle11/repodata/primary.xml.gz

which matches my earlier tests. When I installed the factory kernel-default on a 11.1/x86_64, /var/lib/rpm grew from 61MB (after rpm --rebuilddb) to 63MB. When installing/updating kernels and KMPs, I couldn't observe any slowdown, most time was spent in mkintrd and the weak-modules2 script.

Interesting observation: the kernel exports over 9k symbols, but only about 800 are actually used by KMPs.

icons/user_comment.png A. O. wrote: (8 years ago)

Michael, on openSUSE 11.2 RC1 when i instal multiple kmp modules i always get an message of dependencies error is that a bug ?

icons/user_comment.png M. M. wrote: (8 years ago)

Which kmps and which errors? The kabi is not frozen yet, so make sure you have kmps matching your kernel. And please report problems like this one in bugzilla, this feature is done.

icons/user_comment.png M. M. wrote: (8 years ago)

Andreas - you rejected the feature for 11.2, but in fact this has been in factory since March (see comment #2). And I haven't seen any reports about slow kernel / kmp installation, so I think the impact is relatively low.

icons/user_comment.png A. J. wrote: (8 years ago)

Then why was this not marked as done? I expected there was further work needed.
Michal, you can set features yourself as done.

icons/user_comment.png M. M. wrote: (8 years ago)

We have today found out that is detrimental to the openSUSE 11.2 infrastructure.

Every kernel-flavour RPM takes 800KB, kernel-flavour-base 400KB uncompressed primary size.

As we keep all kernels in the repo.

2 kernel updates for 11.2 so far have caused a primary.xml size increase of 33MB.

Yes. 16 MB more per kernel updates and we have 2 architectures only on 11.2.

This is going to be quite unmanagabkle very soon.

Last change: 8 years ago
Voting
Score: 2
  • Negative: 0
  • Neutral: 0
  • Positive: 2
Tags
Feature Export
Application-xmlXML   Text-x-logPlaintext   PrinterPrint