libzypp: Services

libzypp 17.38.13

Services

Author
  Duncan Mac-Vicar P. dmacv.nosp@m.icar.nosp@m.@suse.nosp@m..de

Services provide a list of repositories. So when the service is refreshed, the repositories specified by the service service is synced with the local repository list.

# Classes of services

There are two classes of services:

## Remote Services

Remote services are a subscription to a remote server, that provide a list of repositories that will be synced with the previous list of repositories for the same service.

By default, the remote services list is stored in the /etc/zypp/services.d directory in standard INI files. Each file can contain more than one service.

The format is the following:

    [servicealias]
    name=My Service
    url=http://customers.maintenance.company.com/service
    enabled=true
    autorefresh=true
    type=nu

The type "nu" stands for Novell Update, which gives a customer the repositories it is entitled to access. The type "ris" is an extended version of the "nu" type. Additionally, the file could contain "repostoenable" and "repostodisable" entries, which contain a list of repository aliases to enable or disable on the next refresh. This line is modified by other programs to force enabling or disabling a certain repository.

From the remote side, the service url needs to provide a repoindex.xml file with the repository list:

    <repoindex>
      <repo url="http://foo.com/repo1" alias="a repository 1"/>
      <repo url="http://foo.com/repo2" alias="a repository 2"/>
    ...
    </repoindex>

## Plugin Services

Plugin services are simple scripts that return a list of repositories. They are installed locally in the system using packages. For each script installed ZYpp will “see” a service of type “plugin”. When you refresh the service, the repositories this script returns as output will be added (or removed and kept in sync if they change afterwards).

A plugin service is a program installed in /usr/lib/zypp/plugins/services.

Once a program called "foobar" is installed, listing services will show it:

    > zypper ls
    #  | Alias            | Name         | Enabled | Refresh | Type
    ---+------------------+--------------+---------+---------+-------
    1  | foobar           | foobar       | Yes     | Yes     | plugin
    ...
    ...

When this service is refreshed, the program will be executed and it will return the list of repositories in the same .repo files format that can be found in /etc/zypp/repos.d:

    # output returned by the plugin

    [repo-oss]
    name=oss
    enabled=1
    autorefresh=0
    baseurl=http://download.opensuse.org//distribution/11.4/repo/oss
    type=yast2
    keeppackages=0

    ...

As you can see, the main advantage of plugin services is that they can read information from the client and calculate the repository list from there. Nothing prevents a plugin service to still interact with a server to get the repository list.

The main disadvantage is that you have logic in the client side that in case of bugs or changes, needs to be updated.

Spacewalk integration is implemented using plugin services. The plugin talks XML-RPC to the server and asks for the list of channels the system is subscribed to. The plugin needs to read the server address and perform the login.

However, the spacewalk plugin service does have an extra indirection, as the repository list returned are not plain urls, but urls of the type plugin:$name?params, which use the urlresolver plugin to get the real url. This is not necessary, and here it is done because it allows to add custom headers to the HTTP requests. You can read more about urlresolver plugins Url Resolver plugins here.

# Refreshing services

Using zypper, you can refresh services by executing

    zypper refs

The repositories that are listed in the service will be added, using the repository alias specified in the service index prefixed by the service alias: e.g. "myservice:myrepository". Repositories that vanished from the service will be automatically removed.

# Example usecases

## Usecase #1: The project with multiple repositories and layers

Imagine the following usecase (with this one I am using some real request from our KDE guys)

The build service provides a KDE4 repository. Which in turn requires the Qt4 repository, because is built on openSUSE 11.0 + the new Qt4 repo.

When looking at this problem, repository dependencies is what comes to head in the first place. But forget about them. If package dependencies are complicated right now, imagine adding a secondary (and duplicated) layer of information. Packages already know their dependencies.

Now imagine our KDE guys can provide an URL, which you add to zypper. And this url returns a dynamic list of repositories. And zypper adds and remove repositories based on the information returned by this url on every refresh.

## Usecase #2: Update repositories based on the customer

This is actually where services where originated. Services were present in Novell ZenWorks. How it works?

The service url nu.novell.com is added to the system. But in this url also a customer id is present as a http username. When you registered, Novell knows the subscription and products this system is linked to and what entitlements the customer has. The service can then return a dynamic list of repositories based on the customer preferences, products and entitlements. The customer does not need to keep them manually in sync.

Now that we don’t have Zenworks in the base system, we still want this cool functionality for our customers, therefore ZYpp now implements services natively.

Technically, this even allows us to offer hotfixes to L3 supported customers on the fly: the system is marked on the server side as being hotfixed, and an extra temporary repository with the PTF (Problem Temporary Fix) packages is added to this system list of repositories.

## Usecase #3: Dynamic repository collections

You are a build service user, and you have an account, and of course you have a list of watched projects you are interested to. What if you could keep your system repositores in sync with your watched project list.

Or what if the build service could offer a service based on keywords or other data: like http://build.opensuse.org/services/mostpopular/repo/repoindex.xml would contain dynamically the 15 most popular repositories. You add that service, and then ZYpp does the work for you of adding new popular repositories, and remove the old ones.

## Usecase #4: Collect openSUSE core repos in one service

Some people prefer to keep and maintain the distros core repos within one local service rather than managing the repos individually. Since version 14.21.0 libzypp supports a simple form of variable replacement within a repoindex.xml file, which makes this even easier.

Since version 17.37.1 the 'mirrorlist', 'metalink' and 'gpgkey' attributes are supported in a repo section.

Since version 17.37.2 '%{url}' is available within a repo tag after the url was defined there.

To use a local RIS service create a directory /somewhere on your disk (or accessible via http, nfs, etc.) and create the file ./repo/repoindex.xml inside. A repoindex.xml for openSUSE Leap 15.1 might look like this:

    <repoindex          ttl="0"
                    disturl="https://download.opensuse.org"
                    distsub="leap/"
                    distver="15.1"
                debugenable="false"
               sourceenable="false">

      <repo           url="%{disturl}/distribution/%{distsub}%{distver}/repo/oss"
               mirrorlist="%{url}/?mirrorlist"
                   gpgkey="%{url}/repodata/repomd.xml.key"
                    alias="repo-oss"
                     name="%{alias} (%{distver})"
                  enabled="true"
              autorefresh="true"/>

      <repo url="%{disturl}/distribution/%{distsub}%{distver}/repo/non-oss"
                    alias="repo-non-oss"
                     name="%{alias} (%{distver})"
                  enabled="true"
              autorefresh="true"/>

      <repo url="%{disturl}/update/%{distsub}%{distver}/oss"
                    alias="update-oss"
                     name="%{alias} (%{distver})"
                  enabled="true"
              autorefresh="true"/>

      <repo url="%{disturl}/update/%{distsub}%{distver}/non-oss"
                    alias="upadte-non-oss"
                     name="%{alias} (%{distver})"
                  enabled="true"
              autorefresh="true"/>

      <repo url="%{disturl}/debug/distribution/%{distsub}%{distver}/repo/oss"
                    alias="debug-oss"
                     name="%{alias} (%{distver})"
                  enabled="%{debugenable}"
              autorefresh="true"/>

      <repo url="%{disturl}/debug/distribution/%{distsub}%{distver}/repo/oss"
                    alias="debug-non-oss"
                     name="%{alias} (%{distver})"
                  enabled="%{debugenable}"
              autorefresh="true"/>

      <repo url="%{disturl}/source/distribution/%{distsub}%{distver}/repo/oss"
                    alias="source-oss"
                     name="%{alias} (%{distver})"
                  enabled="%{sourceenable}"
              autorefresh="true"/>

      <repo url="%{disturl}/source/distribution/%{distsub}%{distver}/repo/non-oss"
                    alias="source-non-oss"
                     name="%{alias} (%{distver})"
                  enabled="%{sourceenable}"
              autorefresh="true"/>
    </repoindex>

%{VAR} always refers to the value defined in the repoindex tag. Reserved names are 'ttl' and 'alias'.

- ttl="SECONDS" defines the matadata's expected time to live in case autorefresh is turned on for the service. It's the number of seconds that should pass before the service is autorefreshed again. It is not available as variable.
- %{alias} is available within a repo tag after the alias was defined there.

Given the file is located at /somewhere/repo/repoindex.xml, add the service by:

    zypper sa /somewhere openSUSE

Manually refresh the service (incl. its repos) with:

    zypper refs -r

Refreshing the service will evaluate the repoindex.xml and adjust the Services repos accordingly. The repos alias will be prefixed by the service name:

    zypper lr
    > #  | Alias                   | Name                  | Enabled | ... | Refresh
    > ---+-------------------------+-----------------------+---------+-...-+--------
    >  9 | openSUSE:debug-non-oss  | debug-non-oss (15.1)  | No      | ... | ----
    > 10 | openSUSE:debug-oss      | debug-oss (15.1)      | No      | ... | ----
    > 11 | openSUSE:repo-non-oss   | repo-non-oss (15.1)   | Yes     | ... | Yes
    > 12 | openSUSE:repo-oss       | repo-oss (15.1)       | Yes     | ... | Yes
    > 13 | openSUSE:source-non-oss | source-non-oss (15.1) | No      | ... | ----
    > 14 | openSUSE:source-oss     | source-oss (15.1)     | No      | ... | ----
    > 15 | openSUSE:upadte-non-oss | upadte-non-oss (15.1) | Yes     | ... | Yes
    > 16 | openSUSE:update-oss     | update-oss (15.1)     | Yes     | ... | Yes

After editing the repoindex.xml you must manually refresh the service to takeover the changes, or you turn on autorefresh for the service (depends on how often you change the content):

    zypper ms -r openSUSE

Rather than hardcoding the distver="15.1", you can also use a repo variable like distver="${releasever}".

# Developers: Implementation

Services are implemented in the following classes:

- zypp::repo::ServiceType (Service types enumeration)
- zypp::parser::ServiceFileReader (.service file parser)
- zypp::ServiceInfo Information about a service (.service file data)
- zypp::repo::PluginServices (Services as plugins)
- zypp::parser::RepoindexFileReader (NU and RIS remote service index XML parser)
- zypp::repo::ServiceRepos (Repositories in a service)