2010 GSoC LSB projects

Go back to the main GSoC Linux Foundation page

Mailing list: lsb-discuss at lists dot linux-foundation dot org

IRC: #lsb on irc.linux-foundation.org

Workgroup Resources

Code License: GPL/BSD, specs: GNU FDL

Mentors: Alexey Khoroshilov (alexey dot khoroshilov at gmail dot com), Jeff Licquia, Denis Silakov (dsilakov at gmail dot com).

Distributing software for Linux is a known pain for many developers who want to provide their products in a compiled (binary) form for all users, not depending on Linux distribution they use. Even if developers are sure that application binaries will work properly on all necessary Linux implementations, they face several problems with proper program installation inside particular systems.

The first problem is a large variety of package managers that exist in the Linux ecosystem. However, package managers in most popular distributions accept packages of either DEB or RPM format; moreover, RPMs can be converted to DEBs using the 'alien' tool, so package formats by themselves is not a critical issue.

A more significant problem is different 'dependency' names used in distributions to designate entities that are actually the same. For example, if someone wants to create an RPM package for application that uses libQtGui, then it will be necessary to create several packages with different dependencies - the one for openSUSE that will depend on ‘libqt4’, another one for Fedora (requiring ‘qt4-x11’), the next one for Mandriva (‘qtguilib’) and so on. In some cases (including our example) a possible approach is to use direct dependencies on libraries, but this works for RPMs only and doesn't cover those dependencies that don't concern libraries.

Public services such as openSUSE Build Farm simplify generation of separate packages for different systems, but developers still have to create separate spec files for every distribution. Another approach is standardization - the LSB specification standardizes some packaging aspects, but the LSB approach is suitable only for LSB-compliant applications; unfortunately, LSB covers only a part of existing libraries and interfaces, and many programs are out of its scope. Other promising approaches such as Berlin Packaging API require support from distribution vendors and haven't got much popularity yet.

Thus, though some approaches and tools exist that help developers to resolve packaging issues, there is no unified and fully automated solution yet, and one of the problem of different dependency names in different distributions remains urgent.

Currently we are working on a web service providing access to a database with information about correspondence between dependency names in different distributions. At the moment, this information can be provided to users as plain text lists via web UI.

The goal of the proposed GSoC project is to develop more complicated tools that will simplify usage of the service by end users. Two tools are to be developed; the first one is a tool for application developers that will automate creation of rpm and deb packages for different distributions on the basis of one package (provided by developer) suitable for one particular system (by repacking files and transforming dependencies). The second tool is for end-users that will allow installation of 'alien' packages (created for other distributions) in user's system, querying the web service for necessary data and resolving transformed dependencies.

Desired knowledge: C/C, packaging basics, rpm and deb formats, Web services. ===LSB test result analytic system=== [[http://ispras.linuxfoundation.org/index.php/About_Distribution_Checker LSB Distribution Checker]] is a user interface front-end, which wraps all the LSB test suites into a user-friendly “single LSB test suite” with unified test execution and analysis capabilities. The checker supports scheduled (e.g. nightly) runs of LSB  tests on multiple test systems with collection of the results on a single server.

An important feature of the Distribution Checker is so called problem_db that is a knowledge base collecting information regarding all known problems detected by LSB tests and defects inside the tests themselves. The problem_db is maintained at the Linux Foundation site and Distribution Checker downloads and applies problem_db generating test result reports.

There is another system that visualize the results collected and facilitate their analysis. However, it has some shortcomings. First of all, it does not allow to reapply updated problem_db to already storing test results. Secondly, URLs of test results are not very suitable to linked from external resources such as LSB Bugzilla. There are some usability issues in user interface as well.

The project is aimed to develop a new test result analytical system that will support hot reapplying problem_db to test results and provide users with  enriched web interface and advanced analytical features. Desired knowledge: Perl, Python, SQL, TET. ===Linux Device Driver Quality Inspector=== Linux device driver program aimed to develop an advanced verification framework detecting typical bugs in Linux device drivers by static source code analysis. A prototype of the framework is almost ready and there are several bugs detected in vanilla kernel drivers already. The goal of this GSoC project is to develop a management system that automatically follows changes introduced into the git repository configured and runs source code analysis for device drivers that has been modified directly or indirectly. Desired knowledge: C, git, Linux device drivers, source code analysis. ===Annotation support for API Sanity Autotest=== It is not a secret that while Linux Ecosystem is rich in applications, many existing programs and libraries lack for tests that would check that they behave as expected in all possible situations, that no regression was introduced with the last modifications, and so on.

Usual reason of lack of tests is lack of resources to create them, and approaches to automate test generation are always welcome. During the LSB tests development, AZOV Framework is used to automate development of shallow-quality tests. With this framework, tests are generated automatically on the basis of data stored in the LSB Database; the core information (e.g., function synopsis) is collected automatically by header analyzers, while additional information (e.g., limitations on function parameter values) should be added manually. A further development of this framework is API Sanity Autotest test generator which can automatically parse header files and automatically generate tests for functions declared there without using intermediate database. This tool is not bound to LSB and can be easily used by any developer. However, tests generated using only information about function declarations are too rough and can't perform any valuable checks. Thus, a way is required for users to provide generator with additional information (similar to AZOV Framework) which will allow generation of more thorough tests.

Different approaches already exist for describing such additional requirements for functions, variables and other program elements. One of the most popular ways is to add 'annotations' (special kinds of comments) directly to the source code - for example, use /*@ null @*/ comment for parameters that are allowed to be NULL. For C and C languages, annotations are usually used to perform static program analysis (see, for example, ACSL specification language or the splint tool).

The goal of this project is to add support for annotations to the API Sanity Autotest - the automated test generator should take such annotations into account and add appropriate runtime checks wherever possible; this should allow to significantly improve quality of the generated tests. First, support for splint annotations should be added; support for more expressive ACSL annotations is an optional task.

Desired knowledge: C and C, Perl, basic knowledge of code parsing techniques. ===Upstream Tracker=== The number of shared libraries in Linux ecosystem is permanently growing up. New versions are released very often, introducing new features and fixing bugs. However, new versions are not always compatible with previous ones (e.g., there can be changes in function behavior, in data type structure, etc.), so applications that use old versions can behave unexpectedly or even fail to run with new ones without recompilation. Information about compatibility between library versions is especially important for distribution maintainers, who should support synchronized set of libraries and applications and ensure that everything works fine in their systems. In particular, it is important to estimate if it is safe to update a particular library and if such an update will require rebuilding of other system components.\\ \\ Several tools exist that can be used to estimate compatibility of different library versions - e.g., icheck or ABI Compliance Checker. However, though these tools are useful when comparing two versions of some library, they provide no way to automate tracking of changes in a huge set of libraries that are used in modern Linux distributions.\\ \\ The goal of this project is to develop a system named "Upstream Tracker" which will provide information about libraries evolution from backward compatibility point of view. The two major components of the system to be developed are like the following: * a monitoring backend that will regularly check for updates of various Linux libraries from LSB Navigator [[http://dev.linuxfoundation.org/navigator/browse/rawlib.php?cmd=display-approved approved list]], download new versions, and build them; * an analytical script that will check backward compatibility of new library versions with previous ones using automated tools (e.g., ABI Compliance Checker). Reports produced by these tools should be combined and transformed into user-friendly HTML (to be published in Web). At the first stage, it will be enough for Upstream Tracker to support x86 platform only, but in future it can be useful to add support for more platforms.

Desired knowledge: C/C, Perl, Web services, knowledge of compatibility basics.