Faf
- Status: in development, unstable
- Programming language: Python
- Operating system: RHEL ≥ 6 or Fedora ≥ 15
- Dependencies:
Faf is a programmable platform for analysis of packages, packaging
issues, bug reports and other artifacts produced within Fedora
operating system development. It's written and interfaced in Python
with small bits in C. Top-level functionality is exposed via command
line tools.
It can be obtained from
its Fedora Hosted project
page.
Faf exists to hidden unnecessary complexities of Fedora operating
system development behind several useful abstractions. These
abstractions already enabled the possibility to develop useful
services that would not exist without them. Here are its unique
selling points for operating system analysis:
- Bugzilla bugs, attachments, users are available instantly and
with 100% reliability. Without this abstraction, people used
Bugzilla's XML-RPC to get only the bugs they needed. However, the
XML-RPC interface is problematic:
- it sometimes returns errors as answers to reasonable requests
- sometimes it is not accessible
- sometimes a new bug is introduced server-side
- various data can be fetched via multiple sub-interfaces, but the
returned data differ in subtle ways (e.g. precision and format of
dates/timestamps returned), and it takes time to find the best way in
Bugzilla's source code
- sometimes you must use a combination of interfaces to get all what
you need
- round-trip time for every operation ranges from 700 milliseconds
to several seconds, so any attempt to develop nontrivial code working
across many bugs becomes uneconomical, as every re-run takes hours or
days
- Build metadata, build logs, RPMs, source RPMs, tags are
available instantly and with 100% reliability. The advantages of
this are similar to the Bugzilla point. Koji-specific points:
- RPMs are huge (tens of gigabytes). Downloading them several times
is slowed down either by internet connection speed or by local network
administrator.
- Koji interface appears bug-free and nonchanging, but it is
sometimes exceptionally slow in providing the RPMs.
- Old versions of packages are available. When developing
some script that checks some attributes of packages, it often takes
several weeks and many re-runs of the script to cover all package side
cases correctly. When this development is done on repositories or
development snapshots, many packages are updated to newer versions
over time. This makes reproducing some issues that occurred on certain
package in past too difficult. It is no longer possible to build a
list of script issues (bugs) to be postponed and solved later, because
the issues become unreproducible over time. Also, a new realm of
possibilities appear when you take the possibility to examine and
compare several versions of packages for granted.
- Data from various sources are accessible in an unified way.
All data in Faf are stored in local cache, which supports both binary
(for RPMs) and textual entries. Textual entries are stored in a
normalized format, that makes it easy to create entries manually, view
them in a text editor, search through them using
grep
— this is vital for problem-solving and development of new
complex analysis scripts. Selected parts of textual entries are also
stored in an SQLite database for quick searches and access. All the
data is accessed the same way, be it objects downloaded from Koji,
Bugzilla, Package Database, reports generated by various checkers, or
intermediate representation of backtraces useful for backtrace
comparsion. Extending Faf to store new type of data in the local cache
as simple as adding single Python file with data description to the
source code (existing data description files can be used as a
template).
- Functionality is exposed as both Python library and
command-line interface. Command line is great for inspection,
prototyping, experiments, bugfixing. Python interface enables tighter
integration and higher performance.
Provided data
Faf provides the following data locally:
- Red Hat Bugzilla:
- bugs
- bug comments
- bug attachments
- users
- Fedora Koji:
- tags
- packages
- builds
- build logs
- rpms
- rpm metadata
- Fedora Package Database:
Provided functionality
Faf provides the following functionality:
- RPMs are easily unpacked and their content analyzed.
- Chroot environment for package rebuilds is easily prepared,
packages can be recompiled, their build roots analyzed.
- Package dependency resolver
- package debuginfo checker
- Statistics scripts
- Backtrace deduplication for ABRT bugs from Bugzilla
Faf tools that pull data from external sources to local cache:
- faf-koji-pull-builds-fedora
- Retrieves builds and build logs from the Fedora Koji to the local cache.
- faf-koji-pull-rpms
- Retrieve rpms from the Fedora Koji for builds in the local cache.
- faf-koji-pull-tags
- Retrieve build tags from the Fedora Koji.
- faf-fedora-pkgdb-pull-collections
- Retrieve package collections from the Fedora Package Database.
- faf-fedora-pkgdb-pull-packages
- Retrieve package metadata from the Fedora Package Database.
- faf-bugzilla-pull-bugs-abrt
- Retrieve ABRT bugs from the Red Hat Bugzilla.
- faf-bugzilla-pull-attachments
- Retrieve attachments from the Red Hat Bugzilla for the bug
reports stored in the local cache.
- faf-bugzilla-pull-comments
- Retrieve comments from the Red Hat Bugzilla for the bug reports
stored in the local cache.
- faf-bugzilla-pull-users
- Retrieve users from the Red Hat Bugzilla for the bug reports
stored in the local cache.
Faf tools that can manually query external sources:
- faf-koji
- Query a Koji instance for build tags, builds, and RPMs.
- faf-fedora-pkgdb
- Query Fedora Package Database for package collections and
package metadata.
- faf-bugzilla
- Connect to the Red Hat Bugzilla and perform bug, comment,
attachment, and user queries.
Faf tools that analyze ABRT bugs and handle the discovered
issues:
- faf-abrt-backtraces
- faf-abrt-check-reports
- Check ABRT-reported bugs from Red Hat Bugzilla to find various
issues with ABRT. Discovers low quality and unparsable backtraces,
missing or invalid duplicate duplicate hashes.
- faf-abrt-dup-bugs
- Find ABRT bug reports with very similar backtraces.
- faf-abrt-dup-improvements
Faf tools that analyze debuginfo packages and report issues:
- faf-debuginfo-analyze-builds
- Analyze all builds and store discovered issues to the local cache.
- faf-debuginfo-analyze-build
- Analyze single build for debuginfo problems and store discovered
issues to the local cache.
- faf-debuginfo-build-debugsources
- faf-debuginfo-report-bugzilla
- Report issues found by faf-debuginfo-analyze-build to the Red
Hat Bugzilla.
- faf-debuginfo-report-log
- Create a logfile containing all the issues found by
faf-debuginfo-analyze-build. The issues are taken from the local
cache. Use faf-debuginfo-analyze-latest-builds to fill the cache
with issues.
Faf tools that generate statistics and graphs from local
storage:
- faf-stats-abrt-bugs-closed
- Generate a report about ABRT bugs closed (resolved) in various
ways in Red Hat Bugzilla over time.
- faf-stats-abrt-bugs-opened
- Generate a report about new ABRT bugs being opened in Red Hat
Bugzilla over time.
- faf-stats-abrt-bugs-remaining-opened
- Generate a report about the ABRT bugs remaining opened in Red
Hat Bugzilla over time.
- faf-stats-abrt-crashes
- Generate a report about the number of reported ABRT crashes
compared to the number of bugs in Red Hat Bugzilla. Several
crashes from various people might be reported in a single
bug.
- faf-stats-abrt-development
- faf-stats-abrt-users
- Generate a report containing the number of Fedora users using
ABRT to report problems.
Other tools:
- faf-cache
- Manage and investigate data in local cache.
- faf-chroot
Use cases
Provide statistics about Bugzilla activity
# Download bugs separately because Bugzilla allows to fetch only max. 100000 bugs at once.
# Add -vvv option to see verbose output.
for version in 14, 15, 16, rawhide
do
faf-bugzilla-pull-bugs --product fedora --product-version $version \
--with-comments --with-attachments -vvv
done
faf-bugzilla-pull-users -vvv
faf-stats-bugs-closed
faf-stats-bugs-opened
faf-stats-bugs-remaining-opened
faf-stats-users
faf-stats-abrt-crashes
This use case shows how some attributes of operating system
packages can be improved by using Faf.
faf-koji-pull-tags
faf-koji-pull-pull-builds fedora --tag dist-rawhide --only-missing --no-build-logs
faf-koji-pull-rpms fedora --only-missing --skip-i686
faf-debuginfo-build-debugsources --only-missing
faf-debuginfo-analyze-builds
faf-debuginfo-report-log
Improve quality of coredump-level duplicate detection
This use case shows how some algorithms can be fine-tuned by
testing them on whole operating system by using Faf.
faf-koji-pull-tags
for tag in dist-f15, f16, f17, dist-rawhide
do
faf-koji-pull-pull-builds fedora --tag $tag --count 2 --only-missing --no-build-logs
done
faf-koji-pull-rpms fedora --only-missing --skip-srpm --skip-i686 --skip-noarch
Backtrace deduplication server
for version in 14, 15, 16, rawhide
do
faf-bugzilla-pull-bugs --abrt --product fedora --product-version $version \
--with-comments --with-attachments
done
faf-btserver-optimize-backtraces
faf-btserver-cgi
faf-btserver-analyze-backtraces
faf-btserver-report-bugzilla
LLVM IR service
faf-koji-pull-tags
faf-koji-pull-pull-builds fedora --tag dist-rawhide --only-missing --no-build-logs
faf-koji-pull-rpms fedora --only-missing --skip-i686
faf-llvmir-build --only-missing
faf-llvmir-cgi
Wild ideas
Requires check
- check that RPM provides all their libraries
- check that RPM requires all what its binaries and libraries
provide
- check that what RPM requires is available in the repository
- check that replacing older RPM with the new one did not affect
Requires or BuildRequires of some other RPM
- check that what RPM BuildRequires is available in the repository
- check that two RPMs do not provide the same item
RPM update check
- Check if a patch is added to SRPM and it is also added to the
spec file, but is not applied
Manpages check
- See rpmlint
- Check via hunspell (custom dictionary needed)
- Check using option comparator
- Check manpage-binary package dependencies
- Check config file paths in manpages (/etc/...), they might be
wrong.
Paths check
- check path collisions
- check file permissions in directories with known purpose
- see http://fedoraproject.org/wiki/Packaging:Conflicts
- filenames that contain non-ASCII characters must be encoded as UTF-8.
- symlinks does not point to file in package which is not installed
Parse build logs
Parse important GCC warnings.
Scan build logs for "operation not permitted" and other failures
"operation not permitted" is shown in Emacs.
Security
http://people.redhat.com/sgrubb/files/rpm-chksec
Package review
- The spec file name must match the base package %@{name@}, in the
format %@{name@}.spec unless your package has an exemption.
- The package must be licensed with a Fedora approved license and meet
the Licensing Guidelines. (check for disapproved licences)
- The License field in the package spec file must match the actual
license. (check the COPYING file etc.)
- If (and only if) the source package includes the text of the
license(s) in its own file, then that file, containing the text of
the license(s) for the package must be included in %doc.
- The sources used to build the package must match the upstream
source, as provided in the spec URL. Reviewers should use md5sum for
this task. If no upstream URL can be specified for this package,
please see the Source URL Guidelines for how to deal with this.
- The spec file MUST handle locales properly. This is done by using
the %find_lang macro. Using %@{_datadir@}/locale/* is strictly
forbidden.
- Every binary RPM package (or subpackage) which stores shared library
files (not just symlinks) in any of the dynamic linker's default
paths, must call ldconfig in %post and %postun.
- Packages must NOT bundle copies of system libraries.
- A package must own all directories that it creates. If it does not
create a directory that it uses, then it should require a package
which does create that directory.
- A Fedora package must not list a file more than once in the spec
file's %files listings. (Notable exception: license texts in
specific situations)
- Permissions on files must be set properly. Executables should be set
with executable permissions, for example. Every %files section must
include a %defattr(...) line
- If a package includes something as %doc, it must not affect the
runtime of the application. To summarize: If it is in %doc, the
program must run properly if it is not present.
- Header files must be in a -devel package.
- Static libraries must be in a -static package
- If a package contains library files with a suffix
(e.g. libfoo.so.1.1), then library files that end in .so (without
suffix) must go in a -devel package.
- In the vast majority of cases, devel packages must require the base
package using a fully versioned dependency: Requires:
%@{name@}%@{?_isa@} = %@{version@}-%@{release@}
- Packages must NOT contain any .la libtool archives, these must be
removed in the spec if they are built.
- Packages containing GUI applications must include a %@{name@}.desktop
file, and that file must be properly installed with
desktop-file-install in the %install section. If you feel that your
packaged GUI application does not need a .desktop file, you must put
a comment in the spec file with your explanation.
- Packages must not own files or directories already owned by other
packages. The rule of thumb here is that the first package to be
installed should own the files or directories that other packages
may rely upon. This means, for example, that no package in Fedora
should ever share ownership with any of the files or directories
owned by the filesystem or man package. If you feel that you have a
good reason to own a file or directory that another package owns,
then please present that at package review time.
- All filenames in rpm packages must be valid UTF-8.
- Usually, subpackages other than devel should require the base
package using a fully versioned dependency.
- The placement of pkgconfig(.pc) files depends on their usecase, and
this is usually for development purposes, so should be placed in a
-devel pkg. A reasonable exception is that the main pkg itself is a
devel tool not installed in a user runtime, e.g. gcc or gdb.
- all Fedora packages must be named using only the following ASCII
characters. These characters are displayed here:
abcdefghijklmnopqrstuvwxyz ABCDEFGHIJKLMNOPQRSTUVWXYZ 0123456789-._+
- Conflicting package names, even if they differ by case alone, are
not allowed.
- No inclusion of pre-built binaries or libraries. Remove all
pre-built program binaries and program libraries in %prep prior to
the building of the package. Examples include, but are not limited
to, *.class, *.dll, *.DS_Store, *.exe, *.jar, *.o, *.pyc, *.pyo,
*.egg, *.so files.
- Binaries placed into /bin and /sbin must not depend upon libraries
stored in /usr/lib (or /usr/lib64). Binaries which depend on
libraries in /usr/lib* must live in either /usr/bin or /usr/sbin.
- Rpmlint errors
- The Packager tag should not be used in spec files.
- The Vendor tag should not be used.
- The Copyright tag is deprecated. Use the License tag instead.
- The Summary tag value should not end in a period.
- Usually, the PreReq tag should be replaced by plain Requires.
- The Source tag documents where to find the upstream sources for the
rpm. In most cases this should be a complete URL to the upstream
tarball. For special cases, please see the Packaging:SourceURL
Guidelines
- Be very careful to watch especially the configure part of the build
process for missing optional libraries or tools that are desirable
for the package. (check for .pc file checks in configure)
- Files marked as documentation must not cause the package to pull in
more dependencies than it would without the documentation. One
simple way to ensure this is to remove all executable permissions
from %doc files (chmod -x).
- Sometimes, code will hardcode specific library paths when linking
binaries (using the -rpath or -R flag). This is commonly referred to
as an rpath. Normally, the dynamic linker and loader (ld.so) resolve
the executable's dependencies on shared libraries and load what is
required. However, when -rpath or -R is used, the location
information is then hardcoded into the binary and is examined by
ld.so in the beginning of the execution. Since the Linux dynamic
linker is usually smarter than a hardcoded path, we usually do not
permit the use of rpath in Fedora. There is a tool called
check-rpaths which is included in the rpmdevtools package. Any rpath
flagged by check-rpaths MUST be removed.
- When a program installs internal libraries they are often not
installed in the system path. These internal libraries are only used
for the programs that are present in the package (for example, to
factor out code that's common to the executables). These libraries
are not intended for use outside of the package. When this occurs,
it is acceptable for the programs within the package to use an rpath
to find these libraries.
- Don't use %config or %config(noreplace) under /usr. /usr is deemed
to not contain configuration files in Fedora.
- .desktop file correct usage, syntax, contents
- MUST run desktop-file-install OR desktop-file-validate in %install
(and have BuildRequires: desktop-file-utils), to help ensure
.desktop file safety and spec-compliance. desktop-file-install MUST
be used if the package does not install the file or there are
changes desired to the .desktop file (such as add/removing
categories, etc). desktop-file-validate MAY be used instead if the
.desktop file's content/location does not need modification.
- Fedora's RPM includes a %makeinstall macro but it must NOT be used
when make install DESTDIR=%@{buildroot@} works.
- All macros in Summary: and %description need to be expandable at
srpm buildtime. Because SRPMs are built without the package's
BuildRequires installed, depending on macros defined outside of the
spec file can easily lead to the unexpanded macros showing up in the
built SRPM.
- Packages which use files itemized as Source# files, must refer to
those files by their Source# macro name, and must not use
$RPM_SOURCE_DIR or %@{sourcedir@} to refer to those files.
- If the package includes translations, add BuildRequires: gettext
- Scriplets are only allowed to write in certain directories. Build
scripts of packages (%prep, %build, %install, %check and %clean) may
only alter files (create, modify, delete) under %@{buildroot@},
%@{_builddir@} and valid temporary locations like /tmp, /var/tmp (or
$TMPDIR or %@{_tmppath@} as set by the rpmbuild process)
according to the matrix on wiki page
- Note that the %pretrans scriptlet will, in the particular case of
system installation, run before anything at all has been
installed. This implies that it cannot have any dependencies at
all. For this reason, %pretrans is best avoided, but if you must use
it, the script must be written in Lua.
- Web applications packaged in Fedora should put their content into
/usr/share/%@{name@} and NOT into /var/www/.
- The "alternatives" tool provides a means for parallel installation
of packages which provide the same functionality by maintaining sets
of symlinks. Check correct usage.
- Fonts in general-purpose formats such as Type1, OpenType TT (TTF) or
OpenType CFF (OTF) are subject to specific packaging guidelines (1),
and should never be packaged in a private application directory
instead of the system-wide font repositories.
- Some applications have specific guidelines written for them, located
on their own pages in the Packaging: Namespace.
Detect arch packages which should be noarch
Generate -ast packages (abstract syntax tree)
SELinux policy checker
Jak se zmenili labels u baliku (rpmdiff) pri posledni policy, kdyz
se napr. nejaky soubor presunul.
Otestovat jake syscally pouziva binarka (pomoci DWARF), a porovnat
s tim jake syscally ma povolene v policy. -- to nestaci, jsou
potreba resit parametry syscallu.
Check `yum whatprovides *lint` for fools to be run on packages
xmllint, pylint, rpmlint, eu-elflint, eu-dwarflint
Check that package has proper dependencies
They depend on binaries, so libs, spec file contents and
policy. Policy example: emacs-el packages should depend on exact
version-release of main package.
Special checks for languages
- ruby (rubygems compatibility etc.)
- python (eg. imports should exist in requires)
- java
Check that package is compiled with Fedora's GCC options (stack protector etc.).
Licensing
Licensing dependencies (GPLv2 depending on GPLv3+ library) and
license of individual files. Also subpackages not depending on the
base package containing the license should include the license file
themselves.
Emacs
This site has duplicate Lisp libraries with the same name. If a
locally-installed Lisp library overrides a library in the Emacs
release, that can cause trouble, and you should probably remove the
locally-installed version unless you know what you are doing.
/usr/share/emacs/site-lisp/flim/sasl-digest hides
/home/karel/devel/hatchery/emacs/emacs-24.0.50/lisp/net/sasl-digest
ABI compliance checker
http://forge.ispras.ru/projects/abi-compliance-checker
Wishlist
- Stastistics scripts cleanup
- Rename
faf-stats-abrt-* to faf-stats-*,
and add the --abrt option. Add option to select the
gnuplot output format (svg, png, pdf).
- Extend
cache --verify to check database
- It should check that database data match the files, and no additional data is present, and no data is missing.
- Add
--tag option to faf-koji-pull-builds
- Currently the tag is hard-coded to dist-rawhide and RHEL-6.2.
- Add
--no-build-logs option to faf-koji-pull-builds
- Add
--count x option to faf-koji-pull-builds
Homepage