Summary of 24th Quattor Workshop - RAL - 25-27/10/2017
Trying to get the last version of the broker in production
- Reduced size of the database by over 30GB by purging old audit log entries
Releasing a new OpenStack infrastructure
- Need for Debian/Ubuntu-based VMs so trying to sort out the the
- Working on template library changes required for last OpenStack version
- UGent also interested
- Found that running services in fail-over mode was a source of complication
- Deployed IPv6 in production with Quattor, using Pan to calculate an IPv6 address for every IPv4 address.
See HEPiX talk for details.
IPv6 configured and working
Need to adapt the current approach for provisioning resources in the OpenNebula cloud: currently based on AII hooks
No progress with Aquilon: no time to deal with it
No time to work on Aquilon since Annecy, despite a working broker
Main focus in the last months
- OpenStack cloud scale up
- Design/procurement phase for a new distributed Ceph infrastructure
- Fixing of some Quattor configuration modules for EL7
Salt used to manage VM: working on integration with Aquilon broker
Working on replacing previous “dump zone files” approach to managing DNS with a system that pushes changes into DNS in real time using the PowerDNS API.
ncm-systemd: would like to be able to disable everything that is not appearing in the
- Used to do it with
- In fact work in progress by Stijn: see https://github.com/quattor/configuration-modules-core/pull/1119
17.8 out this morning: main change is a almost complete rewrite of
- Requires some careful testing by sites, despite it has been extensively tested
Skip 17.10, next release will be 17.12
- Nothing urgent in the backlog
- Release candidate mid-November, release first week of December
Proposal for next year: one release every 3 months
- Does not prevent additional releases if urgently needed or to release one particular
new feature/change (like the
Discussion of issues marked as
discuss at workshop
- Fix the pan issue #163 (applying defaults in
dict) rather than implement a function in
ncm-chkconfigto work around the problem
- How to manage
xinetservices, something that used to be doable with
ncm-chkconfigand is no longer with
- Write a new component
- Other solution is to reconfigure
xinetto use a non standard directory for services and manage services with
ncm-metaconfigbut seen as a little bit hacky
- Write a new component
- Implement service restart: still the feeling that
ncm-metaconfigcould be a replacement if it (or
ncm-ncd) was able to remove no longer used files
- Agreement to keep this document for the time being and enhance it
- Implement restart as it is done in metaconfig by listing the daemons to restart
Clean up of
noaction handling: Stijn will try to look more closely at what is involved
- Seems a good thing to do if it is not too much effort and has no risk to break less maintained components
Store downloaded profiles in Git
- Looks as a good idea
- No real reason to keep the current directory structure in the Git repository: will allow
to really benefit from the change (remove the need to purge)
- Use commit message to handle the meta-data associated with a revision, like CID
- Need to check what is the impacted code but should be pretty well contained in CCM
- Not used by any standard configuration modules any more
- MS is still relying on it: let’s wait for their green light!
- No drawback to keep it without using it
Support active configuration for plug-ins: still no consensus reached
- Not seen as urgent, keep for another workshop where we have all the experts in the room (Stijn, Gabor)
Implement restart-if-running logic in
CAF::Service: agreement to do it using standard
- Up to the component developer to check that
condrestartis supported by the init script
Support JSON in the API (aka JSON formatter): protobuf v3 would make it easy/easier to support JSON, so probably the right way to go.
Problem doing releases… only release candidate can be done
- Not clear why and what could have changed since 10.3
10.4: already tagged despite the releasing problem
- Do not change it
Already a few things for a next release (10.5)
Dependency management: how to improve the handling of a dependency moved in the load path, currently undetected
- A difficult problem as the dependency is registered at a time the
loadpathis not known
Support for keys starting by a number
- Would be great to have: should be allowed with the back-ends we use nowadays (only
the deprecated XML back-end had a problem, in
panxmlkeys are stored as attributes)
- Interfere with escaping/auto-escaping discussion: to be followed up
Internal implementation of
- Keep the
npush()name for backward compatibility and add
Continue with the same numbering scheme, distinct from Quattor releases: makes sense for the language, quite independent from the other components
Merge configuration-modules-core and configuration-modules-grid: no opposition
- Either create a new repository and merge both to it or merge grid into core
- Keep history using
Merge all template-library repositories into one?
- Current branches per version are confusing for users and adding complexity to the
- Multiple masters
- It is impossible to open issues against a specific branch: only against a repository
- Contributing patches to several branches requires several pull-requests when one PR could update several directories
- Often one PR in one repository depends on another PR in another one and this cannot be well expressed
- Start by template-library-core and -standard
- The new repository top-level directories will reflect where the templates are coming from: standard, core…
Merge CAF and LC repositories?
- LC is now sort of a “private code” in the way we use it and it is becoming an internal tool used by CAF, mainly
- Even consider merging all the Perl repository into one?
Open question: how to move the open issues or to make the existing repositories read-only?
Note: Two weeks after the workshop GitHub added functionality for archiving repositories which solves the problem of preserving old repositories in a read-only state.
Quattor and SELinux (Michel)
- SELinux has come a long way. Now supports permissive mode which makes things work just as they were unless you explicitly enable it (e.g. can enable it to prevent
httpddoing evil things without effecting rest of box). Also get some audit logging by default.
- Permissive mode does not work if
SELinux=disabledis set which is what we all do today
- Most important change in Quattor is to ensure we are preserving the appropriate context for files and directories managed by Quattor.
restoreconafter modifying a file (https://github.com/quattor/CAF/blob/master/src/main/perl/FileWriter.pm#L31)
- Proposed creating a new component (
ncm-selinux) to allow managing SE Linux enforcement, managing properties of policies but not full policies which are very complex (they could be downloaded with
ncm-downloador packaged as RPMs).
A collection of scripts to allow OpenStack VM images to be automatically built based on personalities from Aquilon. It uses Packer (https://www.packer.io) to do the image building. Open sourced python scripts: https://github.com/stfc/packuilon
Depends on RabbitMQ but then so does OpenStack so you have one running anyway. Receives servers notifications from Aquilon.
Installation instructions include
aq bind server which invokes a Quattor template: that template just installs some RPMs (packer and what’s in the git repository).
Aquilon GitHub Repository
All issues were reviewed and a number of older ones found to be already fixed. Gintare worked on validating the install process on RH7.
All open pull-requests with no milestone were reviewed. All pull-requests without external dependencies were assigned to 17.12 which has made the release quite large now. Some pull-requests are ready for merging now 17.8 is out, these should be handled by the weekly stand ups.