16th Quattor Workshop Summary (October 2013)

30 September 2013
Michel Jouvin

Quattor Workshop - CNAF (Bologna) - 30/9/2013-2/10/2013


Project Status - L. Munoz

Survey ran before the workshop

  • Significant improvement compared to previous ones
  • QWG : many were not able to move to QWG or forked a long time ago. We need to think more heavily at it.
  • Aquilon adoption making progress (2-3 sites)
  • Community support is perceived as good
  • Documentation: still the weak point, were not able to make significant progresses
  • LAL wiki: certificate problem still perceived as a barrier, a developer only resource

Core modules

Configuration Modules - L. Munoz


  • Count : nine releases, 3 of them tagged as stable
  • New releases are not mandatory
  • Need a releaser manager backup for Luis

Config modules: main activity is on the core components

  • Not much on grid components or on daemons
  • ncm-openldap and ncm-cups were resurrected

New features

  • Preliminary support for SSSD in authconfig
  • YUM-based SPMA ready for production
  • Hooks for ncm-ncd before and after execution of components

Metaconfig stuff

  • Ugent has schemas and templates for many services: would like to share them
  • But we can hardly promise backward compatibility

Work in progress

  • Policy-based component dispatching in ncm-ncd
  • Solaris port
  • ncm-network: still not much progress

ncm-ncd hooks and policy-based component scheduling

See [/wiki/Development/PrePostActionsNCMNCD proposal].

Goal: execute some arbitrary actions before running all components and after.

  • These actions will have no access to node profile
  • Specified by run-init and run-finalizein /etc/ncm-ncd.conf
  • An error during run-init will prevent executing components but run-finalize will run

Hooks may be used to implement site-specific policy for component scheduling, allowing to filter out some components from the initial list based on some local criteria

  • But what to do with dependencies if one is filtered out? Can be very complicated
  • Probably better/enough to implement a “everything or nothing” strategy: at least use case can be cleanly defined

Some use cases for executing only part of the configuration changes for a component. Proposal:

  • Split component configuration into 2 different parts/subtree, e.g. /software/components/filecopy and /software/components/filecopy-magic
  • Bind /software/components/filecopy-magic to type_filecopy
  • Define a component alias name in filecopy-magic saying it will use component filecopy to execute the action
  • New field in deps structure
  • Filter out filecopy-magic in run-init hook
  • See https://github.com/quattor/configuration-modules-core/issues/112 for discussion/implementation follow-up

CDBSQL - Z. Sebestyen

Used at MS to produce/update a SQLite db every 2 hours with the AQDB contents: used by users to query configuration rather than overloading the broker.

  • Reused an older API from MS to query the DB

Code will be pushed to SF at some point

  • Still some concerns with pushing directly to GitHub

Still need to review if it could be integrated with QuatView to replace the internal SQL generator in QuatView and deliver a generic web front-end.

Also, may look at using a NoSQL db rather than SQLite.

  • May give some perf advantage for querying the db but may a price for the update

Maker Quattor Easier

Introduction - D. Zilaskos

Painful path for a new Quattor user

  • Go to website and click download: not exactly what was being looked for…
  • Try documentation: suggest to go to overview… just marketing blurb…

Should not dream of producing a polished documentation as we failed for so long… but we can make things much easier if not very easy. Goal should be to install a workable Quattor installation for testing in 30 mn!

  • Install from Quattor YUM repo
  • yum install quattor-server
  • yum install quattor-client?
  • Sample templates usable with minimum editing (host name, mac address) to install/bootstrap a first machine
  • For the first step, could be take control of already existing machines without doing the initial installation to avoid AII/kickstart difficulties
  • Panc book: improve the tutorial part

Documentation may be organized in 2 different parts:

  • Getting started: build/describe an optimized procedure for a user to start with Quattor both for evaluating AND setting up a production infrastructure (could then be used to produce an appliance)
  • Prerequisite hardware/site config
  • Install server: prefer Aquilon if mature enough. We don’t want to attract new sites to SCDB.
  • Take control of a client (already installed): should include a basic configuration description that is as less intrusive as possible for the existing configuration
  • Mini-demonstration illustrating how to change some service configuration, for example ssh keys, account creations, add a RPM
  • Next steps
  • How to find docs and examples
  • Initial installation: AII/kickstart customization
  • Security
  • Customizing node configurations: common services and recipes
  • Document how to execute an arbitrary command with existing config modules like filecopy or metaconfig

Put documentation on GitHub: start with a non public (review) section

  • Review how to improve the documentation structure on gitHub: see documentation discussion later during the workshop

Documentation: activate robot indexing for Quattor wiki hosted at LAL


  • Getting Started done at next workshop (April 2014)
  • Next steps after

Encourage adoption of new YUM-based ncm-spma: make it clear that YUM is used in a completely standard way and that people may/must refer to YUM docs for details on advance YUM management (like mirroring).

QWG Status - G. Philippon

Recent changes

  • No more release for more than 1 year
  • No update to branches: all changes in the trunk
  • standard/ almost stable: most things are in fact imported from other places

Current trunk status

  • Partly based on Quattor 13.1: most components not yet updated (validation required)
  • Support SL5 and SL6

3 different subsets shipped together

  • “Standard” templates
  • Provide a core node type without any grid dependency
  • Should be really standard: can it be the case for a Web server?
  • Grid templates
  • Templates generated from other Quattor components like AII or config modules

StratusLab templates delivered separately (git repo)

  • Support both ncm-spma v2 and v3

Work in progress

  • Remove templates related other Quattor components from standard/
  • Create a new namespace quattor/ with one sub-namespace per Quattor version: easier to move from one version to another one
  • YUM-based ncm-spma
  • Move to GitHub: do we keep the history?

To be done but not yet handled by lack of time

  • Namespace structure of OS templates
  • Integration of Aquilon
  • NFS configuration independent of grid templates

GitHub migration based on several repositories, no history kept

  • templ-library-standard: standard/ without the stuff imported from other places and without monitoring
  • monitoring not migrated
  • templ-library-core: templates imported from other Quattor components (config modules, AII, schema, Pan types…). Generated by the release process ideally, else some SCDB scripts may help (e.g. updateComponents)
  • master branch + 13.1 branch (ncm-spma v2)
  • Add standard/pan and standard/quattor other than AII
  • templ-library-grid: grid templates
  • 1 branch per major grid MW version
  • templ-library-stratuslab: current strastuslab repository
  • templ-library-examples: current clusters/ and sites in SVN
  • Minor issue: how to keep a “unified view” of these separate repositories for pushing back from a unique place like in the past
  • git submodule are not appropriate for this
  • git subtree: requires additional options when pushing but may be explored
  • Luis will try to find some examples

Core Quattor Schema

standard/pan and standard/quattor general templates were not migrated from SF SVN: need to be added to template-library-core

Some suff added/changed for Aquilon: need to merge asap

  • Mostly additions but a few mods (network part in particular)



  • Getting Started section on quattor.org
  • Allow indexing of Trac wiki by robots
  • Move to quattor.org parts of general interest on QWG wiki

Configuration modules

  • Implement ncm-ncd hooks
  • Add support for component aliasing