« Choosing an insanely small computer | Main | Ruby on rails, Zope on a rope »

ZCML needs to do less

The separation of component wiring from the actual component code is a strong theme in Zope 3. To do the wiring, it was decided very early on not to use Python but something more restrictive. Extensible, but restrictive in terms of its abilities. The result was a configuration engine that allows component configuration and registration ("the wiring") through simple configuration files. The only backend right now, ZCML, uses an XML syntax for that.

Common criticism

Not everyone has been happy with how ZCML turned out. From what I've seen, the common criticism is two-fold:

The Python purists usually complain that the XML is ugly, that it's hard to read, hard to write, just awkward for Python programmers, and so on. They have some points there, but it's really hard to please a Python programmer in aesthetics. (It's all Guido's fault, really. The Perl folk will put up with almost anything!) I'll therefore not get into this debate, at least not now.

I'm speculating that the die-hard Zope 2 coders probably don't worry too much about the aesthetics (It's not like Zope 2 code ever looked good anyways...). According to my observations, their major complaint is the indirection ZCML introduces. The fact that component registration is needed, separated from the code, isn't as much as a problem as the fact that many important component issues lie in ZCML and not in Python. That makes reading and therefore understanding, maintaining and debugging code harder.

Policy vs. automation and shifting the focus of ZCML

I think the latter criticism has a point. Right now, ZCML is used for both policy and automation. I have come to believe that ZCML's focus should only be policy because automation is better done in Python.

For example, I think ZCML should try to do much less on-the-fly construction of objects, let alone whole classes. Ideally, things that are registered through ZCML are importable somewhere from Python (I realize that in reality it would be a bit difficult to do everywhere). Basically, ZCML directives should become mere on/off switches which is what 90% of policy is about: Switching on a certain component in favour of another one. These sort of on/off switches are typically ZCML one-liners, such as:

<adapter factory=".foo.BarFooAdapter" />
<utility factory=".foo.FooUtility" />

The rest of the information (what is adapted and what is provided) doesn't concern ZCML as much as it concerns the Python code, which is where one has to deal with the adapted API and the to-be-provided API after all:

class BarFooAdapter(object):
    adapts(IBar)
    implements(IFoo)
    ... # deal with IBar objects here and provide IFoo API

class FooUtility(object):
    implements(IFoo)
    ... # provide IFoo API here

The good news is that the above is already possible. In my opinion, it should become the preferred way of doings things (perhaps even the only one).

Less magic in ZCML, more explicitness in Python

A good example of what can go wrong in a ZCML directive are browser pages. The on-the-fly creation of classes there definitely needs to end since the extent to how much magic this entails is just ridiculous. It was also awfully painful when we started bringing ZCML to Zope 2 via Five (and it continues to be painful!). Introducing some explicitness will only cost a few lines more Python code, though (optional) base classes should help a great deal there. After all, this kind of automation is what base classes are very good for, even in the Component Architecture!

Another aspect are the names of browser pages. Unlike named utilities, which don't care about the name they're registered with, it does matter to browser pages what name they have (if not to themselves, then to other browser page "siblings"). Imagine the following (over-simplified) example:

class FooPages(BrowserView):

    def form(self):
        return u'<html>...<form action="@@update.html">...</form>...</html>'

    def update(self, data):
        self.context.data = data
        self.request.response.redirect('@@index.html')

    def index(self):
        return u'<html>...</html>'

How do you know that @@update.html actually refers to the update method and that @@index.html refers to the index method here? You don't by looking at the Python code, even though that's where this information would be quite useful. Given Python 2.4's decorator abilities, we can provide such a feature in a nice syntactic form, for example:

class FooPages(Pages):
    adapts(IFoo, IBrowserRequest)

    @page(u'edit.html')
    def form(self):
        return u'<html>...<form action="@@update.html">...</form>...</html>'

    @page(u'update.html')
    def update(self, data):
        self.context.data = data
        self.request.response.redirect('@@index.html')

    @page(u'index.html')
    def index(self):
        return u'<html>...</html>'

The registration would look like that:

<browser:pages class=".browser.FooPages">
  <require permission="zope.View" attributes="index" />
  <require permission="zope.ManageContent" attributes="form update" />
</browser:pages>

Note that we still need and want the security here. Policy is not always about on/off. Security in Zope's understanding, for example, is application policy and therefore doesn't belong in Python. I would say that security is the only big exception to the on/off rule and it's a traditionally very import one in Zope.

Also note how this fictitious directive above has a usage similar to the content directive (regarding the require subdirective). Repeating existing patterns where possible lowers the barrier tremendously, which brings us right to the next topic:

Reducing directive and directive functionality proliferation

Following the ZCML-should-not-create-things-on-the-fly paradigm, we can also get rid of some ZCML directives or directive functionality easily and thus reduce the directive proliferation that has happened over the years. As someone who has written a book and given trainings on Zope 3, I can't tell you how important that is from both a teaching and learning point of view. The more people can reuse what they've learned already when learning about new things, the easier it is for them. Having to write a line more here and there for explicitness' sake is small price to pay in return.

In a current proposal of mine, I already suggest to get rid of browser:layer and browser:skin directives. They were quite a low-hanging fruit, given the simplifications that the skinning system had already seen previously. Half of this proposal is actually about passing on that simplification from under the hood on to the developer.

Here are some other directives and some directive functionality that are on my preliminary hitlist:

factory: This is just a shortcut to the utility directive and saves you one line in ZCML and two lines in Python (for the title and description of the factory). I think it's a goner.
vocabulary: This directive registers vocabularies. Actually, it doesn't register vocabularies but vocabulary factories (because vocabularies are content dependent and might need to be initialized with more than just the context). It is probably one of the most magical ZCML directives of all:

First, it is the only directive to take arbitrary arguments and by this it defies one of the initial aspects of ZCML (which is being restricted to a certain set of directives and parameters). Second, it creates a magical wrapper around vocabularies so that we can register them as vocabulary factories. All of this isn't needed at all, people can deal with these things much better in Python code, e.g.:
```
class MyVocabulary(object):
    zope.interface.implements(IVocabulary)
    zope.interface.classProvides(IVocabularyFactory)

    # the existence of this method satisfies IVocabularyFactory
    def __init__(self, context, **kw):
        ...
```
Now, this class simply gets registered as utility providing IVocuablaryFactory. Voila, one directive gunned down at the expense of only one extra line in Python.
modulealias: This directive can put Python modules into sys.modules under a new alias, typically to preserve backward compatability for old pickles in ZODB. I have strong doubts that this has anything to do with registering and/or configuring components at all, let alone the fact that I feel a bit uncomfortable with putting something like that in ZCML in the first place. If you want to do a sys.modules hack, put it in Python with a proper BBB comment and be done with it, I say.
renderer:renderer: Since the major simplification of the Component Architecture, this is probably one of the most useless directives of all times. It simply registers an adapter and does nothing else. Using the adapter directive here would not even cost you any extra lines.
rdb:provideConnection: This is just a shortcut to the utility directive and saves you exactly one line of ZCML. I call for R.I.P.
dav:provideInterface: This is just a shortcut to a call to the interface directive (with an appropriate type argument) and saves one line in ZCML as well.
xmlrpc:view: I'm certain that, given the correct the usage of decent base classes, this directive could be replaced with the simple view directive.
Many directives of the browser namespace support the registration of menu items in addition to registering the component in question. Though I see that this is useful because it might save some typing, I weigh keeping directives as simple as possible (on/off switches!) higher. I've seen people being intimated by the length of some of the browser directives (such as browser:editform); by taking the menu functionality out, we can reduce many directives by two lines making them easier to understand by themselves (of course, we'll have to add a whole new directive, but it'll only be 3 or so lines).
browser:addview: This is a shortcut to the browser:view directive and saves you one line of ZCML. R.I.P. in the grave next to rdb:provideConnection and dav:provideInterface.
browser:localUtility: This is just a short-cut around the content directive and letting your class implement IAttributeAnnotatable and ILocalUtility. So it saves two lines of Python code, how great.
browser:containerViews: This is a shortcut to a bunch of things, so it actually saves more than just a few lines. Though I'm sure that given the right approach, we can even do with out this directive as well, I just need to think about it a bit longer.
browser:addform, browser:editform, browser:schemadisplay: If the form views these directives register were defined in Python (which would make the overriding of widgets much easier, too), a plain boring browser:page directive could take over. When using subclassing sensibly, a few lines of Python code should be enough for most use-cases.

This is how zope.formlib does it, actually, so perhaps we should simply not worry about zope.app.form and deprecate it all together (after moving out the widgets and other useful parts). In fact, zope.formlib is a good example of how things should be done in many respects.
browser:viewlet, browser:viewletManager: I don't know much about these yet, but from simply looking at their directive handlers, they seem to throw together a class on-the-fly and do pretty much what browser:page would do with it (template, security, adapter registration), except that the adapter registration involves more interfaces. The whole viewlet concept is very new and so are these directives; I'm sure we can think of a way to simplify the architecture so that we won't have to maintain two custom directives.

In total that makes 15 directives of roughly 80 we have in Zope 3.2. If we'd get rid of all 15 of them, we'd end up with 65 which would still be a lot, but also a lot less than 80.

Guidelines for new packages

In conclusion, we can compile some guidelines for writing new packages from the above:

Try to reuse as much of the existing Component Architecture concepts (utilities, adapters, views) as possible.
Try not to invent new ZCML directives unless you have a really really compelling reason to do so. If you're limiting yourself to utilities, adapters, and views anyways (and this should be enough), you shouldn't need new directives, the existing once should suffice.
Don't let ZCML do automation, Python is better at it. People will look for some of the behaviour in the Python code only to find out that it's not there but hidden behind dubious ZCML directives. This can be avoided. Simply try to look at ZCML directives as on/off switches and you're half-way there.

Comments

Subject	Author	Date
Two thumbs up: refactor the h*ll out of zcml	ksmith99	14/12/05 20:00
Z3 beginner concurs	wleftwich	14/12/05 14:23

Posted by Philipp von Weitershausen @ 12/14/2005 07:36 AM. - Categories: Zope 3 - 2 comments