[Irl-dean] Accessible CMS

Joshue O Connor joshue.oconnor at ncbi.ie
Thu Dec 7 11:30:16 GMT 2006

Hi Eamon,

>> I would like to ask the list if they have any thoughts/experiences on
>> accessibility of Content Management Systems (CMSes).

This is an area that CFIT have been looking at over the last year. I put
my findings together and wrote a paper that, was accepted by AXMEDIS
(http://www.axmedis.org/axmedis2006/ but myself and Gez are presenting
another paper there next week,  so I don't think I will be presenting
this one. The paper is in a kind of limbo at the moment looking for a
home, so in the spirit of sharing here it is. I hope you find it useful.

Feel free to use it as you will, and if you do please just give CFIT a nod.



Joshue O Connor
Senior Accessibility Consultant

Expert Screen Reader Evaluation
by Paul Traynor CFIT
joshue.oconnor at cfit.ie


How do you go about choosing an accessible content management system
(CMS)? What are the main criteria for success? And how to ensure ease of
use for authors including screen reader users?

CFIT looked at several popular CMSs in order to assess which would be
most suitable. The successful CMS had to be capable of outputting
standards compliant markup and have an accessible user interface. Other
factors such as the overall ease of use and usability were considered.
It was also necessary for the system to be feature rich and scalable.

Our approach was to look at how these CMSs work “out of the box” and no
complex heuristics were applied in order to simulate how many other
users would approach the adoption of a CMS in the “real world”. The
assessment method was an intuitive approach with some basic core tasks
such as adding content and administration.

1. Introduction

There are many commercial and open source CMSs available today and
making the right choice can be very daunting.

Some factors to consider are the technical skill of the staff who will
use it and the behind the scenes work needed to set up and customise the
system. The amount of time it can take to properly check the viability
of a CMS is itself something that can put many off choosing and
implementing a reasonable open source system, so many will go to
companies who offer a commercial package they have themselves developed
and offer to take care of the content migration and customisation. For
those with a large budget this may be a viable solution, but for
community based groups, disability advocate groups and others with
little or no budget, are there systems available that can make setting
up and administration a plausible option with a nominal amount of
technical knowledge needed?

For the purpose of this paper we are going to look at some popular and
free CMS packages that could suit the needs of groups like these. Aside
from basic set up and any related costs including hosting, etc., we need
to consider if the successful system will be capable of outputting
standards compliant [1] and valid code [2]. This whole exercise can be a
daunting task for people who are not technically minded. Many may not
even be aware that there are efforts to ensure that accessibility is
already considered and, where possible, built into the CMS. This “under
the hood” accessibility is courtesy of the Authoring Tool Accessibility
Guidelines (ATAG) [3]. These guidelines are there to help CMS developers
create tools that have a user interface that is accessible, as well as
outputting accessible content. The guidelines aid the developer who
builds the system to take care of any automatic processes and to
intervene if some input is needed from the user in order to improve or
enhance the accessibility of the content. However, there is also a
responsibility on the part of content authors to ensure that they are
endeavouring to create good accessible content. There may be an
unrealistic expectation that whatever the author throws at the system
will be magically transformed.

2. What are the main criteria for success?

Our goals were to find:

1. A CMS which is functional, accessible and usable to authors who have
basic IT skill level and knowledge of using word processors, but no
fluency with markup languages like HTML.

2. A CMS that can be recommend and promoted as an accessible Web content
management solution for community/disability groups, charities and
others with a limited budget.

3. If possible, the WSYWIG editor should be accessible to screen reader
users. The usability of the mechanism is also important, so we are not
merely considering if it is technically accessible but also if it is
easy and pleasant to use.

2.1 How to ensure ease of use for authors including screen reader users?

All the CMSs that we examined were tested by an experienced screen
reader user with a strong technical background. He is a power user with
many years of experience using various screen readers such as JAWS and
Window Eyes. The CMSs were evaluated in a critical manner with attention
to the quality of the markup and the finer touches and details that can
be employed by developers to enhance the user experience for blind people.

CFIT pay particular attention to usability in our testing of any ICT. We
are not interested in merely “ticking the boxes” or “minimal compliance”
but in a pleasant and rich user experience that facilitates better
interaction, whether the user is creating content or administrating the CMS.

2.2 How accessible and usable are some of the more popular open source CMSs?

2.2.1 Methodology

In order to give these tests a real world flavour and to ensure they
were ecologically valid, we consciously did not use any particular
testing method or script in order to access how intuitive these systems
are out of the box.

For the tests we looked at:

•	Jadu
•	Mambo
•	Joomla
•	Quick and Easy
•	Expression Engine
•	Plone
•	Drupal
•	Textpattern
•	Xoops
•	Typo3

3. Results

Our feedback derives from user testing and observation of a screen
reader user performing the following basic tasks, as well as the same
tasks being performed by a user with no Assistive Technology/Special
User Agent requirements and with an average computer skill level.

These tests were facilitated and observed by an experienced CFIT
accessibility consultant.

The tasks included:

•	Uploading content and, where possible, editing and formatting content
(using a WYSIWYG editor).
•	Creating new pages (Category/Section headings and sub
•	Basic administration of user groups and permissions.

For testing the WYSIWYG editors we used a similar methodology to Peter
Krantz of Standards Schmandards [4].

These tasks were performed for all of the CMSs that we tested. The
results are not conclusive and are included here to be indicative of our
experience and subjective assessment of the pros and cons of using each CMS.

3.1 Jadu

Before we started looking at Jadu, we contacted the developers to ask
about the accessibility of their CMS. They replied, pointing out their
“use of AJAX and JavaScript in the editor that would make for ’issues‘
with our screen reader”.

As a result we did not continue with our screen reader test. However we
noted that the system has an option to display content as high
contrast/text only, which many visually impaired users will find useful.

The developers then stated that they have plans to introduce an
accessible user interface in forthcoming releases.

3.2 Mambo

Visually, the graphic style of the interface was pleasant to work with
and the style of the Windows operating systems graphics would no doubt
be appealing to many users and would not be too much of a departure from
what they are used to, so this could be an advantage.

>From the perspective of “instant usability”, our first impression of
Mambo was quite good, but on starting to perform the tasks we found it
difficult to then find the uploaded content.

Mambo has a peculiar method of structuring its content with a curious
mix of sections/categories that we found to be confusing, but once the
user had familiarized themselves with the way the system works then it
could be comfortably used.

However, using Mambo was found to be difficult for a screen reader user
so we could not confidently recommend Mambo as a usable and accessible CMS.

3.3 Expression Engine

Expression Engine is a blog focused CMS which lacks more advanced

>From a screen reader users perspective it was frustrating to use. There
were no structural headings and the blog feature was very difficult to
navigate and figure out. Title and edit boxes were not marked up well
and our tester found it unintuitive.

>From a usability perspective our first impression is quite good and we
liked the interface. It is visually quite smart and we found it quite
intuitive to use. Adding posts is very easy as is creating links. It
automatically asks for additional information, such as the title
attribute, when creating a link. Creating a link is a matter of
selecting your desired link text and then clicking the link button.
These additional options pop up then.

Module installation is straightforward and there are an extensive list
of utilities such as an SQL and Plug-in Manager.

However, as it is inaccessible we will not recommend it.

3.4 Textpattern

Textpattern is primarily a blogging tool rather than a CMS, and a big
downside to this application is the inability to have more than one
level in the navigational hierarchy. Our testing with a screen reader
revealed similar results to Expression.

However, it has a very simple interface and textile text formatting system.

Textile is a way of formatting text without using a WYSIWYG editor

3.5 Joomla

Joomla is a new version of Mambo and it has made some great improvements
in its overall accessibility.

For a screen reader user, we found it to read very well with JAWS and
Window Eyes, with no focus problems and no mixing up of lists or combo
boxes, etc. In our testing it was found to be “Easy to understand, and
enjoyable to use”.

Some more specific positive observations were:

1. All checkboxes were found to be well labelled and read correctly
using Window Eyes, JAWS and both Firefox and IE6. Various menu settings
such as user menu, main menu, etc. were checked.

2. The log in boxes are fine, no problems with them.

Some problems encountered were:

3.Some links reading "On Mouse Over" could not be activated by pressing
the Enter key.

4. Various items had checkboxes, etc. that weren't very intuitive. The
labels didn't convey their purpose effectively to the screen reader user.

5. Radio buttons read well but their labelling could be improved. They
often were not understandable as to what purpose they served.

If these problems could be addressed, we would recommend Joomla.

3.6 Quick and Easy

We very much liked Quick and Easy. Some feedback from screen reader
testing is as follows:

1. Very simple interface.

2. Easy to get around and not cluttered.

3. All items were well labelled and easy to understand as to their purpose.

4. Quick and Easy is made for a user who wants simplicity and a quick
way to update web content.

5. I enjoyed my experience of using Quick and Easy.

6. All lists, list boxes, combo list boxes edit boxes read well with
both JAWS and Window Eyes.

7. I noticed that none of the various elements read incorrectly with
either screen reader.

Quick and Easy is a low cost commercial product and could be suitable to
a wide variety of community based groups. The administration interface
is accessible and standards compliant. There is a user determined choice
of HTML tool, so different people managing the same site can choose
whatever suits them. Quick and Easy has some advanced functionality,
such as built in blog, a mailing list application and a photo gallery
plug-in, but may not be as extendable as some other open source CMSs
such as Plone or Drupal. However, we do feel it was one of the best we

3.7 Plone

>From a usability perspective our first impression of Plone was that it
is not that intuitive. This is primarily down to the labels and page
types but this could be improved, as Plone is highly customisable. It is
very feature rich out of the box and this may be why it feels rather
unwieldy and a little intimidating.

>From a screen reader users perspective, we did find the following
positive points, in no particular order of importance:

•	Skip to content/Skip navigation links.
•	Good heading structure.
•	No Frames.
•	Form elements are well marked up.

The negative points are:

1. Overall lack of consistency between what elements are visible when in
forms mode/virtual PC mode.

2. The naming conventions for items used in the interface are a little
unintuitive. Use of terms like "Smart Folder" wasn't great, and we had
no idea what a "Smart Folder" does. However, reading the manual would no
doubt shed some light on this, which as previously stated, we have not
done for this test in order to assess the CMS’s level of ‘instant

When we tried to add a page we found two frames not correctly or not
clearly labelled.

We also noted some peculiar (and inconsistent) behaviour when using JAWS
5. Pressing the F key (or tab key) should allow the user pick out
elements such as buttons, edit boxes, list boxes, etc. and this worked
fine. However, when using the arrow keys to navigate through these
elements, no associated labels or identifying information were read out.
Now we have to again test this with JAWS 7 and also other screen
readers, such as Window Eyes, to see if this observed behaviour is
consistent or something peculiar to JAWS 5.

Overall, Plone was a good CMS and highly customisable and extendable. We
would recommend it.

3.8 Xoops

Xoops has turned out to be a favourite during our tests with our screen
reader user who was very pleased with it. He found it easy to use,
understandable and enjoyable. Form elements are well labelled and the
layout is simple and uncluttered.

However, from a usability perspective we did not like Xoops. We found
the interface unintuitive and bare and we just did not like the feel of
the application.

3.9 Drupal

In terms of the interface "out of the box", Drupal takes the opposite
approach to Plone. The interface is simple uncluttered and clean. They
adopt a modular approach which allows you to customise Drupal so it fits
your requirements, adding other modules such as a blog, etc. as you need
to, and thereby presenting the user with a simple to understand and easy
to navigate interface, which does not overload the user when they first
open the application.

Drupal is well laid out, with good headings/structure. Access controls
and checkboxes are marked up well.

It could however be improved. For example the labelling of checkboxes in
the blog administration page is not very good. There are checkboxes that
allow the administrator to set permissions for anonymous and
authenticated users that are also not labelled very well. This makes it
difficult for a screen reader user to administer the site well as they
cannot associate each checkbox with its relevant command.

However, these are our first impressions and we feel that Drupal is one
of the best that we have come across and would recommend it with some

3.10 Typo3

Typo3 has a confusing interface. It uses a tree structure but the
interface is inconsistent. The language used is also very “technical”,
which would put off novice users. It uses inaccessible language for
buttons/commands like “Reload the tree from the server”. Even adding a
page/content is difficult with more unusable tech speak like “the page
module is activated from the backend menu” being used.

Typo3 also uses frames that are not correctly labelled (if at all) and
there is a lack of simple heading structure.

There are however edit boxes/buttons with are labelled well but there
are also peculiar behaviours. For example, when jumping across links our
screen reader user found that he was also jumping through unidentified

We cannot therefore recommend Typo3.

4. Plone and Drupal Advanced Features comparison

With both Drupal and Plone all of these feature are available either out
of the box or as a module plug-in*.

Implementation & Setup
Custom Content Types / Templates
Pod casting
RSS syndication
XML syndication
Content Reuse
Friendly URLs
Support for Multiple file formats
Publication Scheduling
Content Versioning
Granular Privileges
LDAP Authentication
XHTML web output

*However, XML syndication and Publication Scheduling may require
substantially more development time in Drupal. Also Plone is written in
Python, which may be an issue for those without skills in that language.

5. Modifying the Interface

The interface for each system is XHTML / CSS based so each can be
enhanced and improved. Quick and Easy, Joomla, Plone or Drupal would
satisfy the website requirements of many community based authors and
groups. However, the Drupal and Quick and Easy interface, without
modification, are more screen reader friendly than Plone. Drupal and
Quick and Easy have a clean and simple UI that will be a help to
(non-technical) content creators.

Plone has more features available out of the box but many of these
features may not be needed.

6. Accessible WYSIWYG Editors

CFIT have concerns about the general accessibility of WYSIWYG editors.
The two editors we looked at are XStandard and TINY MCE. While using
XStandard we encountered some problems. The screen reader could not
properly interact with it and it seemed to leave the user hanging
without any real idea where he was within it.

It also needs to be keyboard accessible. We contacted the XStandard
development team and they have produced a Mac version that is keyboard
accessible, which we have not yet tested. They also mentioned that they
are currently working on a keyboard accessible Windows version of the
editor and hope to have it out by the end of 2006.

We found Tiny MCE to be “somewhat” accessible but we have concerns about
the quality of output; structure is often not maintained.

7. Practical Alternatives to using the embedded WYSIWYG Editors

Use an editor with an accessible interface such as the freeware editor
PSPPad [5] or the W3C’s editor/browser Amaya [6], to pre-format any
code. The screen reader user can them upload the structured content via
the CMS. The drawback is the learning curve required on the part of the
user but it seems to be the most viable option at the moment.

For more on comparisons of editors Peter Krantz of Standards Schmandards
has done some more extensive testing of a wide variety of CMSs [7].

When embedded WYSIWYG editors improve the accessibility of their
interfaces this will have a major positive impact on the overall
accessibility and enhanced usability of a CMS.

8. References









NOTICE: The information contained in this email and any attachments 
is confidential and may be privileged.  If you are not the intended 
recipient you should not use, disclose, distribute or copy any of 
the content of it or of any attachment; you are requested to notify 
the sender immediately of your receipt of the email and then to 
delete it and any attachments from your system.

NCBI endeavours to ensure that emails and any attachments generated 
by its staff are free from viruses or other contaminants.  However, 
it cannot accept any responsibility for any such which are 
transmitted.  We therefore recommend you scan all attachments.

Please note that the statements and views expressed in this email 
and any attachments are those of the author and do not necessarily 
represent the views of NCBI


More information about the CEUD-ICT mailing list