Evaluating new software suppliers

Is anyone aware of a site where I can find a checklist of issues to consider when evaluating a new software supplier.

I am looking for a simple wysiwyg editorial system for updating web content without the need for html knowledge.

I am reasonably sure of the product questions I want to ask, but would like to know what other issues should be considered, I can think of a few (supplier's financial security, client base, commitment to product support and future development), but am nervous of missing out on topics which may become obvious with experience.

Gareth

Thanks for offering your help. I accept that there is no liability on your part. I don't know your email address. If you would like to email me, send me the questionnaire(s) to

Regards
David

On 22:32:33 19 February 2002 glewis wrote:
>I've got various supplier assessment
>"questionnaires" knocking around my hard disk if
>you'd like to email me - but no liability accepted! Due
>diligence (abbreviated as CYA) is pretty boring, but as
>you say, it's also worrying if you don't ask all the
>questions, particularly as even before the 'net
>revolution, one in three IT suppliers disappeared every
>year . . .
>
>Gareth
>
>
>On 11:29:35 14 February 2002 davidlynam wrote:
>>Is anyone aware of a site where I can find a
>checklist of
>>issues to consider when evaluating a new software
>>supplier.
>>
>>I am looking for a simple wysiwyg editorial system for
>>updating web content without the need for html
>knowledge.
>>
>>I am reasonably sure of the product questions I want
>to
>>ask, but would like to know what other issues should
>be
>>considered, I can think of a few (supplier's financial
>>security, client base, commitment to product support
>and
>>future development), but am nervous of missing out on
>>topics which may become obvious with experience.

Hi Ashley

Thanks very much for the extremely comprehensive list! The product I am evaluating is called Ticto (see www.ticto.net) as I wanted something simple which allows authorised users to be able to edit web pages without requiring them to have technical html knowledge. Its an interesting product, especially in light of some of the questions on the checklist which are not applicable given the nature of their product.
As for financial health, I don't have a very exhaustive list of questions but the following are worth bearing in mind for this company as they are a relatively new start-up:

How much cash did they initially raise, how much is left, how many funding rounds have they gone through. How long have they been trading. When are they aiming to break even - the further off this is, the riskier. If they have venture capitalist behind them what is the timescale for the VC's exit strategy. If they have corporate investors, who are they? What is the competitive advantage their product offers, how 'protectable is it' - might a bigger player buy them out - or bail them out in the event of difficulties. Some companies will be less willing to give out all the above, but some of the information can help you guess at answers to the other questions

In the event of the company catching 'Enron-itis' what protection would their clients have? Do they have Escrow agreements for instance?

On 11:14:22 22 February 2002 Ashley wrote:
>Hi David
>
>Below is a set of questions that I would recommend using
>when going through a selection process for choosing a
>content management system (CMS). These will be too
>exhaustive for what you want, no doubt, and a lot of the
>questions are product-centric. However, some of the
>headings and questions might help you think about the
>kinds of questions you need to ask and issues you need to
>consider.
>
>What I am lacking is the more general questions about the
>company’s health, financial status and so on, so if
>you’ve got anything on that, it would be
>appreciated.
>
>Ashley
>
>
>
>*Implementation*
>
>- What consulting services do you provide? How involved
>would you be in any implementation?
>- How much would you expect this to cost?
>- How long would you expect an implementation of your
>system to take, given our circumstances and who from your
>organisation would be involved?
>- Can you give an example of where your system has been
>successfully deployed to support global and localized
>multi-lingual sites?
>- How would we migrate our existing site(s) into your
>system? Do you provide any tools, support, processes to
>aid this? How long might this take in your experience,
>given our circumstances?
>
>*Technical and Integration*
>
>- What operating systems can your CMS run on?
>- Which web servers are supported?
>- What database systems do you use or support?
>- What hardware, software and network requirements would
>you recommend for your system as we intend to use it? Is a
>proprietary, or dedicated, application server required?
>Which application server platforms does your CMS integrate
>with?
>- Is there any particular software required for client
>machines accessing your system?
>- What browsers and versions does your system support for
>content authoring, management and publishing?
>- What skills, languages and tools are needed to customize
>and extend your system?
>- What content authoring tools do you integrate with and
>to what extent?
>- What translation tools do you integrate with and to what
>extent?
>- Do you integrate with, or provide, tools to aid the
>structuring of unstructured data?
>- Can you launch and use other applications from within
>the system?
>- What connectors do you provide to common external
>enterprise applications, such as supply chain, ERP, HR and
>accounting systems?
>- How does your system share data with other systems? What
>protocols and formats are used?
>- What syndication capabilities does the system have to
>take content into the system and automatically feed it
>out?
>- What documentation do you provide with your system?
>- How would your system typically be configured in terms
>of hardware and software, including clustering
>considerations, networking, firewalls and security?
>- Does your system support user session management and the
>use of cookies?
>- What proprietary code is required, if any, and for what
>purpose?
>- What caching capabilities does your system have?
>- What administration tools and interfaces are provided
>for configuring and administering the system? Can the
>system be entirely managed via a web interface?
>- What skills are needed to administer the system?
>- Can distributed servers that are part of the system be
>administered both locally and centrally?
>
>*Scalability and Disaster Recovery*
>
>- How does your system scale to efficiently handle
>increasing numbers of content objects and authors? How
>does the system scale to meet increasing load demands?
>- What known performance limitations does the system have
>in terms of a maximum number of concurrent users, number
>of templates that can be simultaneously processed, number
>of content objects that can be stored in the repository,
>maximum file sizes and so on?
>- How does the system support the n-tier distribution of
>work environments that are not in the same place?
>- What is the upgrade path for more advanced or future
>versions of the system?
>- What fault-tolerance or automated recovery features does
>the system have to mitigate against system failures?
>- What archiving and disaster recovery features does your
>system have?
>- How reliable is the system? What availability can you
>guarantee?
>
>*Content Authoring and Version Control*
>
>- Do you provide your own authoring tools?
>- What tools does your system directly integrate with?
>- What file formats can your system recognise and manage?
>- How does your system deal with streaming media content?
>- Do you provide, or integrate with, any tools to help
>automatically convert between content formats?
>- Are your Web based forms automatically generated from a
>database schema or manually set up? How can they be edited
>and customised?
>- What level of form validation can your CMS provide for
>form-entered content? Do you provide and spell-checking or
>grammar checking tools?
>- Do you provide levels of undo for authors? How would an
>author go back to an earlier version of a piece of
>content?
>- What preview functions do you provide allowing an author
>to see content in its final state?
>- How do you enable formatting within your Web based forms
>for content capture? And how to you link to or embed rich
>media files associated with the text?
>- What personalisation capabilities does your CMS have
>based on an author’s permission level?
>- Does a content author have to be online to use the Web
>forms?
>- How does an author add metadata to a content object? Can
>metadata be automatically pulled from other sources such
>as a database file?
>- To what degree can content authoring be enforced by
>setting mandatory or optional fields?
>- What version control tools do you provide? Can content
>authors choose to view and work with previous versions of
>a content object? How can earlier versions be restored?
>- What support do you provide for simultaneous,
>collaborative authoring? Can different versions of the
>same content be used at the same time across multiple
>sites? Can the same piece of content be worked on at the
>same time and changes merged later?
>- Does the platform include audit-trail capabilities to
>capture relevant information on each version of a content
>object so that changes to the content can be easily
>tracked?
>- What levels of rollback does your system provide (files,
>content objects, pages, complete sites)? - How are
>‘known good’ versions of the site captured and
>stored for rollback purposes?
>
>*Storing, Managing and Publishing Content*
>
>- How does your system store content in the repository
>(file system, object database, relational database,
>hybrid)?
>- Can you store content to support international character
>sets as well as multiple date and currency formats?
>- What link management features does your system provide?
>Can it validate both internal and external links?
>- What indexing and searching does your system support? Do
>you use your own tools or are they augmented by third
>party search tools?
>- How does the system support the replication and
>distribution of content to multiple locations whilst
>maintaining data integrity, system availability and
>content synchronisation?
>- How does your system fit in with a two or three stage
>production environment (development, staging and live
>servers)?
>- What content deployment features does your system have?
>Can updates be done incrementally or must the whole site
>be replicated? How can updates be automatically scheduled,
>manually overridden, done ad hoc?
>- Can you set go live, archive, review and delete
>attributes to content with the system?
>- Can you use the system to publish content in file
>formats such as PDFs or proprietary formats such as help
>files or for CD-ROM?
>- What tools can the system make available which might
>benefit the end user of the site, such as advanced search
>options?
>
>*Templates*
>
>- How do your templates work? What languages can be used
>to develop templates for your system? Do you provide
>standard templates, or libraries of template code, for
>modification?
>- Does your system provide particular enhanced features
>for publishing to multiple channels, both digital and
>offline? How would your templates recognise and work with
>different end-user devices?
>- Can your system generate pages using templates both in
>real time and according to scheduled processes?
>- How do your templates integrate with other systems?
>
>*Personalization*
>
>- What levels and forms of personalization can your system
>deliver, and how (collaborative filtering, customisation
>of interface and so on)? How do you typically integrate
>with third party tools or systems (CRM or e-commerce
>systems, for example)?
>- Does your system rely on business rules defined by us or
>does it also have automated forms of intelligent
>personalisation such as dynamic navigation created
>depending on inferred site user interests? Do you provide
>any pre-defined rules we can use?
>- What range of rules does the system support for
>personalization? How are these entered and edited? What
>skills are required to configure the business rules?
>- How do you store user profile data? How can user data be
>analysed? How does the system report on user data? How can
>user data be imported into, or exported from, the system?
>What forms of data does your personalization engine work
>with (session data, user profiles, page history, metadata,
>rules and so on)?
>- What range of publications can your system personalise
>for (Web pages, e-mail, WAP, print and so on)? Is your
>personalization engine capable of running personalized,
>outbound, marketing campaigns?
>- Can your system report in any way on the effectiveness
>of the personalization being used in order to improve it?
>
>*Universal Language Support*
>
>- Does your system provide user interfaces in different
>languages?
>- Does your system integrate with any language translation
>aids?
>- What language content cannot be entered or retrieved
>from your system?
>- What are the configuration implications of deploying the
>system to non-English operating systems?
>- How would multi-lingual servers be synched so that
>content is uniformly up to date globally?
>
>*Workflow*
>
>- What workflow tools does your system provide? What tools
>do you integrate with?
>- What activities can have workflows applied to them in
>your system? Can external parties be included in a
>workflow? Can you apply workflows to all levels of
>content, not just files, right down to data elements
>within a content object?
>- How are workflows created and managed? What is the
>interface for designing workflows?
>- What triggers and alerting functions do you support in
>your workflow steps?
>- How do you configure the roles of users in a workflow?
>- Can multiple workflows be seen from the perspective of a
>single user’s participation in them? What task
>queuing functions to do you provide for users?
>- How do you audit workflows? What tracking and reporting
>tools and analysis does the system provide?
>
>*Security and Rights Management*
>
>- Does your system allow for multiple layers of security,
>rather than simple read/write/execute constraints?
>- How are users, groups and roles managed by the system?
>Do you support a hierarchical security model?
>- Can your system work with, or import, existing user
>permission and security level settings?
>- How can your system help us protect our intellectual
>property rights? Do you provide any digital rights
>management tools?
>- Does your system come with any standard encryption or
>security certification features? How can the system help
>protect and encrypt content stored on it?
>
>*Management Information Reporting*
>
>- What reporting and logging features does the system
>have? What tools does the system integrate with?
>- Does the reporting concentrate principally on
>non-customer facing processes or on end user interaction
>with the content?
>- What metrics can the system report against? Can the
>reporting be customer-centric, using an individual user as
>the starting point, rather than content-centric?
>- How accurate and reliable is the reporting? What are the
>margins of inaccuracy? Does the reporting comply with
>standard-defined metrics?
>- What interfaces are provided for configuring reporting
>and retrieving reports?
>- What standard reports are provided? How are custom
>reports set up?
>- How can reporting be configured to run regularly or ad
>hoc?
>- What format can the reports be delivered in and via what
>channels?
>- What technical performance measurement and monitoring
>tools does the system have?
>- How can reporting and analysis be personalized to the
>user?
>- Can the logs that the system generates be exported in
>standard data formats and shared with other applications?
>- Can workflow be applied to the reporting?
>
>*Training and Support*
>
>- What training do you provide as standard and what
>additional training can you provide, at what cost?
>- Where do you hold training sessions?
>- Do you have certified partners who can offer additional
>training services?
>- What levels of technical support do you provide? What
>hours is support available, in which countries and
>languages? What response times do you guarantee? What are
>your escalation procedures?
>- What other support services can you provide through
>partners or self-help facilities (documentation, online
>knowledge bases, support communities and so on)?
>- How do you cost your levels of support on an ongoing
>basis?
>- What is the upgrade path for future releases of the
>system? How are patches and bug fixes communicated and
>distributed?