Project Objective
To process the 2001 Census Forms and deliver clean, consistent and complete data for input into the in-house edit, imputation and output systems.
Background
Traditionally, Census forms have always been processed in-house, but following the 1997 Census Test it was decided that better value for money could be obtained from contracting-out the main scanning and capture services for the 2001 Census.
An Open Options Procurement Project was undertaken in 1998 by the Census Offices for England and Wales (ONS), Scotland (GROS) and Northern Ireland (NISRA), jointly known as the Authority, with the aim of acquiring processing services for the 1999 Rehearsal and the 2001 Census. In December 1998 the Processing Contract was awarded.
During the procurement of processing services, three potential suppliers were short-listed. Their Best and Final Offers were considered against a Public Sector Comparator (PSC), which is an assessment of the cost to the Authority to process the 2001 Census in-house. The PSC included contingency and took account of the value of the risk being transferred. The contract was awarded to Lockheed Martin (LM) who submitted the proposal which best met the Census Offices requirements in terms of solution and price.
The contract was drawn up to achieve value for money and to transfer to LM major risks associated with the design, development and implementation of processing services for the 2001 Census. The Contract was based on an 'outputs' approach with the requirements being stated in terms of 'what' was required and not 'how' things should be done, i.e. descriptive rather than prescriptive. However, LM's solution (their 'how') was captured in the contract, so that they could not change it without the Authority's consent.
Live processing activities for the 2001 Census took place at Widnes, Cheshire, at a processing site selected by LM and ICL, and approved by the Authority. The staff were recruited locally and LM maintained a presence on the site throughout. ONS, together with colleagues from the other 2 Census Offices, also set up and maintained a team on site to resolve queries, and 'oversee' the management of the operation.
The 2001 Census processing system, designed by LM, used Optical Mark Recognition (OMR) and Optical Character Recognition (OCR) to 'lift' the data from the census forms. If characters were not recognised automatically, with a pre-determined degree of confidence, the image was presented to an operator for interpretation and keying.
In addition to the automatic recognition of the data, LM developed an automatic coding system, used for responses that required translation from a textual response into a numerical format. Those responses not coded automatically by the system were presented to manual coders who used computer assisted technology to apply the appropriate codes.
The system was very sophisticated, enabling LM to modify and tune the application, thereby continually improving the throughput and accuracy.
Key Achievements
There are three key indicators by which to measure the success of the Processing project - Quality, Cost and Timeliness. All Quality Standards were met or exceeded and the final costs, allowing for anticipated changes, compared favourably with the original estimates.
Within the 11 month period between July 2001 and May 2002, 24.1 million forms were processed for England and Wales and the cost to ONS for the services provided by LM was £54 million.
External consultancy for the Processing Project, covering advice and assistance on Procurement, Contract Management, Legal and Technical matters cost a little under £2 million. A further £1 million was expended on internal administration. Other in-house processing activities, such as the set up and management of the Authority presence at the processing site, which lay outside the contract but which the Authority undertook in order to ensure a successful interface with the Contractor, cost a further £2 million. Together, these costs for the management of the contract and the processing interface represent 9.2 per cent of the overall costs.
The overall cost to the Authority of £59.5 million compares very favourably with the ONS element of the Public Sector Comparator of £61 million.
There were some timetabling issues which ultimately led to a slippage of 5 weeks in the final data delivery milestone, still a reasonable achievement in the context of a 4 year programme. Overall, this compares extremely favourably with equivalent Government Information Technology projects. It provides a successful model against which future similar initiatives can be compared.
The Authority's relationship with the Contractor was managed in a very professional manner and was the cornerstone of the Project's success.
Methodology
A major difference in processing the 2001 Census, compared with previous censuses, was the contracting-out of the main capture and coding services. The project's success was therefore heavily reliant on the timely delivery of quality services from LM and procedures were established to manage both the contract and the project to achieve a successful outcome, within budget.
The Capture and Coding systems were automated which enabled 100 per cent coding of Occupation and Industry responses compared to 10 per cent in 1991.
Initially, the Authority's role was to provide the contractor with the essential information needed for them to develop a service to meet the requirements. This then developed into one of overseeing the Contractor's management of the live operation.
LM sub-contracted the scannable forms printing to Polestar and operational elements of the contract, including warehousing activities, to Fujitsu (formerly ICL).
Contract Scope and Interfaces
The scope of the contract included:
printing scannable forms;
the capture, coding and delivery of clean data;
warehousing activities;
the production of microfilm from the Census Forms for long-term archiving;
interfaces with the Authority on matters such as quality and the resolution of queries generated during processing; and
the destruction of the Census Forms.
Requirements
The high level Statement of Requirements (SOR), formed the basis of the original invitation to tender. During initial negotiations with the three potential suppliers they were discussed at length and were further honed by this process. This fitted well with the recommended approach of an outputs based procurement, with the requirements being stated in terms of 'what' was required and not 'how' things should be done. However, without constraining the bidders' creativity prior to contract award, the winning bidder's solution was captured in the contract, so that the Authority retained control over subsequent development of the chosen system and services.
Management of the Contract
Following award of the Contract, two committees were formed - the Contract Steering Committee (CSC), responsible for providing strategic direction to the Project and the Contracts Management Board (CMB), internal to the Authority and to its advisors. Regular, less formal meetings complemented those of the CSC and the CMB. The contract administration team acted as the official conduit between the Authority and Contractor.
Testing
A testing plan was set up as part of the contract, where the Contractor and the Authority undertook distinct and specific activities. As the project progressed, there were clear benefits identified in combining these teams but leaving the overall obligation with the Contractor to ensure all systems and services functioned correctly.
Service Levels and Liquidated Damages
The contract required LM to provide the Authority with services of a pre-determined quality to ensure the delivery of accurate and timely data. If the pre-determined quality standards i.e. Service Levels, were not met, Service Default Points (SDPs), were allocated by the Authority. The accrual of SDPs entitled the Authority to have the Contractor take corrective action, if necessary.
Liquidated Damages (LDs), which are payments which compensate against failure to meet contractual milestones, were specified in the contract. Each LD was set at a cost per day for a pre-specified period. If either SDPs or LDs ever reached certain pre-set levels, the Authority would have been entitled, had it so wished, to terminate the contract and have the Contractor transfer it to a third party or bring it back in-house.
Change Control and Issue Management
A contract of the size and complexity of the Processing Services will inevitably need changes made to it over the course of its life. Formal procedures for managing changes and issues were set up and were fundamental to ensuring that changes were costed according to the pricing structure in the contract. It was also important throughout the project to distinguish between changes to requirements - which were chargeable by the Contractor - and measures to rectify failures to achieve requirements - which were not chargeable.
Risk Management
A Risk Consultant was recruited, from an external consultancy firm, to identify and assess risks to the project. Risks were managed by the Authority through the CMB and CSC by securing changes to the contract or rectification measures to enforce the contract (as appropriate) from the Contractor.
External and Internal Expertise
As well as Legal Representatives, the Census Offices enlisted a pool of external consultants. The consultants employed had a wide experience of a variety of aspects of lengthy contracts and practical experience of technical solutions.
An external employment agency was used for recruiting staff to represent the authority at the processing site.
Security and Confidentiality
An Independent Security Review was set up, aimed at providing assurance that both the Authority and LM had adequate practices and standards for the security of 2001 Census data and to meet the principles set out in the 2001 Census White Paper.
Assessment and Lessons Learnt Contract scope and Interfaces
Where there are complex interface issues between separate contracts, there can be problems, such as the responsibility for employing a driver to move trailers from the secure holding area into the warehouse. To avoid this it may be beneficial to award all related and interdependent contracts to one supplier/prime contractor. This needs, however, to be balanced with a risk that in awarding the total contract to one supplier they may not be specialists or best placed in every area within the contract. To overcome this problem, additional experts can be employed to advise. For example, concerns about the print aspects for 2001 Census Forms were mitigated by buying-in the services of an expert in forms design and printing for scanners/Optical Character Recognition (OCR), to support the Authority. Overall, the decision to place the onus for printing scannable forms with the Contractor responsible for processing turned out to be sound and avoided a potentially difficult interface between two contracts.
On balance, the Authority has concluded that a single contract could very likely have improved overall effectiveness. There is scope for combining processing, transportation, all printing, storage and perhaps payroll and call centre activities into one contract, dependant on a detailed assessment at the time.
It should be recognised early on that a contract of this magnitude will inevitably have changes made to it over the course of its life. Any future contract should include a contingency amount on which the project would have first call. Allowance should also be made for a significant level of external contract support.
The contract was awarded as a fixed price contract placing the onus on the contractor to cost accurately at bid time. Formal changes to the requirements were charged in accordance with the contract. All other unexpected costs were borne by the Contractor.
Requirements
Requirements must be reviewed and, if possible, simplified. For 2001, the detailed coding and processing requirements were not totally clear and complete at bid time which meant that LM experienced some difficulties in pricing their proposal. Ideally, a detailed requirement specification should be in place prior to award of contract and examples of the detailed processing rules should be developed well in advance of award. If timing does not allow this, a contract mechanism needs to built in to cope with an evolving requirement.
A contract awarded jointly by three countries for common services requires an Authority lead and in 2001 ONS led each of the elements. This report therefore uses Authority as a generic term to refer to ONS activities.
Management of the Contract
The contribution of the Management Boards convened for the project, together with the overall tone of the less formal communications, was invaluable in building the relationships, trust and mutual understanding that were intrinsic to the project's success.
A dedicated Contract Management team acted as the sole conduit in communications with LM and were key to the smooth running of contractual activities.
Meaningful service levels should be set and closely monitored, to allow the Authority the opportunity to re-evaluate their relevance if the Contractor's failure to meet them threatens. The purpose of Service Level Agreements (SLAs) and the method of calculating Service Default Points (SDPs), if the Contractor failed to meet the SLAs, needs to be reviewed. If too many SDPs had accrued, the contract could have been terminated. A pragmatic approach is needed in determining how to use SDPs to put pressure on the contractor to take corrective action in the event that performance falls below SLAs.
Realistically, if we had terminated the contract during the operational phase (2001/2002), there was no alternative for processing the census on schedule. Considerable thought must be given to how to practically implement an alternative which would still result in the Census being delivered on time, in the event of termination of a contract for non-compliance.
Testing
When testing that the Contractor's system meets the Authority's requirements, the Authority and Contractor need a clear understanding of the objectives of each test and to share a common understanding of what constitutes an acceptable result - acceptance criteria should be agreed up front and met before moving on to the next phase.
The final testing phase should cover all Operational Services as well as Operational Systems/Interface with Field Operations and Interfaces between Authority and Contractor and Contractor and Sub-contractor. The contract should make it clear that the Authority expects to see evidence of full operational capability well in advance of 'live' operations. The number of test forms should be sufficient to simulate full operational processing volumes.
The 2001 Census Capture and Coding service was far more integrated and automated than the equivalent 1991 version which relied heavily on manual input. Although the 2001 Census approach brought many efficiencies, it was considerably more complicated to specify, develop and test than in the past.
Service Levels and Liquidated Damages
Basing the contract on pre-determined quality standards or Service Levels was an extremely effective way of obtaining data and services of the correct standard. The impact of each must be commensurate with the importance of that service to the Authority.
There were no Service Levels against output for small areas, such as groups of several Postcodes, which allowed problems in the data to legitimately slip through. These had to be corrected by the Authority in the later stages of processing. For the future, service levels against such small areas must be in place.
Liquidated Damages (LDs) should be used to incentivise, not only to start on time but also to complete on time. LDs were accrued as debts and could lead to termination of the contract.
Change Control and Issue Management
The Contract should include more detailed provision for defining the costs for change and in particular be clearer about categorising changes occurring between Rehearsal and the Census and establishing which categories were catered for within the original price and which would be charged for.
During the 4 year lifetime of the project, 107 changes were made, 52 sponsored by the Authority and 55 by the Contractor. Optional services such as the destruction of the census forms and the archiving of the images were included in the contract and pre-costed from the outset.
A fully electronic Change Control system should be developed to streamline the procedures and consideration should be given to enabling access by the Contractor.
Risk Management
Training in Risk Management should be provided to all potential risk owners, either as a specific course or as part of an overall project Management course.
External Internal Expertise
An understanding of census issues/requirements takes time, so key external expertise needs to be enlisted early and retained until the end of the contract, as happened for the 2001 Census. Similarly, continuity of the members of the processing team ensured that invaluable census expertise was always available.
Security and Confidentiality
Specify staff security clearance requirements well before recruitment starts. Such processes should be included in the project plans as a milestone date.
Procedures relating to back up and recovery should be tightened up for any future Contracts of this kind. Appropriate procedures need to be in place and the Contractor must demonstrate that they will be implemented correctly.
Conclusions
There is no doubt that the Processing Project has been a resounding success, due to two key features: the carefully drawn up contract with its inbuilt safeguards; and the unstinting commitment and capability of staff working within the Project, both Contractor and Authority.
The selection of the preferred Contractor has also proven to be a sound choice, given their continued commitment to meet their contractual obligations. Where alternative procedures have been recommended this merely reflects the lessons learned during the lifetime of the project which took an innovative approach to a traditional process.
The external expertise brought in to assist in managing the Processing Contract has added undoubted value to the Project. Also, the importance of sound legal and tactical procurement advice throughout the Contract cannot be overstated. It is generally acknowledged that those who have been working on the Project, many of whom had never had the opportunity to work in a commercial environment before, have benefited from the experience of working alongside contractors with a range of expertise.
Independent analysis has confirmed that the accuracy of the data has met or exceeded the service levels laid out in the contract.
Continuity of the processing team members provided one of the most important benefits to the Project. The expertise available for training new staff and providing business assistance to external experts was invaluable.
Census News
