The outcome of a properly designed and implemented sales process is advice that is suitable to the recipient and that is evidenced from the sales records, in particular the fact find and suitability letter, together with other supporting documentation.

A case of unsuitable advice, or a lack of evidence to support a recommendation, is evidence of failure of systems and controls. System failure falls into two categories:

A failure of design. Here controls simply are not adequate to pick up a failure in the advice process. An example of this could be that there is no control systematically to check new business.
A failure of implementation. This is where the control is soundly designed, but it is not operated to the required standard.

The FSA is well aware that firms have existing compliance systems in place which, if they are well-designed and operate effectively, will have prevented the mis-selling of pension switches. Its themed work has highlighted the risk of the extent to which these systems may be ineffective across the industry. In its follow-up work, the FSA is sure to look at whether systems function effectively, and the acid test of that is case sampling. So, whilst firms are told that they need to sample only "if necessary", we don't think that it is possible for a firm to know that its systems and controls are effective without sampling at some level.

Of course, past compliance checking is a guide, but the FSA has set a very specific standard in the published template. Whilst the published literature says that firms "may wish to" use the template, we understand that privately the FSA are indicating to regulated firms that its use in any sampling is mandatory. Almost certainly, the only way to audit past work against the standard of the template is to test sample cases against it. The two main issues are how to determine an appropriate sample and how to execute the testing objectively.

When sampling cases it is important to sample all relevant types of transfer and all advisers. Where cases have previously been through a formal process of compliance checking, sample all compliance checkers as well. Where some cases fail testing, one needs to look for any pattern of failure. If none is apparent, and if the original sample is modest, further targeted sampling may be needed to determine if a pattern exists. Firms need to be confident that there is no significant risk of systemic failure in any quarter. The FSA will expect to see targeted remedial action of sufficient scope to deal with any pattern of failure. Widespread failures without any discernable pattern suggests a failure of system design, for which the only remedy is a case by case review.

All this work needs to be completed by the end of June 2009 when the FSA's follow-up work begins. A firm that finds that it has a systemic issue will find that timescale a real challenge. It makes sense to get on with sampling - you don't know what you will find until you have done it.

It goes without saying that a firm must consider what remedial action is necessary for each identified failure of the advice process.

It can be difficult to assess objectively systems you have been using and have believed are delivering good outcomes. The opinion of an outside expert may be very useful to point out potential flaws in the system. As an absolute minimum, no-one can be expected to assess a case in which they have previously been involved, be that in the role of adviser or compliance checker.

At OAC, we can help you to decide the data that you need to select a valid sample, and the criteria to use in selecting it including determining the optimum size. We can give you insight into the evidential standards expected when assessing cases and provide additional resources where staff levels, or independence of assessment, are an issue. OAC is fully capable of carrying out all, or any part of, the required work on your behalf.